Actions Speak Louder than Words: Measuring Cross-Lingual Policy Retention in Tool-Using Agents
The paper says multilingual agents often change their tool-use policy even when the task is the same.
The authors measured action traces, not just final answers, across 8 models, 6 benchmarks, and 41 languages. After correcting for trace-length, empty-trace, chance-agreement, and self-consistency confounds, the cross-lingual divergence got larger. Four frontier models retained about 71-73% of their action policy across languages under greedy decoding. The paper also says agents route non-English tasks through English, and that one apparent multilingual failure was caused by trace extraction rather than the model itself. ArXiv · AI/CL/LG's note
The authors measured action traces, not just final answers, across 8 models, 6 benchmarks, and 41 languages. After correcting for trace-length, empty-trace, chance-agreement, and self-consistency confounds, the cross-lingual divergence got larger. Four frontier models retained about 71-73% of their action policy across languages under greedy decoding. The paper also says agents route non-English tasks through English, and that one apparent multilingual failure was caused by trace extraction rather than the model itself. ArXiv · AI/CL/LG's note
score 5