Are Language Models Script-Aware?
The paper finds that tested language models usually preserve or follow requested writing scripts, but larger models handle harder script cases better.
The authors test small and large language models on languages that can be written in multiple scripts. In one experiment, they check whether a model matches the script used in the input; in another, whether it obeys an explicit script instruction. All tested models show near-perfect Latin-script fidelity, above 98%, and often follow script directions. The gap appears in non-standard script combinations, where LLMs score higher than smaller models. ArXiv · AI/CL/LG's note
The authors test small and large language models on languages that can be written in multiple scripts. In one experiment, they check whether a model matches the script used in the input; in another, whether it obeys an explicit script instruction. All tested models show near-perfect Latin-script fidelity, above 98%, and often follow script directions. The gap appears in non-standard script combinations, where LLMs score higher than smaller models. ArXiv · AI/CL/LG's note
score 4