Not X, but When: Diagnosing a Mandarin LLM Writing Tic through Contextual Appropriateness
Most annotated uses of Mandarin’s “not X, but Y” were correct but unnecessary.
Five commercial models used the construction 2.5 to 9.2 times as densely as a human reference. Among 307 annotated uses, 52.8% were optional and just 1.30% were defective. The researchers assessed whether context made X a live alternative and whether rejecting it added information. Several reward models favored the construction over quality-matched variants, while supervised fine-tuning and preference optimization tended to increase its frequency in open-weight models.
ArXiv · AI/CL/LG's note
Five commercial models used the construction 2.5 to 9.2 times as densely as a human reference. Among 307 annotated uses, 52.8% were optional and just 1.30% were defective. The researchers assessed whether context made X a live alternative and whether rejecting it added information. Several reward models favored the construction over quality-matched variants, while supervised fine-tuning and preference optimization tended to increase its frequency in open-weight models.
ArXiv · AI/CL/LG's note
score 4