Parsed, Plausible, Wrong: Output Format Controls Silent Errors in Small-Model Semantic Parsing
Ordinary-word output cut silent parsing errors by 2.3× on Qwen and 1.8× on LFM2; shortening output alone did not help.
Researchers tested six encodings of the same target trees across 300 fine-tuning runs on the PIZZA ordering benchmark. The word encoding performed best on both small models, reaching 87.5% exact match on Qwen versus 85.8% for native s-expressions while emitting 4.1× fewer tokens. Much of the improvement came from handling negation. Strict refusal of off-menu inputs carried a cost: models also rejected 2.1–10.1% of valid orders, mostly polite ones.
HF Daily Papers' note
Researchers tested six encodings of the same target trees across 300 fine-tuning runs on the PIZZA ordering benchmark. The word encoding performed best on both small models, reaching 87.5% exact match on Qwen versus 85.8% for native s-expressions while emitting 4.1× fewer tokens. Much of the improvement came from handling negation. Strict refusal of off-menu inputs carried a cost: models also rejected 2.1–10.1% of valid orders, mostly polite ones.
HF Daily Papers' note
score 4