Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents
A 2B model got much better at dialogue games after training aimed at specific, diagnosable failures.
The paper tests a small open-weight model in the LM Playschool Challenge, where agents must track state, use feedback, and make valid moves across turns. It reports that many errors were local and mechanical: repeated guesses, malformed actions, or ignoring feedback just given. The proposed recipe uses supervised fine-tuning for broad participation, targeted preference pairs to repair verifiable failures, and a preservation step for general abilities. Final evaluation scores rose sharply in-domain, while out-of-domain clemscore stayed low, suggesting most transfer stayed within the targeted game family. HF Daily Papers' note
The paper tests a small open-weight model in the LM Playschool Challenge, where agents must track state, use feedback, and make valid moves across turns. It reports that many errors were local and mechanical: repeated guesses, malformed actions, or ignoring feedback just given. The proposed recipe uses supervised fine-tuning for broad participation, targeted preference pairs to repair verifiable failures, and a preservation step for general abilities. Final evaluation scores rose sharply in-domain, while out-of-domain clemscore stayed low, suggesting most transfer stayed within the targeted game family. HF Daily Papers' note
score 4