TraceForge: Distilling with Your Own Thinking
Generated reasoning traces raised coding scores by 3.7 points on HumanEval+ and 7.1 on MBPP+, using the same 5,000 questions and teacher answers.
TraceForge takes a teacher’s answer and generates a reasoning trace in the student model family’s style. It trains a generator and critic through supervised and adversarial stages. Across five benchmarks, general-task results were roughly unchanged, while coding results beat training on Fable 5’s own thinking content. Independent judging measured how genuine the traces appeared, not whether their reasoning was correct.
HF Daily Papers' note
TraceForge takes a teacher’s answer and generates a reasoning trace in the student model family’s style. It trains a generator and critic through supervised and adversarial stages. Across five benchmarks, general-task results were roughly unchanged, while coding results beat training on Fable 5’s own thinking content. Independent judging measured how genuine the traces appeared, not whether their reasoning was correct.
HF Daily Papers' note
score 4