LLM Post-Training as Brownfield Maintenance: An Industrial Perspective on Dataware Engineering
The paper frames post-training as constrained maintenance of an already deployed model, with data mixtures as the thing being engineered.
The authors argue industrial teams are not retraining from scratch; they are patching behavior under fixed compute and mixture budgets while avoiding regressions. Their code-generation case study says improving teacher-distillation yield produced 2.84x more accepted supervision with the same teacher and attempt count. The resulting patch improved CodeForces and LiveCodeBench scores in statistically significant evaluations, while internal AIME and MATH regression suites stayed within tolerance. ArXiv · AI/CL/LG's note
The authors argue industrial teams are not retraining from scratch; they are patching behavior under fixed compute and mixture budgets while avoiding regressions. Their code-generation case study says improving teacher-distillation yield produced 2.84x more accepted supervision with the same teacher and attempt count. The resulting patch improved CodeForces and LiveCodeBench scores in statistically significant evaluations, while internal AIME and MATH regression suites stayed within tolerance. ArXiv · AI/CL/LG's note
score 4