Everything in Moderation: Per-Domain Coverage Optima and Alignment-Resistant Domain Gaps in Multi-Domain Mid-Training
The paper finds that mid-training domain mix leaves performance gaps that later alignment does not erase.
In controlled logical-reasoning runs on Qwen3-8B-Base, each of five KOR-Bench domains performed best with moderate coverage, roughly in the 10% to 40% band. A later fixed-budget supervised fine-tuning pass improved most cells, but almost never closed the domain gaps under the paper’s bridging tests. Zero coverage badly hurt mid-training-only accuracy, though the authors say that result is mixed with generic drift. ArXiv · AI/CL/LG's note
In controlled logical-reasoning runs on Qwen3-8B-Base, each of five KOR-Bench domains performed best with moderate coverage, roughly in the 10% to 40% band. A later fixed-budget supervised fine-tuning pass improved most cells, but almost never closed the domain gaps under the paper’s bridging tests. Zero coverage badly hurt mid-training-only accuracy, though the authors say that result is mixed with generic drift. ArXiv · AI/CL/LG's note
score 5