New LoRA Skills Should Read but Never Write
READ lets each added adapter use earlier skills without overwriting their outputs.
The paper frames LoRA composition as a factorization and direction problem, not just a merging problem. Its method rewrites each adapter into a balanced canonical form, then trains only the new skill’s row in a one-way coupling matrix. The composed update can be folded into the base model, avoiding routing and added inference cost. In tests across four benchmark suites and two model families, the authors report stronger averages than published baselines built from the same adapters. ArXiv · AI/CL/LG's note
The paper frames LoRA composition as a factorization and direction problem, not just a merging problem. Its method rewrites each adapter into a balanced canonical form, then trains only the new skill’s row in a one-way coupling matrix. The composed update can be folded into the base model, avoiding routing and added inference cost. In tests across four benchmark suites and two model families, the authors report stronger averages than published baselines built from the same adapters. ArXiv · AI/CL/LG's note
score 4