LOCUS: Task-Aware Low-Rank Post-Training for Token-Efficient Language Generation
LOCUS cuts generated token length while leaving the preference objective intact.
The paper tests whether low-rank post-training can reduce verbosity without changing the alignment loss. LOCUS chooses a task-aware adaptation subspace, keeps the backbone frozen, and updates only 0.24-0.28% of parameters. On Anthropic HH-RLHF preferences, it reports continuation-length reductions up to 39.84% for Pythia-2.8B and 14.87-17.58% for Qwen2.5-3B, with no material change in its internal preference diagnostic. ArXiv · AI/CL/LG's note
The paper tests whether low-rank post-training can reduce verbosity without changing the alignment loss. LOCUS chooses a task-aware adaptation subspace, keeps the backbone frozen, and updates only 0.24-0.28% of parameters. On Anthropic HH-RLHF preferences, it reports continuation-length reductions up to 39.84% for Pythia-2.8B and 14.87-17.58% for Qwen2.5-3B, with no material change in its internal preference diagnostic. ArXiv · AI/CL/LG's note
score 5