\k{appa}-LoRA: Condition Numbers Reveal Which LoRA Matrices Worth Updating
The paper says LoRA can skip half its matrix updates by ranking them with condition numbers.
The authors argue that high-condition-number LoRA matrices carry the useful adaptation signal, while lower ones add little. Their κ-LoRA method updates only the top 50% of matrices by that measure, cutting trainable parameters in half. Across the reported benchmarks, it matches standard LoRA accuracy while reducing fine-tuning time by 16.2% on average and memory cost by 4.5%. ArXiv · AI/CL/LG's note
The authors argue that high-condition-number LoRA matrices carry the useful adaptation signal, while lower ones add little. Their κ-LoRA method updates only the top 50% of matrices by that measure, cutting trainable parameters in half. Across the reported benchmarks, it matches standard LoRA accuracy while reducing fine-tuning time by 16.2% on average and memory cost by 4.5%. ArXiv · AI/CL/LG's note
score 5