Rethinking Factor Sharing in Federated LoRA: A Rank-Aware Adaptive Approach
FedAS-LoRA chooses which LoRA factor to share by testing whether clients have enough common low-rank structure before training.
The paper compares Share-A/Local-B with Share-B/Local-A and argues they fail for different geometric reasons. Its RSS metric uses frozen-backbone representations to estimate whether a shared rank-r input subspace is sufficient for the clients’ data. Experiments across tasks, distributions, LoRA ranks, and participation settings report that this pre-training choice improves fine-tuning performance. ArXiv · AI/CL/LG's note
The paper compares Share-A/Local-B with Share-B/Local-A and argues they fail for different geometric reasons. Its RSS metric uses frozen-backbone representations to estimate whether a shared rank-r input subspace is sufficient for the clients’ data. Experiments across tasks, distributions, LoRA ranks, and participation settings report that this pre-training choice improves fine-tuning performance. ArXiv · AI/CL/LG's note
score 4