MoE--LoRA Methods for Multilingual Adaptation: A Systematic Empirical Study
LoRA improved model fit on translation data but consistently worsened reading comprehension.
The study compared eight low-rank adaptation methods on a frozen base model across 200 languages and more than 9,000 evaluation settings. Low-resource languages benefited most. Smaller searches identified viable configurations, but their top-ranked configuration was never best at scale. The authors conclude that these methods are targeted tools rather than general-purpose upgrades to multilingual capability.
ArXiv · AI/CL/LG's note
The study compared eight low-rank adaptation methods on a frozen base model across 200 languages and more than 9,000 evaluation settings. Low-resource languages benefited most. Smaller searches identified viable configurations, but their top-ranked configuration was never best at scale. The authors conclude that these methods are targeted tools rather than general-purpose upgrades to multilingual capability.
ArXiv · AI/CL/LG's note
score 4