Selective State-Space Adaptation and Retrieval for Language Model Reasoning
The paper adds recurrent, input-dependent state to LoRA-style adapters and reports consistent reasoning gains over LoRA.
MaLoRA modulates adapter scaling across tokens with Mamba-style recurrence, while MaRA tracks cross-segment context and retrieves segments relevant to the query. The authors test three frozen backbones: Qwen-2.5-7B, Llama-3.1-8B, and Gemma-2-9B. On MuSiQue and 2WikiMultihopQA, the adapter family improves every model-benchmark cell, averaging +6.8 F1 over LoRA. The abstract also says token-level gains carry to RULER QA-2 under length stress. ArXiv · AI/CL/LG's note
MaLoRA modulates adapter scaling across tokens with Mamba-style recurrence, while MaRA tracks cross-segment context and retrieves segments relevant to the query. The authors test three frozen backbones: Qwen-2.5-7B, Llama-3.1-8B, and Gemma-2-9B. On MuSiQue and 2WikiMultihopQA, the adapter family improves every model-benchmark cell, averaging +6.8 F1 over LoRA. The abstract also says token-level gains carry to RULER QA-2 under length stress. ArXiv · AI/CL/LG's note
score 4