Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts
The paper argues LoRA expert routing should spend compute where another adapter can measurably reduce risk, not where the model merely feels uncertain.
VI-MoLE estimates the counterfactual risk left after each expert prefix and uses calibrated upper-risk certificates to choose the next token-layer adapter action. The router allocates a global adapter budget to the largest certified marginal risk reduction per cost, then uses a terminal certificate to answer or abstain. The authors claim validity guarantees, greedy optimality under diminishing certified gains, and regret bounds under estimation error. Evaluation is set up against fixed and dynamic MoE-LoRA routers on matched compute, coverage, shift, and latency.
ArXiv · AI/CL/LG's note
VI-MoLE estimates the counterfactual risk left after each expert prefix and uses calibrated upper-risk certificates to choose the next token-layer adapter action. The router allocates a global adapter budget to the largest certified marginal risk reduction per cost, then uses a terminal certificate to answer or abstain. The authors claim validity guarantees, greedy optimality under diminishing certified gains, and regret bounds under estimation error. Evaluation is set up against fixed and dynamic MoE-LoRA routers on matched compute, coverage, shift, and latency.
ArXiv · AI/CL/LG's note
score 4