Beyond One-Size-Fits-All: Sample-Adaptive Strategy Routing for Vision Token Pruning in MLLMs
VIP-Router chooses a vision-token pruning method per input instead of forcing one pruning rule across all samples.
The paper argues that average benchmark accuracy hides cases where a non-best pruning strategy works better for a specific sample. VIP-Router uses low-cost visual and textual features to pick among candidate pruning strategies, with full-token inference still available when pruning looks harmful. On VTC-Bench Group A, the authors report a 26.9% relative gain in average accuracy over the best fixed strategy baseline and a 22.0% relative gain in average utility after token cost. The router is described as plug-and-play, adding trainable parameters equal to 0.017% of the backbone, without changing model weights or pruning algorithms. ArXiv · AI/CL/LG's note
The paper argues that average benchmark accuracy hides cases where a non-best pruning strategy works better for a specific sample. VIP-Router uses low-cost visual and textual features to pick among candidate pruning strategies, with full-token inference still available when pruning looks harmful. On VTC-Bench Group A, the authors report a 26.9% relative gain in average accuracy over the best fixed strategy baseline and a 22.0% relative gain in average utility after token cost. The router is described as plug-and-play, adding trainable parameters equal to 0.017% of the backbone, without changing model weights or pruning algorithms. ArXiv · AI/CL/LG's note
score 5