Relax Within, Balance Across: Geometry-Guided Load Balancing for Vision-Language Mixture-of-Experts
ReBA targets a load-balancing failure that can be hidden when image and text token loads are mixed together.
The paper says standard Switch auxiliary loss can let image and text load errors cancel out, leaving routers sensitive to shifts in resolution, tiling, image count, and prompt length. The authors derive how the load curve changes with the image-text token mix, then use observed router geometry to split balancing by modality and by source image. Across four split backbones, ReBA lowered reported load imbalance while keeping mean task accuracy comparable to Std-Aux. HF Daily Papers' note
The paper says standard Switch auxiliary loss can let image and text load errors cancel out, leaving routers sensitive to shifts in resolution, tiling, image count, and prompt length. The authors derive how the load curve changes with the image-text token mix, then use observed router geometry to split balancing by modality and by source image. Across four split backbones, ReBA lowered reported load imbalance while keeping mean task accuracy comparable to Std-Aux. HF Daily Papers' note
score 4