SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding
SmartMage routes each 3D-scene query to the modalities the paper says are most useful, instead of forcing a fixed RGB/geometric mix.
The model adds a SMART router to select modalities using semantic priors, text-modality alignment, and modality quality. A MAGE gating module then uses modality priors to steer expert activation for multimodal reasoning. The authors report state-of-the-art results across five 3D scene understanding benchmarks, plus competitive RGB-only video results. They also introduce ScanFacet to inspect which modality combinations different semantic task types prefer. HF Daily Papers' note
The model adds a SMART router to select modalities using semantic priors, text-modality alignment, and modality quality. A MAGE gating module then uses modality priors to steer expert activation for multimodal reasoning. The authors report state-of-the-art results across five 3D scene understanding benchmarks, plus competitive RGB-only video results. They also introduce ScanFacet to inspect which modality combinations different semantic task types prefer. HF Daily Papers' note
score 5