Megadose The AI curve, compressed.

Understanding Cross-Modal Inconsistency through Representation Subspaces in Omni Models

· ArXiv · AI/CL/LG ·
The signals that predict disagreement across audio, vision and text are distinct from the representations whose disruption causes it.

Across three Omni Models and two multimodal reasoning tasks, researchers found that disagreement-predictive information concentrated mainly outside a low-dimensional “Answer Subspace.” Perturbing those predictive signals barely changed answers, while perturbing the Answer Subspace substantially changed predictions. Altering the Answer Subspace for just one modality induced disagreement; applying the same alteration to both largely preserved agreement. The results show that information readable from a model’s representations does not necessarily influence its answers.

ArXiv · AI/CL/LG's note

score 4

Categories: Research