MLLMs Hallucinate when Information Distribution Drifts in Synergy Heads
The paper ties MLLM hallucinations to drift inside “synergy heads,” not simply to weak visual or language attention.
The authors propose HEAL, a head-level method that uses causal noise intervention and counterfactual Difference-in-Differences to sort attention heads by information behavior. Their analysis says hallucinations emerge when synergy heads move away from a “healthy equilibrium” in how they distribute information. HEAL then calibrates the value vectors in those heads to regulate visual-language dependencies and push outputs toward factual evidence. The paper reports reduced hallucinations across multiple MLLMs. HF Daily Papers' note
The authors propose HEAL, a head-level method that uses causal noise intervention and counterfactual Difference-in-Differences to sort attention heads by information behavior. Their analysis says hallucinations emerge when synergy heads move away from a “healthy equilibrium” in how they distribute information. HEAL then calibrates the value vectors in those heads to regulate visual-language dependencies and push outputs toward factual evidence. The paper reports reduced hallucinations across multiple MLLMs. HF Daily Papers' note
score 4