Rethinking in Spikes: Mitigating Hallucinations in MDLMs with Step-Aware Decoding
StepRefine preserves competing token possibilities when a model’s uncertainty spikes, delaying a discrete choice.
The researchers associate brief entropy spikes during multimodal diffusion decoding with hallucinations. Their plug-and-play method detects those steps and uses probability-weighted embeddings for repeated continuous refinement. A controller tracks distributional stability, limits unnecessary refinement and returns the model to standard decoding. The authors report reduced hallucinations across multiple models and multimodal benchmarks, though the supplied abstract gives no numerical results.
HF Daily Papers' note
The researchers associate brief entropy spikes during multimodal diffusion decoding with hallucinations. Their plug-and-play method detects those steps and uses probability-weighted embeddings for repeated continuous refinement. A controller tracks distributional stability, limits unnecessary refinement and returns the model to standard decoding. The authors report reduced hallucinations across multiple models and multimodal benchmarks, though the supplied abstract gives no numerical results.
HF Daily Papers' note
score 4