Rethinking in Spikes: Mitigating Hallucinations in MDLMs with Step-Aware Decoding
Abstract
StepRefine, a plug-and-play decoding strategy, mitigates hallucinations in multimodal diffusion language models by switching between discrete denoising and latent refinement based on entropy spikes and distributional stability.
Recent advancements in multimodal diffusion language models (MDLMs) have exhibited strong global modeling capabilities through iterative refinement. We observe that mask uncertainty follows an overall downward trend during iterative denoising, while a few denoising steps exhibit localized entropy spikes closely associated with hallucinations. We argue that anomalous steps can be identified by measuring the entropy deviation of each denoising step from its neighboring temporal window. We hypothesize that discrete token commitment discards the distributional information at masked positions, limiting the continued interaction between candidate semantics and the context during entropy spikes. With this goal, we present StepRefine, a plug-and-play decoding strategy that leverages semantic context to achieve reliable denoising. The core of our method lies in a step-aware switch between discrete denoising and latent refinement. The model employs probability-weighted embeddings at entropy spike steps to preserve diverse reasoning hypotheses and performs multiple rounds of continuous refinement. Meanwhile, a refinement controller monitors distributional stability to reduce unnecessary loops and switch the model back to standard discrete denoising. Through extensive experiments, StepRefine demonstrates significant hallucination mitigation across different MDLMs on multimodal benchmarks.
Community
• We analyze the relationship between entropy spike steps and hallucinations in multimodal
diffusion models from the perspective of uncertainty dynamics across denoising steps.
• We propose StepRefine, a plug-and-play decoding approach that mitigates hallucinations
through entropy spike detection and step-aware mode switching.
• Extensive evaluations across four MDLM backbones and diverse multimodal benchmarks
show the superior performance of StepRefine, offering an effective solution for multimodal
diffusion hallucinations.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- VIGOR: Visual Gain Ordering for Hallucination Mitigation in Multimodal Discrete Diffusion Language Models (2026)
- DRAMA: Dissecting Attention Redundancy for Accelerating Multimodal Diffusion Large Language Models (2026)
- Alleviating Hallucination in Reasoning Tasks with Training-Free Uncertainty-Guided Steering (2026)
- Entropogram-guided Visual Realignment Decoding: Capturing the Rhythm of Multimodal Reasoning (2026)
- TRACESTEER: Trace Driven Activation Steering for Hallucination Mitigation in Diffusion Language Models (2026)
- HALMES: Knowing When to Intervene in LVLM Hallucination Mitigation (2026)
- Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper