Test-Time Scaling for Diffusion Language Models with Frequency-Aware Remasking
Tracking which token predictions keep changing helped diffusion models solve questions that confidence-based decoding left unanswered.
The method runs a fast sampler first, then applies frequency-aware remasking to harder questions. It preferentially commits tokens whose predictions change frequently across denoising steps, encouraging alternative reasoning paths. Against a strong adaptive allocation baseline, reported gains reached 7.33 percentage points in MATH-500 accuracy and 11.39 points in HumanEval coverage across the two tested models. It also recovered solutions for 3.75%–18.75% of questions still unsolved after 100 samples from the base certainty-aware strategy.
HF Daily Papers' note
The method runs a fast sampler first, then applies frequency-aware remasking to harder questions. It preferentially commits tokens whose predictions change frequently across denoising steps, encouraging alternative reasoning paths. Against a strong adaptive allocation baseline, reported gains reached 7.33 percentage points in MATH-500 accuracy and 11.39 points in HumanEval coverage across the two tested models. It also recovered solutions for 3.75%–18.75% of questions still unsolved after 100 samples from the base certainty-aware strategy.
HF Daily Papers' note
score 5