Query-Efficient Rejection Sampling for Parallel Masked Diffusion
SIRS estimates the log acceptance probability with one score query, replacing an evaluation that queries every unmasked token.
Masked diffusion generates tokens in parallel, but sampling them independently ignores correlations and biases the output. The proposed sampler uses a random number of clipped estimates to make accept-or-reject decisions. It also averages information about remaining tokens returned by each query, reducing clipping at no extra query cost. The authors bound its output’s divergence from the data distribution by a clipping term plus the model’s training error.
ArXiv · AI/CL/LG's note
Masked diffusion generates tokens in parallel, but sampling them independently ignores correlations and biases the output. The proposed sampler uses a random number of clipped estimates to make accept-or-reject decisions. It also averages information about remaining tokens returned by each query, reducing clipping at no extra query cost. The authors bound its output’s divergence from the data distribution by a clipping term plus the model’s training error.
ArXiv · AI/CL/LG's note
score 5