acceptodds
Desk rejected at ICLR 2027

Block Reflection: Training-Free Reasoning Repair for Discrete Diffusion Language Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Discrete diffusion large language models (dLLMs) generate text by iteratively denoising masked sequences under bidirectional context, which gives them a revision mechanism unavailable to autoregressive decoders. Existing block-wise decoding schemes, however, treat each revealed block as a fixed commitment. Once later blocks are drafted, earlier reasoning steps may become inconsistent with the new future context, yet the decoder cannot identify and repair the flawed block without expanding external candidates. We propose Block Reflection, a training-free inference procedure that turns this unused revision capability into online self-correction. To decide which committed block to regenerate, we design a dLLM-native instability signal: the drift of a block's conditional token distribution before and after new future context is drafted. Block Reflection remasks and regenerates only the most unstable block and keeps the rest of the trajectory fixed, with no external verifier, fine-tuning, or full-trajectory regeneration. Across four math and code benchmarks on 4 different dLLMs, Block Reflection consistently improves accuracy, with large gains of 6.2% on MATH-500 and 10.2% on MBPP.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Large Language Model

Related papers

Discussion (0)

Sign in to comment.