REVISE WHAT MATTERS: ATTENTION-PRIORITIZED VERIFICATION FOR DIFFUSION LANGUAGE MODELS

Diffusion language models enable parallel token generation, but existing revocable decoders often treat uncertainty as sufficient evidence for revision, causing many suspicious predictions to be remasked even when revision provides little benefit. We argue that effective revision should consider not only whether a token is unreliable, but also whether correcting it is likely to matter to the evolving decoding context. We propose CAPER (Context-Aware Probation for Efficient Revision), a training-free revocable decoding framework that introduces a one-step probation state between token commitment and remasking. CAPER combines assignment uncertainty with incoming attention from already decoded and newly drafted positions to prioritize contextually consequential candidates for verification. During probation, priority-weighted drafting limits the influence of unresolved predictions on concurrent generation, while verification determines whether their current assignments should be retained or revoked. Drafting and verification are integrated into the same decoding step, allowing CAPER to concentrate revision on actionable predictions without sacrificing parallel generation. Experiments on LLaDA-8B-Base and Dream-Base-7B across code generation and mathematical reasoning demonstrate a consistently stronger quality–efficiency trade-off than existing irreversible and revocable decoding methods.