acceptodds
Under review as a conference paper at ICLR 2027

RETHINKING WORKSPACE CONTROL FOR OPEN-ENDED DIFFUSION LANGUAGE MODELS

Abstract

Masked diffusion language models (dLLMs) generate text by resolving masked positions in parallel, but open-ended generation remains challenging because the final output length is unknown in advance. Existing methods typically rely on model-derived signals such as EOS statistics, confidence, or prediction density to dynamically adjust the workspace, introducing additional estimation and control complexity. We find that, with the decoding state fixed and these control signals bypassed, directly changing future [MASK] exposure can substantially alter current decoding decisions. This suggests a simpler principle: rather than repeatedly estimating workspace demand from model outputs, future workspace can be organized directly around decoding progress. We therefore introduce MaskHorizon, a framework for open-ended dLLM decoding. MaskHorizon maintains a local horizon of future [MASK] positions that advances with block-level decoding progress. Each forward pass predicts over the workspace up to this horizon, while commitment remains restricted to the active block. Experiments on LLaDA and Dream across reasoning and code-generation tasks show that MaskHorizon substantially reduces decoding computation while retaining competitive generation quality. The code for MaskHorizon is available at https://anonymous.4open.science/r/MaskHorizon/.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.