acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Order Policies for Masked Diffusion

Abstract

Masked diffusion models have seen great success in capturing data distributions over discrete sequences in domains such as text and proteins. These models generate data by iteratively unmasking tokens starting from a fully masked sequence, with the unmasking order typically chosen at random or using a heuristic based on denoiser probabilities. In this work, we propose a scheme for learning the unmasking order using an additional lightweight policy network on top of a diffusion model. Our proposed loss reweights terms in the masked diffusion loss according to policy probabilities, and results in a policy that prefers positions where the denoiser is more likely to be correct. We study this loss in two settings: (i) training solely the policy while using a frozen pre-trained denoiser, and (ii) training the policy and denoiser jointly with the weighted loss to allow for mutual adaptation. We demonstrate that our approach outperforms common heuristics on problems that are sensitive to token ordering, such as combinatorial problems, proteins as well as various coding and language tasks.

Primary area:
generative models
Keywords:
masked diffusion, discrete diffusion, fine-tuning, unmasking orders

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zm97OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes learning the token unmasking order in masked diffusion models instead of relying on heuristics based on quantities like confidence or margin. The authors introduce a lightweight policy network trained to favor positions where the denoiser is more likely to predict the correct token, and explore both training the policy alone and jointly training it with the denoiser. They evaluate the approach on across a wide range of tasks, generally finding improvements over heuristic ordering strategies. However, the gains are often modest, especially on larger language models like LLaDA-8B, where the learned policy does not consistently outperform high-confidence ordering.

Critical strengths

1. The method itself is conceptually simple and elegant. It only requires a lightweight network, which doesn't add much to inference compute, and is generally applicable to a wide range of models.

2. The evaluation covers a very good range of domains, including Sudoku, protein generation, and language modeling. The ablations are also useful, demonstrating improved accuracy with fewer diffusion steps, as well as robustness to stochastic decoding.

Critical weaknesses

1. The evaluation primarily compares against decoding order heuristics, such as high confidence, margin, and random. While the learned policy generally improves upon these heuristics, it's unclear to me how the proposed method would perform against other learned order baselines, such as [1,2,3], which aren't included. Such comparison is required to demonstrate the empirical contribution of the proposed method relative to the closest baselines; comparing only against the heuristics isn't sufficient to show the benefit of the proposed method.

2. I'm not convinced by the novelty or impact of the proposed method. While the training objective is elegant and simple, the underlying idea of prioritizing positions where the denoiser is more likely to make correct predictions is pretty intuitive and closely related to existing confidence-based ordering strategies. This would be less concerning if the empirical improvements were substantial, but the gains over training-free heuristics are generally modest; for example, on LLaDA-8B, the learned policy only slightly outperforms high-confidence ordering on some benchmarks, while performing worse on others. Given the limited algorithmic novelty and relatively modest empirical gains, I'm not convinced that the paper demonstrates a sufficiently significant contribution.

Questions for authors
  1. Major: Can the authors provide comparisons against existing learned-order approaches, such as [1,2,3], under comparable experimental settings? If direct comparisons are infeasible, could the authors explain why and clarify what advantages their method offers over these approaches?

  2. Major: The learned policy achieves only marginal improvements over high-confidence ordering on some LLaDA-8B benchmarks and performs worse on others (e.g., BBH and CoQA). Can the authors provide an answer as to why this occurs? Why does the learned order policy not work as well at larger scale?

  3. Major: Since high-confidence and margin ordering require no additional training, can the authors quantify the additional computational cost of policy training and discuss whether the observed improvements justify this cost? How does the training efficiency compare with existing learned-order approaches?

Concerns addressed by author responses, increased score from 2 to 3.

[1] Hong et al., ICLR 2026, "Improving Discrete Diffusion Unmasking Policies Beyond Explicit Reference Policies".

[2] Xu et al., ICML 2026, "Scheduling Thoughts: Learning the Order of Thought in Diffusion Language Models".

[3] Jazbec et al., ICML 2026, "Learning Unmasking Policies for Diffusion Language Models".

Reviewer_ZXyLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Masked diffusion models have emerged as an alternative framework to autoregressive models. Recently, many prior works have recognized that the unmasking order is one of the main factors affecting their performance. This paper aims to address this problem by learning an unmasking policy model that can be utilized at generation time. It considers two approaches: 1) jointly training the unmasking policy and token prediction models, and 2) training the unmasking policy model independently.

Critical strengths

The experiments cover a wide range of domains.

Critical weaknesses

The main weakness of this paper is that it does not provide any new insights or theoretical foundations in this field, nor does it adequately compare its empirical performance with highly related prior works. The three papers that should be compared in detail are:

[1] Peng et al., "Planner Aware Path Learning in Diffusion Language Models Training", ICLR 2026.

[2] Hong et al., "Unifying Masked Diffusion Models with Various Generation Orders and Beyond", ICML 2026.

[3] Peng et al., "Path Planning for Diffusion Language Model Sampling", arXiv 2025.

Among these, [1] is cited in the paper, whereas [2] and [3] are not. I have two main concerns regarding these three prior works:

  1. The concept of learning the unmasking order is the same, and there is no theoretical or empirical advancement.

The paper heavily relies on the theory and concepts of [1]. The only difference I see is that this paper explicitly uses an unmasking policy model. This already makes the contribution marginal. While the authors mention in the Related Work section that their contribution beyond [1] is that they allow joint training of token prediction and unmasking prediction, [2] has already proposed a unified framework for jointly learning both token prediction and the unmasking policy model. Moreover, [2] derives an exact NELBO for the joint training process, whereas this paper relies on a theorem from [1] and even incorporates practical approximations (rather than exact NELBO-based training). Finally, although [1] and [2] explicitly mention and discuss the earlier work [3], and the overall methodology of this paper highly overlaps with that of [3], this paper does not include any discussion of it.

  1. The paper does not adequately compare its performance with prior works [1], [2], and [3].

If the paper does not provide new insights or theoretical results, the authors should at least demonstrate performance improvements over prior works. However, the experiments mainly compare the proposed method with confidence- or margin-based sampling. Only Table 2 provides a comparison with [1].

Questions for authors

Please see weaknesses above, clarify the theoretical or conceptual advancement beyond two prior works, and provide experimental comparisons.

Reviewer_JvpPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a policy network to learn the token unmasking order in masked diffusion models, using a weighted training objective that prioritizes positions where the denoiser is more likely to be correct. Experiments show that the proposed approach outperforms existing benchmarks in various different tasks.

Critical strengths

This paper provides new insight to the order policy for masked diffusion. The numerical performance is very good.

Critical weaknesses
  1. Line 143 "Using an ELBO decomposition for such a process, derived by Peng et al. (2025a), we can show that our loss in Equation (3) results from making approximations to avoid computing intractable terms. We present the details of this derivation in Appendix C." However, the derivation in Appendix C is not rigorous but more like explanations.

  2. Methods seem not explained clear enough and make the reader wonder why this works.

Questions for authors

My questions for authors are to provide more details addressing the critical weaknesses.

Comments on the paper

1 comment
Area_Chair_s6cdOpenReview ↗

Please respond

Hi reviewers,

Thanks for your work so far on this paper. Please respond to the authors before the end of the discussion period if you have not done so already, even if to acknowledge their response.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.