- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper investigates how learning dynamics shape decoding-order preferences and advantages in masked diffusion models. Using a simplified graph-planning task, the authors establish an initial endpoint-first learning asymmetry and provide conditional theoretical results explaining how endpoint revelation can facilitate predecessor retrieval and yield a temporary right-to-left (RTL) decoding advantage. Experiments under joint training exhibit corresponding learning and decoding patterns.
Critical strengths
Well-motivated research question
The paper addresses an important question of how training shapes the effectiveness of different decoding orders, going beyond existing analyses of decoding strategies themselves.
Detailed theoretical analysis
The paper provides explicit theoretical conditions connecting learned attention circuits, prediction correctness, and confidence-based decoding-order selection, supported by detailed mathematical derivations.
Critical weaknesses
Limited generalizability
The theoretical analysis and empirical validation are confined to a highly simplified two-token graph-planning task and an attention-only Transformer, where the destination is already provided in the input. Both the theory and experiments remain within essentially the same restricted setting, leaving it unclear whether the findings extend to more general reasoning tasks or realistic diffusion language models.
Gap between theoretical analysis and joint training
The initial endpoint-first learning asymmetry is proved only locally, while the subsequent analysis relies on a prescribed stage-wise training regime with assumed endpoint readiness and preservation. In particular, the content weights governing destination matching are manually specified (Eq. 10) and frozen during Stage II, rather than learned through the analyzed dynamics. Consequently, the theory does not establish that ordinary joint training develops the proposed reverse-retrieval mechanism.
Conditional nature of the decoding guarantees
The finite-time descent result does not guarantee sufficiently low risk for a nontrivial reverse-circuit certificate, and the RTL advantage theorem relies on additional bootstrap conditions whose attainment is not established. Although joint-training experiments track prediction margins and reverse-route attention masses, they do not directly verify whether the sufficient conditions for the theoretical decoding guarantees are satisfied.
Questions for authors
Applicability beyond masked diffusion
The paper motivates its analysis through bidirectional prediction and flexible token ordering, which are not unique to masked diffusion models. To what extent do the theoretical findings depend on masked-token corruption and monotone unmasking, rather than on bidirectional prediction or the specific graph-planning task? Would similar learning dynamics and decoding-order advantages be expected in other diffusion formulations, such as uniform-state diffusion?
Verification of theoretical conditions under joint training
Are the sufficient conditions in Theorem 4.2 (the endpoint-reveal bootstrap condition) and Proposition 4.3 (the circuit-separation condition) satisfied along the joint-training trajectories shown in Figures 3 and 4? Directly evaluating these conditions would help clarify how closely the theoretical analysis explains the observed decoding behavior.
5 comments
Weakness3. Conditional nature of the decoding guarantees
The results of Sections 3.3 and 4 are certificates: each converts a measurable state of training into a guarantee on the circuit or on decoding. They are not convergence statements, and we have revised the text after Proposition 3.4 and Theorem 3.5 to separate the two roles.
Finite descent and the certificate. Proposition 3.4 establishes how Stage I couples to Stage II: endpoint readiness lower-bounds the predecessor query readiness (Lemma C.2), which gives on a finite interval. It does not claim convergence to arbitrarily low predecessor risk.
Theorem 3.5 is a separate statement: whenever the attained threshold satisfies , the route masses are at least and is at least . Under joint training these quantities do saturate: Figure 3(b) tracks the same four route masses, and all approach one as the losses in Figure 3(a) vanish.
Bootstrap condition. Lemma 4.1 holds at any Stage II state without further conditions: revealing the endpoint strictly increases query readiness () and the share of destination attention on . Since the total destination exposure can also change, this does not by itself imply a larger predecessor margin. Theorem 4.2 therefore states the remaining requirement directly on a measurable quantity, the revealed margin at first contact, , and converts it into a sequence-level guarantee.
Confidence-based decoding (Proposition 4.3). Proposition 4.3 establishes that dominance of the endpoint's token-mass gaps guarantees that both top-probability and top-margin decoding select the endpoint first. Figures 4(c–d) track reverse-order selection and the corresponding gap-dominance condition during joint training. We provide detailed condition checks in our response to Q2.
Theorem 4.4 assumes and characterizes the consequence. Both predictions are observed under joint training: all decoding orders reach accuracy one (Figure 3(c)) and the confidence gaps decay to zero (Figure 4(b)).
Q1. Applicability beyond masked diffusion
Thank you for the question. Our results use two properties, with different scope.
(1) Mask-conditioned training. Section 3 concerns the four conditional predictors, i.e., each answer token with the other one masked or revealed. Models trained on the same conditionals, such as other iterative infilling models, face the same learning problem, so we expect the endpoint-first asymmetry and the reverse-retrieval circuit to carry over.
(2) Monotone unmasking. Section 4 additionally uses that masked positions are identifiable and that decoding commits tokens without remasking, so a correct first prediction produces exactly the revealed state seen in training. The success identities and the bootstrap analysis rest on this.
Uniform-state diffusion [7] differs in both respects: corrupted tokens are ordinary vocabulary tokens and are not marked, and the reverse process may revise a token several times, so there is no fixed reveal order to compare. Our decoding guarantees therefore do not apply as stated. The corresponding question in that setting is which positions the model corrects first and whether early correct tokens help the rest; this requires the uniform corruption states and reverse transitions, and we leave it for future work.
[7] Structured Denoising Diffusion Models in Discrete State-Spaces, NeurIPS 2021
Q2. Verification of theoretical conditions under joint training
We evaluated the conditions of Theorem 4.2 and Proposition 4.3, as stated in the revision, along the joint-training run of Figure 4. Both hold throughout the window in which RTL outperforms LTR.
Theorem 4.2. The condition is , where is the first time the both-masked predecessor margin reaches zero. The revised theorem states this condition directly in terms of the prediction margin. The previous condition involving implies the revised condition, so the theorem now holds under a weaker assumption.
In the experiment, the mean both-masked margin changes sign between steps 180 and 186, while the mean endpoint-revealed margin is positive at both steps. The predicted separation, , is observed during steps 132–180, coinciding with the interval in which RTL outperforms LTR:
| Step | RTL accuracy | LTR accuracy | ||
|---|---|---|---|---|
| 132 | 0.000615 | −0.226 | 57.74% | 0% |
| 156 | 0.00172 | −0.162 | 68.51% | 0% |
| 180 | 0.00404 | −0.0121 | 97.29% | 0% |
| 186 | 0.00508 | 0.0174 | 100% | 100% |
The theorem is stated for symmetry-preserving population gradient flow, where all inputs behave identically and decoding accuracy is either 0 or 1. In contrast, the experiments use random initialization, so prediction margins vary across inputs. Accordingly, RTL accuracy represents the fraction of inputs for which the endpoint is correctly predicted and . This fraction increases from 57.74% to 97.29% while LTR accuracy remains at 0%, consistent with the endpoint-reveal bootstrap mechanism.
Proposition 4.3 The revised sufficient condition is and for all , with strict inequality for at least one , where denotes the sorted token-mass gaps computed from Eq. (22).
The previous route-based condition is not satisfied along the joint-training trajectory, so we revised this condition to characterize reverse-confidence selection directly through the output token-mass distribution.
To evaluate the revised condition, we compute and across the evaluated inputs.
| Step | |||||
|---|---|---|---|---|---|
| 10 | 0.0343 | −4.86e−6 | −8.89e−6 | −1.27e−5 | −1.55e−5 |
| 30 | 0.0510 | −3.44e−5 | −5.12e−5 | −7.91e−5 | −9.77e−5 |
| 50 | 0.0943 | −1.75e−5 | −1.91e−6 | −1.02e−6 | −1.27e−5 |
| 60 | 0.117 | 2.81e−5 | 1.20e−4 | 1.62e−4 | 1.49e−4 |
| 100 | 0.215 | 0.00457 | 0.00932 | 0.0131 | 0.0159 |
| 200 | 0.589 | 0.556 | 1.09 | 1.58 | 2.00 |
| 400 | 0.957 | 0.127 | 0.250 | 0.339 | 0.421 |
is positive at all training steps and initially negative and becomes positive around 50~60 steps (exact turning step depends on ) and then continually positive until it ends. -dynamics is illustrated in Figure 4(d). This also agrees with the positive confidence gaps and the preference for reverse decoding observed in Figures 4(b), 4(c).
Dear reviewer S2Lq,
Thank you for the detailed assessment and identifying the distinction between a stage-wise framework and the attainment under joint training. We address the theoretical scope, the requested condition checks, and the role of masked corruption below.
Weakness1. Limited generalizability
Our goal is to explain a specific, empirically observed phenomenon, namely reverse reasoning [1] and reverse decoding [2] in planning, rather than to give a theory of all dLLMs. As discussed in [3], the key factor is that different decoding orders lead to subproblems of different difficulty. A setting that captures this needs two properties:
(i) at least two prediction targets, one computationally easy and one hard;
(ii) solving the easy one first makes the hard one easier.
The two-token path-star task is the minimal setting with both. Predicting the endpoint a2 is a copy of the destination in the query (Eq. (1)), while predicting the predecessor a1 requires matching the destination edge and retrieving its source. Since the destination is already in the input, revealing a2 adds no new task information; it only changes how the model can access it. Any RTL advantage is therefore attributable to the learned computation, which is what we analyze (Proposition 3.3, Lemma 4.1, Theorem 4.2).
The experiments are not restricted to the theoretical setting. Appendix E uses random initialization, AdamW, minibatches and a train/test split, all of which break the symmetry and gradient-flow assumptions of the theory, and the same circuit and decoding patterns appear for G(2,4), G(3,4) and G(4,4). Extending the analysis to longer paths and to tasks without a copied destination is left for future work.
[1] How Transformers Learn to Plan via Multi-Token Prediction, COLM 2026
[2] Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks, arXiv 2026
[3] Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions, ICML 2025
Weakness2. Gap between theoretical analysis and joint training
The paper contains results at two levels; the distinction is stated in the Introduction and at the beginning of Section 3.2.
Joint training. Theorem 3.1 is proved for ordinary joint gradient flow, without staging: early in training the endpoint loss falls below the predecessor loss. Figure 3(a) shows the same order empirically.
Stage-wise framework. Motivated by this asymmetry, we analyze endpoint preparation and predecessor learning in two stages. Stage-wise training is a standard device for provable dynamics in multi-layer transformers [4, 6], where the joint flow is not tractable. Here the staging is not imposed a priori; it follows the order established by Theorem 3.1.
Content weights in Eq. 10 (Eq. 9 in the revision). The content matrices are chosen from the symmetry-induced family of Lemma 2.1 and kept fixed in Stage II. This does not prescribe the circuit: the positional weights remain trainable and determine paired-source selection and exposure to the graph-role pools, i.e., which route is formed. To check that the chosen form is the one joint training finds, we have added Appendix E.3, which tracks both content matrices when all blocks are trained jointly from random initialization. Layer 1 develops a negative diagonal and a negative [MASK] column, and Layer 2 a positive diagonal, as in Eq. 9. This structure is visible for both G(2,3) and G(3,4).
All experiments in the paper train content and positional blocks jointly, with no staging or content freeze, and show the chronology of the stage-wise analysis (Figure 3). Proving that joint training reaches the Stage II entry conditions remains open, as stated in the Conclusion.
[4] How Transformers Learn Causal Structure with Gradient Descent, ICML 2024
[5] Focus and Dilution: The Multi-stage Learning Process of Attention, ICML 2026
[6] Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers, NeurIPS 2024