acceptodds
Under review as a conference paper at ICLR 2027

How Learning Dynamics Shape Decoding in Masked Diffusion Models

Abstract

Diffusion Language Models have recently emerged as a promising alternative for reasoning, benefiting from bidirectional prediction and flexible token ordering at inference. However, how training shapes the usefulness of this flexibility remains unclear. In this paper, we study stage-wise learning dynamics in a masked diffusion model using a two-layer Transformer with a fixed output head on a graph-planning task. A local endpoint-first asymmetry under ordinary joint training motivates a framework separating endpoint preparation from predecessor learning. Within this framework, we establish a finite predecessor-risk decrease and derive risk-dependent reverse-circuit certificates. We derive conditions for endpoint reveal to bootstrap predecessor prediction and for top-probability and top-margin decoding to favor the reverse order, distinguishing sequence-level correctness from confidence-based selection. We also characterize order independence and confidence agreement when all mask-conditioned risks vanish. Together, our analysis relates stage-wise learning and internal computation to decoding preference and advantage.

Primary area:
learning theory
Keywords:
masked diffusion models, learning dynamics, decoding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_S2LqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates how learning dynamics shape decoding-order preferences and advantages in masked diffusion models. Using a simplified graph-planning task, the authors establish an initial endpoint-first learning asymmetry and provide conditional theoretical results explaining how endpoint revelation can facilitate predecessor retrieval and yield a temporary right-to-left (RTL) decoding advantage. Experiments under joint training exhibit corresponding learning and decoding patterns.

Critical strengths

Well-motivated research question

The paper addresses an important question of how training shapes the effectiveness of different decoding orders, going beyond existing analyses of decoding strategies themselves.

Detailed theoretical analysis

The paper provides explicit theoretical conditions connecting learned attention circuits, prediction correctness, and confidence-based decoding-order selection, supported by detailed mathematical derivations.

Critical weaknesses

Limited generalizability

The theoretical analysis and empirical validation are confined to a highly simplified two-token graph-planning task and an attention-only Transformer, where the destination is already provided in the input. Both the theory and experiments remain within essentially the same restricted setting, leaving it unclear whether the findings extend to more general reasoning tasks or realistic diffusion language models.

Gap between theoretical analysis and joint training

The initial endpoint-first learning asymmetry is proved only locally, while the subsequent analysis relies on a prescribed stage-wise training regime with assumed endpoint readiness and preservation. In particular, the content weights governing destination matching are manually specified (Eq. 10) and frozen during Stage II, rather than learned through the analyzed dynamics. Consequently, the theory does not establish that ordinary joint training develops the proposed reverse-retrieval mechanism.

Conditional nature of the decoding guarantees

The finite-time descent result does not guarantee sufficiently low risk for a nontrivial reverse-circuit certificate, and the RTL advantage theorem relies on additional bootstrap conditions whose attainment is not established. Although joint-training experiments track prediction margins and reverse-route attention masses, they do not directly verify whether the sufficient conditions for the theoretical decoding guarantees are satisfied.

Questions for authors

Applicability beyond masked diffusion

The paper motivates its analysis through bidirectional prediction and flexible token ordering, which are not unique to masked diffusion models. To what extent do the theoretical findings depend on masked-token corruption and monotone unmasking, rather than on bidirectional prediction or the specific graph-planning task? Would similar learning dynamics and decoding-order advantages be expected in other diffusion formulations, such as uniform-state diffusion?

Verification of theoretical conditions under joint training

Are the sufficient conditions in Theorem 4.2 (the endpoint-reveal bootstrap condition) and Proposition 4.3 (the circuit-separation condition) satisfied along the joint-training trajectories shown in Figures 3 and 4? Directly evaluating these conditions would help clarify how closely the theoretical analysis explains the observed decoding behavior.

5 comments

Dear reviewer S2Lq,

Thank you for the detailed assessment and identifying the distinction between a stage-wise framework and the attainment under joint training. We address the theoretical scope, the requested condition checks, and the role of masked corruption below.


Weakness1. Limited generalizability

Our goal is to explain a specific, empirically observed phenomenon, namely reverse reasoning [1] and reverse decoding [2] in planning, rather than to give a theory of all dLLMs. As discussed in [3], the key factor is that different decoding orders lead to subproblems of different difficulty. A setting that captures this needs two properties:

(i) at least two prediction targets, one computationally easy and one hard;

(ii) solving the easy one first makes the hard one easier.

The two-token path-star task is the minimal setting with both. Predicting the endpoint a2a_2 is a copy of the destination in the query (Eq. (1)), while predicting the predecessor a1a_1 requires matching the destination edge and retrieving its source. Since the destination is already in the input, revealing a2a_2 adds no new task information; it only changes how the model can access it. Any RTL advantage is therefore attributable to the learned computation, which is what we analyze (Proposition 3.3, Lemma 4.1, Theorem 4.2).

The experiments are not restricted to the theoretical setting. Appendix E uses random initialization, AdamW, minibatches and a train/test split, all of which break the symmetry and gradient-flow assumptions of the theory, and the same circuit and decoding patterns appear for G(2,4)G(2,4), G(3,4)G(3,4) and G(4,4)G(4,4). Extending the analysis to longer paths and to tasks without a copied destination is left for future work.

[1] How Transformers Learn to Plan via Multi-Token Prediction, COLM 2026

[2] Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks, arXiv 2026

[3] Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions, ICML 2025


Weakness2. Gap between theoretical analysis and joint training

The paper contains results at two levels; the distinction is stated in the Introduction and at the beginning of Section 3.2.

Joint training. Theorem 3.1 is proved for ordinary joint gradient flow, without staging: early in training the endpoint loss falls below the predecessor loss. Figure 3(a) shows the same order empirically.

Stage-wise framework. Motivated by this asymmetry, we analyze endpoint preparation and predecessor learning in two stages. Stage-wise training is a standard device for provable dynamics in multi-layer transformers [4, 6], where the joint flow is not tractable. Here the staging is not imposed a priori; it follows the order established by Theorem 3.1.

Content weights in Eq. 10 (Eq. 9 in the revision). The content matrices are chosen from the symmetry-induced family of Lemma 2.1 and kept fixed in Stage II. This does not prescribe the circuit: the positional weights remain trainable and determine paired-source selection and exposure to the graph-role pools, i.e., which route is formed. To check that the chosen form is the one joint training finds, we have added Appendix E.3, which tracks both content matrices when all blocks are trained jointly from random initialization. Layer 1 develops a negative diagonal and a negative [MASK] column, and Layer 2 a positive diagonal, as in Eq. 9. This structure is visible for both G(2,3)G(2,3) and G(3,4)G(3,4).

All experiments in the paper train content and positional blocks jointly, with no staging or content freeze, and show the chronology of the stage-wise analysis (Figure 3). Proving that joint training reaches the Stage II entry conditions remains open, as stated in the Conclusion.

[4] How Transformers Learn Causal Structure with Gradient Descent, ICML 2024

[5] Focus and Dilution: The Multi-stage Learning Process of Attention, ICML 2026

[6] Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers, NeurIPS 2024

Weakness3. Conditional nature of the decoding guarantees

The results of Sections 3.3 and 4 are certificates: each converts a measurable state of training into a guarantee on the circuit or on decoding. They are not convergence statements, and we have revised the text after Proposition 3.4 and Theorem 3.5 to separate the two roles.

Finite descent and the certificate. Proposition 3.4 establishes how Stage I couples to Stage II: endpoint readiness lower-bounds the predecessor query readiness beb_e (Lemma C.2), which gives L˙1≤−cP<0\dot{\mathcal L}_1 \le -c_P < 0 on a finite interval. It does not claim convergence to arbitrarily low predecessor risk.

Theorem 3.5 is a separate statement: whenever the attained threshold satisfies η≤C1δLε2/2\eta\le C_1\delta_L\varepsilon^2/2, the route masses ρmatch,ρpred,ρtrans\rho_{\mathrm{match}},\rho_{\mathrm{pred}},\rho_{\mathrm{trans}} are at least 1−ε21-\varepsilon^2 and ρend\rho_{\mathrm{end}} is at least 1−ε1-\varepsilon. Under joint training these quantities do saturate: Figure 3(b) tracks the same four route masses, and all approach one as the losses in Figure 3(a) vanish.

Bootstrap condition. Lemma 4.1 holds at any Stage II state without further conditions: revealing the endpoint strictly increases query readiness (b1∣2>b1∣∅b_{1\mid2}>b_{1\mid\varnothing}) and the share of destination attention on d⋆d^\star. Since the total destination exposure can also change, this does not by itself imply a larger predecessor margin. Theorem 4.2 therefore states the remaining requirement directly on a measurable quantity, the revealed margin at first contact, M1∣2(t⋆)>0M_{1\mid2}(t_\star)>0, and converts it into a sequence-level guarantee.

Confidence-based decoding (Proposition 4.3). Proposition 4.3 establishes that dominance of the endpoint's token-mass gaps guarantees that both top-probability and top-margin decoding select the endpoint first. Figures 4(c–d) track reverse-order selection and the corresponding gap-dominance condition during joint training. We provide detailed condition checks in our response to Q2.

Theorem 4.4 assumes L1,L2→0\mathcal L_1,\mathcal L_2\to 0 and characterizes the consequence. Both predictions are observed under joint training: all decoding orders reach accuracy one (Figure 3(c)) and the confidence gaps decay to zero (Figure 4(b)).


Q1. Applicability beyond masked diffusion

Thank you for the question. Our results use two properties, with different scope.

(1) Mask-conditioned training. Section 3 concerns the four conditional predictors, i.e., each answer token with the other one masked or revealed. Models trained on the same conditionals, such as other iterative infilling models, face the same learning problem, so we expect the endpoint-first asymmetry and the reverse-retrieval circuit to carry over.

(2) Monotone unmasking. Section 4 additionally uses that masked positions are identifiable and that decoding commits tokens without remasking, so a correct first prediction produces exactly the revealed state seen in training. The success identities and the bootstrap analysis rest on this.

Uniform-state diffusion [7] differs in both respects: corrupted tokens are ordinary vocabulary tokens and are not marked, and the reverse process may revise a token several times, so there is no fixed reveal order to compare. Our decoding guarantees therefore do not apply as stated. The corresponding question in that setting is which positions the model corrects first and whether early correct tokens help the rest; this requires the uniform corruption states and reverse transitions, and we leave it for future work.

[7] Structured Denoising Diffusion Models in Discrete State-Spaces, NeurIPS 2021

Q2. Verification of theoretical conditions under joint training

We evaluated the conditions of Theorem 4.2 and Proposition 4.3, as stated in the revision, along the joint-training run of Figure 4. Both hold throughout the window in which RTL outperforms LTR.

Theorem 4.2. The condition is M1∣2(t⋆)>0M_{1\mid2}(t_\star)>0, where t⋆t_\star is the first time the both-masked predecessor margin M1∣∅M_{1\mid\varnothing} reaches zero. The revised theorem states this condition directly in terms of the prediction margin. The previous condition involving εint\varepsilon_{\mathrm{int}} implies the revised condition, so the theorem now holds under a weaker assumption.

In the experiment, the mean both-masked margin changes sign between steps 180 and 186, while the mean endpoint-revealed margin is positive at both steps. The predicted separation, M1∣2>0>M1∣∅M_{1\mid2}>0>M_{1\mid\varnothing}, is observed during steps 132–180, coinciding with the interval in which RTL outperforms LTR:

StepE[M1∣2]\mathbb{E}[M_{1\mid2}]E[M1∣∅]\mathbb{E}[M_{1\mid\varnothing}]RTL accuracyLTR accuracy
1320.000615−0.22657.74%0%
1560.00172−0.16268.51%0%
1800.00404−0.012197.29%0%
1860.005080.0174100%100%

The theorem is stated for symmetry-preserving population gradient flow, where all inputs behave identically and decoding accuracy is either 0 or 1. In contrast, the experiments use random initialization, so prediction margins vary across inputs. Accordingly, RTL accuracy represents the fraction of inputs for which the endpoint is correctly predicted and M1∣2>0M_{1\mid2}>0. This fraction increases from 57.74% to 97.29% while LTR accuracy remains at 0%, consistent with the endpoint-reveal bootstrap mechanism.

Proposition 4.3 The revised sufficient condition is g2,2>0g_{2,2}>0 and ∑j=2kg2,j≥∑j=2kg1,j\sum_{j=2}^{k}g_{2,j}\geq\sum_{j=2}^{k}g_{1,j} for all k=2,…,Kk=2,\ldots,K, with strict inequality for at least one kk, where gi,jg_{i,j} denotes the sorted token-mass gaps computed from Eq. (22).

The previous route-based condition 2(1−ρtrans)<min⁡{ρpred,δL(1−ρpred)}2(1-\rho_{\mathrm{trans}})<\min\left\lbrace\rho_{\mathrm{pred}},\delta_L(1-\rho_{\mathrm{pred}})\right\rbrace is not satisfied along the joint-training trajectory, so we revised this condition to characterize reverse-confidence selection directly through the output token-mass distribution.

To evaluate the revised condition, we compute Gmin⁡:=min⁡xg2,2(x)G_{\min}:=\min_{\mathbf{x}}g_{2,2}(\mathbf{x}) and Δkmin⁡:=min⁡x[∑j=2kg2,j(x)−∑j=2kg1,j(x)]\Delta_k^{\min}:=\min_{\mathbf{x}}\left[\sum_{j=2}^{k}g_{2,j}(\mathbf{x})-\sum_{j=2}^{k}g_{1,j}(\mathbf{x})\right] across the evaluated inputs.

StepGmin⁡G_{\min}Δ2min⁡\Delta_2^{\min}Δ3min⁡\Delta_3^{\min}Δ4min⁡\Delta_4^{\min}Δ5min⁡\Delta_5^{\min}
100.0343−4.86e−6−8.89e−6−1.27e−5−1.55e−5
300.0510−3.44e−5−5.12e−5−7.91e−5−9.77e−5
500.0943−1.75e−5−1.91e−6−1.02e−6−1.27e−5
600.1172.81e−51.20e−41.62e−41.49e−4
1000.2150.004570.009320.01310.0159
2000.5890.5561.091.582.00
4000.9570.1270.2500.3390.421

Gmin⁡G_{\min} is positive at all training steps and Δk\Delta_k initially negative and becomes positive around 50~60 steps (exact turning step depends on kk) and then continually positive until it ends. Δk\Delta_k-dynamics is illustrated in Figure 4(d). This also agrees with the positive confidence gaps and the preference for reverse decoding observed in Figures 4(b), 4(c).

Reviewer_AL6LOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the masked diffusion decoding order with learning dynamics by looking into the endpoint preparation stage and the predecessor learning stage. Authors conduct analysis on a two-layer single-head transformer model on star-graph tasks. This paper may possess a sound analysis on the learning dynamics of the masked diffusion model in decoding order. Yet this paper is largely hard to follow and not self-contained. The analysed scenario may be narrow. The experiment setting is restricted.

Critical strengths
  1. Authors studied a timely topic about the decoding strategy of a masked diffusion model.
  2. Authors provide an analysis framework in learning dynamics with a well motivated separation in endpoint preparation and predecessor learning.
  3. Authors provide experiment with an exemplar scenario in star graph task.
Critical weaknesses
  1. The writing of this paper is hard to follow and the paper content may not be self-contained. For example, this paper introduces concepts and notations without sufficient definition and motivation: (1) in Line 060, what is a G(R, L), which is later defined in Line 156 that “The graph G(R,L) consists of R directed arms of L − 1 edges sharing a root u.” But what is an arm in a graph and why do we need to specify u for a root node? Star graph may not be a well-known concept for a general audience. (2) in Line 146, a1 and a2 are introduced without definition, which is later defined in Line 172 as answer positions. (3) in Line 152, what are the four problems? This is previously mentioned in Line 062, “Masking either or both answer positions yields four conditional prediction problems”, which does not illustrate why answer has “both” positions? Is it the answer span position as in a1 and a2? (4) in Line 195, why suddenly do we need to care about “symmetry”? and why reordering/relabeling preserve “the tasks”? What are these tasks? (5) in Line 162, what are the “prompt” in these scenarios? (6) etc. etc. many more writing self-containedness issue.
  2. Only studying star-shape graph may not demonstrate the generalization of the proposed analysis framework. For example, (1) in other shapes of graphs, are the same analysis be still effective? (2) In diffusion language models, are these analysis still valid? (3) The studied problem basically requires the model to recover the answer positions, which is why the authors frame it as a retrieval-like problem. (4) What if the problem requires the model to decode something not in the prompt?
  3. Having additional metrics may help with the validation of the model in different scenarios. Authors rely on mass in different scenario, including token mass, route mass, including in Figure 3, etc. Why do we need to care about the mass? What does it tell?
Questions for authors

Please refer to the weaknesses above.

5 comments

Dear reviewer AL6L,

Thank you for the detailed comments on the presentation. We have revised Sections 2–4 accordingly and respond to each point below.


Weakness 1. Presentation and self-containedness

We have revised the Introduction and Section 2 to make the problem formulation self-contained.

(1) Task and notation. A path-star graph G(R,L)G(R,L) consists of RR directed paths (arms), each containing L−1L-1 edges, that share a common root uu. The root serves as the starting vertex of the planning query. The input consists of a shuffled edge list and a source–destination query (u,vL−1)(u, v_{L-1}). We consider the terminal pair of the target path, represented by two answer positions: a1a_1 contains the predecessor of the destination, and a2a_2 contains the destination itself.

(2,3) Four conditional prediction problems. Masking the two answer positions gives four supervised prediction problems: predicting a1a_1 or a2a_2 when both are masked, and predicting either position when the other is revealed. These correspond to the four sublosses in Eq. (8).

Answer statePrediction targetConditional problem
[MASK, MASK]vL−2v_{L-2} at position a1a_11∣∅1 \mid \varnothing
[MASK, MASK]vL−1v_{L-1} at position a2a_22∣∅2 \mid \varnothing
[MASK, vL−1v_{L-1}]vL−2v_{L-2} at position a1a_11∣21 \mid 2
[vL−2v_{L-2}, MASK]vL−1v_{L-1} at position a2a_22∣12 \mid 1

In the revision, a1a_1 and a2a_2 are defined before they are used in the loss (Section 2.2), and the Introduction no longer refers to G(R,L)G(R,L).

(4) Role of symmetry. Relabeling the vertices changes their names, while preserving the underlying graph and the required answer up to the same relabeling. Reordering the serialized edge pairs changes their presentation, while preserving the graph. These symmetries constrain the learned parameter structure under zero-initialized population gradient flow (Lemma 2.1). These transformations preserve each of the four prediction problems above. They let the analysis focus on content equality, source/destination roles, and whether two positions belong to the same edge, rather than on arbitrary node names or edge-list indices. This property is used in Proposition 3.3 to derive destination-matching bounds and the risk-dependent circuit guarantee in Theorem 3.5. Population symmetry also allows Theorem 4.2 to translate predecessor-margin conditions into sequence-level RTL and LTR accuracy guarantees. Thus, symmetry provides the structural basis for these results without prescribing the learned attention routes.

(5) Meaning of the prompt. We now use 'input sequence' instead of 'prompt' throughout. The input sequence consists of the shuffled graph edge list and the source–destination query. As in Eq. (1), the both-masked input is [π(E)∣u,vL−1∣[MASK],[MASK]][\pi(E)\mid u,v_{L-1}\mid [\mathrm{MASK}],[\mathrm{MASK}]]. The model predicts the masked answer positions using the visible graph and query.

(6) Other presentation issues. We have revised Sections 2-4 to remove undefined notation, delayed definitions and missing motivation.

Weakness 2. Limited generalizability

Our aim is to explain a specific, empirically observed phenomenon, namely reverse reasoning [1] and reverse decoding [2] in planning, rather than to give a theory of all dLLMs. As discussed in [3], different decoding orders can lead to subproblems of different computational difficulty. A setting that captures this asymmetry needs two properties:

(i) at least two prediction targets with different computational requirements;

(ii) solving the easier target first changes the computation available for predicting the harder one.

The two-token path-star task is designed to preserve both properties. Predicting the endpoint a2a_2 requires copying the destination qtq_t, whereas predicting the predecessor a1a_1 requires destination-conditioned edge matching and paired-source retrieval. RTL reveals a2a_2 before predicting a1a_1, while LTR must predict a1a_1 directly. Since the destination is already present in the query, revealing a2a_2 provides no new task information. The resulting advantage therefore depends on how the learned model uses the revealed context, which we characterize in Proposition 3.3, Lemma 4.1, and Theorem 4.2.

(1) Other graph structures. Our analysis uses the path-star graph's symmetries to characterize the learned weights (Lemma 2.1) and establish reverse-route bounds (Theorem 3.5). The underlying decoding-order asymmetry is not specific to path-star graphs. Appendix E.4 shows that binary-tree tasks exhibiting the same asymmetry also reproduce the reverse-decoding phenomenon.

(2) Practical diffusion language models. Our model retains bidirectional masked prediction, flexible decoding orders, and two-layer softmax attention. These components capture how revealing a predicted token changes subsequent predictions. Appendix E further shows that the circuit and decoding patterns emerge under joint training with random initialization, AdamW, and minibatches.

(3) Retrieval-based formulation. The retrieval task is intentional. Predicting the predecessor requires destination-conditioned edge matching followed by paired-source retrieval, rather than simply copying a token. This isolates the computational asymmetry responsible for the decoding-order advantage.

(4) Answers absent from the prompt. The mechanism we study is not copying itself, but how solving an easier prediction first facilitates a harder prediction. Our task captures this through endpoint copying and predecessor retrieval. For non-retrieval tasks, the corresponding question is whether revealing an earlier prediction similarly facilitates subsequent predictions. Investigating this requires extending the analysis to a generation-capable model.


[1] How Transformers Learn to Plan via Multi-Token Prediction, COLM 2026

[2] Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks, arXiv 2026

[3] Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions, ICML 2025

Weakness 3. Interpretation of attention masses

Token mass measures the contribution of each input token to the final representation before the fixed output head. The logit for node vv at position aia_i is α(S(2)S(1)Z)ai,v\alpha(S^{(2)}S^{(1)}Z)_{a_i,v}. Thus, token mass determines the ordering of output logits.

Route mass measures the contribution of a specific two-layer attention path. For the correct predecessor vL−2v_{L-2}, the reverse-route mass satisfies ρpred=Sa1,d⋆(2)Sd⋆,s⋆(1)≤(S(2)S(1)Z)a1,vL−2\rho_{\mathrm{pred}}=S^{(2)}_{a_1,d^\star} S^{(1)}_{d^\star,s^\star}\le (S^{(2)}S^{(1)}Z)_{a_1,v_{L-2}}.

The first factor selects the destination-matched edge, and the second retrieves its paired source. These quantities are directly connected to the theoretical results. Proposition 3.3 lower-bounds destination matching and predecessor retrieval using query readiness and graph-role exposure. Theorem 3.5 further establishes that, conditional on sufficiently low attained risk, ρtrans\rho_{\mathrm{trans}}, ρmatch\rho_{\mathrm{match}}, and ρpred\rho_{\mathrm{pred}} are at least 1−ε21-\varepsilon^2, while ρend\rho_{\mathrm{end}} is at least 1−ε1-\varepsilon.

Figure 3(b) tracks these route masses during joint training, alongside the prediction losses and decoding accuracies in Figures 3(a) and 3(c). Figure 4 further reports prediction margins, confidence gaps and reverse-order choices.

Reviewer_VHEQOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Note: I do not have a strong background in theoretical learning dynamics, reverse-reasoning circuits, or graph-planning theory. Therefore, I have low confidence in evaluating the correctness and novelty of this paper, and my review focuses will primarily be on the paper’s clarity, scope and assumption.

As I understand it, the paper investigates why training may cause a masked diffusion model to prefer reverse decoding. It analyzes a simplified two-layer, attention-only Transformer on a synthetic two-token graph-planning task, proving a local endpoint-first learning asymmetry and deriving conditional results for reverse retrieval and decoding-order preference under a prescribed stage-wise process.

Critical strengths

The paper asks an interesting and important question about how training dynamics create useful decoding-order preferences. I particularly appreciate its distinction among internal reverse retrieval, confidence-based order selection, and sequence-level accuracy. The controlled setting permits an apparently detailed finite-time analysis of a composed two-layer retrieval circuit, and the experimental trajectories provide qualitative evidence matches well with the proposed mechanism. Although I could not verify the mathematical proofs, the theoretical development seems substantial and the conclusion that order sensitivity may be concentrated in an intermediate training domain is conceptually interesting.

Critical weaknesses
  1. The paper is difficult to follow for people not familiar with this domain.
  2. It's motivated by practical dLLMs but studies only a synthetic two-token graph task with a highly simplified Transformer, and the paper does not establish how its conclusions transfer to realistic diffusion language models.
Questions for authors

None

2 comments

Dear reviewer VHEQ,

Thank you for the review. We respond to the two points below. We also added a roadmap of the theoretical results at the beginning of the Appendix.


Weakness 1. Clarity and accessibility of the theoretical analysis

We have revised Sections 2–4 for readability. The task and all token positions are now defined with Figure 1 before the model (Section 2.2). Section 3 describes the two learning stages in words before the analysis and follows each result with its interpretation. Section 4 states the decoding conditions with the same quantities that are plotted in Figures 3 and 4.

The mechanism can be stated with a small example. Suppose the input describes two paths, u→a→du\to a\to d and u→b→eu\to b\to e, and the answer for destination dd is [a,d][a,d]. Predicting dd is a copy of the destination in the query. Predicting aa requires finding the edge that ends at dd and returning its source. Once dd is written in the second answer position, the model has an additional cue for finding that edge.

The results follow this order:

  • Training learns the easy token first (Theorem 3.1).
  • As the loss decreases, the network forms a circuit that matches the destination and retrieves its predecessor (Theorem 3.5).
  • There is a period of training in which the predecessor is correct only after the endpoint is revealed, so right-to-left decoding succeeds while left-to-right decoding fails (Theorem 4.2). Proposition 4.3 gives a condition under which confidence-based decoding chooses this order.
  • When training is complete, every order succeeds and the advantage disappears (Theorem 4.4).

Decoding order therefore matters most at an intermediate stage of training, which is the conclusion highlighted in your review.


Weakness 2. Scope of the simplified setting and relevance to practical dLLMs

The two-token task and attention-only architecture make it possible to trace the composed retrieval path and derive explicit sufficient conditions. The resulting contribution is a controlled mechanism for studying how training changes the value of a decoding order. Our goal is to isolate and explain the underlying mechanism behind the reverse reasoning and reverse decoding phenomena observed in prior works [1, 2].

The existing experiments train all attention blocks jointly and reproduce qualitative circuit and decoding patterns in this controlled setting. The conditions of Theorem 4.2 and Proposition 4.3 are evaluated along the joint-training run (Figure 4 and our response to Reviewer S2Lq, Q2), and the same right-to-left window appears on binary trees (Appendix E.4).

Related theoretical works also use simplified models to understand how specific computations emerge through training. For example, Nichani et al. [3] study how a two-layer Transformer learns causal structures, while Chen et al. [4] investigate how induction heads emerge through gradient-based learning. More closely related to our work, Huang et al. [2] analyze the emergence of reverse reasoning using a simplified Transformer on a graph-planning task. These controlled settings make it possible to analyze learning dynamics and internal computations explicitly. Similarly, our theoretical setup is designed to isolate the computational asymmetry between different decoding orders and explain how training enables the model to exploit an easier prediction to facilitate a more difficult one.


[1] Discrete Diffusion Models Exploit Asymmetry to Solve Lookahead Planning Tasks, arXiv 2026

[2] How Transformers Learn to Plan via Multi-Token Prediction, COLM 2026

[3] How Transformers Learn Causal Structure with Gradient Descent, ICML 2024

[4] Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers, NeurIPS 2024

Dear Reviewer VHEQ,

We have uploaded a revision with all changes highlighted (teal: clarity edits; blue: changes to the theory and new content). On the two points in your review:

(1) Clarity. The task and all token positions are now defined with Figure 1 before the model. Sections 3–4 were rewritten so that each result is followed by its interpretation, and the new Appendix A gives a one-page overview of how the results connect.

(2) Scope. The revision shows the same right-to-left window on binary trees (Appendix E.4) and under random initialization, AdamW, and minibatches (Appendix E.2), and checks the conditions of Theorem 4.2 and Proposition 4.3 along the joint-training run (Figure 4).

These changes address both concerns raised in your review. We therefore ask you to re-evaluate the rating in light of the revision. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.