acceptodds
Under review as a conference paper at ICLR 2027

Low-Confidence Remasking Traps Flexibility: Realizing Arbitrary-Order Potential for Diverse Rollouts in Diffusion LLMs

Abstract

Masked diffusion language models support arbitrary-order generation, suggesting a natural way to produce diverse outputs. However, recent work argues that this flexibility reduces diversity by delaying high-uncertainty tokens that can lead to different generation paths. We trace this diversity loss not to arbitrary-order generation itself, but largely to *low-confidence remasking* (LCR), a widely used decoding rule. At each step, LCR samples a token at every masked position but commits only the sampled token with the highest probability, filtering out the rest. We show that this mechanism can exponentially suppress lower-probability tokens as more positions compete, and observe the same suppression in LLaDA. In contrast, *top-probability position selection* (TPP), which has often been conflated with LCR under the shared label *confidence-based decoding*, avoids this diversity loss. TPP first selects the position whose most likely token has the highest probability, then samples directly from that position's distribution. Replacing LCR with TPP restores diversity and yields Pass@ comparable to left-to-right decoding, suggesting that the reported diversity loss stems largely from LCR's filtering rather than from generating high-confidence positions first. To further exploit order flexibility, we introduce **Entropy-Guided Initialization** (EGI), which samples the first token at the highest-entropy position and then follows TPP. This simple modification further improves rollout diversity and solution coverage beyond left-to-right decoding, with gains extending to downstream policy optimization, highlighting the potential of arbitrary-order generation for diverse rollouts.

Primary area:
generative models
Keywords:
masked diffusion language models, arbitrary-order generation, rollout sampling, confidence-based decoding, low-confidence remasking, group-relative policy optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_FCFGOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper attributes much of arbitrary-order decoding's diversity loss to LCR's proposal filtering rather than confidence-first ordering. Toy analyses and LLaDA diagnostics support this explanation. TPP restores rollout coverage, while entropy-guided initialization further improves coverage and policy optimization.

Critical strengths
  1. The sampler distinction is precise and consequential. Exponential suppression in the toy models is supported by proposal-versus-commitment diagnostics, correcting an overly broad interpretation of the flexibility trap (Sections 3-5).
  2. The explanation leads to a simple intervention: TPP recovers coverage and EGI improves it further. The AR temperature control and gains across three RL objectives strengthen the empirical case (Figure 3, Table 4, Appendix C.5).
Critical weaknesses
  1. Evaluation uses one-token-per-step decoding and two related LLaDA checkpoints. Whether the mechanism and EGI gains extend to practical parallel decoding or another model family remains untested.
  2. RL results select the best of ten test-set checkpoints, without reporting variability across training seeds (Appendix D). This weakens confidence in the smaller downstream gains.
Questions for authors
  1. Do TPP and EGI retain their coverage advantage when multiple tokens are committed per step?
  2. Do the RL gains persist across seeds using final-budget or validation-selected checkpoints?
1 comment

We thank the reviewer for the constructive feedback and helpful suggestions for strengthening the empirical evaluation.

W1. Generalization beyond the LLaDA family

We evaluated Dream 7B Instruct [1] on GSM8K, MATH-500, HumanEval, and MBPP with 32 rollouts per prompt; GSM8K uses a random 500-problem subset. Across all four benchmarks, TPP consistently improves over LCR and recovers rollout coverage close to AR. On the math benchmarks, AR retains a small advantage, which may partly reflect Dream's initialization from the autoregressive Qwen2.5-7B model [1] and the sequential nature of mathematical reasoning. Despite this residual gap, TPP remains close to AR, reproducing the key LCR-TPP distinction beyond the LLaDA family. EGI also remains competitive with AR. Full results are provided in App. D.1 (Fig. 11) of the revised manuscript.

Q1. Coverage under parallel decoding

We evaluated parallel decoding using Fast-dLLM [2] variants on LLaDA-8B-Instruct with a common confidence threshold of 0.90.9. We denote Fast-dLLM with LCR, TPP, and EGI as FD-LCR, FD-TPP, and FD-EGI, respectively. FD-LCR applies LCR's proposal filtering, FD-TPP selects positions using top-token probabilities before sampling, and FD-EGI applies EGI at the first step followed by FD-TPP.

FD-TPP improves coverage over FD-LCR, and FD-EGI improves it further with only a modest increase in NFE. Thus, the same diversity behavior underlying LCR, TPP, and EGI persists under parallel decoding. Pass@K and Potential@K are percentages, and NFE is the mean number of model evaluations. Full results are provided in Sec. 6.2 (Fig. 4) and App. D.2 (Fig. 12) of the revised manuscript.

BenchmarkMethodPass@8Pass@32Potential@8Potential@32NFE
HumanEvalFD-LCR53.0757.9323.9531.6876.7
HumanEvalFD-TPP54.2963.4126.5940.5979.1
HumanEvalFD-EGI59.7270.7336.9453.4782.1
GSM8K (500)FD-LCR90.2594.0061.5376.1971.3
GSM8K (500)FD-TPP93.6497.4075.4289.6873.2
GSM8K (500)FD-EGI94.9797.6080.8490.4873.7

W2 & Q2. RL robustness across seeds and checkpoints

We additionally ran JustGRPO with three training seeds per sampler and evaluated the final-budget checkpoints, while also reporting accuracy averaged over all ten evaluation checkpoints. Both summaries report mean ±\pm sample standard deviation across training seeds. EGI achieves higher mean accuracy than AR on all four benchmarks under both summaries, without relying on best-checkpoint selection. All values are percentages. Full results are provided in App. E.1 (Fig. 15 and Tab. 9) of the revised manuscript.

BenchmarkFinal budget: ARFinal budget: EGI10-checkpoint mean: AR10-checkpoint mean: EGI
GSM8K81.00±0.4481.00 \pm 0.4483.27±0.4683.27 \pm 0.4678.41±0.2078.41 \pm 0.2079.71±0.1179.71 \pm 0.11
MATH-50036.73±0.4236.73 \pm 0.4237.73±0.2337.73 \pm 0.2335.61±0.0935.61 \pm 0.0936.21±0.0536.21 \pm 0.05
HumanEval38.01±0.3538.01 \pm 0.3538.62±0.9338.62 \pm 0.9336.97±0.0736.97 \pm 0.0737.38±0.1837.38 \pm 0.18
MBPP43.80±0.2043.80 \pm 0.2044.60±0.2044.60 \pm 0.2042.55±0.1842.55 \pm 0.1843.16±0.1743.16 \pm 0.17

References

[1] Ye et al., Dream 7B: Diffusion Large Language Models, arXiv, 2025.

[2] Wu et al., Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding, ICLR, 2026.

Reviewer_PduPOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that low-confidence remasking suppresses rollout diversity because it samples token proposals before choosing which position to reveal. Simple models show how this selection step can discard low-probability samples. Experiments with LLaDA support the mechanism. Choosing the position before sampling largely restores pass@k, and an entropy-guided first step (EGI) improves it further. The paper also tests EGI for reinforcement-learning rollouts.

Critical strengths
  1. The strongest contribution is identifying a confound in prior comparisons: LCR and TPP are different samplers. LCR samples candidates at all masked positions and then selects among them; TPP chooses a position before sampling its token. At nonzero temperature, LCR filters out lower-probability proposals. This helps explain behavior that prior comparisons might attribute to generation order alone.

  2. The explanation is clean and convincing. In the toy setting, LCR commits a non-top token with probability (1-c_T)^m, versus 1-c_T for TPP. Here, c_T is the probability of sampling the top token, and m is the number of masked positions. The token-level measurements support this mechanism. The paper is focused and easy to follow, from the initial discrepancy through the analysis and experiments to EGI.

Critical weaknesses
  1. The paper could connect token-level suppression to diversity among solutions more directly. LCR and TPP differ greatly in non-top commitment rates, but their pass@k gaps are more modest on some benchmarks in Figure 3, especially the math tasks. The larger coding gains are encouraging, but it is unclear when the mechanism matters most. Showing how commitment rates relate to pass@k or Potential@k across the tested settings would help establish when this matters in practice.

  2. EGI uses an entropy-guided first commitment and then follows TPP. The comparison with TPP shows the added benefit of EGI in that setting. It does not show whether the same first step could also help LCR. The missing EGI+LCR ablation matters. A more varied first token might partly overcome later suppression, or the decoder might need to keep using TPP. The current results do not distinguish these possibilities.

Questions for authors
  1. Across the tested temperatures, block sizes, and tasks, how closely do changes in non-top commitment rates track gains in pass@k or Potential@k? Can the existing results explain why the pass@k gaps vary across tasks?

  2. Can you compare LCR, TPP, EGI+LCR, and EGI+TPP? Please match first-step temperatures across the relevant controls to separate the effects of position choice and the sampler used afterward. This would show whether an entropy-guided first step can reduce the diversity loss from later LCR steps.

3 comments

We thank the reviewer for the constructive feedback and helpful suggestions for strengthening the connection between our analysis and empirical results.

W1 & Q1. Token-level suppression and solution coverage across tasks

LCR rarely commits non-top tokens, but it can still obtain diversity through variation in generation order, as already discussed in the original submission (Lines 340-343). Therefore, the large difference in non-top commitment rates between LCR and TPP does not need to translate proportionally into Pass@k or Potential@k gaps, and the amount of order-induced diversity can vary across tasks.

To characterize this intermediate rollout diversity, we additionally analyze the last matching step, LpermL_{\mathrm{perm}}, between rollout pairs with generation length 256. LpermL_{\mathrm{perm}} is the last step at which two rollouts contain the same set of committed position-token pairs regardless of commitment order; lower values indicate earlier persistent separation.

BenchmarkLCRTPP
HumanEval81.041.9
GSM8K88.832.6

TPP exhibits substantially earlier persistent separation than LCR on both tasks. Moreover, the LCR-TPP difference in LpermL_{\mathrm{perm}} is larger on GSM8K than on HumanEval, while the corresponding Pass@k gap is smaller on GSM8K. This contrast suggests that the extent to which rollout-level divergence translates into additional solution coverage is task-dependent. Full results are provided in App. D.5 (Tab. 8) of the revised manuscript.

W2 & Q2. Requested ablation: EGI followed by LCR

EGI first selects a position and then samples its token, whereas LCR applies proposal filtering by selecting among sampled token proposals. Thus, EGI is not directly defined on top of LCR. To construct the requested hybrid, we use EGI only for the first step and then apply standard LCR for all remaining steps.

We evaluated LCR, TPP, EGI+LCR, and EGI+TPP (= original EGI) on LLaDA-8B-Instruct with 32 rollouts per prompt. All four variants use T=0.9T=0.9 at the first step and T=0.6T=0.6 thereafter; GSM8K uses a random 500-problem subset. EGI improves coverage even when followed by LCR, while EGI+TPP achieves the strongest results. Full results are provided in App. D.3 (Fig. 13) of the revised manuscript.

BenchmarkSamplerPass@8Pass@32Potential@8Potential@32
HumanEvalLCR54.0459.7624.0633.33
HumanEvalTPP60.3768.9034.7848.48
HumanEvalEGI+LCR60.4468.9035.9448.48
HumanEvalEGI+TPP62.1575.6139.4360.61
GSM8K (500)LCR90.0393.6059.6673.98
GSM8K (500)TPP94.0297.8076.4091.06
GSM8K (500)EGI+LCR94.4797.6078.1190.24
GSM8K (500)EGI+TPP95.4398.4082.2593.50

Thank you for the detailed response and the additional experiments. I appreciate the authors' effort in addressing my concerns.The EGI+LCR experiment with matched temperatures directly answers one of my main questions. I still think the connection between token-level diversity and solution coverage could be studied further, but the additional results have addressed most of my concerns. Overall, I think this is a clear and useful paper. I remain positive about the work and will keep my original rating.

AuthorsOpenReview ↗

Thanks for the constructive feedback

Thank you for the constructive feedback and for your positive assessment. We agree that the relationship between token-level diversity and solution coverage deserves further discussion, and we will expand this point in the final version.

Reviewer_7RD9OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
3
Read review
Paper summary

The paper shows that the loss in diversity of decoded outputs of MDMs, which is typically associated with confidence-based decoding, conflates two types of confidence-based decoding methods. Specifically, if the token is sampled from the original per-position distribution and if the position selection algorithm does not consider the sampled token or its probability in its decision (TPP), then the diversity collapse does not occur. It then goes on to propose EGI, which selects the highest-entropy position at the first decoding step and then follows TPP. This small change increases the pass@k on GSM8K and HumanEval compared to AR decoding. Finally, it also shows that this increase in pass@k translates into improved performance of GRPO.

Critical strengths
  1. Clear writing: The paper is well written, in that it presents a simple claim and supports it with clear empirical evaluation.
Critical weaknesses

Novelty: The most significant weakness of the paper is its incremental novelty. Zhang et. al. already discuss the loss in entropy as a direct consequence of LCR-type sampling. Their Figure 3 shows that remasking using entropy, instead of the selected token's probability, recovers the entropy of left-to-right decoding. The entropy-based remasking procedure of Zhang et. al. is somewhat equivalent to the TPP sampling discussed here, with the only difference that, instead of using top-probability as the position selection criterion, they use lower entropy.

The bridge from observation to understanding is missing. More specifically, why does EGI, which only changes the first step of decoding, achieve a higher pass@k than TPP and AR? Moreover, would unmasking any random position at the first step instead of the highest-entropy position also provide the same effect? What about doing such random sampling periodically during the decoding process? These questions might be easier to answer if the reason for EGI's higher diversity is better understood.

Significance EGI shows significant gains over AR only for HumanEval and to some extent on GSM8K.

[1] Zeyang Zhang, Chengwei Liang, Xingyan Chen, Meiqi Gu, Minrui Luo, Jingzhao Zhang, and Tianxing He. Differences in text generated by diffusion and autoregressive language models. In COLM, 2026.

Questions for authors
  1. Line 453: How do you pick the temperatures of T=0.9 for math and T=0.6 for code?
  2. Could you please include Math500 and MBPP in table 2 as well? Also, please include TPP in Table 4.
  3. I'd be happy to update my assessment if my concerns regarding the understanding of EGI are addressed adequately.

Minor clarifications and Typos

  1. Line 233: "Under TPP, all positions have the same top-token probability...". This does not match the description of TPP in Section 3.
  2. How are std bands in Figure 4 obtained? Are they due to different policy optimization methods, or to multiple runs with the same method using different seeds?
4 comments
AuthorsOpenReview ↗

Rebuttal: Part 1 of 2

We thank the reviewer for the constructive feedback and detailed questions, which helped us clarify the paper's novelty and the role of EGI.

W1. Novelty relative to Zhang et al. [1]

Our focus is to identify LCR's proposal filtering, rather than arbitrary-order generation or confidence-prioritized ordering itself, as the main source of the diversity loss reported by Ni et al. [2]. This requires distinguishing LCR from TPP, which have often been conflated under the shared label of confidence-based decoding. While [1] identifies entropy reduction caused by sampled-token-dependent selection in LCR, this issue is one component of its broader comparison between AR and diffusion LMs.

Our work goes substantially further in characterizing this mechanism. We quantify the severity of LCR's suppression by showing exponential suppression of non-top commitments. We also directly trace the proposal-to-commitment filtering mechanism in LLaDA, identify the broader LCR-TPP conflation in the literature, and show that this distinction changes the interpretation of the flexibility trap. These differences were already discussed in Sec. 4 and App. A.1 of the submitted paper, and we summarize them more explicitly below.

1. Quantifying LCR's suppression.

[1] establishes entropy reduction under independent positions, while our analysis identifies the underlying exponential suppression mechanism and quantifies the severity of the resulting entropy collapse. This sharper analysis was also independently highlighted as a key strength by Reviewers PduP and FCFG. With mm competing positions and tempered top-token probability cTc_T, LCR exponentiates the probability of committing a non-top token from 1−cT1-c_T under TPP to (1−cT)m(1-c_T)^m. Consequently, for a length-LL sequence, LCR's expected non-top fraction decreases as O(1/L)O(1/L) and its entropy per token collapses from O(1)O(1) under TPP to O(log⁡L/L)O(\log L/L), equivalently reducing total entropy growth from O(L)O(L) to O(log⁡L)O(\log L) (Sec. 4.1; Apps. B.1 and B.3, Propositions 2 and 4).

2. Tracing the mechanism in LLaDA.

We directly observe in a real dLLM that temperature-induced exploration is largely removed between proposal and commitment. On GSM8K with L=B=128L=B=128 and T=1.2T=1.2, only 0.7% of LCR commitments are non-top tokens, compared with 37.6% under TPP (Fig. 1). Fig. 2 shows the underlying mechanism. Increasing temperature produces substantially lower-probability proposals, but the tokens that survive LCR and are actually committed remain concentrated at high probability tokens.

3. Clarifying the LCR-TPP conflation.

Another distinction from [1] is that we extend the LCR-TPP distinction to a broader conflation problem in the literature. Although the two coincide under greedy decoding, at T>0T>0, LCR selects among already sampled proposals whereas TPP selects a position before sampling its token, resulting in substantially different sampling behavior. Conflating these procedures can therefore lead to incorrect interpretations of positive-temperature rollout results. We documented such cases in App. A.2 of the submitted paper, including TPP-style descriptions or analyses paired with LCR-style positive-temperature implementations.

4. Reassessing the flexibility trap.

Ni et al. [2] attribute the flexibility trap to confidence-prioritized arbitrary-order decoding, but TPP also prioritizes high-confidence positions while recovering Pass@k comparable to AR (Fig. 3). Together with our LCR-TPP diagnostics, this shows that the observed AR-LCR coverage gap is largely caused by LCR's filtering rather than by confidence-prioritized ordering or arbitrary-order generation itself.

EGI then provides a simple constructive example that order flexibility can serve as an additional source of exploration. It is not intended as an optimal diversity sampler, but rather to demonstrate this potential. Changing only the first decoding position already improves rollout coverage.

References

[1] Zhang et al., Differences in Text Generated by Diffusion and Autoregressive Language Models, COLM, 2026.

[2] Ni et al., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, ICML, 2026.

[3] Hu et al., Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?, ICLR, 2026.

AuthorsOpenReview ↗

Rebuttal: Part 2 of 2

W2, W3 & Q3. Motivation and significance of EGI

We appreciate the reviewer’s suggestions for better understanding EGI. As discussed in our response to W1, we would like to clarify that our goal is to use EGI as a minimal intervention illustrating how order flexibility can provide additional opportunities for exploration.

Although a detailed study of EGI is not our main focus, we further investigated its behavior following the reviewer’s suggestions. We tested a single entropy-guided or random-position intervention on HumanEval, placed at initialization, one-third, or two-thirds through decoding (Sec. 6.3, Fig. 5; App. D.4). Random-position initialization also improves coverage over AR, while delaying either intervention largely removes these gains. These results are consistent with our motivation: an early commitment can influence more subsequent predictions and position choices, allowing additional branching. They also show that the benefit is not specific to the highest-entropy rule.

We agree that periodic interventions could offer further improvements, although their effect on rollout quality would also need to be evaluated. Given our focus on demonstrating the potential of order flexibility through a minimal intervention, we believe that optimizing when and how often to intervene is beyond the scope of this work. We view the reviewer’s suggestion as a promising direction for further study.

We further find that EGI produces more mixed-success rollout groups and fewer uniformly correct or incorrect groups on both GSM8K and HumanEval, consistent with early branching opening useful alternative trajectories (Sec. 6.3, Fig. 5; App. D.5). Although the gains vary across tasks, these results illustrate the potential of even a simple use of order flexibility.

Q1. Temperature choices

For the Pass@k experiments, we selected T=0.9T=0.9 for EGI's first step based on HumanEval and then fixed this setting across all benchmarks, with T=0.6T=0.6 for all subsequent steps rather than tuning temperatures separately for each benchmark. We additionally report a first-step temperature sweep on HumanEval, where EGI already outperforms AR at T=0.6T=0.6 and T=0.9T=0.9 provides further gains at larger rollout budgets (App. C.5, Fig. 9).

For RL, we use T=0.9T=0.9 for math but T=0.6T=0.6 for code. The code setting trains on a subset of AceCoder-87K and evaluates on HumanEval and MBPP, introducing a cross-dataset shift. Prior work shows that transfer of RL gains can depend on task similarity [3]. We therefore conservatively keep the first-step temperature at the base T=0.6T=0.6 for code.

Q2. The requested benchmarks and TPP results

We have added MATH-500 and MBPP results in App. C.4 (Tab. 7), as requested. These additional results are consistent with our original findings, with EGI improving rollout diversity and coverage over AR on both benchmarks.

We additionally evaluated TPP under JustGRPO on GSM8K (App. E.2, Tab. 10). TPP performs comparably to AR under both the final-budget and training-trajectory summaries:

MetricAR (3-seed mean ±\pm SD)TPP (single run)
Final-budget accuracy81.00±0.4481.00 \pm 0.4481.5881.58
10-checkpoint average78.41±0.2078.41 \pm 0.2078.9478.94

Minor 1. The statement on equal top-token probabilities

The statement refers only to the i.i.d. toy setup in Sec. 4.1, not to TPP in general. We have revised it to:

“In this i.i.d. toy setup, all positions have the same top-token probability, so TPP may select any masked position before sampling and committing a token directly from p(T)p^{(T)}.”

Minor 2. The bands in Fig. 6

The Fig. 6 bands show variation in a single training run, with shading indicating ±1\pm 1 standard deviation of logged values within ten equal-width FLOPs bins. We have clarified this in the revised caption.

Separately, we ran JustGRPO with three seeds per sampler and evaluated the final-budget checkpoints. EGI achieves higher mean final-budget accuracy than AR on all four benchmarks. Full multi-seed results are provided in App. E.1 (Fig. 15 and Tab. 9) of the revised manuscript.

BenchmarkAREGI
GSM8K81.00±0.4481.00 \pm 0.4483.27±0.4683.27 \pm 0.46
MATH-50036.73±0.4236.73 \pm 0.4237.73±0.2337.73 \pm 0.23
HumanEval38.01±0.3538.01 \pm 0.3538.62±0.9338.62 \pm 0.93
MBPP43.80±0.2043.80 \pm 0.2044.60±0.2044.60 \pm 0.20

References

[1] Zhang et al., Differences in Text Generated by Diffusion and Autoregressive Language Models, COLM, 2026.

[2] Ni et al., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, ICML, 2026.

[3] Hu et al., Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?, ICLR, 2026.

Reviewer_7RD9OpenReview ↗

Thanks for the rebuttal

Thanks for the thorough rebuttal. I still believe that a better understanding and exploration of ways to exploit EGI and its variants would make the paper much stronger and its contribution more novel. The added discussion in the appendix D4 is interesting and I encourage the authors to give it more space in the main paper.

AuthorsOpenReview ↗

Thanks for the helpful suggestion

Thank you again for the helpful suggestion, which helped strengthen the presentation of the paper, and for your positive reassessment. Following your recommendation, we have moved the key EGI analyses from the appendix into a new main-text Sec. 6.3.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to Reviewers

We thank the reviewers for their constructive feedback. Our central finding is that LCR's proposal filtering is a major source of the diversity loss attributed to arbitrary-order generation. TPP recovers rollout coverage comparable to left-to-right decoding, and EGI illustrates the further exploration potential of order flexibility. The revised manuscript (with all changes highlighted in blue) and responses address the following points:

  • Novelty and the role of EGI (7RD9). We clarify the distinction from prior work and EGI's role as an illustrative use of order flexibility. New random-position and intervention-timing controls support the importance of early intervention (Sec. 6.3, Fig. 5; App. D.4).

  • Suppression, initialization, and coverage (PduP, 7RD9). We add matched-temperature comparisons of LCR, TPP, EGI+LCR, and EGI+TPP, together with trajectory-separation and rollout-success diagnostics (Sec. 6.3, Fig. 5; Apps. D.3, D.5).

  • Broader evaluation (FCFG). We add Dream 7B experiments and parallel-decoding results using Fast-dLLM variants (Sec. 6.2, Fig. 4; Apps. D.1-D.2).

  • RL reliability (FCFG, 7RD9). We add three-seed JustGRPO results at a fixed final compute budget and averaged over ten checkpoints (App. E.1), and report a single-run TPP control on GSM8K (App. E.2). We also clarify that Fig. 6's bands show variation in a single training run.

  • Additional results and settings (7RD9). We add the MATH-500 and MBPP counterparts of Tab. 2 (App. C.4, Tab. 7), explain the temperature choices (App. C.5), and restrict the equal-top-probability statement to the i.i.d. toy setup.

AuthorsOpenReview ↗

Update on the Revised Manuscript

Thank you again to all reviewers for the constructive discussion. We have uploaded a further revised manuscript that incorporates several analyses from the rebuttal into the main paper. In particular, the parallel-decoding results are now included in Sec. 6.2, and the EGI analyses are consolidated in a new Sec. 6.3. As these changes alter the manuscript numbering, we will update only the section, figure, and table references in our earlier general response and rebuttals accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.