acceptodds
Under review as a conference paper at ICLR 2027

Fork-dLLM: Avoiding the Flexibility Trap in Diffusion Language Models

Abstract

Masked diffusion language models (dLLMs) have shown strong potential for faster inference through parallel token generation when combined with confidence-based samplers. However, recent work has shown that such methods can defer unmasking high-entropy *fork* positions at which multiple plausible continuations exist. This results in reduced generation diversity, as shown by worse pass@ scaling, and limits gains obtainable from RL post-training. To avoid this *flexibility trap*, prior work advocated for autoregressive (AR) sampling. Here, we show that discarding confidence-based sampling is unnecessary and, once inference cost is taken into account, wasteful. We first propose *Fork-dLLM*, a simple hybrid sampler that uses AR-style ordering only at uncertain fallback steps while retaining parallel generation otherwise. We then extend the same principle to post-training with *ForkGRPO*, which uses Fork-dLLM rollouts and applies the GRPO objective only at fallback steps, preserving exact policy-likelihood ratios while substantially reducing rollout and optimization cost. In our experiments, Fork-dLLM matches the strong pass@ scaling of AR sampling while being – more efficient, and ForkGRPO achieves downstream performance comparable to or better than AR-based GRPO baselines at a substantially lower training cost.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Models, RL, Flexibility Trap

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_BpYQOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to mitigate the diversity loss associated with confidence-based decoding in masked diffusion language models while retaining parallel generation efficiency. The proposed Fork-dLLM sampler preserves parallel commitment of tokens above a confidence threshold but replaces Fast-dLLM’s highest-confidence fallback with the leftmost masked position whenever no candidate exceeds the threshold. The paper further introduces ForkGRPO, which generates rollouts with Fork-dLLM and applies the policy optimization objective only at fallback steps, reducing both rollout and optimization costs. Sampling experiments on LLaDA and Dream examine pass@k and NFE efficiency, while LoRA post-training experiments on LLaDA-8B-Instruct demonstrate improved compute efficiency relative to JustGRPO-Fast.

Critical strengths

1. The method is simple, inexpensive, and addresses a clear quality–efficiency trade-off. Fork-dLLM changes only the fallback rule of an existing sampler, without additional model evaluations or new sampling hyperparameters. Evaluating both pass@k and pass@NFE is valuable because it accounts for the computational cost of obtaining multiple samples. The results demonstrate that, in the evaluated settings, retaining confidence-based parallel generation does not necessarily preclude solution coverage close to that of autoregressive sampling.

2. The post-training experiments include useful controlled comparisons. The authors train ForkGRPO and JustGRPO-Fast with the same LoRA configuration and compute budget, and supplement this comparison with step-matched results. These experiments support the practical value of reducing training costs. The different stability behavior observed between FastGRPO and ForkGRPO under matched sampling hyperparameters is also potentially informative and motivates further investigation into the relationship between position selection and RL training dynamics.

Critical weaknesses

1. The methodological advance is limited, and its distinct contribution is not sufficiently isolated.

ForkGRPO largely retains JustGRPO’s policy optimization framework, replacing token-by-token autoregressive rollouts with parallel rollouts incorporating a leftmost-position fallback and restricting the objective to fallback steps. Since JustGRPO-Fast already performs sparse optimization at high-entropy positions, the principal changes concern rollout generation and position selection. The matched-compute improvements are useful, but they jointly reflect faster parallel rollouts and fewer optimized positions, without sufficiently isolating the contribution of fallback-based selection itself. In Table 1, the methods achieve similar accuracy at matched training steps, while ForkGRPO requires fewer backward passes. This supports computational efficiency, but does not by itself establish the stronger interpretation that fallback positions are more informative. Comparing fallback-based and entropy-based selection under the same rollout policy and with matched numbers of optimized tokens would more directly test this claim. The observation that FastGRPO collapses while ForkGRPO remains stable is potentially valuable, but its mechanism remains unresolved. The authors hypothesize that leftmost-position selection reduces dependence on RL-induced confidence changes. However, whether a fallback occurs still depends on model confidence, so the modification does not fully remove this coupling. The current evidence supports a stability difference in the tested setting more directly than a general mechanistic explanation. My concern is not simplicity itself, but whether this limited methodological change has been supported by sufficiently discriminating experiments and analysis to establish a substantial new insight.

2. The RL evaluation is narrow and does not establish applicability to deployment-oriented models or longer reasoning trajectories.

All RL experiments use LLaDA-8B-Instruct with LoRA and a generation length of 256 tokens. Although the sampling experiments also include Dream-7B, these do not establish the transferability of ForkGRPO’s training benefits. In particular, the method is not evaluated on more deployment-oriented block-diffusion models, such as the LLaDA 2.x or SDAR families. This is not simply a request for additional model coverage: the method relies on fallback events as a proxy for important decision points, while differences in attention structure, training objectives, and decoding mechanisms could alter confidence distributions, fallback frequency, and the influence of selected positions on final outputs. Consequently, the observed sparse-optimization benefits and training stability may not transfer directly. Likewise, the 256-token experiments do not establish whether optimizing only fallback positions remains effective for credit assignment, stability, and computational savings over longer reasoning trajectories. The reported pass@k scaling concerns solution coverage as the number of samples increases; it does not establish scalability with reasoning length, model size, or training budget. The authors acknowledge several of these limitations, but they still constrain the significance of the contribution. Without evidence from at least one longer-generation or deployment-oriented setting, the conclusions should remain explicitly limited to the demonstrated short-generation, single-backbone RL setup.

Questions for authors
  1. Independent contribution of position selection: With the rollout policy fixed and the number of optimized tokens matched, does fallback-based selection still outperform entropy-based selection? What evidence distinguishes the claim that fallback positions are more informative from the alternative explanation that optimizing fewer positions is sufficient to obtain similar performance at lower cost?

  2. Training stability: Is the contrast between FastGRPO collapse and ForkGRPO stability reproducible across multiple random seeds? Given that fallback triggering still depends on confidence, what evidence supports the proposed decoupling explanation beyond the observed stability difference in the reported configurations?

  3. Scope of applicability: Do the authors have evidence from longer generations or block-diffusion models showing that fallback events continue to identify effective optimization positions while preserving stability and computational advantages? If not, which conclusions should be restricted to the current LLaDA, LoRA, and 256-token setting?

4 comments

We thank you for reviewing our work. Please find our responses below.

Independent contribution of position selection: With the rollout policy fixed and the number of optimized tokens matched, does fallback-based selection still outperform entropy-based selection? What evidence distinguishes the claim that fallback positions are more informative from the alternative explanation that optimizing fewer positions is sufficient to obtain similar performance at lower cost?

Thank you for this question. Reviewer 84P5 raised a closely related point, and we ran a new experiment to address it; please see our response there for further details.

We agree that our claim in Section 4.2 was not fully supported, since Table 1 varies both the rollout sampler and the optimized positions. To isolate the selection rule, we trained a ForkGRPO variant with the same Fork-dLLM rollouts that replaces each rollout's fallback tokens with an equal number of its highest-entropy committed tokens. Both variants perform comparably for the first 200 steps, after which the entropy-selection variant collapses (70.1% vs. 86.4% for ForkGRPO at step 250, with the base model at 80.7%).

Based on these results, we cannot conclude that fallback positions are more informative per update than entropy-selected ones, and we have removed this claim from Section 4.2 (in red). However, our results suggest that optimizing fallback positions leads to more stable training. We have added this comparison to Appendix F.

Training stability: Is the contrast between FastGRPO collapse and ForkGRPO stability reproducible across multiple random seeds? Given that fallback triggering still depends on confidence, what evidence supports the proposed decoupling explanation beyond the observed stability difference in the reported configurations?

During the rebuttal, we additionally run FastGRPO with a different seed (β\beta=0). Similar to the two previous runs, the training begins collapsing around step 355, reaching a reward of 0.24 by step 371. We thank you for this suggestion, which we believe strengthens our paper.

Regarding the mechanism, we agree that decoupling is a hypothesis rather than an established explanation. As you note, whether a fallback occurs still depends on model confidence; Fork-dLLM only makes the choice of position independent of it. Our new experiment adds a further observation: with the same Fork-dLLM rollouts, optimizing entropy-selected positions instead of fallback positions also leads to collapse (Appendix F). This suggests that the choice of optimized positions, and not only the rollout policy, affects stability. We leave a full analysis of the mechanism to future work.

Scope of applicability: Do the authors have evidence from longer generations or block-diffusion models showing that fallback events continue to identify effective optimization positions while preserving stability and computational advantages? If not, which conclusions should be restricted to the current LLaDA, LoRA, and 256-token setting?

We agree that our RL conclusions are established in a single setting (see Limitations, Section 5): LLaDA-8B-Instruct with LoRA and a generation length of 256 tokens. This follows prior work on RL for dLLMs, which trains a single model (LLaDA-8B-Instruct) with a single fine-tuning method and a generation length of 256 tokens, e.g. JustGRPO (Ni et al., 2026), d1 (Zhao et al., 2025), SPG (Wang et al., 2026), and TCT (Olausson et al., 2026). Our sampling results are broader, covering two models (LLaDA and Dream) and four benchmarks.

  • Ni et al., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, ICML 2026
  • Zhao et al., d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning, NeurIPS 2025
  • Wang et al., SPG: Sandwiched Policy Gradient for Masked Diffusion Language Models, ICLR 2026
  • Olausson et al., A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models, COLM 2026

We thank the authors for their detailed response and the new experiments added during the rebuttal.

What the response addressed. The entropy-selection control in Appendix F is precisely the comparison we asked for: with the rollout policy fixed and the number of optimized tokens matched, the fallback-selected variant remains stable while the entropy-selected variant collapses. We appreciate the candid withdrawal of the "more informative per update" claim and the revised, better-supported framing in terms of training stability. The additional FastGRPO seed partially addresses our reproducibility concern, and we agree that a full mechanistic analysis can be left to future work.

Why our assessment remains unchanged. Our primary concern (Weakness 2 / Question 3) is not resolved. The authors note that prior RL work on dLLMs (JustGRPO, d1, SPG, TCT) uses the same single-model, 256-token setup. We acknowledge this precedent, but we do not believe it answers the concern. Our question is not whether the setting is consistent with prior work, but whether the paper's conclusions are supported beyond a single narrow configuration. That a limitation is shared by the literature does not diminish its impact on the significance of this contribution; if anything, it makes evidence that escapes this setting more valuable, not less.

Concretely

  • ForkGRPO's benefits rely on fallback events serving as a proxy for important decision points. This is a property of the confidence distribution of one model under one sampler. Differences in attention structure, training objectives, and decoding mechanisms in deployment-oriented block-diffusion models (e.g., the LLaDA 2.x or SDAR families) could alter fallback frequency, the informativeness of fallback positions, and therefore both the efficiency and the stability conclusions. No evidence is provided for any such model, even at the sampling level.
  • A generation length of 256 tokens does not establish that optimizing only fallback positions remains effective for credit assignment over longer reasoning trajectories — precisely the regime where sparse optimization is most likely to fail. The reported pass@k scaling concerns the number of samples, not reasoning length, model scale, or training budget.
  • The stability contrast, while interesting, is demonstrated in a single configuration. The revised claim that confidence-independent position selection stabilizes sparse RL is a general statement about RL for dLLMs, yet its current support is confined to LLaDA-8B-Instruct with LoRA and 256-token generations.

We recognize that the sampling contribution is more broadly supported (two backbones, four benchmarks) and that the method is simple and inexpensive. However, the RL contribution is a central part of the paper, and as currently established it remains confined to a setting we consider too narrow to support the broader claims. The authors have appropriately restricted their conclusions in the Limitations section, but this restriction also limits the significance of the contribution.

We therefore maintain our rating of 2 (weak rejection). We would reconsider if the authors provided evidence from at least one longer-generation or deployment-oriented (block-diffusion) setting showing that fallback-based selection retains its benefits; even inference-level fallback statistics on such a model would be a meaningful step in that direction.

Thanks for your reply.

We would reconsider if the authors provided evidence from at least one longer-generation or deployment-oriented (block-diffusion) setting showing that fallback-based selection retains its benefits; even inference-level fallback statistics on such a model would be a meaningful step in that direction.

While our computational resources and rebuttal time constraints do not allow us to conduct full RL experiments on LLaDA 2.0-mini, we have performed additional Fork-dLLM sampling experiments on LLaDA 2.0-mini (see Appendix H of the revised manuscript). We observe that the sampling benefits of Fork-dLLM transfer from LLaDA/Dream to LLaDA 2.0-mini. In particular, on the MBPP dataset Fork-dLLM is the only confidence-based sampler to catch-up with AR sampling in terms of pass@k, while outperforming it in terms of pass@NFE (it requires ~3× fewer NFEs to achieve ~90% solution coverage). We hope this addresses your concerns about fallback-based selection retaining its benefits on more modern block-diffusion models such as LLaDA 2.0. We will make sure to include results on additional datasets for LLaDA 2.0-mini in the camera-ready version.

Finally, we would like to reiterate that the two closest works to ours, The Flexibility Trap [1], which received an Outstanding Paper Award at ICML 2026, and TCT [2], both conduct their RL experiments exclusively on LLaDA. Hence, while we agree that a broader evaluation is always desirable, we would find it somewhat unfair to treat our experimental scope as grounds for rejection, given that our RL evaluation follows the established setup of closely related (and well-received) prior work.

[1] Ni et al., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, ICML 2026

[2] Olausson et al., A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models, COLM 2026

Appendix H meets the reconsideration condition stated in our previous comment — inference-level evidence that Fork-dLLM's benefits transfer to a block-diffusion model (LLaDA 2.0-mini) — and together with the Appendix F control and the withdrawn claim addresses our main concerns, so we raise our rating to 3; the RL evidence remains confined to a single setup, and the camera-ready should include the promised additional datasets.

Reviewer_84P5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether diffusion language models can maintain diverse generations without sacrificing parallel decoding. Fork-dLLM introduces a simple modification to confidence-based sampling: when no candidate passes the confidence threshold, it unmasks the leftmost remaining position instead of the one with the highest confidence. Building on this idea, ForkGRPO extends it by applying GRPO updates only at these fallback steps. Experiments on two models and four math and code benchmarks show that the sampler generally approaches autoregressive pass@k performance while requiring substantially fewer model evaluations. The post-training experiments also show better accuracy than JustGRPO-Fast under the same training budget. I view the main contribution as a practical way to retain much of the efficiency of parallel generation while handling uncertain decisions more carefully.

Critical strengths

1/ A small, practical change yields a useful accuracy–compute tradeoff. Fork-dLLM modifies only the fallback rule and introduces no new hyperparameters. The results across two models and four benchmarks, together with comparisons against tuned Fast-dLLM and TCT, make the inference contribution convincing. Reporting both pass@k and pass@NFE also makes explicit the cost of recovering solution coverage.

2/ The training gains under matched compute are meaningful. ForkGRPO and the locally trained JustGRPO-Fast baseline use the same LoRA setup and GPU-time budget. ForkGRPO completes more training steps and delivers substantially larger improvements — for example, +6.7 versus +2.1 accuracy points on GSM8K. This provides concrete evidence that the approach can make post-training more effective within a limited budget.

Critical weaknesses

I find the sampling idea useful and the practical results promising. However, the claimed exactness of the likelihood ratios requires a more convincing justification, and the experiments do not fully isolate the benefit of the proposed training rule from my perspecitve.

1/ The claim of exact policy-likelihood ratios requires a stronger justification. Algorithm 2 determines whether fallback occurs after candidate tokens have been sampled, based on their sampled-token probabilities. Conditioning on this event may change the distribution of the committed token, yet Equation (3) uses the raw token-probability ratio without accounting for this selection effect. It is not clear why committing a single token alone is sufficient to establish exactness for the induced sampling policy. Since this is a central claimed advantage of ForkGRPO, an explicit derivation is needed.

2/ The experiments do not isolate the benefit of training on fallback tokens. Table 1 varies both the rollout sampler and the positions included in the loss. Therefore, achieving similar accuracy with fewer backward passes does not establish that fallback positions are more informative than entropy-selected ones. A comparison using the same rollout sampler and the same number of optimized tokens would help determine whether the proposed selection rule contributes beyond the savings from parallel generation.

3/ The headline training-cost comparison mixes different fine-tuning setups. ForkGRPO uses LoRA, whereas the released JustGRPO checkpoint uses full fine-tuning. Although the paper acknowledges this difference, it still limits what can be concluded from the reported eightfold reduction in GPU-hours. The matched-budget comparison against JustGRPO-Fast is useful; by contrast, the comparison against full JustGRPO does not establish how much of the saving comes from ForkGRPO itself.

Questions for authors

Does the implementation determine fallback from sampled-token confidence exactly as shown in Algorithm 2? If so, can you derive Equation (3) for the implemented sampling policy, accounting for both the fallback selection event and the sampling temperature?

2 comments

We thank you for your reviewing efforts and feedback. Please find our responses below.

1/ The claim of exact policy-likelihood ratios requires a stronger justification. Algorithm 2 determines whether fallback occurs after candidate tokens have been sampled, based on their sampled-token probabilities…

Does the implementation determine fallback from sampled-token confidence exactly as shown in Algorithm 2? If so, can you derive Equation (3) for the implemented sampling policy, accounting for both the fallback selection event and the sampling temperature?

We thank you for raising this point, which made us revisit our claims about the exactness of the policy-likelihood ratios used in ForkGRPO and realize that our original claim was incorrect. We have therefore removed all exactness claims from the paper (shown in red strikethrough) and now present the proposed ratios in Eq. (3) as a practical surrogate.

Our initial reasoning was motivated by DUEL (Turok et al., 2026), which establishes exact likelihood computation for dLLMs with deterministic position-selection rules. However, unlike the samplers considered in DUEL, Fork-dLLM determines which positions to commit based on the confidences of sampled candidate tokens. Consequently, its position-selection rule is stochastic, and the DUEL argument does not directly apply to our case, which we overlooked.

Yes, the implementation follows Algorithm 2. As you correctly point out, Fork-dLLM first samples candidate tokens and then evaluates their confidences to determine whether fallback occurs. We therefore need to account for the conditioning on the fallback event, which changes the distribution of the committed token. In Appendix D, we derive the exact fallback-conditioned token likelihood ratios under Fork-dLLM, showing that they admit a closed-form expression:

=\rho_t^{(\tau)} \cdot \frac{ Z_{\theta_{\mathrm{old}}}^{k_t}(\boldsymbol{x}_t;\tau) }{ Z_{\theta}^{k_t}(\boldsymbol{x}_t;\tau) } \cdot \mathbf{1}[ q_{\theta}^{k_t}(x_0^{k_t}\mid\boldsymbol{x}_t) \leq\lambda]$$ where $\rho_t^{(\tau)}$ is the ratio of Eq. (3) evaluated at the sampling temperature $\tau$ (here we omit the rollout index $i$ for brevity), and $Z_{\theta}^{j}(\boldsymbol{x}_t;\tau)= \sum_{v\in[V]}q_\theta^j(v\mid\boldsymbol{x}_t;\tau)\mathbf{1}[q_{\theta}^j(v\mid\boldsymbol{x}_t)\leq\lambda]$ is the probability that the candidate token sampled at position $j$ fails to exceed the confidence threshold $\lambda$ used in the Fork-dLLM sampler. In ForkGRPO, we compute Eq. (3) at $\tau=1$, following common practice (Zhao et al., 2025), and omit both corrections. Importantly, the new ratios are computationally tractable: the corrections depend only on the distribution at the committed fallback position $q_{\theta}^{k_t}(\cdot \mid \boldsymbol{x}_t)$ and require no additional transformer NFEs. While we do not have the time and compute to evaluate these fallback-conditioned ratios during the rebuttal, we believe this is a very promising direction for future work. We thank you again for bringing this issue to our attention. > The experiments do not isolate the benefit of training on fallback tokens. Table 1 varies both the rollout sampler and the positions included in the loss. Therefore, achieving similar accuracy with fewer backward passes does not establish that fallback positions are more informative than entropy-selected ones. A comparison using the same rollout sampler and the same number of optimized tokens would help determine whether the proposed selection rule contributes beyond the savings from parallel generation. > We agree that Table 1 varies both the rollout sampler and the optimized positions. To isolate the selection rule, we trained a ForkGRPO variant that replaces each rollout's fallback tokens with an equal number of its highest-entropy committed tokens. The table below reports the GSM8K accuracy of both variants across training checkpoints, evaluated with Fast-dLLM at λ = 0.95. We have added this comparison to Appendix F. | ForkGRPO Step | Fallback Rule - Acc. (%) | Entropy Rule - Acc. (%) | | --- | --- | --- | | 50 | 82.9 | 82.6 | | 100 | 83.8 | 83.9 | | 150 | 85.1 | 85.5 | | 200 | 85.4 | 85.1 | | 250 | 86.4 | 70.1 | Both variants perform comparably for the first 200 steps, after which the entropy-selection variant collapses: its step-250 checkpoint reaches 70.1% accuracy, compared to 86.4% for ForkGRPO and 80.7% for the base model. This collapse coincides with a sharp drop in decoding NFEs, from 90 at step 50 to 39 at step 250. Since Fast-dLLM commits every token whose confidence exceeds λ, this means the model’s predictions become increasingly confident, even as its accuracy drops.

Based on these results, we cannot conclude that fallback positions are more informative per update than entropy-selected ones, and we have removed this claim from Section 4.2 (in red). However, our results suggest that optimizing fallback positions as proposed in ForkGRPO leads to more stable training. Interestingly, the collapse observed here resembles the one we previously observed in FastGRPO (Figure 6), which uses confidence-based fallback selection during rollouts. Given our new observation that the policy-likelihood ratios in Eq. (3) are approximate (see our previous response), we hypothesize that these instabilities may be related to the approximation error. Our empirical results suggest that the approximation may be less problematic when optimizing AR-style fallback positions, as in ForkGRPO, than when optimizing confidence- or entropy-selected positions. We leave this important investigation for future work given the rebuttal time constraints.

The headline training-cost comparison mixes different fine-tuning setups. ForkGRPO uses LoRA, whereas the released JustGRPO checkpoint uses full fine-tuning. Although the paper acknowledges this difference, it still limits what can be concluded from the reported eightfold reduction in GPU-hours. The matched-budget comparison against JustGRPO-Fast is useful; by contrast, the comparison against full JustGRPO does not establish how much of the saving comes from ForkGRPO itself.

We agree that the comparison with the released JustGRPO checkpoint does not isolate the savings due to ForkGRPO, since the two also differ in the fine-tuning method and hyperparameters. The matched-budget comparison against JustGRPO-Fast is therefore our main evidence for the efficiency of ForkGRPO. We have clarified this in the Figure 1 caption, Section 4.2, and the limitations (in red).

We hope these clarifications alleviate your concerns. We remain available for any follow-up questions.

  • Turok et al., DUEL: Exact Likelihood for Masked Diffusion via Deterministic Unmasking, arXiv 2026
  • Zhao et al., d1: Scaling reasoning in diffusion large language models via reinforcement learning, NeurIPS 2025
Reviewer_txLAOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Fork-dLLM, a method designed to address the flexibility trap problem. The core idea and implementation is simple, when no token clears the threshold at decoding step, the fallback token choice is replaced to the leftmost token, instead of the most confident one. In this way, the model prefers an AR-style order, which reduces the chances that the branching tokens are generated too late.The simplicity of the method allows one to drop it in as a rollout mechanism during GRPO.

Critical strengths

** The method is a very simple change**: only fixing the fallback rule of fast-dLLM, the method also matches AR's pass@K quality while keeping most of the parallel speedup. This also makes the method a good drop-in on RL experiments.

Critical weaknesses

It is unclear whether branching tokens are picked earlier

The paper emphasizes Fork-mechanism, the decoding of the branching word is the key effect. However, the method does not guarantees this, it is more like an indirect bias towards the left token, which could decode the branching token early. This hypothesis though, is not fully tested.

One direct test could be to measure when fork tokens get committed under Fast vs Fork vs AR. In the ablation, picking the most uncertain slot does worse. So the evidence supports “AR order at uncertain steps helps". But this does not directly mean "forking tokens are helping".

Since this is the key claim of the paper. Clarity here would be very important to this work.

Questions for authors

Can the authors measure when fork tokens (e.g., top-entropy tokens under AR decoding) are committed during generation under Fast-dLLM, Fork-dLLM, and AR?

4 comments

We thank you for your efforts in reviewing our work. Please find our answers below.

The paper emphasizes Fork-mechanism, the decoding of the branching word is the key effect. However, the method does not guarantees this, it is more like an indirect bias towards the left token, which could decode the branching token early. This hypothesis though, is not fully tested.

Can the authors measure when fork tokens (e.g., top-entropy tokens under AR decoding) are committed during generation under Fast-dLLM, Fork-dLLM, and AR?

We first note that fork tokens are not uniquely defined, and identifying corresponding branching points across different samplers is challenging, as they produce different generations, unmasking orders, and predictive distributions.

Nevertheless, in the manuscript we provide several pieces of evidence that Fork-dLLM's fallback steps capture important branching decisions. First, Figure 5 (left) shows that when only fallback positions are sampled stochastically, while parallel positions are decoded greedily, pass@k scaling remains practically unchanged. This suggests that stochastic decisions at fallback steps account for most of the observed generation diversity. Second, the word clouds in Figure 4 show that fallback positions are dominated by connectives such as Since, First, and Thus, closely resembling the fork tokens identified by Ni et al. (2026) and Wang et al. (2025).

During the rebuttal, we additionally measured the entropy of tokens committed during parallel and fallback steps of Fork-dLLM on HumanEval with LLaDA-8B-Instruct (complementing the Fast-dLLM analysis in Figure 2):

StepShare of positionsMean entropyEntropy > 1
Parallel79%0.4817%
Fallback21%2.3295%

Fallback positions exhibit substantially higher entropy than parallel positions: 95% exceed an entropy of 1, compared to only 17% of parallel positions. Together with our stochasticity ablation, these results support our hypothesis that fallback steps capture high-entropy branching decisions that are important for generation diversity. We have added this analysis to Appendix G. We thank you for this question, which we believe strengthens our paper.

We hope these clarifications alleviate your concerns. We remain available for any follow-up questions.

  • Ni et al. The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models. ICML 2026
  • Wang et al. Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning. NeurIPS 2025

Thanks for the rebuttal.

I am aware that the paper provided evidence that Fork-dLLM helps. My question is actually on why it helps. Note that the claim "avoiding the flexibility trap" is a underlying thread of the paper. I would expect the authors to study this hypothesis more thoroughly.

This matters for future researchers to understand which one of the following reasons matters:

  1. Resolve forks early.
  2. Use AR (left-to-right order) when unsure.

The algorithm proposed here is mechanically simply "using AR when unsure". While I believe that this can alleviate the flexibility trap problem, it may also also help in other ways. For example, by using the AR order, this may make generation more coherent at uncertain steps. Figure 5 (right) indicates that picking the most uncertain token performs worse than picking the leftmost one: does that actually support reason 2 instead of reason 1.

We thank you for the prompt follow-up.

This matters for future researchers to understand which one of the following reasons matters: 1. Resolve forks early. 2. Use AR (left-to-right order) when unsure.

Resolving forks is what matters, and AR ordering at uncertain steps is our efficient way of achieving it. This follows the analysis of Ni et al. (2026), who argue that confidence-based ordering defers committing forks until their surrounding context is unmasked. This degrades the entropy of forks and limits generation diversity; AR ordering avoids this by directly confronting each fork, given only its left context. Building on this, Olausson et al. (2026) note that AR ordering is not special: fork entropy is retained by any ordering that tends to resolve forks before their surrounding context.

At Fork-dLLM's fallback steps, the leftmost position has its full left context yet remains uncertain. The evidence in our first reply suggests that such positions are forks (Figure 4, Figure 5 left, Appendix G), which Fork-dLLM resolves instead of deferring. We have revised the related work accordingly (Appendix A, in red).

For example, by using the AR order, this may make generation more coherent at uncertain steps.

We agree that left-to-right order keeps generation more coherent than random or most-uncertain orderings (Figure 5, right). However, coherence does not explain the gain over Fast-dLLM, whose confidence ordering already favors single-sample coherence (Ni et al., 2026). Fork-dLLM has similar or lower pass@1 but higher pass@64 on both models (Tables 4 and 5), and adding the AR-style fallback to EB-Sampler and KLASS shows the same pattern (Appendix E). The gain is in diversity, as expected from resolving forks.

Figure 5 (right) indicates that picking the most uncertain token performs worse than picking the leftmost one: does that actually support reason 2 instead of reason 1.

In this ablation, the most uncertain position is not necessarily a fork. Since forks are difficult to detect, uncertain positions can serve as a proxy for them, but the two are not equivalent: a fork position is uncertain because multiple continuations are plausible, whereas other positions can be uncertain simply because the context they depend on is still masked. In dLLMs, confidence is highest at masked positions directly after the unmasked tokens and decreases further to the right (Gong et al., 2026). The least confident positions therefore tend to lie far from the unmasked tokens, where the context they may depend on is often still masked. As a result, this ablation does not isolate the effect of resolving forks.

We hope this clarifies why Fork-dLLM helps. We remain available for any further questions.

  • Ni et al. The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models. ICML 2026
  • Olausson et al. A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models. COLM 2026
  • Gong et al. DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation. ICLR 2026

Thanks for the clarification, which addresses my main concern. I am raising my score to 3.

Reviewer_TtFLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper points out an important limitation of using AR sampling to address the flexibility trap in dLLMs: while AR sampling improves pass@k scaling, it gives up the core advantage of dLLMs, parallel decoding. To address this, the authors propose Fork-dLLM, which unmasks the leftmost masked position only at Fast-dLLM fallback steps while keeping confidence-based parallel decoding at the other steps; Fork-dLLM achieves pass@k scaling close to AR with 2–3× fewer NFEs and consistently improves over Fast-dLLM, especially at larger k. The paper also proposes ForkGRPO, which performs GRPO updates only at fallback steps of Fork-dLLM rollouts, achieving performance comparable to JustGRPO with much lower compute; for example, the GSM8K comparison uses 8× fewer H100-hours. In addition, the paper shows that properly tuned Fast-dLLM and TCT achieve much stronger pass@k scaling than previously reported and come close to AR, challenging the prior conclusion that avoiding the flexibility trap requires giving up confidence-based parallel decoding and using fully AR sampling.

Critical strengths
  1. It is important to clearly point out the trade-off that AR sampling improves pass@k scaling but gives up parallel decoding, which is a core advantage of dLLMs. The idea of applying AR-style ordering only at Fast-dLLM fallback steps is also simple and convincing, and gives consistent pass@k gains over Fast-dLLM without extra training or additional model evaluations.

  2. The paper is easy to read, the experimental setup and compute costs are reported clearly, and the ablations support the main claims well. In particular, Figure 5(right) convincingly shows that choosing the leftmost position at fallback steps is important for the gain of Fork-dLLM.

Critical weaknesses
  1. Fork-dLLM shows a good NFE–pass@k trade-off compared with Fast-dLLM and TCT, but the comparison does not include several recent training-free parallel decoders such as EB-Sampler [1], KLASS [2], and DAPD [3]. The authors also mention that the Fork-dLLM principle may apply to similar confidence-based samplers, so the contribution would be stronger if they showed that similar modifications also improve these methods.

  2. The paper emphasizes that confidence-based sampling does not need to be abandoned, but Fork-dLLM itself still uses AR-style ordering to recover diversity. It is therefore somewhat unclear to me which conclusion of the prior flexibility-trap work the paper aims to revise.


[1] Ben-Hamu et al., Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking, NeurIPS 2025.

[2] Kim et al., KLASS: KL-Guided Fast Inference in Masked Diffusion Models, NeurIPS 2025.

[3] Kim et al., DAPD: Dependency-Aware Parallel Decoding via Attention for Diffusion LLMs, ICML 2026.

Questions for authors
  1. Can the authors provide even partial results showing whether Fork-dLLM remains competitive in NFE–pass@k trade-off against other strong parallel decoders, and whether a similar idea can be applied to them?

  2. Which conclusion of the prior flexibility trap work is this paper trying to revise? Could the authors clarify their conclusion about confidence-based methods versus AR sampling?

2 comments

We thank you for your questions. Please find the answers below.

Fork-dLLM shows a good NFE–pass@k trade-off compared with Fast-dLLM and TCT, but the comparison does not include several recent training-free parallel decoders such as EB-Sampler [1], KLASS [2], and DAPD [3]…

Can the authors provide even partial results showing whether Fork-dLLM remains competitive in NFE–pass@k trade-off against other strong parallel decoders, and whether a similar idea can be applied to them?

We focused on the Fast-dLLM baseline, since it is the focus of prior work on sampling diversity in dLLMs (Ni et al., 2026; Olausson et al., 2026) and one of the most widely adopted samplers.

During the rebuttal, we additionally evaluated EB-Sampler (Ben-Hamu et al., 2025) and KLASS (Kim et al., 2025). Note that the EB-Sampler selects the lowest-entropy position in its fallback case, similar to Fast-dLLM’s and KLASS’s max confidence position fallback. We therefore evaluate the two methods with their original fallback rule and with our AR-style fallback. All experiments use LLaDA-8B-Instruct on HumanEval with τ=0.8\tau=0.8. The table below reports pass@kk for k∈{1,4,16,64}k\in\{1,4,16,64\} and average NFEs. The full pass@kk and pass@NFE curves are provided in Appendix E of the revised manuscript.

Adding the AR fallback to EB improves pass@kk at larger kk, by +3.0 points at pass@64, at the cost of a small decrease in pass@1 (−0.8 points). KLASS shows the same trade-off more strongly: +8.5 points at pass@64 and −12.5 at pass@1. This mirrors the trade-off between Fork-dLLM and Fast-dLLM, and suggests that the AR-style fallback transfers to other confidence-based samplers. Fork-dLLM outperforms all EB and KLASS variants at pass@8 and above (Appendix E), and uses fewer NFEs. We thank you for this suggestion, which we believe strengthens our paper.

SamplerFallbackpass@1pass@4pass@16pass@64NFE
EB (γ=0.5)default0.2500.4600.6090.726111
EB (γ=0.5)leftmost0.2420.4610.6370.756112
KLASSdefault0.4150.5190.6040.677160
KLASSleftmost0.2900.4980.6560.762173
Fast-dLLMdefault0.4020.5190.6230.70796
Fork-dLLMleftmost0.3170.5150.6630.787108

Which conclusion of the prior flexibility trap work is this paper trying to revise? Could the authors clarify their conclusion about confidence-based methods versus AR sampling

We note that the Flexibility Trap work identifies an important limitation of confidence-based sampling in dLLMs: reduced generation diversity, as measured by pass@kk. To address this issue, the authors advocate switching to AR sampling during post-training. It is this proposed solution that our work aims to revise. Specifically, we show that confidence-based parallel samplers can recover the diversity of AR sampling (in terms of pass@kk) through a simple modification to their fallback step, while retaining their computational efficiency (in terms of NFEs). Fork-dLLM uses AR-style ordering only at fallback steps, where Fast-dLLM also commits a single token. It therefore gives up no parallelism. This is particularly important for RL post-training, where rollout generation is already a major computational bottleneck. Switching to AR sampling, as proposed in Flexibility Trap, is computationally expensive, as it requires substantially more NFEs than parallel samplers such as Fork-dLLM and Fast-dLLM.

We have clarified this distinction in the introduction of the revised manuscript (in red). We remain available for any follow-up questions.

  • Ben-Hamu et al. Accelerated Sampling from Masked Diffusion Models via Entropy Bounded Unmasking. NeurIPS 2025
  • Kim et al. KLASS: KL-Guided Fast Inference in Masked Diffusion Models. NeurIPS 2025
  • Ni et al. The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models. ICML 2026
  • Olausson et al. A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models. COLM 2026

The rebuttal fully addresses my concerns. In particular, I think the paper now presents a clear and convincing extension of the benefits of AR-style ordering discussed in the Flexibility Trap work to parallel decoding. Accordingly, I will raise my score from 2 to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.