- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how to mitigate the diversity loss associated with confidence-based decoding in masked diffusion language models while retaining parallel generation efficiency. The proposed Fork-dLLM sampler preserves parallel commitment of tokens above a confidence threshold but replaces Fast-dLLM’s highest-confidence fallback with the leftmost masked position whenever no candidate exceeds the threshold. The paper further introduces ForkGRPO, which generates rollouts with Fork-dLLM and applies the policy optimization objective only at fallback steps, reducing both rollout and optimization costs. Sampling experiments on LLaDA and Dream examine pass@k and NFE efficiency, while LoRA post-training experiments on LLaDA-8B-Instruct demonstrate improved compute efficiency relative to JustGRPO-Fast.
Critical strengths
1. The method is simple, inexpensive, and addresses a clear quality–efficiency trade-off. Fork-dLLM changes only the fallback rule of an existing sampler, without additional model evaluations or new sampling hyperparameters. Evaluating both pass@k and pass@NFE is valuable because it accounts for the computational cost of obtaining multiple samples. The results demonstrate that, in the evaluated settings, retaining confidence-based parallel generation does not necessarily preclude solution coverage close to that of autoregressive sampling.
2. The post-training experiments include useful controlled comparisons. The authors train ForkGRPO and JustGRPO-Fast with the same LoRA configuration and compute budget, and supplement this comparison with step-matched results. These experiments support the practical value of reducing training costs. The different stability behavior observed between FastGRPO and ForkGRPO under matched sampling hyperparameters is also potentially informative and motivates further investigation into the relationship between position selection and RL training dynamics.
Critical weaknesses
1. The methodological advance is limited, and its distinct contribution is not sufficiently isolated.
ForkGRPO largely retains JustGRPO’s policy optimization framework, replacing token-by-token autoregressive rollouts with parallel rollouts incorporating a leftmost-position fallback and restricting the objective to fallback steps. Since JustGRPO-Fast already performs sparse optimization at high-entropy positions, the principal changes concern rollout generation and position selection. The matched-compute improvements are useful, but they jointly reflect faster parallel rollouts and fewer optimized positions, without sufficiently isolating the contribution of fallback-based selection itself. In Table 1, the methods achieve similar accuracy at matched training steps, while ForkGRPO requires fewer backward passes. This supports computational efficiency, but does not by itself establish the stronger interpretation that fallback positions are more informative. Comparing fallback-based and entropy-based selection under the same rollout policy and with matched numbers of optimized tokens would more directly test this claim. The observation that FastGRPO collapses while ForkGRPO remains stable is potentially valuable, but its mechanism remains unresolved. The authors hypothesize that leftmost-position selection reduces dependence on RL-induced confidence changes. However, whether a fallback occurs still depends on model confidence, so the modification does not fully remove this coupling. The current evidence supports a stability difference in the tested setting more directly than a general mechanistic explanation. My concern is not simplicity itself, but whether this limited methodological change has been supported by sufficiently discriminating experiments and analysis to establish a substantial new insight.
2. The RL evaluation is narrow and does not establish applicability to deployment-oriented models or longer reasoning trajectories.
All RL experiments use LLaDA-8B-Instruct with LoRA and a generation length of 256 tokens. Although the sampling experiments also include Dream-7B, these do not establish the transferability of ForkGRPO’s training benefits. In particular, the method is not evaluated on more deployment-oriented block-diffusion models, such as the LLaDA 2.x or SDAR families. This is not simply a request for additional model coverage: the method relies on fallback events as a proxy for important decision points, while differences in attention structure, training objectives, and decoding mechanisms could alter confidence distributions, fallback frequency, and the influence of selected positions on final outputs. Consequently, the observed sparse-optimization benefits and training stability may not transfer directly. Likewise, the 256-token experiments do not establish whether optimizing only fallback positions remains effective for credit assignment, stability, and computational savings over longer reasoning trajectories. The reported pass@k scaling concerns solution coverage as the number of samples increases; it does not establish scalability with reasoning length, model size, or training budget. The authors acknowledge several of these limitations, but they still constrain the significance of the contribution. Without evidence from at least one longer-generation or deployment-oriented setting, the conclusions should remain explicitly limited to the demonstrated short-generation, single-backbone RL setup.
Questions for authors
-
Independent contribution of position selection: With the rollout policy fixed and the number of optimized tokens matched, does fallback-based selection still outperform entropy-based selection? What evidence distinguishes the claim that fallback positions are more informative from the alternative explanation that optimizing fewer positions is sufficient to obtain similar performance at lower cost?
-
Training stability: Is the contrast between FastGRPO collapse and ForkGRPO stability reproducible across multiple random seeds? Given that fallback triggering still depends on confidence, what evidence supports the proposed decoupling explanation beyond the observed stability difference in the reported configurations?
-
Scope of applicability: Do the authors have evidence from longer generations or block-diffusion models showing that fallback events continue to identify effective optimization positions while preserving stability and computational advantages? If not, which conclusions should be restricted to the current LLaDA, LoRA, and 256-token setting?
4 comments
We thank the authors for their detailed response and the new experiments added during the rebuttal.
What the response addressed. The entropy-selection control in Appendix F is precisely the comparison we asked for: with the rollout policy fixed and the number of optimized tokens matched, the fallback-selected variant remains stable while the entropy-selected variant collapses. We appreciate the candid withdrawal of the "more informative per update" claim and the revised, better-supported framing in terms of training stability. The additional FastGRPO seed partially addresses our reproducibility concern, and we agree that a full mechanistic analysis can be left to future work.
Why our assessment remains unchanged. Our primary concern (Weakness 2 / Question 3) is not resolved. The authors note that prior RL work on dLLMs (JustGRPO, d1, SPG, TCT) uses the same single-model, 256-token setup. We acknowledge this precedent, but we do not believe it answers the concern. Our question is not whether the setting is consistent with prior work, but whether the paper's conclusions are supported beyond a single narrow configuration. That a limitation is shared by the literature does not diminish its impact on the significance of this contribution; if anything, it makes evidence that escapes this setting more valuable, not less.
Concretely
- ForkGRPO's benefits rely on fallback events serving as a proxy for important decision points. This is a property of the confidence distribution of one model under one sampler. Differences in attention structure, training objectives, and decoding mechanisms in deployment-oriented block-diffusion models (e.g., the LLaDA 2.x or SDAR families) could alter fallback frequency, the informativeness of fallback positions, and therefore both the efficiency and the stability conclusions. No evidence is provided for any such model, even at the sampling level.
- A generation length of 256 tokens does not establish that optimizing only fallback positions remains effective for credit assignment over longer reasoning trajectories — precisely the regime where sparse optimization is most likely to fail. The reported pass@k scaling concerns the number of samples, not reasoning length, model scale, or training budget.
- The stability contrast, while interesting, is demonstrated in a single configuration. The revised claim that confidence-independent position selection stabilizes sparse RL is a general statement about RL for dLLMs, yet its current support is confined to LLaDA-8B-Instruct with LoRA and 256-token generations.
We recognize that the sampling contribution is more broadly supported (two backbones, four benchmarks) and that the method is simple and inexpensive. However, the RL contribution is a central part of the paper, and as currently established it remains confined to a setting we consider too narrow to support the broader claims. The authors have appropriately restricted their conclusions in the Limitations section, but this restriction also limits the significance of the contribution.
We therefore maintain our rating of 2 (weak rejection). We would reconsider if the authors provided evidence from at least one longer-generation or deployment-oriented (block-diffusion) setting showing that fallback-based selection retains its benefits; even inference-level fallback statistics on such a model would be a meaningful step in that direction.
Thanks for your reply.
We would reconsider if the authors provided evidence from at least one longer-generation or deployment-oriented (block-diffusion) setting showing that fallback-based selection retains its benefits; even inference-level fallback statistics on such a model would be a meaningful step in that direction.
While our computational resources and rebuttal time constraints do not allow us to conduct full RL experiments on LLaDA 2.0-mini, we have performed additional Fork-dLLM sampling experiments on LLaDA 2.0-mini (see Appendix H of the revised manuscript). We observe that the sampling benefits of Fork-dLLM transfer from LLaDA/Dream to LLaDA 2.0-mini. In particular, on the MBPP dataset Fork-dLLM is the only confidence-based sampler to catch-up with AR sampling in terms of pass@k, while outperforming it in terms of pass@NFE (it requires ~3× fewer NFEs to achieve ~90% solution coverage). We hope this addresses your concerns about fallback-based selection retaining its benefits on more modern block-diffusion models such as LLaDA 2.0. We will make sure to include results on additional datasets for LLaDA 2.0-mini in the camera-ready version.
Finally, we would like to reiterate that the two closest works to ours, The Flexibility Trap [1], which received an Outstanding Paper Award at ICML 2026, and TCT [2], both conduct their RL experiments exclusively on LLaDA. Hence, while we agree that a broader evaluation is always desirable, we would find it somewhat unfair to treat our experimental scope as grounds for rejection, given that our RL evaluation follows the established setup of closely related (and well-received) prior work.
[1] Ni et al., The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models, ICML 2026
[2] Olausson et al., A Tale of Two Temperatures: Simple, Efficient, and Diverse Sampling from Diffusion Language Models, COLM 2026
Appendix H meets the reconsideration condition stated in our previous comment — inference-level evidence that Fork-dLLM's benefits transfer to a block-diffusion model (LLaDA 2.0-mini) — and together with the Appendix F control and the withdrawn claim addresses our main concerns, so we raise our rating to 3; the RL evidence remains confined to a single setup, and the camera-ready should include the promised additional datasets.
We thank you for reviewing our work. Please find our responses below.
Thank you for this question. Reviewer 84P5 raised a closely related point, and we ran a new experiment to address it; please see our response there for further details.
We agree that our claim in Section 4.2 was not fully supported, since Table 1 varies both the rollout sampler and the optimized positions. To isolate the selection rule, we trained a ForkGRPO variant with the same Fork-dLLM rollouts that replaces each rollout's fallback tokens with an equal number of its highest-entropy committed tokens. Both variants perform comparably for the first 200 steps, after which the entropy-selection variant collapses (70.1% vs. 86.4% for ForkGRPO at step 250, with the base model at 80.7%).
Based on these results, we cannot conclude that fallback positions are more informative per update than entropy-selected ones, and we have removed this claim from Section 4.2 (in red). However, our results suggest that optimizing fallback positions leads to more stable training. We have added this comparison to Appendix F.
During the rebuttal, we additionally run FastGRPO with a different seed (β=0). Similar to the two previous runs, the training begins collapsing around step 355, reaching a reward of 0.24 by step 371. We thank you for this suggestion, which we believe strengthens our paper.
Regarding the mechanism, we agree that decoupling is a hypothesis rather than an established explanation. As you note, whether a fallback occurs still depends on model confidence; Fork-dLLM only makes the choice of position independent of it. Our new experiment adds a further observation: with the same Fork-dLLM rollouts, optimizing entropy-selected positions instead of fallback positions also leads to collapse (Appendix F). This suggests that the choice of optimized positions, and not only the rollout policy, affects stability. We leave a full analysis of the mechanism to future work.
We agree that our RL conclusions are established in a single setting (see Limitations, Section 5): LLaDA-8B-Instruct with LoRA and a generation length of 256 tokens. This follows prior work on RL for dLLMs, which trains a single model (LLaDA-8B-Instruct) with a single fine-tuning method and a generation length of 256 tokens, e.g. JustGRPO (Ni et al., 2026), d1 (Zhao et al., 2025), SPG (Wang et al., 2026), and TCT (Olausson et al., 2026). Our sampling results are broader, covering two models (LLaDA and Dream) and four benchmarks.