- Rating
- 3(+2)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The reasoning capabilities of Diffusion Large Language Models (dLLMs) are typically improved through additional post-training, which is computationally expensive. To address this issue, the paper proposes a method for further enhancing model capability after post-training has already been completed. By using the logits of the pre- and post-training models during inference to compute a guidance signal, the method enables a dLLM to self-improve at inference time using only existing checkpoints.
Critical strengths
The paper proposes an efficient inference-time optimization and guidance algorithm that can further improve the capabilities of dLLMs.
Critical weaknesses
First, there is a problem with the contribution of the paper. The paper mentions statements such as “Furthermore, RFG matches or exceeds the gains delivered by resource-intensive RL post-training while remaining fully training-free,” emphasizing that RFG is more resource-saving. However, using RFG requires a model that has already completed RL post-training. It is not reasonable to claim that this part of the cost does not exist simply because one can directly use an RL-post-trained model trained by others.
Second, there is a problem with the related work of the paper. As the authors mention CFG, the related work of this work should be inference-time guidance methods, rather than post-training algorithms such as GRPO.
Questions for authors
Compared with a broader range of inference-time guidance methods, is the advantage of RFG clear? If the authors can demonstrate that RFG is an effective inference-time guidance method, I would be willing to raise my score.
4 comments
Response to Reviewer jYax (2/2): Questions
Comparison with a Broader Range of Inference-Time Guidance Methods (Q)
The reviewer asks whether the advantage of RFG is clear against a broader range of inference-time guidance methods, and states a willingness to raise the score if RFG is shown to be effective.
Yes. RFG is the best or tied-best method in all 48 model-benchmark-length settings of Tables 2 to 5, and the only one that improves the original model in every setting. The initial submission compared RFG with CFG (Tables 2 to 5) and policy-only sharpening (Table 9). We have added two recent inference-time guidance methods designed for dLLMs, A-CFG (Li et al., 2025) and FoCore (Feng et al., 2026), on all 48 settings. Like RFG, they need no training and no external reward model. CFG, A-CFG, FoCore, and RFG share the same cost of two forward passes per denoising step, and they differ in the second branch of the extrapolation form (Table R3).
Table R3: Guidance methods on post-trained dLLMs over the 48 settings of Tables 2 to 5. Changes are in accuracy points relative to the original model.
| Method | Second branch | Settings improved | Mean change |
|---|---|---|---|
| CFG | Same model, prompt masked | 8 of 48 | −3.7 |
| A-CFG | Same model, low-confidence tokens re-masked | 4 of 48 | −2.9 |
| FoCore | Same model, high-information-density tokens re-masked | 24 of 48 | +0.2 |
| RFG | Reference checkpoint, same input | 48 of 48 | +2.7 |
- Against the original model. CFG and A-CFG lower average accuracy, FoCore leaves it nearly unchanged, and RFG improves every setting.
- Head to head. RFG is above CFG in 47 settings with one tie, above A-CFG in all 48, and above FoCore in 46 with two ties. No method is above RFG in any setting.
Note that all methods are evaluated in the same setting as our main results in Tables 2 to 5. The A-CFG paper tests pre-trained models with few-shot prompts, and the FoCore paper relies on different prompts and the full MBPP test set, whereas we use the sanitized one. Absolute numbers therefore differ from those in the original papers. Despite this, FoCore’s ranking above CFG and A-CFG is preserved in our runs (21 and 23 of the 24 settings shared with its paper). Our original-model baseline is higher than the FoCore paper’s in 19 of these 24 settings. We consider this the fairest test of inference-time guidance, since each method must improve a stronger original model.
We thank the reviewer for this question. The revised paper adds A-CFG and FoCore to the experiments in revised Appendix.
Full Results (also in revised Appendix)
| Benchmark | Model | Length | Original | CFG | A-CFG | FoCore | RFG |
|---|---|---|---|---|---|---|---|
| GSM8K | LLaDA-Instruct | 128 | 76.3 | 72.6 | 74.8 | 76.3 | 77.2 |
| 256 | 79.8 | 80.7 | 77.9 | 81.3 | 81.3 | ||
| 512 | 83.7 | 81.5 | 81.3 | 82.8 | 84.6 | ||
| Dream-Instruct | 128 | 67.9 | 63.6 | 65.2 | 68.6 | 74.2 | |
| 256 | 80.9 | 72.2 | 75.4 | 80.2 | 82.1 | ||
| 512 | 84.8 | 80.1 | 81.9 | 84.0 | 85.1 | ||
| d1-LLaDA | 128 | 79.3 | 77.4 | 77.6 | 78.5 | 79.8 | |
| 256 | 82.5 | 81.3 | 80.8 | 82.1 | 84.7 | ||
| 512 | 83.6 | 82.2 | 82.9 | 82.5 | 84.6 | ||
| LLaDA-1.5 | 128 | 76.3 | 75.7 | 76.9 | 76.3 | 78.0 | |
| 256 | 81.6 | 82.0 | 79.6 | 81.3 | 82.1 | ||
| 512 | 84.8 | 81.3 | 84.0 | 84.0 | 85.3 | ||
| MATH-500 | LLaDA-Instruct | 128 | 32.4 | 23.8 | 30.2 | 32.8 | 34.4 |
| 256 | 38.4 | 34.0 | 35.6 | 37.4 | 39.6 | ||
| 512 | 42.6 | 39.0 | 40.0 | 43.6 | 44.0 | ||
| Dream-Instruct | 128 | 32.2 | 18.4 | 20.6 | 32.0 | 33.6 | |
| 256 | 43.6 | 22.8 | 31.6 | 41.2 | 46.4 | ||
| 512 | 50.4 | 29.6 | 41.2 | 51.2 | 52.6 | ||
| d1-LLaDA | 128 | 34.0 | 21.2 | 29.2 | 35.0 | 37.4 | |
| 256 | 38.8 | 33.8 | 36.4 | 39.6 | 41.6 | ||
| 512 | 42.6 | 39.4 | 40.8 | 41.6 | 43.8 | ||
| LLaDA-1.5 | 128 | 32.0 | 23.4 | 28.8 | 31.6 | 34.0 | |
| 256 | 38.6 | 33.6 | 35.0 | 39.4 | 40.6 | ||
| 512 | 42.0 | 36.8 | 39.8 | 44.0 | 44.0 | ||
| HumanEval | LLaDA-Instruct | 128 | 32.3 | 26.8 | 31.1 | 32.9 | 33.5 |
| 256 | 39.6 | 37.8 | 36.0 | 40.2 | 40.9 | ||
| 512 | 45.1 | 39.0 | 40.9 | 45.7 | 47.6 | ||
| Dream-Instruct | 128 | 52.4 | 57.3 | 59.1 | 54.9 | 60.4 | |
| 256 | 61.0 | 66.5 | 57.9 | 61.6 | 66.5 | ||
| 512 | 62.2 | 64.0 | 59.8 | 61.6 | 65.2 | ||
| LLaDA-1.5 | 128 | 32.9 | 25.6 | 30.5 | 34.1 | 37.8 | |
| 256 | 39.6 | 38.4 | 36.6 | 39.0 | 41.5 | ||
| 512 | 45.7 | 39.6 | 40.2 | 43.9 | 46.3 | ||
| diffuCoder | 128 | 70.1 | 68.9 | 65.9 | 72.0 | 73.8 | |
| 256 | 69.5 | 70.7 | 72.0 | 70.7 | 76.2 | ||
| 512 | 69.5 | 70.7 | 72.6 | 71.3 | 78.7 | ||
| MBPP | LLaDA-Instruct | 128 | 51.0 | 49.0 | 48.2 | 49.8 | 54.1 |
| 256 | 54.9 | 49.8 | 48.2 | 54.1 | 55.6 | ||
| 512 | 49.8 | 49.4 | 44.7 | 50.6 | 52.9 | ||
| Dream-Instruct | 128 | 65.0 | 61.9 | 63.8 | 64.2 | 68.1 | |
| 256 | 64.2 | 63.4 | 63.4 | 65.4 | 69.6 | ||
| 512 | 64.6 | 64.6 | 62.3 | 65.4 | 70.4 | ||
| LLaDA-1.5 | 128 | 51.8 | 48.6 | 47.9 | 53.3 | 54.1 | |
| 256 | 54.5 | 52.9 | 46.7 | 54.5 | 57.2 | ||
| 512 | 50.6 | 51.0 | 43.2 | 51.0 | 55.6 | ||
| diffuCoder | 128 | 72.4 | 70.8 | 71.6 | 73.9 | 74.3 | |
| 256 | 72.4 | 71.2 | 70.8 | 72.4 | 75.9 | ||
| 512 | 72.8 | 71.6 | 71.6 | 72.4 | 75.5 |
References
- Li, P., et al. (2025). Adaptive Classifier-Free Guidance via Dynamic Low-Confidence Masking. NeurIPS 2025.
- Feng, J., et al. (2026). Focus on the Core: Empowering Diffusion Large Language Models by Self-Contrast. arXiv:2605.01373.
We hope these results and clarifications address the reviewer's concerns and welcome further discussion.
Thank you for your response. The new experiments show that RFG improves the responses of post-trained models. I’m willing to raise my score.
However, I still have concerns about how the paper presents its contribution. Both SFT and RL take extra compute to train a model from a base checkpoint. That cost still exists even if the model developers release the SFT checkpoint. RL can start from a base checkpoint alone, and RFG needs both the base checkpoint and the post-trained one. This is a limitation of RFG, it’s why I’m still not convinced by the comparison with RL. I don’t think being “training-free” is a contribution in this paper.
We thank the reviewer for the prompt reply and for re-evaluating the paper. Following the reviewer's comment, we'll make these points explicit in the paper:
- RFG needs a post-trained (SFT / RL) checkpoint, so it builds on the cost of post-training and does not remove it.
- The comparison with RL is only a reference for the size of RFG's gains. It does not show that RFG removes the cost of post-training.
We thank the reviewer again for helping make the paper's claims more precise.
Response to Reviewer jYax (1/2): Weaknesses (W1, W2)
We thank the reviewer for the feedback and for recognizing RFG as an efficient algorithm that further improves the capabilities of dLLMs. We address each concern below.
TL;DR
Contribution and Comparison with RL Post-Training (W1)
The reviewer's concern is our claim that RFG "matches or exceeds the gains delivered by resource-intensive RL post-training while remaining fully training-free". The reviewer reads RFG as "requiring a model that has already completed RL post-training", so that the claim omits the cost of that training.
We share the reviewer's view that training cost for RL checkpoints must be accounted for, but we respectfully clarify that this reading does not match our setup.
Using RFG does not require RL checkpoints, as it works with SFT checkpoints as well, and the claim involves exactly that case: RFG applied to an existing SFT checkpoint, compared with RL post-training of the same checkpoint.
RFG works with either SFT or RL checkpoints. Our experiments use both, and half of the settings use SFT checkpoints that have undergone no RL (LLaDA-Instruct, Dream-Instruct). RFG improves all of them (Tables 2 to 5), so RL post-training is not a prerequisite for RFG.
The claim is made under a fair comparison in Table 1. It compares two ways of improving the same existing SFT checkpoint, LLaDA-Instruct:
Both methods thus start from the same existing checkpoint, so the comparison is fair and supports the claim. The revised caption of Table 1 states this setup explicitly.
We also note that
Scope of the Related Work (W2)
The reviewer notes that, since the paper relates RFG to CFG, its related work should cover inference-time guidance methods and not post-training algorithms such as GRPO.
We'd like to clarify that the paper does cover inference-time guidance. Due to the page limit, the main text of the initial submission held only a summary of the works we build on. The complete related work was left to Appendix D, which discusses inference-time guidance (classifier guidance, CFG, autoguidance, contrastive decoding, guidance for discrete diffusion). We include the post-training methods as well because they are a core part of our setting: they produce the SFT or RL checkpoints for RFG (e.g., d1-LLaDA and diffuCoder are trained with GRPO).
With the additional page allowed for the revision, we've moved the complete related work back into Section 5 and extended it with further inference-time guidance methods, including A-CFG and FoCore for dLLMs and Diffusion Blend and RLG for image diffusion.