- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
FastGuide accelerates reward-guided generation in dLLMs. It reuses the guidance vector across multiple token commitments while updating the base model’s predictions after each commitment. Low-confidence tokens are deferred. Experiments on Dream and LLaDA compare generation quality and runtime against sequential and parallel decoding methods.
Critical strengths
-
The method separates guidance updates from base model updates. Figure 3 supports this design. Reusing the guidance while updating token predictions helps reduce the quality loss of parallel generation.
-
The paper reports substantial runtime improvements over sequential reward-guided decoding, with guidance computation included in the measurements. This demonstrates a practical improvement within the tested settings.
Critical weaknesses
-
All generation experiments use Dream-7B-Instruct or LLaDA-8B-Instruct. There is no evaluation on a more competitive model such as SDAR with its optimized inference engine. The results therefore do not establish whether FastGuide remains useful when the underlying model and decoder are stronger.
-
The benefit over simpler alternatives at equal runtime is unclear. On Dream/RM-Bench, FastGuide and unguided BoN take 6.2 and 6.3 seconds, respectively, but their LMUnit scores are 4.04 and 4.06. All experiments also use 128-token outputs, leaving its effectiveness on longer generation untested.
Questions for authors
-
Does FastGuide improve quality at equal runtime on SDAR or other competitive dLLMs, using its optimized inference engine and including all guidance costs?
-
How does FastGuide compare with BoN and DAPD at equal runtime on longer outputs, using task accuracy or code execution rather than reward-model and LLM-judge scores?
Rebuttal Response
We thank the reviewer for their insightful feedback. We now respond in detail to each question.
We agree it is valuable to discuss FastGuide in the context of modern block-diffusion models such as SDAR. We note, however, that our work accelerates gradient-based reward guidance, and to the best of our knowledge there is no prior work on gradient-based reward guidance for block-diffusion models such as SDAR. This is because reward guidance hinges on constructing a completion of the full response at every intermediate sampling step, which is then passed to the reward model to compute gradients. A full-canvas diffusion model provides this completion essentially for free, since every forward pass outputs a marginal distribution at every position of the response (which APS and EntRGi exploit). A block-diffusion model exposes marginals only inside the block currently being denoised, which is not sufficient to construct a response beyond the block boundary.
Nonetheless, FastGuide itself is compatible with any response completion algorithm. To demonstrate this, we try a very simple implementation of gradient-based reward guidance on SDAR that uses a full sampled rollout at every timestep to construct a reward model input. We then compare doing sequential gradient-based reward guidance versus FastGuide using the same response completion algorithm. The below table shows results on 50 prompts from JudgeBench using the SDAR-8B-Chat optimized inference engine with block size 32 and guidance with a Skywork-Reward-V2-Qwen3-1.7B model. Our preliminary experiments show that FastGuide is able to maintain similar performance to sequential guidance while being significantly faster. In contrast, when performing parallel generation with k=8, generation quality drastically drops.
It is important to note that the reported inference time includes the time to generate rollouts at every timestep, which is clearly very expensive. It is an important direction of future work to develop efficient response completion algorithms for block-AR diffusion models.
Below, we include performance on GSM8K and MATH500 benchmarks using 256-length generations and task accuracy as a metric (Top@1 is the answer picked with highest reward model score). These results show similar trends to Table 1, where FastGuide retains a large proportion of sequential guided decoding (first row) while being around 5x faster. Furthermore, BoN sampling requires around 16 generations to be close to FastGuide performance, greatly increasing sampling time. At roughly equal runtime compared to DAPD and BoN, FastGuide has higher task accuracy. Interestingly, on the harder MATH500 benchmark, there is a larger gap between sequential and parallel decoding, possibly due to parallel decoding being harder for more complex reasoning tasks.
GSM8K (300 prompts)
MATH500 (200 prompts)