- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how diffusion language models can reuse previous reasoning after user requirements change. HiPE is a training-free method that inserts selected passages from a previous partial answer during denoising, allowing the model to revise them in later steps. It uses the current generation state, attention, and token predictions to decide when to intervene, which passages to select, and where to place them. Experiments on mathematical reasoning and coding tasks show modest accuracy gains over RESTART on AIME and LiveCodeBench, with similar accuracy on GSM8K and MATH-500. HiPE also achieves higher accuracy than the evaluated warm-start baselines.
Critical strengths
Training-free intervention without additional decoder calls. HiPE uses the accepted-token fraction, along with attention and predictive distributions from existing forward passes, to decide when, what, and where to inject. The inserted text remains revisable during subsequent denoising, offering a practical way to reuse earlier reasoning without modifying model weights or decoding rules.
Controlled comparisons reveal meaningful differences between reuse strategies. Under matched contexts and generation budgets, HiPE achieves higher average accuracy than RESTART on AIME and coding tasks and surpasses all three warm-start baselines across the evaluated tasks. Although gains over RESTART are relatively modest, these results suggest that initialization-based reuse and selective injection during generation may have different effects.
Critical weaknesses
W1. Missing clean-restart and continuation baselines. RESTART keeps the original question and previous reasoning in context, while the warm-start methods begin a new generation rather than continue from the interrupted answer. The paper does not test continuation under the updated conditions, nor does it include a clean restart that solves the fully updated problem without the earlier question or reasoning. Given the relatively small gains over RESTART, it is unclear how HiPE compares with these two alternatives.
W2. No comparison with simple update guidance. The condition-update tasks come from Wu et al. [1], who show that brief prompt guidance can improve adaptation in their evaluated models. However, the paper does not test this simpler approach with DiffusionGemma under the same experimental settings. It is therefore unclear whether HiPE offers an advantage over simply reminding the model to follow the updated requirements.
[1] Wu et al. Are Large Reasoning Models Interruptible? ICML 2026.
Questions for authors
Q1. Practical efficiency. Could you report wall-clock latency under the same hardware and decoding settings, including the overhead of selecting and placing historical passages? NFE alone does not show whether HiPE provides a practical speed advantage over RESTART.
Q2. Intervention timing. How were the default accepted-fraction bounds of 0.25 and 0.45 and the cutoff of t < 12 selected? Table 10 shows that different windows can lead to different accuracy and NFE results, particularly on AIME 2025. Were these settings fixed before evaluation, selected using held-out data, or tuned on the evaluation tasks?
Minor comment on figure readability. The axis labels and tick labels in Figure 3 are relatively small. In Figure 4, some curves and distributions are difficult to distinguish, especially where they overlap. Larger labels and more distinct colors or line styles would improve readability.
hank you for the concrete suggestions concerning baselines, practical efficiency, intervention timing, and figure readability.
Regarding W1 and W2, we conducted follow-up experiments with clean restart and explicit prompt guidance baselines, using the same problems, generation seeds, and output budgets within each dataset. Clean restart solves the fully updated problem without earlier reasoning, while prompt guidance explicitly instructs the model to adapt to the update. These comparisons did not establish a consistent advantage for HiPE over these alternatives. Clean restart achieved higher accuracy point estimates with greater NFE, while prompt guidance showed mixed results. We plan to expand these comparisons to include a strict continuation baseline and evaluate accuracy–computation trade-offs more systematically in a future submission.
Regarding Q2, the original accepted-fraction bounds and the (t<12) cutoff were chosen heuristically and require stronger justification. In our follow-up experiments, we removed this cutoff and selected the margin and accepted-fraction window using a MATH validation subset, then fixed the configuration for evaluation. We intend to further investigate sensitivity to intervention timing and parameter selection.
Regarding Q1, we agree that NFE reductions alone are insufficient to demonstrate wall-clock improvements. A comprehensive, controlled latency evaluation will be important for establishing practical efficiency. We also acknowledge the concerns about figure readability and plan to increase label sizes and make overlapping curves easier to distinguish.
We appreciate these suggestions and will incorporate the lessons from these evaluations into a more carefully controlled study for a future submission.