- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces HOPD, an on-policy distillation method for diffusion language models. It applies teacher supervision to all masked positions, including predictions the student has not yet committed. It also uses the completed decoding trajectory to give more weight to unstable predictions. Experiments show improved average accuracy on math and coding tasks, along with faster decoding on MATH500.
Critical strengths
- The idea is simple and well motivated. HOPD uses teacher guidance for token predictions that TOPD leaves out, without extra model forward passes.
- The experiments cover math and coding, different model sizes, and both static and dynamic decoding. The results show improvements in accuracy and decoding speed.
Critical weaknesses
- The paper does not show why hindsight-based hesitation is a better weighting signal than token confidence or entropy. Token changes may also reflect sampling noise or valid alternatives.
- The paper uses evaluation benchmarks to select checkpoints (Appendix B.4, lines 848–851). This may lead to overly optimistic results on those benchmarks.
Questions for authors
- Do the hindsight weights outperform weights based on token confidence or entropy?
- The paper argues that deferred and retracted predictions need different treatment. How does HOPD address both needs, and what evidence shows the benefit of supervising each group?
Response to Reviewer E7ng
We thank the reviewer for the helpful comments. We address the concerns as follows.
W1 & Q1. Hindsight weights versus confidence-based weights
Thank you for this question. We trained a variant of HOPD that replaces the hindsight weights with confidence-based weights 1 − c, where c is the student's confidence in its proposed token, and keeps everything else unchanged. The results is given by in the following.
HOPD outperforms the confidence-based variant on the five-benchmark average in all four settings, by 1.0/0.3 on SDAR-4B and 1.5/0.9 on SDAR-1.7B. The hindsight weights are therefore the more effective signal. The reason is that confidence, like entropy, is measured at a single step and cannot tell whether a proposal will later be kept or overturned. A deferred prediction, a low-confidence proposal that is later committed unchanged, receives a large weight, while a retracted prediction may receive a small one.
W2. Checkpoint selection
Thank you for raising this point. The selection rule in Appendix B.4 is the same for all methods. To remove its effect, we evaluated every TOPD and HOPD model at round 30, the last round of our fixed training budget, without using any benchmark to choose the checkpoint. Entries are static/dynamic.
Without selection, HOPD is never below TOPD in the six model–decoding settings. The gain is largest on SDAR-1.7B, +0.5/+2.2, and on LLaDA-8B under static decoding, +1.1. On SDAR-4B, the two methods reach comparable accuracy, but HOPD decodes faster. The table below gives the tokens committed per denoising step under dynamic decoding.
HOPD commits 10–12% more tokens per step on SDAR-4B and 20–28% more on SDAR-1.7B. Therefore, HOPD's advantage does not depend on checkpoint selection. It is more accurate on SDAR-1.7B and LLaDA-8B, and on SDAR-4B it reaches comparable accuracy while decoding faster.
Q2. Handling deferred and retracted predictions
Thank you for this question. HOPD supervises both groups with the same loss, but treats them differently in the hindsight weights. The loss matches the teacher's distribution at each position, so its effect depends on the teacher. Where the teacher is confident, the student can learn to commit a deferred proposal earlier, and where the teacher is uncertain, a retracted position stays uncertain. In the weights, a deferral matches the final token and adds no weight, while a retraction raises both the position and block weights.
To test each group, we removed it from the uniform target set. Entries are the math three-benchmark average, static/dynamic.
Overall, supervising both groups improves the average over removing either of them, and HOPD gives the best results in all four settings. Therefore, both groups contribute, and weighting them by hindsight further improves the results.