- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DiT-OPD, which incorporates Diagnostic Block Selection and Action-Conditioned Feedback into on-policy distillation. The frozen teacher determines whether guidance is needed based on the prefix preceding the student’s current block; when a skip is not performed, the selected action simultaneously determines the supervision mask and the teacher scoring prompt for the next 256-token block. The method computes feedback for the original student token and does not require the teacher during inference. The paper uses Qwen3-8B and Qwen3-1.7B to evaluate answer quality and output length across six math benchmarks, transfer tasks, and the LiveCodeBench self-repair task, and provides component-wise ablation results.
Critical strengths
- DBS uses only the prefix preceding the target block for diagnosis, while ACF performs teacher forcing on the original student token; the information boundary of the supervision process is clear, and no teacher is required during inference. This makes the method valuable for practical deployment.
- The length gains are relatively consistent. In Table 2, the average reasoning length across the six tasks decreased from 22,672 to 19,736 tokens compared to Standard OPD; in Table 4, the truncation rates for all six tasks were also lower than those of the three OPD controls.
Critical weaknesses
- It is unclear whether matching each diagnosed action to its teaching prompt is necessary. The generic-guidance ablation also changes the prompt content, so it cannot separate the benefit of action matching from the effect of the prompts themselves.
- The gains in answer quality over OPD baselines are limited. The self-repair evaluation also uses each model’s own failed programs, so the models receive different repair inputs. This makes it difficult to attribute the repair gains to better error correction.
Questions for authors
- Could you shuffle the action–prompt assignments while keeping the selected blocks, prompt frequencies, and training settings fixed, and compare answer quality and output length with DiT-OPD?
- Could you evaluate all checkpoints on the same set of failed programs with the same test feedback, including w/o DBS and Generic guidance?
- Appendix C reports about 148.6 million input tokens for diagnosing 512 trajectories. What are the total teacher token usage and wall-clock time for training?