- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Flash-OPD accelerates on-policy distillation by stopping each student trajectory when its cumulative count of low teacher–student top-k overlap events reaches a budget. Generation alternates with teacher verification; an estimated event rate schedules checks, while observed counts determine stopping. Experiments report 2.23–7.52× training speedups over full OPD across four teacher–student pairs, with broadly comparable accuracy and a larger improvement for the Qwen3-4B student.
Critical strengths
- The measured savings are substantial and include verification overhead. For Qwen3-4B, average accuracy rises from 15.85 to 19.20 while training time falls from 33.09 to 4.40 hours (Table 2). This is a useful practical result.
- The implementation and cost analysis are unusually concrete. Appendices B–C explain exact boundary localization, EOS handling and caching; Tables 6–7 quantify verification costs and decoding overshoot. These details make the reported acceleration easier to assess.
Critical weaknesses
- The experiments do not isolate the benefit of trajectory-specific stopping from simply using much shorter rollouts. TOPD uses 4,096 tokens and Prune-OPD has a 1,024-token minimum, whereas Flash-OPD retains only 304–585 tokens on average in three settings (Appendix D.3; Table 7). A similarly short fixed-horizon baseline, or a control preserving the stopping-length distribution while removing its association with each trajectory, is needed to establish the central contribution. Appendix F acknowledges this gap.
- The cumulative overlap budget is not independently validated as a boundary of useful supervision. Equation (4) permanently accumulates low-overlap events, without distinguishing persistent drift from recoverable disagreement. Figure 2 measures horizon mismatch against this same criterion-defined boundary, while the stopping propositions establish correct boundary detection. Neither directly establishes that the discarded suffix provides less useful supervision. Appendix F acknowledges this limitation, but the reliability interpretation remains insufficiently supported.
- The contribution of adaptive verification scheduling is not established separately from early stopping. With exact boundary localization, fixed-interval verification would retain the same prefix on a fixed trajectory (Appendix B.2). Figure 8 varies the proposed scheduler's discount factor but does not compare against fixed verification blocks. The reported speedups therefore do not establish whether event-rate extrapolation improves the verification-cost/decoding-overshoot trade-off over a simpler cached implementation.
The practical speedups are promising, but the missing matched-budget comparison leaves the value of trajectory-specific allocation unresolved. The interpretation of the cumulative overlap threshold as a useful-supervision boundary also needs direct validation.
Questions for authors
- At matched retained-token or time budgets, how does Flash-OPD compare with fixed horizons near 300–600 tokens or with stopping lengths reassigned across trajectories?
- With the same stopping criterion and prefix caching, how does event-rate scheduling compare with fixed verification blocks in verification cost and decoding overshoot?
- What evidence from the existing full rollouts links the Equation (4) crossing to reduced suffix supervision utility, rather than temporary or recoverable teacher–student disagreement?
3 comments
Rebuttal by Authors (Part 2)
Method-related
-
W2 & W4 & Q3: Is the crossing a boundary of useful supervision?- We think there is a framing mismatch worth making explicit: Eq. (4) is a budget on accumulated unreliability, not a change-point detector, and it is not intended to classify a disagreement as persistent or recoverable. The operative question for a cost-control rule is not "will this trajectory recover?" but "at the point the budget is exhausted, does continuing to supervise buy accuracy?" That question has a direct answer, and it is the measurement you ask for in Q3.
- The allowance is a dose knob on exactly the discarded tokens, so sweeping it is the utility measurement. On G3, the curve saturates around the boundary, as shown in the table below:
- Doubling the retained length beyond the boundary yields a improvement, which falls within the random fluctuation range observed in our measurements. Full OPD represents the limit for this dosage: it supervises to more tokens on G2–G4, yet is not more accurate overall (Table 2 in our paper).
- Therefore, in the only measurable sense, namely its effect on training, supervision beyond the critical point is not a factor on which accuracy relies.
| G3 Setting | Retained Tokens / Trajectory | Average Accuracy | vs Default |
|---|---|---|---|
| (default) | — | ||
Thank you again for your constructive suggestions, which have helped us substantially sharpen the positioning of this work. If possible, we kindly request that you reconsider the score, and we would be very happy to discuss any remaining questions.
Best regards,
Authors
Kindly Request for Reviewer's Feedback
Dear Reviewer tHrP,
We hope this message finds you well. As the author–reviewer discussion period approaches its end, we wanted to gently follow up and thank you once again for your thoughtful and constructive review. Your insightful comments regarding trajectory-specific stopping versus fixed truncation, the utility of supervision beyond the crossing boundary, and separating the stopping rule from adaptive scheduling helped us substantially strengthen the paper. We have provided detailed analyses, allowance-sweeping measurements, and new matched-length experimental results in our response above to address each point.
We truly appreciate your valuable time and guidance. If our responses and new results have satisfactorily resolved your concerns, we would be deeply grateful if you would consider raising your score. Please also let us know if any further clarification would be helpful.
Warm regards,
The Authors
Rebuttal by Authors (Part 1)
Dear Reviewer tHrP,
We sincerely thank you for your constructive feedback and the effort invested in reviewing our work. We have carefully addressed each of your concerns below: (
We wrote this rebuttal entirely by ourselves, word by word, without using any AI assistance. We hope that this can reciprocate your effort in taking the time to review it. Thank you!). Specifically:W1 & W4 & Q1: Trajectory-specific stopping vs. simply shorter rolloutsStudent / Teacher: Qwen3-1.7B-Base / Qwen3-4B (Non-thinking)Student / Teacher: Qwen3-4B-Base / Qwen3-4B (Non-thinking)W3 & Q2: Separating the stopping rule from the adaptive scheduling of Eq. (5)