- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
TIDE addresses two forms of teacher-student mismatch in on-policy distillation: unstable negative updates for student-excess tokens, and the rarity of student-deficit tokens in student rollouts. It combines gated Hellinger suppression with analytic recovery over the teacher's top-K tokens. The experiments cover nine math benchmarks and two teacher-student pairs.
Critical strengths
S1. The controlled-prefix and teacher-continuation experiments help diagnose a failure of OPD: local agreement can coexist with repetitive responses that still fail.
S2. Under strong mismatch, TIDE raises Avg@8 from OPD's 6.9% to 20.3%, a substantial improvement. Responses are also shorter and less likely to omit an answer.
S3. The component ablations and sensitivity studies help assess the method's design choices. Using the final checkpoint for every method avoids selecting checkpoints on the evaluation benchmarks.
Critical weaknesses
W1. It is unclear how much mismatch gating contributes beyond ordinary analytic teacher supervision. The deficit-only variant already reaches 18.32% Avg@8, close to AOPD, yet the experiments include neither ungated top-K cross-entropy nor a coverage-matched random gate. These controls would help establish whether the proposed selection rule accounts for the gains.
W2. The deficit score responds to teacher truncation even when the student matches the teacher. When q=p, Eq. 3 gives dt=−logPt, where Pt is the teacher's probability mass on its top-K set. The score is positive whenever Pt<1, so the deficit loss still pushes probability into that set despite perfect agreement. Positions can thus be selected because of the teacher's probability spread, with no actual student deficit. This also raises a question about the claim that matched positions receive no supervision.
W3. Appendix E.3 renormalizes teacher probabilities and removes the 1K factor from AOPD. The original AOPD objective uses unnormalized probabilities and includes that factor. This modification changes the strength of distribution matching relative to positive policy-gradient updates, so the reported 2.0-point Avg@8 gap needs to be checked using the published loss or a guidance-weight sweep.
W4. The two teacher-student pairs differ in student initialization, teacher size, and teacher strength, making it hard to attribute their different behavior to mismatch. The paper also reports neither variation across training seeds nor accuracy confidence intervals. These omissions matter for the smaller gains: 0.1 Avg@8 points over FiRe-OPD in the weak setting and 0.2 Pass@8 points over AOPD in the strong setting.
Questions for authors
Q1. How does TIDE compare with ungated top-K cross-entropy and a random gate with the same coverage?
Q2. How much of the deficit score comes from teacher truncation? Does TIDE prevent entry into loops, or increase the probability of exiting an existing loop?
Q3. Does the gain over AOPD remain with its published loss and a tuned guidance weight?
Q4. Please report variation across training seeds and the initial overlap values for both teacher-student pairs.