- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes OmniOPD, a logit-free on-policy distillation method that obtains teacher feedback through sampled textual continuations. It selects high-entropy chunks from student-generated trajectories, compares them with multiple teacher continuations, and uses a similarity score smoothed with a student-policy prior to weight the likelihood of the student’s own tokens. A reference-policy KL penalty anchors unaudited positions. Evaluation covers mathematical reasoning and code generation with Qwen3-1.7B/4B students and open-weight or hosted teachers. The paper reports improvements over sequence-level distillation and includes component ablations, teacher-proxy diagnostics, failure analyses, and an analytical teacher-inference cost model.
Critical strengths
The paper addresses a concrete access constraint. Strong teachers may expose generated text without providing token-level probabilities. OmniOPD supplies local feedback on student-generated prefixes using sampled continuations, without requiring teacher logits or a separately trained discriminator. This is a meaningful practical setting, and the inclusion of hosted-teacher experiments directly addresses the motivating constraint.
Critical weaknesses
- Unresolved inconsistencies between the evaluation protocol and reported results undermine the reliability of the experimental evidence.
Lines 277–281 specify 100 problems per coding benchmark and Pass@1 averaged over 16 independent sampling trials. With one binary outcome per problem per trial and no additional averaging, each percentage must equal , for an integer , before rounding. However, Table 6 reports values such as 46.78 for Seq-KD and 54.28 for OmniOPD on APPS, neither of which can arise under this protocol through ordinary two-decimal rounding.
Under this interpretation, at least 21 of the 24 individual benchmark entries are incompatible with the stated setup, affecting all three methods. Related discrepancies arise in the mathematical results under standard benchmark-size assumptions. With 30 AIME problems and 16 binary trials per problem, percentages must equal ; the OmniOPD AIME-2024 score of 68.67 in Table 1 cannot be obtained through ordinary rounding or truncation. Likewise, assuming the commonly used 40-problem AMC23 split, the scores of 90.44 in Table 1 and 92.69 in Table 2 require clarification.
These are substantive reproducibility concerns, not merely formatting issues. Additional averaging, different effective sample counts, or non-binary scoring could explain some discrepancies, but the manuscript does not document such procedures sufficiently to reconstruct the results. If partial test-case credit was used for coding, it must be distinguished from binary program-level Pass@1. Until the discrepancies are reconciled with a reproducible aggregation procedure and supporting records, I cannot regard the affected results as reliable evidence for the paper’s conclusions.
- The claimed semantic advantage and the conditions under which the surrogate improves reasoning are insufficiently established.
The default edit-distance metric and the ROUGE alternative primarily measure surface overlap. Theorem B.4 correctly transfers invariances already possessed by the similarity function; it does not establish that the implemented metrics tolerate meaning-preserving paraphrases while distinguishing small but consequential reasoning errors. For an identity-separating edit similarity, the induced equivalence classes can be trivial. The existing correlation analyses do not directly establish the semantic robustness claimed in the motivation. The optimization evidence also leaves an important gap. Appendix E.7 reports increasing compatibility scores and decreasing training loss alongside reasoning collapse without KL anchoring. Table 11 shows severe sensitivity to chunk length: configurations with and have equal nominal audited-token and teacher-output-token budgets but substantially different outcomes. This does not invalidate the full method, and the authors appropriately disclose these failures. It does, however, leave the operating conditions of the central surrogate insufficiently understood. The conditional variance and drift bounds do not establish when these updates improve reasoning rather than reinforce superficial agreement or existing student preferences.
Questions for authors
-
How were the Table 6 APPS scores of 46.78 and 54.28 obtained from 100 problems and 16 sampling trials? Was there additional sampling, averaging, or filtering?
-
Does coding evaluation use binary program-level correctness or partial test-case credit? If partial credit is used, what are the corresponding binary Pass@1 results?
-
Which AIME and AMC23 subsets and sample counts were used? How were the AIME score of 68.67 and AMC23 scores of 90.44 and 92.69 computed?
-
Can you provide the per-problem evaluation records, scorer configuration, and aggregation script needed to reproduce these entries?
-
What direct evidence shows that the implemented similarity metrics tolerate meaning-preserving changes while distinguishing consequential reasoning errors?
-
How does the method compare with constant or shuffled teacher weights, keeping the update structure, KL regularization, and sampling budget unchanged?
-
Why does fail so severely relative to , despite equal nominal audited-token and teacher-output-token budgets? Does it exhibit the same collapse as the no-KL variant?