- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies a token-coverage problem in teacher-top- on-policy distillation. The authors show that a teacher’s top- set can retain almost all of the teacher’s probability mass while still omitting high-probability tokens under the student distribution. This issue is particularly evident for the special <tool_call> token: although the response teacher has near-complete probability-mass coverage, it may fail to include the student-relevant call-entry coordinate and therefore cannot provide an effective corrective signal. In a routed multi-teacher setting, the tool teacher reinforces the call entry, whereas the response teacher often fails to provide the corresponding negative correction, creating an asymmetric supervision signal. Based on this diagnosis, the paper evaluates three interventions for correcting the finding.
Critical strengths
The paper conducts a comprehensive set of analytical experiments, revealing the biases introduced by top--support distillation and evaluating the effects of several intervention mechanisms.
Critical weaknesses
-
The diagnosis has limited persuasive power and conceptual novelty. The diagnosis is primarily based on a single <tool_call> token in the Qwen3.5 tool-use setting. Although the paper later reproduces the coverage pattern in Llama’s JSON setting, the analysis remains largely confined to tool-calling scenarios, which limits its generality and persuasiveness. Also, the findings do not appear sufficiently novel or insightful to provide the community with a substantially new understanding.
-
The proposed methods are only weakly connected to the diagnosis, offer limited novelty, and yield modest improvements. The student-teacher support-union strategy is relatively straightforward, while the connection between Soft Clamp and the diagnosed probability-mass omission is not intuitive. In terms of performance, Soft Clamp provides only a limited improvement in overall decision accuracy and does not show a clear advantage over vanilla GKD or the support-union strategy.
Questions for authors
Please see the weakness section.
Response to Reviewer dXcv
W1. Scope, persuasive power, and conceptual novelty
The empirical scope is tool use, with the complete matched chain in Qwen and directional behavioral/support replication in Llama. Llama's native JSON results rule out dependence on Qwen's specific XML entry marker, but do not establish generalization to tasks without an identifiable behavioral entry.
We do not base novelty on the general observations that top-K can bias distillation, that tail information matters, or that teacher–student support compatibility matters. These have precedents in Sparse Logit Sampling, Don't Ignore the Tail, and Teachability-Aware OPD (SLS, TAD, TA-OPD).
The specific empirical contribution is the connection from an omitted correction to generation dynamics under the implemented objective. The response route usually removes direct supervision on a student-overestimated entry coordinate, while the tool route reinforces it. Crucially, restoring the first position changes local entry but mostly relocates calls: initial calls on should-respond examples fall from 13.42% to 3.07%, while delayed calls rise from 0.78% to 10.42%. Restoring all response positions yields a similar initial boundary but reduces delayed calls to 0.65% and full-generation over-calling to 3.73% (Table 15).
This temporal behavior is not implied by the zero-gradient observation alone. The matched non-tool controls further narrow the explanation: over-calling is 13.25% with the teacher-probability-matched placebo and 11.65% with gradient rerouting, versus 3.73% with exact all-position restoration (Table 16). The former does not match student probability or gradient strength; the latter matches the frozen forward loss and selected output-logit-gradient magnitude, not full parameter gradients, and has collateral degradation.
We offer this as a bounded behavioral attribution and a caution about local repair and retained-mass audits. It does not establish a universal cause of tool overuse or a general law of multi-teacher capability conflict.
W2. Connection of interventions to the diagnosis, method novelty, and modest gains
The interventions change different parts of the system. Support construction can restore omitted coordinates. Soft Clamp reshapes retained training signals and cannot restore those coordinates. Entry bias adjusts the deployed boundary without retraining. Their comparison asks how much of the observed behavior can be changed at each layer, with different access requirements and capability costs; it does not establish three interchangeable repairs of the same omission.
Teacher/student union is a familiar construction. TA-OPD already uses union support to measure local disagreement for position selection; our experiment uses it as the actual within-position distillation support. The set operation is not an algorithmic novelty claim. Its role here is to test a student-aware construction that does not hard-code the entry token and to measure the resulting behavioral point.
The Soft Clamp accuracy gain is also small: 88.6±0.3% for vanilla versus 88.7±0.7% for Soft Clamp. Its over-calling reduction, 14.2% to 9.0%, accompanies lower call recall, 91.5% to 86.5%. It retains the original teacher top-K interface, without student-selected teacher probes or a union prepass. Union has a stronger single-turn point, while Soft Clamp preserves more required calls and higher dialogue exact success in the multi-turn diagnostic (Table 6); both remain below vanilla on dialogue exact. These differences support a comparison of operating points and access requirements, rather than overall superiority for Soft Clamp.
Accordingly, the mechanism evidence rests on the frozen audit, replay, exact temporal-scope restoration, and matched non-tool controls. Loss shaping and decoding bias are comparators for the scope of behavioral calibration. Their limited gains do not establish a new general-purpose optimization method, and the current evidence does not establish a universally preferable deployment strategy.