- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies gradient interference in multi-teacher on-policy distillation (MOPD), showing that domain-routed supervision can still induce conflicting updates on shared student parameters and degrade capability in high-conflict domain combinations. It proposes ROAD MOPD, which uses non-routed teachers as same-trajectory critics for agreement-aware token reweighting and norm-constrained gradient projection, improving joint performance across math, code, and medical domains.
Critical strengths
- The paper identifies a concrete and practically relevant failure mode of domain-routed MOPD: routing supervision by domain does not prevent conflicting updates to shared student parameters.
- The proposed use of non-routed teachers as same-trajectory cross-domain critics is conceptually distinctive from both standard MOPD and conventional multi-task gradient surgery. And the two components of ROAD MOPD—agreement-aware token credit assignment and norm-constrained preventive gradient projection—are well aligned with the diagnosed token- and parameter-level interference.
- The empirical evaluation is broad and reasonably well controlled, covering high-conflict, low-conflict, multiple student sizes, and three-domain settings, together with relevant MOPD and PCGrad baselines. It also includes useful ablations and mechanistic probes, and provides detailed implementation and reproducibility information.
Critical weaknesses
W1. The central compatibility proxy is not sufficiently validated. The main conceptual concern is that ROAD MOPD relies on a strong but insufficiently validated assumption: a non-routed teacher’s preference or gradient on a target-domain trajectory is treated as a proxy for whether the corresponding update is compatible with the capability represented by that teacher.
W2. The evidence for update incompatibility as a general MOPD failure mode is somewhat narrow and potentially confounded. The paper motivates ROAD MOPD through a strong contrast between Math–Code and the two Medical-containing domain pairs. However, both high-conflict settings involve the Medical domain, while Math–Code exhibits very low conflict. This makes it difficult to determine whether the observed degradation is primarily caused by a general limitation of domain-routed MOPD or by properties specific to the Medical configuration.
W3. Several important design choices remain heuristic and are not sufficiently stress-tested The token-level mechanism uses
\tanh((\ell_t^r-\ell_t^{ref})/\tau) \tanh((\ell_t^a-\ell_t^{ref})/\tau)$$ followed by a linear mapping to $\lambda_t$, while the gradient-level method introduces a norm-retention floor $m$. These choices are intuitive, but the manuscript provides limited evidence that these particular formulations are important for capturing cross-domain compatibility rather than simply being reasonable design choices.Questions for authors
- What is the justification for treating a non-routed teacher’s preference on a target-domain trajectory as a proxy for preserving that teacher’s own-domain capability?
- Why is the same-trajectory auxiliary gradient a more appropriate compatibility signal than the auxiliary teacher’s gradient on its own-domain trajectories?
- Could the authors clarify the rationale behind the specific token-level agreement function in Eq. (4) and its mapping to the reward coefficient in Eq. (5)?
- How sensitive are the results to the particular definition of , the mapping from to , and the retention threshold ? Have the authors considered simpler alternatives such as sign agreement, raw reference-relative log-probability products, teacher-distribution divergence, or several values of ?