- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TransCAP, a framework for on-policy self-distillation (OPSD) that jointly constructs supervision targets and allocates token-level training weights. The method uses a task-conditioned residual to refine privileged teacher supervision and determine initial token priorities. A constrained KL projection then adjusts these weights to limit directional conflict with the task-only anchor and prevent excessive concentration. Experiments across three Qwen model scales demonstrate improvements over existing OPSD baselines on mathematical reasoning, code generation, and function calling benchmarks.
Critical strengths
The paper addresses an interesting and important problem of jointly determining what knowledge to transfer and where to allocate supervision in OPSD.
TransCAP presents a technically interesting framework that combines task-conditioned guidance construction with constrained supervision allocation.
Experiments across multiple model scales and benchmarks demonstrate promising improvements over several recent OPSD methods.
Extensive ablation studies provide useful evidence for the contributions of guidance-based weighting, conflict control, and concentration constraints.
Critical weaknesses
The proposed guidance score is based on the magnitude of task-conditioned distributional differences, but larger differences do not necessarily indicate more transferable knowledge or greater learning benefit. The paper lacks sufficient evidence that this score reliably identifies the most informative token positions.
The directional conflict metric is defined in the output distribution space rather than the model's parameter-gradient space. Reducing this local conflict does not necessarily prevent interference with previously learned capabilities. The connection between the proposed geometric constraints and actual capability preservation remains insufficiently established.
The target construction largely builds upon existing privileged-supervision refinement methods, while the main contribution lies in introducing additional weighting and projection mechanisms. Although the empirical improvements are promising, it remains unclear whether these components provide a sufficiently generalizable insight beyond more sophisticated supervision reweighting.
TransCAP requires evaluating multiple conditional distributions and solving a constrained projection problem during training. While the paper reports projection latency and training-stage timings, a comprehensive end-to-end computational comparison with competing OPSD methods is missing. Additionally, some improvements, particularly on Qwen3-4B, are relatively small and lack uncertainty estimates across independent training runs.
Questions for authors
How can the authors demonstrate that the proposed residual magnitude measures transferable knowledge rather than merely differences in model predictions or conditioning preferences?
To what extent does reducing distribution-space directional conflict translate into better preservation of existing capabilities? Can the authors provide more direct evidence of reduced interference or forgetting?
7 comments
Additional Experiments for General-Capability Evaluation
Thank you for recognizing the motivation behind jointly designing supervision targets and supervision allocation, and for your constructive feedback. We have added general-capability evaluations, an end-to-end training-step timing comparison, and independent training runs. We first present these additional experiments, then address your questions about the guidance score, directional conflict, and methodological contribution.
To examine performance beyond the original mathematics and code benchmarks, we conducted mixed-domain training using mathematical, coding, and scientific examples from OpenThoughts. For each of the 4B and 9B models, we performed five independent TransCAP training runs and evaluated the resulting models on MMLU, GPQA Main, IFEval, and IFBench. We also evaluated the original model under the same protocol as the Raw control.
| Model | Method | MMLU | GPQA | IFEval | IFBench |
|---|---|---|---|---|---|
| Qwen3-4B | Raw | 77.13 | 55.90 | 81.33 | 25.00 |
| Qwen3-4B | TransCAP | 79.12 | 60.20 | 85.66 | 42.13 |
| Qwen3.5-9B | Raw | 84.60 | 77.78 | 89.28 | 33.80 |
| Qwen3.5-9B | TransCAP | 90.12 | 83.48 | 90.09 | 45.07 |
All models in this evaluation use the complete benchmark sets, Avg@12, and the main mathematical decoding settings. TransCAP scores are means across five independent training runs. MMLU and GPQA Main use answer accuracy, IFEval uses strict prompt-level accuracy, and IFBench uses loose prompt-level accuracy.
In this mixed-domain setting, the TransCAP means exceed the Raw scores on all four benchmarks at both scales. These results extend the evaluation to general knowledge, scientific reasoning, and instruction following, providing a direct comparison with the original models after broader-domain training. The results and evaluation settings are included in Table 2 and Appendix F.3. We discuss the specific contribution of conflict control separately in our response to your methodological questions.
Additional Experiments for W4: End-to-End Training Cost
Thank you for pointing out that projection latency alone does not capture the additional training cost. We have supplemented it with an end-to-end training-step timing comparison between OPSD and TransCAP.
For 9B mathematical training, we used 32 NVIDIA H20 GPUs with matched hardware and training settings. After warm-up, we timed 100 complete training steps for each method. Each measured step includes response generation, the required reference-model computations, student updates, and synchronization. Evaluation and checkpoint saving were excluded consistently for both methods.
| Method | Average end-to-end time per training step |
|---|---|
| OPSD | 1.00× |
| TransCAP | approximately 1.18× |
TransCAP therefore takes approximately 18% more time per training step in this setting. This measurement includes the reference-model computations and the projection, rather than measuring the projection module alone.
For context, at the same 200 training steps in the main experiments, the 9B mathematical average increases from 62.31 for OPSD to 80.65 for TransCAP, a gain of 18.34 percentage points. Reporting these results alongside the timing comparison makes the performance–cost trade-off explicit.
Target construction and allocation are used only during training. At inference, TransCAP uses the student model alone, without an additional teacher or projection module.
Additional Experiments for Main Concern 2: Do Higher-Residual Positions Provide More Useful Supervision?
Thank you for distinguishing two questions. The permutation experiment shows that assigning residual-derived weights to their original positions improves performance. Your question goes further: given the same supervision budget, does a student learn more from high-residual positions, as measured by performance on unseen problems? We added a direct comparison on mathematics and code.
1. Learning from high- versus low-residual positions
For each domain, we trained two students from the same 9B checkpoint on the same training trajectories. Within each training example and relative-position bin, we ranked positions by residual magnitude and selected equal numbers from the high and low ends. Both students saw the full response context and used the same target-construction rule. Each student received supervision only at its selected positions, with equal weight per selected token. We matched the number of supervised tokens and training steps and disabled projection in both conditions.
Both groups improved over the starting model, so low-residual positions also provided useful supervision. On AIME 2025, the high-residual student produced 288 correct responses out of 360, compared with 271 out of 360 for the low-residual student. On LiveCodeBench v6, the high-residual student gained 5.73 percentage points over the starting model, while the low-residual student gained 2.93 points. In both domains, spending the same supervision budget on high-residual positions yielded a larger gain on unseen problems.
2. Residual-based allocation in the full method
The permutation experiment addresses the complementary question of how to distribute supervision when all positions contribute to training. We kept target construction and the full projection rules fixed and varied the initial allocation.
We divided each response into eight relative-position bins and shuffled the initial weights within each bin. This preserved the weight values and the total initial weight in each region. Restoring the original assignment improved AIME 2025 by 2.50 percentage points and LiveCodeBench v6 by 3.59 points. Thus, even with the same total initial weight in each region, the choice of which tokens received greater weight affected performance.
Interpretation. At a fixed student-generated prefix, we compare the frozen model’s next-token predictions under two conditions: task plus privileged information, and privileged information alone. The signed, centered log-probability difference adjusts the supervision target. Its mean absolute magnitude across the vocabulary, gt, measures how strongly adding the task changes the model’s relative preferences among candidate tokens. The new experiment connects that quantity to learning benefit: on both benchmarks, the positions selected for their higher residuals provided more useful supervision on average, as measured by the student’s improvement on unseen problems. This supports using gt to set initial supervision priorities.