- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Mixed-Policy Distillation (MPD) to shorten small-model reasoning. A larger teacher compresses student-generated reasoning traces, and the student learns through token-level distribution matching on the rewritten traces. The approach aims to combine student-derived reasoning paths with teacher-guided concision, without ground-truth supervision or length penalties. Experiments report reduced token usage while maintaining or improving Pass@4 across mathematical, code, and long-context reasoning tasks.
Critical strengths
1.The idea is simple and useful: improve reasoning efficiency using supervision tailored to the student’s actual attempts.
- Results show substantial token savings across several tasks and model families. In the main experiment, average tokens fall from 8.2k to 6.1k, while Pass@4 rises from 68.2 to 71.3.
Critical weaknesses
-
The benefit of teacher compression is insufficiently explained. On-Policy (C) already compresses student-generated trajectories, and both methods use the same teacher for distribution matching. Although MPD produces shorter outputs, the paper does not directly compare the compressors’ preservation of essential reasoning steps. Table 2 shows nearly identical semantic similarity, while Appendix C audits only teacher rewrites. A comparison at matched compression lengths would clarify whether the advantage comes from better reasoning-aware selection or simply stronger compression.
-
Limited contribution. While I do agree that the compressing is satisfactory compared to the baseline methods, it appears to be an incremental extension of on-policy distillation, with prompt-based teacher compression as the main addition. The paper provides insufficient insight into why this modification works. Beyond Weakness 1, the large gap between SFT and MPD deserves further explanation. For example, would SFT trained on student-compressed trajectories exhibit the same performance collapse? This comparison could help clarify whether the failure stems from the compression targets or the training objective.
Questions for authors
see weakness
3 comments
Weakness 2:
Limited contribution. While I do agree that the compressing is satisfactory compared to the baseline methods, it appears to be an incremental extension of on-policy distillation, with prompt-based teacher compression as the main addition. The paper provides insufficient insight into why this modification works. Beyond Weakness 1, the large gap between SFT and MPD deserves further explanation. For example, would SFT trained on student-compressed trajectories exhibit the same performance collapse? This comparison could help clarify whether the failure stems from the compression targets or the training objective.
Reply to Weakness 2:
Thank you for your constructive comments. We address these concerns as follows.
1. Novelty and motivation. The key contribution of MPD is to combine teacher-guided trajectory compression while preserving student’s reasoning strategy in a unified distillation framework. Our ablation experiments suggest that neither keeping student reasoning nor trajectory compression alone fully explains MPD's improvements; their combination provides a more favorable accuracy–efficiency trade-off.
2. Performance gap between SFT and MPD. We agree that this comparison deserves further clarification. Appendix B.2 (Table 12) reports SFT performance across training steps. Notably, at comparable output lengths, SFT at step 50 achieves an average Pass@4 of 65.9 with 6.0k tokens, whereas MPD achieves 71.3 with 6.1k tokens. This indicates that MPD's advantage is not merely due to differences in output length or the performance collapse of SFT at later training steps.
3. Additional SFT experiment. Following your suggestion, we conduct an additional experiment that applies SFT to student-compressed trajectories under the same training setup. The result shows that training with trajectories compressed by the student model, the SFT also shows reduction on both token length and performance and degradation with further training steps. We expect this controlled comparison providing further insight into the performance degradation that arises primarily from the SFT objective.
| Method | AIME24 P@4 | AIME24 #Tok | AIME25 P@4 | AIME25 #Tok | GSM8K P@4 | GSM8K #Tok | CodeMMLU P@4 | CodeMMLU #Tok | LongBench2 P@4 | LongBench2 #Tok | Avg. P@4 | Avg. #Tok |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-1.7B (Vanilla) | 63.3 | 16.8k | 53.3 | 17.7k | 94.5 | 2.1k | 81.2 | 3.6k | 48.7 | 0.9k | 68.6 | 8.2k |
| SFT step 50 | 66.7 | 12.7k | 40.0 | 9.6k | 92.6 | 0.8k | 82.7 | 2.2k | 46.7 | 0.7k | 65.7 | 5.2k |
| SFT step 100 | 10.0 | 0.6k | 0.0 | 0.8k | 88.1 | 0.2k | 69.1 | 0.6k | 46.5 | 0.6k | 44.7 | 0.5k |
| SFT step 150 | 3.3 | 0.5k | 10.0 | 0.4k | 88.0 | 0.2k | 66.7 | 0.6k | 45.9 | 0.5k | 41.4 | 0.4k |
| SFT step 200 | 6.7 | 0.3k | 3.0 | 0.3k | 84.2 | 0.2k | 67.8 | 0.6k | 47.9 | 0.4k | 41.3 | 0.4k |
| MPD (Ours) | 73.3 | 13.4k | 56.7 | 12.9k | 94.2 | 1.4k | 82.8 | 2.3k | 50.3 | 0.7k | 71.3 | 6.1k |
Thank you for your response. I will keep my score.
Weakness 1:
Reply to Weakness 1: We thank the reviewer for this insightful comment. We agree that semantic similarity does not necessarily imply preservation of essential reasoning steps.
First, we conducted an additional LLM-as-Judge evaluation using the same evaluation protocol (gemini-3.1-flash-lite) as Appendix C to directly compare student and teacher compression. We further report the compression quality using teacher model or student model to compress the trajectories, as shown in the Table below. The results show that teacher compression achieves 93% faithful trajectories, compared to 87% for student compression, with higher scores in key dimensions such as constraint preservation (4.92 Student-Teacher vs. 4.77 Student-Student) and Equation Preservation (4.92 vs. 4.78). Further, we also compare the compression rate. The teacher as compressor achieves these results despite a higher compression rate (93% vs. 88%). This provides more direct evidence that teacher compression better preserves essential reasoning steps while removing more redundancy.
Second, regarding matched-length comparison, we found that compression rates are primarily determined by the compressor model rather than the prompt. We experimented with different compression instructions, but observed only minor variations in compression rates. Matching compression lengths would therefore require explicit length constraints, potentially changing the natural compression behavior that our method aims to exploit.
Together with the downstream improvements over On-Policy (C) in Table 5, these findings support the benefit of stronger teacher-guided compression beyond simply producing shorter trajectories.