- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Success-Conditioned Self-Distillation (SCSD) for continual learning. It searches for rewriting rules that turn successful model-generated solutions into teacher prompts, aiming to preserve teacher success while limiting drift from a reference model. The resulting teacher provides token-level supervision on student trajectories. The approach is motivated by a characterization of the minimum-KL successful policy and evaluated across mathematics, science, and tool use.
Critical strengths
- The paper highlights how successful trajectories can contain incidental details that induce unnecessary behavioral changes. Optimizing the information provided to the teacher is a meaningful extension to existing self-distillation methods.
- The characterization of the success-conditioned distribution provides a clear rationale for balancing task success with limited distributional drift.
- SCSD shows a good empirical results, which substantially mitigates the mathematical degradation observed with the evaluated SDPO baseline and improves several final-task scores in sequential learning.
Critical weaknesses
- Comparisons against fixed compression, length-matched summaries, and search without the KL objective would help establish whether the gains come from simpler teacher context or the proposed search procedure.
- It would be better to conduct one more experiments on continual learning.
Questions for authors
- How much of the improvement comes from the proposed optimization beyond simply compressing the teacher context? Could the authors compare against the fixed initial rewriting instruction and search without the KL objective, while keeping other settings, including teacher feedback, identical?
We thank the reviewer for recognizing that optimizing the information given to the teacher is a meaningful extension of self-distillation, and that the drift floor gives a clear rationale for balancing success against drift. We address the concerns below.
W1 & Q1. How much of the improvement comes from the search beyond compressing the teacher's context? This is exactly the right question, and we answer it end to end in G1, where all runs share the teacher, the wrapper and every other setting. More specifically, we compare SCSD against (i) the fixed seed instruction g0 without search, (ii) a length-matched generic summary, (iii) the raw success, and (iv) a search with the reward-only objective. Compression alone does not help: g0 and the summary stay at the accuracy and drift of the raw teacher on math. The reward-only search raises accuracy but has the highest drift of all arms. SCSD improves over g0 by +0.168 / +0.221 / +0.059 on math / science / tool use. G1 shows that the gain carries over to the trained student, and that it comes from the minimum-drift objective rather than from a shorter context.
W2. One more continual-learning experiment. We add three seeds, two task orders, a second model (Qwen3-4B) and three continual-learning comparators (G2). Across the three orders, SCSD forgets less than SDPO on 8 of the 9 (order, task) pairs, and on Qwen3-4B it has the largest on-task gain at every stage and ends above both GRPO and SDPO on every task.
We hope these new results address the reviewer's concerns. If so, we would greatly appreciate it if the reviewer could consider raising the score. We are happy to answer any further questions.