- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces LoopDistill, a student-aware distillation framework for looped language models. It generates student reasoning trajectories at selected recurrent depths and asks a teacher to correct them while preserving aspects of the student's reasoning behavior. A verification procedure checks solution quality and compatibility in trajectory length and exploratory structure. Accepted rewrites supervise the corresponding recurrent readouts. Experiments use Ouro-2.6B-Base, two teachers, and four mathematical reasoning benchmarks. LoopDistill consistently outperforms KD and OPD, and achieves the best results in most experimental settings.
Critical strengths
LoopDistill uses distinct reasoning trajectories to supervise different recurrent depths and incorporates teacher-guided rewriting to adapt these trajectories to the student’s behavior. This provides a well-motivated approach to distillation for looped language models.
Critical weaknesses
The paper does not quantify computational cost or data efficiency, making the practical trade-offs difficult to assess. Experiments cover only one student architecture, leaving the method’s generalizability to other looped language models unclear.
Questions for authors
- Could you compare the computational costs of LoopDistill, TRD, OPD, and KD, including both training-data generation and student training? LoopDistill appears to require additional computation for depth-specific rollouts and trajectory rewriting.
- Why were the third and fourth loops selected for supervision? How would performance change if all four loops were supervised?
4 comments
Reply to Reviewer 5AEg
2. Generalizability to Other LoopLMs. To address the concern about generalizability, we conduct additional experiments on another LoopLM architecture and clarify our initial model selection.
- Additional experiments on Nanbeige4.2-3B-Base. We extend our evaluation to Nanbeige4.2-3B-Base, another publicly available pretrained LoopLM. LoopDistill consistently outperforms conventional KD across all four benchmarks, providing further evidence of its generalizability.
Additional results on Nanbeige4.2-3B-Base (student) with Qwen3-30B-A3B (teacher). All values are reported as accuracy (%).
| Method | AIME24 | AIME25 | AIME26 | OlympiadBench |
|---|---|---|---|---|
| Base (no distillation) | 15.6 | 10.0 | 11.1 | 42.6 |
| KD | 53.3 | 41.1 | 42.2 | 63.2 |
| LoopDistill (Ours) | 55.6 | 42.2 | 45.6 | 64.0 |
-
LoopCoder-v2 (no pretrained base checkpoint). LoopCoder-v2 only releases a post-trained checkpoint without a publicly available pretrained base checkpoint, making it unsuitable for our evaluation setting.
-
Huginn-3.5B (poor base-model performance at small loop counts; high computational cost at large loop counts). Our preliminary evaluation of the pretrained Huginn-3.5B base model (without distillation) yielded 0% AIME accuracy at 1, 2, 4, and 8 loops. Its default configuration uses 32 loops, requiring 132 Transformer-layer executions per forward pass, with a computational cost roughly equivalent to a 50B-parameter dense model. This compute-equivalent model size exceeds that of both teacher models used in our experiments (Qwen3-30B-A3B and gpt-oss-20B). Training at this depth would therefore be prohibitively expensive within the rebuttal period.
These considerations motivated our initial selection of Ouro-2.6B-Base, while the additional Nanbeige4.2-3B-Base results demonstrate the effectiveness of LoopDistill on another LoopLM architecture.
Q2. Why were the third and fourth loops selected for supervision? How would performance change if all four loops were supervised?
Answer. We thank the reviewer for this question. We selected loops 3 and 4 based on a preliminary analysis of gradient cosine similarity between different recurrent depths. Using loop 4 as the reference, we observed that:
- Loop 3: Similarity decreases from approximately 0.7 to 0.2 during training.
- Loop 2: Similarity decreases from approximately 0.6 to 0.
- Loop 1: Similarity decreases from approximately 0.3 to −0.1.
These observations suggest that loop 3 has the most aligned optimization direction with the final loop, whereas earlier loops exhibit weaker alignment and potentially conflicting gradients. We therefore selected loops 3 and 4 to provide complementary supervision while minimizing gradient interference.
To further validate this choice, we conduct an ablation study with different combinations of supervised recurrent depths.
Ablation of supervised recurrent depths with Ouro-2.6B-Base as the student and Qwen3-30B-A3B as the teacher. All values are reported as accuracy (%).
| Supervised Loops | AIME24 | AIME25 | AIME26 | OlympiadBench |
|---|---|---|---|---|
| 3+4 (Ours) | 56.7 | 47.8 | 58.9 | 66.5 |
| 2+4 | 55.6 | 47.8 | 54.4 | 66.0 |
| 1+4 | 54.4 | 44.4 | 55.6 | 66.0 |
| 1+2+4 | 52.2 | 42.2 | 48.9 | 65.4 |
| 1+3+4 | 52.2 | 44.4 | 51.1 | 65.6 |
| 2+3+4 | 46.7 | 44.4 | 55.6 | 65.3 |
| 1+2+3+4 | 42.2 | 47.8 | 47.8 | 64.4 |
The results confirm that supervising loops 3 and 4 achieves the best performance across all four benchmarks. In contrast, supervising all four loops reduces average AIME accuracy from 54.5% to 45.9%, a decrease of 8.5 percentage points. This is consistent with our gradient similarity analysis, suggesting that supervision at earlier depths may introduce gradient interference and compromise final-loop optimization.
A Kind Reminder for the Discussion Phase
Dear Reviewer 5AEg,
We sincerely appreciate the thoughtful comments and constructive feedback from you. We hope our responses have adequately addressed your questions and concerns. As the rebuttal phase is nearing its end, we look forward to your feedback on our responses and would be happy to clarify any additional questions you may have.
Thank you for your time and consideration.
Thanks for your comment. I will keep my score.
Reply to Reviewer 5AEg
We thank the reviewer for recognizing the well-motivated design of LoopDistill, particularly its use of depth-specific reasoning trajectories and teacher-guided rewriting to adapt supervision to the student's behavior. We appreciate the reviewer's constructive feedback and have conducted additional experiments and analyses to address the concerns regarding computational efficiency, generalizability across LoopLM architectures, and the selection of supervised recurrent depths, as detailed below.
W1 & Q1. Computational Cost, Data Efficiency, and Generalizability
The reviewer raises concerns about the computational cost and data efficiency of LoopDistill, particularly the additional overhead from depth-specific student rollouts and teacher trajectory rewriting. The reviewer requests a comparison of the computational costs of LoopDistill, TRD, OPD, and KD, covering both data preparation and student training. Additionally, the evaluation is limited to a single student architecture, leaving generalizability to other LoopLMs unclear.
Answer. We thank the reviewer for raising these important concerns. We acknowledge that LoopDistill requires additional computation for student rollouts, teacher rewriting, and verification. Importantly, this additional computational investment translates into substantial performance improvements. However, simply increasing the computational budget of conventional KD does not yield comparable gains. Specifically, KD with three teacher-generated trajectories per problem uses more generated tokens than LoopDistill (214.2M vs. 194.8M), yet achieves substantially lower performance (38.9% vs. 54.5% average AIME accuracy). This suggests that our gains arise from more effective distillation rather than increased computation alone.
We provide detailed computational cost analyses and additional experiments on another LoopLM architecture below.
1. Computational Cost and Efficiency. LoopDistill's additional data-preparation cost is incurred only once offline, rather than repeatedly during training. We compare token consumption and GPU hours for KD, TRD, OPD, and LoopDistill over three training epochs.
Token consumption comparison with Qwen3-30B-A3B as the teacher. All values are reported in millions of tokens (M).
Computational cost comparison with Qwen3-30B-A3B as the teacher. All values are reported in GPU hours.
Performance comparison with Qwen3-30B-A3B as the teacher. All benchmark values are reported as accuracy (%).
Despite tripling the teacher generation budget, KD improves average AIME accuracy by only 2.2 percentage points (36.7% to 38.9%), remaining 15.6 points below LoopDistill (54.5%). These results suggest that increased teacher-generated supervision alone cannot explain LoopDistill's improvements.
Overall, LoopDistill requires more computation than KD and TRD but delivers substantially stronger performance. Compared with OPD, it achieves better performance while using 47.6% fewer GPU hours (82.7 vs. 157.8). This demonstrates that the additional offline computation provides meaningful performance benefits rather than merely increasing the computational budget.