acceptodds
Under review as a conference paper at ICLR 2027

LoopDistill: Student-Aware Knowledge Distillation for Looped Language Models

Abstract

Looped language models (LoopLMs) increase effective depth by repeatedly applying shared Transformer blocks, offering a parameter-efficient approach to reasoning. Knowledge distillation typically transfers reasoning by training a smaller model on chain-of-thought trajectories from a stronger teacher. For a LoopLM, however, readouts at different recurrent depths can generate reasoning trajectories with different lengths and structures. A single teacher trajectory may therefore provide uneven supervision across loops, while training only the final readout leaves intermediate loops without direct reasoning targets. We introduce LoopDistill, a student-aware distillation framework that uses trajectories generated at selected depths to construct loop-specific teacher supervision. Each selected loop is trained on a target adapted to its own reasoning behavior. Using Ouro-2.6B-Base and two teachers, we evaluate LoopDistill on AIME24, AIME25, AIME26, and OlympiadBench. With Qwen3-30B-A3B as the teacher, LoopDistill outperforms the competing baseline on every benchmark, by up to 18.9 percentage points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Looped Language Models, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5AEgOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LoopDistill, a student-aware distillation framework for looped language models. It generates student reasoning trajectories at selected recurrent depths and asks a teacher to correct them while preserving aspects of the student's reasoning behavior. A verification procedure checks solution quality and compatibility in trajectory length and exploratory structure. Accepted rewrites supervise the corresponding recurrent readouts. Experiments use Ouro-2.6B-Base, two teachers, and four mathematical reasoning benchmarks. LoopDistill consistently outperforms KD and OPD, and achieves the best results in most experimental settings.

Critical strengths

LoopDistill uses distinct reasoning trajectories to supervise different recurrent depths and incorporates teacher-guided rewriting to adapt these trajectories to the student’s behavior. This provides a well-motivated approach to distillation for looped language models.

Critical weaknesses

The paper does not quantify computational cost or data efficiency, making the practical trade-offs difficult to assess. Experiments cover only one student architecture, leaving the method’s generalizability to other looped language models unclear.

Questions for authors
  1. Could you compare the computational costs of LoopDistill, TRD, OPD, and KD, including both training-data generation and student training? LoopDistill appears to require additional computation for depth-specific rollouts and trajectory rewriting.
  2. Why were the third and fourth loops selected for supervision? How would performance change if all four loops were supervised?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer 5AEg

We thank the reviewer for recognizing the well-motivated design of LoopDistill, particularly its use of depth-specific reasoning trajectories and teacher-guided rewriting to adapt supervision to the student's behavior. We appreciate the reviewer's constructive feedback and have conducted additional experiments and analyses to address the concerns regarding computational efficiency, generalizability across LoopLM architectures, and the selection of supervised recurrent depths, as detailed below.

W1 & Q1. Computational Cost, Data Efficiency, and Generalizability

The reviewer raises concerns about the computational cost and data efficiency of LoopDistill, particularly the additional overhead from depth-specific student rollouts and teacher trajectory rewriting. The reviewer requests a comparison of the computational costs of LoopDistill, TRD, OPD, and KD, covering both data preparation and student training. Additionally, the evaluation is limited to a single student architecture, leaving generalizability to other LoopLMs unclear.

Answer. We thank the reviewer for raising these important concerns. We acknowledge that LoopDistill requires additional computation for student rollouts, teacher rewriting, and verification. Importantly, this additional computational investment translates into substantial performance improvements. However, simply increasing the computational budget of conventional KD does not yield comparable gains. Specifically, KD with three teacher-generated trajectories per problem uses more generated tokens than LoopDistill (214.2M vs. 194.8M), yet achieves substantially lower performance (38.9% vs. 54.5% average AIME accuracy). This suggests that our gains arise from more effective distillation rather than increased computation alone.

We provide detailed computational cost analyses and additional experiments on another LoopLM architecture below.

1. Computational Cost and Efficiency. LoopDistill's additional data-preparation cost is incurred only once offline, rather than repeatedly during training. We compare token consumption and GPU hours for KD, TRD, OPD, and LoopDistill over three training epochs.

Token consumption comparison with Qwen3-30B-A3B as the teacher. All values are reported in millions of tokens (M).

KDOPDTRDLoopDistill
Data preparation (once)
Student rollouts (generated)––61.4M125.0M
Teacher generation (generated)71.4M–39.8M69.8M
Student training
Student rollouts / epoch (generated)–59.6M––
Teacher scoring / epoch (read)–59.6M––
Training tokens / epoch39.3M59.6M21.6M57.5M
Totals
Tokens generated71.4M178.8M101.2M194.8M
Tokens trained (3 epochs)117.9M178.8M64.8M172.4M

Computational cost comparison with Qwen3-30B-A3B as the teacher. All values are reported in GPU hours.

KDOPDTRDLoopDistill
Data preparation (once)
Student rollouts––26.146.5
Teacher generation9.3–10.710.2
Total data preparation9.3–36.856.7
Student training
Student rollouts / epoch–38.3––
Teacher scoring / epoch–4.3––
Gradient updates / epoch5.310.02.98.7
Training / epoch5.352.62.98.7
Training (3 epochs)15.9157.88.726.0
Total GPU hours25.2157.845.582.7

Performance comparison with Qwen3-30B-A3B as the teacher. All benchmark values are reported as accuracy (%).

MethodGenerated TokensAIME24AIME25AIME26OlyLCBv5
Ouro-2.6B Base–33.030.030.053.140.0
KD (1 seed)71.4M36.734.438.961.168.5
KD (3 seeds)214.2M41.137.837.860.9–
TRD101.2M46.736.740.059.768.3
OPD178.8M48.941.132.256.419.4
LoopDistill (Ours)194.8M56.747.858.966.571.5

Despite tripling the teacher generation budget, KD improves average AIME accuracy by only 2.2 percentage points (36.7% to 38.9%), remaining 15.6 points below LoopDistill (54.5%). These results suggest that increased teacher-generated supervision alone cannot explain LoopDistill's improvements.

Overall, LoopDistill requires more computation than KD and TRD but delivers substantially stronger performance. Compared with OPD, it achieves better performance while using 47.6% fewer GPU hours (82.7 vs. 157.8). This demonstrates that the additional offline computation provides meaningful performance benefits rather than merely increasing the computational budget.

AuthorsOpenReview ↗

Reply to Reviewer 5AEg

2. Generalizability to Other LoopLMs. To address the concern about generalizability, we conduct additional experiments on another LoopLM architecture and clarify our initial model selection.

  • Additional experiments on Nanbeige4.2-3B-Base. We extend our evaluation to Nanbeige4.2-3B-Base, another publicly available pretrained LoopLM. LoopDistill consistently outperforms conventional KD across all four benchmarks, providing further evidence of its generalizability.

Additional results on Nanbeige4.2-3B-Base (student) with Qwen3-30B-A3B (teacher). All values are reported as accuracy (%).

MethodAIME24AIME25AIME26OlympiadBench
Base (no distillation)15.610.011.142.6
KD53.341.142.263.2
LoopDistill (Ours)55.642.245.664.0
  • LoopCoder-v2 (no pretrained base checkpoint). LoopCoder-v2 only releases a post-trained checkpoint without a publicly available pretrained base checkpoint, making it unsuitable for our evaluation setting.

  • Huginn-3.5B (poor base-model performance at small loop counts; high computational cost at large loop counts). Our preliminary evaluation of the pretrained Huginn-3.5B base model (without distillation) yielded 0% AIME accuracy at 1, 2, 4, and 8 loops. Its default configuration uses 32 loops, requiring 132 Transformer-layer executions per forward pass, with a computational cost roughly equivalent to a 50B-parameter dense model. This compute-equivalent model size exceeds that of both teacher models used in our experiments (Qwen3-30B-A3B and gpt-oss-20B). Training at this depth would therefore be prohibitively expensive within the rebuttal period.

These considerations motivated our initial selection of Ouro-2.6B-Base, while the additional Nanbeige4.2-3B-Base results demonstrate the effectiveness of LoopDistill on another LoopLM architecture.

Q2. Why were the third and fourth loops selected for supervision? How would performance change if all four loops were supervised?

Answer. We thank the reviewer for this question. We selected loops 3 and 4 based on a preliminary analysis of gradient cosine similarity between different recurrent depths. Using loop 4 as the reference, we observed that:

  • Loop 3: Similarity decreases from approximately 0.7 to 0.2 during training.
  • Loop 2: Similarity decreases from approximately 0.6 to 0.
  • Loop 1: Similarity decreases from approximately 0.3 to −0.1.

These observations suggest that loop 3 has the most aligned optimization direction with the final loop, whereas earlier loops exhibit weaker alignment and potentially conflicting gradients. We therefore selected loops 3 and 4 to provide complementary supervision while minimizing gradient interference.

To further validate this choice, we conduct an ablation study with different combinations of supervised recurrent depths.

Ablation of supervised recurrent depths with Ouro-2.6B-Base as the student and Qwen3-30B-A3B as the teacher. All values are reported as accuracy (%).

Supervised LoopsAIME24AIME25AIME26OlympiadBench
3+4 (Ours)56.747.858.966.5
2+455.647.854.466.0
1+454.444.455.666.0
1+2+452.242.248.965.4
1+3+452.244.451.165.6
2+3+446.744.455.665.3
1+2+3+442.247.847.864.4

The results confirm that supervising loops 3 and 4 achieves the best performance across all four benchmarks. In contrast, supervising all four loops reduces average AIME accuracy from 54.5% to 45.9%, a decrease of 8.5 percentage points. This is consistent with our gradient similarity analysis, suggesting that supervision at earlier depths may introduce gradient interference and compromise final-loop optimization.

AuthorsOpenReview ↗

A Kind Reminder for the Discussion Phase

Dear Reviewer 5AEg,

We sincerely appreciate the thoughtful comments and constructive feedback from you. We hope our responses have adequately addressed your questions and concerns. As the rebuttal phase is nearing its end, we look forward to your feedback on our responses and would be happy to clarify any additional questions you may have.

Thank you for your time and consideration.

Thanks for your comment. I will keep my score.

Reviewer_YW1MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reasoning distillation into looped language models. LoopDistill first generates complete trajectories from the initial student at fixed recurrent depths. A teacher then rewrites each student trajectory using a correct solution as reference. Accepted rewrites pass final-answer and student-compatibility checks and supervise their corresponding recurrent readouts. After constructing the training data, the student is fine-tuned using a length-normalized, multi-depth cross-entropy objective. Experiments use Ouro-2.6B-Base, supervise loops 3 and 4, and evaluate at loop 4. Training problems come from DAPO-Math-8k-Stratified; teachers are Qwen3-30B-A3B and gpt-oss-20b. Evaluation covers three annual AIME benchmarks and OlympiadBench. LoopDistill outperforms direct KD in all eight teacher–benchmark settings. It outperforms all compared methods with the Qwen teacher, but does not consistently outperform TRD with the gpt-oss teacher. Ablations compare original versus rewritten targets and last-loop versus two-loop supervision.

Critical strengths
  1. The method delivers substantial and relatively consistent gains in the evaluated settings and mitigates negative transfer from direct distillation.
  2. The method combines student-aware target construction with multi-depth supervision into a practical training approach for existing looped models.
Critical weaknesses
  1. The experiments do not directly establish the necessity of depth-specific targets over shared rewritten targets, limiting evidence for the central incremental contribution. Table 2 supports rewriting and multi-loop supervision, but does not compare sharing one high-quality rewrite across both depths against assigning depth-specific rewrites, or test the target–depth assignment while holding the training material fixed. Evaluation also reports only loop-4 performance, without directly showing changes in loop-3 outputs. Consequently, the results establish that the complete pipeline works, but do not sufficiently determine whether its gains arise from recurrent-depth adaptation rather than better training text, greater target diversity, or auxiliary supervision. This materially affects the contribution beyond general trajectory rewriting and multi-depth supervision.
  2. Reporting of the core data-construction procedure is insufficient for reproducibility and interpretation of the comparisons. Section 3.4 specifies length- and branch-ratio acceptance conditions without reporting their actual thresholds. The rewrite-strategy selection and prefix-selection rules in Section 3.3 remain largely qualitative, while Appendix A supplies only prefix-continuation prompts. The manuscript also omits retained-data counts by teacher and depth and the corresponding data coverage of the controls. These choices determine what supervision the student actually receives, rather than constituting peripheral implementation details. Their omission makes the central pipeline difficult to reproduce and leaves uncertainty about the influence of filtering and differences in training-material quantity.
Questions for authors
  1. With comparable training problems, supervision counts, and token budgets, how does sharing the same verified student-aware rewrite across loops 3 and 4 compare with the proposed depth-specific targets? Alternatively, does swapping the two targets’ assigned depths affect performance when the rewrites and training budget are unchanged? Please provide any existing analysis.
  2. Please specify the length/branch-ratio thresholds, rewrite-strategy selection rules, prefix-selection procedure, and per-depth loss weights used in the experiments. Please also report accepted-rewrite counts by teacher and depth and clarify whether the Table 2 settings use the same underlying problem pool and comparable supervision budgets.
3 comments
AuthorsOpenReview ↗

Reply to Reviewer YW1M

We thank the reviewer for recognizing the substantial and consistent performance improvements of LoopDistill. We have conducted additional experiments to validate depth-specific supervision and provide implementation details to improve reproducibility.

W1 & Q1. Necessity of Depth-Specific Supervision

The reviewer questions whether depth-specific targets are necessary compared with sharing or swapping rewrites across loops 3 and 4 under comparable training budgets. Could the improvements instead arise from better trajectories, increased target diversity, or multi-loop supervision? Do the benefits extend to intermediate-loop outputs?

Answer. We conduct three additional experiments to address these concerns.

1. Ablation on depth-specific supervision. We compare LoopDistill against Shared, which applies the same rewrites to both depths, and Swap, which reverses the depth-specific target assignments. All variants use the same training problems, supervised depths, and training protocol.

Depth-specific supervision ablation with Qwen3-30B-A3B as the teacher. All values are accuracy (%).

MethodSupervision AssignmentAIME24AIME25AIME26OlympiadBench
LoopDistill (Ours)Loop 3 supervised by loop-3 rewrites; loop 4 supervised by loop-4 rewrites56.747.858.966.5
SharedLoops 3 and 4 supervised by loop-4 rewrites55.643.351.166.5
SwapLoop 3 supervised by loop-4 rewrites; loop 4 supervised by loop-3 rewrites50.043.357.866.5

Depth-specific supervision ablation with gpt-oss-20B as the teacher. All values are accuracy (%).

MethodSupervision AssignmentAIME24AIME25AIME26OlympiadBench
LoopDistill (Ours)Loop 3 supervised by loop-3 rewrites; loop 4 supervised by loop-4 rewrites40.031.146.757.2
SharedLoops 3 and 4 supervised by loop-4 rewrites35.627.837.854.6
SwapLoop 3 supervised by loop-4 rewrites; loop 4 supervised by loop-3 rewrites40.033.337.854.9

LoopDistill achieves the highest average AIME accuracy under both teachers. Since Shared retains multi-loop supervision and Swap retains the same rewritten targets but changes their assignment, these results support the importance of matching supervision trajectories to their corresponding recurrent depths.

2. Effect of target diversity. To test whether increased target diversity explains the gains, we strengthen KD by sampling three teacher trajectories per problem using different random seeds.

Target diversity comparison with Qwen3-30B-A3B as the teacher. All benchmark values are accuracy (%).

MethodTokens GeneratedOlympiadBenchAIME24AIME25AIME26
KD (1 seed)71.4M61.136.734.438.9
KD (3 seeds)214.2M60.941.137.837.8
LoopDistill (Ours)194.8M66.556.747.858.9

Target diversity comparison with gpt-oss-20B as the teacher. All values are accuracy (%).

MethodOlympiadBenchAIME24AIME25AIME26
KD (1 seed)48.920.023.325.6
KD (3 seeds)50.821.122.222.2
LoopDistill (Ours)57.240.031.146.7

Increasing target diversity yields only modest improvements for KD under both teachers. With Qwen3-30B-A3B, three-seed KD uses more generated tokens than LoopDistill (214.2M vs. 194.8M) but achieves substantially lower performance, suggesting that target diversity alone cannot explain our gains.

3. Intermediate-loop evaluation. We evaluate both loop-3 and loop-4 outputs using Qwen3-30B-A3B as the teacher.

Performance at different recurrent depths with Qwen3-30B-A3B as the teacher. All values are accuracy (%).

MethodAIME24AIME25AIME26OlympiadBench
KD (loop 3)36.736.734.462.5
KD (loop 4)36.734.438.961.1
LoopDistill (loop 3)47.842.242.263.5
LoopDistill (loop 4)56.747.858.966.5

LoopDistill improves loop-3 AIME accuracy by 8.1 percentage points on average over KD, confirming benefits beyond the final loop.

Overall, these experiments provide direct evidence for the benefits of depth-specific supervision beyond shared rewriting, increased target diversity, and multi-loop supervision.

AuthorsOpenReview ↗

Reply to Reviewer YW1M

W2 & Q2. Reproducibility of Data Construction and Training-Data Coverage

The reviewer raises concerns about the reproducibility of the data-construction pipeline, including the length/branch-ratio thresholds, rewrite-strategy and prefix-selection rules, per-depth loss weights, retained-data statistics, and differences in training-data coverage across baselines.

Answer. We thank the reviewer for highlighting the importance of reproducibility. We provide the detailed implementation settings, retained-data statistics, and explanations of training-data coverage below.

1. Data-construction details. We summarize the key implementation choices using Qwen3-30B-A3B as the teacher.

SettingImplementation
Student rollout1 draft per problem and depth, temperature 0.6, max 16K tokens
Prefix selectionFirst hesitation marker after 25% of the draft; otherwise, first paragraph break after midpoint
Teacher generationTemperature 0.6, max 24K tokens, up to 2 retries
Length-ratio threshold[0.7, 1.5]
Branch-ratio threshold[0.5, 2.0]
Acceptance criteriaCorrect and complete answer, valid final answer format, no editorial artifacts
FallbackVerified teacher trajectory if no valid continuation is obtained
Data filteringEnglish-only, max 16K tokens
Per-depth loss weightsEqual weights (1:1) for loops 3 and 4

For trajectory construction, we prioritize verified teacher continuations from the selected student prefix and use a verified teacher trajectory as fallback when no valid continuation is obtained. The selected student prefix is preserved and loss-masked during training. Invalid or incomplete trajectories are discarded, and overlength samples are removed rather than truncated.

2. Retained-data statistics. We report the number of accepted student-prefix continuations and teacher fallbacks at each recurrent depth using Qwen3-30B-A3B as the teacher.

DepthAccepted ContinuationsTeacher Fallbacks
12,8061,692
23,815814
34,116580
44,148528

3. Training-data coverage. All methods start from the same DAPO-Math-8k-Stratified dataset containing 6,971 problems. After removing non-English questions, 5,700 problems remain. Different methods then apply their respective filtering requirements:

  • KD: Retains problems for which the teacher generates a correct solution.
  • OPD: Does not require teacher-generated correct trajectories, as supervision is obtained through on-policy student rollouts and teacher scoring.
  • LoopDistill: Further filters the generated trajectories based on our correctness, length-ratio, and branch-ratio criteria.

We report the resulting training-data sizes below.

SettingKDOPDTRDLoopDistill
Original problem pool6,9716,9716,9716,971
After non-English filtering5,7005,7005,7005,700
Training problems4,9205,7004,9174,676
Training sequences / epoch4,9205,7004,9179,352

Although all methods start from the same problem pool, their retained training sets differ due to their respective filtering requirements. Notably, LoopDistill uses fewer distinct training problems than the baselines, while its larger number of training sequences results from supervising both loops 3 and 4 rather than using additional problems.

We will incorporate these implementation details and data statistics into the revised manuscript to improve reproducibility and clarify the differences in training-data coverage.

AuthorsOpenReview ↗

A Kind Reminder for the Discussion Phase

Dear Reviewer YW1M,

We sincerely appreciate the thoughtful comments and constructive feedback from you. We hope our responses have adequately addressed your questions and concerns. As the rebuttal phase is nearing its end, we look forward to your feedback on our responses and would be happy to clarify any additional questions you may have.

Thank you for your time and consideration.

Reviewer_TrSeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LoopDistill, a student-aware distillation framework for looped language models. It uses student rollouts at selected recurrent depths to guide teacher rewrites, verifies the resulting trajectories, and supervises each selected loop with its corresponding target. This paper evaluates LoopDistill on AIME24, AIME25, AIME26, and OlympiadBench, and the proposed method outperforms the competing baseline on every benchmark.

Critical strengths
  1. This paper proposes a LoopDistill framework to address the distillation problem for looped language models.

  2. The proposed method outperforms direct knowledge distillation in all benchmarks.

Critical weaknesses
  1. The student model only used Ouro-2.6B-Base; whether it can be effective on other LoopLMs has not been verified.

  2. The method proposed in the paper requires multiple rounds of student rollout, as well as teacher rewrite and check. Will this process be very time-consuming? The paper does not analyze the time complexity, nor does it conduct experimental analysis of runtime efficiency.

  3. The framework proposed in the paper is a general framework, but experiments were only conducted on mathematical reasoning datasets; whether it is effective on other tasks and datasets has not been verified.

Questions for authors

See Weaknesse

4 comments
AuthorsOpenReview ↗

Reply to Reviewer TrSe

We thank the reviewer for recognizing LoopDistill's consistent improvements over direct knowledge distillation across all evaluated benchmarks. We appreciate the reviewer's constructive feedback and have conducted additional experiments and analyses to address the concerns regarding generalizability across LoopLM architectures, computational efficiency, and effectiveness beyond mathematical reasoning, as detailed below.

Q1. The student model only used Ouro-2.6B-Base; whether it can be effective on other LoopLMs has not been verified.

Answer. We thank the reviewer for raising this important concern. We conduct additional experiments on another LoopLM architecture and clarify our initial model selection.

  • Additional experiments on Nanbeige4.2-3B-Base [1]. We extend our evaluation to Nanbeige4.2-3B-Base, another publicly available pretrained LoopLM. LoopDistill consistently outperforms conventional KD across all four benchmarks, providing further evidence of its generalizability.

Additional results on Nanbeige4.2-3B-Base (student) with Qwen3-30B-A3B (teacher). All values are reported as accuracy (%).

MethodAIME24AIME25AIME26OlympiadBench
Base (no distillation)15.610.011.142.6
KD53.341.142.263.2
LoopDistill (Ours)55.642.245.664.0
  • LoopCoder-v2 [2] (no pretrained base checkpoint). LoopCoder-v2 only releases a post-trained checkpoint without a publicly available pretrained base checkpoint, making it unsuitable for our evaluation setting.

  • Huginn-3.5B [3] (poor base-model performance at small loop counts; high computational cost at large loop counts). Our preliminary evaluation of the pretrained Huginn-3.5B base model (without distillation) yielded 0% AIME accuracy at 1, 2, 4, and 8 loops. Its default configuration uses 32 loops, requiring 132 Transformer-layer executions per forward pass, with a computational cost roughly equivalent to a 50B-parameter dense model. This compute-equivalent model size exceeds that of both teacher models used in our experiments (Qwen3-30B-A3B and gpt-oss-20B). Training at this depth would therefore be prohibitively expensive within the rebuttal period.

These considerations motivated our initial selection of Ouro-2.6B-Base, while the additional Nanbeige4.2-3B-Base results demonstrate the effectiveness of LoopDistill on another LoopLM architecture.

References

[1] Nanbeige4.2-3B: Unlocking Agentic Capabilities in a Compact Model. Nanbeige LLM Lab. arXiv:2607.22083.

[2] LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling. Yang et al. arXiv:2606.18023.

[3] Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach. Geiping et al. NeurIPS 2025.

AuthorsOpenReview ↗

Reply to Reviewer TrSe

Q2. The method proposed in the paper requires multiple rounds of student rollout, as well as teacher rewrite and check. Will this process be very time-consuming? The paper does not analyze the time complexity, nor does it conduct experimental analysis of runtime efficiency.

Answer. We thank the reviewer for raising this important concern. We acknowledge that LoopDistill requires additional computation for student rollouts, teacher rewriting, and verification. Importantly, this additional computational investment translates into substantial performance improvements. However, simply increasing the computational budget of conventional KD does not yield comparable gains. Specifically, KD with three teacher-generated trajectories per problem uses more generated tokens than LoopDistill (214.2M vs. 194.8M), yet achieves substantially lower performance (38.9% vs. 54.5% average AIME accuracy). This suggests that our gains arise from more effective distillation rather than increased computation alone.

Furthermore, LoopDistill's additional data-preparation cost is incurred only once offline, rather than repeatedly during training. To quantify this trade-off, we compare the token consumption, GPU hours, and downstream performance of KD, TRD, OPD, and LoopDistill.

1. Token consumption and GPU hours. We report the costs of both data preparation and student training over three epochs.

Token consumption comparison with Qwen3-30B-A3B as the teacher. All values are reported in millions of tokens (M).

KDOPDTRDLoopDistill
Data preparation (once)
Student rollouts (generated)––61.4M125.0M
Teacher generation (generated)71.4M–39.8M69.8M
Student training
Student rollouts / epoch (generated)–59.6M––
Teacher scoring / epoch (read)–59.6M––
Training tokens / epoch39.3M59.6M21.6M57.5M
Totals
Tokens generated71.4M178.8M101.2M194.8M
Tokens trained (3 epochs)117.9M178.8M64.8M172.4M

Computational cost comparison with Qwen3-30B-A3B as the teacher. All values are reported in GPU hours.

KDOPDTRDLoopDistill
Data preparation (once)
Student rollouts––26.146.5
Teacher generation9.3–10.710.2
Total data preparation9.3–36.856.7
Student training
Student rollouts / epoch–38.3––
Teacher scoring / epoch–4.3––
Gradient updates / epoch5.310.02.98.7
Training / epoch5.352.62.98.7
Training (3 epochs)15.9157.88.726.0
Total GPU hours25.2157.845.582.7

2. Performance versus computational cost. To further investigate whether increased computation alone explains the improvements, we evaluate KD with three teacher-generated trajectories per problem, increasing its generation budget from 71.4M to 214.2M tokens, exceeding LoopDistill's 194.8M tokens.

Performance comparison with Qwen3-30B-A3B as the teacher. All benchmark values are reported as accuracy (%).

MethodGenerated TokensAIME24AIME25AIME26OlyLCBv5
Ouro-2.6B Base–33.030.030.053.140.0
KD (1 seed)71.4M36.734.438.961.168.5
KD (3 seeds)214.2M41.137.837.860.9–
TRD101.2M46.736.740.059.768.3
OPD178.8M48.941.132.256.419.4
LoopDistill (Ours)194.8M56.747.858.966.571.5

Despite tripling the teacher generation budget, KD improves average AIME accuracy by only 2.2 percentage points (36.7% to 38.9%), remaining 15.6 points below LoopDistill (54.5%). This further demonstrates that increasing teacher-generated supervision alone cannot explain our performance gains.

Overall, LoopDistill requires more computation than KD and TRD but delivers substantially stronger performance. Compared with OPD, it achieves better performance while using 47.6% fewer GPU hours (82.7 vs. 157.8). These results demonstrate that the additional offline computation provides meaningful performance benefits rather than merely increasing the computational budget.

AuthorsOpenReview ↗

Reply to Reviewer TrSe

Q3. The framework proposed in the paper is a general framework, but experiments were only conducted on mathematical reasoning datasets; whether it is effective on other tasks and datasets has not been verified.

Answer. We thank the reviewer for raising this important concern. We agree that evaluating LoopDistill beyond mathematical reasoning is essential to demonstrate its general applicability. To address this concern, we extend our experiments to code generation, a distinct reasoning domain that requires both logical reasoning and precise program synthesis. Specifically, we train on KodCode (10K problems) and evaluate on LiveCodeBench v5 (880 problems) using pass@1 with a maximum generation budget of 32K tokens. We apply the same LoopDistill framework without task-specific methodological modifications and compare against conventional KD, TRD, and OPD under two different teacher models.

Code generation results on LiveCodeBench v5 with Ouro-2.6B-Base as the student. All values are reported as pass@1 accuracy (%).

MethodQwen3-30B-A3Bgpt-oss-20B
Ouro-2.6B Base40.040.0
KD68.535.3
TRD68.347.5
OPD19.424.9
LoopDistill (Ours)71.554.2
AuthorsOpenReview ↗

A Kind Reminder for the Discussion Phase

Dear Reviewer TrSe,

We sincerely appreciate the thoughtful comments and constructive feedback from you. We hope our responses have adequately addressed your questions and concerns. As the rebuttal phase is nearing its end, we look forward to your feedback on our responses and would be happy to clarify any additional questions you may have.

Thank you for your time and consideration.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

General Response

We sincerely thank all reviewers for their constructive feedback. We have conducted additional experiments and analyses to further strengthen our paper.

1. Computational Cost and Efficiency. We provide a detailed analysis of token consumption and GPU hours, covering both data preparation and training. We acknowledge that LoopDistill requires additional computation compared with KD and TRD, but this cost leads to substantial performance gains. To examine whether increased computation alone explains these gains, we evaluate KD with three teacher-generated trajectories per problem. Despite using more generated tokens, this stronger KD baseline improves average AIME accuracy by only 2.2 points, remaining substantially below LoopDistill.

2. Expanded Empirical Evaluation. We extend our evaluation to code generation on LiveCodeBench v5 [1]. Additionally, although not explicitly requested by reviewers, we include more baselines (RSR [2], SE-KD [3], AdaKD [4], and FiRe-OPD [5]) for a more comprehensive comparison. LoopDistill achieves the best average performance on both mathematical reasoning and code generation, further validating its effectiveness.

Expanded baseline comparison with Qwen3-30B-A3B as the teacher and Ouro-2.6B-Base as the student. All values are reported as accuracy (%).

MethodAIME24AIME25AIME26OlympiadBenchLiveCodeBenchv5
Ouro-2.6B Base33.030.030.053.140.0
KD36.734.438.961.168.5
TRD46.736.740.059.768.3
RSR36.736.736.757.669.2
SE-KD32.233.328.957.468.6
AdaKD33.333.335.657.168.3
OPD48.941.132.256.419.4
FiRe-OPD51.143.345.664.714.0
LoopDistill (Ours)56.747.858.966.571.5

Expanded baseline comparison with gpt-oss-20B as the teacher and Ouro-2.6B-Base as the student. All values are reported as accuracy (%).

MethodAIME24AIME25AIME26OlympiadBenchLiveCodeBench v5
Ouro-2.6B Base33.030.030.053.140.0
KD20.023.325.648.935.3
TRD32.235.635.657.647.5
RSR34.427.834.455.049.2
SE-KD20.021.120.045.438.6
AdaKD34.428.945.653.850.0
OPD12.220.016.737.324.9
FiRe-OPD14.414.413.338.410.9
LoopDistill (Ours)40.031.146.757.254.2

References

[1] LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. Jain et al. ICLR 2025.

[2] Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment. Yang et al. ACL 2026.

[3] Rethinking Selective Knowledge Distillation. Tavor et al. arXiv:2602.01395.

[4] LLM-Oriented Token-Adaptive Knowledge Distillation. Xie et al. AAAI 2026.

[5] Filter, Then Reweight: Rethinking Optimization Granularity in On-Policy Distillation. Li et al. arXiv:2606.02684.

Dear reviewers, as the discussion draws to a close, I would appreciate your thoughts on whether the authors’ rebuttal addresses your concerns. Thank you for your time!

AuthorsOpenReview ↗

General Response: revised manuscript

We thank all reviewers for their constructive feedback. We have submitted an updated PDF of our paper, incorporating additional experiments, analyses, and clarifications to address the concerns raised during the review process.

As the rebuttal deadline is approaching, we would greatly appreciate it if the reviewers could check our responses and the revised manuscript and let us know whether their concerns have been adequately addressed. We also kindly invite the reviewers to reconsider their scores in light of the additional evidence and improvements presented in our rebuttal.

Thank you again for your time, thoughtful feedback, and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.