acceptodds
Under review as a conference paper at ICLR 2027

LoOPSD: On-Policy Self-Distillation with Looped Teachers

13 views

Abstract

On-policy self-distillation improves a model using guidance from the model itself, but it still requires a teacher that provides more useful predictions than the student. Existing approaches obtain such guidance by giving the teacher additional context or instructions. We ask whether it can instead come from changing how the same model uses its own layers. To address this, we introduce LoOPSD, which searches over a model's layer execution order to construct a stronger teacher and distills its advantage into the student. Specifically, we keep the model weights unchanged while searching for layers to repeat, and use the resulting teacher's predictions on student-generated prefixes to guide learning. We then alternate search and distillation, rebuilding the teacher from the updated student in each round so that the student progressively absorbs the gains from alternative layer execution. Experiments on mathematical and logical reasoning tasks show improvements across models. On GSM8K with a 400-token thinking budget, LoOPSD raises Qwen3-1.7B accuracy from 42.30% to 79.61%; subsequent distillation rounds yield further gains beyond the first round. Together, these findings show that by searching layer execution to construct teachers, we can turn a model's additional computation into improved student capabilities, opening a new avenue for self-distillation to derive supervision from the model's own computation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Self-Dsitillation, On-Policy Distillation, Layer Routing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_h3s1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a self-distillation method that constructs a stronger teacher by modifying the execution order of a model's internal layers, repeating selected layers to increase computation without changing model parameters. Specifically, the method first uses a small set of labeled examples to identify which layers to repeat for improved accuracy. The resulting looped model then serves as the teacher, while the student follows the original layer execution order and is trained through reverse KL distillation. Experiments show that the proposed method achieves approximately 3 percentage points of improvement over the CRISP baseline on Qwen3-1.7B GSM8K.

Critical strengths
  1. The paper explores an interesting direction by constructing stronger self-teachers through layer looping and distilling the benefits into standard models.

  2. The experimental results show relatively consistent improvements across multiple models and reasoning benchmarks.

Critical weaknesses
  1. The paper lacks sufficiently compute-matched baseline comparisons. Layer search evaluates many candidate execution paths, and LoOPSD often requires multiple distillation rounds. Although the paper includes multi-round baselines, the additional search computation is not adequately accounted for. Could simpler approaches, such as sampling multiple reasoning trajectories and using majority voting or best-of-N selection for self-training, achieve comparable improvements under the same computational budget?

  2. The experiments use only one training seed, and some improvements over strong baselines are relatively modest (around 3–4 percentage points). Without repeated training runs, it is difficult to determine whether these gains are robust or partly due to randomness. Additional seeds and evaluations across different training configurations would strengthen the empirical evidence.

  3. The paper provides limited explanation of why repeating particular layers improves reasoning. Although the layer-search results demonstrate that execution order matters, the underlying mechanism remains unclear. More controlled ablations would be helpful, such as comparing repeated blocks with concentrating the same amount of additional computation in a single layer, or testing alternative repetition patterns under matched computational budgets.

Questions for authors
  1. Could the authors provide a deeper theoretical explanation of why repeating specific layers improves reasoning performance?

  2. Could the authors report results with more random seeds and error bars, and provide compute-matched comparisons with simpler alternatives?

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer h3s1 (Part 1/2)

We thank the reviewer for the suggestions on compute-matched alternatives, reproducibility, and the mechanism of repeated execution. We answer each concern directly.

W1 & Q2. Compute-matched majority-vote and best-of-N self-training

The paper lacks sufficiently compute-matched baseline comparisons. Layer search evaluates many candidate execution paths, and LoOPSD often requires multiple distillation rounds. Although the paper includes multi-round baselines, the additional search computation is not adequately accounted for. Could simpler approaches, such as sampling multiple reasoning trajectories and using majority voting or best-of-N selection for self-training, achieve comparable improvements under the same computational budget?

Could the authors provide compute-matched comparisons with simpler alternatives?

LoOPSD outperforms majority-vote and best-of-N self-training at the same total compute budget. We compare LoOPSD with two trajectory-selection self-training baselines in Table H1:

  • Majority-vote self-training: Samples 16 trajectories per question and selects a trajectory whose final answer agrees with the majority, without using reference answers.
  • Best-of-N self-training: Samples 16 trajectories per question and selects a correct trajectory using reference answers.

Table H1. Qwen3-1.7B/GSM8K-400: trajectory-selection baselines at equal total FLOPs.

MethodTotal PFLOPsStudent accuracy (%)
Majority-vote self-training135.244.58
Best-of-N self-training135.244.81
LoOPSD135.278.14

As shown in Table H1, LoOPSD outperforms both baselines by over 33%, with all methods receiving the same 135.2 PFLOPs. These results demonstrate that spending equivalent computation on sampling and selecting standard-path trajectories does not yield comparable improvements.

W2 & Q2. Robustness across random seeds and training configurations

The experiments use only one training seed, and some improvements over strong baselines are relatively modest (around 3–4%). Without repeated training runs, it is difficult to determine whether these gains are robust or partly due to randomness. Additional seeds and evaluations across different training configurations would strengthen the empirical evidence.

Could the authors report results with more random seeds and error bars?

Multiple Seeds. LoOPSD† consistently outperforms CRISP with low variability across three training seeds. We report the mean accuracy and standard deviation of LoOPSD† alongside the originally reported CRISP results.

Table H2. Qwen3-1.7B: comparison with CRISP across five benchmarks.

BenchmarkCRISP (%)LoOPSD†, 3 seeds (%)Gain (pp)
GSM8K76.6580.44 ± 1.06+3.79
GSM-Plus58.5062.90 ± 1.10+4.40
SVAMP88.3391.45 ± 1.35+3.12
MATH-50037.8047.27 ± 1.14+9.47
ProofWriter43.4053.70 ± 0.53+10.30

As shown in Table H2, even the lowest-performing LoOPSD† seed outperforms CRISP on all five benchmarks, including GSM8K and SVAMP, where the improvements are relatively modest. The low variability across seeds further supports the consistency of these gains.

Different Training Configurations. LoOPSD† maintains consistent performance across different training settings. To further evaluate the robustness of our results, we vary the distillation objective, learning rate, and student sampling configuration in Table H3.

Table H3. Qwen3-1.7B/GSM8K-400: robustness across training configurations.

MethodDistillation objectiveTraining batchLearning rateAccuracy (%)
CRISP (paper setting)76.65
LoOPSD† (paper setting)Sampled-token KL12 questions × 4 responses2e-679.68
LoOPSD†Full-vocab Reverse KL——79.53
LoOPSD†Full-vocab Reverse KL—1e-680.24
LoOPSD†Top-k16 Reverse KL——80.74
LoOPSD†—24 questions × 2 responses—81.27

For LoOPSD† variants, bold configuration entries indicate changes from the paper setting; dashes indicate unchanged settings.

As shown in Table H3, LoOPSD† achieves 79.53–81.27% accuracy across the tested configurations, consistently exceeding the original CRISP result of 76.65%. Together with Table H2, these results demonstrate that LoOPSD† maintains its gains across both training seeds and configurations.

AuthorsOpenReview ↗

Author Response to Reviewer h3s1 (Part 2/2)

W3 & Q1. Why the repeated-layer structure matters

The paper provides limited explanation of why repeating particular layers improves reasoning. Although the layer-search results demonstrate that execution order matters, the underlying mechanism remains unclear. More controlled ablations would be helpful, such as comparing repeated blocks with concentrating the same amount of additional computation in a single layer, or testing alternative repetition patterns under matched computational budgets.

Could the authors provide a deeper theoretical explanation of why repeating specific layers improves reasoning performance?

Structure Ablation. Repeating a useful contiguous block gives a stronger teacher than placing the same additional computation in one layer or splitting it across blocks. On Qwen3-4B/GSM8K-128, we compare four execution structures:

  • Standard path: Execute the original 36 layers without repetition.
  • Single-layer repetition: Execute one layer six additional times.
  • Split-block repetition: Repeat two separate three-layer blocks once each.
  • Contiguous-block repetition: Repeat one contiguous six-layer block once.

We use the same route-search budget for all three repeated-execution structures. All three repetition alternatives add six layer executions, resulting in 42 executed layers.

Table H4. Qwen3-4B/GSM8K-128: alternative execution structures.

Teacher execution structureExecuted layersTeacher accuracy (%)
Standard path3627.22
Six additional executions of one layer4210.08
Two separate three-layer blocks, repeated once4228.81
One contiguous six-layer block, repeated once4233.74

As shown in Table H4, the selected contiguous block outperforms the tested split-block alternative by 4.93%, while single-layer repetition even reduces accuracy below the standard path. Simply adding more layer executions does not guarantee improvement; selecting an effective execution route is essential.

Mechanistic Interpretation. Prior studies suggest that pretrained Transformer middle layers exhibit representational similarity and tolerate certain changes in execution order [1, 2]. Westerhoff et al. [3] further associate the effectiveness of layer repetition with boundaries between different representational regimes.

These findings suggest that a suitable repeated block may further refine answer-relevant representations while remaining compatible with subsequent layers. In contrast, unsuitable repetition may disrupt subsequent computation. This provides a possible explanation for why some pretrained layer compositions benefit from repeated execution while others do not.

Answer Readout. To further examine this interpretation, we compare the standard path and selected loop with answer readout:

Table H5. Qwen3-4B/ProofWriter: answer readout at identical model weights and contexts.

Execution pathMean accuracy, layers 25–35 (%)Final output accuracy (%)
Standard 36-layer path55.0959.80
Selected 42-layer loop60.4366.00

As shown in Table H5, the selected loop improves intermediate-layer answer readout by 5.34% and final output accuracy by 6.20% without changing model weights. Together with Figure 6 and Appendix Figure 11, these results support our interpretation that useful repeated execution can make answer-relevant information more accessible.

References

[1] Sun et al. Transformer Layers as Painters. AAAI 2025.

[2] Lad et al. The Remarkable Robustness of LLMs: Stages of Inference? NeurIPS 2025.

[3] Westerhoff et al. Programs-of-Layers in LLMs through the Lens of Cortical Areas. arXiv preprint, 2026.

AuthorsOpenReview ↗

Gentle Reminder: Rebuttal and Updated Paper Available

Dear Reviewer h3s1,

This is a gentle reminder that we have submitted our rebuttal and updated the paper accordingly. If you have a moment, we would greatly appreciate it if you could take a look and share any further comments during the discussion phase. We are very happy to clarify any points or provide additional information as needed.

Thank you very much for your time and for engaging with our submission.

Best regards,

Authors of Submission 4558

AuthorsOpenReview ↗

Eagerly await your valuable feedback|Just One Day Left in the Discussion

Dear Reviewer h3s1,

Thank you again for your thoughtful review. With the discussion deadline approaching, we wanted to briefly follow up on the three points you raised.

  • W1. Compute-matched simpler baselines. We compare LoOPSD with majority-vote and best-of-N self-training at the same total compute budget. LoOPSD reaches 78.14% accuracy, outperforming both baselines by more than 33%.
  • W2. Robustness across seeds and training settings. LoOPSD† outperforms the reported CRISP results on all five benchmarks across all three training seeds. It also maintains its advantage on GSM8K when we vary the distillation objective, learning rate, and student sampling configuration.
  • W3. Why the repetition pattern matters. With six extra layer executions in each case, repeating a contiguous block yields higher accuracy than repeating a single layer or two separate blocks. We provide a detailed explanation in our rebuttal of why repeating a well-chosen block may improve reasoning.

If you have a moment before the deadline, we’d welcome any thoughts on the new results or any points you’d like us to clarify. Thank you again for your time and feedback.

Best regards,

Authors of Submission 4558

Reviewer_6yrQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes LoOPSD, an iterative self-distillation method that changes how the current model executes its layers in order to build a stronger teacher. In each round, it searches for a contiguous blocks of layers to repeat, selects a route using labeled search and validation questions, and freezes that routed model as the teacher. A student then generates responses and learns from the teacher's token predictions on those student-generated prefixes. The updated student becomes the starting point for the next search round. The paper evaluates several Qwen3 and DeepSeek-based models on mathematical and logical reasoning tasks. It reports large gains under restricted thinking budgets, including a Qwen3-1.7B GSM8K improvement from 42.30% to 79.61% at a 400-token thinking budget. It also studies transfer, generation length, multiple test budgets, label usage, sampling temperature, and layerwise readout behavior.

Critical strengths
  1. The core construction is simple and potentially useful. LoOPSD creates a stronger teacher without introducing a separate larger model, and the final student returns to the original layer order. The large same-weight teacher improvement under a short reasoning budget is a notable empirical finding.
  2. The evaluation goes beyond a single summary table, covering multiple model sizes and tasks, varying thinking budgets, transfer sets, response lengths, sampling temperatures, and iterative behavior. Figure 5 is particularly informative, as it includes multi-round standard-path baselines evaluated under the same update schedule.
Critical weaknesses
  1. The comparisons are not matched in terms of computational budget: the final LoOPSD results can involve multiple rounds of route search and distillation (up to 10 rounds), while the CRISP baseline in Tables 1 and 2 only uses a single round of 200 updates. In addition, route search evaluates many candidate executions before each distillation round. However, the paper does not report the total generated tokens, scored tokens, FLOPs, or wall-clock time for each method. This makes it difficult to assess whether the comparison is fair. In fact, after just one round, LoOPSD underperforms CRISP in many settings in Table 1 and for both models in Table 2. I would like to see stronger controls, such as random or fixed loops, route selection without labels, and compute-matched multi-round baselines without loops. Without these comparisons, it is unclear how much of the improvement comes from selecting better loops rather than simply using more computation.
Questions for authors
  1. Could the authors report the total computational cost for each method and final model, including generated and scored tokens, FLOPs, GPU-hours, number of route evaluations, and distillation rounds?
  2. How does LoOPSD perform against compute-matched, multi-round versions of CRISP and GRPO? It would also be helpful to compare against fixed or randomly selected loop teachers using the same prompts and update budget.
4 comments
AuthorsOpenReview ↗

Author Response to Reviewer 6yrQ (Part 1/2)

We thank the reviewer for the constructive comments on computational fairness and teacher quality.

W1-1 (Q1 & Q2). Total cost and matched-budget comparisons

The comparisons are not matched in terms of computational budget: the final LoOPSD results can involve multiple rounds of route search and distillation (up to 10 rounds), while the CRISP baseline in Tables 1 and 2 only uses a single round of 200 updates. In addition, route search evaluates many candidate executions before each distillation round. However, the paper does not report the total generated tokens, scored tokens, FLOPs, or wall-clock time for each method. This makes it difficult to assess whether the comparison is fair. In fact, after just one round, LoOPSD underperforms CRISP in many settings in Table 1 and for both models in Table 2. I would like to see stronger controls, such as random or fixed loops, route selection without labels, and compute-matched multi-round baselines without loops. Without these comparisons, it is unclear how much of the improvement comes from selecting better loops rather than simply using more computation.

Could the authors report the total computational cost for each method and final model, including generated and scored tokens, FLOPs, GPU-hours, number of route evaluations, and distillation rounds?

How does LoOPSD perform against compute-matched, multi-round versions of CRISP and GRPO?

Matched-budget Comparisons. LoOPSD† achieves higher accuracy than compute-matched GRPO and CRISP even after including teacher-search costs. To test whether the improvements merely reflect additional computation, we compare methods at the same or comparable total FLOPs, accounting for all search and distillation costs.

Table C1. Accuracy and total computation (search + training) under a 400-token thinking budget. Within each setting, CRISP and LoOPSD† use the same number of rounds. LoOPSD† spends part of its budget on route search; CRISP uses that compute for additional training instead.

ModelMethodGSM8K (%)GSM-Plus (%)MATH-500 (%)
Qwen3-1.7BGRPO75.8257.6038.60
CRISP79.3061.2048.00
LoOPSD†79.6861.8048.20
Matched PFLOPs≈861.3≈861.3≈2,459
Qwen3-4BGRPO83.3264.8038.80
CRISP88.4870.8051.60
LoOPSD†91.1373.2060.40
Matched PFLOPs≈2,630≈2,630≈3,373

Total FLOPs are approximately matched across methods within each setting and include route-search costs. GSM-Plus evaluates the GSM8K-trained checkpoints without additional training.

At matched total compute, LoOPSD† outperforms both CRISP and GRPO across both model sizes and all three benchmarks. The gains over CRISP are particularly pronounced on Qwen3-4B, reaching +8.80 pp on MATH-500. These results support the benefit of allocating computation to route search rather than additional baseline training.

Cost details. We count candidate searches, validation and stopping searches, student generation, teacher scoring, and optimization. Table C2 gives representative cost breakdowns, with route evaluations and token counts shown in compact units.

Table C2. Representative cumulative computational costs on GSM8K (400-token thinking budget); estimates.

Model / MethodRoundsRoute evaluations (K)Generated / scored tokens (M)Search / train / total PFLOPsEst. GPU-h
Qwen3-1.7B
CRISP10.03.2 / 5.00 / 96 / 964.8
LoOPSD9806.0375.5 / 48.02,023 / 1,038 / 3,061149.4
LoOPSD†2249.9107.0 / 10.1664 / 197 / 86144.1
Qwen3-4B
CRISP10.03.2 / 5.00 / 224 / 2246.0
LoOPSD7911.0407.3 / 37.35,442 / 1,895 / 7,337243.4
LoOPSD†2349.8141.8 / 10.12,169 / 461 / 2,63097.5
Qwen3-14B
CRISP10.03.2 / 5.00 / 779 / 77911.5
LoOPSD91,335.9571.1 / 48.027,146 / 8,493 / 35,639618.9
LoOPSD†1272.6100.8 / 5.05,633 / 779 / 6,412121.0

We have also included the relevant material in Appendix G.2 (Table 7) and G.3 (Table 8).

AuthorsOpenReview ↗

Author Response to Reviewer 6yrQ (Part 2/2)

W1-2 (Q2). Route selection comparisons

I would like to see stronger controls, such as random or fixed loops, route selection without labels, and compute-matched multi-round baselines without loops. Without these comparisons, it is unclear how much of the improvement comes from selecting better loops rather than simply using more computation.

It would also be helpful to compare against fixed or randomly selected loop teachers using the same prompts and update budget.

Route selection. To verify that the gains depend on finding a useful loop, we compare searched routes with fixed and random alternatives under matched prompts and student updates as below:

  • Naive self-distillation: Use the standard execution path without looping.
  • Random loop: Randomly select a contiguous six-layer block to repeat (report the mean over three routes).
  • Fixed middle block: Repeat a predefined six-layer block in the middle of the model.
  • Fixed final block: Repeat a predefined six-layer block near the end of the model.
  • Searched loop (LoOPSD): Select the repeated block through our route-search procedure.

Table C3. Qwen3-4B/GSM8K-128: student accuracy by teacher route.

Teacher routeAccuracy (%)
Naive self-distillation (no looping)27.37
Random loop14.13
Fixed middle block27.98
Fixed final block28.43
Searched loop (LoOPSD)35.78

The searched loop outperforms fixed and random routes, providing complementary evidence that the quality of the selected teacher, rather than extra compute alone, drives LoOPSD's improvement.

AuthorsOpenReview ↗

Gentle Reminder: Rebuttal and Updated Paper Available

Dear Reviewer 6yrQ,

This is a gentle reminder that we have submitted our rebuttal and updated the paper accordingly. If you have a moment, we would greatly appreciate it if you could take a look and share any further comments during the discussion phase. We are very happy to clarify any points or provide additional information as needed.

Thank you very much for your time and for engaging with our submission.

Best regards,

Authors of Submission 4558

I thank the authors for their detailed responses. After carefully considering the rebuttal and the overall quality of the paper, I have decided to maintain my original score.

Reviewer_BbbVOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies on-policy self-distillation (OPSD) for large language models, where a model improves its reasoning capabilities using self-generated supervision without relying on an external, stronger teacher model. While existing self-distillation methods typically construct a stronger teacher by providing privileged context, reference reasoning traces, or specialized prompt instructions, the authors propose LoOPSD. LoOPSD investigates whether a model can serve as its own stronger teacher by altering how it executes its layers (specifically, repeating a contiguous block of layers) without changing its model parameters.

Critical strengths

A central challenge in self-distillation is avoiding degenerate self-imitation. LoOPSD elegantly leverages the emerging insight that executing contiguous layer blocks recurrently enhances reasoning without retraining weights. Converting this execution-derived advantage into dense token-level training guidance for a standard-path student is conceptually novel and avoids the need for external annotations or privileged prompt contexts.

The authors conduct comprehensive experiments across six models spanning 1.7B to 14B parameters and demonstrate consistent gains across various thinking budgets and out-of-distribution transfers (e.g., GSM-Plus). Furthermore, the logit-lens analysis (Figures 6 and 11) convincingly illustrates that supervising the student with the looped teacher strengthens answer readout in the deeper intermediate layers of the standard architecture.

Critical weaknesses

Omission of Cumulative Route-Search Overhead:According to Appendix A, each round requires a two-stage screening process (evaluating candidate routes on 128 and 872 questions, followed by validation on 100 questions). For multi-round setups (e.g., up to 6–10 rounds), this repeated search incurs substantial rollout and forward-pass computation. In Table 3 ("Label Efficiency"), only the student update steps and labeled sample counts are matched against GRPO, omitting the inference cost required by candidate search. A complete compute-budget comparison (total FLOPs / GPU hours) is needed to assess whether the search cost justifies the performance gain over spending that budget directly on standard RL exploration.

Equation (2) explicitly relies on a reference answer scoring function V(⋅)V(\cdot) to validate whether a candidate looped route is superior to the standard path. This restricts the method's applicability to tasks with exact verifiers or labeled validation sets. The paper does not address how the teacher search could be performed in an unsupervised or verifier-free setting.

Questions for authors

NA

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer BbbV

We thank the reviewer for recognizing the novelty of execution-derived teachers and for the suggestions on computational fairness and route selection. We respond to both concerns below.

W1. Cumulative route-search cost and computational fairness

Omission of Cumulative Route-Search Overhead: According to Appendix A, each round requires a two-stage screening process (evaluating candidate routes on 128 and 872 questions, followed by validation on 100 questions). For multi-round setups (e.g., up to 6–10 rounds), this repeated search incurs substantial rollout and forward-pass computation. In Table 3 ("Label Efficiency"), only the student update steps and labeled sample counts are matched against GRPO, omitting the inference cost required by candidate search. A complete compute-budget comparison (total FLOPs / GPU hours) is needed to assess whether the search cost justifies the performance gain over spending that budget directly on standard RL exploration.

LoOPSD outperforms GRPO at matched total compute, including route-search costs.

To test whether LoOPSD's gains simply come from extra computation, we compare it with GRPO using the same labeled-data budget and total FLOPs, including route-search and training costs. We also report GRPO with the same number of student updates as a reference.

Table B1. Qwen3-1.7B/GSM8K-400: accuracy (%) under matched labeled-data budgets and computation.

LabelsMatched total compute (PFLOPs)LoOPSD (%)GRPO, same compute (%)GRPO, same updates (%)
200135.278.1444.5843.90
400174.678.3944.7343.52
1,100315.978.6247.0844.05

LoOPSD exceeds compute-matched GRPO by over 30% across all three labeled-data budgets. These results show that LoOPSD's gains come from more effective teacher construction and distillation, rather than simply using more computation.

We have also included the relevant material in Appendix G.1 (Table 6).

W2. Teacher search without reference answers

Equation (2) explicitly relies on a reference answer scoring function to validate whether a candidate looped route is superior to the standard path. This restricts the method's applicability to tasks with exact verifiers or labeled validation sets. The paper does not address how the teacher search could be performed in an unsupervised or verifier-free setting.

Our work focuses on teacher construction with limited labeled feedback, while fully reference-free teacher selection remains a separate research problem.

Nevertheless, LoOPSD can be naturally extended to settings without reference answers. To demonstrate this, we replace reference-answer accuracy with agreement among sampled answers (self-consistency) to select a looped teacher.

Table B2. Qwen3-4B/GSM8K-128: student accuracy (%).

MethodAccuracy (%)
Naive self-distillation (no looping)27.37
LoOPSD (self-consistency)34.34

LoOPSD improves student accuracy by 6.97% without using reference answers for teacher selection. This simple extension demonstrates the feasibility of finding effective teachers without reference answers, with further room to explore better approaches.

AuthorsOpenReview ↗

Gentle Reminder: Rebuttal and Updated Paper Available

Dear Reviewer BbbV,

This is a gentle reminder that we have submitted our rebuttal and updated the paper accordingly. If you have a moment, we would greatly appreciate it if you could take a look and share any further comments during the discussion phase. We are very happy to clarify any points or provide additional information as needed.

Thank you very much for your time and for engaging with our submission.

Best regards,

Authors of Submission 4558

AuthorsOpenReview ↗

Eagerly await your valuable feedback| Just One Day Left in the Discussion

Dear Reviewer BbbV,

Thank you again for your thoughtful review. With the discussion deadline approaching, we wanted to briefly follow up on the two questions you raised.

  • W1. Computational fairness: We provide matched-label and matched-total-FLOPs comparisons that explicitly account for cumulative route-search costs. LoOPSD consistently outperforms GRPO across all three budgets, with gains exceeding 30 percentage points.
  • W2. Reference-free route selection: We additionally explore self-consistency-based teacher selection without reference answers for route search, improving student accuracy from 27.37% to 34.34%.

If you have a moment before the deadline, we would appreciate a brief reply on whether these results address your concerns. If so, we would deeply appreciate it if you could consider raising your score. If not, please let us know if you have any further questions. We’d be happy to clarify any remaining points.

Best regards,

Authors of Submission 4558

Comments on the paper

1 comment

Dear Reviewers,

Thank you for your time and effort in reviewing this paper. Please take a moment to read the authors’ rebuttal and the other reviewers’ comments, and participate in the discussion. It would be helpful to indicate which concerns have been addressed and which remain unresolved, and to follow up with the authors where further clarification is needed. Please also update your review and score if warranted, or briefly explain why your assessment remains unchanged. Your engagement will be valuable in reaching a well-informed decision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.