acceptodds
Under review as a conference paper at ICLR 2027

Spend Teacher Tokens Where They Matter: Success-Referenced On-Policy Distillation

Abstract

On-policy distillation (OPD) combines student-generated rollouts with dense token-level supervision from a teacher, but providing such supervision for every rollout requires substantial teacher computation. We introduce Success-Referenced On-Policy Distillation (SR-OPD), which reduces this cost by selecting which prompts and rollouts receive teacher supervision. When the student produces both successful and failed rollouts for the same prompt, a successful rollout can serve as a natural reference for selecting failed rollouts. SR-OPD therefore focuses on such prompts and prioritizes failed rollouts whose hidden-state trajectories show sustained divergence from a successful reference, while accounting for estimated teacher-input cost. Across three teacher–student pairs and six mathematical reasoning benchmarks, SR-OPD uses only 3.46–5.02% of the teacher-input tokens required by Vanilla OPD in the one-pass setting while maintaining comparable reasoning performance. Under a controlled setting matched to 5% of Vanilla OPD’s teacher-input budget, further experiments support both key design choices: focusing supervision on prompts with both successful and failed rollouts, and using successful rollouts to guide failure selection. These results indicate that a student’s own successful behavior can serve as a useful reference for allocating teacher supervision under a fixed teacher-input budget.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, Selective Supervision, Teacher Efficiency, Success-Referenced Routing, Hidden-State Trajectories.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_f82cOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3(+2)
Soundness
3(+1)
Read review
Paper summary

This paper argues that in the on-policy distillation process, a pre-query routing technique analyzing only the student-generated rollouts can dramatically reduce teacher computation costs. When a student model generates multiple rollouts for a single problem, it selects only mixed-outcome prompts where both correct and incorrect answers are present. Hidden-state trajectories are aligned along a normalized progress axis to measure the persistent-departure area (PDA) of a failed rollout from the trajectory of a successful one. Measured by the area relative to the estimated teacher-input tokens, the method selects the single failed answer with the highest efficiency. The student is then updated by calculating token-level advantages under the guidance of the teacher model only for the selected answer. Consequently, SR-OPD maintains a reasoning performance comparable to Vanilla-OPD across three teacher-student pairs and six mathematical benchmarks while using only about 3% to 5% of the teacher-input tokens required by Vanilla-OPD.

Critical strengths
  1. Impressive Performance-Cost Trade-off: The method achieves remarkable efficiency gains, maintaining reasoning performance while consuming only a minimal fraction of tokens.
  2. High Exensibility: The method is easy to plug into existing pipelines.
Critical weaknesses
  1. High Background Dependency: The paper's structure assumes a strong familiarity with advanced on-policy distillation making it unfriendly to readers without prior context.
  2. Omission of Training Details and Hyperparameters: The paper and its appendix lack essential training details such as learning rates, optimizer configurations, maximum token lengths, etc. Omitting these critical hyperparameters severely hinders reproducibility.
  3. Narrow Domain Generalization: Evaluation is exclusively restricted to mathematical reasoning benchmarks
Questions for authors

Question on Base Model Performance and Distillation Efficacy: Table 1 compares various routing methods (Vanilla OPD, TA-OPD, TLR-OPD, and SR-OPD), but it lacks the performance baseline of the initial student model prior to on-policy distillation, as well as the performance of the teacher model. Could authors provide the zero-shot reasoning performance of the base student models across the six benchmarks?

Question on System-Level Overhead and Wall-Clock Time: While the paper successfully demonstrates a dramatic reduction in teacher-input tokens, the overall pipeline introduces additional student-side overhead such as generating multiple rollouts, extracting hidden states, and computing trajectory alignments. How much do the student-side routing computations offset the computational savings gained from reducing teacher queries?

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer f82c

We thank the reviewer for recognizing SR-OPD's efficiency and ease of integration. We address each concern below.

W1. OPD Background and Clarity

Like on-policy RL, OPD often samples multiple rollouts per prompt from the current student. For a math problem, the student generates KK solutions. The teacher provides token-level probability distributions for these student-generated reasoning contexts. In our Vanilla OPD setup, all KK rollouts receive teacher supervision, which is costly. SR-OPD routes rollouts before teacher queries: it first keeps prompts with both correct and incorrect answers, then selects one failed rollout using persistent-departure area (PDA) relative to teacher-input cost. Both methods use K=4K=4; SR-OPD needs no extra rollouts. We will add a workflow figure.

W2. Training Details

We will add the missing settings:

ParameterSetting
DataDAPO-Math-17K
Training1 epoch; 279 steps
OptimizerAdamW, β=(0.9,0.999)\beta=(0.9,0.999); weight decay 0.01
Learning rate5×10−65\times10^{-6}, constant; no warmup
Batch / rollouts64 prompts; K=4K=4
Max lengthsPrompt 2,048; response 8,192 tokens
Hardware8×NVIDIA H20
EvaluationT=0.7T=0.7, top-p=0.95; 16 samples; seeds 42–57

W3. Cross-Domain Generalization

We evaluated the math-trained Qwen 8K checkpoint on GPQA-Diamond (science), IFEval (instruction following), and HumanEval (code). Values are Avg@16 / Pass@16 (%); IFEval uses prompt-level strict accuracy.

ModelGPQA-DiamondIFEvalHumanEval
Student29.10 / 74.7568.23 / 82.9961.55 / 74.39
Teacher43.72 / 78.7983.40 / 89.6587.46 / 93.29
Vanilla OPD34.97 / 79.8068.63 / 85.0364.06 / 78.05
SR-OPD34.47 / 82.3269.39 / 85.0361.55 / 75.61

Against the initial student, SR-OPD improves Avg@16 by 5.37 points on GPQA-Diamond and 1.16 points on IFEval. HumanEval Avg@16 is unchanged, while Pass@16 rises by 1.22 points. Compared with Vanilla OPD, SR-OPD is similar on GPQA-Diamond and IFEval but lower on HumanEval. Performance on these non-math tasks is thus maintained relative to the initial student.

Q1. Initial Models and Distillation Results

We combine initial teacher/student baselines with distillation results. Scores are Avg@16 (%); Avg. is the six-benchmark mean. Input is teacher-input tokens in millions (M), with ratios relative to Vanilla OPD; – means not applicable.

PairMethodAIME24AIME25AMC23HMMT24HMMT25MATH-500Avg.Input (M; ratio)
QwenTeacher58.7544.3893.4426.5729.1793.3357.61–
QwenStudent14.7911.0442.195.835.4271.9325.20–
QwenVanilla OPD34.3119.4461.3011.6013.9684.3037.48374.10 (1.00)
QwenSR-OPD34.1719.5861.5612.0813.6884.2137.5513.60 (0.04)
SkyworkTeacher66.4651.4686.9032.9231.2593.5560.42–
SkyworkStudent28.1923.2662.4512.6413.9683.1837.28–
SkyworkVanilla OPD36.7426.6767.6015.1419.2485.8941.88519.80 (1.00)
SkyworkSR-OPD36.3226.2567.7115.1419.1786.1541.7918.00 (0.03)
GraniteTeacher8.124.5035.003.502.5069.0220.44–
GraniteStudent3.820.5623.962.220.2153.3514.02–
GraniteVanilla OPD3.751.4625.213.130.5656.9915.18101.30 (1.00)
GraniteSR-OPD3.681.6725.052.920.4256.9015.115.09 (0.05)

SR-OPD closely matches Vanilla OPD using only 3–5% of its teacher-input tokens.

Q2. System Overhead and Wall-Clock Time

SR-OPD adds hidden-state alignment, similarity calculation, and PDA scoring. For fixed rollout count and available hidden states, scoring scales approximately linearly with compared tokens and needs no additional teacher queries.

Qwen pair on 8×H20:

MethodTeacher GPU-hTotal GPU-hWall-clock h
Vanilla OPD3.9467.158.64
SR-OPD0.7956.707.54
Change−79.9%−15.6%−12.7%

Including routing overhead, SR-OPD reduces total GPU compute by 15.6% and end-to-end time by 12.7%. Student Actor rollout generation remains the main time cost. Moving some teacher resources to the Actor, subject to memory needs, may improve throughput. We will clarify teacher-query, routing, and end-to-end costs in the revision.

We thank the reviewer for the helpful comments.

Thanks to the authors for additional experiments about cross-domain generalization and base models. As a result of the authors’ response, I have revised the soundness and presentation score.

Reviewer_xrDzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Success-Referenced On-Policy Distillation (SR-OPD), a two-stage procedure for allocating teacher supervision before teacher inference. The student generates multiple rollouts per prompt; the method retains prompts containing both successful and failed rollouts and selects one failure using persistent-departure area (PDA), a hidden-state alignment score relative to successful same-prompt rollouts, normalized by estimated teacher-input cost. The selected rollout receives the unchanged OPD update. Experiments cover three teacher–student pairs and six mathematical reasoning benchmarks. In the one-pass setting, SR-OPD uses 3.46–5.02% of Vanilla OPD’s teacher-input tokens with closely matching mean accuracy. Additional teacher-budget-matched studies provide evidence for mixed-outcome filtering and success-referenced rollout selection, although the distinctive benefit of persistent departure remains uncertain.

Critical strengths

The paper addresses a practically relevant supervision-allocation problem. Selecting rollouts before teacher inference can avoid teacher processing that post-query token filtering has already incurred. The large reduction in teacher-input tokens across three model pairs makes the reported operating point practically interesting. The routing procedure also preserves the underlying OPD objective, making integration into existing distillation pipelines plausible. The paper appropriately distinguishes teacher-input accounting from total computation.

The controlled studies provide initial evidence for the hierarchical design. The paper separately examines prompt filtering and within-prompt rollout selection rather than relying exclusively on the comparison with Vanilla OPD. Table A2 reports positive paired effects for mixed-outcome filtering across all three model pairs, with pointwise intervals above zero. Table A4 reports improvements over Mixed+Random, with intervals excluding zero for Qwen and Skywork. These results support investigating outcome-based allocation and successful-reference selection, even though they do not yet establish the incremental value of PDA.

Critical weaknesses

The headline performance–cost result does not sufficiently isolate the proposed selector’s contribution.

Table 1 (§4.2, p. 6) compares comprehensive teacher supervision with a method that skips many prompts and supervises at most one failed rollout per retained prompt. The unresolved question is whether simple random or cost-based selection can preserve similar performance under the same one-pass, K = 4 protocol and at SR-OPD’s realized teacher-input budget.

We note that Table A3 already reports Mixed+Random at the controlled 5% budget, with Macro Avg values of 36.55, 40.49, and 14.39; Table A1 reports realized budget ranges of 4.98–5.02% across the controlled mixed-gate selectors. However, this arm permits additional data passes and therefore does not resolve the missing comparison under Table 1’s one-pass protocol. Cost-only appears in Appendix A.10 only under K = 8/16 and a 3% cap. Initial-student benchmark results are also absent, limiting interpretation of the absolute distillation benefit.

Appendix A.4 further allows different methods to encounter different amounts of student generation and potentially different numbers of updates before reaching the teacher-input target. This is a legitimate protocol for comparing outcomes at a fixed teacher budget, but it does not isolate trajectory selection from these accompanying differences. The authors acknowledge the accounting limitation; the remaining issue is the lack of descriptive computational records and a directly comparable simple baseline for the headline setting.

The distinctive value of persistent departure remains insufficiently established.

Table A4 (p. 15) reports PDA-minus-StaticSR improvements of 0.191, 0.262, and 0.196 percentage points, with respective 95% intervals of [−0.514, +0.895], [−0.372, +0.895], and [−0.184, +0.576]. All component contrasts in Table A5 also have intervals crossing zero. This does not establish that the components are ineffective, but the evidence for the specifically proposed departure statistic is substantially weaker than the evidence for the broader filtering and reference-selection ideas. The paper acknowledges this distinction, which should also delimit the claimed methodological contribution.

The interpretation of PDA requires further justification. Equations (8)–(10) measure unrecovered decreases from previously attained alignment: relative to a given successful reference, a trajectory with constant low similarity receives zero departure area, whereas one whose similarity decreases from a higher level receives a positive value. The final score takes the maximum across successful references. Moreover, normalized-length alignment need not align equivalent reasoning steps. These are not mathematical errors, and the paper correctly avoids claiming semantic error localization, but they leave the relationship between this statistic and useful supervision unclear. A targeted comparison of actual trajectories and their selection outcomes would be more informative than the current descriptive correlations alone.

Questions for authors

Matched one-pass baselines: Under Table 1’s one-pass, K = 4 protocol and at SR-OPD’s realized teacher-input budget, how do (i) a global-gate random baseline and (ii) cost-only selection under the mixed-outcome gate perform? Mixed+Random at the 5% budget already exists in Table A3 but permits additional data passes, while cost-only appears only in the K = 8/16, 3%-cap study in Appendix A.10. If full runs are infeasible, one representative teacher–student pair would already be informative. Please also report the initial-student results so that the absolute distillation gains can be assessed.

Student computation in controlled runs: Please report student-generated tokens, data passes, and optimizer updates for the 5% budget comparisons. If feasible, one representative teacher–student pair with matched student-generation tokens or matched optimizer updates would suffice to help assess whether the observed selector advantage persists. Otherwise, reporting the descriptive computational records would still clarify the comparison.

PDA versus static distance: Can the authors show actual failure trajectories on which PDA and StaticSR disagree and assess which selection yields more useful supervision? Please examine whether the disagreement is driven by response length or reference aggregation. The request concerns validating the existing selection mechanism, rather than extending the task scope.

Uncertainty and reproducibility: Please provide the full paired intervals against Vanilla OPD and the underlying seed-level results, or identify their location in the submitted supplementary materials. Similar point estimates are encouraging; the complete intervals would show what magnitude of degradation remains compatible with the data. This requests precision for the reported comparable performance, not a formal non-inferiority claim that the paper does not make. Please also identify where complete training/decoding configurations and verifier rules are provided. Section 3.3 states that the estimation of μ is provided in Appendix A.3, but Appendix A.3 only says that μ is estimated once from held-out student states and held fixed. Please specify the provenance and size of this held-out set, the estimator used, and whether student representation drift during training affects comparisons under a fixed μ.

Reviewer_uh3iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SR-OPD to reduce the cost of teacher supervision in on-policy distillation. The method first selects prompts containing both successful and failed rollouts, then uses hidden-state trajectories and estimated teacher-input costs to select failed rollouts for supervision. Experiments on six mathematical reasoning benchmarks with three teacher–student pairs show that SR-OPD reduces teacher-input tokens to 3.46–5.02% of Vanilla OPD while maintaining comparable reasoning performance.

Critical strengths
  1. The paper addresses the high cost of teacher supervision in vanilla OPD and proposes SR-OPD, a two-stage routing method that selects prompts and failed rollouts before querying the teacher. The method uses successful rollouts as references to identify which failed rollouts should receive teacher supervision. The motivation is clear, and the proposed approach is straightforward.

  2. The experimental results are encouraging. Across three teacher–student pairs, SR-OPD uses only 3.46–5.02% of Vanilla OPD’s teacher-input tokens while achieving similar performance on six math benchmarks. The controlled experiments also provide useful evidence for the two routing stages.

Critical weaknesses

1.The paper assumes that failed rollouts with larger PDA are more useful for teacher supervision, but this assumption is not sufficiently validated. In fact, failed rollouts with smaller PDA may be easier for the student to correct, as their hidden states are closer to those of successful rollouts. It would be useful to compare the proposed strategy with selecting the smallest PDA or intermediate PDA values under the same teacher-input budget.

2.PDA measures the area of deviation, but a larger area does not necessarily indicate more persistent divergence. For example, a rollout with a large deviation in the middle that later recovers may have a higher PDA than one that gradually diverges throughout the reasoning process. The latter, however, seems more consistent with the idea of persistent divergence. The authors should further justify this metric and consider comparing it with alternative measures that better capture divergence trends.

3.The paper aligns successful and failed rollouts using normalized token positions, but it is unclear whether these positions correspond to similar reasoning steps, especially when the rollouts have different lengths or follow different reasoning paths. For example, the middle of a successful rollout may represent a different reasoning stage from the middle of a failed rollout. Such mismatches could affect the similarity scores and the resulting PDA values. The authors should further justify this alignment strategy and compare it with alternative alignment methods.

4.SR-OPD only selects prompts that have both successful and failed rollouts. However, for difficult problems or weaker student models, most rollouts may fail, causing these prompts to be skipped entirely. Such prompts may still benefit from teacher supervision. The authors should further investigate how the method performs under different student success rates, especially when successful rollouts are rare.

5.Table 1 shows that SR-OPD significantly reduces teacher-input tokens, but its reasoning performance remains very close to Vanilla OPD. The accuracy on some challenging benchmarks, particularly AIME and HMMT, is also relatively low. Although improving efficiency is the main goal of this work, the experiments provide limited evidence that the proposed rollout selection strategy leads to meaningful performance gains. Further analysis of the performance limitations would strengthen the paper.

Questions for authors
  1. Why does the method prioritize failed rollouts with larger PDA? Would rollouts with smaller PDA be easier for the student to correct? Could the authors compare Max-PDA with Min-PDA and intermediate-PDA selection under the same teacher-input budget?

  2. Does PDA accurately measure persistent divergence? A rollout with a large deviation in the middle that later recovers may have a higher PDA than one that gradually diverges throughout the reasoning process. How do the authors justify this behavior of the metric?

  3. How reliable is the hidden-state alignment between successful and failed rollouts? The method aligns hidden states based on normalized token positions, but these positions may correspond to different reasoning steps. Have the authors examined how this alignment affects the PDA scores, particularly for rollouts with different lengths?

  4. How does SR-OPD handle difficult prompts where all sampled rollouts fail? What proportion of training prompts are excluded for this reason? Could excluding these prompts limit the improvement of weaker students?

  5. Does the reduction in teacher-input tokens translate into lower overall training costs? Could the authors report the total training time or GPU hours, including the additional cost of hidden-state extraction and rollout selection?

3 comments
AuthorsOpenReview ↗

Reply 1/2: PDA Selection, Metric Justification, and Trajectory Alignment (W1–W3)

We thank the reviewer for recognizing the motivation, teacher-cost savings, and two-stage routing of SR-OPD. We address W1–W3 below.

W1. Why select larger PDA?

We agree that smaller-PDA failures may be easier to correct. To test this, we compared Max-, Median-, and Min-PDA selection. Our motivation is informed by MOPD [1]: adding failed rollouts increased teacher Pairwise Accuracy from 0.32 to 0.40 and QA-Chemistry mean@8 from 52.62% to 74.29% (§5.3, Table 4), suggesting that successes and failures provide complementary information.

Model pairMax-PDAMedian-PDAMin-PDA
Qwen37.55 / 13.60M37.17 / 9.72M36.77 / 13.10M
Skywork41.79 / 18.00M41.50 / 15.95M41.08 / 17.42M
Granite15.11 / 5.09M14.89 / 4.47M14.66 / 4.89M

Each cell: Macro Avg@16 (%) / teacher-input tokens.

Max-PDA gives the highest performance point estimate for all three pairs, although token budgets differ. We report both metrics for transparency. Appendix A.9 also reports Spearman correlations of 0.314, 0.239, and 0.371 between PDA and validation-loss reduction.

W2. Does PDA measure persistent divergence?

We agree that the term Persistent Divergence is imprecise. PDA measures the cumulative decline in similarity from its running maximum:

A=∫01[max⁡0≤s≤tc(s)−c(t)] dt,A=\int_0^1[\max_{0\le s\le t}c(s)-c(t)]\,dt,

where c(t)c(t) is the hidden-state similarity between successful and failed trajectories. The area captures both the magnitude and duration of alignment decline.

The similarity curve is approximately U-shaped: it starts high because the trajectories share the prompt, drops as reasoning paths diverge, and may rise near the end because our Instruct models follow the same formatting instruction: Remember to put your answer on its own line after "Answer:".

We will describe PDA as Cumulative Alignment Drawdown and add a schematic. In Appendix A.8, PDA exceeds Max Departure by 0.249, 0.287, and 0.192 percentage points across the three model pairs.

W3. Is normalized token-position alignment reliable?

We agree that equal relative token positions need not correspond to the same reasoning step. Inspired by hidden-state trajectory analysis [2] and relative-progress alignment [3], SR-OPD maps different-length trajectories to normalized progress positions (M=16M=16). We tested three alternatives:

  • Normalized: Compare hidden states at matching relative progress in [0,1][0,1].
  • DTW-PDA: Use dynamic time warping for monotonic alignment at different reasoning rates.
  • Gap-DTW: Penalize non-diagonal DTW moves (λ=0.10\lambda=0.10) to reduce repeated matches.
  • OT-PDA: Use optimal transport to match within-stage hidden-state distributions without enforcing token order.

The first three use the same PDA calculation after alignment. The precise OT-PDA configuration remains to be checked against the run logs.

AlignmentQwen AvgSkywork AvgGranite AvgSelector time
Normalized37.5541.7915.111.0×
DTW-PDA37.4541.7014.994.8×
Gap-DTW37.4341.7015.046.6×
OT-PDA37.4941.7315.0621.4×

Performance is similar, while Normalized has the lowest selection overhead. We will discuss semantic alignment and other alternatives in the revision.

References

[1] Yu et al. (2026), Multi-Rollout On-Policy Distillation via Peer Successes and Failures. https://arxiv.org/abs/2605.12652

[2] Sun et al. (2026), LLM Reasoning as Trajectories. https://aclanthology.org/2026.acl-long.1237/

[3] He et al. (2026), Rethinking Reasoning Paths as Phase-Structured Trajectories (PAIR). https://arxiv.org/abs/2609.36461

AuthorsOpenReview ↗

Reply 2/2: Difficult Prompt Coverage and End-to-End Training Costs (W4–W5)

We again thank the reviewer for highlighting the importance of difficult prompt coverage and practical training costs. We continue by addressing W4 and W5.

W4. Why exclude difficult prompts where all rollouts fail?

Thank you for this suggestion. SR-OPD samples K=4K=4 rollouts per prompt. When all rollouts fail, the absence of a successful reference prevents a teacher query from being triggered in the current round.

We analyzed 17,917 training-prompt observations:

Rollout CategoryCountPercentage
All Failed6,68837.33%
All Correct1070.60%
Mixed3,11017.36%
Truncated / Parsing Failed8,01244.72%

Among valid observations, the All-Failed category accounts for 67.52%. We agree that this may limit difficult prompt coverage, particularly for weaker students.

To further investigate these categories, following Rethinking OPD [1], we measured teacher–student KL divergence and Top-16 Overlap across eight checkpoints:

CategoryInitial KL ↓Final KL ↓Initial Overlap ↑Final Overlap ↑
All Failed0.47210.374636.26%59.08%
Mixed0.50160.288735.89%69.77%

At the end of training, the Mixed category exhibits greater teacher–student distributional agreement.

Furthermore, when fixing the rollout selector to Random, Mixed routing improves performance over Uniform routing by 0.410, 0.467, and 0.338 percentage points across the three model pairs, respectively, with all paired 95% confidence intervals having lower bounds above zero.

These results support prioritizing Mixed prompts under a limited supervision budget.

We also acknowledge the potential value of All-Failed prompts. In the appendix, we will discuss curriculum-based OPD, where difficult prompts can be resampled in later rounds and potentially receive teacher supervision.

W5. Do teacher-token savings translate into lower end-to-end training costs?

Thank you for highlighting this important issue. We agree that teacher-token consumption alone does not fully capture end-to-end training costs.

We conducted an additional comparison of complete training costs for Qwen over 279 updates:

MethodTeacher GPU-hTotal GPU-hWall-clock h
Vanilla OPD3.9467.158.64
SR-OPD0.7956.707.54
Reduction79.9%15.6%12.7%

SR-OPD not only reduces teacher computation but also decreases total GPU-hours and wall-clock training time by 15.6% and 12.7%, respectively.

We will further clarify how each computational cost component is measured in the revised manuscript.

Regarding the performance limitations on AIME and HMMT, we agree that the absolute performance improvements of the current method are limited. Its primary objective is to maintain performance close to Vanilla OPD while substantially reducing teacher supervision costs.

Additionally, under a fixed 5% teacher budget, SR-OPD outperforms Mixed+Random by 0.616, 0.738, and 0.587 percentage points across the three model pairs, respectively (Appendix A.7, Table A4). This provides further evidence supporting the effectiveness of rollout selection.

We sincerely thank the reviewer again for the careful evaluation and constructive suggestions. We will further clarify the applicability and cost–performance trade-offs of our method in the revised manuscript.

References

[1] Li et al. (2026). Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe. https://arxiv.org/abs/2604.13016

Thank you for the authors' response. After considering the rebuttal, I would prefer to maintain my original assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.