acceptodds
Under review as a conference paper at ICLR 2027

Where Should Suppressed Probability Go? One-Step Lookahead KL Routing for On-Policy Distillation

Abstract

On-policy distillation (OPD) supervises a student on its own trajectories, but its sampled-token form leaves the recipients of negative credit implicit. When a sampled token is suppressed, the compensating logit update follows the student's current probabilities rather than explicitly selecting recipients using teacher evidence, even among alternatives that both models rank highly. We formulate this limitation as recipient routing and introduce Lookahead KL Routing (LKR). LKR first defines a dense routing operator. It combines current teacher support with agreement between the student and teacher after appending each candidate, then solves a constrained reverse-KL projection to reallocate a bounded portion of the update within this shared candidate set. Its unique water-filling target defines a routing correction that preserves the sampled token's local logit gradient. Dense routing would otherwise require successor evaluation at every eligible event. LKR makes training practical by sampling a small subset of events and adapting the projection budget to keep each routing update bounded. Across six public mathematical reasoning benchmarks, LKR improves step-matched OPD by 2.23 and 4.54 percentage points in Mean@8 and Cons@8, respectively. The gains persist across three seeds on a 1,000-problem held-out set with a 1.5B student and a 7B teacher. The pattern holds with a 7B student and a 32B teacher.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, on-policy distillation, knowledge distillation, mathematical reasoning, gradient routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_QEUVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Lookahead KL Routing (LKR) for on-policy distillation. LKR redirects part of a negative sampled-token update toward alternatives selected using current teacher probabilities and one-step student–teacher KL divergence. A constrained reverse-KL projection yields a water-filling allocation, which preserves the donor’s local logit gradient. Budgeted event sampling reduces lookahead queries. Experiments cover a held-out mathematical reasoning set and six public benchmarks, with three-seed results for the primary model pair, component ablations, and a single-seed larger-scale comparison.

Critical strengths
  • A coherent and interpretable routing formulation. The donor–recipient decomposition makes the destination of negative updates explicit. The constrained projection provides a well-defined allocation, and the manuscript carefully limits its guarantees to the auxiliary objective and local logit gradients.

  • Meaningful empirical improvements with useful controls. The primary three-seed comparison improves held-out Mean@8 and Cons@8 over OPD by 2.41 and 2.27 percentage points, respectively. Gains also persist against longer OPD training under an approximately matched training-time budget, providing evidence beyond a step-matched comparison.

Critical weaknesses
  • Theory–implementation mismatch. The rollout-dependent deterministic seeding leaves the applicability of Proposition 3’s conditional-unbiasedness guarantee to the released sampler unresolved.

  • Unvalidated KL approximation. The coarse successor KL can reverse candidate preferences, yet its effect on routing allocations is not empirically quantified.

  • Insufficient component-level evidence. Single-run ablations and limited equal-cost comparisons with cheaper routing variants do not firmly establish the incremental benefits of lookahead and water-filling.

Questions for authors
  • How is the rollout-derived sampling seed reconciled with the conditional inclusion probabilities in Proposition 3? Was this sampler used in the reported training runs?

  • On representative training events, how frequently does coarse successor KL change candidate rankings or routing allocations relative to full-vocabulary KL? What are the intersection-size and retained-probability-mass distributions?

  • Do the improvements from lookahead and water-filling persist across matched training seeds and equal measured training-time budgets, relative to Teacher-only and normalized-score routing?

  • Which implementation and configuration produced the k=8k=8 results, given the restriction in the released projection function?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QEUV

Thank you for recognizing the routing formulation and experimental controls. We address your questions below.

1. Sampling seed and Proposition 3

Thank you for highlighting this detail. The reported training runs used this sampler: it selects at most m donors uniformly without replacement from each rollout's negative-feedback positions, using a seed derived from rollout-related inputs.

Proposition 3 considers random donor sampling conditional on a fixed rollout and scores. With the current seed construction, fixing the relevant inputs also fixes the selected donors. To align the implementation with this condition, future experiments will use an independent donor-sampling seed to initialize a separate random generator once at training start, advancing its state throughout training. We will document the historical seed setup and this adjustment in the revision.

2. Successor KL resolution and its effect on routing

Considering the experimental cost, we added successor Top-20/50/100 resolution comparisons, using Top-100 as a finer reference. All three merge the vocabulary outside the teacher-student intersection into one tail bucket.

Using 128 rollouts at the primary pair's step-250 checkpoint, we evaluated 2,311 valid routing events and 8,910 successor prefixes. We fixed the current candidates (cap k=4), current teacher/student probabilities, and clipped dose, changing only successor KL resolution before recomputing water-filling shares.

The table reports successor intersection size and both models' probability mass on it as median [P10, P90].

\text{Top-K}&\text{Intersection size}&\text{Student mass}&\text{Teacher mass}\\\\ 20&11\\ [4,16]&0.827\\ [0.412,0.968]&0.862\\ [0.478,0.981]\\\\ 50&29\\ [13,42]&0.913\\ [0.609,0.989]&0.931\\ [0.682,0.994]\\\\ 100&63\\ [31,85]&0.952\\ [0.738,0.995]&0.968\\ [0.804,0.998] \end{array}$$ Top-20 retains median masses of 82.7% and 86.2% for the student and teacher; higher resolutions represent more probability mass token by token. We also measured changes in rankings and allocations: $$\begin{array}{lrrrr} \text{Comparison}&\text{KL reversal (\\%)}&\text{Top-score change (\\%)}&\text{Mean TV}&\text{P90 TV}\\\\ 20\to50&4.74&2.99&0.0259&0.0746\\\\ 20\to100&7.59&4.72&0.0379&0.1042\\\\ 50\to100&3.03&2.03&0.0151&0.0413 \end{array}$$ Reversal rates use candidate pairs strictly ordered under both resolutions; other metrics use the 2,311 valid events. TV is half the L1 distance between allocation shares, measuring the share requiring reallocation. From Top-20 to Top-100, 7.59% of candidate pairs reverse KL ordering, but only 4.72% of events change their highest-scoring candidate, with an average reallocation of 3.79%. From Top-50 to Top-100, this falls to 1.51%. These results support Top-20 as a practical routing approximation. Based on this resolution trend, we conjecture that using full-vocabulary KL may also have a limited effect on overall allocations. We will add these experiments and metric definitions to the revision. **3. Lookahead and water-filling under matched training-time budgets** Thank you for suggesting controls with matched seeds and measured training time. We added an ablation with one shared training seed, the same initialization and data on eight GPUs, with k=4. The target training-time budget matches the main experiments. Evaluation uses the same 1,000 Held-out questions. Teacher-only retains current teacher scores and water-filling but removes both models' successor queries. Normalized-score retains lookahead scores but replaces water-filling with score normalization. $$\begin{array}{lrrrrr} \text{Method}&\text{Steps}&\text{Time (s)}&\text{Teacher successor queries}&\text{Mean@8 (\\%)}&\text{Cons@8 (\\%)}\\\\ \text{Teacher-only}&334&28{,}857&0&24.96&35.20\\\\ \text{Normalized-score}&252&28{,}879&2{,}327{,}846&24.84&34.90\\\\ \text{LKR}&250&28{,}894&2{,}311{,}492&25.61&36.40 \end{array}$$ Despite Teacher-only completing 84 more steps, LKR gains 0.65/1.20 percentage points in Mean@8/Cons@8, supporting the benefit of successor information. Against Normalized-score, with similar runtime, steps, and query counts, LKR gains 0.78/1.50 points, supporting water-filling. Both components retain their benefits in this time-matched ablation. We will add the comparison and configurations. **4. Implementation and configuration for k=8** Thank you for noting this version discrepancy. The main and new supplementary experiments use k=4; k=8 is a hyperparameter experiment. It used a training version supporting eight candidates, with the cap set to 8. We omitted the updated candidate-count check when uploading the code, leaving the old restriction in the released version. We apologize for this inconsistency. After acceptance, we will open-source the correct code version consistent with the paper's experiments, together with the corresponding launch configurations.

Thank you for the additional experiments. Two concerns remain:

  1. Top-100 is still a coarsened reference. A comparison against full-vocabulary KL on a small representative subset would more directly assess the approximation’s effect on routing.
  2. The time-matched ablations provide useful support for both components, but one shared training seed does not establish robustness across seeds. Results over multiple matched training seeds are still needed.
AuthorsOpenReview ↗

Response to Reviewer QEUV

Dear Reviewer QEUV,

Thank you for following up. Because time was limited during the initial rebuttal, we first ran and reported the time-matched ablation with one training seed shared across methods. We have just obtained the results for two additional matched seeds. We report the three-seed results below, along with the requested full-vocabulary KL comparison.

1. Full-vocabulary KL comparison

We randomly sampled 32 questions, yielding 578 valid routing events and 2,227 successor prefixes. We computed successor reverse KL in FP32 over all 151,665 tokens in the effective vocabulary shared by our 1.5B student and 7B teacher. On the same events, we compared candidate rankings and water-filling allocations using Top-20, Top-100, and full-vocabulary KL.

MetricTop-20 → Top-100Top-20 → FullTop-100 → Full
KL ranking reversal (%)7.619.111.90
Routing score ranking reversal (%)4.955.891.17
Highest-scoring candidate changed (%)4.675.711.21
Water-filling active set changed (%)10.9012.982.77
Mean allocation TV0.03810.04450.0084
Median allocation TV0.01390.01680.0029
P90 allocation TV0.10480.12160.0231
Maximum allocation TV0.3960.4420.118

Ranking reversal rates use pairs strictly ordered in both settings. Allocation TV is half the L1 distance between the two vectors of allocation shares. We summarize it over all 578 events.

The full-vocabulary comparison supports Top-20 as a practical routing approximation in our setting. Compared with full KL, Top-20 changed the highest-scoring candidate in 5.71% of events and shifted 4.45% of the allocation mass on average, with larger shifts in some events. For Top-100, the corresponding change rate and mean allocation TV were 1.21% and 0.0084, supporting its use as a finer reference in our earlier analysis.

2. Time-matched ablations across three seeds

Each method used the same three training seeds and an 8-hour target training budget per run. All runs were evaluated on the same 1,000 held-out questions. We report mean ± sample standard deviation across seeds.

MethodTraining stepsMeasured training time (s)Mean@8 (%)Cons@8 (%)
Teacher-only333.3 ± 1.228,864 ± 1425.03 ± 0.2635.23 ± 0.25
Normalized-score251.7 ± 0.628,876 ± 1224.84 ± 0.2234.97 ± 0.31
LKR250.3 ± 0.628,890 ± 1425.73 ± 0.1636.47 ± 0.21

LKR outperformed both controls on both metrics in all three matched seeds. The paired gains in Mean@8 and Cons@8 were 0.70 ± 0.14 and 1.23 ± 0.06 percentage points over Teacher-only, and 0.89 ± 0.14 and 1.50 ± 0.10 percentage points over Normalized-score. These results support the contributions of lookahead and water-filling under the matched time budget in this setting.

We sincerely thank you for taking the time to read our rebuttal and share your follow-up comments. Your suggestions helped us test both points more directly. We would be grateful for any further thoughts you may have on the new results and would be happy to clarify any remaining questions.

Reviewer_rcUbOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper studies the problem of on policy distillation and provides a new algorithm to improve on it. Previous OPD methods used the teacher model mainly to suppress which student tokens should be suppressed and in this work, authors propose a way to allocate this probability mass to other tokens.

Critical strengths

The problem statement is interesting and empirical results seem solid.

Critical weaknesses

My main concern about the paper is the writing. The paper seems to borrow most of the terminology from the blogpost: https://thinkingmachines.ai/blog/on-policy-distillation/. Most terms seem non-standard and I have a hard time following it. If authors use standard ML notation, that would make the paper much more easy to follow.

Questions for authors

Line 118: Is detached credit a standard terminology? 

Line 122: What is sg in sg(A_i)?

Line 127: how did we get equation (2)?

Section 2.1 beginning: can you please highlight what is the full definition of "donor". 

Can you please add an algorithm box to give the full algorithm at one place. Given the new terminology, its a bit hard for me follow.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rcUb [1/2]

Thank you for your careful reading, your positive assessment of the problem and results, and your helpful suggestions on presentation. We apologize for the extra reading effort. We will clarify the terminology, explain symbols at first use, and add a main-text algorithm box.

Method overview. Sampled-token OPD suppresses a sampled token when the student assigns it higher probability than the teacher. We call this token the donor. The compensating local logit update follows the student's current distribution, without explicitly using teacher evidence to choose its recipients. This need not favor candidates that support better subsequent reasoning.

LKR selects recipients using two signals: the teacher's current support for a candidate, and how closely their next-token predictions agree after appending it. LKR preserves the donor's local logit gradient and redistributes only part of the compensating update. Water-filling uses candidate scores and existing student probabilities to recompute shares at the clipped budget.

Q1. "Detached credit" (Line 118)

This is our shorthand, rather than standard terminology, for a log-probability ratio treated as constant during differentiation. For sampled token uiu_i, let pi(ui)p_i(u_i) and ti(ui)t_i(u_i) be its student and teacher probabilities under the same prefix. We define Ai=log⁡ti(ui)pi(ui).A_i=\log\frac{t_i(u_i)}{p_i(u_i)}. It is a scalar coefficient in the surrogate loss. We will replace "detached credit" with "teacher-student log-probability ratio" and explain stop-gradient. We will also simplify other terms so that the method can be understood without consulting the cited blog post.

Q2. Meaning of sg\mathrm{sg} (Line 122)

sg\mathrm{sg} is the stop-gradient operator: it leaves its input unchanged in the forward pass and treats it as constant during backpropagation. Thus, although AiA_i depends on the student probability, we do not differentiate through it in this loss. We will define sg\mathrm{sg} beside the loss at first use.

AuthorsOpenReview ↗

Response to Reviewer rcUb [2/2]

(Continued from Part 1.)

Q3. Derivation of Eq. (2) (Line 127)

Thank you for asking us to explain this step. Fix position ii, its prefix cic_i, and the student-sampled token uiu_i. Let V\mathcal V be the shared vocabulary and vv any token in it. The vector ziz_i contains the student's logits at cic_i, with one entry per vocabulary token. The vectors pi=softmax⁡(zi)p_i=\operatorname{softmax}(z_i) and tit_i are the student and teacher next-token distributions. The one-hot vector euie_{u_i} has the same length: its entry at uiu_i is 1 and all other entries are 0.

Expanding log-softmax gives log⁡pi(ui)=zi(ui)−log⁡∑v∈Vexp⁡zi(v).\log p_i(u_i)=z_i(u_i)-\log\sum_{v\in\mathcal V}\exp z_i(v). Its derivative at coordinate vv is [eui]v−pi(v)[e_{u_i}]_v-p_i(v), so ∇zilog⁡pi(ui)=eui−pi.\nabla_{z_i}\log p_i(u_i)=e_{u_i}-p_i. Let ℓibase=−sg(Ai)log⁡pi(ui)\ell_i^{\mathrm{base}}=-\mathrm{sg}(A_i)\log p_i(u_i) be the unweighted base loss and GibaseG_i^{\mathrm{base}} its gradient with respect to ziz_i. Holding AiA_i constant gives Gibase=−Ai(eui−pi).G_i^{\mathrm{base}}=-A_i(e_{u_i}-p_i). For Ai<0A_i < 0, define γi=−Ai>0\gamma_i=-A_i > 0. This yields Eq. (2): Gibase=γi(eui−pi).G_i^{\mathrm{base}}=\gamma_i(e_{u_i}-p_i). The donor component is γi(1−pi(ui))\gamma_i(1-p_i(u_i)); other components are −γipi(v)-\gamma_i p_i(v). A gradient-descent step in local logits therefore lowers the donor's logit and raises the others.

We will add these definitions and derivation steps to the main text, then explain how signal clipping and nonnegative weights change the effective coefficient γi≥0\gamma_i\ge0. The derivation concerns local logit gradients.

Q4. Definition of "donor"

A donor is the sampled token suppressed when pi(ui)>ti(ui)p_i(u_i) > t_i(u_i), equivalently Ai<0A_i < 0. Section 2.1 defines it immediately after Eq. (1). We will introduce it at first use as "the suppressed sampled token" and retain the mathematical definition in Section 2.1.

Q5. Complete algorithm box and terminology

We appreciate this suggestion. Algorithm 1 in Appendix D.1 gives the procedure:

(a) Generate a student rollout and compute base OPD at all tokens.

(b) Sample up to mm negative-feedback positions uniformly without replacement. At each, take up to kk candidates from the teacher/student Top-20 intersection, excluding the donor.

(c) Score candidates using current teacher probabilities and successor KL, then compute water-filling shares at the clipped budget.

(d) Add the routing correction, weighted by inverse inclusion probabilities, to base OPD and update the student. Treat routing quantities as constants.

We will add this algorithm box to the main text with key parameters.

We will also add a short glossary:

Recipients/routing: recipients receive the compensating update; routing divides it among them.

Base budget: Bi=γi(1−pi(ui))B_i=\gamma_i(1-p_i(u_i)), the base gradient magnitude at the donor.

Successor KL: after appending a candidate, compare the next-token distributions in the student-to-teacher direction, using a coarse approximation.

Water-filling: determines update shares, not actual probabilities after training.

We again apologize for the extra reading effort and will revise the terminology, derivation, and algorithm presentation to make the paper easier to follow. Thank you again for helping us improve the clarity of the paper.

AuthorsOpenReview ↗

Gentle Follow-up on Rebuttal Responses Before the Discussion Deadline

Dear Reviewer rcUb,

Thank you for taking the time to review our paper and for sharing your thoughtful feedback. Your questions helped us clarify several points in our rebuttal.

If you have a chance to read our response, we would appreciate hearing whether it addresses your main concerns or if there are any points you would like us to clarify further. We would be happy to continue the discussion.

Thank you again for your time and consideration.

Best regards, The Authors of Paper # 21017

Reviewer_YNegOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors attempt to redistribute the update magnitude in the optimization process of OPD, intuitively addressing the question of which tokens should receive increased probability at positions where the student is overconfident. Essentially, by introducing two extra information—the teacher model’s top-k token preferences and one-step-ahead teacher–student prediction agreement—the proposed LKR method uses a water-filling allocation to explicitly redistribute part of the OPD update among candidate tokens. LKR demonstrates some performance improvements on the datasets selected in the paper. Through ablation experiments, the authors also show gains from several components of their algorithmic design, providing some evidence for the method’s effectiveness.

Critical strengths
  1. The motivation is intuitively straightforward and the algorithm is not particularly complex, making it accessible for implementation.

  2. The paper is relatively clearly written and communicates both the overall methodological approach and the questions that the experiments aim to address.

Critical weaknesses
  1. Overall performance: Although the method improves performance, the gains are relatively modest, and the limited sizes of some public benchmarks like AIME 24~26 introduce uncertainty into their interpretation. The practical value of these gains needs further discussion in light of the additional computational cost.

  2. Method design:

  • (a) The candidate selection mechanism relies on sufficient overlap between the teacher’s and student’s top-k token sets. At some sequence positions, this overlap may be limited, reducing the availability of routing information.

  • (b) Incorporating the additional information introduces substantial computational overhead. Although the authors provide an approximate training-time-matched comparison, the performance–cost trade-off across different query budgets warrants further evaluation.

  1. Experimental design: Although the paper includes extensive ablation experiments showing benefits from one-step-ahead prediction agreement, it does not directly explain why lower successor teacher–student KL better shapes the learning gradients rather than merely candidates on which the two models make similar predictions.
Questions for authors
  1. Why should greater one-step-ahead teacher–student prediction agreement identify candidates that are more beneficial for learning?

  2. Have the authors considered how differences between teacher and student model families might affect top-k candidate overlap?

3 comments
AuthorsOpenReview ↗

Response to Reviewer YNeg [1/2]

Thank you for your careful review and constructive feedback. We appreciate your positive assessment of the motivation, ease of implementation, and presentation. We added candidate coverage statistics and time-matched ablations to address your concerns.

Q1. Why can one-step-ahead agreement help identify candidates beneficial for learning?

Increasing a candidate token's probability makes the student more likely to sample it at the current position. The remaining response is then generated from the prefix extended by that token. If the teacher currently supports two candidates equally, but the student's next-step prediction is closer to the teacher after one of them, we aim to allocate more of the update to that candidate to favor continuations with better next-step agreement.

The two-step KL decomposition in Appendix B.2 provides the rationale: successor prediction mismatch is part of the two-step distillation objective. LKR therefore combines current teacher probabilities with successor KL to score candidates. Water-filling then determines allocation shares using the student's existing probabilities. This makes the current update account for both the current choice and next-step imitation quality. We will add this explanation to the main text.

We tested this signal in a time-matched ablation. Teacher-only uses current teacher scores and water-filling and skips successor queries for both models. Normalized-score uses the same lookahead scores as LKR and allocates the update by normalizing the scores. All methods used the same three training seeds, with matched initialization and training data, eight GPUs, k=4, and an 8-hour target training budget per run. We evaluated all runs on the same 1,000 held-out questions. The table reports mean ± sample standard deviation across seeds.

\text{Method}&\text{Steps}&\text{Train time (s)}&\text{Teacher successor queries}&\text{Mean@8 (\\%)}&\text{Cons@8 (\\%)}\\\\ \text{Teacher-only}&333.3\pm1.2&28{,}864\pm14&0&25.03\pm0.26&35.23\pm0.25\\\\ \text{Normalized-score}&251.7\pm0.6&28{,}876\pm12&2{,}324{,}578\pm5{,}275&24.84\pm0.22&34.97\pm0.31\\\\ \text{LKR}&250.3\pm0.6&28{,}890\pm14&2{,}314{,}536\pm5{,}233&25.73\pm0.16&36.47\pm0.21 \end{array}$$ **LKR outperformed Teacher-only on both metrics for all three seeds**, even though Teacher-only completed 83 more steps on average. Across matched seeds, the paired gains were **0.70 ± 0.14 percentage points (pp) in Mean@8 and 1.23 ± 0.06 pp in Cons@8**, supporting the use of successor information for candidate selection.
AuthorsOpenReview ↗

Response to Reviewer YNeg [2/2]

Q2. How might model-family differences affect Top-k overlap?

We agree that differences between teacher and student model families may affect Top-k overlap. However, this paper focuses on improving OPD for the specified teacher-student pairs, so we added coverage statistics for our primary model pair.

The models share a vocabulary. Candidates come from their Top-20 intersection, excluding the sampled donor token, with at most k=4 retained. We use the available candidates when fewer than four remain; an empty intersection leaves the base OPD update unchanged.

At the primary pair's frozen step-250 checkpoint, we generated one rollout for each of 128 questions and sampled 2,560 negative-feedback positions. The table describes the raw intersection after removing the donor and before capping it at four candidates. Brackets report [P10, P90].

\text{Metric}&\text{Value}\\\\ \text{Median intersection size}&9\\ [3,14]\\\\ \text{Empty intersection (\\%)}&3.87\\\\ \text{Nonempty, fewer than four (\\%)}&7.89\\\\ \text{At least four (\\%)}&88.24\\\\ \text{Nonempty, positive clipped dose (\\%)}&90.27\\\\ \text{Median student mass on intersection}&0.476\\ [0.104,0.783]\\\\ \text{Median teacher mass on intersection}&0.569\\ [0.138,0.847] \end{array}$$ All rates use the 2,560 sampled positions as their denominator; mass statistics include empty intersections. For this pair at this checkpoint, **96.13% of positions have common candidates and 88.24% fill the four-candidate cap.** Thus, the intersection provides routing candidates for most sampled negative-feedback positions in our setting. We will clarify the study's scope and add these statistics in the revision. In future work, we will also consider cross-family experiments to examine how family differences affect candidate coverage and performance. **Q3. Are the gains worth the additional computation?** Thank you for raising the performance-cost trade-off. We agree that small benchmarks such as AIME should be interpreted alongside larger evaluations. On the 1,000-question Held-out set, **LKR improves Mean@8/Cons@8 over OPD by 2.41/2.27 pp, averaged over three training seeds.** The paper already includes query-budget experiments with m=10,20,40 and an OPD time-matched comparison estimated from per-step runtime. The ablation in Q1 uses measured training times under the same target budget for all three methods. In this ablation, LKR also outperformed Normalized-score on both metrics for every seed. The paired gains were **0.89 ± 0.14 pp in Mean@8 and 1.50 ± 0.10 pp in Cons@8**. The two methods used similar training time and teacher successor query counts, supporting the added value of water-filling at comparable cost. **Thank you again for these helpful suggestions.** We will include the explanations and experiments to better present performance gains and computational cost.
AuthorsOpenReview ↗

Gentle Follow-up on Rebuttal Responses Before the Discussion Deadline

Dear Reviewer YNeg,

Thank you for taking the time to review our paper and for sharing your thoughtful feedback. Your questions helped us clarify several points in our rebuttal.

If you have a chance to read our response, we would appreciate hearing whether it addresses your main concerns or if there are any points you would like us to clarify further. We would be happy to continue the discussion.

Thank you again for your time and consideration.

Best regards, The Authors of Paper # 21017

Comments on the paper

1 comment

Dear all, The discussion phase will end soon. The authors have provided their responses to your comments. If the provided responses do not address (even in part) your concerns, this is your chance to raise them. Please engage with the authors timely. We really appreciate your efforts! Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.