acceptodds
Under review as a conference paper at ICLR 2027

Rank What Matters: On-Policy Ranking Distillation for Decision Structure Transfer

Abstract

On-policy distillation (OPD) queries a teacher on student-generated trajectories, aligning teacher supervision with the prefixes encountered by the student. Standard OPD objectives express this supervision through numerical differences between token probabilities, while recent variants refine the distribution regions or token positions. At each prefix, however, these probabilities jointly define a local decision structure: which candidates compete and how strongly the teacher prefers one over another. To make this structure explicit, we introduce Rank-OPD, which selects decision-relevant relations, adapts their required separation to teacher preference strength, and complements them with candidate-set mass control. We further show that this ranking mechanism can serve as a plug-and-play component in distribution-matching OPD and propose Rank-OPD-T, which converts relation violations into bounded probability transport and combines the resulting ranking-refined dense target with candidate-set mass matching. Across eight mathematical reasoning benchmarks, Rank-OPD and Rank-OPD-T achieve average scores of 42.01% and 43.69%, respectively, compared with 41.19% for standard OPD. Rank-OPD-T obtains the best score on seven benchmarks. Mechanistic analyses show that ranking supervision corrects preference violations while preserving plausible alternatives, and ablations support both selective relational supervision and distribution target refinement. These results demonstrate that local ranking structure provides effective supervision for OPD.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, LLM post-training

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_vfFmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper does on-policy distillation on prefixes sampled by the student, but replaces numerical next-token matching with local ranking of the teacher's Top-K. Rank-OPD keeps only head–head and head–tail pairs, sets the margin from the teacher's preference strength, and adds a candidate-set mass constraint. Rank-OPD-T turns under-margin pairs into a bounded probability transport and matches the resulting dense target. Experiments distill Qwen3-4B-Instruct into Qwen3-1.7B-Non-Thinking on DAPO-Math-17K and evaluate eight math benchmarks.

Critical strengths

1.Rank-OPD-T has a clear mean gain, and not all of it is dense matching. Table 1 raises the eight-benchmark average from 41.19 (OPD) to 43.69 (Rank-OPD-T), best on seven of eight sets. On the four-benchmark subset, ρ=0 scores 38.86 and ρ=0.05 scores 40.15. The teacher stays frozen and inference uses only the student. This is consequential because it is the main empirical support for replacing OPD's numerical next-token target with a ranking-corrected one.

2.The Rank-OPD pieces can be checked separately. Mass-only 34.70, HH+HT without mass 36.66, and the full objective 38.06 on the same subset; a large fixed margin is worse than the adaptive one. That supports the design as a set of choices rather than a single inseparable recipe.

Critical weaknesses

1.The main-table gain stacks two changes. Reported OPD is sampled-token k1 reverse KL; Rank-OPD-T is dense candidate-set matching plus transport. The +2.50 therefore mixes sparse-versus-dense matching with ranking correction. Standalone Rank-OPD is only +0.28 over EOPD and is lower on AIME24, AIME25, MATH-500, and HMMT Nov. This is consequential because the claim that local ranking is useful supervision is much stronger for Rank-OPD-T than for Rank-OPD, and the headline comparison does not isolate the ranking part.

2.Rank-OPD-T is not shown as a plug-and-play component of existing OPD objectives. The text describes it as a drop-in correction for distribution-matching OPD, but it is trained as a standalone target and is never attached to EOPD or AOPD. All reported results also stay on one Qwen3 4B→1.7B pairing and math contest sets. This is consequential because the evidence then supports a recipe in this setting more directly than a general change to OPD supervision.

Questions for authors

1.Does Rank-OPD constrain the order of tokens in the generated response, or only pairwise next-token scores within the teacher's Top-K at the current prefix? Concretely, is a Rank-OPD-T violation only a pair whose student logit gap is still below the adaptive margin?

2.Table 1's reported OPD run is sampled-token k1 reverse KL, while Rank-OPD-T is dense matching on the candidate set plus transport. Can the main table include a matched FKL / ρ=0 row? If the claim that local ranking is useful supervision rests mainly on Rank-OPD-T, how much margin remains for standalone Rank-OPD against EOPD?

3.Rank-OPD-T is described as a plug-and-play component for distribution-matching OPD. Is there a result inserting the same transport into EOPD or AOPD under the same budget? If not, should the claim be restricted to this standalone target?

7 comments

We sincerely thank you for the thoughtful review and for recognizing the empirical improvements and component analyses of our methods. We appreciate the constructive comments and address each question below.

Clarification of Local Ranking Supervision (Question 1)

Thank you for raising this concern. Rank-OPD constrains the relative scores of alternative next tokens for the same position. At each student-generated prefix, both candidates in a pair come from the teacher’s top-KK set. As described in Section 3.1, we retain head–head (HH) and head–tail (HT) pairs.

Rank-OPD-T uses the same violation criterion as Rank-OPD. For a retained pair (i,j)(i,j) where the teacher prefers ii to jj, a violation occurs precisely when the student’s logit gap is below the teacher-gap-adaptive margin, gt,ijS<mt,ijg^S_{t,ij} < m_{t,ij}. This includes both reversed preferences and correctly ordered pairs with insufficient separation. A gap equal to the margin satisfies the constraint.

A violated pair contributes to the pairwise hinge loss in Rank-OPD and triggers bounded probability transport in Rank-OPD-T. A pair that meets the margin contributes zero hinge loss and triggers zero transport of its own.

Matched No-Transport Baseline and Standalone Rank-OPD vs EOPD (Question 2 & Weakness 1)

Thank you for the suggestion. We provide the comparisons between the forward-KL baseline and our methods in Table 1. The results show that Rank-OPD-T outperforms it by 3.16% in macro-average accuracy. This comparison supports the contribution of ranking-guided transport within the same dense-matching setup. We will include this baseline in the main table of the revised manuscript.

Table 1. Additional comparison with the FKL baseline on eight mathematical reasoning benchmarks.

MethodAIME 2024AIME 2025AIME 2026MATH-500AMC 2023OlympiadBenchHMMT Feb 2026HMMT Nov 2025Macro Avg
FKL (no transport)33.3327.7123.7582.4574.5351.1119.8911.4540.53
Rank-OPD35.4223.7525.0086.6072.0357.4921.0214.7942.01
Rank-OPD-T36.0427.0826.4687.4576.2558.1221.2116.8843.69

Regarding the comparison between Rank-OPD and EOPD, we'd like to clarify that Rank-OPD was initially designed as a baseline to explore whether the teacher’s local ranking structure could provide useful supervision for OPD. Surprisingly, we found that this simple design is effective: without dense distribution matching, this ranking baseline achieves performance comparable to EOPD, a strong OPD variant that additionally requires full-vocabulary teacher entropy computation at every token position, with significantly higher training efficiency, as shown in Table 2. Building upon these findings, we developed Rank-OPD-T, which further strengthens distribution matching with a ranking-refined teacher target , achieving the highest macro-average accuracy among all evaluated OPD variants.

Table 2: Training time, peak GPU memory, and macro-average accuracy for Rank-OPD and EOPD.

MethodTraining TimePeak GPU Memory (GiB)Macro Avg. Accuracy (%)
Rank-OPD6h 04m 59s21.4042.01
EOPD6h 30m 00s27.9241.73

Applicability to Existing OPD Methods (Question 3)

Table 3. Accuracy (%) of EOPD and EOPD-T on eight mathematical reasoning benchmarks.

MethodAIME 2024AIME 2025AIME 2026MATH-500AMC 2023OlympiadBenchHMMT Feb 2026HMMT Nov 2025Macro Avg
EOPD35.8324.1722.7186.8571.7257.4219.3215.8341.73
EOPD-T34.1724.7923.5486.9074.0658.2021.4016.0442.39

Following your suggestion, we apply our ranking-guided transport mechanism to EOPD, yielding EOPD-T. As shown in Table 3, EOPD-T outperforms EOPD on seven of the eight benchmarks and improves the macro-average accuracy from 41.73% to 42.39% (+0.66 pp), with only a slight drop on AIME 2024. This suggests that our ranking-guided transport mechanism can generalize beyond vanilla OPD.

Reviewer_HSMpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes two methods for transferring token preferences during on-policy distillation. Rank-OPD applies selective pairwise ranking constraints, while Rank-OPD-T adjusts the teacher distribution through bounded probability transport. In a Qwen3 teacher–student setup, Rank-OPD-T improves average performance across eight math benchmarks from 41.19 to 43.69.

Critical strengths

Learning important relative preferences instead of matching every teacher probability is a reasonable motivation, and the proposed objectives are clearly specified. The paper includes useful component analyses. In particular, the matched transport ablation provides some direct evidence that target adjustment contributes to the gains.

Critical weaknesses

The contribution needs a clearer comparison with existing ranking distillation. TSD-KD already uses teacher-ranked token candidates on student-generated trajectories, together with selective distribution matching [1]. Ding et al., which the paper does cite, also transfer preferences over teacher-selected top-k tokens [2]. The selective pairwise constraints, adaptive margins, and bounded transport remain distinct design choices, but the paper does not establish their advantage over these close alternatives.

The headline improvement mixes several changes to supervision. The main OPD baseline uses sampled-token reverse KL, whereas Rank-OPD-T uses dense top-k target matching and a probability-mass constraint. The authors do provide a useful matched control: on four benchmarks, disabling transport gives 38.86 and enabling it gives 40.15. However, this control is not reported across all eight benchmarks, and only one training seed is reported. The full 2.50-point gain therefore cannot be attributed to ranking-based transport alone, and the stability of its incremental gain remains unclear.

[1] Kim and Baek. Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation. ICLR, 2026.

[2] Ding et al. Distilling Large Language Models for Code Generation via Ranking Supervision. Neural Networks, 205, Part B, Article 109437. DOI: 10.1016/j.neunet.2026.109437.

Questions for authors
  1. How does the method compare with TSD-KD under the same teacher, student, data, and training budget? Which design difference accounts for the improvement?
  2. Can you report the matched no-transport baseline on all eight benchmarks and check the incremental gain across multiple training seeds?
5 comments

We sincerely thank you for the thoughtful review and for recognizing the motivation and empirical value of our proposed methods. We appreciate the constructive suggestions and address each question below.

Comparison with TSD-KD (Question 1 & Weakness 1)

Table 1. Additional comparison with TSD-KD.

MethodAIME 2024AIME 2025AIME 2026MATH-500AMC 2023OlympiadBenchHMMT Feb 2026HMMT Nov 2025Macro Avg
TSD-KD [1]27.9223.3320.0083.4063.1353.1916.8611.2537.38
Rank-OPD35.4223.7525.0086.6072.0357.4921.0214.7942.01
Rank-OPD-T36.0427.0826.4687.4576.2558.1221.2116.8843.69

We thank you for suggesting a direct comparison with TSD-KD [1]. To address your concern, we evaluate TSD-KD with the same configurations as in our main experiments. As shown in Table 1, both Rank-OPD and Rank-OPD-T achieve higher accuracy than TSD-KD on all eight benchmarks. One design difference that may help explain this pattern is where each method applies explicit ranking supervision. Specifically, TSD-KD uses an entropy-defined opener to provide explicit ranking guidance at early reasoning steps, while Rank-OPD checks selected teacher preference pairs throughout the student rollout. A pair triggers ranking feedback wherever the student’s logit gap falls below the margin set by the teacher’s preference strength. As the student evolves during training, the positions that receive this feedback can change.

In Section 5.3 of our paper, we illustrate three cases in which active pairs transfer the teacher’s local decision structure. They reinforce an agreed preference when the student’s gap is too small, correct a reversed ordering, and require only modest separation among similarly plausible continuations. We believe that this adaptive ranking feedback may help explain the gains over TSD-KD. We appreciate your suggestion and will include the TSD-KD results in our main table.

References

[1] Kim and Baek. Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillation. ICLR 2026.

Matched No-Transport Baseline and Seed Variability (Question 2)

No-Transport Baseline. Thanks for your suggestions. In Table 2, we present the results of the forward-KL baseline without transport (ρ=0\rho=0) on all eight benchmarks. Our Rank-OPD-T demonstrates significant improvements over it by over 3% in average accuracy. We will include these results in our main table.

Table 2. Additional FKL (no transport) baseline and Rank-OPD-T accuracy (%) across three training seeds.

MethodAIME 2024AIME 2025AIME 2026MATH-500AMC 2023OlympiadBenchHMMT Feb 2026HMMT Nov 2025Macro AvgΔ\Delta
FKL (no transport)33.3327.7123.7582.4574.5351.1119.8911.4540.530
Seed 136.0427.0826.4687.4576.2558.1221.2116.8843.69+3.16
Seed 235.2125.8326.8887.7577.5058.0119.3213.1342.95+2.42
Seed 332.7127.2927.0887.5077.6659.6119.1315.8343.35+2.82
Mean ± sample SD————————43.33 ± 0.37+2.80 ± 0.37

Results with different seeds. We present the results of Rank-OPD-T with different random seeds in Table 2. Rank-OPD-T consistently outperforms the FKL baseline by 2.42-3.16 points. These results confirm that the advantage over FKL is stable and robust to random initialization.

We hope our responses address your concerns. Please feel free to ask any further questions or request additional clarification.

Thank you for the detailed responses and the additional experiments. I appreciate the effort you put into addressing my concerns. The TSD-KD comparison and the full eight-benchmark no-transport baseline are helpful.

I have one remaining question about the explanation for the gains over TSD-KD. How much of the improvement comes from applying ranking supervision throughout the rollout, rather than mainly in the opener? For example, have you tried restricting Rank-OPD’s ranking supervision to the opener defined by the same rule as TSD-KD, while keeping other settings unchanged? This could help clarify the role of supervision coverage in your method. Any existing analysis on this point would also be helpful.

Reviewer_j8usOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a limitation in current On-Policy Distillation (OPD) for autoregressive large language models. Traditional OPD methods minimize numerical discrepancies between teacher and student token probabilities, ignoring the underlying local decision structure (i.e., which tokens are actively competing and how strongly the teacher prefers one over another) . To make these relative preferences explicit, the authors propose two ranking-aware OPD frameworks : • Rank-OPD: Partitions the teacher's Top-K candidates into a "head" and "tail" using a cumulative-mass top-p rule . It extracts head–head and head–tail relations, optimizes them using a teacher gap-adaptive pairwise margin loss, and uses a candidate-set mass control term to prevent probability mass from drifting outside the candidate set . • Rank-OPD-T: Extends this mechanism into distribution-matching OPD methods . It converts margin violations into a bounded probability transport from lower-ranked donor tokens to teacher-preferred recipient tokens, creating a ranking-refined dense target distribution . Evaluating on eight mathematical reasoning benchmarks (including AIME, AMC, and OlympiadBench) using a Qwen3-4B teacher and a Qwen3-1.7B student, Rank-OPD and Rank-OPD-T achieve average accuracy scores of 42.01% and 43.69% respectively, outperforming standard OPD (41.19%) and other competitive baselines .

Critical strengths

• Novel Structural Insight: Shifting the paradigm of knowledge distillation from matching exact numerical token allocations to explicitly transferring the teacher's internal local preference/ranking structures is conceptually elegant and highly intuitive . • Strong Empirical Gains: The proposed methods consistently outperform standard OPD, SFT, GRPO, and recent advanced variants (EOPD, AOPD) across a broad array of competitive mathematical benchmarks . Rank-OPD-T leads on 7 out of 8 benchmarks . • Exploration-Friendly Behavior: The mechanistic analysis proves that focusing on ranking relations yields a student policy with higher early entropy . This preserves space for student exploration during rollout training rather than forcing early distribution collapse . • Extensive Ablations: The ablation experiments carefully isolate the performance contributions of separate components, including head/tail relation combinations, fixed vs. adaptive margins, and the exact probability transport budget (ρ\rho) .

Critical weaknesses

• Computational Overhead: Because pairing candidates scales quadratically (O(K2)O(K^2)) prior to filtering, constructing relation graphs, establishing adaptive margins, and resolving linear mass transports introduces distinct implementation and runtime overhead over vanilla token-level divergences . • Lack of Global Order Preservation: As highlighted in the supplementary analysis (Section D.7), the violation-guided transport in Rank-OPD-T does not guarantee that overall teacher ordering is maintained globally . Transporting mass to resolve a single violation can inadvertently flip the relative order of other valid non-violated pairs . • Domain and Architecture Narrowness: The evaluations are tightly focused on the mathematical reasoning domain using a single teacher-student pair (Qwen3-4B to Qwen3-1.7B Non-Thinking) . It remains unverified how well local ranking transfer scales to open-ended text generation, creative writing, or much larger model sizes.

Questions for authors
  1. I wonder whether the comparison results are based on the same amount of computation overhead, given the pairwise loss construction.
  2. When there is a significant gap between teacher and student distributions, does the proposed loss still work effectively in the cold-start scenario, or just the "Thinking" mode that produces much diversified content?
  3. Scaling Candidate Size (KK): In the setup, the candidate truncation parameter is set to K=16K=16 . If KK is widened significantly (e.g., K=64K=64 or K=128K=128) to capture broader distributions, how severely does the quadratic pair construction degrade training throughput, and does performance plateau?
  4. Addressing Order Reversals: Given that the directional transport in Rank-OPD-T can accidentally reverse non-violated token orders (as detailed in Section D.7) , did you test alternative transport schemes—such as bounding transport strictly between adjacent ordinal ranks—to prevent global layout distortions?
7 comments

We sincerely thank you for the thoughtful review and for recognizing the novelty and effectiveness of our proposed methods. We appreciate the constructive comments and address each question below.

Computational Overhead and Fair Comparison (Question 1 & Weakness 1)

Table 1: Training time, peak GPU memory, and macro-average accuracy of different methods using a Qwen3-4B-Instruct-2507 teacher and a Qwen3-1.7B-Non-Thinking student.

MethodTraining TimePeak Memory (GiB)Macro Avg
Rank-OPD6h 04m 59s21.4042.01
EOPD6h 30m 00s27.9241.73
OPD5h 28m 03s20.9041.19

We thank you for raising this important concern. As shown in Table 1, Rank-OPD adds only a small computational overhead relative to vanilla OPD (about 37 minutes of additional total training time), yet it consistently improves macro-average accuracy across eight benchmarks. Moreover, compared with SOTA methods such as EOPD, our proposed Rank-OPD achieves better overall performance with significantly higher training efficiency. In addition, we note that all methods are trained under the same experimental settings, including the teacher–student configuration, training data, batch size, and training steps. In summary, these results suggest that our method achieves a good trade-off between efficiency and performance.

Robustness to Teacher–Student Distribution Gaps (Question 2)

We thank you for this important question. Actually, the teacher-student gaps in our experiments are already significant. As shown in Table 2, the average performance of the teacher is higher than the student by 37% in average.

Table 2: Initial accuracy gap between the Qwen3-4B-Instruct-2507 teacher and Qwen3-1.7B-Non-Thinking student across eight benchmarks.

ModelAIME24AIME25AIME26MATHAMC23Olymp.HMMT Feb26HMMT Nov25Macro Avg
Teacher (4B)61.8846.6751.4694.5593.5974.9631.4443.1362.21
Student (1.7B)11.888.1310.4272.7046.5642.176.253.7525.23
Gap+50.00+38.54+41.04+21.85+47.03+32.79+25.19+39.38+36.98

To further address your concern, we conducted additional experiments using a 0.6B student with the same 4B teacher, further increasing the teacher–student capacity gap. As shown in Table 3, the trends are consistent with our main experiments: Rank-OPD-T performs best on average across the eight benchmarks. These findings further support the effectiveness of our method under larger teacher–student capacity gaps. Additionally, we clarify that both the teacher and student operate in non-thinking mode throughout our experiments. Therefore, the observed improvements do not depend on thinking-mode generation.

Table 3: Accuracy (%) of different methods using a Qwen3-4B-Instruct-2507 teacher and a Qwen3-0.6B student across eight benchmarks.

MethodAIME24AIME25AIME26MATHAMC23Olymp.HMMT Feb26HMMT Nov25Macro Avg
OPD11.4616.2512.9273.6551.4143.217.955.8327.84
AOPD14.5814.5813.9669.1050.1639.435.876.2526.74
EOPD12.2917.7113.3367.4556.4138.807.397.0827.56
Rank-OPD10.8316.6712.9275.7051.0941.887.776.2527.89
Rank-OPD-T15.2116.6715.0076.4556.0945.4710.806.8830.32

Scalability with Larger Candidate Sets (Question 3)

We thank you for this valuable question. To address your concern, we conducted additional ablation studies with different values of KK, and present the results in Table 4.

Table 4: Training time, peak GPU memory, and macro-average accuracy of Rank-OPD under different candidate-set sizes.

Top-KTraining TimePeak Memory (GiB)Macro Avg
85h 46m 42s21.3741.38
166h 04m 59s21.4042.01
326h 13m 11s21.4941.14
646h 28m 09s21.6140.63

Performance sensitivity. The results show that K=16K=16 achieves the best performance among the evaluated settings. A smaller candidate set (K=8K=8) may exclude useful preference relations, resulting in fewer effective pairwise constraints. Conversely, increasing KK to 32 or 64 introduces more low-probability candidates and potentially noisy pairwise relations. This aligns with our central motivation: Rank-OPD aims to transfer the teacher's decision-relevant local ranking structure rather than exhaustively enforce all candidate relations. Expanding the candidate set may introduce less informative constraints, diluting supervision on the preferences that matter most.

Computational efficiency. Despite the quadratic growth in potential pairwise comparisons, our selective relation construction excludes tail–tail pairs, limiting the practical overhead. Increasing KK from 16 to 64 adds only 23 minutes and 10 seconds to total training time, with minimal additional GPU memory usage.

Overall, these results suggest that K=16K=16 provides the best balance between effective ranking supervision and computational efficiency among the tested configurations. Larger candidate sets introduce additional computational overhead without improving performance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of reviewers feedback and authors response

We thank all reviewers for their constructive critiques and thoughtful engagement with our work. We appreciate their recognition of the value of transferring local token preferences in on-policy distillation, the strong empirical performance of Rank-OPD-T, and the careful component analyses and ablation studies supporting our design choices. Their feedback has been invaluable in strengthening the experimental evidence and clarifying the scope of our contributions.

In response to the reviewers’ concerns, we conducted additional experiments and analyses:

  • Efficiency and transport behavior. We measured training time and peak GPU memory, examined candidate-set sizes from K = 8 to 64, quantified ordering reversals under the default transport budget, and evaluated an adjacent-rank transport variant. These analyses clarify the practical computational overhead and the trade-offs involved in selecting candidates and refining teacher targets.
  • Comparisons and attribution of improvements. We added TSD-KD and a matched no-transport FKL baseline across all eight mathematical reasoning benchmarks. Rank-OPD and Rank-OPD-T outperform TSD-KD on all eight benchmarks. Rank-OPD-T improves macro-average accuracy over the matched FKL baseline from 40.53% to 43.69%, providing direct evidence for the contribution of ranking-guided transport beyond dense distribution matching. Across three training seeds, Rank-OPD-T achieves 43.33 ± 0.37%, consistently exceeding the reported FKL baseline.
  • Generalization and applicability. We extended evaluation to a 0.6B student and three benchmarks beyond mathematical reasoning: GPQA-Diamond, LiveCodeBench v6, and BBH. These experiments support the effectiveness of our approach under a larger teacher–student capacity gap and across additional domains. We also incorporated ranking-guided transport into EOPD and AOPD, improving their macro-average accuracy by 0.66 and 1.46 percentage points, respectively.

In the revised manuscript, we incorporated the following additions and updates:

  • Section 5.2, Table 1: Comparisons with TSD-KD and the matched no-transport FKL baseline.
  • Appendix C.1: Generalization to a Smaller Student.
  • Appendix C.2: Out-of-Distribution Generalization, including the additional FKL baseline.
  • Appendix C.3: Applying Ranking-Guided Transport to EOPD and AOPD.
  • Appendix C.4: Robustness Across Training Seeds.

We also refined some definitions and descriptions of our methods. All revisions will be highlighted in blue.Through these revisions, we believe the manuscript now provides stronger evidence for the effectiveness, robustness, and applicability of ranking-aware on-policy distillation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.