acceptodds
Under review as a conference paper at ICLR 2027

ViableOPD: Guiding On-Policy Distillation with Prefix Viability

Abstract

On-policy distillation (OPD) transfers teacher capabilities by providing dense, token-level teacher supervision on student-generated trajectories. This paradigm implicitly assumes that the entire trajectory can benefit from teacher supervision. However, once a student-generated prefix deviates from the task objective, teacher supervision is conditioned on an erroneous context, making distillation at such states ineffective or even harmful. To characterize when teacher supervision remains effective, we introduce prefix viability, defined as the probability that the original task objective can still be achieved from a student-generated prefix. We estimate prefix viability by normalizing the teacher’s success probability conditioned on the prefix by its success probability from the original input. Based on this criterion, we propose Viability-Guided On-Policy Distillation (ViableOPD), which identifies viable rollouts for teacher supervision, adjusts token-level supervision strength within selected rollouts, and adapts the rollout horizon according to the fraction of viable rollouts. Across multiple benchmarks and teacher–student pairs, ViableOPD consistently outperforms strong OPD baselines while reducing training time by 33.6% on average relative to standard OPD. These results suggest that effective OPD requires prefixes to be not only visited by the student, but also viable for continued supervision.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, Large Language Models, Prefix Viability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GUXdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks at on-policy distillation (OPD), where a teacher gives token-level feedback on whatever the student writes, and points out a hidden assumption nobody really questioned: that every state the student wanders into is still worth teaching at. The authors argue this is often false — once a student's partial answer has drifted so far that the original goal is basically unrecoverable, teacher feedback there is useless or even teaches bad habits. They formalize this with "prefix viability": roughly, the chance the task can still be solved if you continue from the student's current prefix, estimated as the teacher's success rate conditioned on that prefix divided by its success rate from scratch (which controls for task difficulty). Since estimating this online is too expensive, they build a small offline-trained "viability judge" that runs during distillation.

Critical strengths

The core observation: "visited does not mean viable" — is clean, intuitive, and genuinely underexplored.

I really liked the diagnostic experiments in Section 3: showing that viability decays along rollouts (38% of rollouts enter a low-viability zone, ~46% of tokens are generated after that point) is a compelling motivation.

The method itself is sensible and each design choice gets its own ablation (random vs overlap-based selection at matched budget, uniform vs shuffled token weights — the shuffled-weights control is a nice touch).

Critical weaknesses

My main concerns. First, the viability ground-truth labels used to train the judge are themselves Monte-Carlo estimates of teacher success probability — the paper never says how many continuations are sampled per prefix, or how noisy these labels are, and a noisy judge trained on noisy labels could compound errors in subtle ways.

Second, the selection is done at the rollout level, but viability declines within a rollout, so a rollout deemed viable at the end can still contain late tokens that already crossed into the hopeless zone — this seems like an inconsistency with the paper's own framing that is only hand-waved away ("requires viability estimation over multiple prefixes").

Questions for authors
2 comments

Response to Reviewer GUXd

We sincerely thank Reviewer GUXd for the thoughtful and insightful comments. The reviewer raises two important questions concerning the reliability of Monte Carlo-based viability labels and the consistency between prefix-level viability and rollout-level selection. We appreciate the opportunity to further clarify these methodological aspects of ViableOPD and discuss the relevant design considerations and limitations.

1. Reliability of Monte Carlo-Based Viability Labels

Sampling Configuration. As reported in Table 4 of Appendix A, we sample 32 teacher-generated scratch completions per problem and 32 teacher continuations per student prefix to estimate the corresponding task success probabilities. The resulting empirical viability scores are converted into binary labels using the threshold τ_v = 0.75.

Since finite Monte Carlo sampling introduces estimation uncertainty, we further examine whether sampling perturbations change these binary labels by causing empirical viability scores to cross the decision threshold.

Additional Statistical Analysis. We conduct three complementary analyses:

  • Bootstrap Stability. We perform 1,000 bootstrap replicates, independently resampling the 32 continuations of each prefix and the 32 scratch completions per problem. Prefixes from the same problem share the resampled scratch-completion denominator. The average label stability is 97.31%. Based on bootstrap classification frequencies, 52.72% of prefixes have stable positive labels (>95%), 37.12% have stable negative labels (<5%), and 10.16% are ambiguous.

  • Leave-One-Out Sensitivity. Removing each continuation in turn yields an average label-flip rate of only 0.459%, with 98.66% of prefixes retaining their original labels across all 32 evaluations. Label changes are concentrated near the threshold: prefixes within 0.05 of the threshold have an average flip rate of 21.47%, compared with only 0.018% for those at least 0.20 away.

  • Sampling Budget Sensitivity. We evaluate sampling budgets N in {8,16,24,32} by randomly downsampling scratch completions and prefix continuations without replacement, using 10 fixed random seeds. Prefixes from the same problem share the same downsampled denominator. Compared with the full 32-sample labels, the agreement rates for budgets of 8, 16, and 24 are 95.51%, 97.20%, and 98.44%, respectively.

Implications for Viability Judge. Overall, these analyses show that Monte Carlo-based labels are highly stable. The Viability Judge further achieves an AUROC of 0.943 against these empirical labels (Figure 4), while downstream results in Tables 1 and 2 demonstrate the practical effectiveness of Judge-based supervision selection. Although local label noise cannot be entirely ruled out, these findings support the reliability of the proposed supervision signal. We will incorporate the statistical analyses, and relevant discussions into the revised manuscript.

2. Consistency between Prefix-Level Viability and Rollout-Level Selection

Clarification of Prefix Viability. We would like to clarify that prefix viability does not measure whether a student-generated prefix is itself correct, but whether the teacher can still provide meaningful, task-aligned supervision conditioned on that prefix. Thus, even if a student prefix contains reasoning errors, it can remain valuable for distillation as long as the teacher can provide effective corrective guidance. Our goal is to exclude states where accumulated errors have severely compromised the teacher's ability to provide useful supervision, while preserving those that still allow effective correction.

Rationale for Rollout-Level Selection. Our rollout-level selection is primarily motivated by computational efficiency. For a rollout classified as non-viable at its final position, earlier prefixes may still retain substantial supervision value. However, identifying the precise point at which supervision becomes ineffective requires evaluating multiple prefixes in reverse order, from the end of the rollout toward its beginning, thereby introducing additional Viability Judge inference overhead. To avoid this cost, we directly discard the entire non-viable rollout, accepting the loss of potentially useful supervision from its earlier portions. Conversely, we retain the entire rollout when its final prefix is viable, since the teacher can still provide meaningful supervision conditioned on the resulting context. Intermediate reasoning errors do not necessarily invalidate this supervision, as they may still admit effective corrective guidance.

Limitations and Future Work. We acknowledge that rollout-level selection does not explicitly identify the supervision boundary at each intermediate prefix. Exploring finer-grained prefix-level supervision allocation to more precisely identify effective supervision regions is an important direction for future work.

Thanks for you response. i keep my score unchaged.

Reviewer_Tz3TOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper identifies a critical limitation of on-policy distillation (OPD): teacher supervision becomes ineffective or misleading when student-generated trajectories deviate from the original task objective. It introduces the concept of prefix viability to measure the probability that the original task goal remains achievable from a student-generated prefix, and proposes the ViableOPD framework which optimizes supervision allocation through viability-based rollout selection, token-level supervision weighting, and adaptive generation horizon. Evaluated on mathematical reasoning and code generation tasks across multiple teacher-student pairs and benchmarks, the approach achieves better task performance while cutting average online training time by 33.6%.

Critical strengths
  1. This paper cuts to the core of the implicit assumption in on-policy distillation (OPD). The authors point out that student-visited states are not necessarily suitable for teacher supervision, and demonstrate the "visited ≠ viable" phenomenon through diagnostic experiments. The problem formulation is clear and carries practical significance.
  2. The proposed viability metric is well-designed. It normalizes for task difficulty using the ratio of the teacher’s success probability when continuing from a prefix to that from the original input, with an intuitive rationale. The pipeline of offline label construction followed by online deployment of a lightweight judge is also pragmatic.
  3. The method has a clear structure with comprehensive ablation studies. Corresponding analyses are provided for all three components: rollout selection, token-level weighting, and adaptive horizon. Controls such as random selection, overlap-based selection, and weight shuffling lend support to the authors’ claims to a reasonable extent.
  4. The experiments feature broad coverage and consistent results. The authors conduct comparisons across multiple teacher-student pairs, mathematical reasoning and code generation tasks, and multiple benchmarks, achieving state-of-the-art performance on most metrics. The reporting of training time and discussion of offline costs also make the analysis thorough.
Critical weaknesses
  1. The empirical evidence remains insufficient. The authors only report results from a single seed without providing variance or confidence intervals. Some benchmarks have limited scale, and the magnitude of performance gains is modest. The evaluation sample sizes, decoding settings, and pass@1 estimation methodology also need further clarification.
  2. Key control experiments are missing. All existing baselines are OPD variants; controls such as oracle selection based on final correctness, PRM/prefix classifiers, SFT/offline distillation, or RL are absent. It is thus difficult to disentangle whether the gains stem from the specific definition of viability or from generic bad trajectory filtering. Experimental comparisons with the closest concurrent work are also lacking. Although Table 2 claims comparison under a matched supervision budget, it does not report actual selection rates or the number of supervised tokens.
  3. Generalization and implementation details remain inadequate. The experiments only cover small-scale student models, teacher-student pairs largely from the same model family, and verifiable tasks.
Questions for authors
  1. Are all reported experimental results based on a single random seed? If so, have you estimated the variance of performance gains, and are the observed improvements statistically significant?
  2. Table 2 states ablations are conducted under a matched supervision budget. Could you report the actual fraction of selected rollouts and the average number of supervised tokens for each variant, to confirm equal computational comparison?
  3. Could you detail the exact pass@1 calculation protocol, decoding configurations (temperature, top-p), and sample size for each benchmark, to ensure result reproducibility and comparability with prior work?
1 comment

Response to Reviewer Tz3T

We sincerely thank Reviewer Tz3T for the constructive feedback. In response to the concerns regarding empirical robustness, methodological validity, and generalizability, we conducted additional experiments and clarified our methodology and evaluation. Our detailed responses follow.

1. Experimental Robustness and Reproducibility

(1) Multi-seed Evaluation.

We conducted additional experiments with three random seeds using the DeepSeek / JustRL configuration. Results are reported as mean ± standard deviation.

MethodAMC23AIME25HMMT25Time (h)
OPD86.82 ± 0.3934.17 ± 0.4217.43 ± 0.6426.9 ± 1.32
Fixed-Horizon86.25 ± 7.5834.58 ± 3.2518.54 ± 3.8612.1 ± 0.19
Random-Horizon86.72 ± 0.4134.03 ± 1.2617.71 ± 0.2117.7 ± 1.07
Prune-OPD86.04 ± 0.1834.79 ± 0.7419.51 ± 1.5113.2 ± 0.44
PG-OPD86.25 ± 1.7432.29 ± 0.7520.49 ± 1.0315.5 ± 0.25
ViableOPD89.11 ± 0.4835.42 ± 0.2122.15 ± 0.2414.5 ± 0.22

(2) Evaluation Protocol.

  • Pass@1: Mathematical benchmarks use average accuracy over 16 samples per problem; HumanEval+ and MBPP+ use one greedy sample; LiveCodeBench-v6 uses 10 samples with empirical accuracy.
  • Benchmark sizes: AMC23 40, AIME25 30, HMMT25 30, HumanEval+ 164, MBPP+ 378, and LiveCodeBench-v6 1,055.
  • Decoding: Mathematics: temperature = 0.6, top-p = 0.95; HumanEval+/MBPP+: temperature = 0, top-p = 1.0; LiveCodeBench-v6: temperature = 0.2, top-p = 0.95.

2. Additional Baselines and Controlled Experiments

Our work focuses on improving OPD through viability-guided supervision. Accordingly, standard OPD and its variants are the primary baselines for evaluating the effectiveness of our approach. Following the reviewer's suggestion, we additionally introduce Oracle Selection, which retains only rollouts with correct final answers, providing a reference for outcome-based trajectory filtering. We report the results using the DeepSeek / JustRL configuration for brevity.

MethodAMC23AIME25HMMT25Time (h)
Oracle Selection80.0029.7919.3827.5
ViableOPD89.2235.4221.8814.3

Unlike Oracle Selection, which filters rollouts solely by final correctness and discards potentially recoverable trajectories, ViableOPD selects supervision based on prefix viability, preserving opportunities for corrective teacher guidance while achieving better performance with less training time.

3. Supervision Budget and Computational Fairness

Table 2 reports ablations on rollout selection and token-level weighting, with the supervision budget controlled separately for each group.

Prefix Selection. For each batch, Random and Overlap-based Selection select the same number of supervised rollouts as ViableOPD, ensuring matched rollout-level supervision budgets while varying only the selection criteria.

Token Weighting. All variants use identical selected rollouts and supervision regions, differing only in token weights. Uniform Weighting assigns equal weights, whereas Shuffled Weighting randomly permutes the original weights within each rollout, preserving their distribution, non-zero count, and total weight while disrupting their alignment with supervision utility.

These controls isolate the contributions of rollout selection and token weighting. We will additionally report rollout selection ratios and average effective token supervision weights in the revised manuscript. The corresponding plots will be provided there, as OpenReview responses do not support direct image embedding.

4. Generalizability and Scope of Applicability

Regarding model scales and families, prefix viability measures whether a student-generated prefix remains conducive to achieving the task objective, making the underlying supervision principle independent of specific model architectures, parameter scales, or teacher-student configurations. Thus, ViableOPD can be extended beyond the configurations evaluated in this work.

Regarding task generalizability, our current evaluation focuses on verifiable tasks, where teacher continuation success rates provide a reliable signal for estimating prefix viability. For open-ended generation, semantic or preference-based signals from learned reward models or LLM-based evaluators could serve as alternative criteria for assessing whether a prefix remains conducive to the desired objective.

We believe our results demonstrate the effectiveness of viability-guided supervision under the evaluated OPD settings and support its potential applicability to a broader range of models and generation tasks. We will clarify the scope of our empirical evaluation and discuss these potential extensions in the revised manuscript.

Reviewer_To29OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that once the student enters a wrong state from which it would be difficult to recover (outputs a prefix for which the probability of successfully solving the task is low) continuing to supervise with teacher signal is not a good idea since it "encourages imitation of the teacher’s continuation from an erroneous context". They introduce prefix viability to capture this probability from a given prefix. Importantly, they also attempt to control for task difficulty by normalising by the teacher's success when starting from scratch. They use this in a few ways to modify standard on-policy distillation OPD namely: select viable roll-outs, adjust token-level weights, and (adaptively) adjusts the roll-out horizon. Despite the prefix-based motivation the method selects entire rollouts rather than identifying where they go wrong, as Figure 2 suggests.

Critical strengths
  • The motivation for the proposed approach is clear and well articulated.
  • Using a viability above a threshold judge to avoid the expensive teacher roll-out during training is a good idea (but comes at substation upfront cost).
  • Section 3.3 is appreciated, the question if there are local signals that can capture prefix viability is natural, and the negative result justifies the need for a viability judge, although it's not 100% clear if simpler alternatives have been exhausted.
  • Orthogonal to other OPD variants so composable I
Critical weaknesses
  • The motivation in lines 96 to 99 makes sense intuitively but is not backed up by evidence. How do we know that "applying distillation on such states may fail to transfer useful teacher capabilities and even reinforce undesirable generation behaviors"? The improvement over standard OPD provides indirect evidence, but doesn't tell us if supervision on low-viability states is actively harmful or just ineffective.
  • The paper claims "reducing training time by 33.6% on average relative to standard OPD" in the abstract, but AFAIK this excludes the cost of constructing/training the viability judge which would need many training runs to amortise cost (despite the sharing). To be fair, this is acknowledged by the authors and the break-even points are discussed.
  • The existing ablation study is useful, but it would be better to isolate each introduced component including adaptive horizon.
  • Since token-level weighting is a component of the approach and a common idea in OPD variants, it would be good to compare against some existing methods. For example, the paper already cites token teachability (Wang et al., 2026) and entropy-aware OPD (Jin et al., 2026), but there are others.
  • Reports only pass@1.
Questions for authors
  1. Can you provide evidence that low-viability supervision is harmful or ineffective, and explain the mechanism behind the improvements?
  2. Can you add some token-level weighting baselines (see weaknesses)?
  3. Can you expand the ablation (see weaknesses)?
  4. Since the proposed method seems composable with other OPD variants, could you test whether combining them leads to further gains (e.g., pick a recent method that achieves strong improvements in your setting)?
  5. Can you report pass@k and mean@k in addition to pass@1 for various reasonable values of k?
  6. Given the judge construction cost, could you compare against compute-matched OPD and other baselines (in terms of total GPU-hours), for example by spending the extra compute on more on-policy rollouts or additional training epochs?
2 comments

Response to Reviewer To29

We thank the reviewer for the thoughtful feedback. We conducted additional experiments and analyses to strengthen the empirical validation and clarify our contributions.

1. Effectiveness of Viability-Guided Supervision

We introduce an Oracle-Incorrect baseline that distills only from rollouts with incorrect final answers, to test whether supervising unsuccessful trajectories is detrimental. We evaluate it on DeepSeek/JustRL.

MethodAMC23AIME25HMMT25Time (h)
OPD86.8834.1716.8826.6
Oracle-Incorrect85.1632.5015.8327.0
ViableOPD89.2235.4221.8814.3

Oracle-Incorrect underperforms OPD on all benchmarks, indicating that indiscriminate distillation from unsuccessful trajectories can be harmful. ViableOPD achieves the best results by excluding low-viability regions.

2. Comparison with Existing Token-Level Weighting Methods

We compare ViableOPD with two representative token-level weighting methods: TA-OPD (token teachability) and EOPD (entropy-aware weighting). Both are evaluated under the same experimental settings. For brevity, we report results only for DeepSeek/JustRL.

MethodAMC23AIME25HMMT25Time (h)
EOPD86.8833.7517.7130.4
TA-OPD86.8833.5416.0428.9
ViableOPD89.2235.4221.8814.3

ViableOPD consistently outperforms both token-level weighting baselines across all benchmarks while requiring substantially less training time.

3. Expanded Ablation Studies

We further isolate the contribution of Adaptive Horizon on DeepSeek/JustRL using two controlled schedules: Fixed Schedule, with a constant cap equal to the rounded mean adaptive cap; and Shuffled Schedule, which permutes the adaptive cap sequence while preserving its mean and empirical distribution.

MethodAMC23AIME25HMMT25Time (h)
Fixed Schedule87.6633.7519.1715.1
Shuffled Schedule87.1933.3319.5814.6
ViableOPD89.2235.4221.8814.3

ViableOPD outperforms both controls across all benchmarks, supporting the benefit of adapting the generation horizon during training.

4. Compatibility with Existing OPD Variants

We combine ViableOPD with EOPD to obtain ViableEOPD and evaluate the combination on DeepSeek/JustRL.

Performance Comparison (Pass@1).

MethodAMC23AIME25HMMT25Time (h)
ViableEOPD87.3434.3719.7914.9
ViableOPD89.2235.4221.8814.3

Generation Diversity (Unique Answers per Problem).

MethodAMC23AIME25HMMT25
ViableEOPD2.506.246.60
ViableOPD2.105.936.33

ViableEOPD achieves competitive distillation performance while retaining the generation diversity encouraged by EOPD's entropy-aware weighting, demonstrating that viability-guided supervision can be integrated with existing OPD variants.

5. Extended Evaluation with Pass@k and Mean@k

We report Pass@k and Mean@k for k = 1, 4, and 8 on DeepSeek/JustRL.

MethodAMC23 (P1/M1/P4/M4/P8/M8)AIME25 (P1/M1/P4/M4/P8/M8)HMMT25 (P1/M1/P4/M4/P8/M8)
OPD86.88/82.50/94.06/87.50/94.90/87.1934.17/36.67/45.52/35.83/49.97/33.3316.88/20.00/27.08/17.50/35.22/18.33
Fixed-Horizon86.41/85.00/93.82/87.50/94.63/86.5632.94/40.00/46.61/33.33/52.69/35.4218.75/16.67/29.67/19.17/30.51/18.75
Random-Horizon87.19/85.00/93.27/86.25/94.41/85.6234.17/33.33/45.06/31.67/48.54/33.3317.71/16.67/28.89/15.83/33.11/17.08
Prune-OPD85.94/82.50/93.82/86.88/94.75/85.6234.79/36.67/44.77/36.67/49.45/33.3318.75/20.00/29.22/18.33/34.41/18.75
PG-OPD86.56/87.50/94.13/86.25/94.97/86.2532.50/36.67/45.93/34.17/51.52/35.0020.00/20.00/29.94/19.17/35.54/19.58
ViableOPD89.22/90.00/94.80/88.12/95.06/87.8135.42/43.33/49.31/37.50/55.53/37.0821.88/30.00/30.13/21.67/36.32/20.83

Within each benchmark, values are ordered as Pass@1/Mean@1/Pass@4/Mean@4/Pass@8/Mean@8.

6. Compute-Matched Evaluation

We perform a compute-matched comparison on OpenMath/JustRL. ViableOPD uses 120.8 GPU-hours for online training. Amortizing the Judge's approximately 500 GPU-hour construction cost over 11 runs adds 45.455 GPU-hours per run, giving 166.26 GPU-hours in total. We extend OPD to 279 steps for a matched budget of 166.20 GPU-hours.

MethodAMC23AIME25HMMT25GPU-hours
OPD93.9160.2127.50166.20
ViableOPD96.1461.1230.71166.26

ViableOPD consistently outperforms compute-matched OPD across all three benchmarks, demonstrating that its improvements cannot be attributed simply to differences in computational expenditure.

Thank you. This addressed all of my questions. I maintain my positive score!

Comments on the paper

1 comment

General Response to Reviewers

We sincerely thank all reviewers for their thorough evaluation and constructive feedback. The reviewers raised valuable questions regarding the methodological design, experimental rigor, evaluation fairness, and generalizability of ViableOPD. Their comments have helped us further clarify the presentation of our approach and improve the completeness of our experimental evaluation. In response, we have conducted additional experiments, expanded the reporting of experimental results and evaluation protocols, and provided further clarifications on several key aspects of our method. The major additions and clarifications are summarized below.

1. Expanded Experimental Validation of Method Effectiveness and Component Contributions

  • Validation of the Core Motivation: We conducted additional experiments to investigate whether supervising low-viability states can be detrimental, providing further empirical evidence for the relationship between prefix viability and supervision quality.
  • Comparison with Existing Token-Level Weighting Methods: We added comparisons with two existing token-level weighting methods, TA-OPD and EOPD, to further evaluate the effectiveness of ViableOPD against alternative OPD variants.
  • Compatibility with Existing OPD Methods: We evaluated the integration of ViableOPD with EOPD to investigate whether viability-based supervision can be effectively combined with existing strategies.
  • Component-Wise Ablation: We conducted an additional ablation study specifically isolating the contribution of Adaptive Horizon, providing a more targeted assessment of the adaptive generation-length strategy.
  • Compute-Matched Comparison: We introduced a compute-matched OPD baseline to control for differences in computational budgets and ensure a fairer comparison of method effectiveness.
  • Robustness across Random Seeds: We added multi-seed experiments to assess the stability and robustness of the observed performance gains.
  • Outcome-Based Oracle Selection Baseline: We introduced an oracle selection baseline based on the final correctness of generated rollouts to examine the differences between outcome-based filtering and viability-based supervision.

2. Improved Experimental Reporting and Evaluation Rigor

  • Evaluation across Multiple Sampling Budgets: We additionally reported Pass@k and Mean@k results for k=1,4,8k = 1, 4, 8, providing a more comprehensive evaluation of performance under different sampling budgets.
  • Detailed Evaluation Protocol: We clarified the computation procedure for the primary metric, Pass@1, including the decoding configurations (temperature and top-pp) and the number of evaluation instances for each benchmark.
  • Clarification of Ablation Fairness: We provided additional details on the budget-control protocols used across different ablation groups, clarifying the distinct controls applied to supervision budgets and comparison conditions to ensure fair and interpretable comparisons.

3. Further Analysis of Methodological Design, Applicability, and Limitations

  • Generalizability and Scope of Applicability: We further discussed the limitations of the current experimental scope and the extent to which the applicability of ViableOPD to larger models, cross-family distillation, and open-ended generation can be established.

  • Reliability of Monte Carlo-Based Viability Labels: We further discussed the potential noise in Monte Carlo-based viability labels and its implications for the reliability of Viability Judge training and subsequent supervision selection.

  • Consistency between Prefix-Level Viability and Rollout-Level Selection: We additionally discussed the methodological rationale, design considerations, and potential limitations of the rollout-level selection strategy.

Overall, these additional experiments, methodological clarifications, and discussions strengthen the empirical evidence supporting ViableOPD, improve the clarity of its methodological presentation, and provide a more transparent account of its applicability and limitations. We believe these additional efforts will contribute to a more rigorous and complete presentation of the work. Detailed responses to each reviewer’s individual concerns are provided in the corresponding reviewer-specific comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.