acceptodds
Under review as a conference paper at ICLR 2027

Repair to Teach: Root-Cause-Guided On-Policy Distillation for Mathematical Reasoning

Abstract

On-policy self-distillation (OPSD) uses reference solutions as privileged hindsight to supervise student-generated reasoning trajectories. However, reference-based guidance may explain a correct solution without addressing why the student's own reasoning fails. This reasoning mismatch between the guidance provided and the correction needed can encourage the student to borrow correct conclusions while leaving its reasoning errors unresolved. Moreover, applying the same hindsight throughout the trajectory risks a distillation trap, where unnecessary constraints on valid reasoning compete with correction of substantive errors. To address these issues, we propose Root-Cause-Guided On-Policy Distillation (RC-OPD), which uses repairs of the student's own reasoning to provide guidance that addresses its specific errors while building on valid progress. For each failed attempt, RC-OPD locates the earliest substantive error, develops a local correction, and uses the corrected intermediate result as an anchor for the valid prefix. An iterative diagnosis-repair-continuation process tests the repairs through student continuation, identifying further errors within a fixed repair budget. For repair chains that reach a correct answer, root-cause-guided distillation uses failure diagnoses and corrective goals to supervise the erroneous segments, while anchor-guided distillation supports the corresponding valid prefixes with reasoning chains leading to the repaired intermediate results. We evaluate RC-OPD across multiple datasets and model scales. Extensive experiments and analyses show that it mitigates reasoning mismatch and the distillation trap, yielding substantial performance gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
On-Policy Distillation, Privileged Information, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rFbmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at on-policy self-distillation (OPSD) for mathematical reasoning. The authors argue that a teacher conditioned on the reference solution causes two problems: "reasoning mismatch," where the student borrows conclusions from the reference that it cannot derive itself, and a "distillation trap," where steps the student already got right are needlessly constrained. They propose RC-OPD. An external LLM (DeepSeek) finds the first erroneous stage in a failed student rollout and repairs it. The student then continues from the repaired prefix, for up to MM rounds, and the repair is accepted only if the student reaches the correct answer. For accepted repairs, the teacher is conditioned on a failure explanation for the erroneous segment and on a short derivation for the valid prefix (weighted by λ\lambda). Failed repairs fall back to plain OPSD. Experiments on Qwen3-1.7B/4B/8B, evaluated on AIME24/25 and HMMT25, show higher average accuracy than OPSD and other recent distillation baselines.

Critical strengths
  1. The motivation is backed by measurement. Before proposing the method, the authors quantify how often reference-conditioned answers are correct only because they borrow the reference's conclusions. This turns "reasoning mismatch" into a testable problem rather than an intuition, and it is the main reason I take the paper seriously.
  2. Accepting a repair only when the student can continue from it is simple and falsifiable. The ablation shows that this validation step contributes the most. This suggests the gain is not just from the extra hints of an external model, but also from checking whether the student can follow the repair.
  3. The method is precisely specified and broadly compared. Equations define the segment-level supervision clearly, and the experiments compare against several recent distillation baselines at multiple model sizes, which makes the method easy to reimplement and position.
Critical weaknesses
  1. Novelty is limited, and overlap with recent work is not discussed. The core loop is: use the reference to locate and repair the first error, accept the repair if the student's continuation succeeds, and then train on it. This is very close to InT (arXiv 2601.14209), which is not cited and differs mainly in using SFT/RL instead of OPSD. Within OPSD, ROSD already localizes the first erroneous span and leaves the valid prefix alone, and the concurrent Woodpecker Distillation (arXiv 2608.05168) also patches locally, verifies the patch by continuation, and then distills. What is new is mainly the iterative multi-round repair and the separate teacher conditioning for error and prefix segments. That reads as an engineering extension of OPSD/ROSD, and it largely determines how much this paper contributes.
  2. The paper says little about what errors the students actually make. The whole method depends on diagnosing the student's first error, yet there is no taxonomy or distribution of the errors found, and no human check of the DeepSeek diagnoses is reported. Without this, it is hard to tell whether the gains come from fixing root causes or from something else.
  3. The statistical evidence for the main results is weak, and hyperparameters were tuned on the test sets. Each benchmark has only 30 problems, the margins over the best baseline are small, and Table 1 reports no variance. In addition, MM (Figure 6) and λ\lambda (Figure 7) were selected by comparing results on the test benchmarks, with no held-out split.
  4. The comparison is not entirely fair. Only RC-OPD gets diagnoses and repairs from an external LLM. No OPSD baseline receives the same hints, and the number of external calls and their cost are not reported. Part of the improvement may simply come from having a stronger external model in the loop.
Questions for authors
  1. How does RC-OPD differ from InT, Woodpecker Distillation and LEMMA? Could you add an InT-style repair-then-train baseline that uses the same diagnostic model and budget?
  2. What kinds of first errors does the diagnosis find? Could you report their distribution by benchmark and model size, the fraction of diagnoses checked by humans (with agreement)?
  3. Could you report per-seed results (or mean ± std) for RC-OPD, OPSD and PW-OPSD in Table 1, with a paired per-problem bootstrap test? Do the conclusions hold if MM and λ\lambda are fixed in advance or chosen on a held-out split?
  4. Could you add an OPSD baseline whose teacher also receives the external model's diagnosis or repair text, but without continuation-based validation or segment-specific conditioning? How many external LLM calls does RC-OPD need per training problem?
5 comments
AuthorsOpenReview ↗

Response to Reviewer rFbm (Part 0)

We sincerely thank the reviewer for their careful reading, valuable time, and positive evaluation of our quantitative motivation, continuation-based repair validation, precise method formulation, and broad experimental comparisons. We have carefully incorporated the reviewer’s helpful suggestions by adding an InT-style repair-then-train comparison, an OPSD + Hindsight baseline, diagnostic verification and error-type statistics, fuller multi-seed and bootstrap reporting, and external-model call statistics. We also clarify how RC-OPD differs from the related approaches and how the existing experimental results were obtained. We therefore provide targeted clarifications and additional explanations below.

AuthorsOpenReview ↗

Response to Reviewer rFbm (Part 1)

Response to Q1 & W1: Comparison with concurrent and related work

We sincerely thank the reviewer for drawing our attention to these concurrent works. We appreciate your concerns and would also like to clarify a few potential misunderstandings. InT, Woodpecker Distillation, and LEMMA all use reflection on errors to construct training signals. We agree that this idea is widely accepted and was not introduced by RC-OPD. Nevertheless, RC-OPD differs from these methods in several significant ways:

  1. RC-OPD converts local repairs validated through student continuation into distributional supervision on the student’s own trajectories. Compared with SFT in InT and SFT on corrected trajectories in LEMMA, RC-OPD conveys corrective information through the full teacher distribution.
  2. RC-OPD constructs separate teacher conditioning for erroneous segments and valid prefixes. This effective mechanism also distinguishes it from Woodpecker Distillation.

In summary,our distinctive contribution is an effective on-policy supervision approach that combines targeted error correction with the preservation of valid reasoning, while using multiple rounds of diagnosis, repair, and continuation to address subsequently exposed errors.

Following your suggestion, we have also added an InT-style repair-then-train experiment. The results are shown below and have been included in Appendix §\S A.3 (Table 6) of the revised manuscript.

MethodAIME24AIME25HMMT25Mean
Initial Student71.6767.5041.6760.28
InT-style repair-then-train71.6766.6746.6761.67
RC-OPD78.3369.1750.8366.11

Response to Q2 & W2: Error categories and agreement with manual verification

Thank you for your valuable suggestion. We manually reviewed the diagnosis records. Specifically, we took the first 500 training examples, collected the diagnosis records generated for these examples, and checked earliest-error localization and anchor repair in the approximately 300 resulting records. As shown below, both earliest-error localization and anchor repair show strong agreement with manual verification. We have added the table to Appendix §\S B.1 (Table 7) of the revised manuscript.

Model SizeDiagnosesCorrect Earliest-Error LocalizationCorrect Anchor Repair
1.7B329263 / 329, 79.94%307 / 329, 93.31%
4B290251 / 290, 86.55%266 / 290, 91.72%
8B261229 / 261, 87.74%246 / 261, 94.25%

We also provide a more detailed breakdown of the models’ error types below and in Appendix §\S B.1 (Table 8).

Model SizeArithmetic or algebraic errorsProblem misinterpretation, overlooked conditions, or incorrect modelingMisapplication of theorems, formulas, or conceptsDouble counting, omitted cases, or incomplete case analysisUnsupported conclusionsFinal answer formulation or reporting errorsTotal
1.7B81 (26.21%)136 (44.01%)52 (16.83%)21 (6.80%)17 (5.50%)2 (0.65%)309 (100%)
4B87 (33.33%)87 (33.33%)53 (20.31%)15 (5.75%)16 (6.13%)3 (1.15%)261 (100%)
8B77 (33.48%)78 (33.91%)38 (16.52%)21 (9.13%)16 (6.96%)0 (0.00%)230 (100%)
Total245 (30.63%)301 (37.63%)143 (17.88%)57 (7.13%)49 (6.13%)5 (0.63%)800 (100%)
AuthorsOpenReview ↗

Response to Reviewer rFbm (Part 2)

Response to Q3 & W3: Multi-seed results and bootstrap tests

Thank you for your suggestion regarding the completeness of the experimental reporting. Table 1 in original manuscript already reports averages over multiple seeds. For completeness, we collected the results for three seeds from the experimental records and report them as mean ± standard deviation below and in Appendix §\S A.2 (Table 3).

MethodQwen3-1.7BQwen3-4BQwen3-8B
AIME2024AIME2025HMMT2025MeanAIME2024AIME2025HMMT2025MeanAIME2024AIME2025HMMT2025Mean
OPSD54.17 ± 2.2039.17 ± 1.6727.50 ± 0.8340.28 ± 0.7375.83 ± 2.2065.00 ± 0.8346.67 ± 3.0062.50 ± 1.2778.33 ± 1.6764.17 ± 3.0047.50 ± 1.4463.33 ± 1.21
PW-OPSD55.00 ± 2.5041.67 ± 1.4430.00 ± 2.2042.22 ± 1.5576.67 ± 0.8368.33 ± 3.0049.17 ± 1.6764.72 ± 0.5680.83 ± 2.2070.83 ± 2.8944.17 ± 3.6365.28 ± 1.82
RC-OPD54.17 ± 3.0047.50 ± 1.4430.83 ± 2.2044.17 ± 1.0078.33 ± 1.6769.17 ± 3.6350.83 ± 0.8366.11 ± 0.8378.33 ± 2.5076.67 ± 2.8945.83 ± 3.3366.94 ± 2.17

We further conducted bootstrap tests using the pooled data from multiple seeds. The results below show that RC-OPD significantly outperforms the baselines. We have included these results in Appendix §\S A.2 (Table 4) of the revised manuscript and highlighted them in blue.

ComparisonQwen3-1.7B│Qwen3-4B│Qwen3-8B
Δ (pp)95% CIp│Δ (pp)95% CIp│Δ (pp)95% CIp
RC-OPD vs OPSD+3.89[0.33, 7.48]0.0304│+3.61[0.17, 7.05]0.0400│+3.61[0.04, 7.16]0.0472
RC-OPD vs PW-OPSD+1.94[0.65, 3.24]0.0062│+1.39[0.09, 2.78]0.0466│+1.67[0.19, 3.15]0.0324

Δ (pp): RC-OPD − baseline in percentage points; CI: 95% percentile confidence interval; p: two-sided equal-tailed bootstrap p-value.

Response to Q4 & W4: Additional baseline and external-model call statistics

Thank you for your suggestion. We understand your concern about the contribution of external-model diagnosis. Following your suggestions, we added an OPSD + Hindsight baseline, in which the diagnosis and repaired text are supplied as hindsight (Appendix §\S A.3, Table 6). As shown below, directly adding this information to OPSD does not yield a significant gain. Under the OPSD setting, the diagnosis and repaired text for the erroneous segment also affect tokens in the other, correct portions of the trajectory. This finding is consistent with the RC-OPD design of using separate teacher conditioning for correct and erroneous segments.

MethodAIME24AIME25HMMT25Mean Avg@4
OPSD75.8365.0046.6762.50
OPSD + Hindsight71.6771.6745.0062.78

We also appreciate your concern about the number of external-model calls. We summarized the experimental records as follows and included these statistics in Appendix §\S B.8 (Table 10):

ModelCalls (Diagnosis/Repair) per Sample
1.7B0.883
4B0.701
8B0.431
Reviewer_ChmJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes RC-OPD, an on-policy self-distillation pipeline for mathematical reasoning. A diagnostic model identifies the first substantive error in a student trajectory, replaces it with an Anchor Stage, and produces separate hindsight for the Error Stage and the preceding valid prefix. The student continues from the repair; further errors can be diagnosed iteratively. Successful repair chains receive differentiated local distillation, while chains that exhaust the repair budget fall back to reference-conditioned OPSD. Experiments on Qwen3-1.7B/4B/8B and AIME24, AIME25, and HMMT25 report higher Avg@4 means than the included distillation baselines.

Critical strengths
  1. The method has a sensible trajectory-level decomposition: preserve a valid prefix, repair the earliest substantive error, and regenerate the affected suffix. Iterative Counterfactual Validation is a meaningful addition because it tests whether the student can actually continue from a proposed repair rather than accepting the diagnosis at face value.

  2. The evaluation spans three Qwen3 scales and three competition-math benchmarks. The matched component ablation supports the value of validation, and the identical-valid-prefix continuation experiment is particularly relevant: RC-OPD reaches 54.75% accuracy versus 46.50% for OPSD and 30.25% for Base on the selected evaluation subset.

Critical weaknesses
  1. The improvement is not fully disentangled from the quality of the external diagnostic supervision. DeepSeek-V4-Flash-0731 sees the worked reference and produces the root-error localization, Anchor Stage, and both hindsight signals, whereas the closest ROSD baseline uses a self-reflector. The internal ablation shows that validation helps within the proposed pipeline, but a matched baseline receiving the same diagnosis and external-model budget is needed to isolate the contribution of iterative validation and differentiated distillation. 
  2. The computation–performance tradeoff is missing. RC-OPD may require several external diagnoses, student continuations, answer checks, and full-vocabulary distillation passes for a single failed trajectory, yet the paper reports no external-token count, call count, wall-clock cost, or recovery/fallback/skip rates by model scale. This makes it difficult to assess whether the reported 3.6–3.9 point mean improvement over OPSD justifies the added pipeline complexity.
Questions for authors
  1. If a ROSD-style error-localized baseline receives exactly the same DeepSeek diagnosis, Anchor/Failure-Reason information, and external-model budget, how much additional gain comes from iterative validation and differentiated supervision?

  2. What are the average numbers of diagnosis and continuation calls, generated external-model tokens, and wall-clock cost per retained example? Please also report the local-recovery, fallback, and skipped-chain rates at each model scale.

  3. Does RC-OPD remain effective when diagnosis is performed by the same-size student/base model, or when the diagnostician receives only the final answer rather than the complete worked reference?

  4. Can the root-error, repair-validity, and reference-borrowing annotations be validated by independent mathematics experts or at least by a judge model different from the diagnostician?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ChmJ (Part 0)

We sincerely thank the reviewer for their careful reading, valuable time, and positive evaluation of our trajectory-level decomposition, continuation-based validation, and evaluation across model scales. We have carefully incorporated the reviewer’s helpful suggestions by adding comparisons of self-generated and external feedback, reporting pipeline costs and supervision-routing rates, and supplementing the diagnosis-quality evaluation. We also noted that a few comments may stem from minor misunderstandings. We clarify the existing ablations of iterative validation and fallback supervision, together with the reference-borrowing analysis, to make their roles and supporting evidence easier to locate. We therefore provide targeted clarifications and additional explanations below.

AuthorsOpenReview ↗

Response to Reviewer ChmJ (Part 1)

Response to Q1, W1 & W3: Ablations of the external model, iterative validation, and differentiated supervision

Thank you for these valuable concerns. Actually, RC-OPD consists precisely of the mechanisms you mention: diagnosis, Anchor/Failure-Reason information, iterative validation, and differentiated supervision. Your question therefore corresponds to ablating iterative validation and differentiated supervision within RC-OPD.

Therefore, we would like to clairfy that Table 2 of the original manuscript already examines these components; please see the ``w/o Validation'' and `w/o Fallback supervision'' rows in table 2. We have highlighted the corresponding discussion in blue in the main text to make it easier to locate.

To further examine the role of the external model, we added a ROSD-style experiment in which only the reflector is replaced with DeepSeek. We also ablated the use of the external model in RC-OPD (Appendix §\S A.3, Table 5). External diagnosis provides a modest improvement over self-diagnosis, but ROSD with external diagnosis still falls behind RC-OPD. This indicates that the main gains of RC-OPD do not come from the external model alone. Together with the existing ablations in Table 2, these results suggest that the mechanisms of RC-OPD work together and reinforce one another.

MethodAIME2024AIME2025HMMT2025Mean
ROSD + Self-diagnosis75.8366.6746.6763.06
ROSD + DeepSeek diagnosis73.3370.0048.3363.89
RC-OPD + Self-diagnosis76.6770.8349.1765.56
RC-OPD + DeepSeek diagnosis78.3369.1750.8366.11
AuthorsOpenReview ↗

Response to Reviewer ChmJ (Part 2)

Response to Q2 & W2: External-model costs and local-recovery, fallback, and skipped-chain rates

Thank you for raising this concern about the cost of external model calls and for your careful examination of our method. We agree that reporting the local-recovery, fallback, and skipped-chain rates would help readers better understand the algorithm’s behavior.

We have therefore summarized the external-model costs below and added these statistics to Appendix §\S B.8 (Table 10) of the revised manuscript to provide a more complete account of our experiments.

ModelCalls per SampleStudent Continuations per SampleGenerated Tokens per SampleTotal Processing Time (s per Sample)
1.7B0.8830.6454,691.5011.56
4B0.7010.5393,488.548.00
8B0.4310.3002,223.1610.94

We also summarized the fraction of examples entering diagnosis and, among those examples, the fractions that are locally recovered, retained through reference fallback, or skipped. These statistics have also been added to Appendix §\S B.8 (Table 11) of the revised manuscript.

ModelSamples Entering DiagnosisLocally Recovered and RetainedReference Fallback and RetainedSkipped: No Usable Training Tokens
1.7B69.33%42.18%50.74%7.08%
4B54.56%74.11%24.51%1.37%
8B33.22%70.27%16.27%13.45%

Response to Q4: Human verification of root-error, repair-validity, and reference-borrowing annotations

Thank you for your suggestion regarding the accuracy of the diagnostic model.

We manually reviewed the diagnosis records. Specifically, we took the first 500 training examples, collected the diagnosis records generated for these examples, and checked root-error localization and repair-validity in the approximately 300 resulting records. As shown below, both root-error localization and repair-validity show strong agreement with manual verification. We have added the table to Appendix §\S B.1 (Table 7) of the revised manuscript and highlighted it in blue.

Model SizeDiagnosesRoot-Error LocalizationRepair-Validity
1.7B329263 / 329, 79.94%307 / 329, 93.31%
4B290251 / 290, 86.55%266 / 290, 91.72%
8B261229 / 261, 87.74%246 / 261, 94.25%

Regarding reference borrowing, we would like to clarify that this phenomenon is a central motivation of our work. The analysis near the beginning of the original manuscript (Figure 2) already includes quantitative statistics and a case study. We have highlighted the corresponding content in blue to make it more prominent.

Reviewer_LwMfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RC-OPD uses an external LLM to locate the earliest substantive error in a failed student trajectory, produce a repaired anchor and failure explanation, and ask the frozen student to continue from the repaired prefix. Diagnosis and continuation may repeat. Successful chains provide separate supervision for erroneous segments and valid prefixes; failed chains fall back to reference-conditioned supervision. RC-OPD reports the best three-benchmark Avg@4 among compared OPD methods for Qwen3-1.7B, 4B, and 8B, with component ablations and analyses of prefix drift, error-stage alignment, and repair rounds.

Critical strengths
  1. The paper identifies an important weakness of reference-conditioned OPD: it may explain a correct solution without repairing the student’s actual error.
  2. The diagnosis–repair–continuation loop is coherent and tests proposed repairs through student continuation rather than trusting them directly.
  3. Performance trends are positive across three model sizes, and all component ablations reduce the 1.7B mean score.
Critical weaknesses
  1. The main attribution is confounded: RC-OPD uniquely uses DeepSeek-V4-Flash for diagnosis/repair and additional generation, while baselines receive no compute- or capability-matched treatment.
  2. Statistical evidence is limited: only 30 problems per benchmark, no paired confidence intervals, and unclear multi-seed treatment for several baselines. SFT/GRPO use a different protocol, and ROSD numbers come from a collaborator.
  3. Diagnostic reliability is under-evaluated. One analysis retains only 58 of 88 trajectories after excluding unreliable diagnoses or formatting failures, creating possible selection bias.
  4. The paper does not report total external-model calls, generated tokens, wall-clock cost, or a cost–accuracy curve.
Questions for authors
  1. What happens under compute-matched controls, or when the same diagnoses are provided without iterative validation?
  2. What is the human-verified accuracy of earliest-error localization and anchor repair?
  3. Can the authors provide paired bootstrap intervals and matched multi-seed results for the strongest baselines?
  4. What are the total model calls, generated tokens, and end-to-end training costs?
3 comments
AuthorsOpenReview ↗

Response to Reviewer LwMf (Part 0)

We sincerely thank the reviewer for their careful reading, valuable time, and positive evaluation of our problem formulation, the diagnosis–repair–continuation framework, and the performance trends across model scales. We have carefully incorporated the reviewer’s helpful suggestions through additional comparisons of diagnostic sources, verification of diagnostic records, fuller multi-seed and bootstrap reporting, and external-model usage statistics. We also noted that a few comments may stem from minor misunderstandings. We therefore provide targeted clarifications and additional explanations below.

AuthorsOpenReview ↗

Response to Reviewer LwMf (Part 1)

Response to W1: How much does the external diagnostic model contribute?

Thank you for your valuable concern about the role of the external diagnostic model. We consider this comment highly valuable for improving the completeness of our ablation study and have therefore conducted additional experiments.

Specifically, we added ablation experiments for two reflection-based methods, including RC-OPD: (1) ROSD with self-reflection versus reflection by an external model, and (2) RC-OPD with self-diagnosis versus diagnosis by an external model. The results show that the external model provides a modest improvement over self-reflection or self-diagnosis. However, ROSD with the external model still falls substantially behind RC-OPD, indicating that the main gains of RC-OPD do not come from introducing an external model. We appreciate this valuable suggestion and have included the results and discussion in Appendix §\S A.3 (Table 5) of the revised manuscript.

MethodAIME2024AIME2025HMMT2025Mean
ROSD + Self-reflection75.8366.6746.6763.06
ROSD + DeepSeek reflection73.3370.0048.3363.89
RC-OPD + Self-diagnosis76.6770.8349.1765.56
RC-OPD (+ DeepSeek diagnosis)78.3369.1750.8366.11

Response to Q1: The effect of the number of iterative validation rounds

Thank you for your valuable suggestions. We understand your concerns regarding iterative validation, a key mechanism in our method. We would like to clarify that the original manuscript includes an analysis of the number of iterative validation rounds.

Specifically, Figure 6 analyzes the effect of the number of rounds, and Table 2 includes a validation ablation, where the ``w/o Validation''row reports the results without validation. For ease of reference, we reproduce the results below and have highlighted the corresponding discussion in blue in the manuscript. We hope this clarifies the point.

Iteration0123
Score40.5641.9444.1742.78

Response to Q2: Human-verified accuracy of earliest-error localization and anchor repair

Thank you for this valuable suggestion regarding the agreement between the diagnostic model’s assessments and those of human experts.

We manually reviewed the diagnosis records. Specifically, we took the first 500 training examples, collected the diagnosis records generated for these examples, and checked earliest-error localization and anchor repair in the approximately 300 resulting records. As shown below, both earliest-error localization and anchor repair show strong agreement with manual verification. We have added the table to Appendix §\S B.1 (Table 7) of the revised manuscript and highlighted it in red.

Model SizeDiagnosesCorrect Earliest-Error LocalizationCorrect Anchor Repair
1.7B329263 / 329, 79.94%307 / 329, 93.31%
4B290251 / 290, 86.55%266 / 290, 91.72%
8B261229 / 261, 87.74%246 / 261, 94.25%
AuthorsOpenReview ↗

Response to Reviewer LwMf (Part 2)

Response to Q3 & W2: Multi-seed results and bootstrap tests

Thank you for your valuable suggestion regarding the stability of the experimental results. However, we would like to clarify that the results reported in Table 1 of the original manuscript are already averages over three random seeds.

Nevertheless, we therefore recalculated the standard deviations from the existing experimental records, as shown below and reported in Appendix §\S A.2 (Table 3).

Model SizeAIME2024AIME2025HMMT2025Mean ± Std
1.7B54.17 ± 3.0047.50 ± 1.4430.83 ± 2.2044.17 ± 1.00
4B78.33 ± 1.6769.17 ± 3.6350.83 ± 0.8366.11 ± 0.83
8B78.33 ± 2.5076.67 ± 2.8945.83 ± 3.3366.94 ± 2.17

We also conducted bootstrap tests using the pooled data from the three seeds. RC-OPD improves over both the standard baseline OPSD and the strongest baseline PW-OPSD at the 1.7B, 4B, and 8B scales. At all three scales, each two-sided comparison is statistically significant (p < 0.05). We have included these results in Appendix §\S A.2 (Table 4) of the revised manuscript.

ComparisonQwen3-1.7B│Qwen3-4B│Qwen3-8B
Δ (pp)95% CIp│Δ (pp)95% CIp│Δ (pp)95% CIp
RC-OPD vs OPSD+3.89[0.33, 7.48]0.0304│+3.61[0.17, 7.05]0.0400│+3.61[0.04, 7.16]0.0472
RC-OPD vs PW-OPSD+1.94[0.65, 3.24]0.0062│+1.39[0.09, 2.78]0.0466│+1.67[0.19, 3.15]0.0324

where Δ (pp): RC-OPD − baseline in percentage points; CI: 95% percentile confidence interval; p: two-sided equal-tailed bootstrap p-value.

Response to Q4 & W4: External-model costs

Thank you for raising this concern about the cost of external model calls. Actually, we share the same concern and therefore chose to deploy DeepSeek-V4-Flash-0731 locally to reduce the costs associated with external API calls.

To further clarify the overhead, we summarized external-model usage from the experimental records, as shown below. We have also included these statistics in Appendix §\S B.8 (Table 10) of the revised manuscript.

ModelCalls per SampleGenerated Tokens per Sample
1.7B0.8834,691.50
4B0.7013,488.54
8B0.4312,223.16

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to All Reviewers

Dear Reviewers,

Thank you for the time and effort you have devoted to our manuscript, as well as for your constructive comments. We have provided point-by-point responses, which we hope will address your concerns. In addition, we have uploaded the revised manuscript (see the attached PDF), in which all modifications are highlighted in blue and additions are highlighted in red. We once again sincerely appreciate your valuable feedback and efforts on our work.

AuthorsOpenReview ↗

A Kind Reminder: We Look Forward to Your Feedback

We sincerely thank reviewers for the thoughtful comments and the time reviewers have devoted to reviewing our paper. As the concerns focused primarily on the experimental completeness, we have conducted extensive additional experiments to address them as thoroughly as possible. We have also clarified a few minor misunderstandings that may have arisen from our original presentation. We hope these additions and clarifications have adequately addressed your concerns regarding the experiments.

We would also like to highlight our identification and analysis of reasoning mismatch, which motivates root-cause diagnosis and targeted distillation of correct and incorrect segments. In particular, we believe that our work on identifying, analyzing, and addressing ``reasoning mismatch'' provides valuable insights for the OPSD community. We hope this helps clarify the motivation and contributions of our work.

If our responses have satisfactorily addressed your concerns, we would be grateful if you could consider raising your score in light of these clarifications and contributions. As the discussion period is nearing its end, please feel free to let us know if you have any further questions or concerns. We would be very happy to address them before the discussion closes.

AuthorsOpenReview ↗

Overall Response

We thank the reviewers and Area Chair for their constructive feedback. We appreciate the reviewers’ recognition of the paper’s strengths: identifying “an important weakness of reference-conditioned OPD” (Reviewer LwMf), introducing Iterative Counterfactual Validation as “a meaningful addition” (Reviewer ChmJ), and presenting a method that is “precisely specified and broadly compared” (Reviewer rFbm). The reviews also raised several experimental concerns, and a few comments may stem from misunderstandings of the existing analyses and experimental setup. To address these points, we have supplemented the manuscript with extensive additional experimental evidence and provided detailed clarifications in our individual responses. Below we summarize the main revisions and clarifications. All locations refer to the revised manuscript.

  1. Tailored baseline comparisons: Building on the original comparison across multiple OPD methods, we added InT-style repair-then-train and OPSD + Hindsight controls tailored to the reviewers’ attribution questions. On Qwen3-4B, RC-OPD achieves 66.11% mean Avg@4, compared with 61.67% and 62.78%, respectively.

    Locations: Section 6.2, Table 1; Appendix § A.3, Table 6.

  2. Diagnostic-source controls: Added self-generated versus DeepSeek-generated feedback comparisons for both ROSD and RC-OPD. RC-OPD maintains higher mean accuracy than ROSD under both feedback sources, strengthening the evidence for its supervision design beyond the choice of diagnostic model.

    Locations: Appendix § A.3, Table 5.

  3. Three-seed consistency and statistical support: The original RC-OPD results were already averaged over three training seeds. We now report means and standard deviations for RC-OPD, OPSD, and PW-OPSD, together with bootstrap tests. The reported 95% intervals for RC-OPD’s mean gains over both baselines exclude zero at every model scale.

    Locations: Section 6.2, Table 1; Appendix § A.2, Tables 3–4.

  4. Diagnosis verification and error coverage: Added earliest-error localization and anchor-repair verification results, alongside error-type distributions across the three model sizes, to provide a more detailed characterization of the corrective signals used by RC-OPD.

    Locations: Appendix § B.1, Tables 7–8.

  5. Explicit pipeline accounting: Documented external diagnosis/repair calls, student continuations, generated external-model tokens, and processing time per sample. Added diagnosis-entry, local-recovery, reference-fallback, and skipped-chain statistics to characterize resource use and supervision routing across model scales.

    Locations: Appendix § B.8, Tables 10–11.

  6. Existing evidence for the core mechanisms: The original manuscript already included component ablations and a validation-round analysis. We have highlighted these results to make the evidence for iterative validation, valid-prefix supervision, and reference fallback easier to locate.

    Locations: Section 6.3, Table 2; Section 6.4, Figure 6; Appendices §§ B.3 and B.6.

  7. Measurement-backed motivation: Highlighted the original quantitative reference-borrowing analysis and illustrative cases. These results establish the motivation for examining reasoning beyond final-answer correctness and connect the observed reasoning mismatch to RC-OPD’s student-grounded corrective supervision.

    Locations: Section 4, Figures 2–3; Appendices §§ A.1 and B.2.

We hope that the additional experiments, expanded reporting, and clarifications help address the reviewers’ concerns and make the supporting evidence easier to assess. We look forward to the reviewers’ feedback on these updates and welcome any further questions. We sincerely thank the reviewers and Chairs for their time and careful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.