- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EvoOPD presents a system for black-box distillation using a closed-weight teacher model. The system has two key parts. First, a system that generates privileged information (PI) from the teacher that acts as hints for the student. During RL, when the student generates an incorrect answer the teacher is prompted using strategies to provide hints. These hints localize the fault and provide information for producing a correct answer. The strategy bank is evolved throughout the training process using a strategy bank update procedure. The second part is a localization aware reward that applies additional negative advantage to the erroneous student span (identified by the teacher). The system is evaluated on two tasks (math and coding) using 1 family of student models (3 sizes) and 1 teacher model (gemini flash). EvoOPD is shown to be better on average than the other systems tested.
Critical strengths
-
The paper presents a new end-to-end blackbox distillation approach that appears to work better than the other systems evaluated. The design is clearly explained and ablations demonstrates that each component of the system add improvements (fault localization, strategy evolution). I especially appreciated the result that evaluated different bank snapshots (early, middle, late) and different points in training.
-
The paper was, on the whole, easy to follow. Figure 2 was more style over substance, and did not help me understand the fault localization component. From the diagram it was not clear that the teacher was required to extract the e (the erroneous step) via prompt. Looking at the appendix resolved this. Otherwise, the paper and its results were easy to follow, and the problem was well motivated.
Critical weaknesses
-
A core part of this paper is the implementation of the strategy bank and generation of PI. The strategy bank is a form of prompt optimization for the teacher model. Only one teacher model is used, therefore it is unclear whether the techniques generalize to other teachers, or whether they provide additional benefit when even stronger teachers with better intrinsic error location and repair abilities are used. My key concern here is that a central part of this paper is prompt optimization, which itself can be hard to position as a robust research contribution, but additionally it is only evaluated on one model.
-
The novelty of the system is modest, with memory based distillation tackling similar problems. The limited novelty caused be offset with a comprehensive evaluation that demonstrates that this is robust and generalizable technique, however the evaluation in the paper is limited to 1 teacher, 1 family of students, and two tasks (math and coding). Showing that the system scales to stronger teachers, domains, or more complex tasks (like agentic tool calling) would strengthen the work.
Questions for authors
-
What is teacher performance on the benchmarks?
-
OPSD uses frozen teacher. Which teacher is this? How do we know that the improvement of EvoOPD, or weakness of OPSD is due to the strengths of the relative teachers (not the techniques).
-
What is the teacher for ROPD?
-
Table 4. You bold EvoOPD as if it performs 'best', in a similar notation as the benchmark results. But these are token numbers and it isn't clear whether higher or lower is better. (And you bold cases where EvoOPD is both most and least). Please improve the clarity of this table. I think tokens / per accuracy points gained would be a more direct and informative presentation, is that correct?
-
The Fixed Strategy Bank token total is unavailable and remains a dash rather than zero.
Why?
-
The gains on coding are slim. How do the gains change between training runs? How significant is variance between trained checkpoints?
4 comments
Q1: Teacher Performance Across Benchmarks
We have added the independent problem-solving performance of both black-box teachers on all six mathematical reasoning benchmarks to provide a clearer picture of their capabilities. The results are shown below.
| Teacher | AIME24 | AIME25 | AIME26 | HMMT25-Feb | HMMT25-Nov | HMMT26-Feb | Overall |
|---|---|---|---|---|---|---|---|
| Gemini-3.6-Flash | 95.00 | 94.58 | 95.42 | 98.74 | 92.50 | 90.15 | 94.33 |
| GPT-6.1-Sol | 100.00 | 99.58 | 99.58 | 99.17 | 99.17 | 97.73 | 99.18 |
The results show that Gemini-3.6-Flash, used in our original experiments, already demonstrates strong mathematical reasoning capabilities, achieving 94.33% Overall accuracy. The alternative teacher, GPT-6.1-Sol, achieves an even higher Overall accuracy of 99.18%. These results have been added to Appendix B.1 (Table 7).
Q2 & Q3: Teacher Configurations of Baselines and OPSD Performance
We clarify that SeqKD, OPSD, ROPD, and EvoOPD all use the same external black-box teacher, Gemini-3.6-Flash, in the original experiments, controlling for differences in external teacher capability. Specifically, SeqKD performs SFT on verified Gemini-generated responses; ROPD uses evaluative feedback from Gemini for reinforcement learning; OPSD obtains PI from Gemini and, following its official implementation, uses a frozen copy of the student as the probability teacher for self-distillation; and EvoOPD converts Gemini's textual PI into localized token-level advantage corrections.
Regarding OPSD's performance, prior studies have shown that privileged-information-based self-distillation can be sensitive to PI construction and training configurations (Zhu et al., 2026a; Yang et al., 2026a; Li et al., 2026d). In our non-thinking mathematics setting, OPSD does not outperform Base in Overall accuracy at any of the three student scales. In one training run, we also observed that early validation gains gradually disappeared as training progressed.
In contrast, EvoOPD does not directly match the full teacher probability distribution conditioned on PI. Instead, it uses changes in the student's own token probabilities to construct bounded advantage corrections within localized windows of incorrect trajectories. This difference in supervision construction may partly explain the different outcomes observed in our experiments.
We have further discussed OPSD's performance in Section 3.2 and clarified the baseline teacher configurations and training details in Appendix B.1.
Q4 & Q5: Cost Reporting in Table 4
We have incorporated the reviewer's suggestions regarding the presentation of Table 4. In the revised manuscript, we have adjusted the bold formatting of token usage and added the Tokens per Accuracy Point Gained metric, measured relative to No PI, to better illustrate the trade-off between performance gains and teacher interaction costs.
Regarding the missing token cost for Fixed Strategy Bank, we have supplemented the complete statistics. This configuration uses 150.16M teacher tokens, corresponding to 34.05M tokens/pp. In comparison, EvoOPD uses 187.05M teacher tokens and achieves 21.98M tokens/pp, demonstrating higher teacher-token efficiency.
These results have been updated in Section 3.4 (Table 4), with the complete breakdown of input, output, and thinking tokens provided in Appendix B.4 (Table 9).
Q6: Stability of Code Generation Performance
To address concerns about the relatively modest gains on code generation and their stability, we conducted three independent training runs of EvoOPD with different random seeds on Qwen3-1.7B. We also analyzed performance variation across checkpoints near the best-performing model.
First, regarding stability across training runs, we independently trained EvoOPD with three random seeds and evaluated each run on five code generation benchmarks.
Table R1: Multi-Run Stability on Code Generation (Pass@1, %).
| Method / Run | HumanEval+ | MBPP+ | BCB-Hard | LCB v5 | LCB v6 | Overall |
|---|---|---|---|---|---|---|
| GRPO | 66.88 | 51.32 | 8.99 | 15.34 | 18.11 | 36.27 |
| EvoOPD (Seed 42) | 68.10 | 50.98 | 11.19 | 17.55 | 20.21 | 37.37 |
| EvoOPD (Seed 7) | 70.08 | 51.19 | 11.44 | 18.86 | 22.04 | 38.32 |
| EvoOPD (Seed 2026) | 65.66 | 54.12 | 13.13 | 19.46 | 22.39 | 39.09 |
| EvoOPD (Mean ± Std) | 67.95 ± 2.21 | 52.10 ± 1.76 | 11.92 ± 1.06 | 18.62 ± 0.98 | 21.55 ± 1.17 | 38.26 ± 0.86 |
EvoOPD achieves Overall accuracies of 37.37%, 38.32%, and 39.09% across three independent runs, all exceeding the original GRPO baseline of 36.27%, with a mean performance of 38.26 ± 0.86%. Moreover, the average performance of EvoOPD across the three runs exceeds GRPO on all five code benchmarks, indicating that the improvements are not concentrated on a single dataset.
Second, regarding performance variation across checkpoints, we examined the validation performance of three checkpoints near the best-performing model for both GRPO and EvoOPD.
Table R2: Checkpoint Stability Around the Best Model.
| Method | Best Val (%) ↑ | Nearby Mean ± Std (%) ↑ | Nearby Range (pp) ↓ | Selected Test Overall (%) ↑ |
|---|---|---|---|---|
| GRPO | 28.73 | 28.12 ± 0.56 | 1.09 | 36.27 |
| EvoOPD | 29.45 | 28.85 ± 0.55 | 1.09 | 37.37 |
EvoOPD achieves a higher mean validation accuracy across nearby checkpoints (28.85% vs. 28.12%), while the standard deviations are 0.55 and 0.56 percentage points, respectively, and both methods have the same range of 1.09 percentage points. These results show that EvoOPD achieves higher validation accuracy while exhibiting checkpoint-to-checkpoint variation comparable to GRPO.
Furthermore, Table 2 evaluates three student scales across five code generation benchmarks. EvoOPD outperforms GRPO by 1.10, 1.46, and 1.35 percentage points on Qwen3-1.7B, 4B, and 8B, respectively, with an average improvement of 1.30 percentage points. EvoOPD also achieves better results in 12 out of 15 model–benchmark comparisons.
Together, these additional results provide further evidence of the consistency of EvoOPD's code generation gains across independent training runs, nearby checkpoints, and different model scales and benchmarks.
We thank the reviewer again for the constructive feedback and hope that these additional experiments and clarifications have addressed the main concerns. We would be happy to clarify any remaining questions.
Dear Reviewer,
We sincerely appreciate the time and effort you have devoted to reviewing our paper, especially given your busy schedule.
As the discussion deadline approaches, we would appreciate knowing whether our responses and additional experiments have adequately addressed your concerns. We would be happy to clarify any remaining questions or engage in further discussion.
Thank you again for your time and valuable feedback.
We thank the reviewer for the careful review and specific suggestions. In the revised manuscript, we have added analyses of the PI supervision mechanism, extended our evaluations across teachers and student model families, and clarified the baseline configurations and computational costs. We address each concern below.
W1 & W2: Novelty and Generalization
We would like to clarify that our main contribution is not simply prompt optimization, but a distinct approach to black-box on-policy distillation. Specifically, we investigate how textual privileged information (PI) from a black-box teacher can be converted into localized token-level supervision on the student's own trajectories, without access to teacher logits, rather than using teacher-generated text only for SeqKD or converting it into scalar rewards.
Our second contribution is the continual evolution of PI acquisition strategies. Rather than merely optimizing prompts, we adapt the type and granularity of acquired PI based on the current student's actual error-recovery performance. Figure 4 in the original submission already showed that the relative effectiveness of different PI strategies changes with the student's training state.
We also recognize that the original experimental analysis placed greater emphasis on PI strategy evolution, without sufficiently highlighting our first contribution. Therefore, we have added token-level mechanism analyses and PI-on/PI-off controlled experiments in Section 3.3 (Figure 3) to investigate which tokens are affected by PI supervision and how these changes influence unassisted learning. The results show that PI generally has a stronger effect on Planning and Calculation tokens than on Style tokens, and that Original PI induces larger probability shifts than Shuffled PI. Moreover, PI-on improves unassisted continuation correctness over PI-off by 1.15, 1.61, and 2.98 percentage points across the three training stages, providing further evidence that localized PI supervision translates into actual learning gains.
Regarding our second contribution, we further examine the relationship between PI acquisition strategies and student states in Section 3.4 (Figure 4). Students at different stages achieve the highest error-recovery utility and subsequent unassisted training gains with their corresponding stage-specific PI Strategy Banks. These measures are also significantly positively correlated (Pearson r = 0.814). With the same number of strategy evaluation rounds, EvoOPD outperforms Offline-9 by 1.16 and 1.69 percentage points on mathematics and code generation, respectively (Table 3), supporting the effectiveness of continual PI strategy evolution.
To address concerns about generalization, we conducted additional cross-teacher and cross-family experiments. We use GPT-6.1-Sol as an alternative black-box teacher and introduce Gemma 4 E4B IT and Ministral 3 8B Instruct as students from two additional model families. The results across six mathematical reasoning benchmarks are shown below.
Across these three additional settings, EvoOPD outperforms ROPD by 3.21, 2.19, and 1.94 percentage points, respectively, with improvements in all 18 model–benchmark comparisons. These results indicate that EvoOPD's advantages are not limited to the original teacher or the Qwen model family. Full results have been added to Section 3.5 (Table 5).