- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MENTOR-SDFT for continual fine-tuning without replay. Before each task, Agent-Guided Prompt Evolution (AGPE) uses GPT-5.4-mini to optimize a teacher prompt while keeping the target model frozen. Subsequent on-policy distillation uses a reference-conditioned EMA self-teacher, with Confidence-guided Soft Trajectory Distillation (CSTD) weighting supervision according to teacher entropy and trajectory position. Experiments on four tasks and three Qwen backbones report improved final average accuracy and reduced forgetting relative to sequential SDFT.
Critical strengths
1.The paper addresses a meaningful question: whether reference conditioning alone provides sufficiently useful supervision for self-distillation. The controlled prompting experiment provides initial evidence that teacher conditioning deserves further investigation.
2.The evolved prompts and reference answers are required only during training. The resulting student incurs no additional prompt optimization or privileged-information requirements at inference time.
3.Improvements over sequential SDFT are consistent across the three evaluated backbones. The progressive ablations provide initial evidence that both AGPE and CSTD contribute to the overall gains.
Critical weaknesses
1.The offline prompt selection metric is not validated as a predictor of downstream distillation gains. Section 4.1 selects prompts using the frozen model’s answer accuracy without reference answers. During distillation, however, the teacher receives references and supplies distributions along student-generated prefixes. The paper does not establish whether prompts ranked highly under the first condition also yield larger student gains under the second. Comparing candidate scores with outcomes after equal-budget distillation would test this assumption. Furthermore, the proposed net gain is equivalent to the accuracy increase on a fixed dataset. Although individual regression cases may inform reflection, the scalar ranking offers no additional preference for fewer regressions at the same accuracy.
2.The evolved prompts provide limited support for the proposed explanation of the gains. The motivation emphasizes helping the teacher interpret reference answers and reconstruct their derivations. However, Appendix E largely shows formatting instructions, exact-matching requirements, and general task constraints. The Hendrycks prompt makes explicit reasoning optional and requests brevity. These examples leave unclear whether improvements reflect more transferable supervision or better compliance with output requirements. A baseline that only strengthens formatting instructions, together with an error breakdown, would help distinguish these explanations. The complete teacher input should also be provided, particularly to clarify what “reference” denotes in the ToolUse prompt when candidate selection excludes reference answers.
3.The necessity of iterative reflection and evolution remains unverified. AGPE generates three candidates per generation and runs for two generations, leaving one refinement round after initial generation. The paper does not report results across evolution depths or compare against simpler alternatives, such as a single prompt rewrite or independently generating the same number of candidates and selecting the best. Comparisons using the same optimizer and evaluation budget would isolate the contribution of iterative feedback from the benefit of prompt generation and selection alone.
4.The continued suitability of a fixed prompt during training is not examined. AGPE optimizes the prompt before each task and keeps it fixed throughout distillation. Meanwhile, both the student and EMA teacher evolve, potentially changing their trajectories and failure patterns. The initially selected prompt may therefore become less suitable as training progresses. Comparing optimization performed entirely before training with periodic refinement based on updated student errors, under the same total search budget, would help justify this design choice.
5.Dependence on the external prompt optimizer and its computational cost are insufficiently characterized. All prompt evolution uses GPT-5.4-mini. The paper does not compare this setup with using the target model itself for diagnosis and refinement, leaving unclear how much the gains depend on external model capabilities. Such a comparison would clarify the requirements of the prompt optimization stage. Reporting API usage, candidate evaluation costs, and total training time would also make the practical trade-off relative to sequential SDFT easier to assess.
Questions for authors
See weaknesses.
2 comments
Q3.7: How does AGPE's performance change when the target model replaces GPT-5.4-mini as the prompt optimizer?
A3.7:
Thank you for this question. We replaced GPT-5.4-mini with the target model itself for prompt evolution. As shown below, self-evolution improves SDFT on both prompt evolvers. The gains are smaller for Qwen2.5-3B due to its weaker prompt-optimization capability, whereas Qwen3-4B already has sufficient ability.
| Model | Prompt Optimizer | ToolUse | Science | MBPP | Hendrycks | Average |
|---|---|---|---|---|---|---|
| Qwen2.5-3B | SDFT | 48.45 | 63.91 | 37.80 | 78.45 | 57.15 |
| Target Model | 49.48 | 64.30 | 41.40 | 78.26 | 58.36 | |
| GPT-5.4-mini | 64.95 | 66.07 | 40.00 | 78.85 | 62.47 | |
| Qwen3-4B | SDFT | 58.76 | 60.36 | 47.80 | 92.17 | 64.77 |
| Target Model | 56.70 | 62.72 | 48.80 | 92.00 | 65.06 | |
| GPT-5.4-mini | 62.89 | 65.48 | 47.80 | 91.66 | 66.96 |
We have included the above results in Table 7 of the Appendix in the manuscript.
Q3.8: What is the cost of AGPE?
A3.8:
Thank you for this question. For four-task continual learning, AGPE requires 63 GPT-5.4-mini API calls and approximately 1.16 million tokens, totaling only $1.03.
For training time, sequential SDFT takes approximately 5.5 hours (Qwen2.5-3B on a single H100 GPU), while AGPE adds only 4 minutes per task on average for prompt optimization and candidate evaluation. Overall, AGPE introduces only a small additional cost.
Q3.1: Does offline prompt accuracy reflect downstream distillation performance?
A3.1:
Thank you for raising this concern. We conducted additional experiments on Qwen3-1.7B to examine the relationship between offline prompt accuracy and downstream distillation performance. We selected three prompts with the highest, median, and lowest offline accuracy from all evolved candidates and performed distillation using identical training settings and budgets. As shown below, higher offline prompt accuracy consistently corresponds to better post-distillation performance, supporting the effectiveness of our offline prompt selection criterion.
Q3.2: Is the net-gain criterion equivalent to the accuracy?
A3.2:
Yes. As defined in Eq. 6, the net-gain G(p) is equivalent to accuracy. G(p)=∣I+(p)∣−∣I−(p)∣.
We adopt this form because the reflection process considers not only G(p) but also I+(p) and I−(p), ensuring that regressions are taken into account during optimization.
The detailed implementation is available in the supplementary code repository at
src/prompt_agent(lines 1177–1183).We have added the corresponding description to the Method section of the manuscript.
Q3.3: Do the evolved prompts improve supervision beyond formatting compliance?
A3.3:
Thank you for this observation.
As shown below, the two methods exhibit similar formatting error rates on Qwen2.5-3B, indicating that the gains are not solely due to formatting compliance.
Notably, the original task prompts already impose format constraints. Such constraints therefore become important content during evolution. Nevertheless, the experimental results show that the evolved prompt still helps self-distillation beyond formatting issues.
We have included the above results in Table 6 of the Appendix in the manuscript.
Q3.4: What information is provided to the teacher during distillation, and what does “reference” mean in ToolUse?
A3.4:
Thank you for this question. During distillation, the complete self-teacher input during distillation can be represented as follows:
Teacher Input:
For ToolUse, the reference denotes the ground-truth answer. For example:
I need to find the most recent changes to travel restrictions in Japan. Action: getLatestPolicyUpdates Action Input: {"limit": 10}.
Q3.5: Is iterative reflection necessary for effective prompt optimization?
A3.5:
Thank you for this suggestion. We compare two strategies under the same budget:
As shown in the figure below, iterative refinement consistently outperforms independent search across all four tasks, demonstrating the benefits of iterative reflection for prompt optimization.
We have included the above results in Table 4 of the Experiments in the manuscript.
Q3.6: Does periodic refinement of teacher prompts during distillation improve performance compared to keeping them fixed?
A3.6:
Thank you for this suggestion. We compared fixed prompts with periodic refinement. The results support keeping teacher prompts fixed for two reasons:
We have included the above results in Table 9 of the Appendix in the manuscript.