- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Differential Prompt Optimization (DiffPO), a framework for APE. The authors argue that existing prompt optimization methods largely rely on final outcome feedback, although the optimizer may observe the model's reasoning trace, it must itself infer which intermediate reasoning step caused the final failure. DiffPO instead uses a stronger teacher model to inspect the student's reasoning trace against a reference solution, identify where the reasoning breaks down, and abstract the failure into a generalizable rule. These rules are aggregated across failures and provided to the prompt optimizer to generate improved prompts.
Critical strengths
(1) The paper addresses automatic prompt optimization, a timely and relevant problem.
(2) While the paper could compare against more baselines, the evaluation protocol is solid. The authors evaluate two model families, average results over five seeds, and include sufficient ablations to clarify the contributions of the main components.
(3) While several recent prompt-optimization methods are missing, the paper covers the most relevant prior work that is conceptually similar to the proposed approach.
Critical weaknesses
Major Weaknesses
(1) Most of the concepts introduced in this paper are already well established in the literature, including learning generalizable concepts for prompt optimization and leveraging intermediate reasoning steps rather than only the final answer (e.g. [1-3]). The authors do compare experimentally against some of the most closely related methods, therefore, I view the paper primarily as a well-executed combination of existing ideas rather than a fundamentally novel approach.
(2) I would like to see a broader evaluation against existing baselines. The authors cite several APE methods but do not compare against them experimentally (e.g., [4-5]). Several important recent methods are also neither cited nor evaluated [6-8]. Without these comparisons, it is difficult to assess the true benefits of DiffPO and whether its additional components are necessary.
(3) DiffPO relies on an offline knowledge-base construction stage, which introduces additional requirements. In particular, the method requires access to a stronger teacher model to construct the knowledge base for each new task or domain.
(4) The paper also lacks a fully fair compute comparison. The authors discuss compute/query budget, but exclude the offline cost of constructing the knowledge base, giving DiffPO additional compute that is not accounted for in the main comparison.
Minor Weaknesses
(1) The related-work section would benefit from discussing approaches that use reinforcement learning for automated prompt optimization, such as [9-10].
(2) Table captions should be above the tables.
References
[1] Boateng, Emmanuel Aboah, et al. "Concept distillation from strong to weak models via hypotheses-to-theories prompting." NAACL'2025
[2] Zhang, Tianjun, et al. "In-context principle learning from mistakes." arXiv'2024.
[3] Agrawal, Lakshya A., et al. "Gepa: Reflective prompt evolution can outperform reinforcement learning." ICLR'2026
[4] Wang, Xinyuan, et al. "Promptagent: Strategic planning with language models enables expert-level prompt optimization." ICLR'2024
[5] Guo, Qingyan, et al. "Connecting large language models with evolutionary algorithms yields powerful prompt optimizers." ICLR'2024
[6] Agarwal, Eshaan, et al. "PromptWizard: Optimizing prompts via task-aware, feedback-driven self-evolution." ACL'2025
[7] Shi, Wenhang, et al. "No loss, no gain: Gated refinement and adaptive compression for prompt optimization." NeurIPS'2025
[8] Batorski, Paweł, and Paul Swoboda. "PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data." ACL'2026
[9] Deng, Mingkai, et al. "Rlprompt: Optimizing discrete text prompts with reinforcement learning." EMNLP'2022
[10] Kwon, Minchan, et al. "Stableprompt: Automatic prompt tuning using reinforcement learning for large language model." EMNLP'2024
Questions for authors
See weaknesses
Response to reviewer MTY6
We sincerely thank Reviewer MTY6 for their constructive feedback and questions; please find our response below.
Paper's novelty
While several ideas have existed before, our work addresses a distinct core problem: how to utilize a larger model to reliably improve the performance of a smaller model. A central finding of our paper is that naively using a stronger model to assist prompt optimization does not reliably improve a smaller student.
Why naive teacher-to-student transfer fails:
Optimizing prompts on the teacher (Teacher Only): As we show in the paper, optimizing prompts directly on the teacher improves the teacher itself on IFBench. Yet deploying those exact prompts onto the student drops student accuracy below optimizing directly on the student. The teacher evolves concise instructions suited to its own reasoning depth, omitting the explicit requirements of the smaller model.
Static teacher rules or raw teacher traces/exemplars: Distilling teacher feedback offline into static rules (CD [1]), or supplying raw teacher reference solutions si directly into reflection (+ Raw Reference Solutions), overwhelms the student or the optimizer with conflicting information. Similarly, attaching retrieved Φinsight exemplars post-hoc to prompts (+ Post-Hoc Φinsight) often degrades the performance as students copy surface patterns.
Our novelty therefore lies in identifying why naive teacher-guided prompting fails under teacher-student capability gaps and introducing a closed-loop differential audit and co-evolution mechanism that transfers gains reliably from the teacher to the student.
Comparison with more baselines (extending Table 1 in the paper)
We thank the reviewer for sharing the additional baselines. We have implemented [2, 4, 5, 6] under identical models (gemini-3.1-flash-lite student, gemini-3-flash optimizer/teacher); the results are shown below:
We will add the missing citations to Related Work, including a discussion positioning the very recent GRACE [7] (NeurIPS 2025) and PIAST [8] relative to our teacher-to-student differential auditing framework.
Require stronger teacher
We agree that leveraging a stronger teacher is part of our primary problem formulation; distilling reasoning guidance from a capable teacher into a compact student. However, in terms of utilizing the teacher for the external knowledge base, as shown in the table above, DiffPO w/o Exemplars retains strong gains with zero Phase-1 knowledge base.
Compute budget
We set the optimization budget up to 20 iterations for the GEPA/APEX baselines and 10 for DiffPO to match online query cost. However, most 20-iteration baseline runs plateaued without finding a better prompt in later rounds, which means increasing the cost does not usually translate to a better performance.
We agree that our paper requires the extra cost of constructing the knowledge base, however, we want to reemphasize that our main goal is not to find a most training efficient method; we look into ways to improve the performance of the student during the test time. The knowledge base construction is a one-time offline cost that requires zero teacher calls during the inference helping us to maximize test-time student accuracy.
We will add this discussion to the paper.
RL methods
As briefly discussed in the paper, prompt optimization spans distinct paradigms: RL-based methods such as RLPrompt [9] and StablePrompt [10] train parametric policy models to guide the optimization, whereas feedback-based methods edit prompts in natural language using a LLM as an optimizer. Since our method is does not train any extra model, our empirical comparison focuses on feedback-based optimizers. We will add an explicit discussion of RLPrompt [9] and StablePrompt [10] to Related Work.
Table Caption
We thank the reviewer for catching this and will move all table captions above their respective tables in the revised manuscript.