- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces P²O. The idea of jointly optimizing prompts and model parameters is insightful, and the method is sufficiently mature. However, some supporting evidence is missing and should be added.
Critical strengths
- The goal of this work is well motivated: jointly optimizing prompts and model parameters is desirable, and this problem has not been adequately addressed by others.
- Context Distillation precisely targets an aspect overlooked by related methods.
- The discussion of off-policy / distribution mismatch makes the method more complete and the paper more insightful.
- The performance improvements are significant.
Critical weaknesses
-
The paper does not discuss the effect of optimizing only the prompt side, nor does it provide examples showing to what extent GEPA itself can improve performance. If introducing prompt optimization during training leads to substantial gains, the paper should show where these gains come from and rule out the effect of using prompt optimization alone. In addition, a single piece of data generally includes a question and a prompt. Only optimizing the prompt's upper limit is somewhat concerning, because I don't think merely optimizing prompts like "reasoning step by step" can bring much benefit. I hope the author demonstrates this through case studies. These are real limitations, but despite that, I think this work is insightful.
-
Most of the experiments focus on accuracy and appear relatively late in the paper, while the motivation of the method is not sufficiently supported by evidence in the earlier sections. Some of these points may reasonably be treated as common knowledge in a submission version, but they should be supplemented in a later version. For example, training only with improved prompts may have poor generalization, and the authors should demonstrate this empirically. Similarly, the Context Ratio Mask should be supported by a quantitative analysis of the degree of off-policy / distribution mismatch. These analyses should ideally be presented before the accuracy experiments to improve the readability of the paper, especially for less experienced readers.
Questions for authors
See Weaknesses.
3 comments
Thank you for the detailed response and additional analyses. These results largely address my concerns about the contribution of prompt optimization and the motivation for context distillation.
My understanding is that the GEPA-evolved templates provide hint-like privileged information during training, and P²O distills the benefits of this guidance into the policy so that it can perform better without the templates at inference time. I find this a useful and well-motivated approach. The prompt-only baseline and context-distillation ablation help clarify this interpretation.
The use of the non-thinking setting does reduce my confidence somewhat, as it remains unclear how much of the benefit would carry over to models operating in thinking mode. Nevertheless, I find the method clever and its progression from prompt-assisted exploration to context distillation coherent. Overall, I remain inclined to accept the paper and maintain my accept rating.
Response to Reviewer UT3i
Thank you for the thoughtful follow-up. Your interpretation captures our intended mechanism: GEPA-evolved templates provide auxiliary, hint-like guidance during training, and context distillation updates the policy under the original query so that it can benefit from this guidance without the templates at inference time.
To address the concern about dependence on a hybrid thinking/non-thinking backbone, we added an experiment using Qwen2.5-7B-Instruct on DeepMath-5K. Qwen2.5-7B-Instruct is a conventional instruction-tuned model without a separate thinking mode. We compare P²O with GRPO-8 on the same six benchmarks, with P²O evaluated on original queries without evolved templates.
All scores are accuracy (%); AVG is the unweighted mean across the six benchmarks.
| Method | AIME24 | AIME25 | AMC | MATH500 | Minerva Math | OlympiadBench | AVG |
|---|---|---|---|---|---|---|---|
| GRPO-8 | 11.40 | 7.90 | 50.90 | 77.20 | 38.60 | 39.90 | 37.65 |
| P²O | 17.90 | 10.40 | 59.10 | 77.40 | 40.10 | 41.30 | 41.03 |
P²O raises average accuracy from 37.65% to 41.03%, a gain of 3.38 percentage points, and achieves higher scores on all six benchmarks.
These results demonstrate that P²O's benefits also hold for a conventional Instruct model, addressing the concern that a hybrid thinking/non-thinking backbone is necessary for the observed improvement. The gains are achieved without evolved templates at inference, consistent with your interpretation that prompt-assisted exploration and context distillation translate training-time guidance into improved policy performance.
Based on the experimental results reported in our previous responses and the reviewers' feedback, we have submitted a revised manuscript. The revision updates the experimental results and conclusions, corrects Table 6 and its associated analyses, and includes additional experiments. All changes are highlighted in blue for ease of review. We invite you to review the revised manuscript.
Response to Reviewer UT3i
Thank you for recognizing the value of joint policy and prompt optimization and for suggesting clearer evidence for the method's motivation.
W1(a). What does prompt optimization alone achieve? We evaluated GEPA with the initial policy weights frozen. On DeepMath-5K/Qwen3-4B, the following results use the same 500-example evaluation set:
With frozen weights, GEPA improves accuracy by 14.00 points over Base. It uses the training-time search configuration, with the rollout cap removed and early stopping after eight search steps without dev improvement.
W1(b). What does joint training add beyond prompt optimization alone?
On the same evaluation set, P²O exceeds GEPA-only by 21.80 points and GRPO by 4.80 points while using original queries at evaluation. Its final performance therefore cannot be explained by retaining optimized inference-time prompts: the policy has learned to perform better without them.
W1(c). What do the templates contribute beyond “think step by step”? Generic step-by-step guidance is already included in the input for the base model, GRPO and P²O, all run in non-thinking mode. Please refer to our response to Reviewer MoGR, Part 1, Q3, for this shared setting. The evolved templates add substantive problem-solving guidance. In Figure 7, the question asks for the smallest sphere enclosing four unit spheres. The bare-query response returns the incorrect radius 1+2. The evolved template directs the model to consider regular-polyhedron configurations and supplies the tetrahedron's centroid-to-vertex relation, R=a3/8. With center-to-center edge length a=2, the guided response obtains the correct enclosing radius 1+6/2. The useful intervention is this geometric structure and relation, beyond a generic reasoning instruction.
This case illustrates the mechanism. Please refer to our response to Reviewer MoGR, Part 2, W2(d)/Q5, for aggregate evidence from the corrected Table 6 analysis. Across 1,980 historical hard records, templates raise mean reward from 0.1495 to 0.2419 and nonzero-variance groups from 876 (44.24%) to 1,025 (51.77%). Context distillation trains the assisted trajectories under the original query. The R1-hint and single-turn-reflection controls further support evolving guidance over the tested simpler alternatives. Please refer to our response to Reviewer MoGR, Part 1, W1(a)/Q2 and W1(d)/Q4, for these comparisons.
Prior work also demonstrates substantial gains from prompt optimization alone: GEPA improves Qwen3-8B’s six-benchmark score from 45.23 to 54.85, while OPRO (Yang et al., ICLR 2024) raises PaLM 2-L’s GSM8K accuracy from 71.8% to 80.2%.
W2. Supporting Evidence and Presentation
Thank you for the suggestion. We will revise the manuscript to present the method’s motivation more clearly and better highlight the relevant supporting evidence. Specifically, we address the concerns about generalization and distribution mismatch as follows:
Generalization without optimized prompts. Table 2 already provides a matched ablation addressing this concern. Optimizing template-assisted trajectories under the augmented context rather than the original query reduces template-free benchmark accuracy from 65.2% (P$^2$O) to 55.6% (P$^2$O$_{\textnormal{w/o-Context-Distillation}}$), below GRPO’s 60.5%. This supports the importance of context distillation for transferring prompt-assisted gains to the original-query setting.
Quantifying context-induced mismatch. We measure the original-query to template-augmented token probability ratio in Eq. 6 and track the fraction of response tokens outside the CRM interval [0.01, 10] throughout training.
The excluded fraction rises from 0.22% at step 40 to 2.10% at step 200, as shown below:
In very rare cases late in training, the context ratio exceeds 2 million, indicating severe mismatch between template-assisted rollouts and the original-query learning context and include training instability. CRM excludes tokens outside [0.01, 10] from the policy objective, retaining most response tokens while filtering these outliers. Please refer to our response to Reviewer MoGR, Part 2, W2(c), for the collapse evidence. Together, these results support CRM's role in stabilizing context distillation.
We will make these analyses more prominent when introducing the corresponding components, improving the connection between the method’s motivation, design, and empirical evidence.