acceptodds
Under review as a conference paper at ICLR 2027

SAGE: Self-Attribution-Guided Evolving

Abstract

Prompt evolution optimizes the instruction given to a frozen language model: it maintains a population of candidate instructions and repeatedly mutates, evaluates, and selects them. Existing prompt optimizers select candidates by task accuracy alone, but accuracy cannot tell whether a correct answer is due to the instruction. Two instructions can be equally accurate even when one induces a distinct solution strategy and the other merely rewords an existing candidate and leads the model to behave the same way. We formalize this blind spot: among candidates whose rewards follow the same distribution, the rewards carry no information about which candidate produced them, whereas the model's trajectories do. To recover this information, we introduce self-attribution, which measures how much better a candidate's own instruction explains its trajectory than the best competing instruction does, and we prove that it yields a computable lower bound on the information that accuracy discards. We then propose SAGE (Self-Attribution-Guided Evolving), which adds this signal to two decisions of GEPA, a state-of-the-art prompt optimizer: which candidates are selected as parents, and which mutated children are kept. Across four open-weight models spanning a range in size and six tasks, SAGE achieves the highest average accuracy on every model and improves over GEPA by up to 4.0 points. It also ends below its starting prompt in only 3 of 24 model–task pairs, compared with 12 for GEPA. These gains persist on harder out-of-distribution problems, under cross-model prompt transfer, and with a black-box API model whose self-attribution is computed by an open-weight surrogate.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Prompt Evolving, Prompt Optimization, Self-Attribution, LLM Inference-Time Methods

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_e6g9OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper integrates self-attribution into LLM optimization pipeline GEPA. Self-attribution measures how much better a candidate's own instruction explains its trajectory than the best competing instruction does. SAGE achieves better results on average over many models and benchmarks.

Critical strengths
  1. The method is straightforward, clear, and novel. Same accuracy but different prompt quality is a problem that many researchers have realized, and this paper provides a simple yet effective way to convert this into learnable signals.

  2. The evolution scale is large enough. The authors use 4 models, 6 benchmarks, and 4 baselines in their main experiments.

Critical weaknesses
  1. The performance improvement is somewhat incremental. As one can see, on Qwen3-4B the improvement is the largest (2.3), but as the model size and capability increase, the improvement becomes smaller. On GPT-OSS-120B, the improvement is only 1.0. On some benchmarks, even improving 1.0 can mean a lot, but on tasks such as AIME/MATH, the gain appears modest. This explains why I am not giving a rating of 4.

  2. The ablation experiment is also worth noting. It appears that on large models (Qwen3-30B-A3B and GPT-OSS-120B), adding SA-Proposer will hurt performance.

Questions for authors
  1. While the paper is easy to understand for me, it is not necessarily the case for the wider community. For instance, figure 2 is borrowed from the overview figure of GEPA and the authors add modifications to the original pipeline. However, for those who did not see GEPA figure before, this figure can be complex and confusing. I would suggest the authors only plot their modifications or omit too detailed elements in the figure.

  2. Can the authors explain why on large models, adding SA-Proposer will hurt performance?

Reviewer_VoabOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SAGE, a prompt optimization method guided by self-attribution. It measures how well a candidate instruction explains its own generated trajectories compared with competing instructions, and incorporates this signal into parent selection and child generation in GEPA. The authors provide information-theoretic analysis and evaluate the method across four open-weight models, six benchmarks, and several transfer settings.

Critical strengths
  1. The paper addresses an interesting limitation of accuracy-only selection in prompt evolution and proposes a complementary signal to guide the search.
  2. The paper is generally well written and clearly organized, with detailed methodological and experimental descriptions.
Critical weaknesses
  1. The connection between the theoretical results and practical optimization gains is insufficiently established. Equations (1)–(2) show that trajectories contain information about candidate identity that rewards cannot capture when reward distributions are identical. However, this does not establish that such information helps identify better prompts. Moreover, GEPA [1] already uses execution traces for reflection and mutation, so it cannot be treated as a purely reward-based method. The relevance of the theoretical results to the improvements over GEPA needs further clarification.
  2. It is unclear whether self-attribution truly captures distinct reasoning strategies. The metric measures the log-likelihood advantage of a trajectory under its own instruction over competing instructions, but this difference may reflect output style, wording, or formatting rather than meaningful differences in reasoning. More direct evidence linking self-attribution to useful reasoning strategies is needed.
  3. The reliability of self-attribution as a selection signal is insufficiently evaluated. The metric depends on sampled trajectories and the current candidate pool, making it potentially sensitive to sampling randomness and pool composition. Furthermore, generation uses truncated sampling and sampling penalties, while attribution is computed using teacher-forced log-likelihoods. The stability of the signal under different settings remains unclear.
  4. The ablations do not fully justify the necessity of all components. Table 2 shows that SA-Pareto alone outperforms full SAGE on Qwen3-30B-A3B and GPT-OSS-120B. Although the authors discuss this observation, the independent benefits of SA-Proposer and the acceptance gate remain unclear, particularly for larger models.
  5. Comparisons with simpler diversity-based approaches remain limited. Although Appendix H includes an embedding-novelty baseline, it is evaluated only on Qwen3-4B with a single optimization run. More comprehensive comparisons with simpler prompt-diversity or trajectory-similarity measures are needed to establish the unique benefits of self-attribution.
  6. The main results lack sufficient evaluation across optimization seeds. Most of the 24 model–task results come from a single optimization run, with three-seed experiments conducted only on Qwen3-4B. Test-instance bootstrapping does not capture the randomness of prompt evolution itself. Moreover, the average improvement on GPT-OSS-120B is not statistically significant, leaving the consistency of gains across models uncertain.
  7. The computational efficiency comparison remains incomplete. Although self-attribution adds only about 5% wall-clock overhead in the reported setting, Appendix C shows substantial additional prefill computation. The experiments primarily match metric calls rather than total FLOPs. The scalability and cost-effectiveness of cross-scoring trajectories against larger candidate pools require further evaluation.

[1] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning.

Questions for authors

See Weaknesses.

2 comments

Dear Reviewer Voab,

Thank you again for your detailed review. We wanted to follow up on our responses. We clarified what self-attribution measures and how the theoretical result connects to the gains we observe in practice. We also added experiments to test how stable the selection signal is when we resample responses and to examine when the proposer helps. Our replies also address the diversity baselines, the results across optimization seeds, and the measured runtime.

If you have a chance before the discussion closes, we’d appreciate hearing whether our replies address your main concerns. If any points remain open, we’d be happy to discuss them. Thank you for your time.

Thank you to the authors for the detailed and thoughtful rebuttal. The rebuttal has addressed most of my concerns. I appreciate the authors' efforts and have decided to raise my score.

Reviewer_sUJLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes SAGE, a prompt optimizer guided by both accuracy and self-attribution, a quantity proposed to assess how distinct/unique each prompt is in leading to the generated trajectory.

Critical strengths
  • Prompt optimization is an important engineering question
  • The idea of self-attribution makes sense as an additional signal
Critical weaknesses
  • The authors could improve on explaining the highlighting the importance of self-attribution through targeted analysis
  • The contribution, changing two steps in a typical prompt optimizer, might not be technically substantive
Questions for authors

Overall I like the idea of self-attribution as a signal, finding new prompts that lead to unique behavior not already explainable by other prompts.

  • I like how the authors motivated with finding prompts that "induce a distinct solution strategy", it would be great to 1) do a quantitative comparison of how often SAGE steered the system away from repetitive prompts and actually led to prompts inducing distinct strategy; 2) examples of "distinct" and otherwise repetitive prompts as a result of applying self-attribution.

  • The authors could better describe/motivate self-attribution in the abstract/introduction, what it means and its value is not immediately clear until the method section.

  • How often would the self-attribution margin introduced in Sec 2.3 be positive/negative? It would be great to help readers understand the level of uniqueness usually seen in prompt optimizers.

  • When introducing self-attribution, the authors rely on an assumed scenario of ``equal-reward-law class’’, essentially prompts scored the same in accuracy and thus could not be distinguished. Is that very common though? Among a minibatch of n problems and k samples per problem, this would require n*k generations to land on the exact same score. Is that a common scenario worth looking into in the first place?

  • On a similar note, the authors suggest that the self-attribution step is only applied when it is hard to separate by accuracy. Again, it would be great to know how often, quantitatively, is the self-attribution step invoked and whether that varies across settings.

  • Why is (gpt-oss-120b, mmlu-pro) in Table 1 negative across the board? What’s so special about this dataset and model combination?

  • This reminds me of research about noisy channel models, where the output->input derivation resembles the trajectory->prompt calculations here. For the authors’ reference, [1].

  • The best-found prompts through SAGE listed in the appendix seem hyper-specific, listing a lot of "domain knowledge" that seems overfitted to the evaluated task, leaving me unsure about its generalization. That is perhaps a common limitation of prompt optimizers though.

  • The title is overly broad… Please scope it to prompt optimization.

  • Overall the contribution seems to be applying the log likelihood comparisons (self-attribution) to two-steps in GEPA, some might say there isn’t too much technical contribution, but perhaps it is not a huge issue either? The authors are free to argue otherwise and highlight the technically interesting parts.

[1] Min et al. "Noisy channel language model prompting for few-shot text classification."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.