- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper integrates self-attribution into LLM optimization pipeline GEPA. Self-attribution measures how much better a candidate's own instruction explains its trajectory than the best competing instruction does. SAGE achieves better results on average over many models and benchmarks.
Critical strengths
-
The method is straightforward, clear, and novel. Same accuracy but different prompt quality is a problem that many researchers have realized, and this paper provides a simple yet effective way to convert this into learnable signals.
-
The evolution scale is large enough. The authors use 4 models, 6 benchmarks, and 4 baselines in their main experiments.
Critical weaknesses
-
The performance improvement is somewhat incremental. As one can see, on Qwen3-4B the improvement is the largest (2.3), but as the model size and capability increase, the improvement becomes smaller. On GPT-OSS-120B, the improvement is only 1.0. On some benchmarks, even improving 1.0 can mean a lot, but on tasks such as AIME/MATH, the gain appears modest. This explains why I am not giving a rating of 4.
-
The ablation experiment is also worth noting. It appears that on large models (Qwen3-30B-A3B and GPT-OSS-120B), adding SA-Proposer will hurt performance.
Questions for authors
-
While the paper is easy to understand for me, it is not necessarily the case for the wider community. For instance, figure 2 is borrowed from the overview figure of GEPA and the authors add modifications to the original pipeline. However, for those who did not see GEPA figure before, this figure can be complex and confusing. I would suggest the authors only plot their modifications or omit too detailed elements in the figure.
-
Can the authors explain why on large models, adding SA-Proposer will hurt performance?
Dear Reviewer Voab,
Thank you again for your detailed review. We wanted to follow up on our responses. We clarified what self-attribution measures and how the theoretical result connects to the gains we observe in practice. We also added experiments to test how stable the selection signal is when we resample responses and to examine when the proposer helps. Our replies also address the diversity baselines, the results across optimization seeds, and the measured runtime.
If you have a chance before the discussion closes, we’d appreciate hearing whether our replies address your main concerns. If any points remain open, we’d be happy to discuss them. Thank you for your time.