acceptodds
Under review as a conference paper at ICLR 2027

Surgical Meta-Optimization for Agent Evolution with Minimal Samples

Abstract

The transition of Large Language Models (LLMs) into autonomous agents requires ways to improve an agent after it is built, without retraining the backbone. However, prevailing paradigms are largely trapped in a brute-force cycle, relying on massive computational resources, extensive annotated data, and unbounded search spaces. This resource-heavy approach hits a strict ”resource wall” in practical, locally deployed scenarios. To address the challenge of extreme compute and data starvation, we propose a frugal offline agent-evolution framework driven by structure-aware surgical intervention. Rather than pursuing computationally prohibitive full-scale global optimization, we restrict the search to two components of the agent—its system prompt and its controller logic—that can be edited and verified without touching model weights or tool APIs. This is achieved through a two-stage meta-optimization in which each stage optimizes one component independently while holding the other fixed at its baseline. Stage I employs a synergistic “Thought-Prompt” co-evolution to rapidly discover a compact, transferable reasoning mindset. Stage II utilizes a contrastive reflection-guided process to independently harden the programmatic execution loop against structural brittleness. Remarkably, operating under a strict budget of merely around 140 candidate evaluations and 30 sparse evaluation samples, our evolved agent successfully tackles expert-level, unseen tasks. Extensive evaluations on exceptionally challenging benchmarks, including Humanity's Last Exam (HLE), demonstrate that strategic, localized component optimization enables stable generalization without requiring massive data or heavy training dependencies. Code is available at Supplement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agent, Self-evolution, Localized optimization, Sparse learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_D3QvOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a "harness evolution" framework consisting of two stages. Starting with "Web Thinker" as the initial harness—while keeping the underlying model API constant—the authors independently search for an improved prompt and an improved harness (controller) code.

Leveraging an external language model (LM), they perform an iterative process—akin to meta-learning—using a very small development set. Finally, they combine the selected prompt and controller (i.e., the harness) and evaluate the resulting agent on specific subsets of the HLE and GAIA benchmarks.

The authors argue that modifying these two components allows for the optimization of agent performance with fewer samples and lower costs. Both components prove effective—yielding even better results when combined—and demonstrate a degree of generalizability.

Critical strengths

None

Critical weaknesses
  1. There is a significant contradiction between the statement in the main text regarding the optimization data’s independence and the information provided in the supplementary material.

In lines 334–336, the authors explicitly state that the 30 optimization problems used across the two stages do not overlap with the GAIA or HLE datasets, and that no problems from these benchmarks were utilized during the evolutionary process. However, the submitted Stage II code directly loads GAIA data. More importantly, a set of 20 problems found in four candidate evaluation logs—including both the problem prompts and the ground-truth answers—matches entries in the final 103-problem GAIA benchmark exactly. Furthermore, these evaluation logs can be used to reproduce the 52.4% figure reported in the main table.

This indicates that the candidate optimization problems in the submission overlap with the final GAIA benchmark, which directly contradicts the claim in the main text regarding data isolation. Consequently, the reported GAIA results cannot be considered a strictly independent generalization evaluation.

  1. The main table lacks sufficient detail regarding the evaluation subsets and the sources of the baselines, making the comparability of the results unclear.

While the authors state generally that they follow the evaluation setup used in WebThinker, the main table fails to clearly distinguish between the full benchmarks and the specific subsets actually employed. It also does not specify, on an item-by-item basis, whether the baselines utilized the same problem sets and scoring protocols. For instance, the HLE results in the supplementary material are calculated using a denominator of 500 problems, rather than the full HLE dataset. While using a subset is not inherently problematic, the composition of the subset, the selection criteria, and whether all direct comparisons are based on the identical subset must be explicitly stated.

Questions for authors

Please show the evolving details(FULL TRAJS ON ALL EPISODES), so reviewers can check whether there is tuning on the test set.

Reviewer_QYgeOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a frugal strategy to perform harness and tool-environment meta-optimization. It proposes a cheap two-stage approach that learns by adjusting only the harness and tool calling, with no gradient updates, and demonstrates how this transfers to other benchmarks and even surpasses RL post-trained and search-specific models.

Critical strengths

The paper introduces a data-efficient paradigm for agentic harness optimization, as well as a novel two-stage approach to perform it. The second part of the two-stage approach is probably more novel than the first.

Critical weaknesses

The ablations are weak, and it is therefore hard to distill where the gains come from. Experiments are performed with just one seed, and the 30 training samples are undisclosed. The paper does not compare its results against other well-established meta-harness baselines like "Meta-Harness: End-to-End Optimization of Model Harnesses" or "SkillOpt: Agent Skills as Trainable Parameters." Also, the choice of benchmarks used is strange, as GAIA and HLE are not very aligned and are not standard for meta-harness evaluation.

Questions for authors

1- What are the 30 training examples? 2- Do the results hold for multiple seeds? 3- Are there per-component ablations on HLE? 4- What about other ablations, such as the budget or the order of Stage 1 and Stage 2, or the components of each stage? What if we drop diversification from Stage 1, or remove the contrastive hint from Stage 2? In general, it is not clear exactly which components drive what.

Reviewer_vfnbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free, offline procedure for improving a tool-using LLM agent under a very small evaluation budget. The agent is WebThinker with a frozen QwQ-32B backbone. It is decomposed into fixed infrastructure, a system prompt P, and controller code L. Each component is optimized in its own track while the other is held at its baseline:

Stage I evolves the prompt using an LLM meta-optimizer with diversify, recombine, and refine operators. It evaluates 77 candidates, each scored on 10 questions.

Stage II evolves the controller through “contrastive reflection.” The meta-optimizer compares a better and a worse controller, writes a repair hint into a persistent memory, and regenerates the code. It evaluates 60 candidates, each scored on 20 questions.

Critical strengths

The setting is practical and well-motivated: improving a deployed agent without weight updates under a hard budget.

Critical weaknesses
  1. The evolution data is unclear and may overlap with GAIA. Section 5.1 states that the 30 evolution questions come from “sources disjoint from both evaluation benchmarks,” but the source is never named. The Stage II meta-optimizer prompt (App. C.5) tells the optimizer: “You will be evaluated on 20 questions from GAIA.” Section 5.4 refers to “the GAIA-optimized components.” If D_L consists of GAIA questions, then the GAIA results are at best in-distribution generalization from 20 examples, not held-out evaluation. Question-level overlap with the 103-question subset would also need to be ruled out explicitly. Under that reading, HLE is the only genuine transfer test, and the gain there is 2.6 points (about 13 of 500 questions).

  2. There are no equal-budget baselines, and novelty relative to prior work is limited. The paper argues that prior training-free optimizers (DSPy, GEPA, ADAS, AFlow, Meta-Harness) need thousands of rollouts, but none of them is run at the same ~140-evaluation budget. There is no random-search or best-of-N baseline (e.g., sampling the same number of prompts or controllers from the meta-optimizer without the evolutionary or reflective operators). Without these comparisons, it is unclear whether the gains come from the specific operators, the two-component decomposition, or simply from asking a strong LLM to rewrite the prompt and controller a few dozen times. Stage I is close to existing evolutionary prompt optimization (EvoPrompt, Promptbreeder, OPRO), none of which is cited. Stage II closely resembles ReEvo-style reflective evolution and GEPA’s reflective mutation.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.