acceptodds
Under review as a conference paper at ICLR 2027

Execution-Grounded Credit Assignment for Self-Evolving Agents

Abstract

# Abstract Recent research on the self-improvement of language model agents has shown that iterative refinement guided by execution feedback is effective. On long-horizon tasks, however, credit assignment remains difficult because task outcomes reflect many interdependent actions. After a change, an agent may succeed without exhibiting the behavior the change was intended to produce, or fail despite exhibiting it. We propose an evolution method that uses execution-grounded component-level credit assignment to guide changes to instructions, tool code, and skills. Context-Anchored Gene Mutation derives mutation intents from outcome-associated behavioral patterns in the parent agent's execution histories and combines them into plans constrained by evolution context accumulated over earlier rounds. Applying these plans to the parent generates candidate agents, with each change linked to its intent. To explore diverse mutation candidates while limiting costly full task evaluations, we use simulation-based candidate pruning. It simulates the parent and each candidate for a bounded number of steps from recorded decision points relevant to each intent, under matched language-model request conditions. Candidates that express the intended behavior more strongly than the parent in these simulations are prioritized for full task evaluation. After evaluation, Gene Distillation combines each change's source pattern, intent, observed behavior, and task outcome to update its credit. Stronger evidence of expression increases the weight of both positive and negative outcomes. Credit accumulates in the evolution context to guide subsequent mutations. We evaluate the method on SEAL-0, which requires iterative retrieval and verification of potentially conflicting web evidence, and on EnterpriseOps-Gym, which tests stateful planning and tool use in enterprise workflows with interdependent steps. After evolution on five SEAL-0 tasks, accuracy on all 111 tasks, including the development tasks, rises from 41.44% to 63.36%; GEPA (adapted) reaches 50.45% on the same 111 tasks. After evolution on eight EnterpriseOps-Gym tasks, the evolved agent is evaluated on 642 tasks, achieving a task pass rate of 35.67% and a verifier-level pass rate of 68.9%, compared with 26.95% and 59.3%, respectively, for the initial agent. These results suggest that credit assignment for long-horizon agent evolution can be grounded in observed behavior as well as task outcomes.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language model agents, self-evolving agents, credit assignment, execution feedback, evolutionary search, long-horizon tasks

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_XTY4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper addresses component-level credit assignment for self-evolving LLM agents on long-horizon tasks, where a single task outcome cannot reveal which edits actually produced their intended behavior. The method registers each mutation intent before editing, then uses paired replay from a shared parent checkpoint to test whether each candidate expresses the intended behavior. The resulting expression signals are used to prune candidates before full evaluation and to weight outcome-based credit. These records accumulate in an Evolution Context that guides later mutations. The method is validated on SEAL-0 and EnterpriseOps-Gym against an adapted GEPA baseline, with additional pruning and context ablations.

Critical strengths

S1: Meaningful problem formulation. The paper separates "an edit that was not executed" from "an edit that was executed but ineffective," which addresses a real and underexplored source of noise in outcome-only credit assignment for long-horizon agent evolution.

S2: Exceptional transparency. The appendices disclose full evolution archives (including negative entries), complete agent configurations, lineage trees, and per-task justifications for benchmark exclusions, and even report details unfavorable to the authors.

Critical weaknesses

W1: Lack of direct validation for the core contribution. The paper's central claim is that credit grounded in behavioral expression is superior to credit based solely on task outcomes. However, no experiment isolates this comparison while holding all other factors fixed: Section 4.5 removes the entire Evolution Context, while Section 4.4 simultaneously changes both candidate generation and pruning. Moreover, 12 of the 13 entries in the final EnterpriseOps archive carry negative importance, including components actually used by the best-performing agent. This mismatch between credit sign and the components' actual contribution undermines the credibility of the claimed "component-level credit assignment."

W2: Severely insufficient baselines and unfair comparisons. The experiments include almost no comparison with related work. On SEAL-0, the only baseline is GEPA, adapted by the authors themselves, which stagnates after its second iteration; on EnterpriseOps, no external method is evaluated at all. The most relevant prior works, such as EVOTOOL, ADAS, DGM, and AFlow, appear only in the related-work section. The most natural baseline is also missing: low-fidelity screening of all candidates on a task subset under a matched compute budget. Section 4.4 justifies fairness by matching the number of full task evaluations, yet the replay overhead alone exceeds the task-evaluation cost, so the pruning condition consumes more than twice the total compute of its counterpart. Consequently, the current results cannot disentangle the contribution of the method itself from the gains brought by additional compute.

W3: The applicability boundary of the core mechanism remains unclear. Distinguishing "an edit that was not executed" from "an edit that was executed but ineffective" is a reasonable motivation, but its effectiveness rests on several premises that the paper does not test, and these premises break down in different ways across model scales and task environments. First, short-horizon continuation from a parent checkpoint implicitly assumes that edits affect only local decisions; the more an edit restructures the overall workflow, or the longer, more stateful, and more nondeterministic the task, the larger the gap between replay and actual execution, and the less reliable the check becomes. Second, "expression" measures compliance with the new instruction rather than whether the edit is effective, and the relationship between the two is never validated. The discriminative power of this signal also depends on model scale: strong models almost always comply with instructions, so expression saturates and offers little additional information, whereas weak models behave stochastically, making the expression signal noisy and requiring many replay repetitions to stabilize. The regime in which the mechanism is genuinely effective may therefore be limited to mid-capability models and a narrow class of tasks where evaluation is expensive and local replay is feasible. Finally, the sign of credit is determined entirely by task outcomes; after normalization and LLM-based adjudication, the actual contribution of expression weighting to final decisions is unclear.

W4: The paper as a whole exhibits the immature characteristics of an auto-research output. The method overview (Figure 1) is a raster image with dense elements and very small text, and its layout resembles the output of a text-to-image model, making it difficult to serve as an accurate description of the method; the remaining figures contain annotations that do not correspond to their axes and residual internal run identifiers. The pseudocode consists mainly of opaque function calls and steps such as "update using an LLM." The writing relies heavily on metaphors, and terms are often used before being defined; the main-text method section is under two pages, while the appendix consists largely of verbatim dumps of prompts and configurations. The experimental setup is simplistic and insufficiently rigorous: the development sets contain only 5–8 tasks, each condition is run with a single evolution run, and only one model family is used. Overall, the paper reads more like an intermediate product of an automated research pipeline than a mature work refined under a unified experimental protocol.

Questions for authors

Q1 : Under a matched compute budget, does paired-replay pruning outperform simply screening all candidates on a task subset, and if not, what does the replay mechanism contribute beyond additional compute?

Q2 : With all other components held fixed, how does expression-weighted credit compare against outcome-only credit, and how do you reconcile the predominantly negative final archive with the fact that the best agent uses those very components?

Q3 : The headline Qwen3.5 SEAL-0 run was restarted from the R4 parent after a bug, and the final agent was generated in R6. Do the results hold across multiple independent, uninterrupted evolution runs?

Reviewer_cX2WOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a fine-grained method to assess the effects of harness evolution. It uses paired replay to evaluate intended behavioral changes and assigns component-level credit to guide subsequent evolution. Experiments show improvements on SEAL-0 and EnterpriseOps-Gym.

Critical strengths

The problem of assigning credit to individual modifications based on execution evidence during harness evolution is well motivated.

Critical weaknesses
  • The soundness of the proposed method remains unclear. Paired replay may not reflect the behavior of modified agents, as earlier decisions may change and execution states may not be accurately restored. Moreover, observing intended behaviors does not necessarily imply that they contribute to task results.
  • More ablation studies are needed. The paper neither directly evaluates the accuracy of component-level credit assignment nor isolates the benefit of expression-weighted credit.
  • The robustness, sensitivity, and generalizability of the method remain unclear. This paper should report the mean and variance across multiple independent evolution runs to distinguish method gains from run-to-run variability. Moreover, evolution is performed on only 5 or 8 development tasks, without analyzing sensitivity to task selection or set size. Additionally, can this method generalize to more other realistic tasks (e.g., swe tasks)?
Questions for authors

See weaknesses.

Reviewer_qCAHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an execution-grounded framework for improving language-model agents through edits to instructions, tools, and skills. It addresses the mismatch between task outcomes and whether individual edits produce their intended behaviors. Context-Anchored Gene Mutation generates candidate edits from execution patterns and an accumulated Evolution Context. Paired replay compares parent and candidate behavior at matched historical decision points, allowing candidates to be screened before full-task evaluation. Gene Distillation then combines behavioral expression with parent-relative outcomes to update component-level credit and guide subsequent mutations. On SEAL-0, evolution using five development tasks improves Qwen3.5 accuracy from 41.44% to 63.36%, versus 50.45% for adapted GEPA; the advantage persists on 101 held-out items. On EnterpriseOps-Gym, evolution using eight tasks increases task pass rate from 26.95% to 35.67% across 642 tasks. Context ablation and reuse experiments provide additional evidence for accumulating evolutionary experience. The results are promising, although the experiments do not yet isolate the contribution of expression-based credit assignment or establish robustness across independent evolution runs.

Critical strengths
  1. The framework explicitly separates whether an edit exists, whether its intended behavior occurs, and whether performance improves. Prospective intent registration and the distinction between NOTEXPRESSED and UNOBSERVED provide a useful structure for interpreting evolutionary feedback.

  2. The SEAL-0 advantage survives excluding all development items used by either method: 63.70% versus 50.17% for adapted GEPA on 101 held-out items. Improvements on EnterpriseOps-Gym extend the empirical evidence to stateful tool-use tasks. (Tables 2–3)

  3. The method supports edits to instructions, tool code, and skills, while retaining evidence from successful and unsuccessful mutations. The context ablation—81.68% with context versus 74.77% without—provides encouraging evidence that accumulated experience is useful.

  4. The appendices document agent prompts, tool inventories, evolution lineages, archive entries, benchmark exclusions, and evaluation failures. The main GEPA comparison also controls the initial agent, editable components, tools, and scorer, improving interpretability. (Appendices A–F)

Critical weaknesses
  1. All eligible edits inherit the same candidate-level outcome sign, with expression evidence only redistributing magnitude. A harmful edit can therefore receive positive credit when bundled with a more beneficial edit. Although the method does not claim causal identification, component interventions or deletion studies are needed to validate the usefulness of its attribution. (Appendix G)

  2. The pruning comparison simultaneously increases proposals from three to nine, introduces diversity guidance, and applies replay selection. The context ablation removes the archive rather than isolating expression-based credit. Necessary controls include selection from identical candidate pools and an otherwise identical archive using outcome-only credit.

  3. Each condition uses only one evolution run. Three evaluations of the selected agent quantify execution variability, not uncertainty in the search process or sensitivity to the five/eight development tasks. Multiple independent evolution runs and alternative development subsets are needed to establish robustness.

  4. Partial-expression weights, the 0.2 expression threshold, entropy-based consistency, and LLM-curated importance lack calibration or sensitivity analysis. Single-target normalization also produces allocation ±1 regardless of raw expression weight, leaving evidence strength partly to downstream curator rules. Independent judgment validation and comparisons with simpler scoring rules would strengthen the method. (§3.3; Appendix G)

  5. The reported 8.34% overhead compares replaying one candidate with fully evaluating one candidate. Using the reported averages across 106 replayed and 21 evaluated candidates implies aggregate replay overhead of approximately 42% relative to candidate-evaluation tokens. Generation, reflection, and archive-management costs remain outside the comparison; complete token and latency accounting is needed. (§4.3; Appendix F)

  6. Adapted GEPA is evaluated only on Qwen3.5/SEAL-0; EnterpriseOps-Gym lacks an external optimizer baseline, and Qwen3.8 lacks a GEPA comparison. Context transfer also reuses the same model, initial agent, and benchmark, establishing reuse within that setting rather than transfer across models or task families. (Tables 2–3; Appendix A.4)

Questions for authors

see weaknesses

1 comment

Overall, I like this paper. If the authors can address my questions during the discussion period, I would consider raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.