acceptodds
Under review as a conference paper at ICLR 2027

From Failures to Evolution: Behavioral Rubrics for Agent Harness Evolution

Abstract

Large language model (LLM) agents increasingly rely on harnesses that manage context and execution strategies beyond the underlying model. Harness evolution provides a promising approach to improving agent capabilities without updating model parameters. Existing approaches increasingly leverage execution evidence and structured feedback to improve agents. However, they usually lack an intermediate behavioral specification that separates **what the agent should do** from **how the harness realizes it**. Such a specification should capture desired behaviors, applicable contexts, and boundaries while remaining independent of specific harness implementations. We propose **rubric-guided harness evolution (RHE)**, which introduces **behavioral rubrics** as an implementation-agnostic interface between execution evidence and harness modifications. We define behavioral rubrics as specifications of desired agent behaviors, applicable contexts, and boundary constraints. By separating behavioral objectives from harness realizations, rubrics enable different implementations of the same behavioral requirement. Hierarchical experience memory further preserves the relationships among execution evidence, rubrics, and historical realizations, enabling reuse across evolution iterations. We evaluate our approach on **SWE-bench Verified** and **AppWorld**. Across three backbone models and two benchmarks, RHE consistently improves agent performance over existing agent improvement approaches. Cross-environment transfer experiments further show that behavioral rubrics discovered from SWE-bench Verified remain effective when transferred to AppWorld. These results demonstrate that behavioral rubrics enable generalizable harness evolution beyond trajectory-specific optimization.

Primary area:
foundation or frontier models, including LLMs
Keywords:
llm agents, harness optimization, self-improvement, harness engineering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4vEjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a method called Rubric-guided Harness Evolution (RHE). It introduces a "rubric layer" that serves as a behavioral guide, positioned between the analysis of execution traces and the modification of the test harness. This layer is decoupled from the underlying implementation details. The system first identifies behavioral flaws within task execution traces and abstracts these flaws into a set of rubrics, which encompass expected behaviors, applicable scenarios, and boundary constraints. Subsequently, the system employs a structural editor and a guided editor to generate modifications for the test harness, selecting the most effective version—based on task evaluation results—for the next iteration. Furthermore, the method incorporates a hierarchical experience memory that stores the relationships between execution evidence (rubrics) and historical modifications, enabling the reuse of past experience across multiple iterations. The authors evaluated the approach using three backbone models on the SWE-bench Verified and AppWorld datasets, including comparisons with AHE, ReasoningBank, and ACE. Ablation studies were conducted to assess the behavioral abstraction module and the experience memory module. The study also analyzed different modification spaces and the transferability of rubrics across diverse environments. Experimental results demonstrate that RHE achieves state-of-the-art performance across all six designated test settings.

Critical strengths
  • The Behavioral Rubric provides a valuable intermediate abstraction of behaviors for Harness Evolution. Instead of generating modifications directly from failures observed in execution traces, the approach first distills behavioral requirements, which are then used to optimize Harness components (specifically Prompts, Memory, Tools, and Middleware). This design decouples behavioral objectives from their concrete implementations. The Rubric can be reused, refined, or expanded based on new evidence; when combined with a hierarchical experiential memory, it forms a continuously updated guidance mechanism—a key contribution of this work.
  • The experimental evaluation is comprehensive, providing evidence of effectiveness across different models, tasks, and modification mechanisms. Across two benchmarks and six distinct test scenarios, covering three backbone models, RHE consistently outperforms baseline methods. Through Tables 2 and 3, the authors validate the impact of behavioral abstraction, experiential reuse, and the design space of Harness implementations; the paper also includes iterative execution results, evolutionary curves, and case studies. Furthermore, cross-environment transfer experiments provide preliminary evidence that the Rubric is reusable. Collectively, these results substantiate the overall efficacy of the proposed approach.
Critical weaknesses
  • Insufficient validation of the independent contributions of the "Behavioral Rubric" and the hierarchical experiential memory. Existing frameworks like AHE and ReasoningBank already leverage trajectory-based experience to optimize the harness. Since the diagnostic phase already identifies generalization deficits, the Rubric merely provides supplementary information regarding specific scenarios and boundary conditions. Furthermore, the two sets of ablation experiments fail to account for differences in API call volume and token consumption. Removing the memory module results in only marginal performance gains, and the study lacks a control group using a "flat" (non-hierarchical) memory structure. Consequently, the current experimental design cannot isolate the individual contributions of each mechanism.
  • Limited evidence for cross-environment generalization; lack of analysis regarding the computational costs of the evolutionary process. The performance improvements gained from transferring the Rubric across environments are modest, and importing the entire set of criteria at once actually degrades performance. The absence of control groups—such as those using randomized criteria or general behavioral benchmarks—makes it difficult to verify whether the observed improvements stem from behavioral knowledge acquired in the source environment. Moreover, the paper reports results based on a fixed budget of ten iterations, without analyzing the impact of iteration-count variability or the total cost of the evolutionary process.
Questions for authors
  • Validation of the contributions of the Rubric and memory modules. Within the constraints of a fixed token budget for alignment analysis, could we compare the full Behavioral Rubric (containing comprehensive behavioral summaries) against a simplified version that retains only the behavioral objectives? Additionally, by including a baseline that utilizes a flat memory retrieval mechanism (based on the same historical information), we could determine whether the hierarchical, relational structure itself provides an independent performance gain.
  • Alignment between the Rubric and the proposed modifications. Could we employ independent behavioral evaluations to verify whether the modifications made to the Harness satisfy the requirements specified in the corresponding Rubric? Furthermore, by comparing the outcomes when the Rubric is correctly matched versus when it is randomly mismatched, we can confirm whether performance improvements are truly derived from effective behavioral guidance.
  • Cross-environment generalization. Could we incorporate a comparable set of randomized Rubrics or generic behavioral guidelines as a control group for transfer experiments? Specifically, if a Rubric learned in a source environment is used to guide modifications to a Harness in a target environment—without undergoing further iterative evolution—can we still observe performance gains resulting from this transfer?
  • Evolutionary overhead and result robustness. It should be clarified whether the task agent and the evolutionary editor share the same backbone model; additionally, the total token consumption and model invocation costs for the entire evolutionary process should be reported. Furthermore, by varying the iteration budget and conducting multiple independent runs, we could analyze the trends and stability of the performance improvements.
Reviewer_mTLhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents an evolution strategy for improving agents without model training, and the modification is applied to the harness. Different from the common approach to have an LLM fix failure directly from the failure traces, which tends to a one time fix, the proposed RHE, which inserts an abstraction step: each failure is first abstrated to a "behavioral rubric" that implies what general behavior should change.

These rubrics serve as requirements for specialized editors, which build executable tools, revised prompt and memory rules/skills. The editor uses a hierarchical experience memory of previous attempts to guide the changes.

The experiments on SWE-bench Verified and AppWorld, with backbones including GPT-5 and etc., indicate RHE outperforms the existing automated harness and memory baselines.

Critical strengths

The paper provided a novel approach to handle agent failures, instead of providing to a quick fix, the proposed approach prepare a clear behavioral rule first, including what to do, when it applies, and what limits. That helps preventing the agent from overcorrecting or accidentally breaking something else in the process.

Since the rules are expressed in general terms, they are not tied to the one specific bug that triggered them. The author demonstrates this with cross-environment transfer experiments, rules learned on coding tasks turned out to help the agent on real-world app tasks as well.

Critical weaknesses

It seems that the rule writing and retrieval processes are not measured in the paper. There's no check on whether the LLM actually writes accurate rubrics, or whether cosine similarity search retrieve the coorect past lessons. The evaluation is focus on the final pass rates, so I was wondering which parts of the pipeline are doing this most work.

The paper doesn't specify how many tasks or what mix of failure types are required to define the rubrics to enable generalizable harness evolution. Also, transferring rules to a new environment seems not to be automaic and relies on human intervention.

Since each round includes running tests, writing rubrics, and patching code, which also require a lot of LLM calls. The paper could provide the actual numbers for cost, wall-clock time, or token consumptions.

Questions for authors

What were the total input/output token counts, and the average wall-clock time per evolution round?

How does RHE do against the baselines once everyone is held to the same inference budget or token cap?

For the cosine similarity search over stored rubrics, do you have any retrieval numbers, like Recall@K, or even a qualitative look at whether the clusters make sense?

1 comment
AuthorsOpenReview ↗

A Kind Reminder Before the Discussion Period Ends (Reviewer mTLh)

Dear Reviewer mTLh, Thank you again for your detailed review. In our replies, we reported human ratings of rubric quality and evaluated retrieval using human judgments of relevance (Recall@3 = 0.72). We also tested whether rubrics selected automatically by an LLM could improve performance on AppWorld and compared RHE with AHE under the same 2.40B-token cap. If your schedule permits, we would appreciate hearing whether these results address your main concerns. We would be glad to clarify any remaining points while the discussion is open.

Reviewer_cUHaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RHE has three main components:

  • Behavioral rubric construction: Execution trajectories are analyzed to identify generalizable deficiencies. Each rubric specifies a desired behavior, applicable context, boundary conditions, and supporting evidence.
  • Harness modification: Structural and guidance editors translate the rubrics into concrete changes to tools, middleware, system prompts, or memory.
  • Hierarchical experience memory: The system stores relationships among execution evidence, rubrics, and historical harness modifications, allowing prior experience to inform subsequent evolution rounds. The authors evaluate RHE on SWE-bench Verified and AppWorld with GPT-5, GLM-5, and MiniMax-M2.5.

The main results show improvements over the initial harness and comparisons with AHE, ReasoningBank, and ACE. Ablations suggest that behavioral abstraction and experience reuse each contribute to performance. A cross-environment transfer experiment examines whether rubrics learned on SWE-bench can improve AppWorld harness evolution.

Critical strengths
  1. Important and practical problem. The paper addresses harness optimization, an increasingly important direction for improving agent capabilities without updating model weights. Separating behavioral requirements from their concrete harness implementations is a sensible approach to improving the generality and reusability of failure-driven agent optimization.

  2. Systematic framework with encouraging experimental results. RHE combines behavioral abstraction, harness modification, and hierarchical experience memory in a coherent framework. Experiments across two benchmarks and three models show consistent held-out improvements over existing agent improvement methods. The ablations provide useful evidence for the contributions of behavioral abstraction, experience reuse, and structural versus guidance modifications.

Critical weaknesses
  1. Limited novelty and unclear necessity of behavioral rubrics. While separating behavioral requirements from harness implementations is reasonable, the distinction from existing trajectory-based diagnosis, reflection, and structured experience learning is not sufficiently established. The proposed rubrics largely capture generalized behavioral guidance that existing methods may also discover. The ablation shows improvements, but the gains attributable specifically to the rubric abstraction are relatively modest. I would like to better understand what capabilities this representation enables that cannot be achieved through structured failure analysis and direct harness optimization.

  2. Rubric quality and reliability are insufficiently evaluated. The approach relies heavily on LLM-generated failure diagnoses and behavioral abstractions. Incorrect attribution of failure causes could produce misleading rubrics, which may then be reused across evolution rounds. Although the framework includes evidence grounding and boundary conditions, the paper does not directly assess the correctness, consistency, or generality of the generated rubrics. It is unclear how reliably the system distinguishes genuine behavioral deficiencies from task-specific failures or coincidental correlations.

  3. Limited evidence for cross-environment generalization. The paper emphasizes transferability as a key benefit of behavioral rubrics, but the evidence is mixed. Importing all 40 SWE-bench rubrics reduces AppWorld held-out performance from 60.49% to 59.26%, while human-selected and LLM-selected subsets improve performance by only 1.11 and 0.62 percentage points, respectively. Furthermore, the transfer experiments still involve target-environment evolution. These results demonstrate some benefit from applicability-aware selection, but do not sufficiently establish generalizable behavioral transfer.

  4. Evaluation robustness and overfitting require further investigation. Harness candidates are repeatedly evaluated and selected using the same held-in task sets over ten evolution rounds. Although the held-out results are encouraging, repeated optimization on a fixed task distribution may favor benchmark-specific strategies. The main baseline comparisons report repeated evaluations of selected harnesses rather than independent evolution runs. More comprehensive analysis of optimization variability, task-distribution shift, and retention of previously acquired capabilities would strengthen the conclusions.

Questions for authors
  1. Contribution of behavioral abstraction: What is the essential advantage of behavioral rubrics over structured trajectory diagnosis or reflection? Can the authors identify examples where the rubric representation enables an improvement that direct harness optimization cannot achieve?

  2. Rubric quality: How is the correctness of the generated behavioral requirements assessed? Have the authors evaluated cases where an incorrect failure diagnosis leads to an inappropriate rubric or harmful harness modification?

  3. Cross-environment transfer: Why does transferring all rubrics degrade performance? Can the authors evaluate zero-shot rubric transfer without further target-environment evolution to better isolate the transfer benefit?

  4. Generalization and overfitting: How sensitive are the results to the evolution task distribution? Do the learned harness improvements transfer to new repositories, task categories, or environments that are not represented in the evolution set?

  5. Stability and retention: As rubrics and harness implementations accumulate, how does the framework detect conflicting requirements or regressions in previously improved behaviors? Can the authors provide longitudinal analysis of capability retention across evolution rounds?

  6. Optimization cost: What is the total cost of rubric construction, memory retrieval, candidate generation, and evaluation compared with AHE? Are the performance improvements maintained under matched end-to-end optimization budgets?

1 comment
AuthorsOpenReview ↗

A Kind Reminder Before the Discussion Period Ends (Reviewer cUHa)

Dear Reviewer cUHa, Thank you again for your detailed review. In our replies, we gave concrete examples of how a behavioral rubric turns a diagnosis from one execution trace into a requirement that can guide harness edits across tasks. We also reported independent human ratings of rubric quality, compared edits guided by correctly matched and randomly mismatched rubrics, and presented additional transfer results on AppWorld and SWE-Pro. If your schedule permits, we would appreciate hearing whether these examples and results address your main concerns. We would be glad to clarify any remaining points while the discussion is open.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.