acceptodds
Under review as a conference paper at ICLR 2027

Harness: Recursive Agent Harnessing for an Open World

14 views

Abstract

LLM agents' capabilities depend not only on their underlying models but also on their execution harnesses. Recursive self-improvement (RSI) offers a promising approach to automatically evolve these harnesses. However, deploying RSI in open-world settings still faces two critical challenges. First, open-world tasks are typically unseen during the harness design phase and novel at test time, leaving no ground-truth verifiers to reliably guide online harness edits. Second, the RSI process remains highly task-specific and fails to generalize to novel scenarios. We introduce Harness, a plug-and-play *recursive agent harnessing* framework that addresses these challenges through two connected levels of recursion. *Task-level recursion* refines the current task's harness using a contrastive Propose–Probe–Compose step that reliably guides edits without ground-truth verifiers and can be scaled further via parallel or sequential recursion. *Harnessing-level recursion* then extracts experience from these edits to update a continual improvement harness, enabling the agent to better adapt to future tasks. Across three professional benchmarks and four model–harness configurations, a single step consistently improves GLM-5 and Gemini 3.5 Flash over their base harnesses, yielding gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB, with further gains as the task-level recursion budget increases.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Harness, Recursive Self-Improvement, Harness Evolution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_j5QFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HARNESS², which improves a frozen LLM agent’s harness at test time. For each task, it uses a Propose–Probe–Compose procedure to generate invariant and domain-specific harness edits, execute them, and combine changes based on observed behavioral differences, without access to ground-truth rewards or benchmark scores. It further stores adopted harnesses and distilled improvement experience in a persistent library to guide harness refinement on future tasks. It is evaluated on WorkBuddy-Bench, JobBench, and LAB, showing consistent gains over several baselines.

Critical strengths
  • The paper is well written and accompanied by nicely made figures. It was easy to follow the paper.
  • The method shows consistent improvements across multiple benchmarks, with extensive ablation studies, which provides solid evidence for its effectiveness.
  • I appreciate that the authors compare against Best-of-k style test-time scaling, which helps isolate the effect of additional compute from the benefit of recursive harness refinement itself.
Critical weaknesses
  • My main concern is the stability of the system over time. I understand why the authors avoid ground-truth feedback, since it is often unavailable in real-world deployment. However, behavioral analysis alone does not necessarily reflect task correctness. This is especially concerning for cross-task lessons, where an incorrect lesson may be reused and cause errors to accumulate. The paper shows some failures caused by shared incorrect assumptions, but does not directly study this long-term accumulation. While the reflector can reject unsupported lessons, I did not find a mechanism for later removing or revising outdated lessons. It would be useful to evaluate this issue or introduce a filtering mechanism for the library.
  • Besides, the evaluated tasks are mostly specialized tool-using agent tasks. I would like to see whether the method also works on qualitatively different tasks, such as mathematical reasoning or more ambiguous conversational tasks.
Questions for authors

Could the authors analyze

  1. the cross-task stability of the library, including whether filtering or removing outdated lessons improves robustness, and
  2. demonstrate the method on qualitatively different tasks such as mathematical reasoning?
3 comments
AuthorsOpenReview ↗

Response to Reviewer j5QF

We thank the reviewer for the positive assessment of the presentation, the breadth of the ablations, and the best-of-k comparison. We address the two concerns below. The additional OfficeQA and IMO-AnswerBench experiments reported below use GLM 5.3 Flash as both executor and editor, with no ground-truth answers or evaluation rewards used during improvement. —

W1. Cross-task stability of the library

We agree that behavioral contrasts do not establish correctness, and that an incorrect lesson could be reused and compound across tasks. The current design limits this risk in three ways, and we add a new experiment evaluating periodic library maintenance.

Existing safeguards:

  • Evidence-gated writing. The Reflector stores a lesson only when execution contrasts support it. Edits that cannot be isolated are labeled confounded, and when evidence is insufficient the Reflector writes NO_ENTRY instead of a guess.
  • No automatic inheritance. Every task starts from the common base harness H₀. A library entry influences a task only if the Editor chooses to adapt it, and the prompts require re-verifying any task-specific fact against the current task's evidence.
  • Re-probing. A reused component must pass the current task's Probe and Compose before it is adopted. A lesson that misleads on a new task therefore has to produce a supported behavioral contrast before it re-enters the library.

Q1. Filtering mechanism for cross-task stability of the library

To further test the effectiveness of whether a filter mechanism can help the long run, We run Harness² on the 172 tasks from OfficeQA [1] stream used in Q2 and compare two library variants:

  1. Harness²: the original design as described in the paper.
  2. Harness² + meta-reflection: the same pipeline. In addition, every 10 tasks a meta-reflection session (same model, label-free) reviews the whole library together with usage evidence from the stream: which later sessions opened, cited, reused or adapted each entry, and the Reflector's verdicts on components carried over from it. For each entry it decides KEEP, MERGE, REVISE, RETIRE or DELETE.
MethodOverall
Base harness55.3
Harness²63.9
Harness² + meta-reflection65.1

The original Harness² improves over the base by 8.6 points in aggregate over this 172-task stream, and periodic meta-reflection adds a further 1.2 points. These results support the usefulness of library maintenance in this setting, but aggregate scores alone do not establish whether errors accumulate over time or whether the improvement persists over longer streams.

W2 / Q2. Qualitatively different tasks

We evaluate Harness² on two benchmarks that differ from our main tool-using, artifact-producing tasks:

  • IMO-AnswerBench [2]: olympiad-level math problems with short answers. There is little tool use, so behavioral contrasts must come mainly from reasoning trajectories.
  • OfficeQA [1]: grounded question answering over long, real-world documents. The deliverable is a precise answer rather than a file, and success depends on locating and reasoning over the right evidence.
MethodIMO-AnswerBenchOfficeQA
Base harness67.555.3
Harness²82.563.9

These results show that Harness² is effective beyond tool-use tasks, with gains of 15.0 percentage points on IMO-AnswerBench and 8.6 percentage points on OfficeQA. In particular, the substantial improvement on IMO-AnswerBench demonstrates that this framework can benefit reasoning-intensive tasks even when tool use is limited. Together, these findings support its broader applicability across tasks with different reasoning demands and output formats.


References

[1] Databricks. OfficeQA: A grounded reasoning benchmark over real-world documents. 2025.

[2] T. Luong et al. Towards robust mathematical reasoning. Google DeepMind, 2025.

Thank for the additional evidence. My concerns are addressed.

Dear Reviewer j5QF,

Thank you for your thoughtful feedback and for confirming that your concerns have been addressed. We would be grateful if you would consider updating your rating to reflect the additional evidence and our discussion.

Best regards, Authors

Reviewer_oepyOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether agents can improve both their task-specific harnesses and the process for building future harnesses. It proposes task-level recursion via scoped edits and composition, and harnessing-level recursion via a library of past harnesses and improvement experiences. The evaluation covers three benchmarks, multiple models, and different harnesses. While the experimental coverage is broad, I have concerns about the novelty of the task-level method and about whether the claimed harnessing-level improvement is distinct from standard experience reuse.

Critical strengths
  1. This paper covers experiments across three benchmarks, multiple models, and many harnesses. The evaluation is broad and includes repeated runs, ablations, and transfer experiments.
  2. The question of whether RSI can improve the method for future harness improvements is important.
Critical weaknesses

From my perspective, this paper has the following two critical weaknesses:

  1. Harnessing-level recursion: overstated claims and insufficient evidence.

The claim of harnessing-level recursion is overstated. The method mainly stores past harnesses and records reflections in EXPERIENCE.md for later use. And the improvement algorithm remains fixed. "Using past experience is useful" has already been well proven in agent research. This paper does not clearly show how this differs from standard experience reuse or why it should be considered a separate level of recursive self-improvement.

Also, the ablation does not isolate harnessing-level RSI. The full method and the “w/o library” variant both improve the harness for each task. The difference is the access to past harnesses and experience. This comparison also only shows that historical information helps, but does not prove that the system learns a better way to improve harnesses. To test the claimed Harnessing-level recursion, the authors should keep the same archived harnesses and retrieval access, removing only EXPERIENCE.md.

2. Task-level recursion: limited methodological novelty.

The paper discusses similar methods in related work and acknowledges that its main differences are two scoped editors and a composition step. These are incremental changes rather than a methodological advance. This is not a big problem if their main contribution is the harnessing-level recursion. But as explained in the first and second points, the only well-proved task-level recursion also can't provide sufficient novelty compared with existing methods.

Questions for authors
  1. How does harnessing-level recursion improve the editing strategy through experience reuse? Please show a concrete example and ablate only EXPERIENCE.md while retaining archived harnesses and retrieval. A comparison with best-of-N sampling using the base harness and the same selector under the same total inference budget would also strengthen the evidence.
  2. How does the system handle false positives in model-based selection and reflection? An incorrect output may be judged better, and a successful run may be wrongly attributed to a harness edit. If this leads to a misleading lesson being stored in the library, can later evidence trigger its revision? Did you observe such errors propagating to later tasks?

I would consider raising my score if the authors could adequately address these concerns.

8 comments
AuthorsOpenReview ↗

Response to Reviewer oepy (1/N)

We thank the reviewer for the thoughtful assessment. Our contribution connects two levels of recursion in a label-free task stream: task-level recursion improves the current harness and generates evidence about edits, while harnessing-level recursion retains that evidence to guide future improvements. We clarify this contribution and its empirical support below.

W1. Harnessing-level recursion versus standard experience reuse

Thank you for bringing this up. We would like to clarify our contribution from three perspectives: the setting, the experience being accumulated, and how that experience is organized and used.

First, harness evolution over an online, label-free task stream remains relatively underexplored. Much existing harness optimization relies on offline search over labeled tasks or optimizes a harness for the current test-time batch. Our focus is on learning across a stream: the system refines the current harness and retains evidence from that process to guide improvements on subsequent tasks. For example, MemoHarness [1] accumulates improvement experience through labeled offline search and uses a frozen experience bank at test time. Harness² instead accumulates this experience during the unlabeled test-time stream itself, without ground-truth labels or task rewards.

Second, we accumulate experience about improving harnesses, rather than primarily about solving tasks. Many agent-memory methods retain task-solving strategies: ReasoningBank [2], for example, distills experience from successful and failed executions and retrieves it to guide future task solving. Harness² records which harness edits were attempted, how they changed execution, and why they were adopted or rejected. This experience guides future harness proposals and composition decisions. The distinction is relevant empirically: ReasoningBank scores 73.4 versus 77.5 for Harness² on WorkBuddy-Bench, and 30.8 on JobBench, below the unchanged base harness at 31.5. These results show that task-solving memory does not consistently improve performance on the complex tasks in our evaluation.

Third, our library retains both improvement lessons and reusable harness components. Instead of relying solely on retrieved textual experience or a single accumulated instruction file, Harness² organizes adopted executable components alongside transferable lessons in EXPERIENCE.md, with an INDEX.md supporting on-demand access.

We have added the reviewer’s requested ablation: removing only EXPERIENCE.md while retaining the same archived harnesses and retrieval access. We also evaluate experience-only reuse. Results on WorkBuddy-Bench use Gemini 3.5 Flash as executor, Claude Opus 4.8 as editor, and the OpenCode harness at k=1:

ConditionAverage
Base harness72.4
No library74.4
Archived harnesses only75.5
EXPERIENCE.md only76.2
Full Harness² (k=1)77.5

Removing EXPERIENCE.md reduces performance from 77.5 to 75.5 despite preserving archived harnesses and retrieval access. Textual experience alone reaches 76.2, while combining lessons and executable components performs best. Thus, the gains extend beyond simply retrieving previously adopted harnesses.

Finally, our contribution also lies in connecting the two levels of recursion. Figure 7 holds the subsequent direct-editing procedure fixed, without fresh reference executions or probes: a library generated at k=1 restores base performance, while one generated at k=5 yields gains over the base of 5.1 points on Office. These results show that richer task-level exploration produces experience that supports better subsequent harness improvement. Harnessing-level recursion should therefore be assessed together with the online, label-free setting and its connection to task-level recursion, rather than reduced to the isolated claim that storing past experience is useful.

AuthorsOpenReview ↗

Response to Reviewer oepy (2/N)

Q1. Concrete examples and additional experiments

In addition to the ablations discussed above, we have added a concrete example in the Appendix F.2, showing how a lesson from an earlier harness revision guides improvement on a subsequent task. This example appears in the highlighted portion of the revised paper.

Figure 5 already compares Harness² with best-of-k on WorkBuddy-Bench using the same selector and final selection pool size. We now extend this comparison on JobBench to N=10 to examine whether additional sampling alone can recover the gains from harness refinement.

At k=1, Harness² directly submits the composed harness’s output without final selection, scoring 36.0. Best-of-N remains below this result at every tested N through 10, reaching at most 34.9. Harness²’s selected score increases to 39.0 at k=5.

Representative results are shown below. Oracle scores report the best candidate in the final selection pool.

Methodk / NSubmitted / selectedOracle
Harness²136.0—
Harness²337.450.1
Best-of-N334.739.2
Harness²539.056.3
Best-of-N534.946.4
Best-of-N1033.850.3

From N=5 to N=10, best-of-N’s oracle rises by 3.9 points, while its selected score is not increasing. Across N=3–10, selected performance remains between 33.2 and 34.9.

Harness refinement produces stronger candidate pools for the same imperfect selector. At k=3, Harness² achieves an oracle of 50.1, nearly matching best-of-N at N=10 (50.3), while obtaining a higher selected score. Our contribution improves candidate generation, while better label-free selection could translate more of this headroom into realized gains.

Q2. Reflection false positives and later revision

Reflection uses execution evidence to limit unsupported lessons. The Reflector is instructed to record a lesson only when behavioral contrasts support it, mark inseparable effects as confounded, and write NO_ENTRY when evidence is insufficient (§2.3, Prompt 6). Stored lessons are not inherited automatically: they inform new proposals, task-specific facts must be re-verified, and reused components undergo the current task’s Probe and Compose.

We additionally evaluated our pipeline on OfficeQA, using GLM 5.3 Flash as both executor and editor, and audited all 172 tasks, identifying one case of harmful transfer from a reused lesson—approximately 0.6% of audited tasks. In this case, a strict output-format skill from an earlier task led the editor to prioritize that constraint over other relevant signals.

To examine the effect of outdated lessons, we introduce label-free meta-reflection every 10 tasks. It reviews stored entries against subsequent reuse and reflection evidence, deciding whether to keep, merge, revise, retire, or delete them. On OfficeQA, this increases performance from 63.9 to 65.1:

MethodOfficeQA
Base harness55.3
Harness²63.9
Harness² + meta-reflection65.1

The audit identified one harmful-transfer case in this evaluated stream, and periodic revision of accumulated experience improved the aggregate score by 1.2 points. These observations do not establish that errors cannot accumulate over longer streams.

References

[1] Yue Huang, Wenjie Wang, Han Bao, Yuchen Ma, Xiaonan Luo, Yi Nian, Haomin Zhuang, Zheyuan Liu, Yue Zhao, and Xiangliang Zhang. MemoHarness: Agent Harnesses That Learn from Experience. arXiv:2607.14159, 2026

[2] Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. arXiv:2509.25140, 2025.

[3] Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052, 2026.

Reviewer_oepyOpenReview ↗

Response to the Authors

I thank the authors for the additional experiments, but my main concern remains unaddressed.

Neither level is recursive. At the task level, each task restarts from H0, and parallel recursion performs a single population update, so the task level is a single round of per-task test-time harness search.

At the harnessing level, only the library changes, while the improvement algorithm remains unchanged. The method runs multiple harness variants and combines their useful edits, but no results show gains that increase along the task stream.

Also, the evidence is weak. With k = 1, each task is executed 4 times, yet best-of-N (N = 3–10) achieves 33.2–34.9 on JobBench, compared to 36.0 for k = 1. The new ablation results are also within one SD.

I therefore maintain my score.

Reviewer_423DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on the open-world setting: (1) the tasks are typically unseen during the harness design phase and novel at test time; (2) the recursive self-improvement process remains task-specific and fails to generalize. Also, this paper focuses on the self-improvement of the model harness.

The proposed method is Harness$^2$ with task-level recursion (refines current task's harness) and harnessing-level recursion (update a continual improvement harness).

The evaluation has been conducted on three professional benchmarks (WorkBuddy-Bench, JobBench, Legal benchmark LAB) and four model-harness configurations.

My initial rating is likely to be a 2.5, to be honest, given the competitiveness of this venue. I believe I've learned something useful, but there are also things I feel unclear about even after reading for over an hour.

Critical strengths

I appreciate the open-world setting of this paper. This is what I personally feel is missing in current RSI research. The ultimate goal is never to fit the distribution of benchmarks, but to generalize to unseen tasks that the system may encounter in the future.

The presentation is clear and extensive experiments have been conducted with the proposed harness2 being the SOTA solution.

Critical weaknesses

W1. It is not very clear what the novelty in task-level recursion is, as it simply harness self-improvement, and a lot of works have studied. The propose - execute - probe - compose pipeline (or similar self-improvement pipeline) is very common in literature, and what's the conceptual difference of this work compared to works discussed in related works? And the motivation of having two forms of recursion is not clear to me as well -- why specifically two forms instead of only using parallel form, or using many mixed forms and picking the best?

W2. The improvement harness R_t is very unclear in its formulation. How do you represent that? is it simply a codebase or something? And what's the mechanism of this improvement (simply letting the agent rewrite code)? This "improvement harness" for me feels like a mystery neural network whose training is unknown but magically works - hope this analogy can help you understand my concern here.

W3. How does your harness$^2$ compare with simply giving the agent how it did previous improvement in text form, rather than "summarizing" it as a harness? There could be many variants of harness$^2$ due to multiple components in design, and the ablation in Table 3 isn't enough. For example, you could also ablate on your propose - probe - compose pipeline, maybe changing it a little bit to create a variant. In short, it's not proven that the current design is already good enough and replacing / changing any component to other design choice would't grant a better performance.

W4. Though I've been excited to see the open-world setting, the experiments are done mostly per-benchmark. Therefore harness$^2$ might still be learning the benchmark distribution by training on that and learn the construction logic of benchmark. It shows only little evidence of cross-benchmark generalization, which I appreciate but don't feel enough. Can we generalize this study beyond JB -> WB, and do some measurement of how similar JB and WB tasks are (if they're too similar, it's still not "open-world")?

Questions for authors

Q1. Figure 1 has strong AI styles embedded. Q2. Since there is harness$^2$, can there be harness$^3$ (to create harness$^2$ in the open world) and so on? After all, current harness$^2$ is to build a harness for generalizing harness, and might still overfit on the generalization between tasks in the training set. Q3. I'm very curious what your "improved" Codex is like, and is there any takeaway for the "future directions" of Codex? Did you get any new feature that seems to be transferable across new tasks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 423D (1/N)

We thank the reviewer for the careful reading and for recognizing the importance of the open-world setting and the strength of the empirical results. We address the concerns about novelty, the improvement harness, and generalization below.

W1. Novelty and motivation: distinguishing Propose–Probe–Compose and motivating parallel and sequential recursion

Thank you for raising these questions. We would like to clarify the setting motivating our design, the contribution of task-level refinement, and the role of parallel and sequential recursion.

First, our design addresses harness evolution over an online, label-free task stream, a practical setting for real-world deployment. Existing harness optimization methods [1] commonly operate offline on a predefined task set or rely on ground-truth labels or task rewards to evaluate proposed changes. In deployment, however, agents encounter tasks as they arrive, and ground-truth answers or reliable task rewards are often unavailable. Our setting therefore requires the system to refine its harness for the current task and accumulate useful improvement experience for subsequent tasks using the available execution evidence. This creates two connected challenges: determining which edits are supported without ground-truth feedback and retaining useful evidence to guide future revisions.

Second, Propose–Probe–Compose addresses the challenge of obtaining useful evidence about harness edits without ground-truth feedback. Unlike Meta-Harness [1], which uses task scores to assess changes, we rely on execution evidence alone. While methods such as ReasoningBank [2] and RHI [3] use contrastive or pairwise feedback, our emphasis is on probing complementary, deliberately scoped harness variants against a common reference and composing supported edits. Our Compose step consolidates useful changes across explored candidates into a single harness at test time, without ground-truth labels. The resulting harnesses and refinement lessons also guide subsequent tasks. Table 3 supports these choices: removing proposal scoping reduces performance by 2.9/3.7 points on WorkBuddy-Bench/JobBench, while replacing composition with candidate selection reduces it by 1.6/1.9 points.

Third, our task-level contribution extends beyond a single contrastive step to a systematic study of recursive harness refinement. Whereas Harness Scaling [4] updates the harness sequentially, we further study both parallel and sequential recursion upon our pipeline, examining how breadth and depth affect harness improvement. Parallel recursion explores complementary candidates, while sequential recursion progressively refines earlier edits; the two coincide at (k=1). In our evaluation, Harness Scaling does not improve consistently as compute increases, whereas Propose–Probe–Compose benefits from additional compute over the evaluated budgets. Compose provides a mechanism for turning additional exploration into a stronger harness by consolidating complementary, evidence-supported edits. These two forms offer practical ways to allocate compute; mixed schedules remain compatible extensions.

W2. Representation and update of the improvement harness

Thank you for highlighting this ambiguity. A harness is the executable scaffolding surrounding an LLM, including instructions, tools, memory, skills and other components, that governs how it performs a task. The improvement harness (R_t) is the evolving part of the improver agent’s harness that supports refinement of the execution harness, as illustrated in Appendix E.2.

Concretely, each domain’s library contains previously adopted executable harness components and an EXPERIENCE.md recording attempted edits, their observed behavioral effects, and supported lessons. An INDEX.md makes entries accessible on demand. Editor and composer sessions receive access to this library alongside the current task, harness, and execution traces. They can reuse or adapt stored components and consult earlier evidence when deciding which changes to propose, retain, or reject. After each task, a reflector examines the improvement record and adds the adopted harness and evidence-supported experience.

Thus, updating (R_t) changes the reusable components and accumulated experience available to guide future harness revisions. We have added a concrete example to the revised paper in Appendix E.2, showing the structure of the R_t.

AuthorsOpenReview ↗

Response to Reviewer 423D (2/N)

W3. Comparison with text-form experience and alternative designs

In the paper, we have already contained a related baseline, ReasoningBank [2]: it stores distilled strategies as text and retrieves them into the agent’s context. Tables 1–2 report this comparison across the evaluated configurations. For example, with Gemini 3.5 Flash and OpenCode, ReasoningBank scores 73.4 versus 77.5 for Harness² on WorkBuddy-Bench, and 30.8 versus 36.0 on JobBench.

Our existing ablations already examine several alternatives raised by the reviewer. Table 3 replaces scoped editors with generic editors and composition with candidate selection; both replacements reduce performance. Appendix F.1, Table 8 additionally examines the retained components: removing skills, scripts, or prompt-related components reduces average WorkBuddy-Bench performance by 5.3, 3.7, and 1.5 points, respectively.

To address the text-form comparison more directly, we have added an ablation retaining only EXPERIENCE.md, which records previous harness edits, their observed effects, and the resulting lessons, while removing archived executable harnesses. We also evaluate the complementary condition retaining archived harnesses without textual experience. Results on WorkBuddy-Bench use Gemini 3.5 Flash as executor, Claude Opus 4.8 as editor, and the OpenCode harness at k=1:

ConditionAverage
Base harness72.4
No library74.4
Archived harnesses only75.5
EXPERIENCE.md only76.2
Full Harness² (k=1)77.5

Textual improvement experience alone improves performance from 74.4 to 76.2, while combining it with archived harnesses raises performance further to 77.5. Conversely, removing textual experience from the full library costs 2.0 points. Harness² therefore already incorporates textual experience reuse, and these results show complementary benefits from textual lessons and reusable harness components.

W4. Open-world scope and cross-benchmark generalization

Thank you for raising this concern. Our protocol uses no separate labeled training set for harness optimization: tasks arrive sequentially, and improvement uses execution evidence without ground-truth labels or evaluation rewards. We agree, however, that within-benchmark evaluation alone does not establish cross-benchmark generalization. We clarify the transfer mechanism and provide complementary evidence below.

We added Tables 5–6 in Appendix C to document the task differences. Table 5 compares inputs, outputs, task requirements, environments, languages, and evaluation procedures. JobBench involves professional analysis, whereas WorkBuddy spans repository changes, interactive pages, and structured file transformations. JobBench itself covers 35 professions across seven occupational groups. These comparisons establish concrete differences, although they are not a quantitative measure of distributional similarity.

To address this concern, we add an experiment combining OfficeQA and IMO-AnswerBench into a single shuffled task stream. These benchmarks involve substantially different task types: document-grounded question answering and mathematical reasoning. Both use GLM 5.3 Flash as executor and editor. We compare this mixed-stream setting with processing each benchmark separately:

MethodOfficeQAIMO-AnswerBench
Base harness55.367.5
Harness², separate streams63.982.5
Harness², shuffled mixed stream64.581.3

Performance in the mixed stream remains close to that in separate streams: OfficeQA increases by 0.6 points, while IMO-AnswerBench decreases by 1.2 points. Both retain substantial gains over the base harness. These results support the applicability of Harness² to a heterogeneous stream in which successive tasks can come from substantially different domains, rather than requiring tasks to remain within a single benchmark distribution.

We have also added concrete task examples and a domain-comparison table for JobBench and WorkBuddy-Bench to the updated paper. These additions make their differences explicit and help contextualize the existing cross-benchmark transfer results in Figure 7. Together, the mixed-stream experiment and benchmark comparison strengthen the evidence for adaptation beyond a single benchmark-specific stream.

Q1. Figure 1

Thank you for the feedback. We will simplify Figure 1 by reducing decorative elements and emphasizing the runtime roles of the harness components.

AuthorsOpenReview ↗

Response to Reviewer 423D (3/N)

Q2. Higher levels of recursion and possible overfitting

Higher levels are possible in principle. In Harness², task-level recursion revises the execution harness, while harnessing-level recursion accumulates experience that guides future revisions. The refinement instructions and Propose–Probe–Compose procedure remain fixed. A further level could revise that procedure itself, including proposal strategies, probe allocation, or reflection rules.

Regarding overfitting, in our framework, each new task starts from the base harness, rather than directly inheriting the previous task’s adapted harness. Prior experience informs fresh, task-specific revisions, which are probed against the current task’s execution evidence. This reset limits the accumulation of task-specific edits across the stream.

Q3. What changes in the improved Codex?

The improved Codex retains the same underlying model while adapting its external harness. In our Codex configuration, always-on instructions reside in AGENTS.md, alongside on-demand skills, helper scripts, and middleware components (Appendix E.1). These components change both the guidance available to the model and how execution is controlled.

Hooks provide a concrete example. Appendix E.3, Table 7 describes middleware operating around tool calls and turn boundaries. Appendix I, Prompt 1 gives the example of a defensive hook preventing the agent from exiting before a required deliverable exists, when repeated execution evidence shows that a textual instruction is being ignored. This makes the completion condition a runtime check rather than relying on the model to follow a reminder.

The takeaway for future Codex-like systems is to place recurring, mechanically checkable constraints in runtime hooks, while using skills for workflows and scripts for deterministic operations. The deliverable-existence hook is a design example specified in our editor prompt; it could apply across artifact-producing tasks because it checks a completion condition rather than a task-specific answer. The improvement library can retain such components and their supporting evidence for reuse and adaptation.

References

[1] Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn.Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052, 2026.

[2] Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, Vishy Tirumalashetty, George Lee, Mahsan Rofouei, Hangfei Lin, Jiawei Han, Chen-Yu Lee, and Tomas Pfister. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. arXiv:2509.25140, 2025. Accepted to ICLR 2026.

[3] Hyunin Lee, Jinglue Xu, Jeffrey Seely, Donghyun Lee, Matei Zaharia, and Yujin Tang.Recursive Harness Self-Improvement. arXiv:2607.15524, 2026.

[4] Yike Wang, Huaisheng Zhu, Zhengyu Hu, Yige Yuan, Zhengyu Chen, Shakti Senthil, Hannaneh Hajishirzi, Yulia Tsvetkov, Noah A. Smith, Pradeep Dasigi, and Teng Xiao.Rethinking the Evaluation of Harness Evolution for Agents. arXiv:2607.12227, 2026.

Dear Reviewer 423D,

Thank you again for your thoughtful feedback. We have provided clarifications and additional experiments addressing your initial concerns. We would appreciate your thoughts on our responses.

Best regards, Authors

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Updates and Revisions to Our Paper

We thank the reviewers for recognizing the importance of online harness improvement, the clarity of our presentation, and the breadth of our experiments and ablations.

In response to their valuable feedback, we have made the following additions to the paper and rebuttal:

  • Added library-component ablations separating textual experience from archived executable harnesses. Experience-only and archive-only variants score 76.2 and 75.5 on WorkBuddy-Bench, compared with 77.5 for the full method.
  • Added OfficeQA and IMO-AnswerBench evaluations, including a shuffled mixed stream, to examine generalization across document-grounded question answering and mathematical reasoning.
  • Added three independently shuffled OfficeQA runs. Full-stream accuracy is 52.18% ± 2.48 for the base harness and 63.66% ± 2.25 for Harness². Quarter-level gains suggest stronger later-stream performance.
  • Extended Best-of-N comparisons and token accounting. On WorkBuddy-Bench, Harness² achieves 77.5 ± 2.5 at 2.32M tokens per task, compared with 74.3 ± 2.2 at 3.16M for Best-of-10. Our total includes all proposal, probing, composition, and final execution tokens.
  • Added a library reliability audit and periodic meta-reflection experiment. The audit identified one harmful-transfer case among OfficeQA tasks; periodic library maintenance improves accuracy from 63.9 to 65.1.
  • Added concrete library and cross-task reuse examples (Appendices E.2 and F.2) and benchmark comparisons clarifying task differences (Appendix C, Tables 5–6).

We also clarified that our definition of recursion. We sincerely thank the reviewers for their time and constructive feedback, and hope these clarifications and additional evaluations help inform their assessment.

AuthorsOpenReview ↗

General Response (1/2): Positioning of Our Contribution

We thank all reviewers for the engaged discussion. Because much of it has turned on how our contribution should be positioned, we state our position in one place.

1. The setting: harness improvement over an online, label-free task stream

In our setting, tasks arrive one at a time, are novel at test time, and come with no ground-truth verifier or reward. The system must (i) refine the harness for the current task from execution evidence alone, and (ii) carry forward what it learned about improving harnesses to later tasks. The methods Reviewer oepy compares us to solve a different problem:

MethodWhen the harness is improvedSignal for accepting editsUpdated during the test stream
Meta-Harness [1]Offline search, before evaluationTask reward on a labeled search setNothing
MemoHarness [2]Offline search, then per-case adaptationReference outputs and verifier-grounded rewards during searchNothing (experience bank is frozen)
RHO [3]Retrospective batch pass over past tasksLabel-free self-preferenceNothing (one global harness)
Harness²Online, per incoming taskLabel-free execution contrasts (Probe)Improvement library RtR_t, after every task

Meta-Harness and MemoHarness obtain their improvement signal from labeled search cases before deployment. MemoHarness adapts per test case without labels, but from an experience bank that was built with labels and is frozen afterwards; its paper states that no learning or distillation is triggered during evaluation. In an open world this assumption does not hold: a labeled search set cannot exist for tasks that have not been seen yet. RHO removes the labels but works retrospectively, re-solving a coreset of past tasks to produce a single global harness. We are not aware of prior work that combines per-task harness improvement with cross-task accumulation of improvement experience inside an unlabeled stream.

2. On "recursion": we agree with Reviewer oepy's distinction, and clarify the sense we use

We agree on the facts. The Propose–Probe–Compose procedure is fixed, but editors, composer and reflector are not. In Duan et al. [4], L4 concerns how the operating agent persistently adapts while the improvement process stays fixed, and L5 concerns persistent revision of a mechanism that governs later improvement, with the improver listed as one such mechanism. In Harness², what persists is RtR_t, the harness of the improver, and it is consulted when later edits are proposed and composed. Harness² is therefore not a pure L4 system. We do not claim full L5 either: the improver's procedure is fixed and only its state evolves, and the archived harness components in RtR_t are also reused on the execution side. We will state this placement explicitly in §1 and §2.1.

Where we differ is on whether "recursive" must be reserved for L5. The community uses the term across a range:

  • Strict. Liu et al. [5] reserve RSI for systems whose improvement mechanism is itself modifiable. This matches the reviewer's reading.
  • Broad. Dai et al. [6] define RSI as systems that "modify themselves and carry those changes into the generation of subsequent improvements," and survey Prompt, Memory, Skill and Harness updates under fixed weights as its mechanisms. Recuris [7] keeps a fixed Meta-Agent, updates Skill Memory, and calls this a "bounded recursive memory-evolution loop." RHI [8] refines a harness with a fixed procedure over its own revision history. Metaⁿ [9] fixes the meta-operation and recurses over its products.
  • Chen et al. [10] note that these terms are "used loosely and inconsistently across the literature."

Our definition in §2.1 follows the broad sense, and Harness² meets its operative criterion. RtR_t belongs to the improver (editors and composers), not to the task executor. It is written from the outcomes of improvement cycles (which edits were tried, how they changed execution, why they were adopted or rejected) and read when the next edits are generated. This differs from task-solving memory such as ReasoningBank.

The revision is therefore a clarification of scope, not a withdrawal of the claim. We will state that the improvement algorithm is fixed, and distinguish the parallel variant's single round of proposal and composition from the sequential variant's chained refinement.

AuthorsOpenReview ↗

General Response (2/2): Summary of Evidence and Discussion

Reviewer j5QF (library stability over time; qualitatively different tasks) has confirmed that the concerns are addressed.

  • An audit of all 172 OfficeQA tasks found one harmful-transfer case. Label-free meta-reflection every 10 tasks raises accuracy from 63.9 to 65.1.
  • Harness² improves IMO-AnswerBench from 67.5 to 82.5 and OfficeQA from 55.3 to 63.9.

Reviewer 423D (novelty of task-level refinement; what RtR_t is; text-only experience; cross-benchmark generalization).

  • RtR_t is a per-domain library of adopted executable harness components, an EXPERIENCE.md of attempted edits and supported lessons, and an INDEX.md (Appendix E.2).
  • Library ablation on WorkBuddy-Bench: no library 74.4, archived harnesses only 75.5, EXPERIENCE.md only 76.2, full 77.5.
  • A shuffled OfficeQA + IMO-AnswerBench stream stays close to separate streams (64.5 / 81.3 vs. 63.9 / 82.5). Tables 5–6 document the task differences between benchmarks.

Reviewer oepy (recursion framing, clarified in Part 1; strength of evidence).

  • Requested ablation. Removing only EXPERIENCE.md reduces 77.5 to 75.5.
  • Sampling vs. refinement. On JobBench, Best-of-N stays at or below 34.9 for N up to 10, against 36.0 for Harness² at k=1k=1 with no final selector. On WorkBuddy-Bench, Best-of-10 reaches 74.3 ± 2.2 at 3.16M tokens per task, against 77.5 ± 2.5 at 2.32M for Harness².
  • Gains along the stream. Over four independently shuffled OfficeQA orders, the relative gain over the base harness rises from 16.0% ± 6.6 in the first quarter to 30.0% ± 9.0 in the last. Full-stream accuracy is 52.18 ± 2.48 (base) vs. 63.66 ± 2.25 (Harness²).

We thank the reviewers again for their time.

References

[1] Lee et al. Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052, 2026.

[2] Huang et al. MemoHarness: Agent Harnesses That Learn from Experience. arXiv:2607.14159, 2026.

[3] Pan et al. Evolving Agents in the Dark: Retrospective Harness Optimization via Self-Preference. arXiv:2606.05922, 2026.

[4] Duan et al. The Last AI Built by Humans: Toward Genuine Recursive Self-Improvement. arXiv:2609.11873, 2026.

[5] Liu et al. The Path to Recursive Self-Improving Agents: Foundation, Framework, and Future Directions. Preprints 202608.0051, 2026.

[6] Dai et al. When, Where, and How Experience Becomes Structure: Mechanisms for Recursive Self-Improvement in LLM Agents. Preprints 202611.0261, 2026.

[7] Yu et al. Recursive Experiential–Working Memory Evolution for Long-Horizon Agent Harnesses. arXiv:2608.24876, 2026.

[8] Lee et al. Recursive Harness Self-Improvement. arXiv:2607.15524, 2026.

[9] Kim et al. Metaⁿ: Recursive Self-Improvement through Emergent Depth. arXiv:2608.24735, 2026.

[10] Chen et al. Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops. arXiv:2607.07663, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.