acceptodds
Under review as a conference paper at ICLR 2027

RSI-Harness: Optimizing the Harness Optimizer via Recursive Self-Improvement

Abstract

The performance of an LLM agent depends heavily on its harness, which manages the model's context, tool use and control flow. Recent approaches automate harness design, yet the improver that edits the harness is written by hand and never changes, so it cannot learn from the outcomes of its past edits. Self-referential systems relax this restriction by letting the improver update itself, but it remains unclear whether they learn a better improver or merely find a better task harness along one search trajectory. We address this question by making the improver itself an explicit optimization target, so that it learns from its own search experience and becomes a better optimizer. We realize this idea in RSI-Harness, a dual-loop recursive self-improvement framework in which an inner loop uses the current improver to refine the task harness, while an outer loop revises the improver from the history of successful and failed attempts. Each revision may start from any archived improver, and an acceptance gate adopts it only if it produces a better task harness than the current improver under matched tasks and budget. Across five benchmarks and three frozen backbones, RSi-Harness achieves the best average score on every backbone, improving the initial harness by 7.7 points on average with 69% fewer optimization tokens on average than competing baselines. More importantly, the improver itself recursively gets better, with its gain on the same starting harness growing from 1.3 to 6.2 points. The final improver also transfers to unseen models, benchmarks and domains, gaining at most 4.5 as much as the initial improver, while the optimized task harness transfers poorly.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agent, recursive self improvement, agent harness, meta optimization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_wYvQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is about automated harness design. In existing methods the procedure that edits the harness is written by hand once and never changes, so it cannot use what it learns from its own rejected and accepted edits. RSI-Harness makes that improver a second optimization target. An inner loop applies the current improver to the task harness, gating each edit on a held-out validation split. Every third step an outer loop reads the stored history of trials, picks any earlier improver as the parent, and rewrites the improver's own code; the rewrite takes over only if it yields a better harness from the same starting point and the same budget. Experiments cover five agent benchmarks on three frozen backbones against fixed and self-referential optimizers. The paper also freezes improvers taken from different points of a run and restarts them from the same starting harness, where the gain rises from 1.3 to 6.2 points, and shows that the learned improver transfers to unseen models, benchmarks and domains. It claims the framing, the gated self-revision, and the gains at lower token cost as its contributions.

Critical strengths
  1. The paper makes its own central question identifiable, which most of this literature does not.

The question — does recursion produce a better improver, or merely a better harness along one trajectory? — is stated explicitly (Section 1), and the paper then designs two controls that can actually distinguish the answers. The load-bearing one is I_offline (Table 8): it is derived by running I0 for all 50 steps to collect the full history and then applying a single outer-loop revision, so it sees exactly as much evidence as RSI-Harness but never takes over the search. On the fresh split it reaches +3.9 while I_final reaches +8.1. The second is the strong hand-written fixed improver I_sf (Table 7), which contains by hand the procedures the outer loop is observed to learn and still lands at 36.1 against 39.9. Either control alone would be suggestive; together they rule out the two deflationary explanations (more history; rediscovering human-writable heuristics) that a reviewer would otherwise be entitled to assume. This is why I do not regard the "improver itself improved" result as an artifact, and it is the part of the paper I would expect to survive.

  1. The empirical section is unusually resistant to the standard objections.

The validation audit (Table 9) scores every accepted edit additionally on audit tasks never shown to the algorithm, and reports a 14% false-accept rate for improver revisions and 18% for task-harness edits instead of suppressing the number. The fresh-split evaluation empties the history and moves to disjoint tasks; Table 3 starts the method from four harnesses spanning 12.5 to 39.9 initial score; Appendix F reports a hierarchical bootstrap and, in the text, concedes that the interval against the strongest baseline crosses zero. Very few submissions in this genre do any of these. It is the reason my assessment is a weak reject on contribution rather than a reject on rigor.

Critical weaknesses
  1. The contribution is a recombination of existing components, and the paper does not establish a delta over the closest contemporaneous work.

Each element has direct precedent: archive-based self-modification with empirical acceptance is the Darwin Gödel Machine / STOP / SICA line; regression-gated candidate acceptance is the standard move in harness optimization; the two-loop task-vs-improver split is RHI's formulation; mutating the mutation procedure is Promptbreeder. What is new is the specific pairing — gating the improver rather than the artifact, with the same regression test used at both levels. That is a real but small step, and the paper does not argue that it is a conceptual one.

The positioning is thin exactly where it needs to be specific. RHI (Lee et al., 2026a) states the same problem and is dispatched in a single subordinate clause ("its refinement procedure is fixed") with no experiment. AHE (Lin et al., 2026a) is a directly comparable harness-evolution method that appears in the paper exactly once, in Appendix I, as the provenance of a seed harness — it is never discussed as related work and never run as a baseline, even though its own seed harness is one of the paper's four starting points. And Escher-Loop (ICML 2026), which proposes closed-loop self-referential mutual evolution, is not cited at all. I want to be precise that the reference list is broad — this is not a case of missing citations in general. It is a case of the two or three systems that a reader of this literature would immediately name being handled without comparison.

The consequence is visible in the appendices. By the authors' own taxonomy, 85% (158/185) of accepted improver edits are C1–C4 search-policy adjustments: cluster failures by error signature, drop near-duplicate candidates, reallocate the gate budget across clusters, read previously rejected proposals. No accepted edit introduces a new diagnostic procedure or a new synthesis algorithm; the knockout (Figure 8) shows the four categories sum to 9.7 points but are strongly interactive, with evidence selection alone accounting for −3.9 and prompt-wording-only recovering 0.9 of the 7.1-point gap. So the demonstrated capability is "an LLM automatically tunes a search heuristic to the benchmark distribution it is run on." That is a legitimate engineering result. It is not the recursive capability the title and abstract advertise, and absent a mechanism contribution the paper's case for acceptance has to rest entirely on its empirical margin.

  1. The two headline numbers do not carry the weight placed on them.

Statistics. Table 11 gives the 95% intervals against the strongest baseline as +4.1 [−0.9, +9.1], +4.2 [−1.1, +9.6] and +3.2 [−1.7, +8.1] — all three cross zero. Appendix F says so explicitly and honestly ("we therefore claim the best average score rather than a significant margin"). But the abstract states "achieves the best average score on every backbone, improving the initial harness by 7.7 points on average with 69% fewer optimization tokens" and the conclusion repeats it without any qualifier. A point estimate that wins on three backbones is a weaker claim than "best average on every backbone" as ordinarily read, and the caveat is available to the authors and withheld from the reader who stops at page 1.

The 69%. Figure 2b gives 0.21B tokens for RSI-Harness against 0.31B (Self-Harness), 0.45B (GEPA) and 1.24B (Meta-Harness). 69% is 1 − 0.21/mean(0.31, 0.45, 1.24) — the denominator is the arithmetic mean of the baselines. Against the strongest and cheapest baseline the reduction is 32%; against Meta-Harness it is 83%; against GEPA 53%. "69% fewer than competing baselines" is therefore true of no single baseline, and the spread across baselines is larger than the headline.

Cost accounting. Appendix C reports that outer-loop comparisons consume 18.6% of rollouts at M = 3 and that this is "charged to the same rollout budget." Appendix J then states that the rollouts spent on outer-loop comparison "count toward this budget, so RSI-Harness uses no more optimization budget than the other methods," and two paragraphs later that "because rollouts are matched, the task side is the same for every method." These cannot both hold: if gate rollouts are inside a matched budget, RSI-Harness performs fewer inner-loop improvement steps than its baselines (which, if true, strengthens the result and should be stated); if they are not, the reported total is an undercount (which weakens it). Nowhere is it stated whether the 0.21B in Figure 2b includes gate rollouts and the improver's own tokens. Since average score and token efficiency are the only quantitative claims the paper makes beyond its mechanism, this must be settled rather than left to the reader.

Questions for authors

1.Cost accounting. Does the 0.21B in Figure 2b include (a) outer-loop gate rollouts and (b) the improver's own token consumption? Please give a per-method table of inner-loop steps, gate rollouts, task tokens, optimizer tokens and wall-clock, and state explicitly whether gate rollouts are inside the matched budget. The answer determines whether the efficiency claim or the effectiveness claim is the true one.

2.Denominator of the token claim. Why is 69% computed against the mean of the baselines rather than against Self-Harness (32%) or Meta-Harness (83%)?

3.Edit substrate vs meta-optimization. RSI-Harness edits code; GEPA edits prompts and SkillOpt edits a skill document; the paper itself says structural edits are "a likely reason" for the margin over text-artifact optimizers. On SpreadsheetBench with Qwen3.5-122B, Self-Harness scores 67.7 against RSI-Harness's 65.4. How much of the average advantage over the text-editing baselines is attributable to the substrate rather than to recursion, and would moving I_sf into Table 1 as a code-editing fixed-improver baseline change the picture?

4.Positioning. What does RSI-Harness do that RHI's recursive refinement of a harness specification does not, and why is AHE not compared as a harness-evolution baseline rather than used only as a seed harness? Please also relate the contribution to Escher-Loop's closed-loop self-referential mutual evolution.

5.What the outer loop actually produces. Is any of the 185 accepted improver edits a new diagnostic or synthesis algorithm, as opposed to an instantiation of a known search heuristic? If all are policy adjustments, what does the result establish beyond "an LLM can tune a search policy on a benchmark distribution"?

6.Statistical power. Does Table 1 use three runs or eight seeds — the caption says three, Table 10 reports eight, and the means are identical to one decimal. With 18 and 19 test tasks on Terminal-Bench and τ³-bank, what is the per-benchmark 95% interval for the reported deltas? A single task moves those benchmarks by more than five points.

5 comments
AuthorsOpenReview ↗

Response to Reviewer wYvQ Part A

Cost accounting and the token-saving figure (Q1, Q2, W2.2, W2.3)

We thank the reviewer for breaking this question down so precisely. Gate rollouts are inside the matched budget, and the 0.21B includes both the gate rollouts and the improver's own tokens. The reviewer's first case is fewer inner-loop steps; ours keeps 50 steps with fewer gate rollouts per step for task-harness edits. The implication is the same: Table 1 is obtained with less inner-loop spending, and the token total is not underestimated.

1. The budget

All methods get the same number of task-agent rollouts, and the outer loop is paid from this budget (Appendix J): RSI-Harness spends part of its rollouts on the outer loop and the rest on task-harness edits, while the baselines spend all of theirs on task-harness edits. On SpreadsheetBench it takes 18.6% of the rollouts and raises the task score by 15.3 points over no outer loop (Table 6); the shares on the other benchmarks will be added to Appendix C.

The two quoted sentences refer to the same rollout total. "The task side is the same" means the same number of rollouts; task-side tokens differ because trajectories differ in length, and we will reword the sentence accordingly.

2. What the 0.21B contains

Both (a) and (b) are included: gate rollouts are task-agent rollouts, improver calls are optimizer calls, and the paper's token count covers both (Appendix J).

3. Per-method cost

MethodInner-loop stepsTask-agent rolloutsof which outer gateTask tokens (B)Optimizer tokens (B)Total tokens (B)Wall-clock (h)
GEPA504,03000.330.120.4517.8
SkillOpt504,03000.310.050.3615.9
Meta-Harness504,03000.340.901.2427.9
STOP504,0304000.270.020.2913.6
Hyperagents504,0308000.330.150.4818.1
Self-Harness504,03000.280.030.3114.2
RSI-Harness504,0307360.200.010.2110.0

Table A1: Token cost per method.

Findings. RSI-Harness has the lowest total of the seven methods, on both the task side and the optimizer side. Since rollouts are matched, task-side differences come only from trajectory length; optimizer-side differences come from how much evidence each method reads (e.g., Meta-Harness reads all earlier candidates at every step, Appendix I).

4. The denominator of the token claim (69%)

69% = 1 − 0.21 / mean(0.31, 0.45, 1.24), i.e., against the mean of three baselines. We will instead report 32%, 53% and 83% fewer tokens than Self-Harness, GEPA and Meta-Harness, led by the 32%.

Key findings.

  • The effectiveness claim holds under a stricter condition: with the same steps and rollouts, RSI-Harness spends part of them on the outer gate and still has the best average on every backbone.
  • The efficiency claim holds against each baseline: 32%, 53% and 83% fewer tokens, with all outer-loop cost included.

Revision. Add Table A1 to Appendix J and change "the task side is the same" to "the number of rollouts is the same"; state the token composition in the Figure 2b caption; report the outer-loop share per benchmark in Appendix C; replace the 69% in the abstract, §1 and §6 with the per-baseline numbers.

AuthorsOpenReview ↗

Response to Reviewer wYvQ Part B

Edit substrate versus recursion (Q3)

We thank the reviewer for this question. The edit substrate explains only 1.2–1.6 points of the lead over text-based optimizers on Qwen3.5-122B; the remaining 3.8–3.9 points are a lead on the same substrate. There, RSI-Harness leads text-based baselines by 5.1–5.4 points and code-based ones by 3.8–3.9. On the other two backbones, the lead over code-based baselines is no smaller than over text-based ones.

1. Table 1 grouped by substrate

Backbonevs. GEPA (prompt)vs. SkillOpt (skill document)vs. Meta-Harness (code)vs. Self-Harness (harness)vs. I_sf (code, fixed)
gpt-5.5+4.7+4.1+6.6+5.4+5.7
Qwen3.5-122B+5.1+5.4+3.9+3.2+3.8
Gemma-4-31B+3.6+4.7+4.7+3.3+3.7

Table B1: Average-score lead of RSI-Harness over each baseline, grouped by edit substrate (points).

Findings. Only on Qwen3.5-122B is the lead over text-based baselines clearly larger than over code-based ones; on gpt-5.5 it is the reverse, and on Gemma-4-31B the two are similar.

2. The code substrate is used only after recursion

I_0 can already edit code but rarely does: 58% of its proposals only reword prompts or context, against 24% for I_final, whose control-flow and verification edits rise from 18% to 45% (Figure 7). The outer loop teaches what to edit.

3. The SpreadsheetBench cell

The 67.7 on Qwen3.5-122B is SkillOpt, not Self-Harness (61.2). SkillOpt's text substrate suits SpreadsheetBench (+19.4) but not the other four benchmarks (between −1.9 and +3.1). RSI-Harness trails a fixed optimizer in only 3 of 15 cells: this one (by 2.3), gpt-5.5 SpreadsheetBench (SkillOpt 67.4 vs. 66.8) and Gemma-4-31B SWE-bench Pro (GEPA 29.9 vs. 29.3). We will list them in §4.2.

4. Adding I_sf to Table 1

The Qwen3.5-122B row is Table 7; the other two rows are new runs with three seeds:

BackboneSWE-bench ProTerminal-Benchτ³-bankSpreadsheetGAIA-2AverageRSI-HarnessΔ
gpt-5.559.378.939.656.149.356.662.3+5.7
Qwen3.5-122B37.945.618.859.019.336.139.9+3.8
Gemma-4-31B26.937.013.953.332.932.836.5+3.7

Table B2: I_sf on three backbones.

Findings. I_sf stays below RSI-Harness on all three backbones and does not change the strongest baseline on any of them.

5. Fixing recursion and restricting the substrate

With recursion fixed, we restrict RSI-Harness to prompt and context edits (setting of Table 2; the other rows are from Tables 1 and 2):

ConfigurationTask scorevs. T0
T048.3—
GEPA (prompt substrate, fixed optimizer)58.1+9.8
RSI-Harness, prompt / context edits only60.6+12.3
RSI-Harness (all edits)65.4+17.1

Table B3: Restricting the edit substrate (SpreadsheetBench, Qwen3.5-122B).

Key findings.

  • The substrate is a small part of the lead: 1.2–1.6 points on Qwen3.5-122B; the gap to the fixed code-based improvers (Meta-Harness 36.0, I_sf 36.1 vs. 39.9) can only come from revising the improver.
  • On gpt-5.5, the lead over code-based baselines is larger than over text-based ones, so switching the substrate is not itself the source of the advantage.

Revision. Add an I_sf row to Table 1; replace "structural edits are a likely reason" in §4.2 with the grouped analysis above and describe the SpreadsheetBench cell.

AuthorsOpenReview ↗

Response to Reviewer wYvQ Part C

Positioning relative to RHI, AHE and Escher-Loop (Q4, W1.1, W1.2)

We thank the reviewer for these pointers. They are genuine precedents, and our claim is narrower: what is optimized is the improver's search procedure itself; every improver change must pass the same gate as the task harness, and the parent-selection rule is also learned.

1. Comparison along three axes

MethodWhat is optimizedOptimization procedureHow improver changes are accepted
STOP [1]Program and itselfRewrittenUnconditional replacement
DGM [2] / SICA [3] / Hyperagents [4]Agent code (incl. meta part)RewrittenArchive + empirical validation; fixed parent rule
Promptbreeder [5]Task prompt + mutation promptRewrittenTask fitness
RHI [6]Prompt-level spec of the agent loopFixed—
AHE [7]File-level harness componentsFixedNo held-out gate
Escher-Loop [8]Task agent + optimizerRewrittenJoins population, Elo-based elimination; no gate
RSI-HarnessTask harness + improver codeImprover rewrites itselfSame gate as the task harness; learned parent selection

Table C1: Comparison with related work along three axes.

2. What each design is worth

Table 2 of the paper ablates both designs (SpreadsheetBench, Qwen3.5-122B):

  • Outer gate: 3.2 points. Without it, the improver gain drops from 8.8 to 5.0; STOP, which has no gate, falls below T0 on all three backbones (Table 1).
  • Learned parent selection: 6.1 points, lost when replaced by greedy selection; DGM / SICA use a fixed rule.
  • The gate also works for the improver: because the improver edits itself under the same gate, no extra meta level is needed [11], and 86% of the 21 accepted improver revisions improve unseen audit tasks (Table 9).

3. RHI, AHE and Escher-Loop

  • RHI refines with a fixed procedure; we learn the procedure. A fixed procedure corresponds to w/o recursion in Table 2 (−13.1 points), and revising once from the same history (I_offline) gives +3.9 vs. +8.1 for I_final (Table 8).
  • AHE is a fixed optimizer of the task harness, like Meta-Harness [9] and Self-Harness [10]. The submission used only its seed harness NexAU0 (Table 3); we now run AHE itself as a baseline (Table C2).
  • Escher-Loop accepts optimizer changes without a gate, ranking them by Elo; our improver takes over only if it yields a better harness from the same T0 and budget, which is worth 3.2 points (Table 2). Escher-Loop is also evaluated on three geometric optimization problems, not on held-out agent benchmarks.

Initial scores, RSI-Harness and the reference column are from Tables 1 and 3:

Starting harness T0Initial scoreAHERSI-HarnessReference: Self-Harness / Meta-Harness
mini-SWE-agent (T0 of Table 1)39.944.8 (+4.9)49.9 (+10.0)45.0 / 47.6
NexAU0 (AHE's own seed)12.531.5 (+19.0)37.4 (+24.9)—

Table C2: AHE and RSI-Harness on Terminal-Bench (Qwen3.5-122B, N = 50).

Revision. Add Table C1 and a paragraph on each of RHI, AHE and Escher-Loop to §5 and Appendix G; cite Escher-Loop [8]; add an AHE column to Table 3; restate the contribution in §1 as the bolded sentence above.

References

  • [1] Zelikman et al. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation. 2023. arXiv:2310.02304
  • [2] Zhang et al. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. 2025. arXiv:2505.22954
  • [3] Robeyns et al. A Self-Improving Coding Agent. 2025. arXiv:2504.15228
  • [4] Zhang et al. Hyperagents. 2026. arXiv:2603.19461
  • [5] Fernando et al. Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. 2023. arXiv:2309.16797
  • [6] Lee et al. Recursive Harness Self-Improvement. 2026. arXiv:2607.15524
  • [7] Lin et al. Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses. 2026. arXiv:2604.25850
  • [8] Liu et al. Escher-Loop: Mutual Evolution by Closed-Loop Self-Referential Optimization. 2026. arXiv:2604.23472
  • [9] Lee et al. Meta-Harness: End-to-End Optimization of Model Harnesses. 2026. arXiv:2603.28052
  • [10] Zhang et al. Self-Harness: Harnesses that Improve Themselves. 2026. arXiv:2606.09498
  • [11] Schmidhuber. Evolutionary Principles in Self-Referential Learning, or on Learning How to Learn: The Meta-Meta-... Hook. Diploma thesis, TU Munich, 1987.
Reviewer_fUehOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper makes the harness optimizer (the "improver") an optimization target: an inner loop edits the task harness, an outer loop edits the improver, and an archive-backed gate accepts both. On five agent benchmarks and three backbones, it reports the best average on every backbone (Table 1), a 7.7-point mean gain over the starting harness, and a frozen-improver gain that grows from 1.3 to 6.2 points (Table 12). Several tables repeat each other's numbers in ways fresh runs would not produce; I cannot yet trust the evidence.

Critical strengths
  • The frozen-improver protocol (HImp@K, Section 4.3) with the Isf and Ioffline controls (Tables 7 and 8) is the right test of a recursive claim.
  • Figure 4 moves only the learned improver across backbones and benchmarks; the copied task harness helps little (+0.2 to +0.9) and hurts across domains (-1.6, -2.3).
  • The paper admits its margin over the strongest baseline is inside the bootstrap interval (Table 11).
Critical weaknesses
  • Table 10 (eight seeds) reproduces every three-run mean of Table 1 in all nine cells (55.6/58.2/62.3, 30.8/36.7/39.9, 29.1/33.2/36.5), and all 30 per-benchmark differences in Table 11 equal Table 1's. Fresh seeds would not reproduce all 39. Say which runs they used.
  • Table 8 gives HImp@50 on split A as +1.7 (I0) and +8.8 (Ifinal), the full-split values of Table 2. Explain.
  • Table 9 (a separate split) pools 21 accepted improver revisions and 96 task-harness edits over three seeds; Table 6 and Appendix A.3 report 44 revisions, and Table 5 reports a 17.3% acceptance rate for Ifinal. Explain the gap.
  • Terminal-Bench has 18 test tasks (Table 13). Three single-attempt runs put means on a 1.85-point grid; none of its 27 entries in Table 1 are. Report attempts per task.
  • The 69% token saving (abstract) rests on Figure 2b: one backbone, three of six baselines, and Meta-Harness's 1.24B drives the mean. Table 5 puts the improver at 38 to 56k tokens per step, so most of the 0.21B looks like matched rollouts (Appendix J). Split rollout and optimizer tokens.
  • The gate sets for Terminal-Bench and tau3-bank hold 9 and 10 tasks (Table 13); the false-accept audit (Table 9) covers only SpreadsheetBench. Repeat it on one small set.
  • Section 3.3 never states how many rollouts each candidate improver gets in the outer-gate comparison. State the allocation.
Questions for authors

STOP (Zelikman et al., COLM 2024) already improves an improver with a utility function and measures the improved improver on downstream tasks. Hyperagents (Zhang et al., arXiv 2603.19461, 2026) edit the meta level and transfer it across domains. Self-Harness (Zhang et al., arXiv 2606.09498, 2026) and Meta-Harness (Lee et al., 2026) optimize harnesses with a fixed procedure. The paper adds the harness setting, the gated outer loop, and the frozen-checkpoint measurement. Table 1 lists Self-Harness as self-referential, while Figure 10 labels its improver harness "fixed". Appendix G cites Lin et al. (arXiv 2605.30621), who find harness-updating flat in base capability, but does not say how that squares with an updater harness that matters this much (1.3 vs 6.2). Contemporaneous: SBCO (arXiv 2608.10157), RSIFlow (arXiv 2610.21791).

The 7.7 gain and the 2.7x ratio check out. Claw-SWE-Bench (Figure 4) is never defined. "Closing the recursive loop" appears on p2 and p3, and p8 has "a notable form of meta-generalization"; cut these.

5 comments
AuthorsOpenReview ↗

Response to Reviewer fUeh Part A

Run provenance, splits and scoring (W1–W4)

We thank the reviewer for checking the tables one by one. Each overlap has a definite source: Tables 1, 10 and 11 share runs; column A of Table 8 and Table 2 share one evaluation; 21, 44 and 17.3% measure different quantities; and each Terminal-Bench task is evaluated 10 times.

W1. Tables 1, 10 and 11

They share the same runs. All methods were run with three seeds, and three rows per backbone received five more for Appendix F:

  • 8 seeds: T0, the strongest baseline and RSI-Harness in Table 1 on each backbone. Tables 10 and 11 reuse these runs, so their means agree.
  • 3 seeds: all other rows and analyses (other baselines, ablations throughout the paper).

Table 11's hierarchical bootstrap uses the per-task results of these runs (seeds resampled within each method, test tasks resampled jointly across methods), so its 30 per-benchmark point estimates match Table 1. The Table 1 caption will state the seed count of each row.

W2. Column A of Table 8 and Table 2

They are the same evaluation (I_0 +1.7, I_final +8.8). The generalization test is column B, a disjoint split, where the two improvers reach +1.6 and +8.1.

W3. Results gap in the tables.

NumberSourceWhat it measures
44Table 6, Appendix A.3Accepted improver revisions on the default split, three seeds summed (14.7 of 16 per seed)
21Table 9Accepted improver revisions on the audit split, three seeds summed
17.3%Table 5Share of task-harness candidates from the frozen I_final that pass the inner gate

Table A1: the table number comparison in the original table

21 in table 9 and 44 in table 6 and appendix a.3 come from different splits, and 17.3% from table 5 is a different quantity. Table 9 is a separate set of runs that re-splits SpreadsheetBench into training, gate and audit tasks, so its count is not comparable to the default split. 17.3% is an acceptance rate of task-harness candidates, not a count of improver revisions. The revised Table 9 caption will state the counting unit of its 96 edits.

W4. Terminal-Bench evaluations

Each Terminal-Bench task is evaluated 10 times, as are τ³-bank and GAIA-2; SWE-bench Pro and SpreadsheetBench are evaluated once. Scores are averaged over tasks and then over seeds. The 1.85-point grid assumes one evaluation per task (100/18 ≈ 5.56 per seed, 1.85 for a three-seed mean). With 10 evaluations, a seed has up to 180 episodes and its score moves in steps of 0.56 points; episodes that fail for environment reasons (e.g., container build failures) are excluded for all methods, so a few seeds have slightly fewer. We will release the full running history later.

Revision. State the seed count and mean ± SD of each row in Table 1; replace "rerun" in Appendix F with "five additional seeds"; state the counting unit in Table 9.

AuthorsOpenReview ↗

Response to Reviewer fUeh Part B

Token cost (W5)

We thank the reviewer for raising this point. The 0.21B includes all outer-loop cost, and we will report the savings against each baseline instead of the 69% average. The 69% is calculated by the mean of three baselines; for each baseline, RSI-Harness uses 32%, 53% and 83% fewer tokens than Self-Harness, GEPA and Meta-Harness.

MethodTask tokens (B)Optimizer tokens (B)Total tokens (B)
GEPA0.330.120.45
SkillOpt0.310.050.36
Meta-Harness0.340.901.24
STOP0.270.020.29
Hyperagents0.330.150.48
Self-Harness0.280.030.31
RSI-Harness0.200.010.21

Table B1: Token cost per method (Qwen3.5-122B, mean over five benchmarks and three seeds).

Table 5's 38k–56k tokens per step exclude the outer loop. The outer-loop rewrites are in the optimizer tokens, and the gate rollouts of both branches are in the task tokens.

Key findings.

  • The 0.21B includes all outer-loop cost: as the reviewer estimated, most of it is task tokens.
  • Savings are reported per baseline: 32%, 53% and 83%, instead of one average.

Revision. Add Table B1 to Appendix J; state the token composition in the Figure 2b caption; replace the 69% in the abstract, §1 and §6 with the per-baseline numbers.

AuthorsOpenReview ↗

Response to Reviewer fUeh Part C

Small-gate audit and outer-gate allocation (W6, W7)

We thank the reviewer for raising this point. Edits accepted on the 9-task gate still help on unseen tasks. We repeat the audit of Table 9 on Terminal-Bench, which has the smallest gate. A small gate does inflate the gain of a single edit, but accepted edits stay positive on the audit tasks on average, with fewer false accepts than a gate without signal.

W6. False-accept audit on Terminal-Bench

Whose editsAccepted editsNumberMean Δaudit ↑False accepts ↓Audit gain of final harness ↑
Reference: gate without signal——0≈57%0
I_0 (w/o outer loop)Task-harness edits18+0.350%+1.6 ± 2.3
RSI-HarnessTask-harness edits36+1.239%+8.6 ± 4.9
RSI-HarnessImprover revisions20+1.335%—

Table C1: False-accept audit on Terminal-Bench (three seeds pooled).

The gate is 9 validation tasks evaluated once (one extra solved task = +11.1 points); the audit set is 27 unseen training-pool tasks evaluated three times; a false accept is an accepted edit with Δaudit ≤ 0, as in Table 9. Table 12 confirms this on the test set: frozen checkpoints improve Terminal-Bench by +1.9 → +3.5 → +5.4 → +7.9.

W7. Outer-gate rollouts

The outer gate is paid from the matched budget. Both branches start from the same harness and history with the same budget: each comparison uses 46 rollouts on average over the five benchmarks, 23 per branch, run on the same training tasks. The resulting share of all rollouts is 18.6% on SpreadsheetBench (Table 6); we will report the share on every benchmark in Appendix C.

Key findings.

  • Gate noise does not accumulate into false gains: accepted edits are positive on unseen tasks on average, and the final harness gains +8.6 there with RSI-Harness versus +1.6 with I_0.
  • The learned improver is fooled less often by the same gate: its per-edit audit gain is about 4 times that of I_0 (+1.2 vs. +0.3), with 11 points fewer false accepts (39% vs. 50%).

Revision. Add the Terminal-Bench audit to Appendix E; state in §3.3 the rollouts per outer-gate branch and how they are charged to the budget.

Reviewer_bRD4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RSI-Harness, which improves an LLM agent by optimizing its harness while keeping the model weights fixed. Besides an inner loop that edits the harness, an outer loop also revises the improver that makes these edits, so the optimizer itself gets better over time. Experiments on several agent benchmarks and backbones show the best average score, and frozen improver checkpoints show growing gains.

Critical strengths
  1. The paper is well-written and easy to follow.
  2. Evaluating frozen improver checkpoints from the same starting point is a clean way to show that the optimizer itself gets better.
Critical weaknesses
  1. Inconsistent Results. Some scores in the main table cannot come from the stated evaluation protocol given the small test sets, and the rerun results match the original ones too closely. The authors should explain how the scores are computed.
  2. Limited Evidence for Recursion. Most of the analysis is done on a single backbone and a single benchmark, so it is unclear whether the recursive design helps in general.
Questions for authors
  1. How are the evaluation runs repeated and aggregated?
  2. What is the extra cost of the outer loop compared with the baselines?
  3. Does the benefit of recursion still hold on other backbones and benchmarks?
4 comments
AuthorsOpenReview ↗

Response to Reviewer bRD4 Part A

Evaluation protocol and aggregation (Q1, W1)

We thank the reviewer for raising this point. The three small test sets are evaluated 10 times per task, Table 1 averages over tasks and then over seeds, and the matching rows of Tables 10 and 1 come from the same runs, which is why their means agree.

1. Repeats per task

The three small test sets are evaluated 10 times per task: as Table A1 shows, Terminal-Bench 2.0 (18 tasks), τ³-bank (19) and GAIA-2 (32) are far smaller than SWE-bench Pro (146) and SpreadsheetBench (183). With one evaluation per task, flipping a single task would move their score by 3 to 6 points and carry straight into the five-benchmark average. We therefore evaluate each task of these three benchmarks 10 times with independent sampling and the other two once, so that every benchmark contributes at similar scale. Test-time repeats are identical for all methods and are not part of the optimization budget.

BenchmarkTest tasksEvaluations per taskEpisodes per seedPoints per episode
SWE-bench Pro14611460.68
SpreadsheetBench18311830.55
Terminal-Bench 2.018101800.56
τ³-bank19101900.53
GAIA-232103200.31

Table A1: Test-set size and evaluations per task for the five benchmarks.

2. Aggregation order and score resolution

Scores are averaged over tasks first and over seeds second: a task's score is its fraction of successful attempts, a seed's benchmark score is the mean over tasks, and the seed scores are then averaged. With 18 test tasks evaluated once each, a per-seed Terminal-Bench score could only be a multiple of 5.56 points; with 10 evaluations per task it moves in steps of 0.56 points. Two further details set the resolution of the other cells: SpreadsheetBench uses the official soft-restriction judge, which scores each instruction by the fraction of its test spreadsheets solved, so per-task scores are fractional; and on tau3-bench, SWE-bench Pro and Terminal-Bench, episodes that fail for environment reasons (e.g., container build failures) are excluded for all methods, so a seed's denominator can be slightly below the test-set size. We will release the full history later.

3. Number of seeds

The T0, strongest-baseline and RSI-Harness rows have eight seeds, all other rows three: every method was first run with three seeds, and three methods received five additional seeds for the uncertainty analysis of Appendix F.

  • 8 seeds: the T0, strongest-baseline and RSI-Harness rows of Table 1 on each backbone. Tables 10 and 11 are ablations with same runs, so their means agree.
  • 3 seeds: the other baselines in Table 1 and throughout the paper.

The original caption was not updated after the additional runs; the revised Table 1 will state the seed count of each row.

Revision. Add the clear evaluation protocol to the Metrics paragraph of Appendix J; state the seed count of each row in the Table 1 caption, report every cell as mean ± SD and cite the evaluations per task; state the evaluations per task in each benchmark entry of Appendix H; change "rerun" in Appendix F to "five additional seeds".

AuthorsOpenReview ↗

Response to Reviewer bRD4 Part B

Extra cost of the outer loop (Q2)

We thank the reviewer for the question. The outer loop adds no rollouts, and its tokens are already in the 0.21B, which is still below every baseline.

1. Rollouts

The outer loop uses rollouts from the same budget, not extra ones. All methods get the same number of task-agent rollouts over N = 50 steps, and outer-loop comparisons count toward it (Appendix J). RSI-Harness therefore spends part of its own share on the outer loop, while the baselines spend all of theirs on task-harness edits. With the default setting, the outer loop takes 18.6% of the rollouts on SpreadsheetBench and raises the task score by 15.3 points over no outer loop (Table 6); 18.6% is the share on this benchmark, and we will report it for the other four.

2. Tokens

Both parts of the outer loop are already in the 0.21B. Gate rollouts are counted as task tokens, and improver calls, including the 16 outer-loop rewrites, as optimizer tokens (Appendix J).

MethodInner-loop stepsTask tokens (B)Optimizer tokens (B)Total tokens (B)
GEPA500.330.120.45
SkillOpt500.310.050.36
Meta-Harness500.340.901.24
STOP500.270.020.29
Hyperagents500.330.150.48
Self-Harness500.280.030.31
RSI-Harness500.200.010.21

Table B1: Token cost per method (Qwen3.5-122B, mean over five benchmarks and three seeds).

Key findings.

  • No extra rollouts: RSI-Harness gives 18.6% of the same budget to the outer loop and still has the best average score on every backbone.
  • No hidden tokens: with the outer loop included, 0.21B is 32%, 53% and 83% below Self-Harness, GEPA and Meta-Harness.

Revision. Add Table B1 to Appendix J and state that "the task side is the same" refers to the number of rollouts; state the token composition in the Figure 2b caption; report the outer-loop share on every benchmark in Appendix C.

AuthorsOpenReview ↗

Response to Reviewer bRD4 Part C

Does the benefit of recursion hold on other backbones and benchmarks? (Q3, W2)

We thank the reviewer for the question. Removing recursion loses most of the gain on four backbones and five benchmarks, and the learned improver transfers without further updates. We extend the two recursion ablations of Table 2, originally run only on SpreadsheetBench with Qwen3.5-122B.

1. Direct evidence in the paper

  • Removing recursion removes most of the gain: fixing the improver at I_0 (50.1) or letting I_0 write each new improver (52.3) loses over three quarters of the full gain (65.4 vs. T0 48.3; Table 2).
  • The gain comes from repeated revision, not one pass over the history: on a disjoint split, I_final gains +8.1 vs. +3.9 for I_offline (Table 8).
  • Recursion adds to hand-written procedures: RSI-Harness beats I_sf, which starts with the learned procedures, by 3.8 points on average (Table 7).

2. Generalizability across backbones

The gain holds on every backbone. The Qwen3.5-122B row is Table 2; T0 and RSI-Harness for gpt-5.5 and Gemma-4-31B are from Table 1.

BackboneT0RSI-Harnessw/o outer loopw/o recursion
Qwen3.5-122B (Table 2)48.365.450.152.3
gpt-5.552.066.853.354.5
Gemma-4-31B43.759.445.347.6
Qwen3-8B31.541.932.633.9

*Table C1: Recursion ablations on four backbones (SpreadsheetBench). *

3. Generalizability across benchmarks

The gain holds on every benchmark. The T0, RSI-Harness and w/o outer loop columns are from Table 7 (w/o outer loop is its Fixed I_0 row); w/o recursion is new.

BenchmarkT0RSI-Harnessw/o outer loopw/o recursion
SpreadsheetBench (Table 2)48.365.450.152.3
SWE-bench Pro33.841.335.136.0
Terminal-Bench 2.039.949.941.843.0
τ³-bank15.321.616.217.0
GAIA-216.921.217.618.1

Table C2: Recursion ablations on five benchmarks (Qwen3.5-122B).

4. Further transferability results.

What transfers is the improver, not the task harness. We freeze the I_final learned on SWE-bench Pro with gpt-5.5 and apply it to new settings (Figure 4; the Qwen3-8B row is new). Gains are over each target's T0:

TransferTargetI_0T_final (copied)I_finalI_final / I_0
New backboneQwen3.5-122B+1.3+0.6+5.84.5×
Gemma-4-31B+1.5+0.3+4.42.9×
Qwen3-8B+1.1−0.2+3.22.9×
New benchmarkTerminal-Bench+2.4+0.2+4.92.0×
Claw-SWE-Bench+2.5+0.9+4.61.8×
New domainτ³-bank+1.1−1.6+2.92.6×
GAIA-2+1.2−2.3+3.12.6×

Table C3: Gains of the frozen I_final over the target T0 after transfer to new settings.

Key findings.

  • Recursion is the source of the gain on every backbone and benchmark: in every row, full > w/o recursion > w/o outer loop.
  • The learned improver transfers: I_final beats I_0 in every new setting, while the copied task harness adds under 1 point or hurts.

Revision. Add Tables C1 and C2 to Appendix D as an extended Table 2; add the Qwen3-8B row and a numeric table to Figure 4; state in §4.3 which backbones and benchmarks each conclusion covers.

Thank you for the response. Clarification of the evaluation protocol and additional ablations are helpful. However, my main concerns remain that the test sets are too small to reflect the method's true effectiveness and that the method is not fundamentally different from existing self-referential optimizers. I will keep my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.