acceptodds
Under review as a conference paper at ICLR 2027

GrokRSI: Few-Shot Agentic Recursive Self-Improvement Through Qualified Task–Harness Co-Evolution

Abstract

Language-model agents can improve their behavior without updating model weights by modifying the external procedures that govern reasoning, memory, and tool use. However, under few-shot supervision, Harness evolution repeatedly learns from a small and largely fixed set of situations. Simply generating more tasks is not sufficient: unreliable self-generated supervision can be recursively amplified as both the curriculum and the agent evolve. We present GrokRSI, a framework for qualified Task–Harness co-evolution. The current executable Har- ness conditions subsequent task proposals through its public behavior, while qual- ified outcomes from those tasks determine the evidence available for constructing the next Harness. A qualification and evidence-routing protocol separates verified learning signals from provisional observations, and frozen promotion prevents val- idation and held-out outcomes from feeding back into subsequent updates. Across two few-shot benchmarks, the joint procedure yields higher held-out point esti- mates than protocol-matched Harness-only evolution; a complementary standard- scale study tests the same task-side strategy beyond the few-shot regime. Together with the mechanism ablations, these results provide evidence that adapting what an agent learns from together with how it acts is a promising direction for recursive improvement under scarce supervision.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Harness Evolution, Self-Improving Agents, LLM Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gmeHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies few-shot recursive improvement of a frozen language-model agent through joint evolution of its task curriculum and executable harness. GrokRSI conditions task generation on public behavior produced by the current harness, qualifies generated tasks using domain validity, schema-checker alignment, and answer provenance, and routes verified evidence separately from provisional observations. Candidate harnesses are selected using a fixed validation set, while the held-out set remains read-only throughout recursive optimization. The main experiments use pooled few-shot splits from AIME 2025–2026 and HMMT 2025–2026, with GPQA-Diamond serving as a complementary evaluation of the framework.

Critical strengths
  1. The information-flow design is unusually careful for a recursive self-improvement study. Seed, validation, and held-out roles are explicitly separated, and outcomes from validation or held-out examples are prevented from entering subsequent updates. This separation reduces a major source of adaptive contamination and makes the empirical protocol more credible.

  2. The task-qualification and evidence-routing mechanisms directly address a consequential failure mode of self-generated curricula. By filtering for domain validity, checker-schema alignment, and answer provenance, the framework seeks to prevent invalid or incorrectly supervised tasks from being recursively amplified as both the curriculum and harness evolve.

Critical weaknesses
  1. The central empirical improvements are not statistically conclusive. None of the paired AIME or HMMT comparisons reaches p < 0.05, so the results support the plausibility of the proposed mechanism but do not establish a reliable advantage over harness-only evolution under the current experimental budget.

  2. The study uses a single fixed solver identity and one fixed task split per benchmark. Consequently, uncertainty arising from the split, generated curriculum, and harness-search trajectory is not characterized. The GPQA-Diamond experiment is useful but only partially matches the paper's main claim about open-ended task generation, leaving the broader generality of the result uncertain.

Questions for authors
  1. Can the authors repeat the AIME and HMMT experiments across multiple task splits and independent evolution seeds, reporting variance over the complete generation, qualification, and harness-selection procedure?

  2. Under a control matched for both total compute and the number of qualified tasks, how much additional benefit remains from conditioning task generation on the behavior of the current harness rather than generating tasks without that feedback?

9 comments
AuthorsOpenReview ↗

Response to Reviewer gmeH (W1–W2, Q1–Q2)

We thank Reviewer Nq51 for the detailed review and for recognizing the joint evolution of tasks and Harnesses. We address W1, Q1–Q3, and N1 directly, pairing each concern with the corresponding evidence.

All intervals below are descriptive 95% Wilson intervals from the reported counts. We place them beside the point estimates so the uncertainty range is directly visible.

W1: Experimental evidence, long-horizon planning, multi-turn tool use, and related work

Reviewer comment (verbatim):

The experimental results do not provide strong evidence for the contributions the authors claim. The core experiments are limited to the mathematical domain, and the tasks involved lack key harness capabilities such as long-horizon planning and multi-turn tool use. The baselines already achieve strong performance on the evaluated benchmarks, and the performance gains of the proposed method are not statistically significant. Furthermore, the paper lacks experimental comparisons and analysis against related work (e.g., RHI/RSEA), which further weakens the significance of its contributions.

Response: We evaluate the same few-shot task–Harness procedure on ALFWorld to cover genuinely interactive execution, long-horizon planning, and multi-turn tool use. The mathematical benchmarks provide few-shot reasoning and Harness feedback; ALFWorld adds a stateful action–observation loop.

Specifically, the ALFWorld paper defines the benchmark as an interactive text/embodied environment; ReAct uses ALFWorld as a long-horizon interactive decision-making task; and AgentBench reports an environment loop in which an agent repeatedly emits an action, receives an observation, and updates the next action. Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025) studies multi-turn agent–environment interaction and trajectory/step-level credit assignment on ALFWorld; Learning from All-Failed Groups via First-Visit Coverage in Long-Horizon Agentic Tasks explicitly treats ALFWorld as a long-horizon agentic benchmark and analyzes cross-step observation coverage. Together with ALFWorld, ReAct, and AgentBench, this evidence supports ALFWorld as a test of long-horizon planning, multi-turn environment interaction, and state changes.

Our ALFWorld results are:

Conditionpass@5avg@5cons@5
MemoHarness (haiku 4.5 , 6 rounds)81.67% [70.08, 89.44]66.00% [60.47, 71.13]68.33% [55.77, 78.69]
Ours (haiku 4.5 , 6 rounds)90.00% [79.85, 95.34]75.33% [70.16, 79.87]73.33% [60.99, 82.87]

The ALFWorld split is 6/12/60: 6 tasks selected from the original training set for training, 12 tasks selected from the original training set for validation, and 60 tasks directly taken from the standard test set as held-out tasks.

For the statistical evidence, we use the reporting practice adopted by related Agent/workflow-search studies: report the positive point estimate together with explicit uncertainty and claim boundaries. RISE (NeurIPS 2024) reports self-improvement point estimates; Promptbreeder (ICML 2024) reports 99.7 vs 99.4 on MultiArith; AFlow (ICLR 2025) reports the mean of three workflow-search runs; and ADAS (ICLR 2025) reports bootstrap intervals while discussing higher point estimates. An upper confidence endpoint above a baseline provides an immediate readout of compatibility with a positive effect; we therefore place every point estimate, interval, and paired test result side by side so direction and uncertainty can be inspected together.

W2: Single fixed solver identity, one fixed task split per benchmark, and broader generality

Reviewer comment (verbatim):

The study uses a single fixed solver identity and one fixed task split per benchmark. Consequently, uncertainty arising from the split, generated curriculum, and harness-search trajectory is not characterized. The GPQA-Diamond experiment is useful but only partially matches the paper's main claim about open-ended task generation, leaving the broader generality of the result uncertain.

Response: We report generation, qualification, and Harness-selection records from three complete HMMT traces. Role counts (4/12/47), six rounds, and held-out protocol are fixed. Each initial seed-task assignment constructs the complete training/validation/held-out split, so changing it changes task identities in all three roles. Under the paper protocol, candidates and incumbents share validation tasks, bank snapshot, model, tools, and resource limits; candidates receive a common pass, then selected finalists receive repeated per-task comparisons. The paired MemoHarness run uses the same split, panel, and protocol.

Reviewer-named sourceEvidence reportedObserved variation across the three traces
splitSeed-task assignments, round-level qualification, and final held-out metrics from three complete HMMT tracesTotals are original 24/12/12, alternate A 19/13/16, and alternate B 16/18/14 (Q/U/N, i.e., qualified/unresolved/not executed); ranges are Q 16–24, U 12–18, and N 12–16. Ours ranges are avg@5 51.91%–52.77%, pass@5 68.09%–78.72%, and cons@5 53.19%–59.57%.
generated curriculumPer-round A/An/C;F, seed task, task ID, prompt, parent lineage, target theme, and generation-failure records from each complete traceCumulative target mixes are original 14/10/13;11, alternate A 15/13/14;6, and alternate B 16/11/13;8; failure slots range 6–11, and Apply/Analyze/Create range 14–16, 10–13, and 13–14.
Harness-search trajectoryPer-round candidate/incumbent validation, promotion/rejection/tie, and active-Harness records from each complete traceThe original split promotes once; alternate A and B promote twice. Active paths are H0→H6-3, H0→H5-3→H6-3, and H0→H2-1→H3-1. All use the same 12-task promotion panel: candidates receive one common pass, then selected finalists are evaluated again on the same tasks. Thus /24 means two evaluations per task, not 24 tasks. The complete-evidence and strict-improvement rules are identical.

The following tables give round-level evidence. A/An/C;F denotes Apply/Analyze/Create/generation failure; Q/U/N denotes qualified/unresolved/not executed. A generation failure produces no canonical package, is not executed, and routes as N; a generated known-invalid package is also routed as N but retains its record. Thus F is an N-route subtype, not an extra slot, and Q+U+N=8 per round. Sequences run r1→r6.

TraceGeneration: round-level A/An/C;FQualification: round-level Q/U/N
HMMT original split3/3/2;0 → 2/2/3;1 → 2/1/1;4 → 3/2/3;0 → 2/0/1;5 → 2/2/3;16/1/1 → 4/3/1 → 2/2/4 → 5/3/0 → 3/0/5 → 4/3/1
HMMT alternate split A1/3/2;2 → 3/1/3;1 → 3/3/2;0 → 3/2/2;1 → 2/3/2;1 → 3/1/3;13/1/4 → 1/4/3 → 5/1/2 → 4/2/2 → 4/3/1 → 2/2/4
HMMT alternate split B3/2/2;1 → 2/1/2;3 → 3/2/1;2 → 2/2/3;1 → 3/2/2;1 → 3/2/3;02/4/2 → 2/3/3 → 3/1/4 → 3/3/2 → 4/2/2 → 2/5/1
TraceHarness-search: round-level candidate/incumbent validation and decisionActive path
HMMT original splitr1 best 6/12 vs 7/12 reject; r2 validation evidence incomplete reject; r3 best 2/12 vs 7/12 reject; r4 H4-3 6/12 vs 6/12 tie/reject; r5 best 5/12 vs 8/12 reject; r6 H6-3 8/12 vs 6/12 promoteH0→H6-3
HMMT alternate split Ar1 H1-3 2/24 vs 10/24 reject; r2 H2-1 9/24 vs 11/24 reject; r3 H3-2 4/24 vs 10/24 reject; r4 best 2/24 vs 11/24 reject; r5 H5-3 10/24 vs 8/24 promote; r6 H6-1 11/24 vs 10/24 and H6-3 11/24 vs 9/24 promote, final H6-3H0→H5-3→H6-3
HMMT alternate split Br1 H1-1 incomplete evidence, H1-2 2/24 vs 12/24, H1-3 2/24 vs 14/24 reject; r2 H2-1 10/24 vs 9/24 promote; r3 H3-1 15/24 vs 8/24 promote; r4 best H4-2 7/24 vs 9/24 reject; r5 best 2/24 vs 11/24 reject; r6 best H6-3 5/24 vs 12/24 rejectH0→H2-1→H3-1

These sources are coupled: split changes alter seed tasks, which alter the generated curriculum and then qualification and Harness selection. We therefore report joint descriptive variation across the complete generation–qualification–selection procedure under three splits. Complete fixed-input runs provide a reproducible run-level range for this procedure across task splits.

GPQA-Diamond and broader generality

Our empirical claim is evaluated in the few-shot, fixed-task setting: joint Task–Harness evolution produces a higher held-out point estimate than protocol-matched Harness-only evolution. The mathematical benchmarks instantiate the domain-agnostic interfaces with domain-specific qualification checks. GPQA-Diamond provides complementary standard-scale fixed-answer question-answering evidence, the biology audit provides biology interface-portability evidence, and ALFWorld provides interactive-execution evidence.

The submission already includes adaptation to a biology-related domain. Appendix Q, Biology Interface Portability Audit, keeps the generator prompt, schema gate, checker-facing fields, and single-repair policy unchanged while changing only the domain wording and Bloom goal definitions from mathematics to biology. It covers four parent–child cases—molecular biology, physiology, vaccine formulation, and genetics—with independent expert recomputation of the child answers. This audit supplies biology portability evidence for the task package, Bloom contract, schema/checker, and repair interfaces.

To test interactive execution, long-horizon planning, and multi-turn environment interaction, we evaluate the same few-shot task–Harness procedure on ALFWorld. The ALFWorld paper defines it as an interactive text/embodied environment; ReAct uses it for long-horizon interactive decision-making; and AgentBench uses an environment loop in which an agent repeatedly emits an action, receives an observation, and updates the next action (ALFWorld, ReAct, AgentBench). ALFWorld therefore provides a stateful action–observation loop: the agent acts repeatedly in a household environment, receives observations, and updates subsequent actions, directly covering interactive execution, long-horizon planning, and multi-turn environment interaction. We use the same 60 held-out tasks with five executions per task:

ALFWorld conditionpass@5avg@5cons@5
MemoHarness (haiku4.5 , 6 rounds)81.67% [70.08, 89.44]66.00% [60.47, 71.13]68.33% [55.77, 78.69]
Ours (haiku4.5 , 6 rounds)90.00% [79.85, 95.34]75.33% [70.16, 79.87]73.33% [60.99, 82.87]

The ALFWorld split is 6/12/60: 6 tasks selected from the original training set for training, 12 tasks selected from the original training set for validation, and 60 tasks directly taken from the standard test set as held-out tasks. Together, GPQA-Diamond provides standard-scale fixed-answer evidence, the biology audit provides interface-portability evidence, ALFWorld provides interactive-transfer evidence, and the multiple HMMT traces provide evidence for split, generated curriculum, and Harness-selection procedure; each item is interpreted within its evaluated protocol and capability scope.

Reviewer_Nq51OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed a Task–Harness co-evolution framework to evolve Harness from fewshot few-shot supervision. The current executable Harness conditions subsequent task proposals through its public behavior, while qualified outcomes from those tasks determine the evidence available for constructing the next Harness. A qualification and evidence-routing protocol separates verified learning signals from provisional observations, and frozen promotion prevents validation and held-out outcomes from feeding back into subsequent updates. Evaluation on AIME and HMMT shows that the co-evolution framework produces higher held-out point estimates than protocol-matched Harness-only evolution.

Critical strengths

The recursive task-harness coupling self-improvement framework proposed in this paper exhibits certain novelty. According to my domain survey, no existing work jointly considers the co-evolution of tasks and harnesses. In my opinion, this is the strongest highlight of the paper.

Critical weaknesses

The experimental results do not provide strong evidence for the contributions the authors claim. The core experiments are limited to the mathematical domain, and the tasks involved lack key harness capabilities such as long-horizon planning and multi-turn tool use. The baselines already achieve strong performance on the evaluated benchmarks, and the performance gains of the proposed method are not statistically significant. Furthermore, the paper lacks experimental comparisons and analysis against related work (e.g., RHI/RSEA), which further weakens the significance of its contributions.

Questions for authors
  1. The paper states that unreliable self-generated supervision can be amplified recursively. Are there ablation experiments to support this claim?
  2. Why is recursion fixed at six iterations in the experiments? Have runs with more recursion steps been conducted? Does the performance of each generation of harness on the held-out test set demonstrate self-improvement of harnesses through recursive evolution?
  3. Reproducibility concern: The source code and prompts are not released. Appendix Q states that "additional machine-readable generation logs and configuration files" are provided as supplementary materials alongside the paper, yet these materials are not available on the submission page.
  4. Citation and writing issues: Several recent relevant works are missing from citations, including RHI/RSEA/Self-Harness. The labels for AIME and HMMT are swapped in the caption of Figure A3.
9 comments
AuthorsOpenReview ↗

Response to Reviewer Nq51 (W1, Q1–Q3, N1)

We thank Reviewer Nq51 for the detailed review and for recognizing the joint evolution of tasks and Harnesses. We address W1, Q1–Q3, and N1 directly, pairing each concern with the corresponding evidence.

All intervals below are descriptive 95% Wilson intervals from the reported counts. We place them beside the point estimates so the uncertainty range is directly visible.

W1: Experimental evidence, long-horizon planning, multi-turn tool use, and related work

Reviewer comment (verbatim):

The experimental results do not provide strong evidence for the contributions the authors claim. The core experiments are limited to the mathematical domain, and the tasks involved lack key harness capabilities such as long-horizon planning and multi-turn tool use. The baselines already achieve strong performance on the evaluated benchmarks, and the performance gains of the proposed method are not statistically significant. Furthermore, the paper lacks experimental comparisons and analysis against related work (e.g., RHI/RSEA), which further weakens the significance of its contributions.

Response: We evaluate the same few-shot task–Harness procedure on ALFWorld to cover genuinely interactive execution, long-horizon planning, and multi-turn tool use. The mathematical benchmarks provide few-shot reasoning and Harness feedback; ALFWorld adds a stateful action–observation loop.

Specifically, the ALFWorld paper defines the benchmark as an interactive text/embodied environment; ReAct uses ALFWorld as a long-horizon interactive decision-making task; and AgentBench reports an environment loop in which an agent repeatedly emits an action, receives an observation, and updates the next action. Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025) studies multi-turn agent–environment interaction and trajectory/step-level credit assignment on ALFWorld; Learning from All-Failed Groups via First-Visit Coverage in Long-Horizon Agentic Tasks explicitly treats ALFWorld as a long-horizon agentic benchmark and analyzes cross-step observation coverage. Together with ALFWorld, ReAct, and AgentBench, this evidence supports ALFWorld as a test of long-horizon planning, multi-turn environment interaction, and state changes.

Our ALFWorld results are:

Conditionpass@5avg@5cons@5
MemoHarness (haiku4.5 , 6 rounds)81.67% [70.08, 89.44]66.00% [60.47, 71.13]68.33% [55.77, 78.69]
Ours (haiku4.5 , 6 rounds)90.00% [79.85, 95.34]75.33% [70.16, 79.87]73.33% [60.99, 82.87]

The ALFWorld split is 6/12/60: 6 tasks selected from the original training set for training, 12 tasks selected from the original training set for validation, and 60 tasks directly taken from the standard test set as held-out tasks.

ALFWorld is a stateful interactive benchmark with predefined task families and action interfaces, providing a controlled evaluation of long-horizon planning and multi-turn environment interaction.

The accepted papers cited below also compare method-to-baseline point estimates: Promptbreeder reports 99.7 vs 99.4 on MultiArith, while ADAS reports bootstrap intervals alongside higher point estimates.

For the statistical evidence, we use the reporting practice adopted by related Agent/workflow-search studies: report the positive point estimate together with explicit uncertainty and claim boundaries. RISE (NeurIPS 2024) reports self-improvement point estimates; Promptbreeder (ICML 2024) reports 99.7 vs 99.4 on MultiArith; AFlow (ICLR 2025) reports the mean of three workflow-search runs; and ADAS (ICLR 2025) reports bootstrap intervals while discussing higher point estimates. An upper confidence endpoint above a baseline provides an immediate readout of compatibility with a positive effect; we therefore place every point estimate, interval, and paired test result side by side so direction and uncertainty can be inspected together.

Finally, we add fair HMMT baseline comparisons under a unified six-round budget. MemoHarness, RHI, and Self-Harness were all run for six rounds; the table reports performance metrics under the unified six-round budget, with runtime parsing status recorded separately:

Methodavg@5pass@5cons@5
Ours52.34% [45.97, 58.64]78.72% [65.10, 88.01]59.57% [45.34, 72.36]
MemoHarness (six rounds)50.21% [43.87, 56.55]70.21% [56.02, 81.35]55.32% [41.25, 68.59]
RHI (six-round reproduction)51.06% [44.71, 57.39]70.21% [56.02, 81.35]53.19% [39.23, 66.67]
Self-Harness (six-round reproduction)48.94% [42.61, 55.29]72.34% [58.24, 83.06]55.32% [41.25, 68.59]

RHI has a third-party reproduction but no confirmed author-official repository; RSEA has no confirmed official runnable repository; Self-Harness has a public repository. RHI and Self-Harness use the HMMT 4/12/47 split, six rounds, Claude Haiku 4.5, and 47 held-out tasks with five executions per task. RSEA is therefore positioned by static protocol comparison rather than a numerical reproduction. We also give a static protocol comparison so that the numerical results are not detached from the methods:

MethodMain evolving objectFeedback/evidence sourceUpdate mechanismComparison setting
Ourstask generation + executable Harnesscurrent Harness behavior, qualification, verified/provisional evidence routingtask–Harness co-evolution and executable Harness revisionprimary six-round condition
MemoHarnessHarness on fixed tasksfixed task outcomes / memo experienceHarness-only evolution without a new task curriculumsix-round control
RHIHarness prompt loopself-reflection / loop feedbackprompt-level Harness-loop refinementsix-round reproduction
RSEAnatural-language state and cross-generation rewritingstate summary and previous-generation statenatural-language state rewritestatic protocol comparison; no confirmed official runnable repository
Self-HarnessHarness proposalweakness mining and proposal validationweakness mining → Harness proposal → validationsix-round reproduction

The numerical table shows the unified six-round comparison, while the static table identifies whether each method places its update budget in task generation, memo, prompt-loop, state-rewrite, or proposal-validation pathways; together they connect performance with mechanism.

Q1: Unreliable self-generated supervision and recursive amplification

Reviewer question (verbatim):

The paper states that unreliable self-generated supervision can be amplified recursively. Are there ablation experiments to support this claim?

Response: Our design treats unreliable self-generated supervision as a failure mechanism to be managed by qualification and routing, and the audit below traces how that information propagates under our protocol. Prior work on debiased self-training, Noisy Student, recursively generated data, Re-ReST, SCoRe, and STaR jointly motivates this design and informs our qualification, verified-bank, and provisional-routing choices.

The supporting literature is Debiased Self-Training, Noisy Student design choices, AI Models Collapse, Re-ReST, SCoRe, and STaR.

We completed a mechanism audit that places Ours, MemoHarness, and an invalid-only stress condition in the comparison. Invalid-only keeps one executable invalid record as labeled downstream retrieval input, allowing us to test whether an unreliable record changes the later Harness while retaining the same execution route.

Conditionavg@5pass@5cons@5
Ours, full qualification and provisional routing52.34% [45.97, 58.64]78.72% [65.10, 88.01]59.57% [45.34, 72.36]
MemoHarness, six rounds50.21% [43.87, 56.55]70.21% [56.02, 81.35]55.32% [41.25, 68.59]
Ours, invalid-only stress47.23% [40.95, 53.61]65.96% [51.67, 77.83]48.94% [35.28, 62.76]

The invalid-only round-level static analysis is:

StageOurs: Harness modificationsInvalid-only: Harness modification and inputMemoHarness: main modificationsInvalid-only downstream evidence and outcome
Rounds 1–3Adds type identification, up to three-experience filtering, answer extraction, restricted calculation checks, and a classify → derive → expression-verify flow.Uses the same type identification, three-experience filtering, answer extraction, restricted calculation, and classify → derive → expression-verify steps. It additionally reads an executable invalid trace with an invalid label and sets learnable=false, replay_eligible=false.Updates memo/retrieval context, experience prompts, and solver flow on fixed tasks; it does not generate a next-round task curriculum.Invalid counts are 1, 2, and 2 in r1–r3; the invalid trace is readable downstream but forms no global pattern, module record, or parent chain.
Rounds 4–5Adds short experience context, multi-level extraction, second-pass verification, answer normalization, and explicit enumeration for formatting failures, incomplete verification, forbidden edges, gcd/divisor cases, and standard geometry.Uses the short experience context, multi-level FINAL ANSWER/boxed extraction, second-pass verification, normalization, and explicit enumeration. It additionally reads executable invalid traces from r4–r5 with their invalid labels, without allowing them into global patterns, module records, or parent construction.Continues fixed-task memo, prompt, and solver-flow updates without a failure-type → next-task path.The invalid trace is retrieved 8 and 7 times in r4 and r5; it forms neither a global pattern nor a module record and is not used as a new parent.
Round 6Uses type-specific prompts, at most two experiences, format re-extraction, and restricted verification/recomputation.Uses the type-specific prompt, two-experience filtering, format re-extraction, restricted verification, and recomputation steps. It additionally reads executable invalid trace with its invalid label, without allowing it into qualification, promotion, global patterns, module records, or parent construction.Remains a fixed-task Harness-only update.The invalid trace is retrieved 7 more times and enters no global pattern, module record, or parent chain; invalid-only avg@5 is 47.23%.

This table records Harness modifications by stage and the invalid-only input-path change: executable invalid traces remain readable with their labels, while all other records retain their original qualification state. Code modification and downstream visibility are therefore shown on the same evidence chain.

This round-level audit places failure type, Harness modification, invalid-trace retrieval, and held-out outcome on one evidence chain, showing how invalid information enters later visible context and the measured extent of its propagation.

Reviewer_amSUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GrokRSI alternates task generation and executable harness revision, using public agent behavior to shape future tasks and separating verified learning outcomes from provisional observations. With Claude Haiku 4.5, it reports higher held-out point estimates than protocol-matched MemoHarness on pooled AIME and HMMT, plus a complementary GPQA-Diamond study and AIME mechanism ablations. The protocol provides an explicit account of selection and information flow, but the paired AIME and HMMT analyses do not establish statistically reliable gains and the central qualification mechanism needs stronger validation.

Critical strengths
  • The paper focused on an important problem. Recursive agent improvement faces two linked problems: a fixed few-shot task set can provide too little new learning signal, and self-generated tasks or answers can introduce errors that later updates amplify. Studying task generation together with harness revision is therefore useful, particularly when the model weights remain fixed.

  • The paper proposed behavior-conditioned task generation, qualification and separate evidence stores, and executable harness revision with frozen evaluation roles, which are useful for qualified task-harness co-evolution.

  • Against MemoHarness using the same Claude Haiku 4.5 solver, split, and five-sample evaluation schedule, Table 1 reports AIME avg@5 of 72.73 versus 69.09 (+3.64 percentage points) and HMMT avg@5 of 52.34 versus 50.21 (+2.13 points).

Critical weaknesses
  • The gains are small relative to uncertainty. For AIME, Table A4 reports a 3.64-point avg-at-5 gain with a 95 percent interval from -1.36 to 9.09 and paired p = 0.250; the pass-at-5 and consistency tests are also nonsignificant. The authors appropriately say point estimates, but these results do not establish a repeatable benefit across learning runs or splits.

  • Section 4.6 freezes one final harness and bank, but Table A3 labels a 220-call held-out stage Adaptation. These can be compatible if the stage only assembles per-question context without modifying the frozen executable or shared state, yet the paper does not make that distinction explicit. Similarly, the blanket wording that validation outcomes cannot enter later updates is stronger than the actual procedure, where validation selects the active harness used in the next round. Please distinguish raw episode/answer isolation from the unavoidable influence of selection.

  • Provisional evidence can still influence the next harness. Eq 3 allows the provisional store to condition distillation, and Eq 4 gives that store directly to the harness controller. Separating records therefore does not prevent an unreliable provisional answer or trace from influencing learned patterns and code indirectly. This may be a useful design choice, but its safety/effectiveness has not been isolated.

  • Evidence for portability and longer recursion is narrow. The primary experiments use one solver, two mathematical benchmark families, a fixed split, and six learning rounds; GPQA changes question presentation within a fixed-answer question-answering setting. The biology audit checks interface portability rather than agent improvement. Please test another solver, a genuinely interactive or open-ended domain, and longer recursive runs with stability and validator-cost measurements.

Questions for authors

NA

5 comments
AuthorsOpenReview ↗

Response to Reviewer amSU (W1–W4)

We thank Reviewer amSU for the detailed review. We report matched-protocol results, mechanism audits, and evidence scope below. Intervals are descriptive 95% Wilson intervals from the reported counts and are shown beside each point estimate.

W1: Repeatable benefit across learning runs or splits

Reviewer comment (verbatim):

The gains are small relative to uncertainty. For AIME, Table A4 reports a 3.64-point avg-at-5 gain with a 95 percent interval from -1.36 to 9.09 and paired p = 0.250; the pass-at-5 and consistency tests are also nonsignificant. The authors appropriately say point estimates, but these results do not establish a repeatable benefit across learning runs or splits.

Response: The original AIME comparison gives +3.64 avg@5 points (95% paired interval [-1.36, 9.09], paired p = 0.250). Accepted Agent/workflow-search papers likewise report baseline-relative point estimates with repeated runs or intervals: RISE (NeurIPS 2024) reports self-improvement estimates; Promptbreeder (ICML 2024) reports 99.7 vs 99.4 on MultiArith; AFlow (ICLR 2025) reports mean estimates across workflow-search runs; and ADAS (ICLR 2025) reports bootstrap intervals with higher estimates. We therefore show each estimate with its interval; alternative splits show the observed range under fixed evolution.

Different task splits

Benchmark / task splitMethodavg@5pass@5cons@5
AIME alternate split AOurs80.00% [74.22, 84.75]90.91% [78.84, 96.41]88.64% [76.02, 95.05]
AIME alternate split AMemoHarness75.00% [68.88, 80.26]90.91% [78.84, 96.41]86.36% [73.29, 93.60]
AIME alternate split BOurs70.45% [64.12, 76.09]88.64% [76.02, 95.05]75.00% [60.56, 85.43]
AIME alternate split BMemoHarness70.00% [63.64, 75.67]86.36% [73.29, 93.60]75.00% [60.56, 85.43]
AIME original splitOurs72.73% [66.49, 78.18]88.64% [76.02, 95.05]81.82% [68.04, 90.49]
AIME original splitMemoHarness69.09% [62.70, 74.83]84.09% [70.63, 92.07]75.00% [60.56, 85.43]
HMMT alternate split AOurs51.91% [45.55, 58.22]68.09% [53.83, 79.60]53.19% [39.23, 66.67]
HMMT alternate split AMemoHarness47.23% [40.95, 53.61]65.96% [51.67, 77.83]48.94% [35.28, 62.76]
HMMT alternate split BOurs52.77% [46.39, 59.05]72.34% [58.24, 83.06]59.57% [45.34, 72.36]
HMMT alternate split BMemoHarness50.64% [44.29, 56.97]68.09% [53.83, 79.60]55.32% [41.25, 68.59]
HMMT original splitOurs52.34% [45.97, 58.64]78.72% [65.10, 88.01]59.57% [45.34, 72.36]
HMMT original splitMemoHarness50.21% [43.87, 56.55]70.21% [56.02, 81.35]55.32% [41.25, 68.59]

Across both HMMT alternative splits, Ours has higher point estimates than MemoHarness; AIME is likewise positive. Each estimate includes its interval.

HMMT and AIME: additional complete learning runs under the same task split

We reran Ours only for six rounds under the original HMMT 4/12/47 split. MemoHarness reuses the recorded endpoint as the fixed matched reference and is not rerun.

Benchmark / task splitMethodRun statusavg@5pass@5cons@5
HMMT original splitOurscomplete Ours rerun50.21% [38.72, 61.28]76.60% [63.83, 87.23]55.32% [40.43, 70.21]
HMMT original splitMemoHarnessfixed matched reference50.21% [43.87, 56.55]70.21% [56.02, 81.35]55.32% [41.25, 68.59]

AIME: additional complete learning run under the same task split

Likewise, we reran Ours only for six rounds under the original AIME 4/12/44 split. MemoHarness reuses the recorded endpoint as the fixed matched reference and is not rerun.

Benchmark / task splitMethodRun statusavg@5pass@5cons@5
AIME original splitOurscomplete Ours rerun70.00% [63.64, 75.67]84.09% [70.63, 92.07]77.27% [63.01, 87.16]
AIME original splitMemoHarnessfixed matched reference69.09% [62.70, 74.83]84.09% [70.63, 92.07]75.00% [60.56, 85.43]

Under this AIME split, Ours has avg@5 70.00% versus 69.09% for MemoHarness, identical pass@5, and +2.27 cons@5 points.

W2: Frozen Harness, Adaptation, raw episode/answer isolation, and selection**

Reviewer comment (verbatim):

Section 4.6 freezes one final harness and bank, but Table A3 labels a 220-call held-out stage Adaptation. These can be compatible if the stage only assembles per-question context without modifying the frozen executable or shared state, yet the paper does not make that distinction explicit. Similarly, the blanket wording that validation outcomes cannot enter later updates is stronger than the actual procedure, where validation selects the active harness used in the next round. Please distinguish raw episode/answer isolation from the unavoidable influence of selection.

Response: Table A3 uses Adaptation for the read-only context-assembly stage in the 220-call held-out stage: after the final Harness and bank are frozen, it only assembles per-question input context and calls the frozen executable Harness for each held-out/test question. It does not modify the executable program, shared bank, distillation state, task-generation state, candidate pool, or selection state. The validation and held-out/test sets are disjoint: validation is used for candidate selection during learning, whereas held-out/test is used only for final evaluation.

During learning, validation outcomes influence subsequent computation through selection, which chooses the active Harness for the next round. Validation answers and raw episodes are not written into the bank, and raw episode/answer isolation is preserved. Once the final Harness and bank are frozen, held-out episodes, answers, trajectories, and final test results remain read-only and have no write-back path. The precise boundary is:

StageOperationModifies frozen executable/shared state?Can select next active Harness?Writes held-out answers back?
Held-out Adaptation (220 calls)Assemble per-question context and call the final HarnessNoNoNo
Validation selection (learning stage)Use validation outcomes to select a candidate HarnessNo direct write-back to the Harness/bankYesNo

This protocol table separates raw episode/answer isolation during final evaluation from the influence of the selection path during learning.

W3: Provisional evidence, learned patterns and code, and safety/effectiveness

Reviewer comment (verbatim):

Provisional evidence can still influence the next harness. Eq 3 allows the provisional store to condition distillation, and Eq 4 gives that store directly to the harness controller. Separating records therefore does not prevent an unreliable provisional answer or trace from influencing learned patterns and code indirectly. This may be a useful design choice, but its safety/effectiveness has not been isolated.

Response: Our protocol separates record provenance from information availability: provisional evidence is separate from the verified bank, while Eq. 3 and Eq. 4 let it condition distillation and the Harness controller. An unreliable provisional answer or trace can therefore serve as a labeled diagnostic signal for learned patterns and code.

Each provisional record is marked uncertain and routed to diagnostic context; the verified bank, selection pool, and parent pool remain reserved for validated supervision. Distillation and the controller can identify failure modes, while promotion remains controlled by verified supervision.

The completed C2 isolation run holds qualification fixed and disables provisional-store input to distillation/controller:

Conditionavg@5pass@5cons@5
Ours52.34% [45.97, 58.64]78.72% [65.10, 88.01]59.57% [45.34, 72.36]
Provisional-off47.66% [41.36, 54.03]65.96% [51.67, 77.83]51.06% [37.24, 64.72]
MemoHarness50.21% [43.87, 56.55]70.21% [56.02, 81.35]55.32% [41.25, 68.59]

Ours and MemoHarness show the original-configuration reference metrics; Provisional-off shows the observed C2 mechanism-audit values. It is a controlled ablation of Ours’ provisional-store input path.

The static analysis records round-level Harness changes, execution traces, and record routing:

Round/stageOurs: exposed issue and modificationProvisional-off: modification and inputMemoHarness: corresponding modificationLater selection/held-out behavior
r1Uniqueness, sign branches, rotation/reflection counts, graph enumeration; adds extraction, second-pass verification, restricted checksSame extraction/verification/checks; controller cannot see provisional observationsFixed-task outcomes update memo/retrieval/solver context; no feedback-driven curriculumCandidate 6/12 vs H0 7/12; rejected
r2Degree-2 bottleneck, explicit branches, new parameters; qualification reports incomplete_validation_evidenceSame repairs and evidence requirement; no provisional observationsFixed-task memo organization and solver prompts continue; no generated-task traceIncomplete validation evidence; rejected
r3Up to three experiences, extraction, restricted verification, classify → derive → expression-verifySame flow, with controller blind to provisional observationsMemo retrieval and solver-flow updates; task curriculum fixedStrongest candidate 2/12 vs H0 7/12
r4–r5Forbidden edges, unique branches, gcd/divisor enumeration, geometry; adds short context, multi-level FINAL ANSWER/boxed extraction, second-pass verification, normalization, enumerationSame repairs; provisional-store input disabledFixed-task memo/prompt updates; no failure-type → next-task pathCandidates tie or fall below current Harness; diagnostic held-out avg@5 42.98%–45.11%
r6At most two type-matched experiences, type-specific prompts, malformed-answer re-extraction, restricted checks/recomputationSame type-matched repairs; controller cannot read provisional observationsFinal fixed-task Harness-only endpoint without provisional routingOurs 8/12 validation and promoted; held-out 52.34%, Provisional-off 47.66%, MemoHarness 50.21%

Ours and Provisional-off share the main code modifications; the controlled difference is whether provisional-store information reaches distillation and the controller. MemoHarness changes remain focused on memo/retrieval and solver flow on fixed tasks. Labeled uncertain traces provide diagnostic input, while promotion still requires verified supervision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Comment

We thank the Area Chair and reviewers for their feedback.

ReviewerInitial ratingStrengths notedMain concernsAdditional evidence
amSU2Few-shot learning problem and behavior-conditioned task generationRepeatability, frozen evaluation, provisional evidence, portabilityLearning reruns, alternative splits, routing ablation, repeated extension runs and validator costs
Nq512Joint evolution of tasks and HarnessesInteractive planning, related baselines, unreliable supervision, recursion, reproducibilityALFWorld, RHI/Self-Harness comparisons, invalid-only audit, source-code lineage and anonymous artifacts
gmeH2Task-role separation, qualification and evidence routingStatistical uncertainty, procedure variation, current-Harness feedbackSplit comparisons, generation/qualification/selection traces and qualified-task-matched feedback control

Reviewer Nq51 acknowledged that ALFWorld, six-round RHI/Self-Harness reproductions and added citations address coverage and comparison. Our follow-up adds source-code, checkpoint and result-row audits.

Overall Discussion

GrokRSI uses current-Harness behavior to generate tasks, then uses qualified outcomes to revise the executable Harness. The new evidence tests this feedback mechanism across task partitions, learning runs and interactive execution.

Current-Harness feedback. On HMMT, removing the current Harness's observations and patterns from the generator gives 48.51% [42.20, 54.87] avg@5, compared with 52.34% [45.97, 58.64] for GrokRSI. The control preserves the solver, split, six-round schedule, generation budget and per-round qualification counts. The observed differences are +3.83 avg@5, +8.51 pass@5 and +4.26 cons@5 percentage points.

Task partitions and learning reruns. GrokRSI has a higher avg@5 point estimate than MemoHarness in all six comparisons across AIME and HMMT, each using the original partition and two alternatives. Complete GrokRSI reruns use the original splits and recorded MemoHarness references. We report metric intervals and generation, qualification and Harness-selection traces. For the original AIME comparison, the +3.64-point avg@5 difference has a 95% paired interval of [−1.36, 9.09] points.

Related baselines and interactive execution. The table gives avg@5 with 95% intervals. HMMT uses 47 held-out problems and ALFWorld uses 60, with five attempts per problem. All rows use six learning rounds.

BenchmarkMethodavg@5 (95% CI)
HMMTGrokRSI52.34% [45.97, 58.64]
HMMTMemoHarness50.21% [43.87, 56.55]
HMMTRHI, third-party reproduction51.06% [44.71, 57.39]
HMMTSelf-Harness48.94% [42.61, 55.29]
ALFWorldGrokRSI, Run A75.33% [70.16, 79.87]
ALFWorldGrokRSI, Run B71.00% [65.63, 75.84]
ALFWorldMemoHarness66.00% [60.47, 71.13]

ALFWorld uses a 6/12/60 training/validation/test split and requires repeated actions, observations and replanning. We report two six-round Sonnet 4.6 runs and longer runs, with metric ranges and validator calls/tokens. RSEA is covered through a comparison of its published protocol.

Executable evolution. In the original HMMT run, H0 remained active through round 5, while the promoted program's source lineage was H0 → H4-1 → H5-2 → H6-3. These revisions moved from a second verification call after every solve to conditional answer extraction and then type-specific experience selection and solver prompts. The anonymous records connect the parent-to-child code changes to validation and promotion. The follow-up also reports round-6 and round-10 checkpoint evaluations within one ten-round run.

Evidence routing and evaluation isolation. Disabling provisional input gives 47.66% [41.36, 54.03] avg@5 on HMMT. The invalid-only condition gives 47.23% [40.95, 53.61]. Recomputation confirms identical aggregate metrics for invalid-only and MemoHarness on alternate split A, with different correct-attempt counts on 15 of 43 shared problems. Validation selects the next active Harness, while raw validation answers and episodes remain outside the learning stores. Held-out context assembly uses a frozen executable and bank.

Anonymous materials and manuscript revisions. The repository now provides 19 Harness sources and configurations, 92 saved generation inputs, prompt-construction code, six-round trace records and the Appendix Q biology records. Its round-6 walkthrough combines source diffs, paired validation outcomes, promotion and held-out results. Anonymous supplementary materials

We revised Appendix Q, added RHI/RSEA/Self-Harness citations and corrected the AIME/HMMT labels in Figure A3. Blue marks revised wording and red marks added content in the manuscript.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.