acceptodds
Under review as a conference paper at ICLR 2027

Compositional Recursive Self-Improvement

Abstract

Recursive self-improvement is becoming an important direction for building agents that can learn from their own experience and improve their future behavior. Yet current RSI systems do not clearly expose which parts of a successful harness should be retained, how compatible discoveries from different executions should be combined, or how the value of one control changes with its collaborators. Whole-harness updates couple these decisions and make reusable improvement units difficult to preserve. We address this gap with , a compositional harness-adaptation method for RSI systems. retains typed harness modules while evaluating the complete assembled harness: base inheritance preserves unchanged controls, guarded assembly combines compatible discoveries, and cooperation memory informs later task-conditioned composition. Across multiple domains, datasets, and baseline systems, improves transfer performance and rubric-scored analysis quality while producing traceable workflow and artifact evidence. The method provides a compositional interface for retaining and reusing control improvements in recursive self-improvement systems.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent System, Recursive Self-Improvement, Large Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cTHoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces compositional recursive self-improvement. By breaking the harness into six distinct parts, it constrains each improvement down to the smallest possible unit: the module and its corresponding fields. The authors argue that this approach effectively reuses local modifications from previous recursive improvement steps and harnesses positive synergies between different modules. They also conduct comprehensive experiments to demonstrate that the proposed method significantly boosts RSI performance.

Critical strengths
  1. the motivation and intuition behind this approach are very clear. It modularizes and abstracts modifications to the RSI, making it easier to combine changes across different modules and achieve synergy. That motivation and idea are completely clear to me.
  2. the experiments in the paper are comprehensive, evaluating tasks across many different fields and showing significant improvements across multiple tasks. There are also extensive ablation studies, which help us understand the role of different components.
Critical weaknesses
  1. although the design is clean, I haven't seen clear evidence or examples in experiments showing that combining different components and module modifications achieves a true 1+1 > 2 effect. That positive synergy just hasn't come through.
  2. I think this method comes with a trade-off. By restricting the search space to six preset dimensions and their corresponding fields, along with a fixed workflow, it could severely cap performance potential. Since we're likely focused on long-term, sustained gains with RSI, this setup might let you search faster and reach a decent solution more quickly, but in the long run, it ultimately limits performance.
Questions for authors
  1. Could the authors clarify or provide evidence regarding this compositional idea? Specifically, in the actual experiments, did they observe scenarios where the combined modification of two modules exceeded the sum of modifying each module individually, or are there any corresponding ablation experiments?
  2. Could the authors discuss further the trade-offs introduced by constraining the search space? Personally, I feel this might impact the performance limit. However, in actual experiments, will it actually affect things or hit that limit? I don't really have a good sense of this myself, so I hope the authors can elaborate more on this issue.
  3. Although the experiments cover a lot of benchmarks, the scale of some experiments is still quite small, and the sample size is limited. For example, in CMT-bench, each partition has only five tasks, so a single task's result can shift accuracy by around twenty percentage points. I feel this has a huge impact on the interpretation of the results. Can the authors provide some further clarification on this issue?
  4. While compositional RSI is the primary design, the actual algorithm also incorporates a fixed multi-agent workflow. To what extent does the performance boost come from compositional RSI, and how much is instead driven by this multi-agent workflow?
Reviewer_EWmYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Compositional Recursive Self-Improvement (CoRSI), a framework for continual adaptation through modification and reuse of an external agent harness without updating the parameters of the underlying model. The harness is represented along six editable dimensions: Context, Tools, Generation, Orchestration, Memory, and Verification. Rather than storing only complete harnesses, CoRSI extracts field-level typed patches from evaluated configuration changes and retains them in a module catalog. It also records pairwise co-occurrences of modules in successful episodes. For each task, a Planner recalls candidate modules, assigns them to participants, and specifies the artifact-dependency graph and execution order before the complete workflow is evaluated.

The experiments study frozen-bank transfer across tasks, categories, competition years, and model families on competition mathematics, GPQA-Diamond, CMT-Bench, and BiomniBench-DA. The problem is practically relevant, and the framework contains several valuable design ideas, particularly the distinction between the local unit of modification and the complete workflow that receives the evaluation signal, as well as the preservation and routing of executable artifacts in tool-using tasks. However, the paper does not sufficiently specify the implementation of Recall, the Controller, or the update-admission mechanism. It also does not provide sufficient evidence that a small number of bank-update rounds yields stable recursive improvement. My current assessment is therefore below the acceptance threshold.

Critical strengths
  1. The problem is important. Enabling an agent harness to accumulate experience and adapt to new tasks while keeping the underlying model fixed is practically valuable. Treating field-level changes as reusable objects while evaluating complete workflows is a reasonable systems principle.

  2. The experimental scope is broad. The paper covers competition mathematics, scientific multiple-choice questions, condensed-matter problems, and tool-driven biomedical data analysis. The BiomniBench-DA cases are particularly useful because they involve scripts, tables, analysis traces, workspace artifacts, and subsequent verification rather than only prompt optimization.

  3. The execution records and case studies are relatively detailed. The appendix provides patch provenance, inherited configuration values, participant groupings, artifact routes, and selected call records. The AIME/HMMT cases illustrate how multiple derivations are preserved and merged, while the Biomni cases show how intermediate analysis products can be checked by later participants.

  4. Several results are promising. CoRSI outperforms MemoHarness in multiple reported held-out settings. On the 20 held-out BiomniBench-DA tasks, its mean rubric score is 73.20, compared with 68.55 for MemoHarness. All 20 CoRSI runs receive nonzero scores, and none scores below 40, suggesting that the benefit is not limited to a change in the mean. Although the current paper does not yet isolate the source of these gains, the field-level representation and explicit workflow interface may still have practical value.

  5. Some of the principal results are reasonably consistent. GPQA-Diamond contains 100 held-out questions, and CoRSI outperforms MemoHarness in all six reported model settings. The mathematics evaluation covers AIME 2025, HMMT 2025, and HMMT 2026 across both GPT and Claude model families. On GPQA, the complete system also outperforms all four reported ablations. These results do not resolve the mechanism-attribution concerns below, but they provide stronger evidence than a single improvement on one small test set.

Critical weaknesses
  1. The limited and non-monotonic training trajectory does not support a strong recursive-self-improvement claim

Bank construction is run for only five rounds on GPQA, mathematics, and Biomni, and for ten rounds on CMT-Bench. In Biomni, each round executes the same panel of 30 adaptation tasks, yielding 150 completed episodes over five rounds, but only five outer retained-state updates are observed. The full-rubric success rate—the fraction of the 30 tasks receiving a rubric score of exactly 100—is 6.7%, 13.3%, 16.7%, 13.3%, and 20.0% across the five rounds, corresponding to approximately 2, 4, 5, 4, and 6 full-credit tasks. The corresponding mean training scores, including zeros for incomplete tasks, are 67.37, 61.10, 52.43, 65.63, and 62.13. Although Round 5 produces the largest number of full-credit episodes, its mean score is below that of Round 1 and slightly below the Round-5 MemoHarness score.

These results show that the bank and reusable module collection grow, and that the final checkpoint improves some metrics. However, they are insufficient to establish whether the improvement loop can sustain net gains over longer horizons, how or when its performance stabilizes, or how regressions are controlled.

The paper does include admission mechanisms. Algorithm 1 states that a Controller proposal must pass an evaluator or canary gate, while Algorithm 2 performs guarded admission based on conflicts, dependencies, and compatibility. The latter is specified deterministically, but the former is not: the paper does not provide the benchmark-specific evaluation data, acceptance rule, or minimum improvement threshold, nor does it explain how rejected proposals are handled or how checkpoints are retained, selected, or rolled back when performance deteriorates.

There is also an apparent protocol inconsistency. Section 4.6 states that the Biomni training panel is run for ten rounds, whereas Figure 7 reports only Rounds 1–5, Appendix D.2 explicitly describes 150 episodes from 30 tasks over five rounds, and Appendix D.3.1 likewise defines a single five-round training run. The paper does not state that only the first five of ten rounds are shown.

  1. The initialization, generation, and selection of modules are not specified well enough for reproduction

The paper describes module updating as extracting typed patches from episodes. Formally, however, Extract compares the evaluated configuration HqevalH_q^{\mathrm{eval}} with its predecessor HqprevH_q^{\mathrm{prev}} and stores the changed configuration fields as modules. It does not derive a new prompt or strategy from the reasoning trajectory itself; the new field value must already have been proposed by the Controller, supplied as an initial seed, or produced by another candidate-generation mechanism.

The AIME case study shows a Context instruction and token-limit patch originating in Round 1 and a temperature patch originating in Round 2, but it does not explain how the Round-1 candidates were generated. The initial contents of B0B_0, M0M_0, and C0C_0 are also not specified. If both B0B_0 and M0M_0 are empty, the first-round Top-K recall over Mt∪BtM_t \cup B_t has no candidates. Figure 9 further notes that the Controller’s proposal-generation rule is unspecified. Consequently, the process that creates the semantic content of new modules is not reproducible from the paper.

Even assuming that candidate modules have already been generated, their selection is also underspecified. The recall score is defined as a weighted combination of task fit, base-harness fit, cooperation evidence, and compatibility, but the paper does not provide benchmark-specific definitions of these components, their weights, or the candidate-pool sizes. The paper therefore lacks executable definitions for two central stages: how candidate modifications are generated and how they are selected. This makes it difficult to determine whether the reported gains arise from a general compositional-learning mechanism or from unreported Controller prompts, Recall implementations, and other engineering choices.

  1. The practical efficiency and long-term scalability of per-task dynamic composition are not established

During held-out evaluation, CoRSI freezes the contents of BB, MM, and CC, but it does not freeze a single harness that can be executed repeatedly. According to Algorithm 1, every new task requires candidate scoring, Top-K recall, Planner compilation, participant materialization, and workflow execution. The deployment pattern is therefore not “learn one harness and reuse it,” but “retrieve and compile a task-specific workflow for every input.” Per-task retrieval is not inherently unreasonable in a task-conditioned system, but it makes retrieval, planning, and composition recurring online costs for every request.

This concern becomes increasingly important as the number of recursive rounds and retained modules grows. Recall is defined by scoring candidates from Mt∪BtM_t \cup B_t and selecting the Top-K. The value KtK_t bounds the number of candidates ultimately exposed to the Planner, but it does not by itself bound the cost of finding those candidates in a continually growing bank and module catalog. The paper does not provide retrieval complexity or describe the use of indexing, pre-filtering, approximate retrieval, candidate deduplication, or workflow caching. It also does not report aggregate per-task Planner, participant, and synthesis calls, token consumption, wall-clock latency, or monetary cost.

The available case studies indicate that the online execution structure can be substantial. For example, HMMT task 3 uses a Planner call, three participant calls, and a final merge. Without systematic efficiency measurements, the existing results do not establish the practical performance–cost trade-off of per-task dynamic composition or whether the approach remains usable with more recursive rounds, larger module catalogs, and large-scale task streams.

  1. BiomniBench-DA lacks component ablations for the mechanisms most relevant to tool-using workflows

The paper does include a meaningful range of ablations. On GPQA, the complete system outperforms four ablated conditions, and the AIME/HMMT experiments examine cooperation memory, cross-source recombination, multi-participant workflows, artifact handoff, and synthesis. Therefore, the paper should not be characterized as lacking ablations in general.

However, these systematic ablations are concentrated on GPQA and mathematics. BiomniBench-DA is the setting in which Tools, Memory, Verification, artifact routing, and multi-participant collaboration should matter most, yet this section primarily reports aggregate scores, round-wise checkpoints, rubric decompositions, and case studies rather than corresponding component ablations. The cases demonstrate how the system can work, but they do not establish whether these mechanisms are individually necessary for the biomedical results. Ablating at least one of cooperation memory, module-guided workflow construction, artifact handoff, or multi-participant execution would substantially strengthen the mechanism attribution in this setting.

Questions for authors
  1. What are the complete initialization procedures for B0B_0, M0M_0, and C0C_0? Are first-round retrievable modules manually seeded, extracted from H0H_0, or generated by the Controller before execution? How does Extract decide whether a configuration difference should become one typed patch or several?

  2. What algorithm and complete prompt does the Controller use to generate new patches? How are stasks_{\mathrm{task}}, sbases_{\mathrm{base}}, scoops_{\mathrm{coop}}, and scompats_{\mathrm{compat}}, and their weights, instantiated for each benchmark? What data and acceptance criteria are used by the evaluator/canary gate, and how are rejected proposals or subsequent performance regressions handled? Please provide one complete trace from a Controller proposal through evaluation and delta extraction to recall in the next round.

  3. Was the Biomni training protocol run for five rounds or ten? Given that both the round-wise mean score and full-rubric success rate are non-monotonic, how is the “improved” checkpoint defined and selected? Could the authors provide a longer training trajectory, or otherwise narrow the progressive/recursive-improvement claim?

  4. Could the authors ablate key mechanisms—such as cooperation memory, module-guided workflow construction, artifact handoff, or multi-participant execution—on BiomniBench-DA to test whether the mechanism-level conclusions from mathematics and GPQA extend to tool-using biomedical analysis?

Reviewer_JhHjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoRSI, a compositional framework for recursive self-improvement of agent harnesses. The method decomposes harness configurations into reusable, typed modules and combines compatible modifications from previous executions, guided by task-conditioned retrieval and cooperation memory. The authors evaluate CoRSI across mathematical reasoning, scientific question answering, cross-model transfer, and biomedical data analysis. The results demonstrate improvements over existing baselines, supported by ablation studies and detailed execution case studies.

Critical strengths
  • Practical and potentially useful framework. CoRSI introduces an explicit mechanism for retaining and recombining fine-grained harness modifications across previous executions. This provides a flexible and traceable approach to experience-based harness adaptation.

  • Broad empirical evaluation. The paper evaluates the proposed method across multiple reasoning and scientific domains, reporting promising improvements over existing baselines. The ablation studies and detailed execution traces further strengthen the empirical contribution.

Critical weaknesses
  • Limited methodological novelty. Several key components, including experience-based harness adaptation, modular workflow optimization, and cooperation-aware agent selection, are closely related to existing approaches. While cross-source recombination of fine-grained harness modifications is a potentially meaningful contribution, the overall framework appears to be an incremental extension of prior methods rather than a substantially new algorithmic approach.

  • Insufficient isolation of the core contribution. The reported improvements may partly result from multi-agent execution, additional verification, and answer synthesis rather than the proposed compositional reuse mechanism itself. In particular, the ablation evidence specifically supporting cross-source field-level recombination is relatively limited. Without more controlled comparisons that account for workflow structure and actual inference computation, it is difficult to determine how much of the improvement can be attributed to the paper's most distinctive contribution.

Questions for authors
  • Can the authors provide additional evidence quantifying the benefit of cross-source field-level recombination independently of other workflow mechanisms? In particular, how much performance improvement remains when cross-source recombination is disabled while preserving the same workflow structure and other components?

  • Although the paper reports aligned task-level resource budgets, some examples involve substantially different numbers of actual model calls between CoRSI and the baselines. Could the authors clarify the actual inference costs (e.g., model calls, tokens, and latency) and whether the improvements persist under matched computation and comparable multi-agent workflows?

1 comment

Thank you for the detailed rebuttal and additional experiments. The empty-bank control, cross-source recombination ablation, and inference cost comparisons meaningfully address my concerns about mechanism attribution. I appreciate these efforts and consider this concern substantially alleviated.

However, some reservations remain. First, the gains attributable to compositional reuse are relatively modest (4–5 percentage points), and stronger statistical evidence would help establish their robustness. Second, while the additional baseline comparisons are useful, the methodological novelty still appears somewhat incremental, and matching evolution rounds does not necessarily ensure comparable search budgets. Finally, the discrepancies between the original algorithm description and the actual implementation raise concerns about method specification and reproducibility.

Overall, the rebuttal strengthens the empirical contribution, but the remaining concerns prevent me from changing my overall assessment at this stage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.