- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
RSI-Env provides a persistent environment where agents modify model weights and executable harnesses, retain experiment records, and pass updated systems to subsequent iterations. The study compares Self and Teacher configurations across three guidance regimes and evaluates 14 resulting systems on four benchmarks. Nine systems improve on at least one benchmark, while all regress on at least one. Trajectory analysis examines capability validation, update selection, and retention of local gains.
Critical strengths
- The environment integrates model and harness updates with persistent state and controller-managed evaluation, providing a useful experimental platform.
- Evaluation across all four benchmarks exposes regressions beyond the optimization target.
- Concrete trajectories and transparent reporting of negative results help identify practical obstacles to self-improvement.
Critical weaknesses
-
The related-work discussion omits several directly relevant approaches: Gödel Agent, which recursively modifies agent logic and its improvement procedure; Hyperagents, which makes task and meta agents editable; STOP, which applies a code improver to itself; and Huxley-Gödel Machine, which uses descendant performance to guide self-modification. These omissions make the incremental contribution difficult to assess. The paper should compare editable components, feedback access, and successor handoff, and explain what research questions RSI-Env enables beyond these systems.
-
The 67 harness updates yield limited reusable guidance for harness design. Appendix C describes compaction recovery, bounded retries, output filtering, and answer finalization, but does not systematically connect these changes to capability gains, costs, or regressions. For example, AIME Self combines several changes, so its endpoint gains cannot establish which intervention helped. Representative code changes and component ablations would turn the artifact descriptions into more useful, testable design lessons.
-
Table 2 reports one trial per item at temperature 1.0, without reported independent repeats of the evolution process or uncertainty estimates. The 13.3-point AIME gain represents four questions, and the local 9/13 to 10/13 to 9/13 sequence does not establish capability loss. Unequal budgets, unavailable feedback, and runtime failures further limit attribution to agent research decisions. Incomplete logs also make audit counts lower bounds, as Appendix A.4.2 acknowledges. The findings are informative about these runs, but broader or causal conclusions require stronger evidence.
Questions for authors
- What distinguishes RSI-Env from the omitted systems in terms of editable improvement procedures and experimental capabilities?
- Could three representative harness interventions be evaluated separately on fresh tasks, reporting accuracy, interaction or token cost, and regressions?
- Could key settings be repeated with matched budgets and reliable evaluation services, with uncertainty estimates and separate accounting for infrastructure failures?
5 comments
Response to Reviewer 2F5h (2/3)
Weakness 2. The 67 harness updates yield limited reusable guidance for harness design. Appendix C describes compaction recovery, bounded retries, output filtering, and answer finalization, but does not systematically connect these changes to capability gains, costs, or regressions. For example, AIME Self combines several changes, so its endpoint gains cannot establish which intervention helped. Representative code changes and component ablations would turn the artifact descriptions into more useful, testable design lessons. Question 2. Could three representative harness interventions be evaluated separately on fresh tasks, reporting accuracy, interaction or token cost, and regressions?
Response. The three most representative harness modifications identified from the recorded trajectories are as follows.
-
Evidence-guided knowledge workflow
- Failure mode. The initial harness specified tool-use and completion protocols but did not guide the agent in defining the scope of a knowledge question, processing evidence, or organizing its conclusion.
- Harness modification. Knowledge–Self added a structured workflow that identifies the relevant entity, time period, units, and definitions; inspects available sources; preserves qualifiers; distinguishes supported facts from inference; and calibrates conclusions to the available evidence.
- Insight. A harness can make the model's existing knowledge and tool-use capabilities more reliable by explicitly structuring how evidence is processed, rather than merely instructing the model to reason or verify carefully.
-
Budget-aware delivery pacing
- Failure mode. The agent could spend most of the task budget on extended reasoning and exploration, leaving insufficient time to write, verify, and deliver the requested artifact.
- Harness modification. Terminal-Bench–Self exposed elapsed time and introduced staged reminders to write the result to the required path before verifying and refining it.
- Insight. A harness should expose elapsed time and reserve part of a finite task budget for artifact delivery and verification.
-
Artifact-aware completion
- Failure mode. The agent could continue requesting model calls after the required answer file had already been written, spending additional interactions merely to decide or declare that the task was complete.
- Harness modification. AIME–Self checked whether the explicitly requested artifact had been written and was nonempty, and terminated the run once this observable delivery condition was satisfied.
- Insight. Completion for artifact-based tasks should be tied to observable artifacts rather than an additional model declaration.
We isolate the three modifications by applying each one separately to the same Seed system. The model, tools, task budget, sampling configuration, and evaluator are held fixed. We evaluate Seed and the three one-change variants on the same four benchmarks. Our evaluation focuses on capability gains and regressions under the same task budget.
| System | AIME 2026 | GPQA | SimpleQA | Terminal-Bench 2.0 |
|---|---|---|---|---|
| Seed | 53.3 | 71.2 | 73.4 | 9.0 |
| + Evidence-guided knowledge workflow | 50.0 | 64.4 | 82.0 | 5.6 |
| + Budget-aware delivery pacing | 46.7 | 64.0 | 59.8 | 11.2 |
| + Artifact-aware completion | 66.7 | 60.6 | 81.8 | 5.6 |
Response to Reviewer 2F5h (3/3)
W3. Table 2 reports one trial per item at temperature 1.0, without reported independent repeats of the evolution process or uncertainty estimates. The 13.3-point AIME gain represents four questions, and the local 9/13 to 10/13 to 9/13 sequence does not establish capability loss. Unequal budgets, unavailable feedback, and runtime failures further limit attribution to agent research decisions. Incomplete logs also make audit counts lower bounds, as Appendix A.4.2 acknowledges. The findings are informative about these runs, but broader or causal conclusions require stronger evidence.
Question 3. Could key settings be repeated with matched budgets and reliable evaluation services, with uncertainty estimates and separate accounting for infrastructure failures?
Response: Our experiments use the same allocated exploration budget across the compared settings, including the same iteration limit and maximum wall-clock time per iteration, with early completion allowed. Differences in realized runtime therefore reflect how each Improver uses the available budget, rather than unequal budget allocation. Similarly, the absence of benchmark feedback in Capability-Directed and Open-General is an intentional part of these task settings, not an unavailable evaluation service.
Estimating the distribution of outcomes across repeated evolution runs is not the primary research objective of this study. Our contribution is to provide a persistent system-level environment for RSI and to characterize how current agents organize research, modify system components, construct and use validation evidence, and retain improvements under different levels of external guidance. Variation across evolution runs is intrinsic to open-ended RSI exploration, in which agents may select different research directions, construct different evidence, and pursue different system modifications. Nevertheless, to address your concern about variability across runs, we ran the experiments three times. The results are reported below.
| System | GPQA | AIME 2026 | Terminal-Bench 2.0 | SimpleQA |
|---|---|---|---|---|
| Seed | 71.2 | 53.3 | 9.0 | 73.4 |
| General–Self | 63.1 / 73.2 / 68.2 | 66.7 / 56.7 /66.7 | 0.0 / 1.1 /0.0 | 60.0 / 77.2/ 66.8 |
| AIME–Self | 65.7 / 67.2 /70.1 | 66.7 / 66.7 / 70.0 | 5.6 / 4.5 /2.2 | 82.4 / 79.5 / 80.0 |
Although scores vary across runs, the results consistently show that current agents can achieve local gains but do not yet demonstrate reliable self-improvement without regressions.
Response to Reviewer 2F5h (1/3)
We thank the reviewer for the insightful comments. Our reponses are as follows.
Response: These related works provide important precedents for editable improvement procedures. The distinction between RSI-Env and these works lies in its combination of system-level modification capabilities and experimental design, as summarized below. Here, Self as Improver denotes that the evolving system conducts subsequent research.
RSI-Env complements these systems by extending the focus beyond code modification to research autonomy and sustained system-level exploration.