- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents Ouroboros, a self-developing agent harness. The harness operates in two modes: recursive free evolution, where the harness specifically attempts to evolve itself, and experience-driven core evolution, where the harness performs regular tasks. The authors present the architecture of the system. The system is evaluated on 3 benchmarks: TerminalBench 2.1, OSWorld-Verified, and CL-Bench and attains state-of-the-art performance on all three. The authors also provide a qualitative report on Hope, a deployment of Ouroboros that has been running for 210 days at the time of submission. Finally, the authors discuss guardrails they use to maintain operational safety of the self-evolving system.
Critical strengths
- The authors study a timely and relevant problem: self-improving agent harnesses
- The qualitative study on Hope, an extremely long-horizon AI system, is interesting and novel
Critical weaknesses
-
Presentation. The paper is in general very difficult to follow.
- Table 1 presents several items that are either present or not in various self-evolving systems. However, these items are (a) not of the same type (being able to edit prompts and having every change implemented as a "reviewed commit" are two very different features of a system), (b) insufficiently described. For example, how is a workflow distinct from core code? What is deployment state in this context?
- Figure 1 introduces many different concepts which are never discussed in the figure or the text. Take, for example, 'Typed Outcomes'. There is no description of this component and it is left to the reader to imagine what it entails or why it's important. It is also unclear what the arrows in the figure are meant to denote.
- Several sections of the text introduce concepts without fully describing them. Example: "The runtime represents identity and continuity through a versioned constitution, an editable identity profile, scratchpad and chronicle projections, project memory, review ledgers, and Git history. ". It is generally hard for a reader to understand what these components are and why they are necessary.
- The two modes of core evolution are unclear. Arguably, the two-way decomposition of the evolution process is the primary distinguishing element of Ouroboros. The authors should make the distinction clearer, potentially with a symbolic description or examples. Is the difference purely whether evolution is a primary of secondary goal of the task?
-
Missing details on Hope. The description of Hope is lacking in several ways.
- The authors state that Hope has been running since 27 February, 2026. However, time is arguably a poor metric for measuring the lifecycle of an asynchronous agent. The number of tasks and evolution cycles would be better metrics to report.
- The authors describe many features of Hope (Multi-channel state, Controls on self-directed work), but it is not clear whether these are common to all Ouroboros deployments. If these are common, these would be better placed in the architecture description.
-
Evaluation protocol is entirely missing. Section 5 describes a number of results, but these are largely meaningless without descriptions of the evaluation settings, which are missing. Were these run with Hope or without Hope? At what stage of the lifecycle were these experiments run? What data did the agent have access to in order to evolve?
-
Missing baselines. The authors do not benchmark against other self-evolving systems.
-
Missing architectural ablations. The authors present several aspects of the Ouroboros architecture that are not adequately justified or ablated. While many of these features are interesting, based on the paper I am not sure why they are necessary. Examples include features described in Figure 5, sub-agents, and the 3-tiered commit pipeline.
-
No justification for choice of safety controls. The authors do not provide any concrete quantitative evidence that their "Operational Safety Controls" are effective. An ablations demonstrating misaligned behavior would be good evidence that these measures are justified.
Questions for authors
- Can you provide an example of a task from each of the two evolution phases?
- How does the agent decide when to terminate each phase?
- What Ouroboros system are the evaluation results in section 5 for? i.e. what evolution process did it undergo?
- What data was the Ouroboros system evaluated in section 5 developed on? How did you ensure there was no leakage of test set data?
- What is the minimal set of Ouroboros components needed to achieve the evaluation results in section 5?