acceptodds
Desk rejected at ICLR 2027

DataOS: A Recursive Self-Improvement System Driven by Environment Data

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Recursive self-improvement (RSI) seeks to make AI systems capable of revising the procedures through which they develop and improve their capabilities. Through successive experiments, these systems accumulate environment data (e.g., datasets and execution traces) that can inform subsequent improvements. Using these data effectively requires preserving the conditions behind past outcomes and identifying evidence and resources relevant to the evolving system. We introduce DataOS, a four-layer system for data-driven RSI that couples task execution and system evolution through continuously maintained environment data. DataOS preserves provenance and dependencies across versioned artifacts and execution records, enabling agents to use accumulated evidence and reusable resources to diagnose limitations and make targeted revisions to data, models, and the harness. We evaluate DataOS on PostTrainBench for autonomous post-training and Curation-Bench for multimodal data selection. Under matched researcher models, reasoning effort, and budgets, DataOS improves aggregate scores over native coding agents by up to 33.5% and 6.4%, respectively. Ablations support the contributions of structured history, reusable operators, and harness evolution, while transfer experiments show that accumulated research state can benefit subsequent runs even when target-model weights are reset.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Recursive Self-Improvement, Data Centric AI, Autonomous Research

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ogCoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DataOS, a four-layer framework that organizes experimental artifacts and execution histories to guide revisions to training data, target models, and agent harnesses. It reports improvements over native coding agents on PostTrainBench and Curation-Bench, with ablations and transfer experiments intended to support the value of structured history and harness evolution.

Critical strengths

The evaluation covers both open-ended post-training and fixed-recipe data curation. The component ablations also provide some evidence that structured history and harness evolution contribute beyond the underlying coding agents.

Critical weaknesses
  1. The RSI claim is stronger than the demonstrated feedback loop. GPT-5.6 Sol or Claude Opus 5 serves as the researcher while smaller, separate target models are trained. The paper demonstrates an agent improving target models and parts of its harness, but does not show whether an improved target model can itself become the researcher for subsequent improvement. This distinction is central to interpreting the “self-improvement” claim.
  2. The evolution mechanism and its dynamics are insufficiently evidenced. The appendix defines the three Plan types and provides one joint-optimization trajectory, but does not clearly show representative filled-in plans, actual prompt/operator/pipeline revisions, or comparable round-by-round performance trajectories on both benchmarks. Consequently, it is difficult to assess what evolved in practice and when the reported gains arose.
Questions for authors
  1. Could you provide a concrete executed example of each Plan type, including its specifications, version changes, and resulting actions? For harness evolution, which of the four predefined role prompts, operators, or pipelines were actually changed, and why was this initial role decomposition chosen?
  2. What training interventions were permitted and actually attempted on PostTrainBench? Please distinguish these from Curation-Bench, where the training recipes are fixed and only data subsets are selected.
  3. Can an improved target model replace the frontier researcher in a subsequent iteration, and does this yield further improvement? If not, how do the authors delimit the paper’s RSI claim?
  4. Could the authors report per-iteration candidate and best-so-far scores, alongside harness-revision events, for both benchmarks? The existing joint-optimization table does not provide a time-aligned comparison, and Curation-Bench lacks an analogous trajectory.
3 comments
AuthorsOpenReview ↗

Official Response to Reviewer ogCo (1)

We are grateful to the reviewer for the thorough review and helpful suggestions for improving the manuscript.

Q1. RSI scope and researcher replacement

We agree that target-to-researcher replacement is a key question for RSI. This is particularly challenging because frontier researcher models benefit from large-scale training and partly undisclosed data, while comparable compute and data resources are difficult to obtain for target training. RSIBench-Data [1] reports a related difficulty: its exploratory same-family researcher–target experiment failed to improve upon the unadapted target.

Our scope is environment-data-driven, system-level RSI with a fixed researcher backbone. DataOS uses accumulated experimental evidence to revise data, target models, and the research harness. Adopted prompt, operator, and pipeline revisions are reused in later iterations, changing the procedures that produce further improvements. Our PTB and Curation-Bench targets are trained and evaluated on downstream tasks; we have not tested researcher replacement. Reviewer kCXr also recognizes persistent environment-data management as a unifying perspective and the paper’s most important contribution.

These mechanisms provide a foundation for future researcher replacement. A successor researcher still needs to understand prior experiments, reuse effective artifacts and procedures, and propose and validate further changes. DataOS preserves this experience across iterations and system versions by maintaining evidence, provenance, and dependencies. We will explore turning accumulated research trajectories, experimental feedback, and reusable strategies into resources for training and evaluating researcher capabilities, then test whether improved models can take over the researcher role and continue producing improvements on bounded research tasks with controlled budgets.

References

[1] Fanqing Meng, Lingxiao Du, Qiguang Chen, Ziqi Zhao, Haocheng Lu, Mengkang Hu, and Michael Qizhe Shieh. RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvement. arXiv:2607.25886, 2026.

Q2. Executed Plan instances and concrete component revisions

We decompose the system by decision objective, evidence scope, and action space—connected through versioned Plans and shared environment data—to prevent low-level execution and debugging traces from polluting high-level research decisions and to preserve cross-iteration state. The Leader plans experiments under task objectives and budgets (Task Plan); when cross-iteration evidence reveals procedural bottlenecks, the Evolver diagnoses limitations and specifies system revisions and validation criteria (Evolution Plan); the Worker handles implementation and execution; and the Maintainer commits artifact provenance and component versions (Maintenance Plan) for downstream reuse.

In the documented example, the Leader provides experimental feedback and improvement objectives; the Evolver analyzes linked records and specifies revisions and validation; the Worker adds and registers ref_loss_margin_selector and integrates it into the Task Plan's data.recipe, enabling checkpoint-conditioned preference-data selection; and the Maintainer preserves component versions and dependencies for subsequent reuse.

We have clarified the rationale for this initial role decomposition in Sections 3.1–3.5 of the revised manuscript, and added executed instances of the Task, Evolution, and Maintenance Plans in Appendix D.1–D.3 (including specifications, version relationships, executed actions, and outcomes). Appendix D.4 provides role handoffs and an index of supporting records, while Appendix D.5 details the operator and pipeline changes and their subsequent reuse.

Q3. Training interventions on PTB

On PTB, DataOS supports full-parameter SFT, LoRA SFT, full-parameter DPO, and GRPO, together with hyperparameter tuning and checkpoint reuse. It autonomously selects and executes interventions based on task requirements and experimental feedback, within fixed data, compute, and evaluation constraints.

We analyzed its actual training-method choices and obtained the following distribution:

Training methodStandard PTB (1 H100, 10 h)Scaling (4 H100s, 20 h)
Full-parameter SFT70.0%42.3%
LoRA SFT25.0%7.7%
Full-parameter DPO0%50.0%
GRPO5.0%0%

DataOS predominantly selects SFT in standard runs (95%), while DPO accounts for 50% of its choices in scaling.

We document the supported interventions, observed choices, and distinction from Curation-Bench’s fixed training recipes in Appendix B.1.4 and Table 6 of the revised manuscript.

AuthorsOpenReview ↗

Official Response to Reviewer ogCo (2)

Q4. Per-iteration performance trajectories

We have added Figures 2 and 3 in Appendix D.6, presenting PTB trajectories aligned by elapsed time and per-iteration Curation-Bench trajectories, both with candidate and best-so-far scores. These trajectories illustrate our view of improvement: sustained exploration guided by experimental feedback, where candidate performance may fluctuate or regress while better solutions are discovered and retained. In Curation-Bench, the model is reset and the training configuration fixed at each iteration, yet later iterations still identify better data subsets. This illustrates improvement in the research process through feedback, rather than continuous accumulation of model-weight updates.

I appreciate the authors’ response and their careful answers to my questions. However, I still believe my initial overall rating reflects my assessment of the paper, and therefore I would like to keep it.

Reviewer_gq2JOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes DataOS, a multi-agent framework for autonomous post-training and data curation, combining structured experiment histories, reusable data operators, and editable research pipelines. It reports improvements over native coding agents on PostTrainBench and Curation-Bench, alongside ablations and transfer experiments.

Critical strengths
  • The authors take on real difficult benchmarks to show the validity of their work
  • They also provide ablations for certain components of their design
  • They are transparent in providing their prompts and implementation details
  • They use strong baselines for their comparison
Critical weaknesses
  • The proposed architecture feels overengineered and underjustified.
  • The four-layer architecture, four specialized agents, three types of plans, and numerous interfaces are described extensively, but there is little justification for why this particular decomposition is necessary.
  • The distinction between Leader, Evolver, Worker, and Maintainer is not clear.
  • Only few components are ablated, and without any clear motivation the design choices are quite confusing
  • The paper emphasizes provenance, dependency relationships, and state-conditioned diagnosis. However, there is little direct evaluation of whether these mechanisms actually work. The authors measure downstream task scores, but the claimed mechanism is largely treated as a black box.
  • Most importantly, the scores are incorrectly compared, against models with medium reasoning effort instead of max. The claimed large gains suddenly vanish if the comparison is made against a simple single model with increased reasoning effort.
  • There is also no mention of cost/budget in dollar spend on APIs anywhere. If the entire pipeline consumes much more resources, then the benefit is not clear. It’s possible the gains simply come from increased inference time, displayed by model variants
  • BFCL is now excluded from PostTrainBenchv1.2, as it was very noisy and prone to being cheated on.
Questions for authors
  • Could you provide exact cost the pipeline adds, in replacement of running a single model?
  • Can you defend and explain the reasoning behind implementing such an exhaustive pipeline?
2 comments
AuthorsOpenReview ↗

Official Response to Reviewer gq2J (1)

We appreciate the reviewer's time and effort in evaluating our work and providing valuable feedback.

Q1. Architecture and role design

DataOS's four-layer architecture is designed to prevent execution reproducibility problems and cross-iteration state loss when long-horizon research relies on a single continuous context. Outside the model context, decisions in the Agent layer are translated into versioned Plans in the Pipeline layer with explicit inputs, operators, and dependencies, which the Runtime layer executes across heterogeneous tools and backends; the Environment Data layer preserves provenance and version links across iterations, allowing Plans to reuse existing artifacts and execute only incremental operations.

Dividing the workflow into four roles and three Plan types bounds the evidence scope and action space at each decision step, preventing lengthy execution and debugging traces from interfering with experiment planning or anchoring revisions to the current recipe. The Leader and Evolver differ in decision purpose: the Leader advances task performance within the current setup (Task Plan), whereas the Evolver is triggered only when cross-iteration evidence indicates a systemic bottleneck, diagnosing limitations and revising the research procedure itself (Evolution Plan, referencing Task Plans for validation). The Worker handles implementation and execution details within an assigned scope, while the Maintainer executes Maintenance Plans to preserve actual records, execution deviations, and version dependencies beyond individual task summaries. We have expanded Sections 3.1–3.5 of the revised manuscript to explicitly clarify this architectural rationale, role boundaries, and Plan handoff mechanism.

Q2. Ablation design and rationale

Ablations of a complex agent system require prioritization. We therefore select comparisons around the core mechanisms of environment-data-driven improvement. Historical records and their relationships provide evidence for diagnosis and subsequent decisions; reusable operators support turning those decisions into experiments; and harness revisions use experimental feedback to improve subsequent research procedures. Accordingly, we evaluate:

  • History accumulation: Compare current-record feedback with sequential history to assess the value of retaining past experimental experience.
  • Structured organization: Compare sequential history with the full system to assess the joint contribution of explicit relationships and structured queries beyond history retention.
  • Operator reuse: Remove the operator library while allowing the agent to write its own data-processing code, assessing the value of reusable operations.
  • Procedure evolution: Freeze the initial harness while retaining data and training-strategy search, assessing the additional contribution of revising research procedures.

These comparisons use the same researcher model and per-task resource budgets. We have clarified their rationale and control settings in the appendix.

Q3. Evidence for environment-data mechanisms

Environment-data management links experimental outcomes to execution conditions and historical resources, supporting targeted diagnosis and revision. For example, the Evolver examines training configurations, checkpoints, sample responses, and evaluator feedback from T6–T9, identifies incomplete code and missed edge cases, and proposes preference optimization. The Worker reuses the T8 checkpoint to score candidate preference data and integrates a new selection operator into data preparation.

The Maintainer then records dependencies among source data, the scoring model, operator versions, selection results, and downstream training and evaluation. This enables T10–T12 to reuse existing resources and trace the conditions behind different experimental outcomes.

We have added this example to Appendix D (Sections D.2–D.5), including concrete plans, diagnostic evidence, dependencies, operator changes, and validation records.

Q4. Comparison with higher-effort baselines

The original submission already reported native-max results for both researcher families (Table 1), with an explicit comparison in Sec. 4.2. DataOS uses medium effort and outperforms native-max on both benchmarks. After excluding BFCL, the six-task PTB margins remain +5.40 (Claude) and +3.32 (Codex). On Curation-Bench, DataOS scores 43.95/43.16 versus 40.84/40.68 for native-max.

AuthorsOpenReview ↗

Official Response to Reviewer gq2J (2)

Q5. API cost and resource budget

We report researcher-side API usage and estimated costs (from official token pricing) for GPT-5.6 Sol/Codex averaged over 10 sessions (one target model × one benchmark per session). Target-model training and evaluation compute is identical across all settings (1 × H100, 10h budget). For DataOS, all four roles run at medium effort, and totals include all role handoffs, retries, and auxiliary calls.

MethodEffortTokens/session (M), in / out (reasoning)Est. API cost/session (USD), PTB / CurationPTB (6-task)Curation
Codex CLImedium120.4 / 2.40 (1.58)$72.9 / $58.322.7641.09
Codex CLImax132.8 / 9.10 (7.60)$152.4 / $121.928.9740.68
DataOSmedium139.8 / 2.95 (1.92)$84.7 / $67.832.2943.16

Note: Input tokens include cached reads; output tokens include reasoning tokens.

DataOS adds only 16% to the API cost of the single medium-effort baseline while costing 44% less than native-max, yet outperforms native-max by +3.32 (PTB) and +2.48 (Curation-Bench). Simply scaling single-model inference time (max effort) is thus both substantially more expensive and less effective than DataOS's structured workflow.

Q6. Benchmark validity and PTB v1.2

PTB v1.2 was released on September 30, shortly after the September 25 submission deadline. Following the updated benchmark specification, we have removed BFCL and updated all corresponding tables and analyses in the revised manuscript. Recomputing the unweighted average across the remaining six clean tasks, DataOS maintains consistent gains over both native-medium (+6.01 / +9.53 for Claude / Codex) and native-max (+5.40 / +3.32), confirming that our improvements are robust and not driven by BFCL noise.

Reviewer_kCXrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the management of "environment data" (e.g., datasets and execution traces that accumulate during agent sessions and can inform subsequent improvements) in the context of Recursive self-improvement (RSI), which aims to enable AI systems to revise the procedures by which they develop and improve their capabilities. The paper introduces DataOS, a four-layer system for data-driven RSI that couples task execution and system evolution through continuously maintained environment data. DataOS preserves provenance and dependencies across versioned artifacts and execution records to link outcomes to their actual inputs and configurations, enabling agents to use accumulated evidence and reusable resources to diagnose limitations and make targeted revisions. The paper evaluates DataOS for autonomous post-training and multimodal data selection. Under matched conditions, the paper reports DataOS improves aggregate scores over native coding agents and dedicated research agents. The paper conducts several ablation studies and validates the contributions of its proposed modules (e.g., structured history, reusable operators, and harness evolution).

Critical strengths
  • The framing of this work is attractive, unifying multiple lines of data-centric investigations for agentic systems (e.g., world modeling, memory, harness updates) under a shared theme of persistent (environment) data management for RSI. This view is its most important contribution to the community.

  • The writing (language) is generally good, and the elaborations are clear. References and discussed related works are timely and relevant.

Critical weaknesses
  • Moderate to substantial lack of conceptual depth and technical detail. The proposed data management techniques are mostly heuristics and not inherently novel. Also, the paper does not include substantial discussions on its design, considerations, or conceptual contributions. Further, with its appealing framing, the description of the technical components is substantially vague. So it is not immediately clear how or why they are innovative.

  • All the empirical results do not include a range of variance/standard deviation. Agentic sessions often carry significant variance. Are all these results from a single run?

Questions for authors
  • Can you help me (i.e., the readers) understand how this DataOS is different from a skill or a combination of skills?

  • Can you provide examples of how the management helps agents process/react to environment data (differently)? For example, how is the harness rewritten, with what focus, and changed in which aspects, etc.?

2 comments
AuthorsOpenReview ↗

Official Response to Reviewer kCXr

We thank the reviewer for carefully reading our manuscript and providing thoughtful comments and constructive suggestions.

Q1. Mechanism and system-level novelty

Allowing a general-purpose agent to explore freely within a continuous context depends largely on implicit, context-bound heuristics, making multi-round trials and long training runs prone to state drift, overwritten artifacts, and loss of traceability. DataOS's four-layer architecture enforces two system-level invariants outside the model context: decisions in the Agent layer are translated into versioned Plans in the Pipeline layer with explicit inputs, operators, and dependencies, which the Runtime layer executes deterministically across heterogeneous tools; meanwhile, the Environment Data layer links actual artifacts, checkpoints, and evaluation records to the experiment state tree, providing structured evidence for cross-iteration diagnosis and reuse.

The role and Plan decomposition bounds the evidence scope and action space of each decision stage to prevent context interference. The Worker handles low-level execution and debugging traces so they do not dilute the Leader's high-level experiment planning (Task Plan); when progress stagnates, the Evolver queries cross-iteration provenance and dependencies to propose and validate revisions to prompts, operators, or pipelines (Evolution Plan); and the Maintainer independently commits execution records, component versions, and dependencies back to the environment state (Maintenance Plan), turning individual runs into reusable resources for subsequent evolution.

Q2. Relationship to skills

Skills can encapsulate domain knowledge, instructions, and tool scripts, but fundamentally remain local guidelines that depend on the model following them within a single context window, without cross-iteration state persistence or system-level control. Relying solely on Skills for long-running experiments makes them prone to state drift or artifact loss during repeated iterations, rollbacks, and lengthy training runs.

DataOS therefore implements a dedicated runtime harness that enforces system-level invariants outside the model's context. It orchestrates multiple roles, executes versioned Plans, and advances the experiment State Tree. For example, when an agent selects a historical checkpoint and proposes a revision plan, the harness validates the node, creates a separate experimental branch, schedules training and evaluation, and binds the new operator/pipeline versions, model weights, and scores to that same branch.

Q3. A concrete harness-revision example

Thank you for the suggestion. DataOS uses environment data as both diagnostic evidence and reusable resources. For example, to investigate regressions across T6–T9, the Evolver examines linked training configurations, historical checkpoints, sample responses, and evaluator feedback. Observations of incomplete code and missed edge cases motivate the hypothesis that preference optimization could improve answer completeness.

This diagnosis leads to a concrete harness revision: the historical T8 checkpoint is reused to score candidate preference pairs, and the Worker implements and registers ref_loss_margin_selector, integrating checkpoint-conditioned selection into data preparation. The Maintainer then incorporates the new operator, pair scores, selected subsets, and their links to source data, checkpoints, and training results into the environment data. Subsequent T10–T12 experiments reuse these resources, while the linked records support tracing and comparing outcomes under different data combinations and training configurations. Accumulated environment data support the current revision, while the resulting tools, data, and records provide resources for subsequent research.

We have added this example, including its diagnostic evidence, concrete changes, validation requirements, and subsequent reuse, to Appendix D (Sections D.2 and D.5).

Q4. Repeated runs and standard deviations

The main results are averaged over three runs per configuration, as stated in Appendix B.3.4 of the original submission. We have added standard deviations for the main results of DataOS and the baselines on both benchmarks in Table 1 of the latest revised submission. The original PTB allows agents to download training data, which can increase run-to-run variability. We therefore use a shared, fixed data pool across methods to reduce this source of uncertainty.

Thanks for providing the detailed response and revisions to the manuscript. The updates are positive. I've also read through other reviews and responses. Overall, my initial concerns on the conceptual depth, scientific innovation, and presentation remain and I'd wish to keep my score as is.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.