acceptodds
Withdrawn from ICLR 2027

Agent S4: Evolutionary Computer Use

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Computer-use agents are limited by the interfaces humans give them. An agent may type a spreadsheet cell by cell when a single line of code would do, or overwrite a file with a script while the application still holds unsaved edits. Both the interfaces and the knowledge of when to use each are built by hand, which cannot keep pace with new applications, workflows, and models. We introduce Agent S4, an autonomous computer-use scientist that studies computer-use agents and improves their interfaces using population-based evolution. A meta-agent examines task agents' trajectories on synthetic training tasks, without success labels, and writes new interfaces and usage knowledge into a repository. Population-based evolution then selects which repositories to develop further by their performance on held-out tasks the meta-agent never sees, favoring changes that generalize beyond the seen tasks. The evolved repositories contain interfaces no one specified, such as a single request that batches dozens of GUI actions, and observations that condense PDFs and archives into compact text. On OSWorld V2, Agent S4 with Qwen3.8-27B reaches 21.3% binary task success, on par with the 20.6% reported for Claude Opus 4.8 with batched actions, at an estimated task-solving cost of USD 1.10 per task versus the reported USD 72.40, approximately 1.5% of the cost. These results suggest that interfaces are an axis of improvement complementary to the model, one that can keep evolving as models and harnesses change.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Computer Use, Self-Evolving Agents, Software Synthesis, Population-based Evolution, In-Context Learning

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_BxP6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Agent S4, a framework for automatically improving computer-use agents by evolving repositories of executable interfaces and associated usage knowledge. Rather than modifying the underlying model or agent harness, Agent S4 employs a meta-agent to examine execution trajectories from synthetic training tasks and propose repository modifications through mutation and crossover. A population-based evolutionary procedure selects candidate repositories based on held-out validation performance, while validation trajectories and scores are kept hidden from the meta-agent.

The authors evaluate the approach on OSWorld V2 and ScienceBoard. On OSWorld V2, the binary success rate of Qwen3.8-27B increases from 12.04% without a repository to 19.44% with the seed repository and 21.30% with the evolved repository. On ScienceBoard, the evolved repository improves mean success from 15.6% to 28.0% on 94 held-out tasks. The paper also presents examples of automatically developed interfaces for LibreOffice automation, batched GUI interactions, and browser access.

The central contribution is treating computer-use interface repositories as objects of automated optimization, with the goal of improving agent performance without changing the task model or harness.

Critical strengths
  1. A practically relevant optimization target for computer-use agents.

The paper investigates an important aspect of agent performance: the interfaces through which agents observe and manipulate computer environments. Treating executable interfaces and their usage knowledge as persistent, optimizable artifacts is a meaningful system-level contribution. The distinction between the fixed model/harness and the evolving repository is particularly useful, as it provides a way to improve agent behavior without modifying model weights. The separation between trajectory-based construction and held-out validation-based selection is also a sensible design choice.

  1. Promising empirical results supported by concrete interface examples.

The ScienceBoard experiment demonstrates a substantial improvement in mean success from 15.6% to 28.0% on 94 held-out tasks. This provides encouraging evidence that the proposed framework can be useful beyond the primary OSWorld setting. The LibreOffice, GUI batching, and browser-interface examples further illustrate how trajectory-driven software modifications can make useful operations reusable and reduce repeated implementation overhead. These examples help substantiate the practical value of automatically improving computer-use interfaces.

Critical weaknesses
  1. Limited evidence for the necessity and distinct contribution of population-based evolution.

The evolutionary procedure largely follows established mechanisms, including mutation, crossover, tournament selection, and validation-based candidate selection. Similar ideas involving LLM-guided program modification, agent optimization, and automated harness improvement have already been explored in related work such as ADAS, the Darwin Gödel Machine, Self-Harness, and GEPA.

I do not consider the use of established evolutionary techniques inherently problematic. The potential novelty lies in the optimization target, namely persistent repositories of computer-use interfaces and usage knowledge. However, the paper does not sufficiently establish whether population-based evolution offers meaningful advantages over simpler alternatives.

In particular, a natural baseline would be iterative repository refinement, where a meta-agent repeatedly modifies the current best repository using training trajectories, without maintaining a population or performing crossover. Comparisons with mutation-only evolution and alternative parent-selection strategies would also help isolate the contributions of the proposed design.

Furthermore, the repository jointly contains executable interfaces and textual usage knowledge, but the experiments do not sufficiently distinguish their respective contributions. Some observed improvements may result from improved documentation, tool discoverability, or the reuse of capabilities already accessible through the underlying code-execution environment, rather than the discovery of fundamentally new interaction capabilities.

Consequently, the current evidence establishes the usefulness of improved repositories more convincingly than the necessity of the specific evolutionary procedure.

  1. Limited robustness and attribution of the primary OSWorld results.

On OSWorld V2, the largest improvement comes from introducing the seed repository: binary success increases from 12.04% to 19.44%. Subsequent evolution improves success from 19.44% to 21.30%, corresponding to only two additional successful tasks out of 108.

Although the increase in partial score is encouraging, the paper does not provide repeated independent evolution runs, sufficient uncertainty analysis, or paired task-level results to establish the reliability of this relatively small incremental improvement.

Moreover, repository selection is performed using only 15 validation tasks. With such a small validation set and repeated selection among candidate repositories, selection noise and overfitting are concerns. The relationship between these validation tasks and the final 108-task OSWorld evaluation also deserves clarification, particularly regarding whether the final evaluation is entirely independent of the tasks used for repository selection.

The ScienceBoard results are considerably stronger and partially alleviate these concerns. Nevertheless, the paper would benefit from clearer evidence that the main improvements are robust and specifically attributable to the proposed evolutionary framework.

A related concern is the interpretation of computational efficiency. The comparison with Claude Opus 4.8 uses different underlying models, harnesses, execution budgets, and provider-specific pricing assumptions. Moreover, the reported task-solving costs exclude repository evolution and infrastructure costs. Therefore, the large difference in monetary cost should be interpreted as a comparison between particular system configurations rather than direct evidence of the computational efficiency of the evolutionary method itself.

Questions for authors

Q1. Validation and test separation in OSWorld V2.

Could the authors clarify the exact relationship between the 15 OSWorld validation tasks used during evolution and the 108 tasks used for the final evaluation? Are the validation tasks included in the reported full-set results? If so, could the authors report performance separately on tasks that were never involved in repository selection?

This clarification is important for assessing the generalization claims.

Q2. Contribution of population-based evolution.

How does Agent S4 compare with a simpler iterative repository-refinement baseline that uses the same meta-agent, training trajectories, validation procedure, and computational budget, but does not employ population-based selection or crossover?

If existing experiments with mutation-only evolution, alternative parent selection, or independent evolution runs are available, reporting them would help establish the necessity and stability of the proposed evolutionary procedure.

This question would materially affect my assessment of the methodological contribution.

Q3. Contributions of executable interfaces versus usage knowledge.

Can the authors provide further evidence separating improvements due to newly generated executable interfaces from those due to documentation, workflow guidance, or improved discoverability of existing capabilities?

For example, an ablation that retains only interface-code modifications or only knowledge modifications would help clarify what the system actually learns. A concise comparison of the seed and final repository capabilities would also be informative.

Q4. Cost accounting and efficiency claims.

Could the authors report the total computational or monetary cost of repository evolution, including meta-agent construction and candidate evaluation? How would the reported cost-efficiency comparison change when these costs are amortized over downstream task executions?

Additionally, can the authors clarify the sensitivity of the estimated Qwen inference cost to the assumed 80% input-cache hit rate?

A more controlled comparison would help distinguish efficiency gains from differences in model and provider pricing.

Reviewer_ASkBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Agent S4, a framework that treats the interfaces available to a computer-use agent as an object that can be improved automatically from experience. Instead of modifying the task model or its harness, Agent S4 evolves a persistent repository containing executable interfaces together with knowledge about when and how to use them. The topic and the idea are interesting.

The proposed meta-agent analyzes trajectories generated by a fixed task agent, identifies apparent failures or inefficiencies, and creates new repository variants through mutation or crossover. Training trajectories guide repository construction, while held-out validation performance is used for population selection and remains hidden from the meta-agent. The final repository is evaluated on a separate test set.

On OSWorld V2, the task agent achieves relatively good results.

Critical strengths

The paper introduces an interesting and relatively novel research direction. Treating the computer-use interface itself as a learnable and evolvable object is a meaningful departure from approaches that assume a fixed set of observations and actions.

The separation between repository evolution and the fixed task agent is also conceptually good. The task model and harness remain fixed while executable interfaces and their supporting knowledge evolve. The asymmetric use of training trajectories for construction and held-out validation for selection is also a sensible mechanism for encouraging changes that generalize beyond the trajectories that directly motivated them.

The ScienceBoard experiment also sounds promising and shows evidence that interface evolution can be useful beyond one task setting.

Critical weaknesses

The main OSWorld V2 experiment does not clearly demonstrate a substantial benefit from evolution itself. It seems most of the improvement comes from introducing the seed repository, while subsequent evolution provides a relatively modest additional gain.

It is unclear whether the benefits of interface evolution persist with stronger computer-use models. The paper motivates interface evolution as a general axis of improvement complementary to advances in the underlying model. However, the current experiments do not sufficiently test this complementarity. Stronger models may already possess better capabilities for tool use, coding, planning, and switching between interaction modalities, which could reduce the marginal value of evolved interfaces.

The robustness of the evolutionary process is insufficiently evaluated. The OSWorld experiment evolves one population with 15 descendants, and validation performance varies non-monotonically, which makes it difficult to assess how reliably the proposed method discovers useful improvements.

Questions for authors

Please provide stronger evidence that the observed gains are attributable to autonomous evolution rather than primarily to the initial seed repository.

How does the effectiveness of Agent S4 change with substantially stronger task-agent models?

How stable is the evolutionary process across independent runs, initial repositories, and search trajectories?

What is the full computational cost of producing an evolved repository?

Reviewer_zztKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose a framework for automatically improving the interfaces and usage knowledge for agents on computer-use tasks, where a meta-agent modifies the interfaces and knowledge based on training trajectories and repositories, and validation tasks are used to select good modifications, enabling evolution on computer-use tasks.

Critical strengths
  1. The authors propose an easy-to-use, training-free framework that supports rapid iteration and adaptation across computer-use tasks. The method is easy to understand and broadly applicable.
Critical weaknesses
  1. In the OS-World experiments, the authors do not clearly distinguish performance on the validation set from performance on the test set. As I understand it, the authors use 15 tasks from OSWorld V2 for validation, but Table 1 reports results on all 108 tasks of OSWorld V2 without distinguishing performance on the validation and test sets, which substantially weakens the persuasiveness of this part of the experiments.

  2. The overall experimental design is rather limited, lacking multi-seed experiments to demonstrate the stability of the improvements and the necessary ablation studies to demonstrate the value of each component. In the generalization section, the authors simply switch to another benchmark without actually demonstrating that the method enables out-of-distribution generalization from the training set to the test set.

Questions for authors

Could the authors explain why performance on the validation and test sets is not reported separately in the OS-World experiments? Could they provide multi-seed experimental results and the necessary ablation studies?

Reviewer_ndmFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper starts from the observation that computer-use agents act through fixed, hand-designed interfaces, and that extra test-time compute is often spent working around interfaces that fit the task poorly. Agent S4 searches over the interfaces instead. A repository holds executable helpers and notes on when to use them, and a population of repositories is evolved. On OSWorld V2, a meta-agent reads the task agent's trajectories on 30 synthetic training tasks without success labels and either mutates one repository or crosses over two. Parents are chosen by tournament on validation scores the meta-agent never sees, and the task agent's model and harness stay fixed.

With Qwen3.8-27B as both task agent and meta-agent, OSWorld V2 binary success goes from 12.0% with no repository to 19.4% with the seed repository and 21.3% after 15 evolution steps, at an estimated $1.10 per task. The paper compares this with the 20.6% and $72.40 reported for Claude Opus 4.8. ScienceBoard is a separate experiment with a different setup: Qwen3.5-397B-A17B in the OpenCode harness as task agent, Claude Opus 4.7 in Claude Code as meta-agent, and 50 benchmark tasks for evolution. There, mean success on 94 held-out tasks rises from 15.6% with the seed repository to 28.0%, above Agent S3 (18.4%) run with the same model and budget.

Critical strengths

The strongest part of the paper is how it separates building interfaces from keeping them. On OSWorld the meta-agent edits repositories based on unlabeled synthetic trajectories, while validation tasks, traces and scores are withheld from it and only affect which repositories become parents. That is the right asymmetry if the worry is a repository that memorizes the tasks that prompted each edit. ScienceBoard then shows a large held-out effect under a declared split with three attempts per task: mean success goes from 15.6% to 28.0% and Pass^3 from 8.5% to 18.1%, both above Agent S3 under the 10-minute budget used in the paper. I find the design and this result more convincing than the OSWorld cost comparison.

Critical weaknesses

The experiments do not support the OSWorld headline as written. The abstract reports 21.3% for Agent S4 next to Opus 4.8's 20.6% without saying that the hand-built seed repository alone already reaches 19.4%. The evolutionary part is an unreplicated gain of about two tasks, and the Opus figure comes from a different step budget. The abstract also leaves out ScienceBoard entirely, even though it is the stronger evidence for evolution. The seed and evolution gains should be reported separately, ScienceBoard should be in the abstract, and the cost comparison should be described as indicative.

Questions for authors
  1. Are the 15 validation tasks among the 108 test tasks? If so, what are the seed and evolved binary success rates on the remaining 93?
  2. On ScienceBoard, did the meta-agent see evaluator outcomes for the 50 evolution tasks? How much of the gain remains if Qwen3.5-397B-A17B itself is the meta-agent?
  3. What exactly does the seed repository contain? Section 5.2 and Appendix B.3 mention a few of its capabilities, but a full list would show what evolution had to discover.
  4. Could you add a no-repository baseline on ScienceBoard, and an evolution run that starts from an empty repository? Together they would show how much of the gain depends on the hand-built seed.
  5. A 500-step OSWorld evaluation would make the comparison with the reference runs interpretable.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.