acceptodds
Under review as a conference paper at ICLR 2027

CUAEvolve: Self-Improving Computer-Use Agents for Evolving Applications and Workflows

Abstract

Computer-use agents (CUAs) can solve increasingly complex, long-horizon tasks, but application updates and evolving workflows can erode their acquired capabilities. Maintaining up-to-date training data requires frequent collection and curation, a costly and time-consuming process that is difficult to sustain. In this work, we introduce CUAEvolve, a computer-use agent that adapts its policy and reinforcement learning (RL) curriculum to application and workflow evolution through a self-improving loop. To improve robustness to visual and interface changes, CUAEvolve interleaves GUI actions with executable code that can consolidate multiple GUI operations into fewer coding sequences and bypass visual grounding when programmatic interfaces remain compatible across updates. To reduce environment data curation as applications and workflows evolve, we introduce a self-improving RL loop that uses failure diagnoses to synthesize targeted task instructions, initial environment states, and evaluators at scale, enabling continual policy improvement. Concretely, 1) application variants modify one interface property with surface appearance, geometry, naming, or interaction sequences; 2) workflow variants revise user instructions and introduce previously unseen workflows. To measure performance on evolved tasks, we introduce OSEvolveBench, which pairs original tasks with application and workflow changes. Experimental results show that the self-improving RL loop raises task scores on evolved applications and workflows to 25.0% and 22.3%, outperforming prior open-source CUAs at comparable scales. CUAEvolve also shows strong retention and cross-platform generalization, achieving 52.7% and 46.7% on OSWorld-Verified and WindowsAgentArena.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer-Use Agents, Self-Improving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fH8cOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents CUAEvolve, which combines a unified GUI-and-code action policy with a self-improving reinforcement-learning curriculum designed for evolving applications and workflows. A diagnoser analyzes rollout failures, a task generator proposes new application- and workflow-evolution tasks, and an admission gate checks task solvability, evaluator validity, evolution validity, and application integration before admitting generated data. The paper also introduces OSEvolveBench, a benchmark of paired base and evolved tasks spanning synthetic application changes and workflow changes. Experiments compare self-improving, random, and static curricula under matched budgets and evaluate transfer to five held-out applications, as well as performance on OSWorld-Verified and WindowsAgentArena.

Critical strengths
  1. The paper addresses a timely and practically important failure mode of computer-use agents: capabilities can degrade as applications and user workflows change. OSEvolveBench is a substantial accompanying contribution because it covers both application-level and workflow-level evolution, specifies explicit validation criteria, and includes a human audit of the generated tasks.

  2. The empirical comparisons are designed around matched data and training budgets for random, static, and self-improving curricula. Together with evaluation on five applications excluded from reinforcement learning, these experiments provide useful evidence that failure-driven curriculum generation can improve transfer beyond the applications used to construct the training curriculum.

Critical weaknesses
  1. The paper primarily reports point estimates from single training runs, without repeated SFT/RL seeds, confidence intervals, or statistical significance tests. Several central improvements are only approximately two to three points, so the current evidence does not establish whether these differences exceed ordinary training and evaluation variance.

  2. The application changes in OSEvolveBench are synthetic, predominantly single-axis mutations of React mock applications. Furthermore, an agent may sometimes bypass the changed GUI through code or a stable programmatic interface. These properties limit the extent to which the benchmark demonstrates robustness to compound, chronological changes in real production software.

Questions for authors
  1. What is the variance across independent SFT and RL seeds for the self-improving, random, and static curricula, and do the principal comparisons remain significant under that variance?

  2. How much of the reported improvement remains on compound or chronological changes drawn from real application versions, particularly in settings where coding actions cannot bypass the modified GUI through a stable programmatic interface?

1 comment

Response to Reviewer fH8c

We thank the reviewer for describing OSEvolveBench as "a substantial accompanying contribution" and for recognizing the value of comparing curricula under "matched data and training budgets". We understand that your concerns are mainly related to variation across training runs and the gains on real software updates, and we hope our response resolves your concerns.

[About variability across independent training seeds]

For each of three training seeds, we trained all three RL curricula from the same SFT checkpoint (see the table below). Self-improving RL scored highest on both tracks for every seed. On application evolution, its average lead was 6.0 points over SFT, 3.1 over static combined RL, and 3.3 over random RL. On workflow evolution, the corresponding gains were 4.1, 3.3, and 3.2 points. The lower panel reports variation across repeated evaluations of each fixed checkpoint. Values with ±\pm give the mean and sample standard deviation.

TrackTraining conditionSeed 1Seed 2Seed 3Mean
Application evolutionSFT19.020.218.619.3±0.819.3 \pm 0.8
Application evolutionRL + static combined tasks21.521.923.122.2±0.822.2 \pm 0.8
Application evolutionRL + random curriculum21.921.222.822.0±0.822.0 \pm 0.8
Application evolutionRL + self-improving curriculum25.026.024.725.2±0.725.2 \pm 0.7
Workflow evolutionSFT18.517.120.118.6±1.518.6 \pm 1.5
Workflow evolutionRL + static combined tasks20.617.819.519.3±1.419.3 \pm 1.4
Workflow evolutionRL + random curriculum19.917.820.619.4±1.519.4 \pm 1.5
Workflow evolutionRL + self-improving curriculum22.321.124.522.6±1.722.6 \pm 1.7
Repeated evaluationsApplication evolutionWorkflow evolution
CUAEvolve SFT19.0±0.719.0 \pm 0.718.5±2.018.5 \pm 2.0
CUAEvolve RL25.0±0.925.0 \pm 0.922.3±1.622.3 \pm 1.6

[About gains on real software updates with GUI-only interaction]

We agree that single-axis changes in mock applications alone do not establish robustness to real software releases. OSEvolveBench isolates individual interface changes, whereas real updates may combine structural redesigns, added or removed functionality, and changes to programmatic interfaces. To address this concern, we conduct an additional evaluation on 20 tasks in total across four real applications, comparing a base release and an updated release of each application. The selected version pairs include the following changes:

  • Thunderbird (102.15.1 →\rightarrow 128.0esr): A unified toolbar and redesigned application menu, new message-list views, removal of the Account Provisioner, and changes to extension API permission requirements.
  • Firefox (128.0 →\rightarrow 141.0): A redesigned sidebar with support for vertical tabs, new tab groups, and removal of experimental Chrome DevTools Protocol support.
  • GIMP (2.10.30 →\rightarrow 3.0.0): Relocated layer-lock controls, non-destructive filter editing, and updated plug-in and scripting APIs.
  • Blender (3.6.0 →\rightarrow 4.0.2): Redesigned node-group interface controls and modifier menus, new Geometry Nodes tools, and changes to node-group and OBJ import/export APIs.

We evaluate the SFT model, CUAEvolve, and GUI-Owl-1.5-32B-Instruct, keeping the task instructions unchanged across versions. CUAEvolve outperforms SFT by 4.5 points on the base releases (29.7 vs. 25.2) and by 6.6 points on the updated releases (28.3 vs. 21.7). All three models score lower after the updates, with declines of 1.4 points for CUAEvolve, 3.5 points for SFT, and 5.5 points for GUI-Owl. CUAEvolve retains its advantage over SFT after the updates and exhibits the smallest average performance decline among the three models.

With coding, terminal, and scripting actions disabled, CUAEvolve outperforms SFT on both releases (see the table below). The GUI-only gain is 3.1 points on the base releases (26.1 vs. 23.0) and 4.7 points on the updated releases (23.9 vs. 19.2). On the updated releases, the GUI-only gain is about 71% of the 6.6-point gain observed when coding is available.

ModelBase AppUpdated App
SFT23.019.2
CUAEvolve GUI-only26.123.9

Again, we appreciate the reviewer's valuable reviews and believe your concerns are mainly about variation across training runs and the gain on real software updates. In this rebuttal phase, we will try to address them carefully, and we would be grateful if you could update your score if our responses resolve your concerns.

Reviewer_DZdHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents CUAEvolve, a computer-use agent that adapts to changing application interfaces and workflows through self-improving reinforcement learning. Its policy combines GUI actions with executable code, while a feedback-driven loop diagnoses failures, generates targeted training tasks, and filters them before adding them to the curriculum. The paper also introduces OSEvolveBench, which evaluates four types of interface changes and evolving workflows.

Critical strengths
  1. The method uses observed failures to construct targeted training tasks. Its ablations show consistent gains over static and random curricula, supporting the value of adapting the training distribution to the agent's weaknesses.
  2. OSEvolveBench covers interface changes and workflow evolution, with held-out applications and workflows for evaluating generalization. It provides a useful testbed for a problem not captured well by conventional fixed-task benchmarks.
Critical weaknesses
  1. OSEvolveBench mainly tests single-axis changes in mock applications. It remains unclear whether the method generalizes to real updates involving coupled interface and functionality changes.
  2. Programmatic execution may bypass the interface changes under evaluation. Without isolating GUI-only and hybrid performance, it is difficult to determine whether the gains reflect improved interface adaptation.
  3. The diagnoser, task generator, and admission judge all rely on GPT-5.6 Sol. The paper does not adequately assess sensitivity to generated-task quality, errors in diagnosis and validation, or the cost of repeated generation and filtering.
Questions for authors
  1. Can the authors separate the contribution of GUI interaction from programmatic execution, for example by evaluating GUI-only and hybrid policies on the same evolved tasks? How much of the improvement remains when code execution cannot bypass the modified interface?
  2. How sensitive is the self-improving loop to errors in failure diagnosis and task admission? Can the authors report task-generation and validation costs, and provide evidence that the gains are robust to the choice of generator and judge?
4 comments

Response to Reviewer DZdH (1/2)

We thank the reviewer for highlighting the curriculum's "consistent gains over static and random curricula" and for describing OSEvolveBench as "a useful testbed". We understand that your concerns are mainly related to real software updates, GUI-only performance, and the reliability and cost of generating tasks, and we hope our response resolves your concerns.

[About GUI interaction versus programmatic bypass]

To assess how much of the improvement remains without code execution, we evaluate each SFT and RL checkpoint of the hybrid policy with code disabled. As the table below shows, with code disabled CUAEvolve improves over SFT by 4.3 points on application evolution (22.4 vs. 18.1) and 3.3 points on workflow evolution (21.1 vs. 17.8). These improvements retain 71.7% and 86.8% of the corresponding gains with code enabled, showing that most of the RL gains persist under GUI-only evaluation.

PolicyCheckpointCode at evaluationApplication evolutionWorkflow evolution
GUI-onlySFT/16.816.2
GUI-onlySFT + RL/18.919.7
GUI+CodingSFTEnabled19.018.5
GUI+CodingSFT + RLEnabled25.022.3
GUI+CodingSFTDisabled18.117.8
GUI+CodingSFT + RLDisabled22.421.1

[About task quality, diagnosis and admission errors]

Following CUA-Gym (Wang et al., 2026a), each task consists of an instruction, an initial environment state, and an executable reward function. In CUAEvolve, the admission judge reviews execution logs, rendered states, and output artifacts against four criteria: 1) the evaluator checks the intended task objective; 2) a reference solution receives full credit; 3) the intended change is observable; and 4) the deliverable can be run and saved in the target application. A candidate is admitted only if the judge determines that all four criteria are met (Appendix D.4). As reported in Section 3, we also manually reviewed a random sample comprising 50% of the benchmark tasks using the same four criteria. Approximately 92% of the reviewed tasks passed all four checks, and we discarded those that failed.

To assess sensitivity to diagnosis errors, we randomly selected 50% of the failure cases used to generate training tasks. For each selected case, we replaced the recommended mutation axis or workflow type with another applicable option. We held the SFT checkpoint, rollout budget, and evaluation tasks fixed (see the table below).

With perturbed diagnoses, CUAEvolve achieved scores of 23.2 on application evolution and 20.8 on workflow evolution. Relative to the unperturbed setting, it retained 70% of the gain over SFT on application evolution (4.2 of 6.0 points) and 61% on workflow evolution (2.3 of 3.8 points). Both scores remained above those of the random-curriculum baseline (21.9 and 19.9, respectively), which omits diagnosis and samples uniformly from the applicable mutation axes or workflow types. It suggests that diagnosis-guided curriculum generation remains beneficial under this perturbation setting.

Training conditionApplication evolutionWorkflow evolution
SFT19.018.5
RL (CUAEvolve)25.022.3
RL with 50% perturbed diagnoses23.220.8
RL with random curriculum21.919.9

[About the choice of generator and judge]

To address the reviewer's question about sensitivity to model choice, we conduct experiments in which we replace GPT-5.6 Sol with Claude Opus 5 as either the task generator or the admission judge, keeping the model in the other role fixed (see the table below). With Claude Opus 5 as the task generator, CUAEvolve improves over SFT by 7.2 points on application evolution and 6.1 points on workflow evolution. With Claude Opus 5 as the admission judge, the corresponding gains are 6.6 and 4.6 points. Relative to the original configuration, replacing the task generator produces larger changes in downstream scores than replacing the admission judge in these experiments. Both substitutions preserve the benefits of curriculum-based RL.

Training conditionTask generatorAdmission judgeApplication evolutionWorkflow evolution
SFT——19.018.5
RL (CUAEvolve)GPT-5.6 SolGPT-5.6 Sol25.022.3
RLClaude Opus 5GPT-5.6 Sol26.224.6
RLGPT-5.6 SolClaude Opus 525.623.1

Response to Reviewer DZdH (2/2)

[About task generation and filtering cost]

For OSEvolveBench workflow tasks, generation averaged 25,065 recorded tokens per task (16,738 input and 8,327 output), or approximately USD 0.25 per task. We use 4.86 minutes per task to estimate validation time. One RL run comprises 130 steps, each sampling 32 tasks with 8 rollouts per task. Generating a fresh workflow task for each of the 4,160 task draws would cost about USD 8 per step, or USD 1,040 per run.

[About coupled changes in real software releases]

We agree that single-axis changes in mock applications alone do not establish robustness to real software releases. OSEvolveBench isolates individual interface changes, whereas real updates may combine structural redesigns, added or removed functionality, and changes to programmatic interfaces. To address this concern, we conduct an additional evaluation on 20 tasks in total across four real applications, comparing a base release and an updated release of each application. The selected version pairs include the following changes:

  • Thunderbird (102.15.1 →\rightarrow 128.0esr): A unified toolbar and redesigned application menu, new message-list views, removal of the Account Provisioner, and changes to extension API permission requirements.
  • Firefox (128.0 →\rightarrow 141.0): A redesigned sidebar with support for vertical tabs, new tab groups, and removal of experimental Chrome DevTools Protocol support.
  • GIMP (2.10.30 →\rightarrow 3.0.0): Relocated layer-lock controls, non-destructive filter editing, and updated plug-in and scripting APIs.
  • Blender (3.6.0 →\rightarrow 4.0.2): Redesigned node-group interface controls and modifier menus, new Geometry Nodes tools, and changes to node-group and OBJ import/export APIs.

We evaluate the SFT model, CUAEvolve, and GUI-Owl-1.5-32B-Instruct, keeping the task instructions unchanged across versions. Models could use code when their action space permitted it. The table below reports each model's mean score on the base and updated releases. CUAEvolve outperforms SFT by 4.5 points on the base releases (29.7 vs. 25.2) and by 6.6 points on the updated releases (28.3 vs. 21.7). All three models score lower after the updates, with declines of 1.4 points for CUAEvolve, 3.5 points for SFT, and 5.5 points for GUI-Owl. CUAEvolve retains its advantage over SFT after the updates and exhibits the smallest average performance decline among the three models.

ModelBaseUpdated
SFT25.221.7
CUAEvolve29.728.3
GUI-Owl21.816.3

Again, we appreciate the reviewer's valuable reviews and believe your concerns are mainly about adaptation to real software updates, GUI-only performance, and the reliability and cost of generating and filtering tasks. In this rebuttal phase, we will try to address them carefully, and we would be grateful if you could update your score if our responses resolve your concerns.

A follow-up question on the actual cost of the self-improving loop. The reported \1,040perrunassumesthatall4,160taskdrawsrequirenewlygeneratedworkflowtasks.However,asdescribedinAlgorithm1,theagentsamplesfromanexistingtaskpoolandgeneratesnewcandidatesonlyatselectedtrainingsteps.Hence,thisestimatemaynotreflecttheactualcostofanRLrun.Couldtheauthorsreporttheactualnumberoftasksgeneratedandadmitted,alongwiththetotalLLMandvalidationcostsforoneRLrun?Also,arethe1,040 per run assumes that all 4,160 task draws require newly generated workflow tasks. However, as described in Algorithm 1, the agent samples from an existing task pool and generates new candidates only at selected training steps. Hence, this estimate may not reflect the actual cost of an RL run. Could the authors report the actual number of tasks generated and admitted, along with the total LLM and validation costs for one RL run? Also, are the \\0.25 per task and 4.86 minutes per task based on actual online training or benchmark construction?

Thank you a lot for the follow-up question about how many tasks we generate and admit! We agree with you that the estimate of $0.25 per task come from benchmark construction. For the RL curriculum, CUAEvolve samples 4,160 times from the evolving task pool, where we generate 2,582 candidate tasks and admit 1,962 distinct tasks (76.0%).

In actual RL run, across all 2,582 candidates including rejected task, the estimated LLM cost for curriculum generation, diagnosis, and admission is \1,343.Taskgenerationaccountsforapproximately1,343. Task generation accounts for approximately \\1,033 (\0.40percandidate),anddiagnosisandadmissionjudgingtogetheraccountforapproximately0.40 per candidate), and diagnosis and admission judging together account for approximately \\310 (\$0.12 per candidate).

Reviewer_kgx9OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CUAEvolve, a computer-use agent designed to adapt to evolving applications and workflows. The method has two main components. First, it uses a unified policy that interleaves GUI interaction with executable Python or Bash actions, allowing the agent to bypass visual grounding or compress multiple GUI operations when programmatic interfaces remain usable. Second, it introduces a self-improving RL loop in which failures observed during training are diagnosed and used to generate targeted application- and workflow-evolution tasks.

The paper also introduces OSEvolveBench, which evaluates application evolution through changes in appearance, geometry, naming, and interaction sequence, and workflow evolution through revised requirements, new deliverables, and unseen workflow combinations. CUAEvolve shows improvement on this benchmark as well.

Critical strengths

The paper addresses an important problem beyond static computer-use benchmarks. Applications, interfaces, and workflows do change over time, and explicitly studying whether agents can adapt rather than simply retain performance on fixed tasks is valuable.

The self-improving curriculum is supported by meaningful ablations. The comparison against static and random curricula is relatively convincing. The ablation study is also good.

Critical weaknesses

OSEvolveBench evaluates a controlled notion of application evolution rather than real longitudinal software evolution.

The broader general-computer-use results are good but do not establish SOTA performance.

Questions for authors

Can the authors evaluate CUAEvolve on real application-version transitions or substantially more complex changes? The current four application-evolution axes provide a controlled testbed, but it would be valuable to know whether the method adapts to real software updates involving multiple simultaneous changes, structural redesign, new or removed functionality, or altered programmatic interfaces.

How well does the self-improving method generalize to evolution types that are not represented during curriculum generation?

1 comment

Response to Reviewer kgx9

We thank the reviewer for your constructive review and for recognizing that our work addresses "an important problem beyond static computer-use benchmarks" and that the self-improving curriculum is supported by "meaningful ablations". We understand that your concerns are mainly related to adaptation to real software updates and generalization to unseen evolution types, we hope our response resolves your concerns.

[About real application updates and more complex changes]

We agree that single-axis changes in mock applications alone do not establish robustness to real software releases. OSEvolveBench isolates individual interface changes, whereas real updates may combine structural redesigns, added or removed functionality, and changes to programmatic interfaces. To address this concern, we conduct an additional evaluation on 20 tasks in total across four real applications, comparing a base release and an updated release of each application. The selected version pairs include the following changes:

  • Thunderbird (102.15.1 →\rightarrow 128.0esr): A unified toolbar and redesigned application menu, new message-list views, removal of the Account Provisioner, and changes to extension API permission requirements.
  • Firefox (128.0 →\rightarrow 141.0): A redesigned sidebar with support for vertical tabs, new tab groups, and removal of experimental Chrome DevTools Protocol support.
  • GIMP (2.10.30 →\rightarrow 3.0.0): Relocated layer-lock controls, non-destructive filter editing, and updated plug-in and scripting APIs.
  • Blender (3.6.0 →\rightarrow 4.0.2): Redesigned node-group interface controls and modifier menus, new Geometry Nodes tools, and changes to node-group and OBJ import/export APIs.

We evaluate the SFT model, CUAEvolve, and GUI-Owl-1.5-32B-Instruct, keeping the task instructions unchanged across versions. The table below reports each model's mean score on the base and updated releases. CUAEvolve outperforms SFT by 4.5 points on the base releases (29.7 vs. 25.2) and by 6.6 points on the updated releases (28.3 vs. 21.7). All three models score lower after the updates, with declines of 1.4 points for CUAEvolve, 3.5 points for SFT, and 5.5 points for GUI-Owl. CUAEvolve retains its advantage over SFT after the updates and exhibits the smallest average performance decline among the three models.

ModelBase AppUpdated App
SFT25.221.7
CUAEvolve29.728.3
GUI-Owl21.816.3

[About generalization to evolution types absent from training]

To assess generalization to an evolution type absent from our training curriculum, we evaluate pagination on 20 tasks in total across seven React-based mock applications modeled on Instacart, GitHub, Shopify Admin, Jira, Salesforce, Twitter, and Trello. For each task, we split the original single-page list (Base) into multiple pages (Evolved), while preserving the task instruction, record order, and evaluator. For example, an Instacart task asks the agent to clip all eligible offers. The evolved version distributes eight offers across two pages, with one eligible offer on the first page and two on the second.

In the table below, CUAEvolve outperforms SFT by 3.6 points on Base (24.3 vs. 20.7) and by 4.7 points on Evolved (21.3 vs. 16.6). Pagination reduces CUAEvolve's mean score by 3.0 points, compared with 4.1 points for SFT and 6.1 points for GUI-Owl-1.5-32B-Instruct. These results show that CUAEvolve generalizes to held-out evolution type, retaining its advantage over both baselines and exhibiting the smallest performance decline.

ModelBaseEvolved
SFT20.716.6
CUAEvolve24.321.3
GUI-Owl-1.5-32B-Instruct9.73.6

Again, we appreciate the reviewer's valuable reviews and believe your concerns are mainly about adaptation to real software updates and generalization to unseen evolution types. In this rebuttal phase, we will try to address them carefully, and we would be grateful if you could update your score if our responses resolve your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.