- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents CUAEvolve, which combines a unified GUI-and-code action policy with a self-improving reinforcement-learning curriculum designed for evolving applications and workflows. A diagnoser analyzes rollout failures, a task generator proposes new application- and workflow-evolution tasks, and an admission gate checks task solvability, evaluator validity, evolution validity, and application integration before admitting generated data. The paper also introduces OSEvolveBench, a benchmark of paired base and evolved tasks spanning synthetic application changes and workflow changes. Experiments compare self-improving, random, and static curricula under matched budgets and evaluate transfer to five held-out applications, as well as performance on OSWorld-Verified and WindowsAgentArena.
Critical strengths
-
The paper addresses a timely and practically important failure mode of computer-use agents: capabilities can degrade as applications and user workflows change. OSEvolveBench is a substantial accompanying contribution because it covers both application-level and workflow-level evolution, specifies explicit validation criteria, and includes a human audit of the generated tasks.
-
The empirical comparisons are designed around matched data and training budgets for random, static, and self-improving curricula. Together with evaluation on five applications excluded from reinforcement learning, these experiments provide useful evidence that failure-driven curriculum generation can improve transfer beyond the applications used to construct the training curriculum.
Critical weaknesses
-
The paper primarily reports point estimates from single training runs, without repeated SFT/RL seeds, confidence intervals, or statistical significance tests. Several central improvements are only approximately two to three points, so the current evidence does not establish whether these differences exceed ordinary training and evaluation variance.
-
The application changes in OSEvolveBench are synthetic, predominantly single-axis mutations of React mock applications. Furthermore, an agent may sometimes bypass the changed GUI through code or a stable programmatic interface. These properties limit the extent to which the benchmark demonstrates robustness to compound, chronological changes in real production software.
Questions for authors
-
What is the variance across independent SFT and RL seeds for the self-improving, random, and static curricula, and do the principal comparisons remain significant under that variance?
-
How much of the reported improvement remains on compound or chronological changes drawn from real application versions, particularly in settings where coding actions cannot bypass the modified GUI through a stable programmatic interface?
Response to Reviewer fH8c
We thank the reviewer for describing OSEvolveBench as "a substantial accompanying contribution" and for recognizing the value of comparing curricula under "matched data and training budgets". We understand that your concerns are mainly related to variation across training runs and the gains on real software updates, and we hope our response resolves your concerns.
[About variability across independent training seeds]
For each of three training seeds, we trained all three RL curricula from the same SFT checkpoint (see the table below). Self-improving RL scored highest on both tracks for every seed. On application evolution, its average lead was 6.0 points over SFT, 3.1 over static combined RL, and 3.3 over random RL. On workflow evolution, the corresponding gains were 4.1, 3.3, and 3.2 points. The lower panel reports variation across repeated evaluations of each fixed checkpoint. Values with ± give the mean and sample standard deviation.
[About gains on real software updates with GUI-only interaction]
We agree that single-axis changes in mock applications alone do not establish robustness to real software releases. OSEvolveBench isolates individual interface changes, whereas real updates may combine structural redesigns, added or removed functionality, and changes to programmatic interfaces. To address this concern, we conduct an additional evaluation on 20 tasks in total across four real applications, comparing a base release and an updated release of each application. The selected version pairs include the following changes:
We evaluate the SFT model, CUAEvolve, and GUI-Owl-1.5-32B-Instruct, keeping the task instructions unchanged across versions. CUAEvolve outperforms SFT by 4.5 points on the base releases (29.7 vs. 25.2) and by 6.6 points on the updated releases (28.3 vs. 21.7). All three models score lower after the updates, with declines of 1.4 points for CUAEvolve, 3.5 points for SFT, and 5.5 points for GUI-Owl. CUAEvolve retains its advantage over SFT after the updates and exhibits the smallest average performance decline among the three models.
With coding, terminal, and scripting actions disabled, CUAEvolve outperforms SFT on both releases (see the table below). The GUI-only gain is 3.1 points on the base releases (26.1 vs. 23.0) and 4.7 points on the updated releases (23.9 vs. 19.2). On the updated releases, the GUI-only gain is about 71% of the 6.6-point gain observed when coding is available.
Again, we appreciate the reviewer's valuable reviews and believe your concerns are mainly about variation across training runs and the gain on real software updates. In this rebuttal phase, we will try to address them carefully, and we would be grateful if you could update your score if our responses resolve your concerns.