acceptodds
Withdrawn from ICLR 2027

GUI-Evolver: Scaling GUI Agents via Data–Model Co-Evolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We introduce GUI-Evolver, an autonomous data–model co-evolution framework for GUI agents. Student interactions update a persistent State Model Graph (SMG), which supplies continual GUI knowledge and failure-driven remedial tasks. Accumulated knowledge and admitted trajectories train the next student, selected by Validation without per-round human decisions. Teacher-assisted intervention and diagnosis-guided rerollouts provide alternative targets when teacher reasoning is unavailable. Three-evaluation means on Qwen3.5-0.8B/2B/4B/9B and Qwen3.8-27B improve over Base by 9.9–25.0 points on AndroidWorld and 4.8–13.1 on MobileWorld. Controlled comparisons support the contributions of explicit knowledge, targeted practice, and student-compatible supervision within the evaluated settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
GUI-Evolver autonomously co-evolves GUI agents and their training data through persistent environment memory, incremental GUI knowledge, failure-driven practice, and student-compatible supervision.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_gWXGOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes GUI-Evolver, an iterative data–model loop for mobile GUI agents. A State Model Graph (SMG), built by visual BFS over 20 AndroidWorld apps and continually updated from training interactions, feeds two cumulative data streams: graph-derived GUI knowledge QA (IGKS) and failure-directed remedial subtasks. Each round continues the same LoRA adapter on all accumulated data, and a Validation set selects the checkpoint. Smaller models receive teacher-assisted intervention, while the 27B model uses diagnosis-guided self-rerollouts, which the authors motivate as “supervision compatibility” with its long CoT. Across five Qwen models, the method reports gains of 9.9–25.0 points on AndroidWorld and 4.8–13.1 on MobileWorld over self-measured baselines. Controlled experiments on 9B and 27B are offered to attribute these gains to knowledge supervision, targeted practice, persistent memory, and the choice of supervision source.

Critical strengths

The paper does more than report end-to-end gains. It includes a 2×2 factorial on knowledge vs. task increments, an acquisition comparison (reruns vs. ordinary synthesis vs. failure-directed subtasks), and a repair cohort that reports operation reach and conditional operation success. The last one is a sensible way to tell genuine bottleneck repair apart from “easier practice.” If the numbers were reliable, this design would make the mechanism claims testable rather than anecdotal.

Improvements appear across five model sizes on both benchmarks. Checkpoints are selected by Validation rather than benchmark maxima, and all rounds, including regressions, are reported. This lowers the risk of cherry-picking at the checkpoint level.

Critical weaknesses

The results are hard to trust. Several reported means cannot come out of 116 or 117 tasks averaged over three runs. Every Validation score is a round integer, and the per-round knowledge increments shrink by the exact same ratios for all five models, which is not what independent runs look like. There is no variance and no training seeds, and many ablation gaps amount to one or two tasks, so the controlled comparisons can’t support the mechanism claims. Training tasks, the graph and the AndroidWorld test set also share the same 20 apps. The paper never says whether they share task templates or where the Validation set comes from, so the AndroidWorld gains may just be in-distribution fitting. Since each component already exists, the paper needs a plain iterative self-training baseline to show the added machinery matters, and it doesn’t have one.

The paper defends itself instead of explaining. Almost every sentence ends with a disclaimer of what the result does not show, which reads more like generated text tuned to preempt reviewers than authors describing their work. The method is scattered across the appendix, the only worked example is hypothetical, and no real trajectory or failure case appears anywhere. The overview figure is AI-generated, as the AI usage statement admits: it is cluttered, nearly illegible, and shows an “illustrative” phone screen rather than the actual system. For a systems paper whose credibility depends on showing the pipeline at work, this is a serious gap.

Questions for authors

Clarifications:

Do the training tasks or Validation100 share task templates with AW116? How was Validation100 built?

Please provide raw per-task, per-run results that reproduce Tables 1, 2, 14 and 15.

Please also explain why all Validation means are integers and why the IGKS increments decay identically across models.

Suggestion: Add a matched-budget iterative RFT baseline, and report results over multiple training seeds.

Reviewer_bEncOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes GUI-Evolver, which performs multiple rounds of joint data–model updates by continually accumulating GUI knowledge and interaction trajectories. The system first constructs a persistent state model graph (SMG): nodes store page screenshots, semantic descriptions, and interactive elements, while edges record observed action transitions. The student model’s interaction experience is then used to update the graph, generate incremental knowledge supervision, and create remedial practice tasks targeting failures. The system reviews and executes new tasks, accumulates knowledge and accepted trajectories for LoRA SFT, and uses a validation set to select the student model for the next round.

Smaller models obtain supervision through teacher-assisted interventions, while the 27B model uses new trajectories generated by the student with diagnostic guidance. The paper reports multi-round training results for five models and evaluates success rates for complete tasks on AndroidWorld and MobileWorld. It also provides controlled comparisons of knowledge supervision, memory representations, task generation methods, and supervision sources.

Critical strengths
  1. The research question has practical significance, and the evaluation covers complete tasks across multiple rounds. GUI agents may repeatedly fail at specific operations, and simply adding full-task trajectories may not ensure that practice is effectively allocated. The paper attempts to connect persistent interface memory, explicit knowledge, and targeted practice, reporting multi-round results across five models and two evaluation environments. This is more convincing than reporting only success rates on remedial subtasks.

  2. The controlled experiments account for several important confounding factors. The paper compares settings with and without knowledge supervision, additions of knowledge and tasks, flat versus graph memory, standard versus failure-oriented tasks, and different supervision sources. Some controls hold raw observations, semantic information, retrieval context budgets, and training target token budgets fixed, helping avoid attributing all gains to simply adding more training data. The substantial overall performance gains also suggest that the combined approach has potential practical value.

Critical weaknesses
  1. The independent contribution of “data–model co-evolution” has not been directly validated.

    The online experiments simultaneously add knowledge, tasks, trajectories, and cumulative training, so improvements across rounds do not establish that updating the student improves subsequent data. The incremental-data controls in Appendix J use data generated in advance by a frozen collection policy and primarily assess the value of additional data. The paper still lacks a budget-matched comparison between “updating the student used for data collection” and “freezing the student while continuing to generate new data,” leaving the core closed-loop gains unclear.

  2. The empirical chain from SMG to effective remedial tasks is incomplete.

    SMG records observed pages and action transitions but does not guarantee that task prerequisite states can be reconstructed. The examples in Figure 4 and Appendix K.1 are schematic, with no real records linking graph evidence, remedial tasks, successful execution, and acceptance into training. The aggregate results also do not adequately report rates of state-setup failures, invalid tasks, and final acceptance, making it difficult to assess the reliability of this core component.

  3. The reporting of evaluation details and knowledge increments is insufficient for independent verification.

    AW116 and similar labels indicate only nominal task counts. The paper does not report both actual evaluation denominators and per-run success counts, and some results cannot be recalculated using the fixed-denominator formula in Appendix H. Reporting only averages over three runs is also insufficient to assess the stability of small differences in ablation results. Furthermore, the per-round IGKS increments for all five models in Tables 8–12 strictly follow the same decay ratios, yet the paper does not explain whether preset quotas were used. The data generation and retention process therefore requires further clarification.

Questions for authors
  1. Closed-loop gains: Could the authors compare updating versus freezing the student model used for data collection, while holding the initial data, memory and generation mechanisms, and budgets for interactions, teacher calls, and training tokens fixed? Please report actual costs to assess whether updating the student improves the next round’s data.
  2. Execution evidence for remedial tasks: Could the authors provide real examples covering failure trajectories, SMG page/edge IDs and graph versions, prerequisite state setup, execution verification, acceptance into training, and unassisted full-task performance after training? Please also report the rates of page-matching errors, state-setup failures, invalid tasks, and final trajectory acceptance.
  3. Evaluation denominators and aggregation: Please report task IDs, actual denominators, exclusion rules, and success counts for each run in the main and controlled experiments. With 116 tasks and three evaluation runs with binary outcomes, no integer number of successes yields the 60.2% in Table 2 when the success rate is rounded to one decimal place. Was a different denominator or aggregation method used? Please also report evaluation variability and uncertainty for the key ablations.
  4. Pattern of IGKS increments: For all five models, the per-round increments equal the first-round count multiplied by 1, 0.6, 0.4, 0.25, 0.15, 0.09, 0.05, and 0.03, rounded to the nearest integer. Are these preset quotas, counts retained through sampling, or the total additions after review? Please explain the rule and report the corresponding graph growth, candidate counts, counts passing review, and final export counts.
  5. Training sources and per-round records: Where do the initial 2,970 tasks, inherited SMG, initial knowledge pool, and samples added in each round come from? Please provide each round’s parent checkpoint, graph version, active task list, cumulative number of accepted trajectories, and data sources, and explain how Validation/MobileWorld tasks and information derived from them were excluded from the complete training history.
Reviewer_ppJ9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes GUI-Evolver, an automated loop for training mobile GUI agents. Student rollouts on AndroidWorld update a page/transition graph. The graph generates GUI-knowledge questions and failure-targeted practice tasks. Teacher interventions or judge-guided rerollouts supply targets, and the student is retrained each round. Five Qwen students improve over their base models on AndroidWorld and MobileWorld. The controls ask the right questions, but many control numbers cannot arise from the stated evaluation sets. The AndroidWorld gains also look in-distribution.

Critical strengths
  • The right controls are attempted. Trajectory-only versus joint training, a 2×2 knowledge/task-increment factorial, a curriculum matrix, acquisition policies and supervision sources, all at matched target-token budgets. On paper, this is the kind of attribution study that data-loop papers usually skip.
  • Full round history and clean selection. All rounds, including regressions and rounds after the selected checkpoint, are reported. Checkpoints are chosen by Validation, never by benchmark maxima. MobileWorld is kept out of training and selection.
  • An interesting supervision finding. Teacher short explanations help the 9B student but hurt 27B, while diagnosis-guided rerollouts help 27B. If it holds, "supervision compatibility" is a useful and non-obvious message for distillation into reasoning models.
  • Five model sizes, two benchmarks. Qwen3.5 0.8B to 9B and Qwen3.8-27B run under one public harness. Base scores are re-measured in the same setup rather than copied from vendor reports, which avoids a common mismatch.
Critical weaknesses
  • Many control numbers cannot come from the stated evaluations. TSR averages three binary runs over 116 AW and 117 MW tasks. Yet 40 of 58 control values (Tables 2, 14–17) match no integer success count, even for one run (e.g. 9B AW 60.0). Main tables pass 97%.
  • AndroidWorld results are likely in-distribution. Training uses 2,970 AndroidWorld tasks of unexplained origin, on the graph's 20 apps. AW's 116 test tasks are parameterized templates. Nothing states that training tasks, Validation100 and test templates are disjoint, so AW gains do not show generalization.
  • Plain distillation explains most of the gain. Teacher-action-only SFT reaches 56.4 (9B) and 76.8 (27B) on AW. The full curriculum reaches 60.2 and 80.8. The headline +9.9 to +25.0 compares against an untrained Base; over standard in-domain distillation the method adds about four points.
  • Format learning is not separated. Base 0.8B and 2B score 9.5% and 12.1% on AW in a harness they were never tuned for. Any in-domain SFT fixes output format. The paper admits there is no format-only control, so small-model gains cannot be credited to graph knowledge or curricula.
  • No variance, so the controls are within noise. Each configuration is one training run; the three evaluations reuse the same checkpoint at temperature 0. Control differences of 1–4 points equal one to five AW tasks. The paper itself declines to claim significance, yet the controls carry every attribution claim.
  • Novelty is a recombination. The authors acknowledge OpenMobile is closely related: global screen memory, task synthesis and monitored expert correction. GUI ripping and DroidBot cover exploration; WebRL, UI-Genie and EvoCUA cover failure-driven curricula; DAgger and on-policy distillation cover intervention. The paper concedes none is new in isolation.
  • Hard to read and underspecified. Defensive caveats ("does not imply", "not necessarily") dominate the text. Teacher and judge models are not fully named; only "Gemini-3.8-flash" for small students. Main-run compute is not reported. The algorithm cannot be reimplemented from the paper.
Questions for authors

The deciding issues are whether the reported numbers come from real per-task outcomes, and whether the AndroidWorld gains reflect anything beyond in-distribution training.

  1. Please release per-task, per-repeat binary outcomes for every reported cell, including all controls. Explain how values such as 9B AW 60.0 and MW 30.0 arise from 116×3 and 117×3 evaluations. If some cells used other effective denominators, give the actual denominator for each cell.
  2. How were the 2,970 AndroidWorld training tasks and the Validation100 set constructed? Do they share templates, parameter ranges or goals with the 116 test tasks? Please report AW results on templates excluded from training, generation and graph construction.
  3. Please add a full-budget teacher-distillation baseline and a format-only SFT baseline. Run at least three training seeds for Base→Final and for the key controls. Does the method still beat plain distillation beyond seed variance?
  4. (Clarification) Which teacher and judge models fill each role for each student? How much total teacher, judge and environment compute does one full run consume? How many tasks and samples does each round actually admit?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.