acceptodds
Under review as a conference paper at ICLR 2027

GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training

Abstract

Post-training GUI agents in interactive environments is critical for developing generalization and long-horizon planning capabilities. However, training on real-world applications is hindered by high latency, poor reproducibility, and unverifiable rewards relying on noisy visual proxies. To address the limitations, we present GUI-Genesis, the first framework to automatically synthesize efficient GUI training environments with verifiable rewards. GUI-Genesis reconstructs real-world applications into lightweight, task-conditioned web environments using multimodal code models and equips them with code-native rewards, executable assertions that provide deterministic reward signals and eliminate visual estimation noise. Extensive experiments on the WeChat mini-app platform (serving over one billion active users) show that GUI-Genesis reduces interaction latency by 10× and cuts estimated recurring rollout and verification costs by over 98% compared with training on real mini-apps, excluding one-time preparation costs. Notably, agents trained with GUI-Genesis-synthesized environments achieve performance comparable to agents post-trained in real-world environments, demonstrating robust zero-shot sim-to-real transfer and establishing automated environment synthesis as a scalable and cost-effective alternative to direct real-world training. We open-source GUI-Genesis on our anonymous website: https://anonymous.4open.science/r/gui_genesis-B23B/.

Primary area:
reinforcement learning
Keywords:
environment synthesis, reinforcement learning, GUI agent

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7mXPOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GUI-GENESIS, a framework for automatically synthesizing lightweight GUI training environments with code-native rewards. Given interaction traces collected from real-world mini-apps, the framework uses multimodal code models to reconstruct task-specific web environments and embed deterministic, state-based reward functions. It further employs static self-reflection and dynamic execution tests to verify the generated environments. The authors evaluate the framework by post-training a GUI agent using reinforcement learning in the synthesized environments. Experiments demonstrate substantial reductions in interaction latency and estimated rollout costs, while achieving performance comparable to real-environment training when evaluated on unseen real-world mini-apps.

Critical strengths
  1. The paper addresses an important bottleneck in interactive GUI agent training by automatically constructing lightweight, task-specific environments. Integrating environment synthesis with code-native rewards provides a practical approach to reducing interaction overhead and reliance on expensive visual reward models.
Critical weaknesses
  1. GUI-GENESIS synthesizes task-specific environments from limited interaction traces, potentially restricting the state and action space available for RL exploration. Effective RL training may require agents to explore alternative paths and learn to recover from failures. However, the paper does not demonstrate whether the synthesized environments support such off-trajectory behaviors, as the verification process only checks for positive rewards along generated test paths. This raises concerns about whether these environments provide sufficient interaction diversity for RL training and real-world generalization.
  2. Although GUI-GENESIS is proposed as a general framework for GUI agent post-training, its evaluation is limited to a proprietary WeChat mini-app dataset with only 149 test tasks. Experiments on established public benchmarks (e.g., WebArena or AndroidWorld) would provide stronger evidence of generalizability and reproducibility.
Questions for authors
  1. How does GUI-GENESIS ensure that synthesized environments support meaningful exploration beyond the collected trajectories? An evaluation of off-trajectory state transitions and error-recovery behaviors would help establish whether these environments are sufficiently comprehensive for RL training.
  2. Table 2 shows a substantial gap (15–20 percentage points) between human and VLM evaluations. While the paper provides a qualitative example of VLM false positives, could the authors further analyze the sources of this discrepancy and clarify how the reliability of human annotations is ensured?
  3. Could the authors clarify whether both reward mechanisms use the same granularity during RL training, and how intermediate rewards are incorporated into Multistep GRPO?
Reviewer_8uEBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GUI-GENESIS, an automated environment synthesis framework for GUI Agents in online reinforcement learning, aiming to address the issues of high interaction latency, high training costs, and the difficulty in accurately verifying task rewards in real mobile applications. Specifically, the authors first collect interaction trajectories using GUI Agents in the real WeChat mini-program, and then transform the real application into a task-conditioned lightweight Web environment through multimodal code models and hierarchical code generation strategies. At the same time, Code-Native Reward based on program states is automatically generated at the backend of the environment. The authors further validate the synthesized environment using LLM static checks and Playwright dynamic tests, and conduct reinforcement learning training on GUI Agents in these environments using Multistep GRPO.

Experiments show that the synthesized environment can significantly reduce interaction latency and the cost of estimated repetitive training environments, while achieving similar performance to directly using the real environment for training on the real WeChat mini-program.

Critical strengths

The issues discussed in this paper have significant practical implications. Real GUI applications often encounter problems such as high interaction latency, complex external dependencies, and difficulty in accurately obtaining reward signals, which limit the efficiency of large-scale online reinforcement learning. This paper provides a practical and valuable solution for reducing the interaction and reward verification costs of GUI Agents in online reinforcement learning through automatic synthesis of lightweight interactive environments and combining Code-Native Reward based on program states.

In the experiments, this paper demonstrates significant efficiency advantages and verifies the transferability of the synthetic environment to real scenarios. Compared to training directly in real applications, the synthetic environment significantly reduces interaction latency and the repetitive costs of estimation Rollout and reward verification. At the same time, it achieves a similar success rate on unseen real WeChat mini-programs as training in real environments. This result provides valuable experimental evidence for using low-cost synthetic environments to replace part of the training of real GUI reinforcement learning.

Critical weaknesses

The method of this paper is relatively limited in terms of innovation. Interactive environment synthesis, programmatic rewards, and using the synthesized environment to train GUI Agents are ideas that have been explored in related studies such as InfiniteWeb and VeriEnv. Although this paper generates task-conditioned environments based on the interaction trajectories of real mobile applications, overall it still leans towards integrating and extending existing technologies in mobile GUI scenarios. Its independent method innovation is not particularly prominent.

The verification of the behavioral fidelity of the synthetic environment is still insufficient. This paper only reconstructs the interaction states related to the task, which may simplify the error branches, exploration difficulty, and failure recovery processes in real applications. However, Figure 2 mainly assesses the complexity of the environment through the length of successful trajectories, which is not sufficient to fully prove that the synthetic environment retains the interaction difficulty and behavioral characteristics of the real task.

The representativeness and generalization of the experiments still have certain limitations. Although this paper verified the transfer effect on real WeChat mini-programs that it has not seen, the initial task selection criteria and performance analysis of different task categories are still insufficient. Therefore, it is difficult to further determine whether the experimental results are affected by task selection bias and the applicability of this method in more complex tasks and other GUI scenarios.

Questions for authors

InfiniteWeb has studied interactive environment synthesis, programmatic rewards, and the use of synthetic environments to train GUI Agents. Although the authors discuss the differences from existing work in Section 2 of this paper, could they further clarify where the main independent method innovations of GUI-GENESIS lie, apart from the scenario difference of targeting real mobile applications?

GUI-GENESIS only reconstructs the task-related interactive environment, while Figure 2 mainly assesses the complexity of the environment through the length of successful trajectories. Could the authors further explain to what extent the synthetic environment retains the error branches, exploration difficulty, and failure recovery of interactive behaviors in real applications? Is a similar trajectory length sufficient to support the conclusion of task-related behavior fidelity?

The experiments in this paper are mainly based on a self-built task set of WeChat mini-programs. Could the authors further explain the selection criteria for the initial tasks, and whether the difficulty of task reconstruction affects the task selection? Additionally, could they provide the success rates of the real environments on different task categories to further verify the generalization of the experimental conclusions?

Reviewer_BPHUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GUI-GENESIS generates task-specific web environments from real mini-app interaction traces. A multimodal code model produces the interface, mocked backend, and state-based subgoal rewards. Model self-reflection and Playwright tests check the generated code. Qwen-2.5-VL-32B is then post-trained with GRPO and evaluated on 149 tasks from unseen real mini-apps. Synthetic/code-reward training reaches 42.28% human-annotated success, compared with 40.94% for real-environment/VLM-reward training and 36.91% for the base model. The paper also estimates lower rollout and verification costs.

Critical strengths
  1. Table 1 measures an environment interaction latency reduction from 4.81 s to 0.42 s and a smaller full-rollout speedup. The paper separates recurring costs from preparation costs, so the savings have a stated accounting basis.
  2. Evaluation uses unseen real mini-apps. The synthetic-environment/VLM-reward condition also provides a control for changing the environment while retaining the reward source. This is stronger evidence of transfer than evaluation only in generated environments.
Critical weaknesses
  1. The combined static and dynamic checks do not independently establish reward semantics. The dynamic test requires reward>0\mathrm{reward}>0 (§4.5), which may represent partial progress. Table 3 reports 17.35% unverified tasks across all 1,118 training and evaluation construction tasks; it does not give the training-only rate. Final candidates are retained when verification fails. Appendix I finds checker/test faults in seven of ten inspected unverified cases and environment defects in three, so unverified status is not itself evidence of a defective environment. The paper does not report an independent audit of whether reward=1\mathrm{reward}=1 corresponds to full task completion, including checks of incorrect terminal states.
  2. Table 2 demonstrates transfer, but it does not quantify how reliably real-task performance is preserved across training runs. The synthetic/code condition is 1.34 percentage points above real/VLM training and 0.67 percentage points above synthetic/VLM training, approximately two and one tasks respectively. The paper appropriately stops short of a general superiority claim. Paired task outcomes and independent training runs would help determine whether the observed differences are robust. For any tasks whose completion cannot be judged from final screenshots, Appendix F leaves the use of additional evidence unclear.
Questions for authors
  1. Under an independent audit of verified and unverified environments, how often does reward=1\mathrm{reward}=1 correspond to an incorrect or only partially completed task? This directly affects the reliability of the training signal.
  2. Are paired task outcomes and any independent training repeats available for the three trained conditions? For tasks whose completion cannot be inferred from final-state screenshots, was any additional evidence, such as execution logs or backend state, used to determine success?
  3. Please specify the actual GRPO training budgets, including optimization steps and total rollouts, as well as the commercial VLM reward/evaluator versions. Which model inference and policy-optimization costs are excluded from the estimates in Table 1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.