GUI-GENESIS: Automated Synthesis of Efficient Environments with Verifiable Rewards for GUI Agent Post-Training
Abstract
Post-training GUI agents in interactive environments is critical for developing generalization and long-horizon planning capabilities. However, training on real-world applications is hindered by high latency, poor reproducibility, and unverifiable rewards relying on noisy visual proxies. To address the limitations, we present GUI-Genesis, the first framework to automatically synthesize efficient GUI training environments with verifiable rewards. GUI-Genesis reconstructs real-world applications into lightweight, task-conditioned web environments using multimodal code models and equips them with code-native rewards, executable assertions that provide deterministic reward signals and eliminate visual estimation noise. Extensive experiments on the WeChat mini-app platform (serving over one billion active users) show that GUI-Genesis reduces interaction latency by 10× and cuts estimated recurring rollout and verification costs by over 98% compared with training on real mini-apps, excluding one-time preparation costs. Notably, agents trained with GUI-Genesis-synthesized environments achieve performance comparable to agents post-trained in real-world environments, demonstrating robust zero-shot sim-to-real transfer and establishing automated environment synthesis as a scalable and cost-effective alternative to direct real-world training. We open-source GUI-Genesis on our anonymous website: https://anonymous.4open.science/r/gui_genesis-B23B/.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.