- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes WM-R1, a new approach to training GUI agents with reinforcement learning, where a pretrained world model replaces real environment interactions during training. Instead of relying on real Android devices to collect trajectories, the method uses Code2World to simulate the next GUI state after each action, allowing the agent to perform multi-step rollouts and learn through GRPO. The agent can also use the world model during reasoning to preview candidate actions and adjust its decisions based on the simulated outcomes. The authors further introduce a reward function that considers task success, trajectory length, and world-model usage. Experiments on several GUI benchmarks show promising improvements over the selected baselines at different model scales. The paper also includes multiple ablation studies to examine the contributions of individual components, along with training cost comparisons and additional experiments on Qwen3-VL models to evaluate the effectiveness and generality of the approach.
Critical strengths
One strength of this paper is that it addresses a practical challenge in GUI agent RL: collecting multi-step trajectories through real environment interactions is costly and difficult to scale. WM-R1 uses a world model to simulate these interactions during training and allows the agent to preview possible actions during inference. While using world models for agent training and reasoning is not entirely new, I think applying this idea to multi-step GUI RL is a useful contribution. It provides a promising alternative to training agents through real environment interactions. Another strength is the relatively thorough experimental evaluation. The paper reports consistent improvements across several GUI benchmarks and model sizes. The authors also examine the effects of CoT, trajectory-length rewards, and world-model usage rewards through ablation studies, along with additional hyperparameter and cross-model experiments. I particularly appreciate the training time comparison, as it helps demonstrate the practical benefit of the approach beyond improvements in task success rates. Taken together, these results give me reasonable confidence in the value of the proposed method.
Critical weaknesses
The main contribution of this paper is to use the World Model to replace the real environment for multi-step reinforcement learning training, and to utilize the World Model to assist decision-making during the reasoning stage. However, the existing ablation experiments mainly focus on components such as CoT, trajectory length rewards, and World Model invocation rewards, lacking experiments that can directly verify the independent contributions of these two core designs. Although this paper achieved good results on multiple Benchmarks, some GUI Agent methods that have already demonstrated higher performance were absent from the main experiments, such as MobileRL, GUI-Owl, and MAI-UI. The models of these works have similar scale to some of the settings in this paper, especially showing strong performance on AndroidWorld. Although the training data, training costs, and evaluation settings of different methods may vary, and a completely fair comparison cannot be made directly, the authors should at least discuss these methods and explain the advantages and limitations of WM-R1 compared to them. The current experiments can support the conclusion that WM-R1 outperforms the Baseline selected in the paper, but they are not sufficient to support a more general SOTA claim. This makes the performance positioning of this paper in the existing GUI Agent research less clear.
Questions for authors
The core idea of this paper is to use the World Model to replace the real environment for multi-step reinforcement learning training. However, both the training stage and the inference stage employ the World Model. Therefore, it is currently difficult to determine how much benefit the simulation training has brought. Can the author directly deploy the Agent trained by the World Model to the real GUI environment, without invoking the World Model to preview the actions, to evaluate its task success rate and compare it with the corresponding Baseline? I believe this experiment can more directly verify whether the strategies learned in the simulated environment can be transferred to the real environment, and also help to distinguish the benefits of the training stage from the improvement brought by the additional use of the World Model in the inference stage.
I have some questions regarding the selection of the Baseline in the paper and the evaluation settings of some Benchmarks. Firstly, existing methods such as MobileRL, GUI-Owl, and MAI-UI reported significantly higher success rates on AndroidWorld, with some models exceeding 70%, but these methods were not included in the main comparative experiments. The authors could explain the reason and further clarify the applicable scope of the SOTA statement in the paper? Secondly, the paper describes both ScreenSpot-Pro and ScreenSpot-V2 as mobile Grounding evaluations, but ScreenSpot-Pro itself is mainly targeted at desktop professional software and does not have a standard Mobile subset. Could the authors clarify the specific data division used here? Additionally, I also hope to further confirm whether the final results of AndroidWorld come from official task checks in the real Android environment and whether all methods have adopted the same evaluation protocol. These issues will directly affect my judgment on the comparability of the experimental results and the performance conclusions of the paper.
Dear Reviewer, thank you again for your thoughtful comments and suggestions. In our response and revised manuscript, we have added experiments to distinguish the contributions of world-model training and inference, expanded comparisons with stronger baselines, and clarified the evaluation protocols. We would greatly appreciate any further thoughts you may have when you have a chance to review these updates.