acceptodds
Under review as a conference paper at ICLR 2027

DreamUI: Scaling Interactive Mobile Agent Environments with Generative UI World Model

Abstract

Training mobile agents for diverse, long-horizon tasks requires scalable interactive environments. However, real apps require authentication and permissions and are difficult to reset, while mock apps rely on predefined data and transition logic that limit task coverage and interaction diversity. In this paper, we present DreamUI, a scalable reinforcement learning framework that leverages a Generative UI World Model as the training environment. Our core idea is to generate diverse environments while preserving task consistency. The UI world model preserves task-essential facts while introducing plausible variations in unspecified content, exposing agents to richer training contexts. It also generates task-relevant states conditioned on instructions, reducing reliance on predefined app databases when expanding training tasks. Given a task instruction, the current observation, and an agent action, the UI world model generates HTML that renders into the next UI observation, forming a closed interaction loop. To address imperfect environment predictions and sparse task-level feedback, we introduce Environment-Aware Trajectory Masking to exclude corrupted rollouts and a Rubrics-Based Reward System that provides graded rewards through fine-grained task criteria. We train the UI world model through supervised fine-tuning followed by step-wise reinforcement learning. Our DreamUI-World-8B achieves average Instruction Accuracy and Visual Similarity scores of 82.9% and 75.6%, respectively. As for mobile-agent training, RL in the simulated environment raises our DreamUI-4B's success rates to 73.3% on AndroidWorld and 36.2% on MobileWorld. These results demonstrate that our UI world models can serve as effective training environments for mobile-agent RL. We will release our code, data, and models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Mobile Agent, Mobile World Model, Reinforcement Learning.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Hi5WOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents DreamUI, a framework that uses a generative UI world model as a reinforcement learning environment for training mobile agents. The main motivation is clear and practical: training directly on real apps is often cumbersome due to reset difficulties, login requirements, and the cost of building and maintaining large-scale mock application environments. DreamUI addresses this by generating the next interface state from the task instruction, the current screenshot, and the agent’s action. Specifically, the model produces HTML, which is then rendered into the next UI, enabling an interactive simulated environment for agent training.

To improve training stability, the paper proposes two additional designs. Environment-Aware Trajectory Masking aims to filter out trajectories affected by errors in the world model, reducing the impact of low-quality simulated interactions. Rubrics-Based Reward System provides more fine-grained supervision by decomposing tasks into intermediate criteria, so that agents can receive partial rewards even when they do not fully complete the task. Empirically, DreamUI-World-8B achieves 82.9% instruction accuracy and 75.6% visual similarity on MWMBench, suggesting that the generated environments are reasonably faithful. When used for training, DreamUI-4B improves downstream performance from 69.8% to 73.3% on AndroidWorld and from 29.0% to 36.2% on MobileWorld.

Critical strengths
  1. The paper is well written, with a clear motivation and a strong focus on an important problem. It does a good job of identifying the practical challenges of scaling training in real mobile environments, as well as the limitations of existing mock-app setups, which often depend on manually curated databases and hard-coded state transitions.

  2. The proposed approach is quite novel. It introduces a UI World Model, built around generated HTML, to serve as a closed-loop reinforcement learning environment. This makes it possible to dynamically generate interface states and task progressions without relying on interaction with real physical devices.

  3. The paper also directly tackles the reliability issues that come with simulated environments. Trajectory Masking is designed to reduce the negative impact of incorrect environment feedback during policy training, while the Rubrics-Based Reward mechanism helps address the sparse-reward problem in long-horizon tasks by providing more fine-grained supervision.

Critical weaknesses
  1. The paper motivates generative UI world models mainly through their advantages over manually built mock apps, but this central comparison is not evaluated systematically. In particular, the paper does not provide matched comparisons on app coverage, realism, long-horizon reliability, and downstream RL performance. Although the introduction states that mock-app training saturates at around 3,000 tasks while DreamUI scales to 5,000 tasks, the corresponding quantitative results are not reported in the main experiments or appendix.

  2. It remains unclear whether DreamUI simulates an underlying application state machine or primarily generates locally plausible action-conditioned screens. Since the model treats the current screenshot as the complete state, it may suffer from state aliasing: visually identical screens with different latent application states cannot be distinguished. Moreover, the framework does not appear to maintain an explicit navigation stack or persistent application state. Consequently, when the agent performs a Back action, there is no guarantee that the model restores the actual previously visited state, including its content, scroll position, and task-relevant modifications, rather than generating another plausible-looking screen. The paper does not introduce explicit constraints or evaluations for state reversibility, back-stack fidelity, or cycle consistency. Therefore, the current visual-fidelity metrics are insufficient to establish that the generated transitions form a closed and logically consistent application state machine.

Questions for authors
  1. Could the authors provide a controlled comparison between DreamUI and mock-app environments under matched task counts and training budgets, including app coverage, environment realism, long-horizon reliability, and downstream RL performance?

  2. Does DreamUI maintain any latent application state or navigation stack beyond the current screenshot? For example, after �0��0�, how does the model ensure that it restores the original state A, including its content, scroll position, and previous modifications?

  3. (suggestions for future work) As a broader question, concurrent work such as REA (https://github.com/morluto/rea) has substantially improved the realism and scalability of automatically generated mock apps. Under this development, what unique advantages does a generative UI world model retain over executable mock apps, particularly considering their differences in diversity, transition correctness, controllability, latency, and verifiability? Do the authors view these two paradigms as competing approaches, or could they be combined into a hybrid environment?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Hi5W (1/2)

Dear Reviewer Hi5W,

Thank you for the thoughtful comments. It is encouraging that you found our paper well written, with a clear motivation that targets the limitations of both real mobile environments and existing mock apps. We are grateful that you considered our HTML-based UI world model a novel closed-loop RL environment, and that you highlighted how trajectory masking and the rubrics-based reward mitigate unreliable feedback and sparse rewards in long-horizon tasks. We respond to each of your comments below.

[Weakness 1 / Question 1] Controlled Comparison between Mock Apps and DreamUI

We thank the reviewer for suggesting this comparison. We conduct an additional experiment using 3,000 training tasks in each environment, with application types matched as closely as possible across the two environments. Both task sets cover five functional categories: System and Utilities, Productivity and Lifestyle, Communication and Social, Media and Creation, and Browsing and Navigation. The mock-app environment covers 34 apps, while DreamUI-World covers 31 apps. During RL, we set an identical maximum number of interaction turns for both environments and train each policy until performance saturates.

Success Rate (%)

EnvironmentAndroidWorldMobileWorld (GUI-Only)
Mock App71.129.9
DreamUI-World72.433.3

As shown in the table, with matched task counts and training budgets, DreamUI achieves success rates of 72.4% on AndroidWorld and 33.3% on MobileWorld, compared with 71.1% and 29.9% for the mock-app environment, respectively. These results provide additional evidence that DreamUI is an effective environment for downstream agent RL. We further discuss environment realism and long-horizon reliability below.

Environment realism. The mock-app baseline reproduces application interfaces and behaviors through implemented logic and configured data. DreamUI instead learns action-conditioned UI transitions from real mobile interaction trajectories and generates interfaces conditioned on the task, current observation, and action. It is designed to preserve task-essential facts while introducing plausible variations in unspecified content and appearance, as illustrated in Figure 1. The two approaches thus offer complementary strengths: mock apps provide explicitly specified interfaces and behaviors, while DreamUI supports contextual variation and instruction-driven construction of task-relevant screens.

Long-horizon reliability. Mock apps maintain application state through explicit program logic, making transitions within their implemented scope inspectable and controllable. In DreamUI, the multi-step trajectories in Appendix E illustrate how generated interfaces preserve task-relevant information and reflect successive actions as tasks progress. To support reliable policy training, environment-aware trajectory masking additionally examines complete rollouts for action–outcome mismatches, missing task-relevant entities, inconsistent transitions, and persistent rendering failures, excluding flagged trajectories from policy optimization. This mechanism allows DreamUI to use generated multi-step interactions for agent RL while screening out detected environment errors.

AuthorsOpenReview ↗

Response to Reviewer Hi5W (2/2)

[Weakness 2 / Question 2] State Restoration Capability of DreamUI-World

We thank the reviewer for raising this interesting question. We considered incorporating historical observations to support accurate state restoration, but ultimately chose not to adopt history conditioning as the default setting for agent RL, based on its computational cost and the observed downstream results. We describe the evaluation behind this choice below.

To examine state restoration, we provide previously visited observations as additional context to DreamUI-World without updating its parameters. Appendix J (Figures 22–24) illustrates three backtracking trajectories. With historical context, the model closely reconstructs earlier interfaces, preserving key visible content, layout, and item ordering in these examples. These results show that DreamUI-World can use historical observations to support state restoration in a training-free manner.

For agent RL, the essential requirement is to preserve task-relevant facts and valid action-dependent transitions. Exact reproduction of every incidental UI detail is not required. Meanwhile, retaining extensive observation history increases context length and can make rollout generation substantially slower. We therefore examine whether the additional context improves downstream policy learning by comparing agent RL in DreamUI environments with and without history conditioning.

Context SettingAndroidWorldMobileWorld (GUI-Only)Time / Traj. (s)
w/o history73.336.2358
w history73.734.2484

History conditioning increases the time per trajectory by 35.2%, without a clear overall improvement in downstream performance in this comparison. We additionally find that backtracking actions account for only approximately 3% of all actions in our RL rollouts. Together, these observations motivate our default RL setting without historical observation context, balancing rollout efficiency and downstream performance in the evaluated setting. History conditioning remains an option when restoring previously visited interfaces is a priority.

We appreciate the reviewer’s emphasis on state restoration and consistency across interactions. While this work focuses on using world models as effective training environments for agent RL, we recognize the importance of efficient history representations for accurate state restoration. Developing memory mechanisms that preserve task-relevant state and support fast, reliable backtracking remains a broader research challenge in world modeling, and we view this as an important direction for future work.

[Question 3] Discussion for REA

We thank the reviewer for highlighting REA. Advances in automatic app reconstruction reduce the effort required to build executable environments. We view executable mock apps and generative UI world models as complementary approaches: mock apps provide a stable application framework with explicit state and executable operations, while world models introduce task-conditioned GUI dynamics and contextual variation.

We see an agentic workflow as a promising way to combine them. Given application context describing supported functionality and workflows, an environment-building agent could establish the application scaffold and initial configuration. During interaction, an orchestration agent could invoke the UI world model as a state-conditioned interface-generation tool, generating the next UI from the current application state, task instruction, and GUI action. The executable component would maintain application state across interactions.

Task-relevant content could be instantiated on demand and retained in this application state, avoiding the need to manually populate or modify a predefined database when expanding task coverage. This would allow interfaces and contextual details to vary within a consistent application framework, combining the state management of executable environments with the flexible GUI generation of world models.

My questions have been clarified, and I have no further questions. Since I had already given a score leaning toward acceptance, I will keep my rating unchanged.

We sincerely thank the reviewer for the time and effort devoted to reviewing our work. We are glad to hear that our responses have clarified your questions. Your valuable feedback has helped us improve the quality and clarity of our manuscript.

Reviewer_P9xEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces DreamUI, which trains a next-state prediction model for phone UI HTML and then trains an agent inside this world model with RL. To train the world model, large interaction trajectories are converted to HTML and filtered based on visual quality. A reasoning model is then trained on the collected data with SFT, followed by RL to generate realistic pages. The paper shows that the resulting world model produces more realistic trajectories that better follow instructions than baselines. An agent trained with RL inside the world model also outperforms baselines in mobile UI settings.

Critical strengths
  1. The world model achieves strong empirical results on the world model metrics in MWMBench, outperforming other world model approaches that rely on either direct image generation or next-state HTML prediction.
  2. The strength of the world model is particularly demonstrated by the ability to train a policy within it and improve downstream performance on AndroidWorld and MobileWorld relative to baselines. Training with 5k instructions also suggests that the pipeline may be more scalable for synthesizing training data than prior approaches.
  3. The work introduces several important techniques for data quality filtering during world model training and RL training. The paper empirically demonstrates that these filtering steps are important for the RL phase.
  4. The dense rubric reward empirically leads to substantial gains during RL training.
  5. The paper includes details about all the prompts used in the supplementary material.
Critical weaknesses
  1. The world model itself has limited novelty relative to prior works such as Code2World and gWorld. These approaches also appear to train world models that predict the next state as an HTML representation. Code2World also uses a similar RL training phase to DreamUI for training the world model. Thus, it is unclear which components of DreamUI contribute to its improvements over prior work. For example, the gains could be due to better dataset curation, which is orthogonal to the proposed method. For the world model training approach to constitute a meaningful contribution, the paper needs to empirically identify which aspects of the method drive improvements over baselines.
  2. The additional training in DreamUI appears to provide only marginal benefits based on the results in Tables 2 and 3. Most of the gains seem to come from the dense rubric-based reward rather than the ability to scale training data within the DreamUI world model itself. Training in the DreamUI world model yields only a 1.3 percentage point improvement, which could easily be attributed to evaluation or training noise. Furthermore, this dense RL reward formulation is not specific to DreamUI and could also be applied directly in a real environment.
  3. The RL evaluations do not report standard deviations, making it difficult to assess the significance of the empirical improvements. The paper should report results across at least three random seeds for Table 2, given the potentially high variance in RL training and the small performance margins.
  4. A central claim of the work is that DreamUI provides a scalable RL training system. However, the paper does not evaluate how performance changes with the number of RL training tasks. Figure 9 does a good job of demonstrating this relationship for world model training, but a corresponding analysis is missing for RL training.
  5. The main paper should provide more details about the RL training environment instantiated through the DreamUI world model. A primary concern is the potential overlap between the 5k training instructions and the evaluation sets. How is data contamination prevented? Are the 5k training instructions simulated in unseen applications? What is the application coverage and overall diversity of these instructions? These dataset details are important for understanding how RL training transfers to downstream benchmarks. Furthermore, it is unclear why training in DreamUI produces relatively small gains. Is this due to a mismatch between the 5k training instructions and the downstream tasks, differences in application coverage, suboptimal convergence in the simulated environments, or other factors?
  6. The paper does not present RL learning curves for training within the DreamUI world model. These are necessary to substantiate the claim that DreamUI enables stable RL training.
Questions for authors

See the weaknesses section.

7 comments
AuthorsOpenReview ↗

Response to Reviewer P9xE (1/2)

Dear Reviewer P9xE,

Thank you for your valuable feedback. We are pleased that you recognized our world model’s strong performance on MWMBench and its ability to support scalable policy training. We also appreciate your acknowledgment of the effectiveness of our data filtering and dense rubric reward. We address your questions point by point below.

[Weakness 1] Novelty and Contribution of DreamUI-World

Both Code2World and gWorld predict the next UI state as HTML, and DreamUI-World adopts a similar representation. Code2World uses its world model for inference-time look-ahead, while gWorld uses it to estimate candidate action values. Our goal is to make the world model reliable enough to serve as a closed-loop environment for multi-step agent RL. This goal motivates three design choices.

  • Agent-Collected Online Trajectories. Code2World is built only from Android Control, while gWorld draws on four offline datasets. In contrast, 124,420 of our 200,656 training samples are collected by deploying strong policy agents to interact with real mobile environments. Such agent-driven collection broadens the range of apps and interactions available for learning UI transitions.
  • Fine-grained HTML Filtering. Code2World filters HTML annotations using SigLIP similarity between rendered and ground-truth screenshots, while gWorld reports no explicit filtering step. We combine semantic filtering with separate VLM-based checks for layout, content, and style to catch defects such as garbled text and overlapping elements. This helps retain HTML annotations that faithfully represent the ground-truth UI states.
  • Probability-weighted Reward. Directly decoded judge scores can change when token probabilities shift near a rating boundary. We instead compute the expected score over the judge’s rating tokens (Lines 259–264, Eq. 3). Across three runs, this lowers mean sample-wise deviation from 1.17 to 0.28 and run-level variance by 79.1%, providing a more stable reward signal for RL (Appendix C.3, Table 9). Figure 9 in our revised manuscript compares training reward curves under both formulations. Probability-weighted judging shows a more sustained upward trend than discrete judging, supporting its effectiveness for stable RL optimization.

To isolate each component's contribution, we compare against Code2World and gWorld on MWMBench. All 8B models share Qwen3-VL-8B-Instruct as the base model. The results of our variants come from Table 8 (Appendix C.2):

ModelTraining DataReasoningRLIAcc. / Sim. ↑RF ↓
Code2World-8B80K (offline)✗✓63.2 / 70.92.4
gWorld-8B260K (offline)✓✗76.8 / 71.71.4
Ours (SFT only)200K (offline + online)✗✗72.7 / 71.82.5
Ours (SFT only)200K (offline + online)✓✗76.9 / 74.01.7
DreamUI-World-8B200K (offline + online)✓✓82.9 / 75.60.5

As shown in the table, our SFT-only variants already achieve higher IAcc. and Sim. than Code2World-8B and higher Sim. than gWorld-8B with fewer training samples, while step-wise RL brings a further +6.0 IAcc., indicating that the gains come from both the curated data and our training recipe.

[Weakness 2] Marginal Benefits on AndroidWorld

We thank the reviewer for raising this point. To examine whether the modest gains observed with GUI-Owl-1.5-4B also hold for another starting policy, we apply the same DreamUI training pipeline to Qwen3.5-9B and repeat the reward ablation.

Base ModelSettingAndroidWorldMobileWorld (GUI-Only)
GUI-Owl-1.5-4BBaseline69.829.0
GUI-Owl-1.5-4BDreamUI (Binary Naive Reward)71.1 (+1.3)32.7 (+3.7)
GUI-Owl-1.5-4BDreamUI (Graded Rubrics Reward)73.3 (+3.5)36.2 (+7.2)
Qwen3.5-9BBaseline57.818.0
Qwen3.5-9BDreamUI (Binary Naive Reward)68.1 (+10.3)27.4 (+9.4)
Qwen3.5-9BDreamUI (Graded Rubrics Reward)70.3 (+12.5)30.8 (+12.8)

On Qwen3.5-9B, RL training in DreamUI with a binary 0/1 reward improves AndroidWorld and MobileWorld by +10.3 and +9.4 percentage points, respectively. Graded rubric rewards add a further +2.2 and +3.4 points, bringing the total gains to +12.5 and +12.8. These results show that substantial transfer gains can be obtained through RL in DreamUI even without graded rewards. Graded rewards further improve performance for both backbones, while their relative contribution depends on the starting policy.

We agree that rubric-based rewards are not specific to DreamUI and could also be computed on real-device trajectories. DreamUI provides an interactive training environment that supports task expansion without requiring real-device interaction during agent RL. The reward ablation measures the additional benefit of graded rewards within this environment, while the Qwen3.5 results demonstrate substantial transfer gains even with binary rewards.

AuthorsOpenReview ↗

Response to Reviewer P9xE (2/2)

[Weakness 3] Stability of RL Training

We follow your suggestion and repeat the full RL training with three random seeds for both base models, keeping the tasks, reward, and evaluation protocol unchanged. The table reports Pass@1 success rates (%) as mean ± sample standard deviation across different random seeds.

Base ModelAndroidWorldMobileWorld (GUI-Only)
GUI-Owl-1.5-4B73.73 ± 0.7535.97 ± 0.87
Qwen3.5-9B70.23 ± 1.3029.33 ± 1.33

The results show small fluctuations across different random seeds, and every run improves over its corresponding baseline on both benchmarks, indicating the stability of RL training in DreamUI.

[Weakness 4] Performance Change with Training Task Scaling

We thank the reviewer for raising this point. Figure 1 shows the average success rate across AndroidWorld and MobileWorld as the number of RL training tasks increases. We provide the per-benchmark breakdown below at five task counts, plus the 0-task GUI-Owl-1.5-4B baseline. Each policy is trained until performance saturates.

Success Rate (%)

# RL Training Tasks01,0002,0003,0004,0005,000
AndroidWorld69.870.771.672.473.373.3
MobileWorld (GUI-Only)29.031.632.533.336.236.2
Avg.49.451.252.052.954.754.7

As shown in the table, the agent performance improves overall as the number of training tasks increases, with gains of +3.5 on AndroidWorld, +7.2 on MobileWorld, and +5.3 on average from 0 to 5,000 tasks, demonstrating the scalability of DreamUI as an RL training environment.

[Weakness 5] RL Task Coverage and Transfer

We thank the reviewer for these questions. We address each point below.

  • RL Training Environment and Train–test Separation. In Appendix B.1, we describe how we construct the tasks and select compatible initial screenshots. Specifically, we give an LLM app profiles that describe what users can accomplish in each app and how it fits into common workflows. The LLM then generates instructions that specify a user goal and the criteria for completing it. Given the limited availability of mobile app interaction data, training and evaluation may share applications. However, during task synthesis, we use an LLM to identify and remove instructions that exactly match those in the evaluation sets, thereby mitigating the risk of evaluation-task leakage into the training data. Regarding interaction horizons, we limit RL training rollouts to 16 turns and allow up to 50 turns during evaluation. The gains on AndroidWorld and MobileWorld suggest that RL training in DreamUI enables the policy model to learn generalizable GUI interaction skills.

  • Application Coverage and Task Diversity. Our 5K RL tasks cover 31 applications across five functional categories. Appendix G (Table 12) of the revised manuscript provides the complete list. The table below lists representative applications from each category.

CategoryRepresentative Applications
System and UtilitiesSettings, Clock, Files
Productivity and LifestyleCalendar, Notes, Expense Tracker
Communication and SocialContacts, Messages, Mail
Media and CreationCamera, Gallery, Music Player
Browsing and NavigationChrome, Maps

Our training set spans a wide range of goals, constraints, and workflows, from single-app tasks to cross-app tasks with explicit information dependencies (Appendix B.1), which ensures diversity in both task objectives and interaction patterns.

  • Explanation for Performance Gains. We first clarify that the relatively modest gain is mainly observed on AndroidWorld, where DreamUI improves GUI-Owl-1.5-4B by 3.5 percentage points, compared with 7.2 points on MobileWorld. To investigate why the AndroidWorld gain is smaller, we apply the same DreamUI training pipeline to Qwen3.5-9B. As mentioned in our response to Weakness 2, DreamUI improves this model by 12.5 percentage points on AndroidWorld and 12.8 points on MobileWorld. These results suggest that GUI-Owl-1.5-4B, having already undergone GUI-specific training and achieved strong performance on AndroidWorld, may have limited room for further improvement under our current training setup. The substantial gains obtained with Qwen3.5-9B further demonstrate the effectiveness of our method.

[Weakness 6] RL Training Curves

Thank you for this suggestion. We have added GRPO training curves to Appendix I (Figure 21) of the revised manuscript, showing policy entropy, policy-gradient loss, training reward, and gradient norm during agent RL in DreamUI.

As shown in Figure 21, the training reward increases overall, policy entropy remains broadly stable, and policy-gradient loss and gradient norm stabilize after early fluctuations. Together, these trends support our claim that DreamUI enables stable RL for mobile agents.

Dear Reviewer P9xE,

We hope you are doing well. Thank you again for your careful review and constructive feedback. As the discussion period nears its close, we would like to kindly follow up on our responses above. These include further analyses of our world-model training components, additional agent-training results with Qwen3.5, and multi-seed evaluations, along with clarifications on task scaling, training-task coverage, and RL stability.

If our responses have addressed your concerns to your satisfaction, we would sincerely appreciate it if you would consider raising your rating in light of the additional results and clarifications. Please let us know if you have any remaining questions or concerns. We would be more than happy to provide further details and continue the discussion.

Best regards,

The Authors

Reviewer_B5jkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents DreamUI, a framework that uses a generative UI world model as a training environment for mobile GUI agents, enabling closed-loop reinforcement learning without interacting with real devices during training. DreamUI-World-8B is developed by fine-tuning Qwen3-VL-8B on a self-constructed dataset of around 200K UI state transitions, followed by step-wise reinforcement learning to improve the fidelity of rendered UI states. The resulting world model is then used to train GUI-Owl-1.5-4B with GRPO on approximately 5K synthesized tasks. DreamUI further introduces Environment-Aware Trajectory Masking and a Rubrics-Based Reward System to mitigate errors in simulated interactions and address sparse reward signals. Experimental results show that DreamUI-World-8B outperforms the baseline on MWMBench, while the trained agent achieves improved performance on AndroidWorld and MobileWorld.

Critical strengths
  1. Valuable motivation. Real-world applications are difficult to reset and control, while conventional simulated environments are often constrained by predefined databases, states, and transition logic. Developing scalable and interactive training environments is therefore a meaningful direction for online RL.
  2. A systematic end-to-end framework. The work covers the pipeline from training data construction and UI world-model development to downstream agent training via online RL. It also addresses error accumulation during long-horizon interactions, an important challenge when a learned world model serves as the training environment.
  3. A potentially valuable open contribution. The authors claim to release the data, models, and code, including the UI world model and a dataset of approximately 200K training examples. If made publicly available as described, these resources could facilitate further research on world-model-based GUI agent training.
Critical weaknesses

It remains unclear whether DreamUI-World can reliably model applications and UI transitions beyond its training distribution. Since the ability to generalize to unseen applications is important for scaling the proposed environment beyond the domains covered by the training corpus, evaluating the world model on genuinely held-out applications would provide stronger evidence for this claim.

Questions for authors
  1. How well does DreamUI-World generalize to applications and UI states that are absent from its training data?

  2. Which applications and application categories are covered by the 5K RL tasks mentioned in the paper?

5 comments
AuthorsOpenReview ↗

Response to Reviewer B5jk

Dear Reviewer B5jk,

Thank you for your insightful feedback. We appreciate your recognition of our motivation to build scalable, interactive environments for online RL. We are also glad that you acknowledged our systematic end-to-end framework, including its handling of error accumulation in long-horizon interactions. Moreover, we are committed to open-sourcing our data, models, and code to facilitate future research. Below, we address each of your questions in detail.

[Weakness / Question 1] Generalization to Unseen Applications

We thank the reviewer for raising this concern. We agree that evaluation on genuinely held-out applications is important for assessing generalization beyond the training corpus. To address this point, we have added rollout examples on Alipay, 12306, and McDonald’s in Appendix H (Figures 18–20). All three applications are held out from world-model training.

Starting from an initial screenshot of each application, DreamUI-World generates subsequent UI states conditioned on the agent’s actions. The examples cover several types of interaction:

  • Alipay (Figure 18). The agent accesses the bill list, opens the most recent bill, and reviews its transaction details.
  • 12306 (Figure 19). The agent enters the train number G2120 in the search interface and selects the matching result.
  • McDonald’s (Figure 20). The agent navigates to the coupon wallet and scrolls through the available offers to locate a target coupon.

Across these rollouts, DreamUI-World generates coherent, action-consistent transitions while preserving task-relevant content across successive states. These examples provide evidence of generalization to unseen applications. The observed behavior further suggests that DreamUI-World can draw on knowledge acquired during VLM training, such as common UI structures and interaction patterns, to support this generalization.

[Question 2] Application Coverage of RL Tasks

Our 5K RL tasks cover 31 applications, which we group into five functional categories. Representative applications of each category are listed below:

CategoryRepresentative Applications
System and UtilitiesSettings, Clock, Files
Productivity and LifestyleCalendar, Notes, Expense Tracker
Communication and SocialContacts, Messages, Mail
Media and CreationCamera, Gallery, Music Player
Browsing and NavigationChrome, Maps

Beyond single-app tasks, the task set also includes cross-app tasks with explicit information dependencies, where information obtained from one application is required to complete an operation in another (Appendix B.1). The complete list of applications and their categories is provided in Appendix G of the revised paper.

Dear Reviewer B5jk,

We hope you are doing well. Thank you again for your positive assessment and thoughtful feedback. As the discussion period nears its close, we would like to kindly follow up on our response above. It includes additional rollout examples on applications held out from world-model training, along with clarifications on the coverage and diversity of our RL training tasks.

If our response has addressed your concerns to your satisfaction, we would sincerely appreciate it if you would consider raising your rating in light of the additional evidence and clarifications. Please let us know if you have any remaining questions or concerns. We would be more than happy to provide further details and continue the discussion.

Best regards,

The Authors

Dear Reviewer B5jk,

Thank you again for the time and effort you have devoted to reviewing our paper. As the discussion period draws to a close, we would be grateful if you could take a moment to read our response and let us know whether it addresses your questions about generalization to unseen applications and the coverage of our RL training tasks.

If you have any further questions or concerns, we would be more than happy to discuss them. We appreciate your thoughtful feedback and consideration.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers and Area Chair

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their time and effort. The reviewers' constructive feedback has been invaluable in strengthening our work. We have carefully considered all comments, revised our manuscript accordingly, and provided point-by-point responses to each reviewer. We would like to highlight the following points.

  1. Strengths Highlighted by Reviewers
  • Clear Motivation and Novelty: The motivation was found valuable and practical, as real apps are hard to reset and control, while mock apps rely on curated databases and hard-coded transitions (Reviewers B5jk, Hi5W). Using an HTML-based UI world model as a closed-loop RL environment was considered quite novel (Reviewer Hi5W).
  • Systematic End-to-End Framework: The pipeline spans data construction, world-model training, and online RL, with Environment-Aware Trajectory Masking mitigating environment errors and the Rubrics-Based Reward System alleviating sparse rewards (Reviewers B5jk, Hi5W).
  • Strong Empirical Results: DreamUI-World outperforms prior image- and HTML-based world models on MWMBench, enabling agents trained within it to achieve consistent gains on AndroidWorld and MobileWorld (Reviewers P9xE, Hi5W).
  • Effective Training Techniques: Our data quality filtering and dense rubric reward were noted as important techniques with substantial gains (Reviewer P9xE).
  1. Revisions and Additional Experiments

To address the reviewers' questions, we conducted several new experiments during the rebuttal period and incorporated the following key additions and clarifications into our revised manuscript.

  • Contributions over Prior UI World Models (Reviewer P9xE): Unlike Code2World and gWorld, which use world models for look-ahead or action-value estimation, DreamUI-World serves as a closed-loop environment for multi-step agent RL. This motivates (1) agent-collected online trajectories (124K of 200K samples), (2) fine-grained VLM-based HTML filtering of layout, content, and style, and (3) a probability-weighted judge reward that cuts reward deviation from 1.17 to 0.28 (Appendix C.3, Table 9; Figure 9). Our SFT-only model already matches or exceeds both baselines in IAcc. and Sim. Step-wise RL further adds +6.0 IAcc. while reducing RF from 1.7 to 0.5 (Appendix C.2, Table 8).
  • Effectiveness of RL Training in DreamUI (Reviewer P9xE): Beyond the GUI-Owl-1.5-4B results in the paper, we additionally trained Qwen3.5-9B as the agent model and conducted RL in DreamUI. With only a binary reward, it improves AndroidWorld / MobileWorld by +10.3 / +9.4, and rubric rewards raise the gains to +12.5 / +12.8, confirming that substantial gains come from DreamUI itself. The smaller AndroidWorld gain of GUI-Owl-1.5-4B suggests limited headroom after extensive GUI-specific training.
  • Training Stability (Reviewer P9xE): Across three seeds, every run improves over its baseline (73.73 ± 0.75 / 35.97 ± 0.87 for GUI-Owl-1.5-4B; 70.23 ± 1.30 / 29.33 ± 1.33 for Qwen3.5-9B). GRPO training curves further show stable optimization (Appendix I, Figure 21).
  • Scaling with RL Tasks (Reviewer P9xE): Performance improves overall as RL tasks scale from 0 to 5,000, with gains of +3.5 on AndroidWorld, +7.2 on MobileWorld, and +5.3 on average (Figure 1).
  • Generalization to Held-Out Apps (Reviewer B5jk): Rollouts on Alipay, 12306, and McDonald's, all unseen during training, show coherent, action-consistent transitions that preserve task-relevant content (Appendix H, Figures 18–20).
  • RL Task Coverage and Train–Test Separation (Reviewers B5jk, P9xE): The 5K RL tasks cover 31 apps in five categories (Appendix G, Table 12), including cross-app tasks (Appendix B.1). We ensure no training instruction overlaps with evaluation sets.
  • Controlled Comparison with Mock Apps (Reviewer Hi5W): With matched task counts, app categories, and training budgets, RL in DreamUI reaches 72.4% / 33.3% on AndroidWorld / MobileWorld, versus 71.1% / 29.9% with mock apps, directly supporting our central motivation.
  • State Restoration (Reviewer Hi5W): With historical observations, DreamUI-World closely restores previously visited screens without training, preserving key content, layout, and item ordering (Appendix J, Figures 22–24). As this adds 35.2% rollout time without clear downstream gains and backtracking accounts for only ~3% of actions, we do not adopt history conditioning in our default RL setting. We also discussed hybrid environments combining world models with executable mock apps (e.g., REA).

All revisions and new results are marked in blue in our revised manuscript.

We believe these additions and clarifications have substantially strengthened our paper, and we are grateful that Reviewer Hi5W confirmed all questions have been clarified. We respectfully invite the Area Chair and reviewers to consider our responses, and we remain open to further discussion.

Best Regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.