MobileViews: Scalable and High-Fidelity Data Collection for Mobile GUI Agents
Abstract
Training capable mobile GUI agents requires diverse interaction data, yet collecting such data across a broad range of apps remains challenging. We introduce MobileViews, a mobile GUI dataset built through a scalable hardware–software pipeline combining native SoC clusters with VLM-assisted traversal and selective human fallback. It contains over 1.2M unique screenshot–view hierarchy pairs and associated interaction traces from more than 30K Android apps on Google Play. To assess the dataset’s utility, we construct MobileViewsIDM, comprising over 81K inverse dynamics samples from 26K apps for training models to understand GUI transitions. Across the three evaluated VLMs, training on MobileViewsIDM generally outperforms training on prior mobile GUI datasets that are limited in app diversity and scale. Our hardware–software co-design also improves native mobile app compatibility from 56% to 96% and increases the traversal success rate from 16.7% to 89.6%. MobileViews will be fully open-sourced.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.