From Tapping to Hopping: Augmenting Mobile GUI Agents with App-Native Deeplinks
Abstract
Mobile GUI agents complete tasks using GUI actions like taps and swipes. These actions are broadly applicable across applications, but reaching a task-relevant screen can require many intermediate navigation steps, lengthening trajectories and creating more opportunities for error. Fortunately, many Android applications expose deeplinks—external entry points to specific in-app screens—that provide a higher-level navigation interface. A single deeplink call can replace a sequence of screen-by-screen GUI actions. We therefore introduce hybrid interaction, using deeplinks for direct navigation and GUI actions for other on-screen operations and fallback. To enable this, we discover candidate deeplinks through static analysis, validate them on real devices, and describe their observed landing screens. This process creates a verified and grounded deeplink catalog that pairs each working deeplink with a description of its landing screen. Using this catalog, we introduce GUI-Hopper, a deeplink-augmented GUI agent that learns when to use a deeplink and when to rely on GUI actions. On MobileWorld, GUI-Hopper achieves higher task success with fewer steps than its GUI-only counterpart; its gains persist even without deeplinks at inference. On MobileWorld-Real, it also improves task success in commercial applications on real devices, further demonstrating the benefits of hybrid interaction.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.