Building Autonomous GUI Navigation via Agentic-Q Estimation and Step-Wise Policy Optimization

Recent advances in Multimodal Large Language Models (MLLM) have substantially driven the progress of autonomous agents for Graphical User Interface (GUI). Nevertheless, in real-world applications, GUI agents often face non-stationary environments, leading to high computational costs for data curation and policy optimization. In this paper, we introduce a novel MLLM-centered framework for GUI agents, which consists of two components: agentic-Q estimation and step-wise policy optimization. The former one aims to optimize a Q-model that can generate step-wise values to evaluate the contribution of a given action to task completion. The latter one takes step-wise samples from the state-action trajectory as inputs, and optimizes the policy via reinforcement learning with our agentic-Q. Our framework enjoys two favorable advantages: (i) all trajectories are produced by the policy itself, so that data collection costs are manageable; (ii) the policy update is decoupled from the environment, ensuring stable and efficient optimization. Empirical evaluations show that our framework endows Ovis2.5-9B with powerful GUI interaction capabilities, achieving remarkable performances on navigation and grounding benchmarks and even surpassing contenders with larger scales.