acceptodds
Under review as a conference paper at ICLR 2027

Knowing When to STOP, RECOVER, and SEARCH: A Modular Framework for GUI Automation

Abstract

Autonomous GUI agents face two fundamental challenges: early stopping, where agents prematurely declare success without verifiable evidence, and repetitive loops, where agents cycle through the same failing actions without recovery. Existing agent planners track progress through generated subtasks, yet in GUI environments, completing planned steps does not guarantee that the user's requirements are met. Moreover, existing anti-loop mechanisms typically detect only local repetition and lack mechanisms to escalate recovery when the agent remains stuck. To this end, we present GUI-Pro Agent, a modular GUI agentic framework organized around three decisions: when to Stop, Recover, and Search. To prevent premature termination, a mandatory Completeness-Aware Planner tracks UI-observable success criteria throughout execution and independently verifies completion claims before termination. To recover from stalled agent action loops, a mandatory Loop Breaker detects repeated ineffective agent actions and progressively forces the agent to change its action strategy. When the current observation is insufficient to determine a viable workflow, an on-demand Search Agent supplies external procedural knowledge to the agent. To validate GUI-Pro Agent, we evaluate it on two benchmarks containing Linux and Windows tasks with five LLM backbones, achieving top performance on both (77.5% on OSWorld and 61.0% on WindowsAgentArena). Notably, three of the five backbones surpass human performance (72.4%) on OSWorld in a single pass. Ablation studies further show that completion checking, loop recovery, and search provide complementary gains that vary with backbone capability and interaction budget, while the Loop Breaker nearly halves wasted steps for loop-prone backbones (e.g., Gemini 3 Flash). Moreover, transferring three proposed mechanisms to an existing GUI agent (CoAct-1) also improves it by over 5% to 70.3% on a controlled OSWorld subset, showing the strong portability of these components.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.