AdapGuide: Turning Tutorial Knowledge into Evolving Guidance for GUI Agents

GUI agents rely on external tutorials to acquire procedural knowledge for unfamiliar software tasks. Existing methods retrieve relevant tutorials and convert them into action guidance. Yet tutorial guidance remains tied to its source interface and example, so it may not fit the current task and can become invalid when actions produce unexpected effects. Fixed guidance then propagates local errors through subsequent stages of task execution and may mistake an incorrect state for progress. To bridge this gap, we introduce AdapGuide, a training-free framework that reconstructs reusable operation–effect relations and revises task-specific guidance throughout execution. Each relation links a configured operation to its applicable conditions and expected interface change, enabling AdapGuide to bind tutorial knowledge to the objects and settings required by the current instruction and interface. During execution, mismatches between observed and expected effects trigger revisions to object bindings, operation settings, and subsequent stages. Experimental results across three benchmarks demonstrate the effectiveness and generalizability of AdapGuide, with average task score gains of 8.08–15.86 pp across five backbones on OSWorld-Verified, 14.73 pp on Windows Agent Arena, and 12.47 pp on DeskCraft.