The World Doesn't Wait: Windowed Action Commitment for Real-Time GUI Agents

Real-time GUI agents choose actions from screenshots while the interface continues to change. An action that is correct for the observed state can therefore be invalid when executed. Preparing several actions reduces repeated inference, but leaves later actions tied to increasingly old observations. We study how to preserve useful decisions within such a plan. Our central insight is that task choices can outlive their original execution contexts. We introduce windowed action commitment, which binds a bounded action sequence to observable targets. The executor uses fresh visual feedback to retain compatible actions, skip expired independent prefixes, and rebind moved targets, replanning when the remaining commitments cannot be used. Compared with single-step execution in the faster settings of two instrumented Android tasks, successor reuse raises correctness from 12% to 32% at equal mean model-call count; target rebinding raises correctness from 53.0% to 71.7% while reducing calls by 53.1%. Cross-model diagnostics distinguish constructing a usable window from executing it successfully. These findings identify selective reuse of prepared decisions as a mechanism for maintaining task progress while interfaces change.