One Step, Four Clocks: How GUI Agents Fall Out of Step

GUI agents can choose the right action and still fail because the interface changes while they think. We model the timing of agent responses through four events: an environment change, observation availability, decision commitment, and action receipt. These events distinguish failures in which relevant evidence is missed, a decision uses outdated evidence, an action becomes invalid after commitment, and an action arrives after the window of opportunity closes. We introduce ChronotopeGym to make these timing conditions experimentally controllable. We evaluate agent behaviors in the same task under three conditions: the interface waits; observations change while the response remains valid; or the interface continues changing while the agent thinks. Agents that succeed when the response remains valid can fail when it expires before their action arrives. But response speed alone does not explain the failures. On a moving-target task, we keep the model call and the action’s commitment and arrival times unchanged, changing only which observation supplies the click coordinate. Using the newer observation raises success from 0/100 to 100/100: the original click targets where the object was, while the revised click targets where it has moved. Guided by the framework, Chronotope allows new evidence to revise a decision before commitment, preserving useful reasoning while updating action variables that depend on the changing interface.