Keeping Recorded GUI Workflows Reliable When Interfaces Change

Conventional GUI record and replay is fast and cheap, but it breaks under interface reflow, restyling, distractors, and renamed controls. Vision-language computer-use agents adapt to such changes, yet re-planning an already demonstrated workflow on every run adds latency, cost, and error. We study the middle ground: given one successful demonstration, keep that workflow reliable under interface perturbations without generating a new plan. Our method preserves the recorded action sequence and re-ground each target from pixels on the live interface. Our resolver ranks OCR, template, and geometry candidates with 82 scene-relative features and uses calibrated confidence to blend learned evidence with the Hybrid prior and to trigger bounded recovery. We trained on a synthetic corpus of 38 designed workflows rendered under identity with 9 perturbation classes and evaluated on three benchmarks covering 464 replayable workflows from MiniWoB++, WebArena/WONDERBREAD, and WebShop under identity and six task-preserving perturbation classes, yielding 2,670 admitted task–perturbation cells and 16,020 method episodes. Averaged over the six perturbation classes, our method obtains 88.4% MiniWoB++ success versus 67.2% for Hybrid, and 83.9% WebArena success versus 55.2%. On WebShop it reaches 97.6 reward points against 96.3. Code is available at: https://anonymous.4open.science/r/ANONYMIZED.