acceptodds
Under review as a conference paper at ICLR 2027

THE GUI IS NOT THE STATE: DIAGNOSING STATE ALIASING IN GUI WORLD MODELS

Abstract

GUI World Models (GUI-WMs) are increasingly used to predict future states for agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the visible interface omits transition-relevant environment state, so identical observable conditions can correspond to different valid futures. To diagnose this failure mode, we introduce StateAliasBench, a diagnostic benchmark that explicitly isolates such ambiguities via strict pairing. We further propose lightweight predictivestate recovery that infers structured state from history and augments otherwise frozen GUI-WMs through a deterministic state interface. Family-specific specialists provide state recovery across heterogeneous state types, and multi-teacher distillation consolidates them into a single unified estimator. Experiments show that existing GUI-WMs exhibit systematic failures under observation-only conditioning, while predictive-state augmentation substantially restores state-sensitive prediction across evaluated WMs, preserves generative fidelity, and improves downstream performance of GUI agents on AndroidWorld. These results suggest that reliable GUI world modeling should account not only for what is visible, but also for the hidden transition state that determines what happens next.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.