How State-Change Prediction Shapes Representations in Latent World Models

Latent world models commonly predict the full representation of a future observation. We study how predicting its change from the current representation affects action-related information and its use in control. Within LeWM, we compare next-state and state-change prediction under matched architectures, training data, and representation regularization. State-Change LeWM predicts a latent update and adds it to the current representation. It achieves higher mean planning success on all four control tasks, raising the equally weighted average from 80.90% to 86.15%. Policies trained on frozen State-Change representations also improve success by 5.2 and 3.8 percentage points on Reacher and Cube. Crossed encoder–predictor comparisons and action readouts connect these gains to both the prediction form and the learned representation. The identity path carries the current state forward while the predictor models change; multi-step rollouts further show lower latent error on three tasks and higher decoded PSNR on all four.