PRP: Diagnosing State Readability Beyond Prediction Error in JEPA World Models
Abstract
World models predict environmental changes from observations and actions to support planning. Latent world models predict learned representations, typically evaluated through aggregate error. We introduce Paired Rollout Probing (PRP), a state-specific evaluation protocol combining three paired representation paths, fixed observation-trained probes, and persistence baselines. Comparing observed and predicted representations measures state readability through probe accuracy at each horizon; comparison with the initial representation measures decoding gains beyond initialization. Two Meta-World-trained models maintain high position readability over 15 rollout steps, whereas a DROID-trained V-JEPA 2-AC model evaluated on Meta-World shows sharp decay within two predictions. In the shifted checkpoint, matched observation restarts improve next-step object decoding by 22 percentage points, with the benefit fading over the next two to three steps. One-step adaptation lowers prediction error while the displayed adapted readout remains below frozen-latent persistence over steps 2–8. PRP thus adds a physical-state criterion to rollout evaluation: whether prediction and adaptation improve access to the required quantities beyond retaining the initial observation.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.