Adaptation-Ready JEPA: Source Training for Test-Time Adaptive World Models

Latent world models often require test-time adaptation under deployment shifts, while existing methods mainly rely on self-supervised objectives that can be con- structed directly at deployment. In contrast, supervision that explicitly shapes control-related representations, such as physical states, is often available only during source training. Such source-only supervision can improve the initial rep- resentation, but its constraints do not automatically carry over once the model is further updated by a different self-supervised objective. We therefore study a supervision-availability mismatch under a fixed deployment update: how can training-only supervision improve the model produced by that update? We pro- pose Adaptation-Ready JEPA (AR-JEPA), which keeps AdaJEPA’s deployment update unchanged, differentiably simulates it during source training, and opti- mizes the source model through the resulting adapted model. A meta-learning- style post-update physical objective alone is insufficient in this setting, because the adapted object is itself the world model that must continue to support dy- namics prediction and MPC. AR-JEPA therefore uses Post-Phys to preserve control-related physical information in the updated representation and Post-Dyn to preserve future latent-dynamics prediction of the same model, jointly main- tainingrepresentation content and predictive functionality after adaptation. After training, physical labels and the auxiliary head are removed, while real deploy- ment retains AdaJEPA’s original adaptation loss, updatable parameters, and MPC procedure. On Push-Object, AR-JEPA achieves the highest 13-OOD adapted success among the compared configurations (40.56%) and shows smaller future- dynamics and physical-representation degradation under an identical AdaJEPA update. PointMaze further shows that the relative roles of pre- and post-adaptation physical supervision depend on the environment