Invariant Imagination: Learning What Should Not Change the Future in JEPA World Models

Joint-Embedding Predictive Architectures (JEPAs) support latent prediction and visual planning, yet strong clean performance does not guarantee reliable predictions under degraded observations. We characterize this vulnerability through the predictive brittleness gap, defined as the increase in prediction error when clean and degraded contexts are evaluated against the same clean target. We introduce **FAPI** (Future-Anchored Predictive Invariance), an adaptation framework that transforms observation-derived contexts while preserving clean target representations as shared predictive anchors. This asymmetric design encourages predictive stability without an explicit pairwise consistency loss or changes to the prediction architecture. We evaluate **FAPI** on LeWorldModel and Sub-JEPA across PushT, OGB-Cube, TwoRoom, and DMC under Gaussian noise, patch masking, and incomplete history. Under Gaussian noise, FAPI+LeWM improves mean planning success from 49.5% to 61.5% and reduces the mean brittleness gap by 24.6%; **FAPI+Sub-JEPA** improves success from 40.0% to 57.0% with a 43.1% gap reduction. Both models also improve mean clean success and planning under patch and history masking. Extensions to V-JEPA on Something-Something-v2 and I-JEPA on ImageNet yield lower prediction error and stronger clean–corrupt consistency, supporting the applicability of future anchoring across action-conditioned, video, and image prediction settings.