acceptodds
Under review as a conference paper at ICLR 2027

JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts

Abstract

World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world model, test-time training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LaHqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose JEPA-TTT, a latent world model that supports test-time training to adapt to changes in environment dynamics. Unlike conventional JEPA models, which remain frozen after training, JEPA-TTT collects data in a replay buffer at test time and uses these data to update the model, particularly its predictor. Across eight dynamics-shift conditions, JEPA-TTT reduces latent prediction error by up to 83% and improves performance by 153% compared with a frozen JEPA model.

Critical strengths
  • The paper extends JEPA to support test-time training and demonstrates that the model can be updated stably using data collected at test time.
  • The paper is clearly written and easy to follow, from the problem formulation and methodology to the experiments.
  • The experiments evaluate the method under various dynamics shifts and examine how different replay strategies affect performance.
Critical weaknesses
  • Limited generality of the assumed setting. The proposed method assumes that the task (reward function) and observation distribution remain fixed, while only the dynamics governing the effects of actions change. In my view, this does not appear to be a particularly general setting. It seems more common for observations to change while the underlying dynamics remain broadly similar to those encountered previously: for example, a dropped ball still bounces, and an object still moves when pushed.

  • Insufficient comparison and analysis relative to AdaJEPA. AdaJEPA appeared on arXiv approximately three months before this paper’s submission. My understanding is that ICLR considers work published in a peer-reviewed venue within two months of submission to be concurrent work. Since AdaJEPA has not undergone peer review, there may nevertheless be grounds for treating it as concurrent work. Even so, if this paper is published, the two works will occupy closely related positions in the literature, and a more detailed comparison would be valuable. In particular, although the experiments include a comparison with AdaJEPA, a more thorough analysis is needed to explain which methodological differences account for the observed performance gap.

  • Exclusion of continual learning and catastrophic forgetting. Test-time training is closely related to continual learning. However, as acknowledged in the conclusion, this paper does not address continual adaptation or catastrophic forgetting. Adapting to a single shift using a substantial amount of data, without considering continual adaptation, may amount to overwriting the model’s existing knowledge with knowledge of the new environment.

Questions for authors
  1. Could the authors describe the methodological similarities and differences between JEPA-TTT and AdaJEPA in more detail, particularly regarding their objective functions? For example, the paper states that AdaJEPA requires a goal image. However, my reading of Section 3.2 of AdaJEPA suggests that the adaptation objective itself does not use a goal image.

  2. In Figure 5, JEPA-TTT generally outperforms AdaJEPA. Have the authors conducted a more detailed analysis of the reasons for this performance gap?

  3. The images in Figure 3 alone do not provide enough context to understand the circumstances under which JEPA-TTT produces predictions close to the ground truth. Could the authors also show the planned trajectories and explain where the frozen model fails and how JEPA-TTT improves upon it?

  4. Is the visual encoder in Equation 2 frozen? How is the visual encoder’s training objective formulated?

  5. I am not sure I understand the statement in Lines 391–393, “It is not a deployable baseline…” Could the authors clarify its meaning?

Reviewer_Wj3HOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies test time adaptation of JEPA style world models when environment dynamics change. They freeze the encoder and reward model but update the predictor using the online test transitions and replay. They show improvements in prediction accuracy and planning performance.

Critical strengths
  1. The problem of deploying world models in environments with changing dynamics is interesting and relevant.
  2. The method is simple and effective, and their experiments show clear improvements over the equivalent frozen world model.
Critical weaknesses
  1. Updating a dynamics model using online transitions and replanning with MPC is well established in model-based RL and adaptive control. It is not clear to me what is new beyond applying these ideas to a JEPA model. The authros should discuss its relationship to prior work on online dynamics adaptation.
  2. The proposed "dense replay" seems to just be standard experience replay with overlapping trajectory windows, which is commonly used in model-based RL/world-model training. The sparse baselines discard valid training windows so I am not surprised that dense replay performs better... While the ablations are useful, they do not establish a novel method.
  3. I did not find the comparison against PPO-TTT particularly informative, given that PPO adapts through reward driven policy updates, whereas JEPA-TTT uses a supervised loss. A more relevant comparison would be to a pre-trained dynamics model adapted online using the same data, replay, and MPC planner. This would isolate whether JEPA offers any advantage over standard online model adaptation.
Questions for authors
  1. What is the main algorithmic novelty compared to standard online dynamics model adaptation in model-based RL?
  2. How does dense replay differ from the standard replay buffer used in almost all model-based RL algorithms?
  3. Why do you compare against PPO-TTT rather than existing adaptive model-based RL methods? How would JEPA-TTT compare to a world model adapted using the same online data and compute budget?
  4. Do you have any evidence that JEPA representations are useful for test-time adaptation when compared to other pretrained latent world models?
Reviewer_9pisOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes JEPA-TTT, which keeps fine-tuning the predictor of a pretrained LeWorldModel online after a fixed dynamics shift, while the image encoder and an offline-trained reward head stay frozen. Updates use the standard latent prediction loss on the transitions of the policy sampled from a dense replay buffer. Planning is done with CEM on the predicted reward instead of a goal image. On eight hand-designed shifts across PushT, Two-Room, Reacher and OGBench-Cube, the mean best held-out score goes from 0.267 (frozen model) to 0.678 after 500 episodes and 5-block latent MSE drops by 83% on average. The paper compares against AdaJEPA and PPO fine-tuning and ablates the update rule and persistence. Overall JEPA-TTT beats all baselines on every shift and comes within 0.08 of an offline model trained on the shifted environment on 6/8 shifts. The ablations show most of the gain coming from persistence and dense windows.

Critical strengths
  • The design is simple and cleanly decoupled. Only the action encoder/predictor/prediction projection are updated, with the plain latent prediction loss, at most one AdamW update per replan, no decoder and no reward at test time, while encoder and reward head stay frozen. This makes the adaptation cheap and makes any change in plan ranking attributable to the dynamics model alone (Fig. 3), which would make it a practical recipe even if the individual components are not new.
  • The experimental design is clean and the ablations are well controlled. Only the transition dynamics change, the update-rule ablation matches budgets (sparse stream vs compute-matched sparse vs dense stream vs dense replay), the persistence control matches the number of updates (best 0.729 vs 0.289) and the no-shift control shows no gain without a shift (best 0.679 to 0.691). The gains are consistent and large across all eight shifts in four environments, both in planning (mean best 0.267 to 0.678, all three baselines beaten on every shift) and in 5-block latent MSE (45-99% reduction).
Critical weaknesses
  • The method is standard online fine-tuning of a dynamics model with a replay buffer, which is common practice in MBRL/adaptive MPC (e.g. [1] and [2]). The loss is LeWM’s prediction loss without SIGReg on predictor parameters only, and the two parts presented as new (keeping the model across episodes, cutting windows at every offset) feel like default design choices. Their own ablations support this, since persistence mainly shows that not resetting beats resetting (best 0.729 vs 0.289) and dense replay barely beats the dense stream (best 0.678 vs 0.671). This matters most for my assessment because I can’t identify a methodological contribution beyond applying known practice to LeWM.
  • The evaluation can’t show whether JEPA-TTT adds anything over plain fine-tuning. 7/8 shifts transform the commanded action before an unchanged simulator with identical visuals/reward, so predictor-only adaptation is sufficient by construction. The chosen baselines do not test this either, since AdaJEPA is reset every episode and stripped of its goal conditioning, and PPO-TTT only updates every 10 episodes. Minimal and standard alternatives are missing (action-encoder-only adaptation, sparse replay, online fine-tuning of the full world model, established adaptive model-based methods). This matters because the large gains over these baselines are expected given the setup, so they don’t support the paper’s broader claim about adapting latent world models under dynamics shifts.

[1] Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2019). Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning. ICLR 2019. arXiv:1803.11347

[2] Feng, Y., Hansen, N., Xiong, Z., Rajagopalan, C., & Wang, X. (2023). Finetuning Offline World Models in the Real World. CoRL 2023. arXiv:2310.16029

Questions for authors
  • How does dense replay differ from the standard sequence replay in Dreamer/TD-MPC2, which already samples training subsequences from random start indices in a growing buffer?
  • How does JEPA-TTT compare to plain online fine-tuning of the full world model (encoder + predictor with SIGReg) with standard replay?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.