acceptodds
Under review as a conference paper at ICLR 2027

AdaJEPA: An Adaptive Latent World Model

Abstract

Latent world models enable planning from high-dimensional observations by predicting future states in a compact latent space. However, these models are typically kept frozen at test time: when their predictions become inaccurate, planning can fail, especially under test-time distribution shift. To address this, we propose AdaJEPA, an adaptive latent world model that performs test-time adaptation within the closed loop of model predictive control (MPC). After training, AdaJEPA plans and executes the first action chunk, uses the observed next-state transition as a self-supervised adaptation signal, and replans with the updated model. This closed-loop update continuously recalibrates the world model without additional expert demonstrations. Across a range of goal-reaching tasks, AdaJEPA substantially improves planning success with as few as one gradient step per MPC replanning step.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
World Model, Test-Time Adaptation, JEPA, Latent Planning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_B2fhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AdaJEPA, a test-time adaptation framework for JEPA-based latent world models used in model predictive control.

Critical strengths
  • The paper demonstrates substantial practical benefits from a lightweight adaptation procedure. The comparison against the same frozen model provides convincing evidence that online updates can improve planning in the evaluated settings.
  • The empirical study goes beyond a single favorable configuration and provides useful guidance about applicability. The paper evaluates multiple world-model implementations and examines layer selection, LoRA, learning rates, update counts, replay buffers, and pretraining data scale.
Critical weaknesses
  • The fine-tuning comparison does not sufficiently establish the claimed sample-efficiency advantage. In Section 4.2, the main offline fine-tuning baselines receive only one epoch over the target-domain trajectories, whereas AdaJEPA repeatedly updates on a small rolling buffer throughout execution. Matching learning rates and trainable parameters does not match optimization effort, data reuse, or the relevance of the adaptation data to the current trajectory. The number of optimizer updates and the convergence behavior of these baselines are not reported.

  • It remains unclear whether local adaptation reduces compounding errors in multi-step rollouts. Reporting prediction error across rollout horizons would better support the claimed connection between adaptation and improved planning.

Questions for authors
  • AdaJEPA addresses domain shift, but compounding errors may persist in multi-step rollouts. Even introducing rollout-reliability estimation and adaptive termination, as in MBRL-LDN [1], does not eliminate this problem. Have the authors considered approaches that avoid error accumulation altogether?

  • JEPA-based planning is constrained by compounding errors and the need for action-space alignment, making it difficult to scale. Have the authors considered having the model plan directly in state space?

[1] Improving Model‐Based Deep Reinforcement Learning with Learning Degree Networks and Its Application in Robot Control

4 comments

Thanks for your positive recommendation and experiment suggestions! We have added all requested experiments to address your concerns.

In Section 4.2, the main offline fine-tuning baselines receive only one epoch over the target-domain trajectories, whereas AdaJEPA repeatedly updates on a small rolling buffer throughout execution.

Thanks for the suggestion! We updated the offline fine-tuning baseline and find the sample-efficiency advantage is still robust.

  • Matched budget: AdaJEPA takes at most 30 update steps per episode for default Adapt and at most 150 for Adapt+. The new baseline applies the identical update (same parameters, loss, optimizer, batch of 5 transitions) offline to one target-domain trajectory for 30, 150 and 500 steps, and to 32 trajectories for 1 and 4 epochs (153 and 612 steps).
  • Learning rate: The baseline's rate is selected per shape and budget on a validation seed from 0.5x, 1x and 2x the online rates (predictor 5e-4, encoder 1e-5). Note that AdaJEPA uses the same learning as training on every task, with NO tuning.

[Table 1] Offline finetuning baseline (success in %, three seeds):

frozen1 traj, 30 steps150 steps500 steps32 traj, 153 steps612 stepsAdapt (up to 30)Adapt+ (up to 150)
GD, seen shapes47.549.547.741.754.556.082.882.8
GD, unseen shapes31.031.328.729.141.848.252.055.0
CEM, seen shapes48.352.046.843.258.257.781.585.0
CEM, unseen shapes34.035.133.129.648.748.052.057.7

At the same step count, offline fine-tuning on one trajectory provides only modest gains. More steps on one trajectory cause overfitting: the training loss keeps falling with no gain in success. With more trajectories (N=32) and updates, offline finetuning still significantly below AdaJEPA that only sees one trajectory.

It remains unclear whether local adaptation reduces compounding errors in multi-step rollouts. Reporting prediction error across rollout horizons would better support the claimed connection between adaptation and improved planning.

Thanks for the suggestion! We found that AdaJEPA reduces multi-step prediction error on recent, untrained transitions from the agent's executed trajectory.

We evaluate Adapt on the PushObj shapes (GD, 150 episodes per shape). Starting from an observed frame, we roll the model forward autoregressively using the recorded actions and measure latent MSE against the observed frame at each horizon, encoded by the corresponding frozen or adapted encoder. We use two diagnostics:

  • Recent untrained transitions: in a separate diagnostic run, we delay adaptation by excluding the latest four action-chunk transitions (20 environment steps) from updates. Before each update, we retrospectively score rollout windows whose targets lie within this untrained tail, under frozen and current adapted weights. The delayed schedule lowers success by about 2 percentage points relative to the paper's protocol.
  • Held-out slices: in separate runs using the paper's standard update schedule, we score 20 fixed segments from other trajectories with the same object, which are never used for adaptation.

[Table 2] Open-loop rollout error under frozen → adapted weights (adapted / frozen ratio of means), MSE x1e3.

horizon (env steps)recent untrained, seen shapesrecent untrained, unseen shapesheld-out, seen shapesheld-out, unseen shapes
518.9 → 14.4 (0.76)31.8 → 20.8 (0.66)12.4 → 13.3 (1.07)50.5 → 51.4 (1.02)
1036.2 → 26.0 (0.72)59.7 → 40.6 (0.68)26.4 → 27.6 (1.05)133.8 → 134.1 (1.00)
1555.4 → 38.2 (0.69)86.8 → 60.4 (0.70)47.8 → 49.7 (1.04)259.8 → 258.2 (0.99)
2074.5 → 51.4 (0.69)112.9 → 80.0 (0.71)81.1 → 83.4 (1.03)414.7 → 412.9 (1.00)

On recent untrained transitions, the adapted model has lower mean prediction error at every measured horizon for both seen and unseen shapes. Held-out errors increase slightly on seen shapes and change little on unseen shapes. These results are consistent with local correction along the executed trajectory.

AdaJEPA addresses domain shift, but compounding errors may persist in multi-step rollouts.

The results above show that AdaJEPA reduces multi-step prediction error along the executed trajectory, though error still grows with rollout horizon. Avoiding error accumulation remains an open problem. Combining AdaJEPA with rollout-reliability estimation as studied in MBRL-LDN is a promising future direction.

Have the authors considered having the model plan directly in state space?

Yes, AdaJEPA can also be applied directly in state space when the underlying state is accessible. However, we consider the more challenging setting where the model only has access to images.


We are happy to further revise the paper if you have more suggestions!

Thank you again for helping us improve our paper! In summary, we have updated the pdf (in blue) according to your suggestions. We show that step-matched, learning-rate-tuned offline FT baselines still underperform AdaJEPA (App. B.3). We also found that adaptation helps to decrease the rollout error on the transitions the agent is about to execute (Fig. 12). Error still grows with the horizon, and avoiding accumulation is a common challenge in world models that requires future work. We hope the revision addresses your concerns and would be grateful if you would consider raising your score.

Thank you for the authors' response. The authors may have misunderstood "plan directly in state space." What I mean is that both image space and latent space are state space. My question is: why not do video generation or latent-space sequence generation? Because explicit planning with MPC in JEPA is actually very inefficient.

Sorry for the misunderstanding and thank you for clarifying!

If we understand correctly, we think you're suggesting an image or latent trajectory conditioned on the current observation and goal. This is a promising alternative, though a controller or inverse-dynamics model would still be needed to translate the generated states into actions (e.g. a video of picking an apple does not itself specify the robot’s actions). We agree that planning with WMs can be slow, but trajectory generation is not necessarily faster or free of accumulated errors: diffusion involves iterative computation, and autoregressive video generation can still accumulate errors. Generated trajectories may also violate physical constraints[1], making them difficult to execute reliably.

Our focus is on improving world models through online adaptation within JEPA-based MPC. Extending this idea to adapting the generator or controller using execution feedback is an interesting future direction. Incorporating policy to accelerate and improve planning is also another future direction.

[1] VideoPhy-1 (ICLR 2025) and VideoPhy-2 (ICLR 2026)

Reviewer_qxUROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes test-time adaptation of a JEPA-style latent world model inside closed-loop MPC. After each executed action chunk, the observed transition is added to a small replay buffer (5 most recent transitions), and the last layers of the encoder and/or predictor are updated with one gradient step (or up to five, "Adapt+") on the same latent prediction loss used in pretraining, with stop-gradient on the target. The model is reset per episode. Experiments on PushT/PushObj (shape and visual shifts), PointMaze (mass/damping shifts and unseen layouts), and OGBench cube report planning-success gains over the frozen model, e.g. unseen layouts 64.7 → 83.9% (GD, Adapt+), and over offline fine-tuning on N ∈ {1, 8, 32} target trajectories. Added latency is 0.01–0.03 s per replan. Ablations cover which layers to adapt, learning rate, number of steps, buffer design, and training-data scale.

Critical strengths
  1. Simple, cheap, and broadly applicable. The method adds one gradient step on a handful of parameters per replan (≤0.03 s, Table 2), needs no rewards or labels, and is shown on three JEPA implementations (global/spatial features, DINO-WM), two planners (GD, CEM), and several shift types. This is the kind of drop-in procedure practitioners can adopt immediately.
  2. Useful empirical characterization. The paper honestly reports where adaptation fails (red-block shift, App. B.3, with a probing analysis showing that orientation is lost in early encoder layers). It also includes a training-data-scale study (Fig. 6/10) showing gains are largest in low-data regimes, which clarifies when the method is worth using.
Critical weaknesses
  1. Novelty is overstated relative to prior online model adaptation inside MPC. The plan–execute–adapt loop is the core of adaptive MPC and of gradient-based online adaptation of learned dynamics models in model-based RL. Most directly, Nagabandi et al. (ICLR 2019, "Learning to Adapt in Dynamic, Real-World Environments through Meta-RL") adapt a neural dynamics model with gradient steps on the most recent M transitions before each MPC replan, under dynamics shifts closely resembling those in Table 1. Hansen et al. (ICLR 2021, "Self-Supervised Policy Adaptation during Deployment") perform self-supervised test-time updates of the encoder during deployment under visual shifts like those in Fig. 3. Neither is cited or compared. The claim "the first to adapt a JEPA world model during planning" holds only in the narrow sense that the backbone is a JEPA. The paper should position itself against these works and explain what is specific to latent (non-reconstructive) world models: for example, the risk of representation drift or collapse when the encoder is adapted, which also changes the goal embedding z_g used in the planning cost.
  2. The large in-distribution gains may mostly compensate for under-trained base models, and the FT baseline is weak. On PushObj in-distribution shapes the frozen model reaches only 40–55%, and adaptation lifts it to 78–87%. On the PushT models of Table 2, frozen success is 68–91% and gains shrink to +0.7–7.3. The PushObj models are trained for 3 epochs, and the data-scale study fixes the number of optimization steps across data sizes (App. B.5). Adapting to the current episode's object is then a form of task identification that a better-trained model, or one with more context, might do in-context. Meanwhile the FT baseline fine-tunes for one epoch on N trajectories with the training learning rate. For N=1 this is very few gradient steps, while AdaJEPA takes up to 30 (or 150 with Adapt+) per episode. When FT is run properly (full model, 4 epochs, Table 5) it is much stronger (52% vs 15–17%). The conclusion that AdaJEPA "is more sample efficient than offline fine-tuning" therefore rests on an under-tuned baseline. Please add FT with matched gradient steps and tuned epochs, and frozen baselines trained to convergence.
Questions for authors
  1. How does AdaJEPA differ, conceptually and empirically, from GrBAL-style online gradient adaptation (Nagabandi et al., 2019) and PAD (Hansen et al., 2021)? A direct comparison on the PointMaze dynamics shifts and the PushT visual shifts would be very informative.
  2. When the encoder is adapted, is the goal latent z_g re-encoded with the updated encoder at each replan? Have you monitored representation statistics (e.g., latent variance or effective rank) over an episode? Shrinking the latent scale can reduce the prediction loss without improving the model.
  3. What happens to the in-distribution PushObj gains if the frozen model is trained to convergence (more epochs) or given a longer history window? This tests whether the gains reflect test-time adaptation or under-training.
  4. Can you report FT with the same total number of gradient steps as an AdaJEPA episode, and FT with learning-rate and epoch tuning on a held-out target split?
  5. Adapt+ performs up to 5× more updates and uses a gate tied to the copy baseline. Is a compute-matched frozen planner, with more CEM samples or GD iterations at the same wall-clock, competitive?
  6. All results are per-episode adaptation with reset. Does carrying adapted parameters across episodes in the same shifted environment help further, or cause drift?
4 comments

Thanks for your positive recommendation and detailed feedback! We revised the related work and added all experiments you requested, and found that AdaJEPA still outperforms baselines even with more training, more data, longer context, and more tuning.

[W1] Comparison with prior methods

How does AdaJEPA differ, conceptually and empirically, from GrBAL-style online gradient adaptation (Nagabandi et al., 2019) and PAD (Hansen et al., 2021)?

Thanks for pointing us to GrBAL and PAD! We have rewritten the adaptation paragraph of the related work. We provide more detailed explanations below.

PAD adapts the encoder of a reward-trained policy during deployment through an inverse-dynamics head, and the frozen policy head acts on the adapted features; it requires a policy trained by RL together with that head. AdaJEPA has no policy and no inverse-dynamics head: it adapts the world model itself with the pretraining prediction loss, and the planner uses the updated model at the next replan.

GrBAL adapts an MLP dynamics model over measured states, whereas AdaJEPA plans from images and adapts the encoder (ResNet or ViT) together with the predictor (transformer). GrBAL's main contribution is meta-learning: the initial parameters are meta-trained on a distribution of environment variations so that one gradient step on recent transitions adapts them fast. AdaJEPA is pretrained without prior knowledge on test-time distribution shifts and adapts with no meta-training.

Empirically, neither method runs as published in our setting: GrBAL operates on states and needs a reward and a task distribution for meta-training, and PAD needs a reward-trained policy with its inverse-dynamics head. The closest is the update rule of the non-meta-trained baseline of GrBAL, which the paper reports as performing poorly. That rule, applied to the dynamics model only, corresponds to the frozen-encoder variant in our layer ablation. Figure 10 (App. B.5) shows that adapting the encoder as well helps where the shift enters through the observation. AdaJEPA updates both the encoder and predictor with a single adaptation rule to handle both observation and dynamics shifts.

[W2] Base model and FT baseline

PushObj in distribution vs PushT models of Table 2.

Thanks for raising this! We first want to clarify that PushObj tasks are defined as contact-rich segments (App. A.2), where reaching the goal requires more movement than the standard PushT tasks in DINO-WM. We clarify this in App. A.2. On DINO-WM's standard held-out PushT tasks, the same checkpoints start more than 10 points higher and adaptation still improves significantly.

[Table 1] Test-task statistics, PushObj T segments vs standard PushT (means over tasks):

tasksblock movedblock displacementblock rotationagent-block distance
PushObj T segments99%120 px42°62 px
standard PushT78%35 px28°106 px

On DINO-WM's standard held-out PushT tasks, the same checkpoints start more than 10 points higher, close to the global-feature model in Table 2 of the paper, and adaptation keeps improving them to the level of the patch-feature models there, which are trained on PushT alone, with far less planning time.

[Table 2] Standard PushT tasks against the paper's contact-filtered T segments, frozen / Adapt / Adapt+:

checkpointplannerstandard PushT (paper Table 2)contact-filtered T segments (paper Figure 3)
shape modelGD69 / 87 / 9055 / 83 / 82
shape modelCEM75 / 86 / 8856 / 84 / 84

[Q3] What happens to the in-distribution PushObj gains if the frozen model is trained to convergence (more epochs) or given a longer history window?

Longer pretraining does not remove the gain. Continuing training of the PushObj model to epochs 5 and 7 under the same recipe improves the frozen model on seen shapes at epoch 5 but then overfits by epoch 7, while the adapted model stays ahead by a wide margin at every epoch.

[Table 3] Success per pretraining epoch, CEM, seen / unseen shapes:

epochfrozenAdaptAdapt+
3 (paper)48.3 / 34.081.5 / 52.085.0 / 57.7
555.0 / 33.678.3 / 48.982.0 / 52.0
747.8 / 31.868.5 / 48.075.8 / 50.4

A longer history window helps the frozen model too, but does not replace adaptation. We retrained the PushObj model with a window of 3 (the paper), 5 and 10 frames under the same recipe and planned from the current frame as in the paper, then also planned the 10-frame model with the last 3, 5 and 10 observed frames as context. Both raise the frozen model, while the adapted model stays ahead at every window and context length, so context and adaptation are complementary.

[Table 4] Success per history window and observed context, CEM, seen / unseen shapes:

window / contextfrozenAdaptAdapt+
3 frames54.2 / 32.276.7 / 49.681.7 / 57.1
5 frames58.5 / 38.478.8 / 53.382.7 / 57.6
10 frames59.8 / 44.784.7 / 58.287.0 / 59.6
10 frames, 3 observed in context62.3 / 46.984.2 / 56.987.7 / 58.9
10 frames, 5 observed in context62.8 / 44.286.5 / 55.888.0 / 60.0
10 frames, 10 observed in context63.7 / 47.885.5 / 59.388.8 / 56.4

[Q4] Can you report FT with the same total number of gradient steps as an AdaJEPA episode, and FT with learning-rate and epoch tuning on a held-out target split?

Thanks for the suggestion! We updated the offline fine-tuning baseline to match the update steps and tune its learning rate and update steps, and found the sample-efficiency advantage is still robust.

  • Matched budget: AdaJEPA takes at most 30 update steps per episode for Adapt and at most 150 for Adapt+. The new baseline applies the identical update (same parameters, loss, optimizer, batch of 5 transitions) offline to one target-domain trajectory for 30, 150 and 500 steps, and to 32 trajectories for 1 and 4 epochs (153 and 612 steps).
  • Learning rate: selected per condition and budget on a held-out validation seed from 0.5x, 1x and 2x the online rates (predictor 5e-4, encoder 1e-5), then reported on the test seeds. AdaJEPA uses the training rates on every task, with NO tuning.

[Table 5] Step-matched offline fine-tuning on PushObj:

frozen1 traj, 30 steps150 steps500 steps32 traj, 153 steps612 stepsAdapt (up to 30)Adapt+ (up to 150)
CEM, seen shapes48.352.046.843.258.257.781.585.0
CEM, unseen shapes34.035.133.129.648.748.052.057.7

At the same step count, offline fine-tuning on one trajectory provides only modest gains. More steps on one trajectory cause overfitting: the training loss keeps falling with no gain in success. With more trajectories (N=32) and updates, offline finetuning still significantly below AdaJEPA that only sees one trajectory.

[Q5] Is a compute-matched frozen planner, with more CEM samples or GD iterations at the same wall-clock, competitive?

Thanks for the question! We found that a larger planner budget costs more than Adapt+ and gains far less. We ran the frozen model at GD 200 steps and CEM 300 and 400 samples (the paper uses GD 100 and CEM 200).

  • Adapt+'s update (up to five update steps) costs 0.06 to 0.08 s per replan on one L40S GPU, which is much cheaper than adding more CEM samples or GD iterations but significantly outperforms both.
  • Adaptation keeps its gain when it is given the larger budget itself. The revised appendix adds a figure of success against measured seconds per replan.

[Table 6] Success rate with larger planner budgets:

conditionfrozen GD 100frozen GD 200Adapt+ GD 100frozen CEM 200frozen CEM 300frozen CEM 400Adapt+ CEM 200
seen shapes (4)47.549.082.848.348.747.885.0
unseen shapes (3)31.030.755.034.032.734.057.7
visual shifts (7)46.349.866.352.955.357.070.9
mass x0.177.384.779.783.382.084.786.7
damping x5052.758.075.060.060.761.376.3
layouts easy64.768.083.954.564.060.777.6
layouts hard31.531.350.026.129.329.342.4

Other questions

[Q2] When the encoder is adapted, is the goal latent z_g re-encoded with the updated encoder at each replan? Have you monitored representation statistics (e.g., latent variance or effective rank) over an episode? Shrinking the latent scale can reduce the prediction loss without improving the model.

Yes, the goal frame is re-encoded with the adapted encoder at every replan. We monitored the latent over the episode, and it does not shrink.

[Table 7] Latent statistics of the same frames (20 held-out slices) under the frozen / adapted encoder over the episode:

replanlatent normper-dimension varianceeffective rank
121.18 / 21.181.031 / 1.03135.1 / 35.1
1021.18 / 21.181.031 / 1.03135.1 / 35.1
2021.18 / 21.181.031 / 1.03035.1 / 35.1
3021.18 / 21.181.031 / 1.03035.1 / 35.1

[Q6] Does carrying adapted parameters across episodes in the same shifted environment help further, or cause drift?

Using the default setup of AdaJEPA, carrying generally leads to light performance degradation for seen shapes, and can help on the most unfamiliar shape (square). This matches what we found for Reviewer B2fh (W2): the update is local, it corrects the model along the executed trajectory, and carried updates might lead to drift and cause forgetting, while an unfamiliar shape keeps benefiting from the accumulated updates. A simple continual variant, keeping an episode's updates only when the episode succeeds and rolling back otherwise (about 35 of the 50 episodes are kept), is within noise of the reset on seen shapes and keeps the gain on square. We’re exploring continual adaptation across episodes as future work.

TLZ+Ismall Tsquare
GD, Adapt, reset84.0 ± 0.090.7 ± 3.483.3 ± 5.779.3 ± 1.946.7 ± 6.269.3 ± 8.138.7 ± 3.4
GD, Adapt, carry78.0 ± 7.185.3 ± 2.580.0 ± 1.679.3 ± 1.944.7 ± 4.169.3 ± 3.449.3 ± 6.2
GD, Adapt, carry on success81.3 ± 4.186.7 ± 4.182.0 ± 2.880.0 ± 4.942.0 ± 11.470.0 ± 5.951.3 ± 7.4

We are happy to further revise the paper if you have more suggestions!

Thank you again for helping us improve our paper! In summary, we have updated the pdf (in blue) according to your suggestions:

  • [W1, Q1] Compare with GrBAL and PAD in the related works, with more explanations above
  • [W2, Q4] Show step-matched, learning-rate-tuned offline FT baselines still underperform AdaJEPA (App. B.3)
  • [Q2] Clarify we do goal re-encoding at every replan (Sec. 3.2); Emphasize the collapse risk with the no-collapse evidence (App. B.6)
  • [Q3] Longer pretraining (in our response above) and longer history windows help the frozen model to some extent, but it is still much worse than adaptation (App. B.7)
  • [Q5] The frozen planner at larger CEM budgets is slower and worse than Adapt+ at the default budget (App. B.2)
  • [Q6] Carrying parameters across episodes helps on the most novel shape, but might cause a drift. Carrying conditioned on success is a partial fix. We think continual adaptation is an interesting and important future direction (Discussion, App. B.8)

We hope the revision addresses your concerns and would be grateful if you would consider raising your score.

Reviewer_f3MqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AdaJEPA, which updates a pretrained JEPA world model online inside the MPC loop: after each executed action chunk it takes one (Adapt) or up to five (Adapt+) self-supervised gradient steps on the latent prediction error of recent transitions, then replans. Motivated by frozen planners failing "especially under test-time distribution shift" [p.1, Abstract], it claims substantial gains in and out of distribution, "particularly strong" when training data is limited [p.2, Section 1].

Method: The world model is a JEPA (ResNet encoder, transformer predictor, stop-gradient target, temporal-straightening regularizer) [p.5, Section 4.1], planned with GD or CEM on the latent distance to a goal image [Equations 1-3]. At test time the encoder's last stage and the predictor's last block are updated on a buffer of the five most recent transitions, with the pretraining prediction loss and a stop-gradient target [Equations 4-5; Algorithm 1], at the training learning rates [p.5, Section 4.1]. No reward or demonstrations are used, and each episode restarts from the pretrained weights [Section 3.2].

Experiments: The paper tests shape shifts (unseen PushObj shapes) and visual shifts (blur, noise, lighting, recolouring) on PushT, dynamics (mass, damping) and unseen-layout shifts on PointMaze, three in-distribution PushT models, and the OGBench cube [p.4-5, Section 4.1; Appendix B.6]. Baselines are the frozen model and offline fine-tuning on N in {1, 8, 32} target trajectories. With GD, Adapt+ raises success from 64.7 to 83.9 on easy unseen layouts and from 52.7 to 75.0 under damping x50, above FT(N=32) in every Table 1 cell [Table 1]. The paper reports that success on unseen shapes nearly doubles [p.6, Section 4.2; Figure 3]. With one shape and 1k trajectories, adaptation lifts success on the training shape from 28.1% to 60.8%, above a frozen model trained on 16x more data [Section 4.4]. One step adds 0.01-0.03 s per replan [Table 2]. On a red-block recolouring, adaptation does not help on its own but adds to full fine-tuning [Appendix B.3, Table 5].

Critical strengths

1. Showing that in-loop adaptation works for reward-free, goal-image JEPA planners.

  • Each executed transition gives a free self-supervised example from the test environment, and updating the JEPA on it before the next replan lets the planner finish tasks on which the frozen model stalls [Algorithm 1; Section 3.2; Figure 3; Table 1].

2. A cheap, bounded update that stacks with fine-tuning.

  • Only a few final layers take one step per replan, so the update can run at every replan; each episode restarts from the pretrained weights, so the model cannot drift; and the update adds to full offline fine-tuning on a shift that neither fixes alone [p.5, Section 4.1; p.7, Section 4.2; Table 2; Appendix B.3, Table 5].
Critical weaknesses

1. The update rule is an uncited 2019 baseline, and adapting the encoder, the main difference from it, adds little outside layout shifts.

  • One gradient step per control step on the model's recent transitions, inside MPC, is GrBAL's MB+DE baseline, "adapted by taking a gradient step at each timestep using the past M observations, akin to dynamic evaluation" [1], and MOLe's baseline (d) [2]. Neither is cited, and Thrun et al. [3], which is cited, already adapt the model "in small steps" from its prediction error after each action, inside a gradient-descent planner.
  • MoVie [4], also uncited, adapts a pixel-based latent world model's encoder at test time with its frozen latent dynamics loss and no reward.
  • With the predictor's last block adapted, adding the encoder's last stage changes success by 0 to +3 points on shape, visual and dynamics shifts (+6 and +7 on layouts) [Appendix B.4, Figure 8], against "for visual ... shifts, updating only the predictor is less effective" [p.8, Section 4.3].
  • Fix: cite these works, rewrite "usually coupled to policy or value learning" [p.2, Section 2] and "unlike existing world models that are kept frozen during planning" [p.3, Section 3], and state the delta (a goal-image JEPA planner, an encoder update against a stop-gradient target, a per-episode reset). GrBAL [1] and MOLe [2] found this plain update slow or insufficient without meta-learning, so its working here is a finding the paper could claim.

References

  • [1] A. Nagabandi, I. Clavera, S. Liu, R. S. Fearing, P. Abbeel, S. Levine, C. Finn. Learning to Adapt in Dynamic, Real-World Environments through Meta-Reinforcement Learning. ICLR 2019. arXiv:1803.11347.
  • [2] A. Nagabandi, C. Finn, S. Levine. Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RL. ICLR 2019. arXiv:1812.07671.
  • [3] S. Thrun, K. Möller, A. Linden. Planning with an Adaptive World Model. NeurIPS 1990.
  • [4] S. Yang, Y. Ze, H. Xu. MoVie: Visual Model-Based Policy Adaptation for View Generalization. NeurIPS 2023. arXiv:2307.00972.
Questions for authors

I don't have any specific questions. It would help if the authors could state what AdaJEPA adds over GrBAL's MB+DE baseline [1] and MoVie [4], as asked in Weakness 1.

2 comments

Thanks for your positive recommendation and for the detailed writing feedback! We have added the missing prior works and revised the paper as suggested. We provide more detailed explanations below.

GrBAL adapts an MLP dynamics model over measured states, whereas AdaJEPA plans from images and adapts the encoder (ResNet or ViT) together with the predictor (transformer). GrBAL's major contribution is meta-learning: the initial parameters are meta-trained on a distribution of environment variations so that one gradient step on recent transitions adapts them fast. Its non-meta-learned baseline, MB+DE, performs poorly, below even the non-adapted model where fast adaptation is needed; MOLe meta-learns in the same way and reports the same for its online baseline without meta-learning. AdaJEPA is pretrained without prior knowledge on test-time distribution shifts, and adapts a JEPA planning on images with no meta-training.

MoVie adapts the encoder of a reward-trained TD-MPC to a new camera view against its frozen latent dynamics model. The world is unchanged there, so they insert spatial-transformer blocks that re-align the encoder's features to the training view while the dynamics model stays frozen. The adaptation is carried across episodes and takes many gradient steps per environment step to converge, against one step per replan on a tiny replay buffer in AdaJEPA.

AdaJEPA updates both the encoder and predictor with a single adaptation rule to handle both observation and dynamics shifts. This is more general, and it poses a challenge the other two settings do not have: with both adapted, the prediction target and the planning cost move with the model, and the latent can collapse. The paper restricts the update (encoder head and one predictor block, stop-gradient target, per-episode reset), and in our reply to Reviewer qxUR (Q2) we confirm that the latent's norm, variance and effective rank do not move over an episode (App. B.6).

Regarding whether updating the encoder is necessary, we have regrouped the layer ablation (Figure 10, App. B.5) into seen and unseen shapes, visual shifts with and without recoloring, dynamics shifts and unseen layouts: adapting the encoder helps most on the recoloring shifts and on unseen layouts. Moreover, when changing the block color in PushT, the block's orientation is lost in the encoder's early layers, so updating the encoder is crucial there (App. B.4). We have revised the description in Section 4.3 to be more precise and also note that updating both the predictor and the encoder covers shifts in the dynamics and in the observation without knowing in advance which one has changed.


We are happy to further revise the paper if you have more suggestions!

Thank you again for helping us improve our paper! In summary, we have updated the pdf (in blue) according to your suggestions: (1) added comparison with GrBAL, MOLe, and MoVie in related works, (2) revised the Section 4.3 sentence: adapting the encoder helps most on recoloring shifts and unseen layouts, (3) rewritten the flagged sentences and removed "unlike existing world models that are kept frozen during planning". We hope the revision addresses your concerns and would be grateful if you would consider raising your score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.