acceptodds
Under review as a conference paper at ICLR 2027

Adapt for Planning, Not Just Prediction: Optimistic Test-Time World Model Adaptation

Abstract

Latent world models are typically pre-trained offline and kept frozen at test time, so planning with them can degrade under distribution shift. Test-time adaptation (TTA) can mitigate this degradation by updating the model during deployment. Unlike pre-training on fixed data, TTA for planning is closed-loop: the planner uses model rollouts to select actions, and the resulting transitions drive the next update. This raises the question: which objective should guide TTA for planning? Under a prediction-only objective, the model is fit only to the outcomes of its own actions, so it can remain wrong about the actions that would reach the goal, and low prediction error need not translate into better planning. Inspired by reward-biased maximum likelihood from adaptive control, we introduce Optimistic Test-Time Adaptation (OpTTA). Its objective has two terms, the prediction loss on observed transitions and the planner's goal-reaching cost on its candidate rollouts, both minimized over the model parameters. The planning term biases adaptation toward models that are optimistic about reaching the goal, without any additional environment interaction. Across visual and environment shifts on manipulation, reaching and navigation tasks, OpTTA raises final planning success over the frozen model by 13.5 percentage points on average, compared with 6.9 for the prediction-only objective (AdaJEPA), when only the predictor is adapted. When the encoder is adapted as well, the gains are 13.5 and 7.5 points. These experiments span three JEPA world models with different encoders and training objectives and, for temporal straightening, both CEM and gradient-based planners. Our results suggest that test-time adaptation of world models for planning benefits from an optimistic, task-aware term in addition to the prediction loss.

Primary area:
applications to robotics, autonomy, planning
Keywords:
test-time adaptation, world models, joint-embedding predictive architecture (JEPA), goal-conditioned planning, optimism in the face of uncertainty, adaptive control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mvpROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies test-time adaptation of pretrained latent world models for goal-conditioned planning under distribution shift. It proposes Optimistic Test-Time Adaptation (OpTTA), which augments prediction loss on observed transitions with the goal-reaching cost of imagined candidate trajectories, differentiated with respect to the predictor parameters.

Critical strengths
  • The problem is relevant and the intervention is simple. Prediction accuracy on the agent’s own transitions need not imply effective planning. Incorporating the downstream planning objective into model adaptation is a useful direction.

  • The evaluation includes useful ablations and transparent limitations. The paper examines adaptation layers, optimism strength and scheduling, and candidate populations. The appendix provides results by corruption and severity, and the paper explicitly reports negative results on TwoRoom and the larger computational overhead for DINO-WM.

Critical weaknesses

The justification for optimizing the world model toward the goal under fixed candidate actions is insufficient. In Equation (7), candidate action sequences are held fixed while the predictor is updated to reduce their imagined distance to the goal. However, these candidates are not known to reach the goal in the real environment. An accurate prediction of an ineffective action sequence can therefore be penalized by the planning term, encouraging the model to make that sequence appear more successful without changing its actual consequences. I recognize that this bias is intentional and motivated by optimistic model estimation. Nevertheless, the paper does not sufficiently establish why this particular surrogate produces useful exploration or better action selection, rather than misleading predictions of action effectiveness. The prediction loss on a short buffer provides an anchor, but does not directly constrain the outcomes of unexecuted candidates. Figure 4 provides useful diagnostics, yet does not directly test this concern.

Questions for authors

Justification of the goal-directed model update. Candidate action sequences in Equation (7) are not guaranteed to reach the goal in the real environment. Why should their predicted outcomes be pushed toward the goal by updating the world model?

4 comments
AuthorsOpenReview ↗

Response to Reviewer mvpR (1/2)

Thank you for your review. We are glad you found the problem relevant, the algorithm simple and the ablations useful. In short: execution can correct an optimistic model error, but a pessimistic error can keep a good plan from ever being tested; Eq. 7 trades the second kind for the first. The revised manuscript adds a paragraph to Section 4 (Why add an optimistic term?) that justifies this design, and background on RBMLE in Appendix B.

1. Eq. 7 and CEM are the two coordinate steps of one RBMLE-inspired joint objective.

At fixed αk\alpha_k, the joint objective over the model parameters θ\theta and CEM's sampling distribution π\pi is

min⁡θ,πLpred(θ;Bk)+αkEa∼πJ(a,θ).\min_{\theta,\pi}\mathcal{L}_{\rm pred}(\theta;\mathcal B_k)+\alpha_k \mathbb E_{a\sim\pi}J(a,\theta).

Its two coordinate problems are:

  • Fix the model; optimize the actions (CEM): min⁡πEa∼πJ(a,θk)\min_\pi\mathbb E_{a\sim\pi}J(a,\theta_k).
  • Fix the candidate actions; optimize the model (OpTTA, Eq. 7):

min⁡θLpred(θ;Bk)+αkM∑m=1MJ(a(m),θ).\min_\theta\mathcal{L}_{\rm pred}(\theta;\mathcal B_k)+\frac{\alpha_k}{M}\sum_{m=1}^M J(a^{(m)},\theta).

Minimizing over π\pi first gives min⁡θLpred(θ;Bk)+αkmin⁡aJ(a,θ)\min_\theta\mathcal{L}_{\rm pred}(\theta;\mathcal B_k)+\alpha_k\min_a J(a,\theta), the RBMLE objective that Theorem B.2 analyzes. The candidates in Eq. 7 are therefore not arbitrary: they are CEM's current solution to the same objective.

2. Execution can correct optimistic errors, but not pessimistic errors on plans it never tries.

A plan whose imagined cost is too low tends to be selected and executed, and its observed transitions then enter the prediction loss. A plan whose imagined cost is too high may never be selected, and then nothing corrects it. Prediction-only updates can create this second kind of error on good plans; the planning term trades it for the first kind, which execution can correct.

This is easy to see in the two-armed bandit case. Suppose the prediction-only estimates of the two arms are μ^1\hat\mu_1 and μ^2\hat\mu_2. Then the optimistic estimates of UCB (Auer et al., 2002) are μ1UCB>μ^1\mu_1^{\rm UCB}>\hat\mu_1 and μ2UCB>μ^2\mu_2^{\rm UCB}>\hat\mu_2, so the estimates are optimistic for both the good and the bad arm. A bad arm that is played is eventually corrected under either algorithm, but a good arm that is never played, as can happen with prediction-only learning, is never corrected. In OpTTA, the planning term plays the role of the optimistic bonus.

We test this argument in a controlled experiment that compares a single OpTTA update with a single AdaJEPA update on the same data (Table R1). The results support both halves:

  • Prediction-only updates make good plans look worse: AdaJEPA's update raises the imagined cost of 61.9% of goal-reaching elites, against 30.2% for OpTTA's (row 1a).
  • OpTTA's update still learns from what is executed: it lowers the prediction loss on the transitions just executed from 0.0460 to 0.0389, against 0.0374 for AdaJEPA's update (row 5).
  • One update barely changes the next decision: rows 3–4 are within noise, with OpTTA's update ahead (31.9% vs 30.8% of selected plans reach the goal). Over full episodes, these update differences can accumulate as the closed-loop trajectories diverge, consistent with OpTTA's overall improvement in final success (Table 1).
  • The paper's α\alpha ablation tests the asymmetry: reversing the sign of the planning term (ρ=−0.1\rho=-0.1), which makes good plans look worse, drops final success below AdaJEPA's on both tasks (32.7% vs 44.7%, 60.7% vs 70.7%), while every optimistic ρ\rho we tested, from 0.5 to 2, stays above it.

Table R1. One AdaJEPA update vs one OpTTA update, applied to the same predictor and buffer at each of the 1,884 updates along AdaJEPA's trajectory (LeWM PushT, 3 seeds); every plan is run in the simulator. Rows 1a–1b: the 30 best candidates of the executed plan's CEM population, split by whether they reach the goal. Row 2: pairs of a goal-reaching and a non-reaching candidate in which the goal-reaching one has the lower imagined cost. Rows 3–4: CEM re-solved at the next replan. Row 5: prediction loss on the segment just executed. ±: s.e. over seeds; pp: Wilcoxon signed-rank test over episodes.

Before updateAdaJEPA's updateOpTTA's updatepp
(1a) Goal-reaching elites whose imagined cost rises (%)–61.9 ± 1.630.2 ± 4.0<0.001
(1b) Non-reaching elites whose imagined cost falls (%)–25.6 ± 1.578.4 ± 0.9<0.001
(2) Goal-reaching candidate ranked cheaper (%)60.1 ± 2.062.5 ± 1.964.3 ± 1.60.002
(3) Goal-reaching plans among the next 30 elites8.90 ± 0.198.89 ± 0.199.07 ± 0.210.061
(4) Next selected plan reaches the goal (%)30.8 ± 0.630.8 ± 0.831.9 ± 1.10.471
(5) Prediction loss on the executed segment0.0460 ± 0.00170.0374 ± 0.00140.0389 ± 0.0016<0.001

(continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer mvpR (2/2)

3. An accurate prediction of an ineffective sequence can be penalized.

Yes, by design: this is the optimistic bonus of point 2, applied to effective and ineffective candidates alike. Table R1 measures it: OpTTA's update lowers the imagined cost of 78% of the elites that do not reach the goal, as it does for 70% of those that do (rows 1a–1b). CEM uses only the ordering of candidates, so a bonus that lowers both alike can mislead it only by changing that ordering, and row 2 shows no such harm on average: a goal-reaching candidate is ranked cheaper than a non-reaching one in 64.3% of pairs after OpTTA's update, against 62.5% after AdaJEPA's. Two mechanisms keep it in check:

  • Eq. 8: it bounds the planning gradient on the adapted predictor block relative to the prediction gradient, $\Vert \alpha_k g_{\rm plan}\Vert \le \rho\sqrt{k/K_{\max}}\Vert g_{\rm pred}\Vert $; with ρ=1\rho=1 in every result outside the ρ\rho ablation, the planning gradient never exceeds the prediction gradient, so the prediction loss stays the anchor of every update.
  • Execution: once the planner selects an ineffective candidate that the update makes look effective, executing it adds new transition evidence to the prediction loss, which OpTTA's update still fits (Table R1, row 5); the better the model fits a transition, the less optimistic its prediction of it. The planner executes one step per replan, so this evidence accumulates over successive replans.

A planning term that lowers the imagined cost of effective candidates more than ineffective ones (row 1b) would be a better choice still; our α\alpha design is a reasonable one that improves planning, and we do not claim it is optimal.

4. Figure 4 (Figure 5 in the revised manuscript) does not directly test this concern.

Agreed. Figure 4 measures prediction error and imagined cost, both inside the model; Table R1 tests the concern against real outcomes:

  • Same starting point: at every update along AdaJEPA's trajectory, we apply both updates to the same predictor and buffer, one Adam step each as in the paper: AdaJEPA's with the prediction loss alone, OpTTA's with the planning term added, so the comparison isolates the objective from the trajectory.
  • Real outcomes: every scored candidate is run in the simulator from the state it was planned from; the probe leaves the agent unchanged, so its episodes are identical to those of the run without it.
  • The direct test (rows 1a, 2 and 5): prediction-only updates penalize good plans that OpTTA's update leaves alone, goal-reaching candidates are ranked cheaper at least as often, and its update still fits the executed transitions.

5. The short buffer does not supervise unexecuted candidates.

Agreed: neither AdaJEPA's nor OpTTA's objective directly supervises unexecuted continuations using the current buffer. This is where the two objectives differ:

  • Prediction-only adaptation: when several models fit the buffer equally well, the prediction loss cannot choose between them, so which one prediction-only adaptation picks has nothing to do with the task.
  • OpTTA: Eq. 7 breaks the tie toward the model under which the goal is cheaper to reach, the RBMLE principle; executing that model's plan then produces the evidence to confirm it or, through the prediction loss, rule it out (point 2).
  • Guarantee and test: Appendix B proves that this limits repeated errors when the evidence is retained and plans are executed in full. OpTTA's one-step execution falls outside those conditions, and for a five-transition buffer the appendix's finite-buffer bound is weak, so we test it empirically: OpTTA's update still fits the executed transitions, and the next plan's success differs only within noise (rows 4–5).

The main results of this response are added to the revised manuscript. We also request you to take a look at our general response, which lists the new results (TS on further tasks, the Sandwich-Residuals comparison, persistent runs without resets and the equal wall-clock comparison) and the improvements made in the manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer mvpR,

Thank you again for your review. Our response and revised manuscript give a detailed justification for optimism together with a direct test of your concern (Table R1) and a supplementary formal analysis (Appendix B). The revised manuscript highlights these additions in blue. We would appreciate it if you could read the response and consider updating your assessment, and we are happy to answer any remaining questions.

Regards, Authors

Dear Reviewer mvpR,

As the discussion period closes today, we would like to note that our response and revision include a detailed justification for Eq. 7 along with background on optimism (Appendix B), and a matched single-update probe that tests the concern directly (Table 13). These additions further strengthen the paper. We hope that the revision provides the requested justification of the algorithm based on the optimism principle and would be grateful if you would revisit your assessment. Thank you!

Regards, Authors

Reviewer_srvSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to improve goal-conditioned planning with pretrained latent world models under distribution shift. The proposed method extends prediction-only adaptation with a task-aware planning objective. It minimizes a combination of prediction error on observed transitions and goal-reaching costs over imagined trajectories, using adaptive gradient-based weighting. Experiments cover three JEPA world models, including manipulation and navigation tasks, visual corruptions, and environment shifts. Empirically the proposed method outperforms prediction-only adaptation methods in most of the tested settings.

Critical strengths
  1. The paper provides a meaningful motivation for task-aware adaptation via clearly identifying the discrepancy between fitting observed transitions and improving downstream planning.
  2. Empirical improvements are strong within the tested settings.
  3. Ablations are informative to demonstrate benefits of optimism.
  4. Implementation transparency is improved by disclosing details in the appendix, including hyperparameters, per-shift results, and success trajectories.
Critical weaknesses
  1. The manuscript provides no formal characterization (propositions or theorems) of when and how much the optimistic objective improves actual planning performance.
  2. The experiments focus on JEPA-based visual planning with task-local adaptation, so generalization to other world-model families or extended deployment without resetting model parameters remains untested.
  3. The main experiments only focus on simulation-based toy examples: PushT (sim-based), Mazes, etc. No real-world application appears to be included (note that at least PushT has real-robot version). Therefore, the methods applicability and robustness in realistic scenarios are not demonstrated. This suggests insufficient significance of the work when theoretical analysis is also absent (i.e., Weakness 1).
  4. Computational overhead seems nontrivial: On DINO-WM PushT, the reported time per replan increases from 2971 ms for AdaJEPA to 5233 ms for OpTTA. Restricting the candidate population to 30 elites (line 408) substantially reduces computation, but the corresponding DINO-WM planning-success trade-off is not reported. Moreover, although OpTTA often solves tasks in fewer replanning steps, comprehensive comparisons under equal wall-clock computation budgets appear to be missing.
Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer srvS (1/2)

Thank you for your review. We are glad you found the motivation meaningful, the empirical improvements strong, the ablations informative and the appendix transparent.

W1: Appendix B characterizes the benefits of optimism and the failure of prediction-only adaptation.

We added Appendix B, which opens with background on RBMLE. Proposition B.1 shows that prediction-only adaptation can incur linear cumulative excess planning cost despite zero prediction loss on observed transitions; Theorem B.2 bounds the cost of optimistic adaptation:

Theorem B.2. Let the finite class Θ\Theta of deterministic models contain the true model θ⋆\theta^\star. At round kk, let the adaptation select $\theta_k\in\arg\min_{\theta\in\Theta}\lbrace L_k(\theta)+\alpha_kV_k(\theta)\rbrace $ with αk>0\alpha_k>0, where LkL_k is the negative log-posterior of all observed transitions and Vk(θ)V_k(\theta) the optimal imagined cost under θ\theta, and let the planner return a plan within κ\kappa of Vk(θk)V_k(\theta_k). Let MK=⌊(P⋆+Δmax⁡k≤Kαk)/ℓ⌋+1M_K=\lfloor(P^\star+\Delta\max_{k\le K}\alpha_k)/\ell\rfloor+1, where P⋆P^\star is the prior penalty of θ⋆\theta^\star, Δ\Delta bounds the optimistic underestimation of the optimal cost and ℓ\ell is the smallest nonzero per-transition loss. Then each incorrect model is selected on at most MKM_K rounds whose predictions are refuted, every other round has excess cost at most κ+P⋆/αk\kappa+P^\star/\alpha_k, and the number of refuting rounds NKN_K and the excess cost RKR_K over the first KK rounds satisfy

NK≤min⁡{K,(∣Θ∣−1)MK},RK≤Kκ+P⋆∑k=1K1αk+Cmin⁡{K,(∣Θ∣−1)MK},N_K\le\min\lbrace K,(|\Theta|-1)M_K\rbrace ,\qquad R_K\le K\kappa+P^\star\sum_{k=1}^K\frac1{\alpha_k}+C\min\lbrace K,(|\Theta|-1)M_K\rbrace ,

with CC a bound on all costs.

In words:

  • Optimistic adaptation selects a model that later observations refute only a bounded number of times; on every other round its plan is within κ+P⋆/αk\kappa+P^\star/\alpha_k of the best plan.
  • With a bounded optimism weight and all observations retained, refutations stop after finitely many rounds (Corollary B.3); with no prior penalty (P⋆=0P^\star=0) and exact planning, the total excess cost is bounded.

Optimism is well established in theory, from RBMLE (Kumar and Becker, 1982; Mete et al., 2021) to UCB; the open problem is a practical algorithm for latent world models. To our knowledge, OpTTA is the first RBMLE-based method for goal-conditioned MPC with them.

W2 (extended deployment without resetting parameters): OpTTA still outperforms AdaJEPA.

With the adapted predictor carried across all 50 episodes of a seed (Table R4), OpTTA outperforms AdaJEPA in all 5 seeds of every setting, by 3.9, 3.2 and 6.7 points of final success on LeWM PushT, LeWM Reacher and TS PushT. The revised manuscript discusses these results in Section 5.2.

Table R4. No reset within a seed: the adapted predictor carries over across all 50 episodes. Visual corruptions, predictor adapted; otherwise as Table 1.

Setting (shifts)Final: FrozenFinal: AdaJEPAFinal: OpTTAMean over replans: FrozenMean over replans: AdaJEPAMean over replans: OpTTA
LeWM PushT (7)58.5 ± 0.857.8 ± 1.4 (−0.7)61.7 ± 0.9 (+3.1)43.9 ± 0.743.1 ± 1.0 (−0.8)47.1 ± 0.7 (+3.2)
LeWM Reacher (7)52.8 ± 0.560.9 ± 0.4 (+8.1)64.1 ± 0.5 (+11.3)38.4 ± 0.640.5 ± 0.4 (+2.0)42.8 ± 0.6 (+4.3)
TS PushT (7)47.8 ± 1.562.1 ± 1.3 (+14.3)68.8 ± 0.9 (+21.0)34.0 ± 1.341.7 ± 1.1 (+7.7)48.4 ± 0.8 (+14.4)

W2 (generalization to other world-model families): The principle is not specific to JEPA models.

  • Within JEPA models, our evaluation is broader than prior work: AdaJEPA covers TS and DINO-WM, and we add LeWM.
  • Beyond JEPA models, Optimistic World Models (Mete et al., 2026) applies the same principle to DreamerV3 and STORM during reward-based training; OpTTA instead adapts a pretrained model at test time, without reward.

W3 (simulation only, no real-world application): We use the standard benchmarks of LeWM, TS and DINO-WM.

  • The papers that introduce LeWM, TS and DINO-WM evaluate on these simulated tasks; we follow their protocols, so our results compare directly with theirs.
  • OpTTA uses only what the planner already computes, the goal and the CEM candidates, so it can potentially be adapted to real robots; we do not currently have physical robot hardware to test this.

W4a: On DINO-WM, the number of candidates trades success for compute.

  • DINO-WM's overhead comes from its latent size: the planning term backpropagates through 200 rollouts of 98,304-dimensional states, while LeWM and TS add only 22 and 21 ms per replan (Table 2).
  • With the kk lowest-cost candidates (Table R5), success under visual corruptions grows from 56.2% (k=30k=30) to 58.8% (k=200k=200), and the overhead over AdaJEPA from 11.2% to 76.1%.

(continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer srvS (2/2)

Table R5. DINO-WM PushT, OpTTA's planning term restricted to the kk lowest-cost CEM candidates (k=200k=200: Table 1's OpTTA); success as in Table 1 (3 seeds). Time per replan on an RTX PRO 6000 (as in Table 2) and an H200 (these runs); parentheses: overhead over AdaJEPA.

Methodms per replan: RTX PRO 6000ms per replan: H200Final: In dist.Final: Corr. (7)Mean over replans: In dist.Mean over replans: Corr. (7)
Frozen2955241676.7 ± 3.545.3 ± 3.058.9 ± 2.234.2 ± 2.4
AdaJEPA2971243377.3 ± 5.2 (+0.7)57.1 ± 4.5 (+11.8)59.7 ± 5.0 (+0.8)42.7 ± 4.0 (+8.5)
OpTTA, k=30k=303304 (+11.2%)2635 (+8.3%)77.3 ± 1.8 (+0.7)56.2 ± 4.0 (+10.9)57.7 ± 0.8 (−1.2)40.7 ± 3.4 (+6.6)
OpTTA, k=100k=1004103 (+38.1%)3044 (+25.1%)79.3 ± 5.9 (+2.7)57.2 ± 5.0 (+11.9)61.4 ± 4.1 (+2.5)42.1 ± 4.2 (+8.0)
OpTTA, k=150k=1504672 (+57.3%)3333 (+37.0%)82.0 ± 3.1 (+5.3)57.6 ± 4.1 (+12.3)64.6 ± 2.7 (+5.6)43.4 ± 2.8 (+9.2)
OpTTA, k=200k=200 (all)5233 (+76.1%)3663 (+50.6%)81.3 ± 4.7 (+4.7)58.8 ± 3.2 (+13.4)63.1 ± 3.9 (+4.2)44.8 ± 2.5 (+10.6)

W4b: At equal wall-clock time, OpTTA reaches AdaJEPA's final success early and outperforms it in 19 of 24 settings.

In Table R6, OpTTA's success is taken at the last replan it completes within AdaJEPA's 30-replan time, computed from each method's mean time per replan.

  • On LeWM and TS, OpTTA reaches AdaJEPA's final success within the budget in 18 of 22 settings, at a median of 58% of it, and then plateaus.
  • At equal time it outperforms AdaJEPA in 19 of 24 settings, by up to 14.7 points (TS PushT under visual corruptions).
  • The exceptions are three TwoRoom rows (−2.4-2.4 to −1.0-1.0), where the paper already reports a negative result, Maze dynamics with the encoder adapted (−0.6-0.6), and DINO-WM under visual corruptions (−1.0-1.0).

We have rewritten the computational-time discussion in the manuscript to include these results and to improve its clarity, with a new figure of success against wall-clock time (Figure 4).

Table R6. Final success (%) within the time AdaJEPA needs for its 30 replans, on Table 1's runs: (a) predictor adapted, (b) predictor and encoder adapted. OpTTA replans: replans OpTTA completes in that time, from each method's mean time per replan on the H200 GPUs these runs used (measured as in Table 2). Parentheses: OpTTA − AdaJEPA, paired by seed. Time to match: the time OpTTA needs to reach AdaJEPA's final success, in % of AdaJEPA's budget (–: not reached within 30 replans).

Setting (shifts)OpTTA replansFinal, equal time: AdaJEPAFinal, equal time: OpTTATime to match (%)
(a) Predictor adapted
In distribution
LeWM PushT2688.8 ± 1.492.0 ± 1.9 (+3.2)53
LeWM Reacher2798.8 ± 0.8100.0 ± 0.0 (+1.2)66
LeWM TwoRoom2593.2 ± 1.290.8 ± 0.5 (−2.4)–
TS PushT2781.6 ± 2.590.4 ± 1.8 (+8.8)51
DINO-WM PushT1977.3 ± 5.280.7 ± 4.8 (+3.3)75
Visual corruptions
LeWM PushT (7)2662.4 ± 1.068.2 ± 0.7 (+5.8)49
LeWM Reacher (7)2660.3 ± 0.568.5 ± 0.9 (+8.1)71
LeWM TwoRoom (7)2657.6 ± 1.556.6 ± 1.4 (−1.0)–
TS PushT (7)2758.3 ± 1.273.0 ± 1.3 (+14.7)56
DINO-WM PushT (7)1957.1 ± 4.556.2 ± 3.1 (−1.0)110
Environment shifts (TS)
PushT recolors (3)2740.8 ± 1.653.9 ± 0.8 (+13.1)51
PushObj shapes (3)2755.2 ± 2.058.1 ± 1.9 (+2.9)63
Maze dynamics (2)2687.6 ± 2.088.0 ± 1.6 (+0.4)99
(b) Predictor and encoder adapted
In distribution
LeWM PushT2786.8 ± 1.590.0 ± 2.0 (+3.2)47
LeWM Reacher2799.2 ± 0.5100.0 ± 0.0 (+0.8)85
LeWM TwoRoom2192.4 ± 1.091.2 ± 0.8 (−1.2)–
TS PushT2783.2 ± 2.990.8 ± 1.6 (+7.6)55
Visual corruptions
LeWM PushT (7)2763.5 ± 0.868.3 ± 1.0 (+4.8)48
LeWM Reacher (7)2662.3 ± 1.168.1 ± 1.1 (+5.8)76
LeWM TwoRoom (7)2657.1 ± 1.257.7 ± 1.4 (+0.6)90
TS PushT (7)2660.7 ± 1.973.6 ± 1.5 (+12.9)56
Environment shifts (TS)
PushT recolors (3)2744.5 ± 1.353.9 ± 1.7 (+9.3)59
PushObj shapes (3)2755.5 ± 1.157.3 ± 2.2 (+1.9)77
Maze dynamics (2)2687.6 ± 1.787.0 ± 1.5 (−0.6)107

The main results of this response are added to the revised manuscript. We also request you to take a look at our general response, which lists the new results (TS on further tasks, the Sandwich-Residuals comparison and a controlled analysis of the planning term) and the improvements made in the manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer srvS,

Thank you again for your review. Our response adds a supplementary formal analysis (Appendix B), persistent adaptation without resets (Table R4) and the comparison at equal wall-clock time (Tables R5 and R6). The revised manuscript highlights these additions in blue. We would appreciate it if you could read the response and consider updating your assessment, and we are happy to answer any remaining questions.

Regards, Authors

Reviewer_F4S3OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose Optimistic Test-Time Adaptation (OpTTA), a method for adapting pretrained latent world models during goal-conditioned planning. OpTTA introduces an additional planning objective that encourages the world model to predict trajectories closer to the goal. Experiments across three JEPA world models, including manipulation, reaching, and navigation tasks, demonstrate improved planning performance under visual and environment distribution shifts compared with prediction-only adaptation.

Critical strengths
  • The proposed approach is straightforward and effective. The authors address a meaningful limitation of prediction-only world-model adaptation, where improved prediction accuracy on observed transitions does not necessarily translate into better planning. The improvement over AdaJEPA is substantial across most tasks.
  • The experimental evaluation is comprehensive. The authors evaluate OpTTA on LeWM, TS and DINO-WM across different visual and environmental shifts. The comparison with AdaJEPA is well controlled, as both methods share the same pretrained models, planning algorithms, and adaptation configurations.
Critical weaknesses
  • The experimental coverage is limited. Although the paper evaluates three different world models, the experiments on DINO-WM are limited to PushT, while TS is evaluated only on PushT considering visual distribution shifts. Moreover, the paper only compares against AdaJEPA as an existing test-time adaptation method.
  • Insufficient analysis of the reliability of optimistic model predictions. Although the authors analyze prediction errors and optimism in Figure 4, they do not investigate whether OpTTA improves the relative ranking of candidate actions.
Questions for authors
  • While I am not very familiar with the test-time world-model adaptation literature, I came across Sandwich-Residuals (Soni et al., 2026), which also addresses test-time adaptation of world models. Could the authors clarify why this method and other potentially relevant adaptation approaches were not included as baselines? Are there methodological differences that make direct comparisons difficult?
  • Does optimistic adaptation preserve the model's ability to distinguish good and bad actions? OpTTA minimizes the average predicted goal-reaching cost across all candidate action sequences. However, this could encourage the world model to predict favorable outcomes even for actions that cannot actually reach the goal. How does the method ensure that optimism does not distort the relative ranking of candidate actions?
5 comments
AuthorsOpenReview ↗

Response to Reviewer F4S3 (1/2)

Thank you for your review. We are glad you found OpTTA effective, the evaluation comprehensive and the comparison with AdaJEPA well controlled.

W1 (coverage): We add TS under visual shifts on Medium Maze and PushObj.

Table R2 reports TS on Medium Maze and PushObj under the seven visual corruptions, with the protocol of Table 1. OpTTA raises final success over Frozen by 8.2 points on Medium Maze and 37.8 on PushObj, against 4.6 and 31.6 for AdaJEPA, and beats AdaJEPA in all 5 seeds of both. We will add DINO-WM results on further environments in the final version; we kept it to PushT here because of its cost, about 3 s per replan (Table 2).

Table R2. TS, visual corruptions, predictor adapted; protocol and format as Table 1. Medium Maze averages four severity levels per corruption; PushObj averages over its four training shapes.

Setting (shifts)Final: FrozenFinal: AdaJEPAFinal: OpTTAMean over replans: FrozenMean over replans: AdaJEPAMean over replans: OpTTA
TS Medium Maze (7)49.8 ± 1.254.5 ± 1.2 (+4.6)58.0 ± 0.7 (+8.2)37.9 ± 1.139.6 ± 1.0 (+1.7)41.6 ± 0.7 (+3.8)
TS PushObj (7)34.7 ± 1.666.3 ± 0.7 (+31.6)72.6 ± 0.9 (+37.8)24.6 ± 1.242.7 ± 0.8 (+18.1)50.8 ± 0.8 (+26.2)

W1, Q1 (baselines): Sandwich-Residuals is concurrent work; our optimistic planning term improves it too.

  • Concurrent work: Sandwich-Residuals (SR) appeared on 18 September 2026, the ICLR abstract deadline.
  • A different question: SR addresses which parameters to adapt, whereas OpTTA addresses which objective to adapt them with. Because the two are complementary, we added our planning term to SR to isolate the effect of the objective.
  • Setup: SR's released code has no LeWM version, so we ported the term into its TS code and compared SR with SR + OpTTA on five TS settings (Table R3).
  • Result: The optimistic planning term raises SR's final success in all five settings: by 9.2 points under visual corruptions and 6.7 in distribution, each in all 3 seeds, and by 0.7 to 6.0 points on the environment shifts.

The revised manuscript discusses the SR results in Section 5.2.

Table R3. TS, Sandwich-Residuals (SR) adapted, with and without our planning term; format as Table 1, with 3 seeds and 20 replans.

Setting (shifts)Final: FrozenFinal: SRFinal: SR + OpTTAMean over replans: FrozenMean over replans: SRMean over replans: SR + OpTTA
In distribution
TS PushT67.3 ± 1.378.0 ± 1.2 (+10.7)84.7 ± 2.7 (+17.3)41.1 ± 0.448.2 ± 0.9 (+7.1)52.1 ± 1.3 (+11.0)
Visual corruptions
TS PushT (7)42.7 ± 1.056.0 ± 0.3 (+13.3)65.2 ± 1.2 (+22.5)26.0 ± 0.832.4 ± 0.7 (+6.3)38.7 ± 0.4 (+12.7)
Environment shifts
PushT recolors (3)29.8 ± 1.243.3 ± 4.7 (+13.6)45.8 ± 1.4 (+16.0)19.9 ± 1.126.4 ± 2.5 (+6.5)29.3 ± 1.4 (+9.3)
PushObj shapes (3)33.3 ± 2.953.3 ± 2.1 (+20.0)54.0 ± 2.8 (+20.7)20.7 ± 1.729.9 ± 1.7 (+9.2)35.6 ± 1.8 (+14.9)
Maze dynamics (2)80.7 ± 2.984.0 ± 1.2 (+3.3)90.0 ± 2.1 (+9.3)54.2 ± 2.457.3 ± 1.6 (+3.1)57.4 ± 1.3 (+3.2)

W2, Q2 (ranking): After OpTTA's update, goal-reaching candidates are ranked cheaper at least as often.

Eq. 7 can change the relative ranking of candidates, so we evaluate those changes against simulator outcomes (Table R1). By design, the planning term weights all of CEM's candidates equally rather than favoring particular ones, and Eq. 8 keeps its gradient no larger than the prediction gradient (with ρ=1\rho=1). CEM uses only the ordering of candidates, so what matters is whether the update changes that ordering:

  • Goal-reaching candidates are ranked cheaper at least as often: within the executed plan's final CEM population, a goal-reaching candidate is ranked cheaper than a non-reaching one in 64.3% of pairs after OpTTA's update, against 62.5% after AdaJEPA's and 60.1% before the update (row 2).
  • Effective and ineffective candidates are lowered alike: 70% of goal-reaching and 78% of non-reaching elites (rows 1a–1b).
  • Prediction-only updates penalize good plans: they raise the imagined cost of 61.9% of goal-reaching elites, against 30.2% for OpTTA's (row 1a).
  • The next decision differs only within noise, and the fit holds: the next selected plan reaches the goal in 31.9% of decisions after OpTTA's update, against 30.8% after AdaJEPA's (row 4, p=0.47p=0.47), and the prediction loss on the executed transitions is 0.0389 vs 0.0374, from 0.0460 (row 5).

The revised manuscript adds a paragraph to Section 4 explaining why this design helps planning.

(continued in the next comment)

AuthorsOpenReview ↗

Response to Reviewer F4S3 (2/2)

Table R1. One AdaJEPA update vs one OpTTA update, applied to the same predictor and buffer at each of the 1,884 updates along AdaJEPA's trajectory (LeWM PushT, 3 seeds); every plan is run in the simulator. Rows 1a–1b: the 30 best candidates of the executed plan's CEM population, split by whether they reach the goal. Row 2: pairs of a goal-reaching and a non-reaching candidate in which the goal-reaching one has the lower imagined cost. Rows 3–4: CEM re-solved at the next replan. Row 5: prediction loss on the segment just executed. ±: s.e. over seeds; pp: Wilcoxon signed-rank test over episodes.

Before updateAdaJEPA's updateOpTTA's updatepp
(1a) Goal-reaching elites whose imagined cost rises (%)–61.9 ± 1.630.2 ± 4.0<0.001
(1b) Non-reaching elites whose imagined cost falls (%)–25.6 ± 1.578.4 ± 0.9<0.001
(2) Goal-reaching candidate ranked cheaper (%)60.1 ± 2.062.5 ± 1.964.3 ± 1.60.002
(3) Goal-reaching plans among the next 30 elites8.90 ± 0.198.89 ± 0.199.07 ± 0.210.061
(4) Next selected plan reaches the goal (%)30.8 ± 0.630.8 ± 0.831.9 ± 1.10.471
(5) Prediction loss on the executed segment0.0460 ± 0.00170.0374 ± 0.00140.0389 ± 0.0016<0.001

The main results of this response are added to the revised manuscript. We also request you to take a look at our general response, which lists the new results (persistent runs without resets, a supporting theoretical analysis, and the equal wall-clock comparison.) and the improvements made in the manuscript.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer F4S3,

Thank you again for your review. Our response adds TS results on Medium Maze and PushObj (Table R2), the comparison with Sandwich-Residuals (Table R3) and the candidate-ranking analysis (Table R1). The revised manuscript highlights these additions in blue. We would appreciate it if you could read the response and consider updating your assessment, and we are happy to answer any remaining questions.

Regards, Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response and Revised Manuscript (1/2)

We thank the reviewers for their careful reading. They found the problem relevant and well motivated (mvpR, srvS), the intervention simple (mvpR) and the method effective (F4S3), the comparison with AdaJEPA well controlled (F4S3) and the improvements strong (F4S3, srvS). The main questions concerned the optimistic objective (mvpR), formal analysis (srvS), coverage and baselines (F4S3), and compute (srvS); each is addressed in the individual responses. The revised manuscript adds the results below. The code is available at https://anonymous.4open.science/r/ICLR_46655_codebase/.

1. Broader evaluation (Table R2; F4S3).

On TS Medium Maze and PushObj under visual corruptions, OpTTA outperforms AdaJEPA in all 5 seeds. Averaged over the shifted settings of the expanded Table 1, OpTTA raises final success over the frozen model by 12.9 points, against 6.7 for AdaJEPA, with the predictor adapted (15.5 against 9.8 with the encoder).

2. Sandwich-Residuals (Table R3; F4S3).

Our planning term improves this concurrent method in all five settings (Section 5.2).

3. Persistent adaptation without resets (Table R4; srvS).

OpTTA outperforms AdaJEPA in this setting without reset (Section 5.2).

4. Mechanism of the planning term (Table R1; mvpR, F4S3).

When we apply one AdaJEPA and one OpTTA update to the same data, prediction-only updates make good plans look worse, while OpTTA's update still fits the executed transitions; the paper's sign ablation supports the benefit of the optimistic sign. Section 4 adds a paragraph on this design, and Appendix B background on RBMLE.

5. Formal analysis (new Appendix B; srvS, mvpR).

Prediction-only adaptation can incur linear excess planning cost despite zero prediction loss (Proposition B.1). Theorem B.2 and Corollary B.3 bound how often optimistic adaptation is refuted by observations, and its excess planning cost.

6. Compute (Tables R5 and R6; srvS).

Table R5 (reply to srvS) shows the trade-off between candidates and overhead on DINO-WM; at equal wall-clock time on H200 GPUs, OpTTA outperforms AdaJEPA in 19 of the 24 settings of Table R6. We have rewritten the computational-time discussion in the manuscript to include these results, with a new wall-clock figure (Figure 4).

Table R2. TS, visual corruptions, predictor adapted; protocol and format as Table 1. Medium Maze averages four severity levels per corruption; PushObj averages over its four training shapes.

Setting (shifts)Final: FrozenFinal: AdaJEPAFinal: OpTTAMean over replans: FrozenMean over replans: AdaJEPAMean over replans: OpTTA
TS Medium Maze (7)49.8 ± 1.254.5 ± 1.2 (+4.6)58.0 ± 0.7 (+8.2)37.9 ± 1.139.6 ± 1.0 (+1.7)41.6 ± 0.7 (+3.8)
TS PushObj (7)34.7 ± 1.666.3 ± 0.7 (+31.6)72.6 ± 0.9 (+37.8)24.6 ± 1.242.7 ± 0.8 (+18.1)50.8 ± 0.8 (+26.2)

Table R3. TS, Sandwich-Residuals (SR) adapted, with and without our planning term; format as Table 1, with 3 seeds and 20 replans.

Setting (shifts)Final: FrozenFinal: SRFinal: SR + OpTTAMean over replans: FrozenMean over replans: SRMean over replans: SR + OpTTA
In distribution
TS PushT67.3 ± 1.378.0 ± 1.2 (+10.7)84.7 ± 2.7 (+17.3)41.1 ± 0.448.2 ± 0.9 (+7.1)52.1 ± 1.3 (+11.0)
Visual corruptions
TS PushT (7)42.7 ± 1.056.0 ± 0.3 (+13.3)65.2 ± 1.2 (+22.5)26.0 ± 0.832.4 ± 0.7 (+6.3)38.7 ± 0.4 (+12.7)
Environment shifts
PushT recolors (3)29.8 ± 1.243.3 ± 4.7 (+13.6)45.8 ± 1.4 (+16.0)19.9 ± 1.126.4 ± 2.5 (+6.5)29.3 ± 1.4 (+9.3)
PushObj shapes (3)33.3 ± 2.953.3 ± 2.1 (+20.0)54.0 ± 2.8 (+20.7)20.7 ± 1.729.9 ± 1.7 (+9.2)35.6 ± 1.8 (+14.9)
Maze dynamics (2)80.7 ± 2.984.0 ± 1.2 (+3.3)90.0 ± 2.1 (+9.3)54.2 ± 2.457.3 ± 1.6 (+3.1)57.4 ± 1.3 (+3.2)

(continued in the next comment)

AuthorsOpenReview ↗

General Response and Revised Manuscript (2/2)

Table R1. One AdaJEPA update vs one OpTTA update, applied to the same predictor and buffer at each of the 1,884 updates along AdaJEPA's trajectory (LeWM PushT, 3 seeds); every plan is run in the simulator. Rows 1a–1b: the 30 best candidates of the executed plan's CEM population, split by whether they reach the goal. Row 2: pairs of a goal-reaching and a non-reaching candidate in which the goal-reaching one has the lower imagined cost. Rows 3–4: CEM re-solved at the next replan. Row 5: prediction loss on the segment just executed. ±: s.e. over seeds; pp: Wilcoxon signed-rank test over episodes.

Before updateAdaJEPA's updateOpTTA's updatepp
(1a) Goal-reaching elites whose imagined cost rises (%)–61.9 ± 1.630.2 ± 4.0<0.001
(1b) Non-reaching elites whose imagined cost falls (%)–25.6 ± 1.578.4 ± 0.9<0.001
(2) Goal-reaching candidate ranked cheaper (%)60.1 ± 2.062.5 ± 1.964.3 ± 1.60.002
(3) Goal-reaching plans among the next 30 elites8.90 ± 0.198.89 ± 0.199.07 ± 0.210.061
(4) Next selected plan reaches the goal (%)30.8 ± 0.630.8 ± 0.831.9 ± 1.10.471
(5) Prediction loss on the executed segment0.0460 ± 0.00170.0374 ± 0.00140.0389 ± 0.0016<0.001

Table R4. No reset within a seed: the adapted predictor carries over across all 50 episodes. Visual corruptions, predictor adapted; otherwise as Table 1.

Setting (shifts)Final: FrozenFinal: AdaJEPAFinal: OpTTAMean over replans: FrozenMean over replans: AdaJEPAMean over replans: OpTTA
LeWM PushT (7)58.5 ± 0.857.8 ± 1.4 (−0.7)61.7 ± 0.9 (+3.1)43.9 ± 0.743.1 ± 1.0 (−0.8)47.1 ± 0.7 (+3.2)
LeWM Reacher (7)52.8 ± 0.560.9 ± 0.4 (+8.1)64.1 ± 0.5 (+11.3)38.4 ± 0.640.5 ± 0.4 (+2.0)42.8 ± 0.6 (+4.3)
TS PushT (7)47.8 ± 1.562.1 ± 1.3 (+14.3)68.8 ± 0.9 (+21.0)34.0 ± 1.341.7 ± 1.1 (+7.7)48.4 ± 0.8 (+14.4)

Table R6. Final success (%) within the time AdaJEPA needs for its 30 replans, on Table 1's runs: (a) predictor adapted, (b) predictor and encoder adapted. OpTTA replans: replans OpTTA completes in that time, from each method's mean time per replan on the H200 GPUs these runs used (measured as in Table 2). Parentheses: OpTTA − AdaJEPA, paired by seed. Time to match: the time OpTTA needs to reach AdaJEPA's final success, in % of AdaJEPA's budget (–: not reached within 30 replans).

Setting (shifts)OpTTA replansFinal, equal time: AdaJEPAFinal, equal time: OpTTATime to match (%)
(a) Predictor adapted
In distribution
LeWM PushT2688.8 ± 1.492.0 ± 1.9 (+3.2)53
LeWM Reacher2798.8 ± 0.8100.0 ± 0.0 (+1.2)66
LeWM TwoRoom2593.2 ± 1.290.8 ± 0.5 (−2.4)–
TS PushT2781.6 ± 2.590.4 ± 1.8 (+8.8)51
DINO-WM PushT1977.3 ± 5.280.7 ± 4.8 (+3.3)75
Visual corruptions
LeWM PushT (7)2662.4 ± 1.068.2 ± 0.7 (+5.8)49
LeWM Reacher (7)2660.3 ± 0.568.5 ± 0.9 (+8.1)71
LeWM TwoRoom (7)2657.6 ± 1.556.6 ± 1.4 (−1.0)–
TS PushT (7)2758.3 ± 1.273.0 ± 1.3 (+14.7)56
DINO-WM PushT (7)1957.1 ± 4.556.2 ± 3.1 (−1.0)110
Environment shifts (TS)
PushT recolors (3)2740.8 ± 1.653.9 ± 0.8 (+13.1)51
PushObj shapes (3)2755.2 ± 2.058.1 ± 1.9 (+2.9)63
Maze dynamics (2)2687.6 ± 2.088.0 ± 1.6 (+0.4)99
(b) Predictor and encoder adapted
In distribution
LeWM PushT2786.8 ± 1.590.0 ± 2.0 (+3.2)47
LeWM Reacher2799.2 ± 0.5100.0 ± 0.0 (+0.8)85
LeWM TwoRoom2192.4 ± 1.091.2 ± 0.8 (−1.2)–
TS PushT2783.2 ± 2.990.8 ± 1.6 (+7.6)55
Visual corruptions
LeWM PushT (7)2763.5 ± 0.868.3 ± 1.0 (+4.8)48
LeWM Reacher (7)2662.3 ± 1.168.1 ± 1.1 (+5.8)76
LeWM TwoRoom (7)2657.1 ± 1.257.7 ± 1.4 (+0.6)90
TS PushT (7)2660.7 ± 1.973.6 ± 1.5 (+12.9)56
Environment shifts (TS)
PushT recolors (3)2744.5 ± 1.353.9 ± 1.7 (+9.3)59
PushObj shapes (3)2755.5 ± 1.157.3 ± 2.2 (+1.9)77
Maze dynamics (2)2687.6 ± 1.787.0 ± 1.5 (−0.6)107

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.