acceptodds
Withdrawn from ICLR 2027

Adapting World Models through Response-Preserving Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

A world model can predict the next observation almost perfectly and still be the wrong model to plan with, because deployment physics rarely match the simulator it was trained in. A controller must read the change from a short prefix of frames and actions, then apply the same correction to every future it imagines. We show that ordinary forward training quietly undermines the first step: a history-dependent predictor explains part of a shifted successor from the history itself, absorbing the change and leaving too little residual for any calibration procedure to read. Forward and controlled-pair objectives cannot tell this allocation from an informative one, since models that divide the change differently make identical predictions. Response-preserving representations (RPR) fix the allocation rather than the symptom: the encoder keeps a grid of spatial motion tokens, controlled source pairs anchor an explicit response channel, and encoder, predictor, and response are trained through the very weighted ridge solve that runs at deployment. With the networks frozen, one solve on the prefix returns the change coefficient and a calibrated covariance; the coefficient is reused at every imagined step and refreshed as observations arrive. A linear-Gaussian analysis ties retained response to prefix estimation risk. On three simulated visual-control tasks, RPR recovers the change from eight transitions in under nine milliseconds, matches twenty-step gradient adaptation on the all-profile mean, and surpasses it wherever the change lies in the source family, while true-coefficient controls and absorption diagnostics attribute the gain to a better estimate and a better forward model alike.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
world models, test-time adaptation, dynamics shift

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_73iJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

It seems this paper is about adapting world models from simulation to test environment.

Critical strengths

None

Critical weaknesses

The text is full of jargon and is very difficult to understand, with words and terms that are not standard in academia. I did not understand the paper at all, and everything is unclear to me. Therefore, I do not have any assessment of this paper and recommend that the authors revise the text to make it understandable to human readers.

Questions for authors

None

Reviewer_fBqnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The setting is a model trained in simulation, then deployed in a different scenario where the transition dynamics of the environment may change. Given a short sequence of transitions, the goal is to identify how the dynamics in the current episode differs on the established set of axes, then update the world model to account for these changes for the duration of the episode. The work identifies a problem associated with this approach called predictor absorption, where a part of the network constructed for the static dynamics absorbs part of the changing dynamics, limiting the extent to which the changes can be modeled during test time. They propose RPR as a solution, which introduces an objective that encourages the network to keep the changing dynamics only in the prescribed part of the network. The experiments directly measure how this affects predictor absorption and how this affects performance in comparison to related methods.

Critical strengths

Starting the work with the identification of predictor absorption serves as a clear motivation for the proposed method. The visual examples helped convey understanding of the phenomenon. The solution appropriately addresses the problem and is thoroughly evaluated. I particularly appreciated the comparison to test-time-training (TTT), which I believe is a good more general baseline despite being a different solution class.

Critical weaknesses

1. The paper is hard to follow because of the density and writing style. The introduction and related works are particularly difficult to follow because it uses a number of terms that are only defined in later sections. Examples of ambiguous terminology include forward objective, forward training, spatial token grid, controlled-pair objective, prefix, prefix estimate, and response channel. Even if some terms are commonly used in this subfield, they are not common enough in the general AI/ML literature to forgo an explanation. I think rewriting the introduction without jargon would significantly improve readability and more clearly convey the story. I would consider increasing my score if the writing and presentation of the work is improved.

2. The proposed algorithm only works in very limited settings. The method requires knowing what will change in advance and being able to simulate each variable that may change in the target environment. It is also only practical for environments where the axes of non-stationary are small in number because it requires a large number of controlled pairs for each axis. Such approaches have inherent limitations in complex environments like the real world, where all the ways in which the work can change are inexhaustible and cannot be fully anticipated. Methods like TTT perform similarly on most of the benchmarked problems while making significantly less assumptions. For these reasons, I do not see how this work could have a lasting and meaningful impact.

Questions for authors

My main concern of the work is its limited generality as posed in Weakness 2. Is there a path where this line of work leads to more general algorithms? Or am I missing any other ways in which this work could have a meaningful impact on the research community?

Reviewer_roKWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies test-time calibration of latent world models under changed physics. The model has an explicit linear response to a few physical coefficients, which are estimated at deployment from a short prefix with a single ridge solve and then kept fixed during planning. The authors argue that standard forward training lets the predictor absorb the shift, so the estimate gets no signal, and propose to train the representation through the ridge solve, supported by controlled simulator pairs. Experiments on planar drag, Push-T and OGBench cube with injected gain/drag show results comparable to gradient-based test-time training at a fraction of the fitting cost.

Critical strengths
  • The diagnostics separate estimation error from representation error (true-cc intervention, absorption/retention measures, ablations; Tables 2, 7, 8). This is more than most world-model adaptation papers do.
  • Cheap closed-form calibration with a calibrated posterior (Table 14), and the authors are explicit about where it stops working.
Critical weaknesses
  • Limited novelty
    • Learning features such that a closed-form linear adaptation works at test time is the core of R2-D2 and ALPaCA (cited) and of Richards et al. (2021, cited).
    • Neural-Fly (O'Connell et al., 2022) learns a basis such that the residual dynamics are linear in a low-dimensional, adapted coefficient. This is very close to F+BcF+Bc and not cited.
    • CAVIA (Zintgraf et al., 2019) and CoDA (Kirchmeyer et al., 2022) adapt a low-dimensional context of a shared dynamics model at test time.
    • What remains is the visual latent setting, the controlled pairs and the absorption diagnosis. The paper presents training through the deployed estimator as its main conceptual contribution.
  • The empirical advantage is small and partly built in
    • In Table 1, the lead over TTT-FF/TTT-LoRA is there but on Planar drag and Cube with the six-profiles RPR loses. In the native-contact setting (Table 4) it ties at p=4p=4 and loses on composition (39.8 vs 44.8). The claim "surpasses it wherever the change lies in the source family" (abstract) is not supported.
    • The gain–drag tasks are exactly affine in cc by construction (App. C.1), which favours a model F+BcF+Bc with a ridge solve. In the more realistic native-contact setting there is no advantage.
  • The 100 ms fitting cap (Table 13) is not motivated, and it is exactly where 20-step TTT times out. The fit runs once per episode, and the setup is not real-time anyway: planning alone takes ~72 ms per step (Table 3), longer than the 50 ms control interval (App. C.1). Without the cap, TTT-LoRA (69.6 ± 20.0) and RPR (78.8 ± 12.0) are within the confidence intervals.
  • Absorption is not shown to matter. In Fig. 1 the absorbed model still predicts the shifted successor correctly, it only returns c^=0\hat c = 0. It can only hurt planning if the absorption breaks down on imagined histories ("context drift", L73), and this is never measured. The poor true-cc result of the forward-pair baseline (Table 7) may just reflect a weakly trained BB (KB=0.29K_B = 0.29).
Questions for authors
  1. Can you report paired differences (RPR minus TTT-LoRA / TTT-FF per scenario, with confidence intervals) for Tables 1 and 4?
  2. Can you show directly that absorption breaks down during imagination, e.g. one-step error of the forward-pair model with its own c^\hat c on real vs. imagined histories?
  3. How does RPR compare to a Neural-Fly-style objective, and to a context-only adaptation baseline (CAVIA-style: a few gradient steps on a pp-dimensional context)? The latter sits between the ridge solve and full TTT.
  4. Why is a 100 ms cap the right budget for a once-per-episode fit? What is the end-to-end episode wall-clock of TTT-LoRA vs. RPR? And what does the 14.0 s "adaptation seconds" for fixed-prefix RPR in Table 17 include?

Suggestions:

  • A main task where the affine form is only an approximation (e.g. native contact with more factors).
  • Can RPR work without controlled pairs, which require simulator state restore? This matters for real data.

Minor points / Details

  • Fig. 1 appears before any notation is introduced. cc, c^\hat c, FF, BB, rr and λ\lambda are defined only in Sec. 3, and mm, gg and yy only in App. A. Neither the caption nor panel (a) ("local crops", tn∼tn+αt_n \sim t_{n+\alpha}) is understandable on first reading. Please define the toy example in the caption.
  • Prop. 1 is the standard bias of ridge regression with a mis-scaled regressor; please say so.
  • Eq. 1: "B=∂G/∂cB=\partial G/\partial c exactly" holds by construction; for the gain–drag tasks it is exact in physical space, so it does not test the approximation.
  • Sec. 3.3: a diagonal Σ^\hat\Sigma over d=8192d=8192 correlated channels makes the posterior overconfident; please report the typical correction γT\gamma_T.
  • Sec. 4.1: are the five seeds training seeds? Please state it.
  • Table 13: TTT-FF with 5 steps reaches 62.2% in 77.5 ms, within the cap and the CI of RPR (78.8 ± 12.0).
  • Table 15: please report the sensitivity to λinv\lambda_{\text{inv}} and λ\lambda.
  • Table 2: why does "Anchored FF" have the lowest absorption but also lower retention?
  • Fig. 4 is too small.
  • The 2026 works central to the motivation (LeWM, AdaJEPA, etc.) should be discussed in the main text.
  • Missing related work, most relevant:
    • M. O'Connell, G. Shi, X. Shi, K. Azizzadenesheli, A. Anandkumar, Y. Yue, S.-J. Chung. Neural-Fly enables rapid learning for agile flight in strong winds. Science Robotics 7(66), 2022. (Closest prior method.)
    • L. Zintgraf, K. Shiarlis, V. Kurin, K. Hofmann, S. Whiteson. Fast Context Adaptation via Meta-Learning (CAVIA). ICML 2019.
    • M. Kirchmeyer, Y. Yin, J. Donà, N. Baskiotis, A. Rakotomamonjy, P. Gallinari. Generalizing to New Physical Systems via Context-Informed Dynamics Model (CoDA). ICML 2022.
    • V. Shaj et al. Multi Time Scale World Models (MTS3). NeurIPS 2023. (Slow latent context inferred with closed-form Gaussian updates.)
    • S. Prasanna, K. Farid, R. Rajan, A. Biedenkapp. Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization. RLC 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.