acceptodds
Under review as a conference paper at ICLR 2027

In-Context Adaptation of Dynamics Models for Control

Abstract

Learned dynamics models typically fail to generalize to environments with unseen physical parameters. Standard approaches add adaptation machinery, such as test-time gradient updates or an inferred latent variable trained with an auxiliary objective. We show that this machinery is not what decides the matter, and that whether a model adapts in context is settled by how it is trained and by where it is tested. A plain sequence model, with no module for inferring the environment, no loss beyond next-state prediction, and no positional encoding, adapts to unseen physics in a single forward pass with its weights frozen. Its context is simply the recent transitions, each kept whole so that a state, the action taken, and the resulting next state stay together. On the six continuous-control environments of lee2020cadm, under their protocol and interaction budget, it outperforms the vanilla, stacked, and meta-learned model-based baselines everywhere, and an ensemble variant clears both variants of the purpose-built context-aware method outright on three of the six while staying inside seed noise on two more. Interventions that leave the weights untouched show what the model uses. Permuting which next state follows which transition costs almost as much as removing the context outright, and replacing the transitions by their average costs two-thirds of the benefit, so the model addresses individual transitions rather than a summary of them. What the context is worth also depends on where the model is tested, moving by a factor of five within one environment between held-out ranges that sit equally far from training. We report where the approach does not work. On Ant, the context is worth nothing, and no intervention we tried changes that, including a fivefold widening of the training distribution.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
in-context learning, dynamics generalization, world models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bkChOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies dynamics generalization, where a learned dynamics model must adapt to a new environment at test time. Using the continuous-control environments of the CaDM benchmark, the authors demonstrate that a standard sequence model using recent state-action-next-state transitions as context can outperform vanilla, stacked, and meta-learning baselines while remaining competitive with specialized context-encoder approaches. The paper further investigates the role of context through a series of ablations that modify the transition history and analyzes how performance varies under different train-test distribution shifts.

The central finding is interesting: full transition histories appear to provide a powerful mechanism for context-based adaptation, enabling effective zero-shot generalization through a single forward pass without requiring additional adaptation procedures or test-time optimization. However, the exposition of the paper is lacking and he work would benefit from deeper analysis to support some of its broader conclusions

Critical strengths

I found the main direction of the paper interesting and worthwhile. If I were to summarize its strengths:

(+) Introduces full state-action-next-state transitions as a simple yet effective mechanism for representing context

(+) Enables genuine zero-shot adaptation through a single forward pass without requiring additional optimization procedures, gradient updates, or specialized adaptation modules at test time

(+) Achieves competitive performance compared to both meta-learning and context-encoder approaches

(+) Includes an extensive collection of benchmarks, ablations, and failure-case analysis

Critical weaknesses

While I appreciate the paper's central empirical finding, I believe the current version has a number of issues related to clarity, experimental analysis, and interpretation of results:

(-) The text needs polishing since it is hard to parse at times obscuring the potential contributions of the proposed work.  For example

  • Lines 25-26:  "Permuting which next state follows which transition costs almost as much as removing the context outright,"
  • Line 51: "...and an ensemble variant clears both variants of the purpose-built context-aware method outright on three of the six while staying inside seed noise on two more."   - Final contributiom bullet point
  • Line 156; "Keeping the next state inside the item": what item?
  • Lines 266-268: "On HalfCheetah, where CaDM maintains...or search budget."

In addition, certain paragraphs feel too AI generated for my liking. See, for example, "Changing what the context is" in line 365 and all subsequent paragraphs in the results section

(-) The ensemble results play an integral role in the experimental evaluation, yet the paper provides very limited information. It is unclear what differs across the independently trained models and how ensemble diversity is achieved. The authors should provide more information rather than expecting reviewers to refer to refer to a 2018 paper to understand the experimental setup.

(-) One of the central claims is that the model extracts useful dynamics information from individual transitions rather than from long temporal contexts. However, I do not believe the presented evidence fully supports this conclusion. Figure 4 is a useful starting point, but similar analysis should be reported for all environments and moved into the main paper. Furthermore, in the extreme settings, performance appears to continue improving as additional context transitions are provided. This observation contradicts with the authors claim that performance saturates after only 2-4 transitions.

(-) Several analysis focus on 1-2 environments while providing limited discussion of the remaining environments. For example, the sections "Corrupting the context" and "Changing what the context is" focus primarily on HalfCheetah. Similarly, Figure 5 is difficult to interpret. It is unclear what varies in the reported "above-training-range" and "below-training-range" settings, and what is the baseline for the reported mean differences (is t no context?) In any case, statements like "This confirms that the direction of distribution shift, rather than its magnitude, governs context utility" are difficult to justify based on a small subset of environments.

(-) The authors argue that individual transitions are particularly informative. However, I believe the sequential structure of the context may itself play an important role. In particular, the proposed approach leverages a history of observed transitions to infer latent environment dynamics, thereby utilizing information that is not available under a strict first-order Markov assumption. In this sense, results such as those in Figure 2, where context shuffling or the use of transitions from different episodes substantially degrades performance in challenging environments such as Ant and Humanoid, seem to suggest that temporal consistency and sequence structure remain important factors.

(-) Finally, it's not clear how well the approach can extend to more challenging domains. The authors note that performance degrades in the out-of-training Ant setting, raising questions about scalability to more complex environments. While evaluating additional domains such as robot manipulation may be beyond the current scope, the paper would benefit from further anlysis of the failure cases and the limitations of the proposed context mechanism

Questions for authors
  • What do the numbers in table 14 report? Improvement over the single context model or the no-context one?

  • Similarly, see my comments above and clarify the results in Figure 5

  • Could you provide a full version of Figure 4 that includes all environments

Reviewer_SUeWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies dynamics generalization under changes in physical parameters. It uses recent real state transitions as a Transformer's context, trains solely through next-state prediction, and freezes the weights at test time while using MPC for control. The reported experiments suggest that this simple context-conditioned model is competitive. Interventions that remove or corrupt the context examine when online history is useful and where its benefits are limited. I find the topic interesting and particularly appreciate this analysis.

Critical strengths
  • The method is simple and directly addresses prediction and control under unknown dynamics. Inference is straightforward, and the additional computation appears modest to me based on the reported results.
  • Context interventions with frozen weights partially isolate the role of online history. These ablations provide evidence that context can support dynamics inference and adaptation.
  • I appreciate the analysis of cases where context provides little benefit. Reporting the lack of adaptation gains on Ant and failures under extreme shifts helps clarify the method's boundaries.
Critical weaknesses
  • Existing recurrent models can adapt through forward computation. It remains unclear what demonstrated advantages retaining individual transitions. References [1, 2, 3] provide further context on in-context dynamics modeling.

[1] Nagabandi, A., Clavera, I., Liu, S., Fearing, R. S., Abbeel, P., Levine, S., & Finn, C. (2018). Learning to adapt in dynamic, real-world environments through meta-reinforcement learning. arXiv preprint arXiv:1803.11347.

[2] Bazzi, M. B., Shahid, A. A., Agia, C., Alora, J., Forgione, M., Piga, D., ... & Roveda, L. (2024). RoboMorph: In-context meta-learning for robot dynamics modeling. arXiv preprint arXiv:2409.11815.

[3] Wang, F., Chen, Z., Zhong, Y., Zheng, S., Shao, P., Yu, B., ... & Kang, Y. (2026, April). Context and diversity matter: The emergence of in-context learning in world models. In International Conference on Learning Representations (Vol. 2026, pp. 100857-100880).

Questions for authors
  • Do the key performance advantages hold when the same simulator is used? Could you reproduce CaDM and PE-TS using the same environment version, interaction budget, and planning settings?
  • Could you provide a systematic analysis of exactly how much computational cost and latency are added when using context compared with not using it? I would like to see whether it is actually worth the cost.
  • Could you explain the design rationale for how the training setup encourages the model to infer dynamics adaptation from context?
Reviewer_arfVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies whether a learned dynamics model can generalize to unseen physical environments without explicit adaptation machinery, such as test-time gradient updates or an auxiliary adaptation objective. The method uses a Transformer trained with next-state prediction. It receives a context of recent transitions and predicts the next state for a candidate action. At deployment, model weights remain frozen, and the context is updated using real transitions. The controller is a MPC (model-predictive control). The method is evaluated on 6 continuous-control environments from the CaDM benchmark.

Critical strengths
  • This is a well-motivated and valuable empirical paper. The authors show that an ordinary next-state trained Transformer can produce useful in-context dynamics adaptation in model-based control. They also provide a careful investigation of when this capability helps and fails.

  • The ablation studies and interventions are thoroughly studied: context removal, scrambing, etc. These experiments move beyond reporting aggregate returns and attempt to explain why context helps. This is significantly stronger than many empirical RL papers.

  • Negative results discussion for the Ant environment. Discussing this failure case adds credibility to the paper.

Critical weaknesses

The main claim of the paper is that a Transformer trained for dynamics prediction can infer physical parameters from context without an adaptation mechanism. This idea is closely related to a growing body of works on in-context learning and in-context world models. The papers cites a few of these works: Wang et al. (2025), Cole et al. (2024), Schaeffer et al. (2024), Prasanna et al. (2024). Positioning against these works is missing : if these works already have adaption capability from context alone, what is the novelty beyond the evaluation of this capability on the CaDM benchmark ? Can you add these baselines to the comparison ?

Other relevant references that are not cited and discussed:

  • T-MCL (NeuriPS 2020) from the same group as CaDM
  • Transformers are Meta-Reinforcement Learners, ICML 2022
  • “Human-timescale adaptation in an open-ended task space” (2023)
  • Supervised pretraining can learn in-context RL (NeurIPS 2023)
  • TransDreamer (Chen et al., 2022) and IRIS (Micheli et al., 2023) : transformer world models for control
  • Trajectory Transformer (Janner et al., 2021)
Questions for authors
  • Could you compare against at least a modern Transformer world model without context ?
  • Can you add at least one additional benchmark family outside Lee et al. (2020) ?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.