acceptodds
Under review as a conference paper at ICLR 2027

BRICKS-WM: Building Reusable Dynamics Networks for Compositional World Models

Abstract

Model-based reinforcement learning (MBRL) has achieved remarkable success in continuous control by leveraging latent world models. However, monolithic world models couple distinct dynamics in a single transition dynamics model, making it difficult to reuse learned dynamics separately from the original predictor. To address this, we introduce BRICKS-WM (Building Reusable dynamICs networKS for compositional World Models), a framework for composing world models from reusable dynamics modules. Motivated by the insight that the physical world is composed of entities, we model global dynamics as a composition of distinct dynamics modules interacting via latent interfaces. As a minimal instantiation, we factorize the environment dynamics into an actuated Agent dynamics module and an external Background dynamics module, bridged by a learned latent interface. For reuse, we recompose the world model by combining a previously learned Background dynamics module with a newly learned Agent dynamics module through an adapted latent interface. Empirically, BRICKS-WM achieves control performance comparable to strong monolithic baselines when trained from scratch, and enables the reuse of background dynamics across scenarios.

Primary area:
reinforcement learning
Keywords:
Model-Based Reinforcement Learning, Structured World Model, Modular Dynamics, Object-Centric Representations

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_pykKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes BRICKS-WM, a Dreamer-style world model that is split into an agent module and a background module with slot attention, where the background receives information from the agent only through a small latent interface. For reuse, the background module, encoder, and decoder are taken from a source task, and a new agent module, interface, and adapter are trained on the target task. The authors report that BRICKS-WM trained from scratch reaches scores comparable to DreamerV3 and TD-MPC2, and that reusing the background dynamics improves early learning in related tasks. The experiments use four DMC locomotion tasks and two CarRacing variants with 500K environment steps.

Critical strengths

The split of a DreamerV3-style world model into an agent module and a background module, where the agent affects the background only through a small latent interface, is simple and clear, and the paper gives a concrete algorithm for reusing the background module. I think whether parts of a learned world model can be reused for a new agent is an interesting question, and it is good that the appendix reports all 14 transfer pairs including the ones where reuse fails (Tables 4, 5, and 7) and says openly what is not tested, e.g., that the condition of Proposition 4.3 is not assessed (lines 1296-1297).

Critical weaknesses

My main problem is that the results do not seem to support the claim of modular dynamics reuse. Compared to DreamerV3 trained from scratch (Table 6), reuse reaches a higher mean final return in only 2 of the 14 pairs in Table 4, and in some pairs only about half (e.g., Cheetah Run -> Walker Run 318 vs. 636). The clearest gains are in pairs where source and target are closely related (Walker Walk <-> Walker Run, and Sport Car <-> Heavy Car compared to BRICKS-WM from scratch). For these pairs I would expect that simply fine-tuning DreamerV3 from the source checkpoint also works well, but this baseline is missing. So I cannot tell whether the transfer works because of the separate background module or simply because source and target are so similar, and this is the central claim of the paper.

The second problem is that Tables 6 and 8 only include the baselines and not BRICKS-WM trained from scratch, although performance "on par with" DreamerV3 and TD-MPC2 is one of the three contributions. The final DMC returns in Figure 2 seem close, but the few numbers I found point to slower learning, e.g., a 200K AUC on CarRacing of 164 and 141 vs. 406 and 345 for DreamerV3 (Table 8 and lines 1112-1113).

Questions for authors
  1. How does pretraining DreamerV3 on the source task and fine-tuning on the target task compare to reuse, for example on Walker Walk -> Walker Run and Sport Car / Heavy Car? If reuse is not better, I think the benefit of the modular split is not shown.
  2. Could you add BRICKS-WM from scratch to Tables 6 and 8, state the number of seeds for the DMC transfer and baseline results, and clarify which parameters (e.g., actor, critic, agent decoder) are re-initialized, frozen or fine-tuned in reuse?
Reviewer_DNhbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

BRICKS-WM splits a Dreamer-style world model into two modules, one for the agent and one for the background, connected by a small learned interface. The authors report that, trained from scratch, the model matches DreamerV3 and TD-MPC2 from pixels, and that the background's dynamics can be reused for a new agent: freeze the background's recurrent core, then train only a new agent module and interface. Experiments cover four DMC locomotion tasks and two CarRacing cars.

Critical strengths
  • The framing changes what can be tested rather than adding a component. Once the world model is a set of modules joined by an interface, you get a falsifiable prediction that the usual single-model view doesn't make: freeze the background, retrain only the agent side, and a new agent in the same environment should learn faster than one that inherits perception alone. The code implements this, and on CarRacing reuse clearly outperforms the paper's own scratch model (early AUC goes from 164 to 461 for the sports car), roughly reaching DreamerV3. This improvement by itself doesn't tell me whether the frozen background core or the inherited perception is doing the work.
Critical weaknesses
  • The reuse effect hasn't been separated from noise. The only control that splits reused dynamics from reused perception is BRICKS-WM-RBG in Table 2. Reuse has the higher mean in most cells, but every gap is within about one standard deviation and there's no statistical test. The paper also doesn't say how many DMC seeds were run, or whether the RBG background core is trained or frozen. And with no DreamerV3 baseline warm-started from the source checkpoint, there's no way to know whether modular reuse does better than the obvious alternative. As it stands, the data are just as consistent with no effect.
  • The approach was only tested where it can't fail. In every task the background moves only because the camera follows the agent; it has no motion of its own, so these benchmarks can't distinguish dynamics reuse from perception reuse. Parity with the baselines rests on the curves in Figure 2 without a table, and on CarRacing the scratch model is far behind DreamerV3 early in training. When the morphology changes, transfer often ends well below DreamerV3 trained from scratch (into Walker Run, about 318 versus 636). Scoped down to what the evidence shows, what's left is a modular architecture that roughly matches single-model baselines plus a small head start on related tasks, an ordinary incremental result.
Questions for authors
  1. How many seeds are behind each cell of Table 2 and each curve in Figure 2, and what does "±" mean for DMC? If reuse clearly exceeds RBG with five or more seeds and a paired test, and isn't worse than a warm-started DreamerV3 on the related pairs, I would consider the reuse claim established.
  2. During target training, is the reinitialized RBG background core trained or frozen?
  3. Can you report BRICKS-WM-from-scratch numbers on DMC and CarRacing at 200K and 500K steps?
  4. What do you expect when the background has its own dynamics, for example moving external objects?
Reviewer_RDtxOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a factored approach to learning a recurrent state space world model called BRICKS-WM. Broadly speaking, this is a version of of Dreamer that incorporates additional structure that distinguishes the background'' dynamics from the dynamics of various agents'' that act in the world. There are several key ideas: 1) learning a dynamics model with the factoring just described; 2) the idea that the background depends upon the behavior of the agents only through an interface latent variable, z; 3) an information bottleneck can be used to minimize the effect of agent on background; 4) each agent is grounded in a masked part of the scene, obtained by using slot attention; 5) composition of these various model pieces at inference time happens fundamentally as a protocol matching problem. The paper presents results (fig 2) that show: 1) this approach to modeling via a composition of models underperforms the best monolithic modeling approachs, ie dreamerv3, but it is not a huge gap; 2) when transferring dynamics knowledge to new, related, problems the compositional factoring can lead to outperformance relative to a monolithic model learned from scratch. There are appropriate ablations in tables 1 and 2.

Critical strengths

I found the overall compositional approach compelling. Although this model does not outperform in a huge way, I think it is interesting and has potential for further gains down the road

I think the info bottleneck idea and the interface idea were intuitively appealing.

The paper outperforms dreamerv3 (a strong baseline) in transfer learning scenarios.

Critical weaknesses

The model does not outperform dreamerv3 by a huge amount.

There are a lot of design choices that probably warrant followup work to understand tradeoffs.

Questions for authors

None

Reviewer_TW8HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes BRICKS-WM, a framework for building compositional world models from reusable dynamics modules. BRICKS-WM factorizes the latent world model into an Agent dynamics module and a Background dynamics module, connected through a learned low-dimensional latent interface. The framework uses Slot Attention and an asymmetric reconstruction decoder to learn disentangled representations from pixels, while an information bottleneck regularizes the interface to encourage transferable interaction signals. For cross-task reuse, the authors retain the pretrained background dynamics and perceptual components while learning a new Agent dynamics module and adapting the latent interface. Experimental results show that BRICKS-WM achieves broadly comparable control performance to monolithic baselines when trained from scratch and that reusing background dynamics can improve learning efficiency in certain transfer settings.

Critical strengths
  1. The paper addresses an important problem in world modeling: how to learn reusable dynamics components that can be transferred and recomposed across different agents and environments.

  2. BRICKS-WM introduces a structured decomposition into Agent and Background dynamics modules connected through a learned latent interface. The design is intuitive, and the use of an information bottleneck to encourage a compact interaction interface provides a reasonable inductive bias.

Critical weaknesses
  1. The paper mainly compares against DreamerV3 and TD-MPC2 trained from scratch, without considering pretrained monolithic world models with fine-tuning or partial freezing. Such comparisons are important to establish the advantages of modular dynamics reuse over conventional transfer learning.

  2. The effectiveness of dynamics reuse varies substantially across source–target pairs, with some transfers providing limited benefits or performing poorly compared with scratch baselines.

  3. The proposed unidirectional factorization does not explicitly model the influence of background dynamics on agent dynamics. This assumption may be overly restrictive for environments involving bidirectional interactions, such as contact-rich manipulation and locomotion.

Questions for authors
  1. The interface policy generates its latent signal from the predicted next Agent state and action. Why is this conditioning chosen?

  2. The background RSSM recurrent core is frozen, while its output layers remain trainable. What motivates this particular partition, and how much of the pretrained background dynamics is actually updated during target adaptation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.