- Rating
- 4
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work presents a Bilinear World Model that imposes structure on the latent dynamics. Instead of learning a representation and a forward model using separate neural models, which can lead to representation collapse, the proposed method is constructed in a way where the learned state representation must adhere to the specified dynamics. Given such a method, an action can be recovered directly from the latent transition. The authors show that this Bilinear World Model reduces planning time by three orders of magnitude while matching or improving over current SOTA World Model approaches.
This work takes a novel and promising direction. I recommend accepting it.
Critical strengths
-
The idea is clever and effective. Restricting the dynamics model and forcing structure into the latent representation is timely and creative. The results show that this is a promising direction to improve World Model representation and planning.
-
The authors compare against SOTA World Models (SMWM and LeWM). They show that the Bilinear model matches or surpasses performance while significantly reducing planning time.
Critical weaknesses
- A conclusion and, especially, a limitations section are missing completely. This needs to be addressed.
Questions for authors
- What's still a bit unclear to me is the source of the speedup. In the main text, the source is unclear, while in the appendix it reveals a bit that the speedup is not due to the proposed bilinear model, but a lot of it seems to come from the Gauss-Newton planner itself and related tricks like the reduced action space mentioned in B.2.3. It would be helpful to be more explicit in the main text.
We sincerely thank the reviewer for their encouraging assessment of our work. We particularly appreciate their recognition of the proposed structured dynamics as a promising direction for improving world-model representations and planning efficiency. We agree that the manuscript would benefit from a more explicit discussion of its limitations and a clearer explanation of the source of the computational speedup. We address both points below.
1. Conclusions and limitations
We agree that adding a conclusion and stating clearly the limitations is necessary. We combined the requests into a Conclusion and Limitations section, which reads:
In addition, we invite the reviewer to consider the additional experiments presented in our response to reviewer 7NTT, which provide further insights into the learned representations and the limitations of our approach in more complex environments.
2. Source of Speedup
We agree with the reviewer that we should make more explicit how the bilinear dynamics enable faster planning, and appreciate the opportunity to improve the presentation of our work.
The speedup comes from using the Gauss–Newton (GN) planner instead of CEM. The proposed bilinear dynamics enable the use of GN without losing performance.
The GN can also be applied to unstructured neural-network dynamics. However, our experiments in Appendix C.2 (Figure 7) show that applying the same planner to SMWM and LeWorld results in substantially degraded control performance.
This is because GN relies on first-order information about the dynamics (i.e., derivatives). In our method, these derivatives are directly available through the matrix (M(z)) of the bilinear parameterization. For neural-network dynamics, these derivatives can be computed through automatic differentiation. However, neural networks are trained for discrete transition accuracy (i.e., predicting the next state given the current state and action), which does not guarantee that differentiating their predictions with respect to the action yields useful information for gradient-based planning, which is exactly the limitation we see in practice.
In summary, structured bilinear dynamics give access to accurate first-order information, enabling the use of planners such as GN. We hope this addresses the reviewer's question and would be happy to provide further clarification.