- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
World models are neural networks that learn to predict how an environment evolves, usually inside a compressed internal representation, and they are increasingly used for planning. What dynamics they have actually learned is opaque, and it is hard to tell when their multi-step predictions have drifted away from reality. Most prior work probes these representations for individual physical quantities such as position or velocity. This paper goes further and tries to extract an explicit, interpretable dynamical system from a trained world model.
The proposed method, H-Lens, works on a frozen world model. It rolls the model out with no actions and trains a small network that compresses each internal state into a few position-like and velocity-like variables. A Hamiltonian neural network is trained alongside it: this is a network that learns an energy-like function and, assuming energy is roughly conserved, uses that function to determine how the variables evolve. Plotting the resulting flow gives a picture of the learned dynamics. The accumulated disagreement between this flow and the world model's own predicted steps is proposed as a way to detect rollout degradation without ground truth. The experiments cover three world models trained on low-resolution images of three classic control tasks (pendulum, mountain car, lunar lander), with both normal and altered physics. They show qualitatively plausible flow plots that change in the expected direction when the physics is altered. The error signal rises alongside the world model's prediction error in most, but not all, model–task combinations.
Critical strengths
- Works on any trained world model without retraining it. H-Lens only needs predicted sequences from a frozen model. The paper applies it unchanged to three world models built in very different ways (Dreamer, LeWorldModel and DINO-WM), so it is a reasonable starting point for inspecting what dynamics a world model has learned.
Critical weaknesses
- The first claimed contribution, recovering the dynamics a world model has learned, is not demonstrated. The evidence is plots, and the main comparison cannot show what it claims.
- The Limitations section states that the recovered coordinates are only determined up to shifts, rescaling and sign flips. Yet Sec. 5 and Figs. 2–4 argue that the altered physics is recovered by comparing axis ranges between two separately trained Hamiltonian networks: pendulum velocity about 0.04 vs. 0.10, and mountain-car velocity 0.3 vs. 1.0 with position 0–1.25 vs. 2–7. Each network can rescale its coordinates freely, so these differences say nothing about the world models.
- The axes are labelled "Angle (θ)", "Velocity (ẋ)" and so on, but the paper never measures how well the recovered variables match the true angle, position or velocity. The true states are available, since they are used to train the probes.
- The method's core assumption, that energy is roughly conserved (Sec. 1), is never checked, and it does not hold for lunar-lander free fall or for the capped velocities and walls of the Gym pendulum and mountain car.
- Even taken at face value, the plots show nothing that the paper's own nonlinear probe does not already show: the probe recovers the true state well for most models (Table 2), and every reading of the flow plots is checked against physics that is already known. The paper gives no case where H-Lens reveals something about a world model that was not known in advance.
- The second claimed contribution, detecting rollout drift without ground truth, is not demonstrated. The score would correlate with prediction error even if it carried no information, and no alternative signal is compared.
- The phase-error score (Eq. 14) adds up a non-negative mismatch at every step, so it can only increase as the rollout gets longer. The world model's prediction error also tends to grow with rollout length. A positive correlation across rollout steps (Table 3, Fig. 5) is therefore expected from any quantity that increases over time, including the step number itself.
- Where the two diverge, the score fails. For LeWM on LunarLander the prediction error falls while the score keeps rising, giving a correlation of −0.90 (Fig. 5h).
- The score is never compared with step number, ensemble disagreement, or Dreamer's built-in gap between its predicted and observation-based states. The paper does not test whether it picks out which individual rollouts will fail, and it reports one training seed per model (Tables 4–6, 9).
- The paper is inconsistent about what was trained and evaluated. The training objective in the main text differs from the one in the appendix. The rollout length is given as at most 10 steps, 5 steps and 30 steps in different places (Secs. 3.1, 4.2, Fig. 5). A reader cannot tell exactly which model produced the reported results.
Questions for authors
- (Weakness 2: rollout-error score) Does the score carry any information beyond how long the rollout has run? Fix the rollout step and compare rollouts: does a higher per-step (not summed) phase error pick out the rollouts with larger prediction error? Does it do better than simple alternatives such as the step number itself, disagreement within an ensemble of world models, or Dreamer's built-in gap between its predicted and observation-based states? If it ranks failing rollouts well at a fixed step, my concern would largely go away. If it does no better than the step number, the second contribution does not hold.
- (Weakness 1: recovery of learned dynamics) On held-out data, how well do the recovered variables match the true angle or position and velocity after the best-fitting rescaling, for both normal and altered physics and across several seeds? Can the altered-physics claims be shown with a measure that does not depend on coordinate scale? One example is the oscillation period implied by the recovered flow, which should be shorter for the shorter pendulum. A close match and the expected change in period would support the first claim. Without them, the plots do not show that the learned dynamics were recovered.
- (Weakness 1: what the method adds) The nonlinear probe already recovers the true state well for most models (Table 2). Is there a case where H-Lens reveals something about a world model's learned dynamics that the probe misses and the known physics would not have predicted, such as a model whose dynamics are wrong in a way the flow plot exposes? Every interpretation in the paper is checked against physics that is already known. Without such a case, it is unclear that the method would help with its stated aim of understanding opaque world models.
2 comments
Thank you for the response. It reports no new results, and my main concerns remain.
Recovered dynamics (Weakness 1). The response agrees that the coordinates differ in basis between models. The paper's evidence for the altered physics compares velocity ranges across separately trained networks, which depends on that basis. No basis-independent measure is reported.
Drift score (Weakness 2). A running sum of non-negative terms grows with rollout length regardless of prediction error. The response does not compare the score with the step number or other simple signals.
We thank the reviewer for the thoughtful comments. W2, Q1. We agree with the reviewer. In fact, we explicitly acknowledge that latent coordinates differ in basis. These changes do not alter the underlying dynamics. A pendulum does not physically swing faster simply because its coordinates are represented in a different basis. Eq. 14 measures cumulative phase-space inconsistency. Local vector-field error and accumulated rollout error are different quantities.
W1. Q2. Great question. H-Lens does not aim to recover exact physical coordinates. It aims to reveal meaningful dynamical structure. Appendix A.6 provides reference phase-spaces, while Appendix A.1 (updated) gives formal guarantees with regards to recovered dynamics.
W1. Q3. Probing requires ground-truth labels. It measures physical information in the latent representation, not the learned vector field. We use known systems for validation and unknown dynamics remain an exciting area for future work.
We thank the reviewer again for all the valuable feedback!