acceptodds
Under review as a conference paper at ICLR 2027

Conservation Buys Stability and Factoring Buys Counterfactuals in Physical World Models

Abstract

A learned simulator can reproduce its training conditions accurately yet fail in two distinct ways once those conditions change. Over long rollouts, small errors accumulate until the trajectory drifts away from physically plausible behavior; under an intervention on a physical parameter, the model may continue to follow the law seen during training rather than the intervened one. We show that these two failures require different structural remedies. Evolving a learned energy with a symplectic integrator preserves the geometry of the conservative dynamics and keeps rollouts bounded and physically meaningful for up to the training horizon, while equal-capacity predictors, an energy-regularized predictor, and a tuned neural ODE diverge. By contrast, encoding the physical coupling through an explicit linear factorization enables the model to follow a never-seen sign of that coupling, whereas an unrestricted parameterization remains locked to the training law. Crucially, the two mechanisms are separable: removing the structure responsible for long-horizon stability leaves counterfactual transfer intact, while removing the factorized coupling destroys counterfactual transfer without eliminating stability. This double dissociation, established with matched controls that remove or replace one structural component at a time, persists beyond the headline three-body system and remains visible when the physical state must be inferred from pixels rather than provided directly. The result is a concrete design principle for physical world models: long-horizon stability and changed-law generalization arise from distinct structural commitments, and each can be imposed deliberately without requiring the other.

Primary area:
foundation or frontier models, including LLMs
Keywords:
World Model, Long-term Stability, Counterfactual Robustness, Physics-law aware

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ya5fOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies two issues when learning physical world models: drift over long-horizon prediction, and zero-shot generalization when a key physical parameter is changed at inference. The authors argue that these issues are addressed by distinct structural properties: evolving a learned energy with a symplectic integrator for the former, and encoding the physical coupling through an explicit linear factorization for the latter. The paper also shows that the two mechanisms are separable (a “double dissociation”), so each can be imposed independently. Experiments are mainly conducted on softened three-body gravity with oracle state, trained on attractive gravity and evaluated on 2000-step rollouts and on the unseen repulsive regime, with extensions to other systems (MuJoCo double pendulum, contact, many-body, Coulomb, dissipative dynamics) and to synthetic pixel observations.

Critical strengths
  • The paper addresses two key limitations of learned world models, long-horizon drift and generalization to changed physical laws, and asks a clear question: which structural commitment fixes which failure?
  • The core experiments are systematically controlled. Controls remove or replace one structural ingredient at a time (unstructured, energy-penalized and neural-ODE baselines; unfactored variants) across several formalisms. They are complemented by a capacity sweep, a set-transformer, domain randomization and a “shortcut” experiment.
  • Reporting is transparent: learned-energy conservation is distinguished from physical-energy drift, non-finite rollouts are counted explicitly, and limitations are discussed.
Critical weaknesses
  • Gap between motivation and setting. The introduction motivates the work with the physical failures of video generators, but the study covers low-dimensional systems with known Hamiltonian structure, mostly with oracle or render-anchored state. The claim of a “design principle for physical world models” goes beyond what this setting supports.

  • Limited theoretical contribution. The two propositions of Section 3 are not new results.

    • Proposition 1 restates classical backward error analysis for symplectic integrators (Hairer et al., 2006). It also concerns the modified learned Hamiltonian, not the physical energy whose drift is measured.
    • Proposition 2 holds by construction: a potential linear in G yields a force odd in G (one-line proof). It guarantees the sign of the force, not its correctness.
    • Factoring G linearly amounts to supplying the exact functional form of the intervened parameter, so the “counterfactual generalization” reduces to extrapolating along a hard-coded dependence, a form of conditioning the paper itself describes as standard (Sec. 2). The long-horizon stability of Hamiltonian and symplectic models is likewise well established.
    • Consequently, the double dissociation is largely expected by construction. The sign inversion of Proposition 2 follows from the linearity in G alone, whatever the integrator, so removing symplectic integration cannot remove it. Conversely, Proposition 1 holds for any learned Hamiltonian, however G enters it, so removing the factoring cannot remove stability, and symplectic integration alone says nothing about the behavior at G < 0. The experiments thus confirm what the parameterizations imply rather than revealing an unexpected separation.
  • Narrow counterfactual setting. The only intervention is the sign flip of a single coupling that enters linearly. Parameters entering nonlinearly (masses, softening, force-law exponent), magnitude extrapolation, and cases where it is unknown how a parameter enters the dynamics, which is the realistic situation for world models, are not addressed.

  • Limitations in experiments.

    • Experiments mostly confirm the construction. The core evidence (Tables 1–3) comes from oracle-state softened three-body gravity, where regime match = 1 under the sign flip is guaranteed by Proposition 2. Only the trajectory error is informative.
    • “Stable” means finite rather than accurate. The symplectic model’s true-energy drift reaches 7.1 at 2000 steps (710%, Table 1), contradicting the abstract’s claim of “physically meaningful” rollouts, while the energy-penalized model and the neural ODE are far more accurate at short horizons.
    • Unfair or missing baselines.
      • Domain randomization (Fig. 4) is tested only on an unstructured black box. The key control, a non-factored symplectic model trained on both signs, is missing; it would likely be both stable and invert, showing that factoring is unnecessary when data cover the regime.
      • Domain-adaptation and context-adaptation approaches for dynamics, which the paper cites, are not compared: e.g., LEADS (Yin et al., 2021), CoDA (Kirchmeyer et al., 2022), or meta-learned dynamics (Wang et al., 2022). Few-shot adaptation to the new regime with a handful of trajectories is the natural data-driven alternative to hard-coding the functional form.
      • Multi-step rollout training, a standard remedy for drift, is also missing as a stability baseline, although the authors themselves report that a short-rollout loss halves the drift.
    • Breadth over depth. The extensions (MuJoCo, contact, many-body, Coulomb, thermal bath) are reported in one or two sentences each, with details deferred to the appendix and no advanced baselines. More generally, Section 4 is very dense, with many results reduced to a single number and most details deferred to a long appendix, which makes the paper hard to assess. The pixel settings use synthetic renders of the same N-body system (Gaussian or disk splats at body positions), not natural video. Inversion is recovered only when the latent is anchored to physical coordinates (renderer-grounded position, imposed canonical kinetic form, image centroid, or a tracker over a fixed detector), while the fully unsupervised branch fails (App. C). The claim that the dissociation “remains visible when the physical state must be inferred from pixels” therefore overstates what is shown.
    • Expected boundaries. Section 4.4 largely restates the assumptions (a conservative model cannot dissipate; a misspecified coupling fails), although the silent failure under coupling-power misspecification is a useful warning.
    • Inconsistent numbers. The drift of the symplectic model is 7.1 in Table 1 but 3.9 in Table 3, and the factored inversion error varies across Tables 2–4 without explanation.
Questions for authors

In the rebuttal, I would mainly expect answers to the following:

  • Baselines: results for a non-factored symplectic model trained on both signs of the coupling, for domain/context-adaptation methods (e.g., LEADS, CoDA) given a few trajectories of the new regime, and for multi-step rollout training as a stability baseline.
  • Beyond the construction: what does the double dissociation show beyond what Propositions 1 and 2 imply? How does the approach behave when the functional form of the intervened parameter is unknown or nonlinear? Can the factorization be learned rather than supplied?
  • Accuracy and consistency: how is “physically meaningful” justified given a 710% true-energy drift at 2000 steps? Why does the symplectic model’s drift differ between Table 1 (7.1) and Table 3 (3.9), and why does the factored inversion error vary across Tables 2–4?
Reviewer_uTQ1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether long-term stability and generalization to changed physical laws come from the same model structure. Its experiments suggest that Hamiltonian structure with symplectic integration helps keep rollouts stable, while explicitly factoring the coupling parameter enables transfer to unseen signs. The authors test this distinction across several simulated systems.

Critical strengths
  1. The problem is clear and ablations help separate stability from generalization to changed laws and offer useful guidance for model design.
  2. The experiments cover several systems and include failure cases. These make the scope of the findings easier to understand.
Critical weaknesses
  1. I am not fully convinced by the novelty. The benefits of Hamiltonian models and symplectic integration are well established. And if the potential is written as V(q,G)=GUθ(q)V(q,G)=G U_\theta(q), reversing the force when GG changes sign follows directly from the model design. The experiments are useful, but it is less clear what we learn beyond these expected effects. I see the separation of the two effects as the paper’s main contribution, rather than either structural component itself. My main concern is how much new insight the separation provides. Since the two components target different aspects of the dynamics, some separation seems expected. The crossed ablations offer useful evidence, but I am not convinced that this finding alone is sufficient to acceptance.
  2. The meaning of “stability” needs more discussion. In Table 1, the symplectic model has a true-energy drift of 7.1 at 2,000 steps, and Appendix C.1 reports large trajectory errors. This is much better than the diverging baselines, but it still seems far from an accurate physical simulation.
Questions for authors
  1. How useful is the approach when the dependence on the intervened parameter is not known in advance? The misspecified-coupling experiment raises this concern, but it is unclear how a practitioner should choose the right structure.
  2. Appendix C.1 selects learning rates based on one-step error, while the main claim concerns long-term stability. Would the observed separation still hold if models were selected based on long-horizon?
Reviewer_C1ECOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper argues for a double dissociation in learned physical simulators. A learned Hamiltonian with a symplectic update improves long-horizon boundedness but does not transfer to an unseen sign change in the interaction law. Explicitly factoring the physical coupling enables this counterfactual transfer but does not prevent divergence. Combining both structures produces a model that does both. The claim is tested primarily on three-body gravity and extended to other mechanical systems, pixel-derived states, dissipation, and driven dynamics.

Critical strengths
  1. The controlled double dissociation is a useful design result: conservation and coupling factorization address different failures.
  2. The paper tests meaningful boundaries, including dissipation, misspecified couplings, and insufficient perceptual grounding.
Critical weaknesses
  1. "Stability" mainly means avoiding divergence. At 2,000 steps, the symplectic model still has relative energy drift and normalized rollout error around 10–14, so bounded trajectories are not necessarily accurate?
  2. Table 3 uses a factored graph network as the “inverts but is unstable” cell, but Appendix C.2 says its inversion label is descriptive because short-horizon fidelity and finiteness were not verified.
Questions for authors
  1. Can the same learned Hamiltonian be compared under leapfrog, RK4, and implicit midpoint on the main three-body benchmark?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.