acceptodds
Under review as a conference paper at ICLR 2027

Noether World Models: An Exactly Conserved Family of Latent Invariants

Abstract

World models predict by rolling a learned latent transition forward over many steps. Current methods pursue conservation through a soft penalty, a symplectic integrator or an implicit projection, so each exact guarantee covers a small set of scalars fixed before training and paid for with an implicit update. We propose the Noether World Model (NWM), which assembles the generator as a graph Laplacian over object subspaces, , and advances the latent with a Cayley transform. The resulting unitary operator conserves a -dimensional family of quadratic invariants exactly at every value of its weights, and that family is exactly the set of quadratic forms with that property. Experiments on three physics datasets and three planning environments show that NWM holds the entire family at double precision whereas no baseline holds more than the norm, with fewer parameters than the strongest physics-structured baseline. Decoded quality in distribution separates the nine designs by only 1.3 dB, whereas under a zero-shot change of force law NWM leads the unconstrained transitions by 1.9 dB, so the family pays where the data leaves the training distribution. Exactness has a price, since the operator fixes every member of the family and most goals in a latent it did not shape lie off the reachable set. The same family can nonetheless enter the training objective of a world model whose predictor we do not replace, and it improves planning only where that predictor already drifts, a condition measurable before retraining.

Primary area:
learning on time series and dynamical systems
Keywords:
world models, latent dynamics, conservation laws, unitary operators, Cayley transform, graph Laplacian, object-centric representations, model-based planning, JEPA

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SE5xOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Noether World Models, a latent dynamics architecture designed to preserve a family of quadratic invariants. The method combines graph-Laplacian generators with Cayley-transform updates to enforce conservation by construction. The paper also investigates the limitations of imposing hard conservation constraints on existing latent representations and explores relaxed alternatives for prediction and planning.

Critical strengths
  1. A principled structural contribution. The conservation properties follow from the architecture rather than relying solely on a training penalty. The theoretical analysis explains which quadratic quantities are preserved, and the controlled comparisons help distinguish the contribution of the proposed structure from that of the Cayley transform alone.
  2. An informative investigation of practical limitations. The paper reports cases where hard conservation constraints perform poorly in planning, rather than presenting conservation as universally beneficial. The capacity controls and relaxation experiments provide useful evidence about the trade-off between structural constraints and predictive flexibility.
Critical weaknesses
  1. The numerical evidence for conservation precision requires clarification. The supplementary results report substantially different relative drift magnitudes for selected invariants and randomly sampled members of the invariant family. These differences may reflect normalization, small initial quadratic values, or numerical implementation details rather than a failure of the theory. Nevertheless, the claim of double-precision conservation would be more convincing with a consistent account of numerical precision and normalization, accompanied by absolute or appropriately scaled residuals.
  2. The practical contribution of preserving the invariant family is not sufficiently isolated. Exact conservation establishes a structural property, but does not by itself ensure accurate dynamics or effective planning. A model can preserve quadratic quantities while predicting incorrect transition directions or action responses. The paper includes useful controls and acknowledges failures of hard constraints; however, it remains unclear which downstream improvements can be specifically attributed to invariant preservation. The discussion would benefit from a clearer connection between the preserved quantities and measurable gains in prediction or planning, including the conditions under which conservation helps or hurts.
Questions for authors
  1. Can the authors reconcile the conservation precision reported for selected invariants with that reported for randomly sampled quadratic forms, including the effects of normalization and numerical precision?
  2. What concrete benefits does preserving the proposed invariant family provide beyond reducing invariant drift? Could the authors clarify whether, and under what conditions, it improves prediction accuracy, long-horizon rollout quality, or planning performance, and which controlled comparisons support attributing these improvements to invariant preservation?
  3. Could the authors clarify the guarantees associated with the relaxed variants? In particular, a finite conservation penalty generally encourages approximate conservation but does not guarantee exact conservation at the optimum of the combined objective.
4 comments
AuthorsOpenReview ↗

Response to Reviewer SE5x (1/3): Overview, and W1/Q1 on conservation precision and scaled residuals

We thank the reviewer for recognising that the conservation properties follow from the architecture and that the controls isolate the Laplacian structure. Part 1 shows that the gap between the named invariants and the sampled forms comes from numerical precision and normalisation, since in double precision every member of the family stays at round-off, and reports scaled residuals (W1, Q1). Part 2 shows with matched controls and new ablations that preserving the family makes prediction robust to a change of force law, and states when it helps and when it does not (W2, Q2). Part 3 states the guarantee of each relaxed variant (Q3).

W1 and Q1. Precision of the named invariants and of the sampled forms. The discrepancy stems from numerical precision and normalisation, not from the theory. Table 10 was computed with the whole rollout in double precision. Table 11, the ratios of Tables 1 and 13, and Table 14 were computed from the single-precision rollout of the trained model. Recomputed in double precision, every member of the family, named or sampled, drifts at the level of double-precision round-off. Line 338 and Appendices D.1, D.2 and D.11 state that all of these tables were computed in double precision from the same rollouts, which holds for Table 10 only, and we have corrected these four statements.

(i) Where precision enters. We recomputed all three datasets and seeds in three settings: (A) the published single-precision rollout, where HH is assembled and the state is rounded in single precision at every step, (B) a double-precision rollout with HH still assembled in single precision, and (C) a fully double-precision rollout, which is the setting of Table 10. The Cayley solve is in double precision in all three, and on the trained checkpoints ∥U†U−I∥F≤2.6×10−14\lVert U^\dagger U - I \rVert_F \le 2.6 \times 10^{-14}. Single-precision assembly keeps HH exactly Hermitian and block-scalar but leaves its row sums nonzero at about 4×10−84 \times 10^{-8} relative, which affects only the block sum and the members with a 11⊤\mathbf{1}\mathbf{1}^\top component. Setting B isolates this term and setting C removes it.

(ii) Scaled residual. The relative drift divides by ∣Q(z0)∣\lvert Q(z_0) \rvert, which for a sampled member can be close to zero: over the in-family forms, ∣Q(z0)∣/(∥B∥2∥z0∥2)\lvert Q(z_0) \rvert / (\lVert B \rVert_2 \lVert z_0 \rVert^2) falls to 8×10−78 \times 10^{-7}, so a mean of relative drifts is set by a few small denominators. We therefore report rB=∣QB(zh)−QB(z0)∣/(∥B∥2∥z0∥2)r_B = \lvert Q_B(z_h) - Q_B(z_0) \rvert / (\lVert B \rVert_2 \lVert z_0 \rVert^2), which does not depend on the scale of the form or on QB(z0)Q_B(z_0). Median at h=29h = 29 over trajectories and forms, range over the three datasets and three seeds:

setting∥z∥2\lVert z \rVert^2∑izi\sum_i z_iin-family formsout-of-family forms
A2.0e-8 to 2.7e-87.5e-8 to 1.4e-73.0e-9 to 6.1e-94.4e-3 to 2.4e-2
B3.9e-16 to 1.0e-157.4e-16 to 1.2e-74.8e-17 to 2.5e-94.4e-3 to 2.4e-2
C3.4e-16 to 1.0e-154.6e-16 to 6.3e-152.0e-17 to 2.3e-164.4e-3 to 2.4e-2

In setting C the largest in-family residual over all trajectories and forms is 1.6×10−151.6 \times 10^{-15}. In the same measure the in-family residuals of the eight alternative transitions lie between 3.7×10−33.7 \times 10^{-3} and 7.5.

(iii) Growth with the horizon. In setting C the median residual of the norm, the block sum and the in-family forms grows from h=1h = 1 to h=100h = 100 with a log-log slope between 0.39 and 0.62, close to the square-root growth of round-off in exactly conservative schemes (Hairer et al., BIT Numerical Mathematics 2008), and stays below 6.6×10−156.6 \times 10^{-15} at h=100h = 100. Extending the latent rollout to h=104h = 10^4 on all three datasets, the median in-family residual remains below 3.1×10−143.1 \times 10^{-14} and its maximum over trajectories below 1.4×10−131.4 \times 10^{-13} (at most 1.2×10−71.2 \times 10^{-7} in the published single-precision setting). For the other transitions it is at least 1.2×10−21.2 \times 10^{-2} and reaches 3.6×1083.6 \times 10^{8}, and GNN diverges before h=103h = 10^3.

(iv) Family ratio. The ratios of Tables 1 and 13 in the submission (4.3e6, 3.6e6, 6.0e6) are single-precision values. Our rerun in setting A gives 5.4e6, 3.1e6 and 9.7e6, with the difference coming from single-precision round-off and one Balls trajectory. In setting C they become 6.3e14, 5.1e14 and 6.3e14, while the largest ratio among the other eight transitions, evaluated in double precision as well, is 5.0. The separation between the family and the other forms is therefore eight orders of magnitude wider in double precision, and the conclusions of §4.1 hold with a wider margin.

The revised PDF reports the family ratios of Table 1, Tables 11 and 15 and the corresponding panels of Figures 3 and 5 in double precision, marks Tables 13 and 14 as single precision, adds the scaled residual (Table 23), and states the precision of every table in Appendix D.11.

AuthorsOpenReview ↗

Response to Reviewer SE5x (2/3): W2/Q2 on what the family buys, and under which conditions

W2 and Q2. Benefits of preserving the family, and the conditions under which they arise. Beyond lower drift, preserving the family makes prediction robust to a change of force law and slows degradation over long horizons. It leaves in-distribution accuracy unchanged, and it is detrimental where a fixed encoder changes the norm of its latent. Matched controls attribute the gain to the family.

(i) Controlled comparison. Ctrl-cayley shares the coupling network and the Cayley step with NWM and differs only in placing the couplings on a random Hermitian basis instead of a Laplacian, so it holds the norm but not the family (Appendix C.4.2). Ctrl-free is an unconstrained transition with the same parameter count to within 1%. Each sector variant breaks one part of the family, as verified on the trained weights. NWM, its variants and the two controls share the encoder, decoder and training recipe. PSNR in dB at h=29h = 29, mean over three seeds (Tables 12 and 17, Appendix D.13):

ModelConservedIn distributionCoulomb 1/r21/r^2
NWMfull family18.8620.41
NWM without zero row sumsS(z)S(z)18.8720.37
NWM without block-scalar HH∥z∥2\lVert z \rVert^2 and PP18.7220.33
HNN (128K parameters)learned energy18.8520.41
Ctrl-free (same size)none18.6718.58
Ctrl-cayley (same size)norm only17.6517.50

Ctrl-cayley is precisely the case the reviewer describes: it preserves a quadratic quantity yet exchanges two objects (Figure 4). NWM matches HNN without assuming Hamiltonian dynamics, with 32 times fewer parameters. Preserving either sector retains the gain, while preserving the norm alone does not.

(ii) Breaking the family by a controlled amount. NWM interpolated with the MLP transition through Eq. 9, three seeds each, one shared recipe, so that α=0\alpha = 0 retrains NWM itself (PSNR in dB):

α\alpha00.010.030.10.250.51
In distribution18.8618.9518.8818.1017.8618.9618.96
Coulomb 1/r21/r^220.3917.6217.5717.6517.9718.3418.44
Energy rescaling κ=0.5\kappa = 0.521.3021.9121.9920.2120.3621.2621.24

Every α>0\alpha > 0 breaks the family and costs 1.95 to 2.82 dB under the Coulomb law, while α=0.01\alpha = 0.01 and α=0.5\alpha = 0.5 match NWM in distribution, so the loss is not due to a weaker fit. Under κ=0.5\kappa = 0.5 breaking the family has no consistent effect. The gain is therefore attributable to the family and arises under the change of force law.

(iii) Conditions.

ConditionEffect of preserving the familyEvidence
In distribution, h=29h = 29neutral: the nine designs lie within 1.31 dB, NWM 0.10 dB behind the bestTable 12
In distribution, h=10h = 10 to 100slower degradation: 0.68 dB, against 1.57 and 1.53 dB for Ctrl-free and Ctrl-cayleyTable 16
Coulomb law, exponents 1 to 31.8 to 3.0 dB over MLP, GRU, Ctrl-free, Ctrl-cayley and UWM, 25 paired tests, each p<0.01p < 0.01Appendix D.13
Coulomb law, 10% of the data20.20 dB, above every unconstrained or norm-conserving transition trained on all dataAppendix D.13
Fixed encoder whose norm variesexact operator detrimental (planning at 3.17%), as bounded by Proposition 3§4.4
Same, members imposed as a penaltyplanning +8.50, +6.50, +7.00 points on Tworoom, where the host drifts most (p=0.007p = 0.007)Table 2

The neutral in-distribution result agrees with the finding that physics priors on pixels do not significantly improve standard models (Botev et al., NeurIPS Datasets and Benchmarks 2021), and the gain under the change of force law agrees with the finding that symmetry built into a dynamics model improves generalization under distributional shift (Wang et al., ICLR 2021). The energy decoded from the NWM latent by a fixed probe also remains the most accurate of the nine models at every horizon up to 100 steps (see our response to Reviewer 7e1N, Q2).

We have added a paragraph to §4.3 that states these conditions and the controls that support each, with the new results in Appendix D.13. All changes in the revised PDF are marked in blue, and line numbers refer to the submitted version.

AuthorsOpenReview ↗

Response to Reviewer SE5x (3/3): Q3 on the guarantees of the relaxed variants

Q3. Guarantees of the relaxed variants. We agree that a finite penalty encourages approximate conservation and does not guarantee exact conservation at the optimum of the combined objective. The two relaxations carry different guarantees, and the revision states each precisely. With Δ=Fψ(z)−Uz\Delta = F_\psi(z) - Uz, a family member Q(w)=w†BwQ(w) = w^\dagger B w and a linear member ℓ(w)=⟨v,w⟩\ell(w) = \langle v, w \rangle such as the block sum:

VariantUsed whereGuarantee at every parameter valueViolation after one step
NWM (Theorem 1, Corollary 2)encoder trained with the operatorexact, for the whole familyzero up to round-off
Interpolated operator, Eq. 9 (Proposition 5)NWM mixed with FψF_\psi by weight α\alphaexplicit bound, controlled by α\alpha∣Q(z′)−Q(z)∣≤α∥B∥∥Δ∥(2∥z∥+α∥Δ∥)\lvert Q(z') - Q(z) \rvert \le \alpha \lVert B \rVert \lVert \Delta \rVert (2 \lVert z \rVert + \alpha \lVert \Delta \rVert) and ∣ℓ(z′)−ℓ(z)∣≤α∥v∥∥Δ∥\lvert \ell(z') - \ell(z) \rvert \le \alpha \lVert v \rVert \lVert \Delta \rVert
Penalty, Eq. 10fixed host whose predictor is not replacednone for a finite λH\lambda_Happroximate at a minimiser, exact only as λH→∞\lambda_H \to \infty

(i) For the interpolated operator the violation is therefore controlled by α\alpha, and Theorem 1 is recovered at α=0\alpha = 0.

(ii) For the penalty, as for any quadratic penalty, the constraint is met exactly only in the limit of a large weight (Nocedal and Wright, Numerical Optimization, 2006, Chapter 17). The penalty serves a different purpose, namely to impose the invariants on a host whose predictor must remain unchanged (§3.5).

We have replaced "holds at a minimiser of L" in §3.5 by "holds approximately at a minimiser of L, with no exact guarantee for any finite λH\lambda_H".

We hope these responses resolve the reviewer's concerns, and we would be grateful if the reviewer reconsidered the score.

AuthorsOpenReview ↗

Response to Reviewer SE5x: each point of the review and its revision

As the discussion closes soon, we thank you for your positive assessment. We have answered each of your points, and each is listed below with its revision in the PDF, marked in blue.

W1 and Q1. Table 1, Tables 11 and 15 and the corresponding panels of Figures 3 and 5 now report the family ratios in double precision, and Appendix D.11 states the precision of every table. Table 23 adds the scaled residual. In double precision every member of the family, named or sampled, stays at round-off, and the family ratio exceeds 5×10145 \times 10^{14}, against at most 5.0 for every other transition. The statements on precision in §4.1 and Appendices D.1 and D.2 are corrected accordingly.

W2 and Q2. §4.3 now states what the family contributes and under which conditions, with the new results in Appendix D.13. Matched controls, seven interpolation weights and two sector variants attribute the gain to the family (Tables 19 and 20). Under a change of force law NWM outperforms the matched baselines in all 25 paired comparisons on a newly generated test set (p<0.01p < 0.01), also when trained on a tenth of the data (Table 21).

Q3. §3.5 now states that the penalty holds only approximately at a minimiser, with no exact guarantee for a finite λH\lambda_H, and Proposition 5 gives the explicit bound of the interpolated operator for quadratic and linear members of the family.

We would be grateful to hear whether these revisions resolve your concerns. If they do, we would sincerely appreciate your consideration of a higher score. Thank you for your time and your support of the paper.

Reviewer_61NYOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The authors introduce a method to construct a non-linear latent predictor that conserves a set of pre-specified latent quantities. They introduce exact and relaxed versions. They evaluate how well this construction conserves certain quantities compared with a suite of baselines (which were not built to conserve such quantities), and show that it indeed outperforms them. They test of visual dynamics forecasting (where their model performs on par or worse than baselines) and simple planning tasks (where their model performs worse on 2 of 3 tasks).

Critical strengths

The authors develop clever mathematical machinery for inducing a broad family of conserved quantities in the update of a latent world model.

Critical weaknesses

My major concern with the paper is that the results are not sufficiently convincing that the method (introducing conserved latent quantities) is sufficiently beneficial for improving true downstream or predictive performance of world models. For example, the authors repeatedly demonstrate that their designed conserved quantities are conserved, but the connection of these quantities to downstream performance is missing (in-fact the authors appear to somewhat hide that fact by trying to flash the bold conserved quantity numbers without highlighting that these are just latent conserved quantities). They only show improved planning performance on one of thee environments compared to the baselines. On the visual prediction task, the authors leave the reconstruction accuracy to the appendix (Table 12) where it is seen that their model performs generally on-par or worse than the baselines. The general idea of this approach has been investigated many times in the literature (as correctly identified by the authors references such as Hamiltonian Generative Networks), and the results to date lack a significant performance improvement over unconstrained baselines. This historical precedent for the general idea, combined with the virtual lack of positive results from the present paper make the contributions of the work unconvincing.

The second major concern with this paper is that the readability is very poor. Some examples:

  • (lines 173-177): "Two properties separate Equation 1 from existing designs, in that the identity holds at every parameter value, whereas a penalty reaches it only at a minimiser and an integrator only as the step size vanishes, and in that U reads both zt and at, whereas a fixed unitary matrix reads neither and an action-conditioned one reads only the action."
  • Many sentences are extended excessively, and frequently have in-line comparisons which make the main points difficult to follow (the phrase 'whereas' is used 23 times in the paper).
Questions for authors

Can the authors demonstrate tasks or datasets where these conserved quantities actually encode something meaningful about the task or the system being modeled. Similarly, can these quantities be shown to improve performance in a more robust and meaningful way? Answering these questions in the affirmative would be necessary to change my rating.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 61NY (1/3): Overview, and what NWM and the baselines conserve

We thank the reviewer for the detailed review and the concrete readability example. Part 1 clarifies that NWM conserves a 3,200-dimensional family fixed by its structure rather than a set of quantities chosen in advance, and that several baselines conserve the norm or a learned energy. Part 2 answers Q1 and Q2 affirmatively with new experiments: members of the family predict the energy and the angular momentum of the simulated systems and stay fixed along the rollout, and preserving the family improves prediction under a change of force law and with less data (W1, W2, W4). Part 3 completes Q2 with further test sets, data efficiency and planning, and answers W5 and W6, including a rewrite of lines 173 to 177. All changes in the revised PDF are marked in blue, and line numbers refer to the submitted version.

Which quantities NWM conserves. The exact operator of NWM does not conserve a set of quantities chosen in advance, and only the penalty of Eq. 10 imposes chosen members on a fixed host (§3.5). The set follows from the structure of the transition: "The set Q is determined by the operator rather than fixed in advance" (lines 176 to 177). The latent z=(z1,…,zN)z = (z_1, \dots, z_N) holds NN objects of width mm. Corollary 2 shows that one step of NWM conserves every quadratic form z†(A⊗C)zz^\dagger (A \otimes C) z with A∈span⁡{IN,11⊤}A \in \operatorname{span}\lbrace I_N, \mathbf{1}\mathbf{1}^\top \rbrace and any m×mm \times m matrix CC, Hermitian for the real-valued members. This family is a real vector space of dimension 2m22m^2, which is 3,200 for m=40m = 40, and Corollary 2 also shows that no other quadratic form is conserved at every value of the weights. The three quantities of Eq. 2 serve as examples for reporting, not as the design target.

What the baselines conserve. Several baselines were designed to conserve one quantity, as stated after Eq. 2 (line 182). UWM and Ctrl-cayley are unitary and conserve the norm ∥z∥2\lVert z \rVert^2 to the same precision as NWM (Table 10). HNN and SymODEN conserve a learned energy. The comparison therefore addresses a sharper question than whether a constrained model outperforms unconstrained ones, namely whether conserving the norm suffices or whether the family is required. Part 2 shows that the family is required.

AuthorsOpenReview ↗

Response to Reviewer 61NY (2/3): Q1, and Q2 on prediction (W1, W2, W4)

Q1 and W2. Do the conserved quantities encode something meaningful? Yes. Members of the conserved family are predictive of the energy and angular momentum of the simulated systems, and NWM keeps these members fixed along the entire rollout.

(i) The family admits a simple characterisation. Every member is a linear function of the matrix S(z)=∑izizi†S(z) = \sum_i z_i z_i^\dagger and of PP†P P^\dagger with P=∑iziP = \sum_i z_i, and NWM conserves S(z)S(z) and the vector PP itself (Corollary 2, Theorem 1). If each object is written as a position and a momentum, the angular momentum is one member of the family and the total momentum is a block of PP. The family therefore contains the conserved quantities of rotation and translation.

(ii) Which physical quantities a learned encoder actually encodes is an empirical question, since without supervision a latent need not align with a named physical factor (Locatello et al., ICML 2019). We fitted a kernel probe on the family at t=0t = 0 (test R2R^2, three seeds). A frame shows positions but not velocities, so we compare with the best a single frame allows: the same probe on the true positions, or on visual N-body, where the true state is not stored, a supervised network on the frame. The last column is the median probe error against the ground truth at step 100, in units of the test standard deviation.

QuantityNWM familySingle-frame limitRecoveredError at h=100h = 100, NWM vs. best other
Energy, visual N-body0.540.66 (supervised)82%0.35 vs. 0.43
Energy, HNN 3-body0.370.40 (true positions)93%0.56 vs. 0.75
Angular momentum, HNN 3-body0.390.42 (true positions)94%0.57 vs. 0.76

Adding the true velocities raises the limit to 1.00, so the family, learned without physical supervision, recovers most of what a frame contains. The other eight latents give energy R2R^2 between 0.08 and 0.55 on visual N-body and between 0.33 and 0.39 on HNN 3-body, so what separates NWM is that it preserves what is encoded. Under NWM these probes stay fixed up to round-off over 10410^4 latent steps, under every other transition they drift, and the NWM probe has the lowest error of the nine models at every horizon from 1 to 100 steps. We have added this analysis to Appendix C.5.

(iii) We agree that the conserved quantities are latent and that this should be stated where the numbers appear. The captions of Table 1 and Figure 3 now state that these quantities are latent, and the decoded PSNR of Table 12 now appears in Table 1 next to them.

Q2 and W1, W3, W4. Does the family improve performance in a robust way? Yes, under the zero-shot change of dynamics of Table 17, where the gain is reproduced across test sets, force exponents and data sizes, and in planning where the host drifts most. In distribution NWM performs on par with the baselines, as stated in the Abstract, §1 and §4.1 (lines 21 to 22, 89 to 91 and 368 to 370). Symmetry built into a dynamics model is known to pay off under distributional shift (Wang et al., ICLR 2021).

(i) The same model with and without the family. Ctrl-cayley is NWM with the Laplacian replaced by a random basis, and each sector variant breaks one part of the family (Appendix D.13). PSNR in dB at h=29h = 29, mean over three seeds:

ModelConservedIn distributionCoulomb 1/r21/r^2
NWMfull family18.8620.41
NWM without zero row sumsS(z)S(z)18.8720.37
NWM without block-scalar HH∥z∥2\lVert z \rVert^2 and PP18.7220.33
HNN (128K parameters)learned energy18.8520.41
Ctrl-free (same size)none18.6718.58
Ctrl-cayley (same size)norm only17.6517.50
MLPnone18.9618.45

NWM matches HNN without assuming Hamiltonian dynamics and with 32 times fewer parameters (3,963 against 128K). Preserving either sector retains the gain, while preserving the norm alone does not.

(ii) Breaking the family by a controlled amount. NWM interpolated with an MLP of weight α\alpha (Eq. 9), three seeds each:

α\alpha00.010.030.10.250.51
In distribution18.8618.9518.8818.1017.8618.9618.96
Coulomb 1/r21/r^220.3917.6217.5717.6517.9718.3418.44

Every α>0\alpha > 0 breaks the family and costs 1.95 to 2.82 dB under the Coulomb law, while α=0.01\alpha = 0.01 and α=0.5\alpha = 0.5 match NWM in distribution, so the loss comes from breaking the family, not from a weaker fit.

AuthorsOpenReview ↗

Response to Reviewer 61NY (3/3): Q2 on further test sets, data and planning (W3), W5 on prior work, and W6 on readability

Q2 (continued). Further test sets and less data. On a newly generated Coulomb test set, with the force exponent varied from 1 to 3, NWM outperforms the MLP, the GRU, Ctrl-free, Ctrl-cayley and UWM by 1.8 to 3.0 dB in all 25 paired comparisons over three seeds (each p<0.01p < 0.01), and remains on par with HNN. Under the Coulomb law NWM is also data-efficient (PSNR in dB, Appendix D.13):

Training data10%25%50%100%
NWM20.2020.7720.6120.41
Ctrl-free18.2218.4818.6318.58
Ctrl-cayley18.7218.0617.6417.50
MLP18.4218.3918.4518.45

Trained on a tenth of the data, NWM is above every unconstrained or norm-conserving transition trained on all of it (at most 19.14 dB, Table 17), and from 10% to 50% of the data it leads by 1.5 to 3.0 dB in nine paired comparisons over three seeds (each p<0.02p < 0.02).

Q2 (continued). Planning (W3). We compare LeWM with and without our penalty (Table 2, drift of ∥z∥2\lVert z \rVert^2 at h=7h = 7):

EnvironmentHost driftWith penaltyChange in success (points)
Tworoom31.65%21.16%+7.33±0.60\mathbf{+7.33 \pm 0.60} (p=0.007p = 0.007)
Push-T6.27%3.67%−0.50-0.50
Reacher6.78%5.14%−1.89±1.20-1.89 \pm 1.20 (p=0.25p = 0.25)

The penalty lowers drift on all three tasks, and planning improves where the host drifts most, in each of three training seeds (+8.50, +6.50 and +7.00 points). On Push-T and Reacher the change is not a measurable loss. Applied directly to the pixel latent, the exact operator plans poorly, which Proposition 3 explains: no norm-preserving step can track a latent whose norm varies between frames (Appendix D.9).

These results show that conserving the family is beneficial under a change of dynamics, that the gain vanishes when the family is broken, and that two of its members, the norm and the block sum imposed as a penalty, improve planning where the host drifts most (Remark 6).

W5. Prior work on physics priors. We agree that physics-inspired priors on pixel data have not significantly improved over standard models in distribution (Botev et al., NeurIPS Datasets and Benchmarks 2021), and our in-distribution result agrees with this. Conservation has nonetheless improved prediction in related settings: a meta-learned conservation loss on learned embeddings improved video prediction (Alet et al., NeurIPS 2021), and learned quadratic conserved quantities improved test accuracy on n-body systems (van der Ouderaa et al., NeurIPS 2024). Our results exhibit the same profile as the strongest physics prior in the comparison: HNN is 0.11 dB below the MLP in distribution (Table 12) and leads it under the Coulomb law (20.41 against 18.45 dB, Table 17). NWM reaches the same level as HNN, without assuming Hamiltonian dynamics and with 32 times fewer parameters. NWM differs from that line in what it guarantees. Hamiltonian models such as HGN (Toth et al., ICLR 2020) conserve one learned energy, and the guarantee holds for the continuous-time flow, so the discrete step keeps it only up to integration error (§2). Discrete-gradient networks (Matsubara et al., NeurIPS 2020) and FINDE (Matsubara and Yaguchi, ICLR 2023) conserve a finite number of quantities exactly in discrete time, at the price of an implicit step. NWM conserves a 3,200-dimensional family of quadratic forms at every step and for every value of its weights, by Theorem 1 and Corollary 2, with one explicit update that depends on the state and the action (Table 3). The question the paper addresses is therefore different: what exact conservation of an entire family costs, as bounded by Proposition 3, and where it is beneficial, as Part 2 shows with matched controls.

W6. Readability. We agree. We have rewritten lines 173 to 177 as follows:

"Equation 1 differs from existing designs in two ways. First, the identity holds at every parameter value. A penalty satisfies it only approximately, and only at a minimiser of the loss. An integrator satisfies it only in the limit of a vanishing step size. Second, UU depends on both the state ztz_t and the action ata_t. A fixed unitary matrix depends on neither of them. An action-conditioned unitary matrix depends only on the action."

The reviewer indicated that affirmative answers to Q1 and Q2 are necessary for a change in rating. We believe Part 2 and Part 3 provide them, and we would be grateful if the reviewer reconsidered the score.

Reviewer_7e1NOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a structured latent transition based on a graph Laplacian and Cayley transform. The construction guarantees conservation of a family of quadratic latent invariants, with accompanying theoretical characterization. The authors evaluate these properties on physical prediction tasks and investigate whether similar constraints can improve existing JEPA-based world models.

Critical strengths

The mathematical construction is the strongest aspect of this work. Establishing an exactly conserved family of invariants, rather than enforcing individual quantities through loss penalties, is an interesting contribution. The matched architectural controls also help demonstrate which guarantees come specifically from the Laplacian structure.

Critical weaknesses

It remains unclear how useful the conserved quantities are for modeling actual physical systems. The guarantees concern the latent representation, but the analysis in Appendix C.5 provides limited evidence that most of these quantities correspond to meaningful physical invariants. Exact conservation alone does not necessarily imply more accurate dynamics.

The practical results are mixed. The exact transition performs poorly when transferred to pretrained world-model representations, while the soft constraint improves planning in only one of the three evaluated environments. I also found the nearly zero full-Jacobian log-determinant in Table 1 surprising: pointwise unitarity does not generally imply volume preservation when the operator depends on its input.

Questions for authors

Can the authors clarify how the full Jacobian was computed, particularly whether the derivatives of the state-dependent coupling coefficients were included?

Beyond the latent norm, can the authors identify conserved quantities that correspond to measurable physical properties and demonstrate their contribution to prediction accuracy?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7e1N (1/3): Overview, W1, and Q2 on the conserved family

We thank the reviewer for recognising the mathematical construction and the matched controls. We address every point with the paper and new experiments. Part 1 shows that the conserved family contains the angular momentum and the total momentum (W1, Q2). Part 2 shows that members of the learned family predict the energy and the angular momentum close to what a single frame allows, that NWM keeps them fixed along the rollout, and that preserving the family is what maintains accuracy under a change of force law (Q2). Part 3 explains the transfer and planning results by Proposition 3 and the drift criterion of §4.4, and confirms that the full Jacobian includes the derivatives of the couplings (W2 to W5, Q1). All changes in the revised PDF are marked in blue, and line numbers refer to the submitted version.

W1. Evidence in Appendix C.5. We agree that Appendix C.5 probes only three members of the family. As Appendix C.5 states, the operator fixes the algebraic shape of the conserved quantities and the encoder fixes their content. The relevant questions are thus whether the family can represent physical invariants and whether a trained encoder actually encodes them. Q2 addresses both for the full 3,200-dimensional family, and we have added this analysis to Appendix C.5.

Q2. Conserved quantities beyond the norm, and their contribution to accuracy. Yes. Corollary 2 characterises these quantities explicitly, members of the learned family are predictive of energy and angular momentum, and preserving the family is what maintains accuracy under a change of force law.

(i) What the family is, and (ii) what it contains. A member has the form z†(A⊗C)zz^\dagger (A \otimes C) z with A∈span⁡{IN,11⊤}A \in \operatorname{span}\lbrace I_N, \mathbf{1}\mathbf{1}^\top \rbrace and CC any m×mm \times m matrix, Hermitian for the real-valued members (Corollary 2). Writing each object as zi=(qi,pi)z_i = (q_i, p_i) with pip_i its momentum:

AAValue of the memberPhysical quantities it contains
INI_Ntr⁡(CS(z))\operatorname{tr}(C S(z)) with S(z)=∑izizi†S(z) = \sum_i z_i z_i^\daggerthe norm (C=ImC = I_m) and the planar angular momentum ∑i(qi,xpi,y−qi,ypi,x)\sum_i (q_{i,x} p_{i,y} - q_{i,y} p_{i,x}) (a real symmetric CC)
11⊤\mathbf{1}\mathbf{1}^\topP†CPP^\dagger C P with P=∑iziP = \sum_i z_ifunctions of the block sum PP, which contains the total momentum

NWM therefore preserves the entire m×mm \times m matrix S(z)S(z) and, by Theorem 1, the vector PP itself, and the family contains the conserved quantities of spatial translation and rotation. Whether a learned encoder actually encodes these quantities is an empirical question, since without supervision the factors captured by a latent are not identifiable (Locatello et al., ICML 2019). Part 2 addresses it.

AuthorsOpenReview ↗

Response to Reviewer 7e1N (2/3): Q2 continued, the physical invariants and their effect on accuracy

(iii) Which physical invariants the learned family encodes. The members of the family are exactly the linear functions of S(z)S(z) and PP†P P^\dagger. For each invariant we fitted a probe on these quantities at t=0t = 0, by ridge regression (linear) and by RBF kernel ridge (nonlinear), with hyperparameters chosen by cross-validation on the training split and R2R^2 on the test split over three seeds, for all nine latents. A flexible probe can fit targets that the representation does not encode (Hewitt and Liang, EMNLP 2019), so we compare against the best a single frame allows: the same probe on the true positions, or on visual N-body, where the true state is not stored, a supervised network with the same encoder trained on the first frame. The last column is the median probe error against the ground truth at h=100h = 100, in units of the test standard deviation.

QuantityKernel R2R^2Linear R2R^2Single-frame limitRecoveredError at h=100h = 100, NWM vs. best other
Energy, visual N-body0.54±0.030.54 \pm 0.030.400.66 (supervised)82%0.35 vs. 0.43
Energy, HNN 3-body0.37±0.010.37 \pm 0.010.370.40 (true positions)93%0.56 vs. 0.75
Angular momentum, HNN 3-body0.39±0.010.39 \pm 0.010.390.42 (true positions)94%0.57 vs. 0.76
Angular momentum, visual N-body, and momentum, HNN 3-bodybelow 0.03below 0.03below 0.03not identifiable from one framen/a

A frame shows positions but not velocities, and adding the true velocities raises the limit to 1.00, so the family, learned without physical supervision, recovers most of what a frame contains. With the same probe the other eight latents give energy R2R^2 between 0.08 and 0.55 on visual N-body and between 0.33 and 0.39 on HNN 3-body. Hence the encoder determines what is encoded, and the transition determines whether it is preserved. Under NWM each probe changes by less than 4×10−134 \times 10^{-13} of the standard deviation of its target over 10410^4 steps in double precision, under the other eight transitions it drifts, and the NWM kernel probe has the lowest error of the nine models at h=1,10,29,50h = 1, 10, 29, 50 and 100 for all three quantities. The linear probes give the same ordering at h=100h = 100.

(iv) Contribution to accuracy. Ctrl-cayley shares the coupling network and the Cayley step with NWM and differs only in placing the couplings on a random Hermitian basis, so it conserves the norm but not the family (Appendix C.4.2). Ctrl-free is unconstrained, with the same parameter count to within 1%. Each sector variant breaks one part of the family, as verified on the trained weights. PSNR in dB at h=29h = 29, three seeds (Table 17, Appendix D.13):

ModelConservedIn distributionCoulomb 1/r21/r^2
NWMfull family18.8620.41
NWM without zero row sumsS(z)S(z)18.8720.37
NWM without block-scalar HH∥z∥2\lVert z \rVert^2 and PP18.7220.33
Ctrl-freenone18.6718.58
Ctrl-cayleynorm only17.6517.50

We also trained NWM interpolated with the MLP transition through Eq. 9, three seeds each, one shared recipe, so that α=0\alpha = 0 retrains NWM itself:

α\alpha00.010.030.10.250.51
In distribution18.8618.9518.8818.1017.8618.9618.96
Coulomb 1/r21/r^220.3917.6217.5717.6517.9718.3418.44

Every α>0\alpha > 0 breaks the family (out-of-family to in-family drift ratio 1.04 to 5.8) and costs 1.95 to 2.82 dB under 1/r21/r^2. At α=0.01\alpha = 0.01 and α=0.5\alpha = 0.5 the in-distribution accuracy matches NWM, and α=0.5\alpha = 0.5 is even more accurate at h=100h = 100 (21.27 against 19.75 dB), yet both lose under the force-law change. The gain is therefore attributable to the family rather than to the in-distribution fit, and preserving either sector retains it while preserving the norm alone does not.

AuthorsOpenReview ↗

Response to Reviewer 7e1N (3/3): Q2 summary, W2, W3, W4, and W5 with Q1

Q2, in summary. The conserved family encodes most of the information a single frame provides about the physical invariants, keeps it fixed along the rollout, and is the component responsible for robustness under a change of force law. We have added (iii) to Appendix C.5 and the α\alpha sweep to §4.3, with the full results in Appendix D.13.

W2. Conservation and accuracy. We agree that exact conservation alone does not imply more accurate dynamics. The paper draws the same line for in-distribution prediction, where "what distinguishes them is what the rollout holds fixed rather than how well it predicts" (lines 89 to 91). This agrees with prior findings. Physics-inspired priors on pixels do not significantly improve over standard models in distribution (Botev et al., NeurIPS Datasets and Benchmarks 2021), and the gains of Hamiltonian networks have been traced to modelling acceleration directly rather than to energy conservation (Gruver et al., ICLR 2022). Symmetry built into a dynamics model pays off instead under distributional shift (Wang et al., ICLR 2021). Our controls isolate such a condition for conservation itself. Under a change of force law the gain appears only when the family is preserved, and it disappears as soon as the family is broken, at unchanged in-distribution accuracy (Q2 (iv), Part 2).

W3. The exact operator on a pretrained latent. This result is the boundary established by Proposition 3, and the second route of the paper is designed around it. Proposition 3 shows that every norm-preserving step has a one-step error of at least ∣∥zt+1∥−∥zt∥∣\lvert \lVert z_{t+1} \rVert - \lVert z_t \rVert \rvert, for every value of its parameters, so the bound depends only on the encoder and the data. On the LeWM latent the encoded norm changes by a median of 4.9% per frame (§4.4), which no unitary step can track. Capacity is not the cause (Appendix D.9):

Transition on the LeWM latentParametersPlanning success
Exact operator, matched size16,6573.83%
Unconstrained transition16,70414.83%
Exact operator, host size10.8M3.17%

This trade-off between exact norm preservation and expressivity is known for unitary and orthogonal models (Vorontsov et al., ICML 2017, Kerg et al., NeurIPS 2019). Proposition 3 makes it quantitative and computable from the pretrained encoder before the transition is trained, which motivates the two routes of the paper (§3.4): the exact operator where the encoder is trained with it, and the penalty of Eq. 10 where the encoder is fixed. The revised §4.4 states this boundary in its opening sentence, and Theorem 1 and Corollary 2 are unaffected by it.

W4. Planning with the penalty. We agree that the planning gain appears in one of the three environments. This matches the criterion stated in §4.4, which relies only on the drift of the host, measured before any planning run (Table 2, drift of ∥z∥2\lVert z \rVert^2 at h=7h = 7):

EnvironmentHost driftWith penaltyChange in success (points)
Tworoom31.65%21.16%+7.33±0.60\mathbf{+7.33 \pm 0.60} (p=0.007p = 0.007)
Push-T6.27%3.67%−0.50-0.50
Reacher6.78%5.14%−1.89±1.20-1.89 \pm 1.20 (p=0.25p = 0.25)

The penalty lowers drift everywhere and changes planning only where the host drifts about five times more. On Tworoom the gain holds in each of three training seeds (+8.50, +6.50 and +7.00 points, paired tt-test), and on Push-T and Reacher the change is not a measurable loss. A lower model error need not translate into control performance (Lambert et al., L4DC 2020). The revised §4.4 states this criterion in its opening sentence.

W5 and Q1. The full Jacobian. We agree that pointwise unitarity does not imply volume preservation when the operator depends on its input. The derivatives of the couplings are included. Because the step is not holomorphic in zz, the column is the real 240×240240 \times 240 Jacobian of x↦F(x)x \mapsto F(x) with x=(Re⁡z,Im⁡z)x = (\operatorname{Re} z, \operatorname{Im} z), computed by automatic differentiation with gradients through the pairwise distances, the coupling network, the Laplacian assembly and the Cayley solve. We recomputed it in full double precision: central differences with step 10−610^{-6} reproduce it on all 64 anchors to a relative Frobenius error below 1.5×10−101.5 \times 10^{-10}, and log⁡∣det⁡J∣\log \lvert \det J \rvert remains of order 10−1510^{-15}. The near-zero value is an empirical property of the trained model on visual N-body rather than a consequence of unitarity. The caption of Table 1 now states this. The guarantees of the paper concern the conserved family (Theorem 1, Corollary 2) and do not rest on volume preservation.

We hope these responses resolve the reviewer's concerns, and we would be grateful if the reviewer reconsidered the score.

AuthorsOpenReview ↗

Response to Reviewer 7e1N: each point of the review and its revision

Before the discussion closes, we wish to confirm whether our responses have addressed your concerns. For convenience, each point of the review is listed below with its revision in the PDF, marked in blue.

W1. Appendix C.5 now probes the full conserved family rather than three members. The learned family recovers 93% and 94% of what a single frame determines about the energy and the angular momentum, and stays fixed along the rollout (Table 22).

W2. §4.3 now states what the family contributes and under which conditions. In Appendix D.13, breaking the family by an interpolation weight or by removing one sector costs 1.95 to 2.82 dB under a change of force law while in-distribution accuracy is unchanged (Tables 19 and 20).

W3. §4.4 now opens with the bound of Proposition 3, which explains the result on the pretrained latent. Appendix D.9 shows that capacity is not the cause: the exact operator plans at 3.17% even at the host's 10.8M parameters.

W4. Table 2 now reports Tworoom over three training seeds (+7.33±0.60+7.33 \pm 0.60 points, p=0.007p = 0.007), and §4.4 states the drift criterion, measured before retraining, that separates the three environments.

W5 and Q1. The caption of Table 1 now states that log⁡∣det⁡J∣\log \lvert \det J \rvert is computed from the full Jacobian, including the derivatives of the couplings, and that its near-zero value is a measured property of the trained model rather than a consequence of unitarity.

Q2. Appendix C.5 and Table 22 show which physical invariants the family encodes, and §4.3 with Appendix D.13 shows that preserving the family is what maintains accuracy under a change of force law (Tables 19 to 21).

We would be grateful if you could let us know whether these revisions address your concerns. If any point remains unclear, we would be very happy to answer it. We believe the core of the paper, a transition that conserves an entire family of quadratic forms exactly and the bound that tells where this applies, offers something new to the study of world models, and we would sincerely appreciate it if you would consider raising your score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the rebuttal and the revision for the Program Chairs, Senior Area Chairs, Area Chairs and Reviewers

We thank the Program Chairs, the Senior Area Chairs, the Area Chairs and the three reviewers for their time and careful reading. As the discussion period draws to a close, we summarise our responses in one place.

We have answered every weakness and question of each reviewer in detail, and the revised PDF marks every change in blue. The main additions are as follows.

(i) Physical meaning (7e1N, 61NY). The conserved family contains the angular momentum and the total momentum. Probes on the learned family recover 93% and 94% of what a single frame determines about the energy and the angular momentum, and stay fixed over 10410^4 steps (Appendix C.5).

(ii) Where the family helps (7e1N, SE5x, 61NY). Under a change of force law NWM leads the matched controls by 1.8 to 3.0 dB in all 25 paired comparisons (p<0.01p < 0.01), also when trained on a tenth of the data. Breaking the family removes the gain at unchanged in-distribution accuracy (§4.3, Appendix D.13).

(iii) Precision (SE5x). In double precision every member of the family stays at round-off, and the family ratio exceeds 5×10145 \times 10^{14}, against at most 5.0 for every other transition (Appendix D.11).

(iv) Pretrained latent and planning (7e1N, 61NY). The result of the exact operator on the pretrained latent follows the bound of Proposition 3, and a capacity control rules out model size (Appendix D.9). With the penalty, the Tworoom gain holds over three training seeds (+7.33±0.60+7.33 \pm 0.60 points, p=0.007p = 0.007), and §4.4 now opens with the drift criterion that separates the three environments (Table 2).

(v) Guarantees and the Jacobian (SE5x, 7e1N). §3.5 states the guarantee of each relaxed variant precisely, and the full Jacobian, including the coupling derivatives, was verified in double precision (Table 1).

(vi) Scope and prior work (61NY). We clarified that the conserved family is fixed by the operator rather than chosen in advance, and positioned NWM against prior work in which conservation improved prediction.

(vii) Clarity (61NY). The passage flagged as hard to read is rewritten (§3.1), and the latent quantities of Table 1 and Figure 3 are labelled as such, next to the decoded PSNR.

We would be very grateful for the reviewers' feedback on whether our responses address their concerns, and we remain glad to answer any further question.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.