- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces LeHami (Hamiltonian JEPA), an action-conditioned latent world model framework designed to prevent distribution contraction and instability over long autoregressive rollouts. LeHami replaces the standard unconstrained predictor with a learned separable Hamiltonian flow over a canonical latent split (), integrated via an exactly symplectic leapfrog step. Momentum () is derived via a zero-parameter Savitzky-Golay (SG) stencil on the encoder's position trajectory. The authors prove that this symplectic construction provably preserves phase-space volume and eliminates collapse modes (such as uniform contraction or asymptotic stability) at every rollout step. Experiments on conservative and actuated pendulum environments demonstrate that LeHami maintains constant latent volume (where baselines contract), improves long-horizon velocity prediction, and supports time-reversal and continuous-time re-stepping.
Critical strengths
- Well-motivated geometric framework for world models.
The paper introduces a principled Hamiltonian formulation for JEPA world models. The use of symplectic integration and structured latent dynamics is theoretically elegant and directly addresses the problem of long-horizon latent-space contraction.
- Strong theoretical and empirical consistency.
Theoretical guarantees regarding volume preservation, avoidance of contraction, and reversibility are clearly derived and empirically validated. The architectural choices are well aligned with the underlying mathematical framework.
Critical weaknesses
-
The guarantee concerns ambient Lebesgue measure, not the data. • Pendulum data lives on a roughly 2-D manifold inside a 64-D latent space. Volume preservation in R⁶⁴ allows contraction along the data manifold, compensated off-manifold. • Fig. 4 (right) and Fig. 13 show log|det J| = 0 for LeHami, which holds for any symplectic net, even untrained. • LeWM's det < 1 is also not evidence of collapse, because off-manifold directions are never trained. The needed diagnostics are the Jacobian restricted to the data tangent space, rollout effective rank, or pairwise distances between rollouts from distinct anchors, to test the "merging" claim.
-
It doesn't prevent collapse, and Table 8 suggests near-collapse. • A constant encoder with ∇V(c,u)=0 gives a neutrally stable fixed point, which Prop. 1(ii) does not exclude. • With the default q-only SIGReg, only 15% of latent variance is within-episode, against 42.8% for the baseline. The caption says near zero means "an episode identifier rather than a state". Joint SIGReg gives 0.01%. • std(p)/std(q) is about 0.10, so the principled regularization of (q,p) fails, and p is effectively unregularized.
-
No ablation isolates symplecticity. • Missing controls: (a) LeWM plus the same SG momentum channel and 4-frame anchor; (b) a plain MLP predictor on (q,p,u) with matched parameters; (c) LeWM at d=32. • The velocity gain at K=0 comes from giving LeHami's encoder a 4-frame stencil while the baseline is deliberately per-frame. • The FLOP savings come from MLP vs. 6-layer transformer, not from Hamiltonian structure. • Capacity is also unmatched: p is a function of q, so LeHami has 32 free dimensions against 64, which plausibly explains its 6–16× worse angle error in Table 1.
-
Planning is negative, and the framing is generous. • Table 2: LeHami succeeds 0.48/0.53/0.30 on the three clauses (position/velocity/both), against 0.80/0.80/0.64 for the best LeWM variant. • The tied kernel is a strawman, since no one would use it for velocity. Untied is the real baseline, and it wins at 13.65 vs. 10.98 TFLOP. • Matching budgets by varying R per model confounds the comparison, so compute-matched runs at equal R are needed. • With std(p) about a tenth of std(q), the latent goal cost barely weights velocity. Block normalization is untested and may explain the velocity gap.
-
Long-horizon claims lack chance baselines. Wrapped angle MAE of 0.6–1.0 rad and ω MAE above 2 rad/s at K>20 are likely near trivial predictors for both models. Add a predict-mean or persistence baseline. • A conservative model can keep the right energy yet drift in phase, so "stays on its level set" (by construction, Figs. 10–11) says little about accuracy. • Attributing LeWM's errors to "compounding contraction" is not tested.
-
Time reversal and re-stepping are properties of the integrator. Strang splitting satisfies Ψ₋ₕ = Ψₕ⁻¹ exactly for any H, so the roughly 10⁻⁵% round-trip error (Fig. 5 right) is floating-point precision. The meaningful test is backward-rollout accuracy against held-out earlier frames. Sub-stepping consistency shows convergence of the integrator, not correct physics at unseen frame rates. The mixed-rate experiment is described but not run.
-
Scope. • There is one 1-DoF pendulum whose Hamiltonian is exactly separable, which perfectly matches the prior. The unit-mass assumption p = q̇ fails for M(q), as the authors admit. • No dissipative, noisy or contact environment is tested, even though real systems have friction and stabilizing controllers. Ruling out stable fixed points can be a bug for world models. • The title says "world models", but the evidence is one toy system with 210 training episodes.
-
Statistics. There are three seeds, but Table 4 (the "one-eighth compute" claim) is a single seed with non-monotone LeWM numbers (0.590 → 0.871 → 0.487), so it is fragile. It also uses a k=20 probe where the rest use k=25. Fig. 13 is one seed.
-
Questions for authors
Can the authors demonstrate LeHami on substantially more complex control and visual world-model benchmarks beyond pendulum systems?
How sensitive is the framework to the quality of the estimated momentum variable obtained via the fixed Savitzky-Golay derivative estimator?
Can the authors provide a quantitative analysis relating latent-volume preservation to rollout prediction accuracy across multiple environments?
What are the results with a non-symplectic MLP on (q,p,u), and with LeWM plus the SG channel?
What is the contraction on the data tangent space, not ambient?
Why does q-only SIGReg leave 85% of variance between-episode?
LeHami argues that a contractive deterministic rollout is a collapse mode, while LIFS-JEPA [1], enforces contraction for stability. [1]Belloulata, 'Contractive MASO-Generalized Predictors for Stable Latent-Space Learning in JEPA, in Transactions on Machine Learning Research (TMLR), 2026, https://openreview.net/forum?id=k2Z2gPOtlq.
The reconciliation is that the right predictor class depends on the dynamics. For stochastic or dissipative processes such as OU, the conditional-mean map should contract. For deterministic conservative systems it should not. Stating what is contracted, state volume versus error dynamics, would pre-empt the objection.
Could the authors discuss the expected advantages and limitations of the two approaches with respect to rollout stability, latent-space structure, and long-horizon prediction accuracy?
6 comments
Response to m2u2 (2/3): planning, chance baselines, backward rollout
3.4 Planning results are negative.
Tables A–D: second general comment. Changes: in blue in the revised PDF.
We agree. Planning is no longer a main result: revised Section 4.5 is one paragraph presenting it as an application with its limits, and the table moves to the appendix (revised Table 4), where the untied LeWM, which captures both the sign and the size of the velocity, is the strongest baseline. LeHami plans at a lower cost per step; it does not reach goals better. On equal R: at the same budget both were run at R = 9 (4 solves) on the position clause (0.48 against 0.80) and on the sign of the velocity (0.80 against 0.96 for untied LeWM; App. Table 10, now Table 12), so its lower success there is not due to R. The goal cost under-weights p for the reason given above for Table 8; normalising each block would be a good fit, but we did not have time to try it. Future work can use integration at a coarser step and planning backward from the goal.
3.5 Long-horizon claims lack a chance baseline.
We added both baselines you propose, repeat the start (persistence) and predict the mean, on all five environments: Tables B and B', and the chance line in revised Figs. 4 and 18. Partly right: on the pendulums at K = 70 the velocity errors are close to chance for every model except LeWM W=3; LeHami is below it, narrowly (2.19 against 2.50; 2.28 against 2.54). In position LeHami beats chance at every step shown; at K = 70 on the conservative pendulum the free predictors are worse than chance and the paper's LeWM is at chance (1.24, Table D). Energy and phase: we agree that energy conservation says little about whether a rollout is in step with the truth; our errors score the decoded angle and velocity at each step, so a rollout that runs ahead or behind is penalised. Compounding contraction is now tested on the data, at every step (Table A, revised Figs. 4 and 17).
3.6 Reversal and re-stepping are integrator properties.
We agree the round trip is an integrator identity, now presented as an implementation check. We ran the test you propose, and the revised paper now includes it (Fig. 5, right, and App. F.6): from a mid-episode frame of a held-out episode, roll LeHami back K steps (with the earlier torques when actuated) and score against the true earlier frames. Conservative pendulum: backward matches forward in angle and velocity, and both beat both baselines up to K = 20 (angle at K = 20: 0.51 backward, 0.53 forward, against 1.05 and 1.27). Actuated pendulum, angle [rad], MLP probe, mean ± std over three seeds (the baselines use no model):
| angle θ [rad], K | forward | backward | repeat the start (backward) | predict the mean (backward) |
|---|---|---|---|---|
| 1 | 0.077 ± 0.009 | 0.074 ± 0.006 | 0.242 | 1.047 |
| 5 | 0.343 ± 0.030 | 0.332 ± 0.031 | 1.120 | 1.041 |
| 10 | 0.531 ± 0.036 | 0.506 ± 0.021 | 1.487 | 1.025 |
| 20 | 0.716 ± 0.050 | 0.684 ± 0.096 | 1.038 | 1.004 |
The angle is recovered as well backward as forward; the velocity beats chance up to 10 steps. A free predictor has no backward step. We did not run the mixed-rate experiment. It matters for planning and control: a model that integrates at any step size can be trained on data recorded at several rates (Doshi et al., 2024) and can plan on non-uniform time grids, fine near the present and coarse further ahead (Tan et al., 2016).
Response to m2u2 (3/3): scope, statistics, LIFS-JEPA, citations
3.7 Scope and assumptions.
Tables A–D: second general comment. Changes: in blue in the revised PDF.
We accept each point and narrow the claims. Separable, constant mass: a constant mass is absorbed by the latent coordinates; it may differ between episodes, but not within one (reply to 4fNR). A configuration-dependent M(q) is not: we added the cart pole to test LeHami where this assumption fails. Its rollouts still do not merge, but it is less exact (Table A). One system: we now have five. Dissipation, noise, contact: not covered; for friction a volume-preserving step is the wrong prior, and the extension is a conformal or port-Hamiltonian step (Limitations). Stable fixed points: a stable fixed point is a state that nearby states converge to, like a pendulum with friction coming to rest at the bottom. Ruling them out is a defect for dissipative or stabilised systems, and correct for conservative ones, where nearby states never converge; it is what stops distinct states merging. Title: we keep "world models", and the revised abstract and introduction state the scope.
What we mean by a world model: a model that predicts how a system evolves, including under the agent's actions, far beyond its training horizon (Ha & Schmidhuber, 2018; LeCun, 2022). Every use needs long rollouts that stay close to the true states. We test that four ways: prediction to 70 steps against chance, four measures of collapse (Table A), running backward to the true past, and keeping the system's properties (the mass, R² ≥ 0.98 after 70 steps). LeHami supports actions by construction: they enter the potential, so every step stays symplectic under any action sequence; the last two tests are on the actuated pendulum, under the recorded torques. Planning is one application: it has limits, but shows that a symplectic Hamiltonian predictor can be used to plan actions.
3.8 Statistics.
Table 4 at three seeds (conservative pendulum, rollout step K = 20, 25-NN probe; mean ± std over three seeds); it replaces App. Table 4 (now revised Table 5):
| model | 0.75 PFLOP | 1.5 PFLOP | 3.0 PFLOP | 6.0 PFLOP | |
|---|---|---|---|---|---|
| MAE θ [rad] | LeHami | 0.304 ± 0.031 | 0.213 ± 0.008 | 0.247 ± 0.013 | 0.207 ± 0.041 |
| LeWM | 0.642 ± 0.048 | 0.710 ± 0.140 | 0.462 ± 0.025 | 0.332 ± 0.020 | |
| MAE ω [rad/s] | LeHami | 1.045 ± 0.093 | 0.939 ± 0.062 | 0.916 ± 0.128 | 0.854 ± 0.075 |
| LeWM | 1.604 ± 0.023 | 1.694 ± 0.408 | 1.345 ± 0.315 | 1.178 ± 0.132 |
LeHami has the lower error at every budget. The non-monotone LeWM number was one seed (new seeds: 0.632, 0.625 at 1.5 PFLOP). "One-eighth compute" holds as "on par" (0.304 rad at 0.75 PFLOP against 0.332 at 6.0). The 20 in Table 4 is the rollout step, not the probe's neighbours; the revised caption says so. All new results are at three seeds; Fig. 13 (one seed) is replaced by Table A.
3.9 Relation to LIFS-JEPA (Belloulata, 2026).
Thank you for bringing this work to our attention. LIFS-JEPA (Belloulata & Belalia, 2026) is a JEPA for static images. Its predictor refines one prediction by iterating a contractive map towards a fixed point: the iterations are refinement steps, and converging is the goal. In a world model each iteration is a step in time, and a contracting predictor makes rollouts from different states converge: the collapse we measure. So the right predictor class depends on what the iterations mean; revised Section 2 now says so.
3.10 Citation concern.
We thank the reviewer for bringing these works to our attention. We now cite SympNets (Jin et al., 2020), the general class of symplectic networks; our leapfrog step is one of them, chosen because it preserves volume, which we use against rollout collapse, while SympNets target learning Hamiltonian systems. LNN (Cranmer et al., 2020) is cited in revised Section 2 and App. C.3, where we explain choosing the Hamiltonian over the Lagrangian form. For HamJEPA, see our reply to the public comment.
Response to m2u2 (1/3): collapse on the data, Table 8, the ablation
3.1 The guarantee is about ambient measure, not the data.
Tables A–D are in our second general comment (Table A = revised Table 1; Tables B, B' = revised Table 2). Changes are in blue in the revised PDF; "revised" refers to it, plain numbers to the submission.
You are right: Prop. 1 holds for any leapfrog step of any H, trained or random, and says nothing about the roughly 2-D surface the data occupy; LeWM's det J < 1 is not by itself evidence of collapse, and the revised paper drops that panel.
What we measure instead. Each measure compares the rollouts at step K with the truth, the encoder's latents of the true frames at the same step, as a ratio (1 is right, below 1 the rollout contracts):
Results (Table A). Both free predictors contract on every environment and measure. LeHami's rollouts never merge: their distance and spread stay at or above 1; where its tangent area or rank falls below 1 (two-body, cart pole), it squeezes some directions and stretches others. On plain LeWM you are right (Table D): its rollouts keep their distances and rank; its tangent area falls below 1 only on the conservative pendulum (0.78) and the cart pole (0.76). Trained or not: with random weights LeHami keeps the whole latent's volume exactly, yet its tangent area ranges from 0.32 to 5.09: the guarantee alone says nothing about the data. Revised Fig. 4 (right) shows it building up with K: the distance between rollouts falls steadily for the two unstructured predictors and stays at 1 for LeHami.
3.2 Near-collapse in Table 8.
Both points are right.
(a) A latent that does not move. Prop. 1 rules out contraction of the whole latent, not a latent that stands still (a constant encoder with ∇V=0). Revised Section 3 now says so.
(b) What the 85% is. Each actuated episode has one of nine (mass, length) settings. 85% of the variance of q is its average over each episode, which encodes the setting; only 15% is the motion within the episode. That motion lies almost on a straight line in the latent (one direction holds 99.2% of it). A swing fits on such a line, but 30% of test episodes go over the top and turn full circles, where the angle wraps around, which a line cannot represent. This, not a loss of volume, is LeHami's failure mode on the actuated set.
(c) Why std(p)/std(q) ≈ 0.10. p is a time derivative, so it has no episode average and only sees the motion within episodes: std(p)/std(q)=ρwwq=0.267×0.151=0.104 (Table 8's 0.10, now revised Table 10), with wq the within-episode share of the variance of q and ρw the same ratio within episodes. So p is not collapsed: it looks small only next to the episode averages that q also carries, and within episodes its ratio to q (0.27) matches the data's own (0.24). The real cost is in the loss: the squared error weights p by ρw2≈0.07 of q. Two fixes would fit well, but we had no time to run them: scaling q and p by their within-episode spread, and keeping the episode constants out of the flow, V(q,u,c).
3.3 No ablation isolates symplecticity.
We ran the first two controls, on five environments with three seeds. All three models share the paper's per-frame ConvNet, the 32-dimensional q, the state [q, SG(q)], the losses and the 8-step rollout; only the predictor differs: (i) LeWM's transformer, i.e. LeWM with the SG momentum channel; (ii) a parameter-matched MLP; (iii) LeHami's Hamiltonian step. Results: Tables A, B, B' (second general comment) and revised Figs. 4, 17, 18.
Capacity: the controls learn the same 32 dimensions as LeHami, so latent capacity is matched and we did not run LeWM at d = 32. Stencil and predictor: at K = 0 the probe reads the encoded state, so we agree the K = 0 velocity gain comes from the stencil. The rollout gain does not: with the same stencil, the free predictors contract on all five environments (Table A), and on the conservative pendulum their velocity error at K = 10 is 2.5 to 3 times LeHami's (Table B').