acceptodds
Withdrawn from ICLR 2027

Learning Physical Dynamics through Relational Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Predicting physical trajectories in unfamiliar configurations requires reusing local interactions. Trajectory supervision evaluates their combined effect on motion, so fitting training trajectories need not capture how individual relations evolve. We introduce **HORME**, which trains the same pair features for motion computation and prediction of relational evolution. To supervise this evolution during recursive prediction, we propose Reference-Anchored Joint-Embedding Predictive Architecture (RA-JEPA). It matches predicted and observed feature changes pair by pair in a shared space, retaining initial features as a fixed reference. The forecasts condition physical pair features, allowing trajectory errors to train the predictor. Computing displacement further requires resolving when acceleration acts within each step. To capture timing effects missed by sparse acceleration samples, we propose Temporal Response Decomposition (TRD), separating displacement contributions from mean acceleration and its timing. A readout shared across integration stages corrects the timing-dependent component while preserving the current step's velocity update. HORME achieves the lowest trajectory root mean square error (RMSE) among compared methods on I-PHYRE and gravitational N-body, including unseen games, larger systems, and extended horizons.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Physical Prediction, Relational Learning, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_UaDhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes HORME, a framework that models relational evolution using pair features for motion computation, rather than simply fitting trajectories. It introduces RA-JEPA to predict how each pair feature changes relative to the initial state of a window and supervise these predictions by matching them with encoded observed states on a pairwise basis. It also introduces TRD to compensate for displacement errors caused by the timing of acceleration within a step, using a learned correction term applied only to position without modifying velocity updates. These two components are connected through forecast conditioning. On both the I-PHYRE and N-body benchmarks, HORME achieves the lowest trajectory RMSE compared to baselines such as NFF, and ablation studies demonstrate the necessity of both RA-JEPA and TRD.

Critical strengths
  1. Strong performance. The method clearly outperforms the baselines on both I-PHYRE and N-body experiments.

  2. Effective combination of existing ideas. Although the individual components build on prior work, the paper effectively combines existing components to improve performance. The ablation studies also clearly demonstrate that both RA-JEPA and TRD are necessary.

Critical weaknesses
  1. Unclear contributions of physical constraints and the proposed model on the N-body dataset. The effects of invariant inputs and antisymmetric readout are not separated from those of the proposed model. In particular, the model shows strong performance on N-body, but it is unclear whether this improvement comes from the two proposed modules (RA-JEPA and TRD) or from the physical constraints (invariant inputs and antisymmetric readout) that are not included in NFF. Therefore, an ablation study without these constraints is needed.

  2. Lack of experiments validating the physical accuracy of pairwise force decomposition. The paper only shows improvements in overall performance, without demonstrating whether the identifiability problem mentioned in the introduction has been resolved. Experiments are needed to show that the learned pair features are physically accurate.

  3. Writing clarity. Overall, the paper lacks consistency and is difficult to follow, making it hard to understand the main points. The introduction does not clearly present the key ideas, the related work section lists previous studies without discussing their relation to HORME, and many parts of the method section are unclear. The conclusion also repeats the abstract. The following are specific unclear points in the methodology:

  4. R_φ is not defined, and the operations of “stores” and “query projections” are not described. The construction of the P_θ query, the mapping from the bank to K/V, and the encoding format of u_k are not described. The differences between the encoders used for each benchmark are not explained in detail.

Questions for authors
  1. The model shows particularly strong performance on N-body compared to other baselines, and I would like to see a more detailed analysis of the reasons behind this improvement. What would the results be if only RA-JEPA and TRD were applied without invariant inputs and antisymmetric readout on N-body?

  2. Contribution of RA-JEPA. The claimed contribution of RA-JEPA is that it correctly learns individual interactions by predicting changes in pair features. However, Section 4.3 only shows that the predicted features are 9–14% closer to the encoder outputs than the zero-change baseline. This demonstrates self-consistency between the predictor and encoder, rather than providing evidence that the features contain physically accurate relational information. Since pairwise forces can be obtained from the simulators in both benchmarks, I would like to request experiments that either (a) compare the pairwise contributions of the head with the actual pairwise forces or (b) compare HORME and NFF using a probe that reconstructs pairwise forces from pair features. In addition, the MAE in Figure 2(c) does not distinguish between direction and magnitude errors, and the only baseline is zero-change. Please report R² or cosine similarity and norm ratios separately, and include stronger baselines such as linear extrapolation.

  3. Comparing Row 2 of Table 3 (TRD ✓, RA-JEPA ✗) with Row 1 of Table 4 (Auxiliary-only JEPA: TRD ✓, RA-JEPA predictor and prediction loss ✓, conditioning ✗), adding only the prediction loss without conditioning leads to worse performance on both benchmarks (I-PHYRE Cross: 0.119 → 0.126, N-body Cross: 0.685 → 0.943). A common assumption in this line of research (Section 2.2) is that an auxiliary latent prediction loss improves shared representations, but the results here show the opposite. This suggests that (i) the pair feature prediction task may push the shared encoder in a direction that is unfavorable for physical computation, and (ii) an explanation is needed for how conditioning resolves this conflict. The paper does not discuss this comparison. I would like to hear the authors’ interpretation of these results.

  4. Please clarify the unclear parts of the methodology mentioned in Weakness 3.

Reviewer_2QvxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies open-loop trajectory prediction for systems where there are interacting objects where the test configuration is different from those the network is trained on (different number of objects on N-body problem etc.). The authors propose HORME, which include two main components, namely RA-JEPA and TRD.

RA-JEPA focuses on predicting how pairwise relationships between particles will change over a window of time. The relationship descriptors are predicted forward in time while all being anchored to the observed initial descriptor state, minimizing drift accumulation compared to a RNN-style implementation. In addition to pairwise relationship the model also attends to the entire scene as a whole. These predictors are feed into the physics of the system, providing both the learning signal as well as the output dynamics. The pairwise and scene attention allows RA-JEPA to extend to unseen configurations as it never needs a whole picture view of the entire system.

TRD completes the other side by taking the conditioned pairs of features and transforming them into motion over the prediction time steps. Across a single step the change in speed depends only on the average acceleration, but when considering position it also depends on when within a step the acceraltion begins to act. TRD thererfore uses a shared response head at the start, center, and end of each step, where at each position a learned displacement coefficient is learned. The displacement coefficient adds corrections to the position only and are learned from the end points.

Critical strengths

S1. The motivation is clear and the solution is elegant. Figure 1 illustrates the issue of network strained solely to produce the predicted trajectory can split the forces incorrectly but still obtain correct dynamics. These hidden errors will only affect the accuracy if the scene or initial conditions drifts significantly away from the training distribution, which HORME targets. The relationship predictions are then fed into the physics, as act as the whole signal that the model must reconstruct the dynamics from. That sets it apart from prior JEPA-style work, where prediction is just an extra loss.

S2. TRD derivation and math is sound. Splitting the position change into avg.accel. and timing is exact and not an approximation, and the paper shows that the timing part a change without affecting the velocity, which isolates exactly what TRD is designed to learn. Lastly, although slightly system specific, the N-bopdy momentum and CoM guarantees hold regardless of the network learning.

S3. HORME shows the largest gains at situations with the hardest generalization gap. The N-body Cross 3T test uses 8–10 bodies instead of the 2–3 seen in training, wider mass and radius ranges, and rollouts three times longer than training. There, error drops by about 72% relative to NFF, and correlation rises from 0.80 to 0.96. This demonstrates that HORME performs best when there is significant gap between training and testing dsitbruitions.

Critical weaknesses

W1. My biggest gripe is the evaluation protocol of the paper. Firstly the baseline results use those reported by the baseline paper, while HORME is trained under a new loss with an extra N-body loss term, and the checkpoint is selected from validation data. Furthermore HORME is also significantly larger than baselines. A question that I have for the authors is that with the two new contributions turned off HORME is weaker than baselines, what happens if the two components are stitched to for instance NFF? Lastly the baselines are mossing some methods that the authors already discuss in the paper - Hypersolvers and IGNS. Lastly the error bars seem to indicate variance across test trajectories? This is not how they are typically reported and seems a little disingenuous.

W2. The qualitative analysis of the paper leaves a little to be desired. RA-JEPA's central thesis is that wrong pair-wise interactions can cause degradation in performance, however pair-wise correctness is never measured or reported. For systems such as the N-body where pair-wise force is well known it would be relatively easy to resolve this.

Questions for authors

Q1. For I-PHYRE dataset, why does the conditioning change between benchmarks, can the authors elaborate on how this was selected and how they affect model performance please?

Q2. Table values are reported as being taken from NFF, so can the authors clarify if the rollout trajectory were from retrained checkpoints released by the papers or were these from retrained models? If the models were retrained can the authors provide the corresponding metrics?

Q3. Seed to seed variance will show training stability and consistency in the results.

Minor suggesiton: I think the paper can be made much clearer is the notation did not rely so much in scripts. Right now it is a handful to read.

My original score is a 2, simply because for some freaking reason the PCs decided to use the WORST review scale of all time (1-4), but this paper is extremely borderline, so I hope the authors are not discouraged by the score and the score can be easily made a 3. Best of luck.

Reviewer_Gr7SOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a new architecture for learning relational simulator for multi-object dynamics. The proposed model called HORME combines RA-JEPA, which recursively forecasts how the simulator’s pair features evolve and feeds these forecasts back into the force computation, with TRD, a learned correction for intra-step acceleration timing. On the I-PHYRE and N-body benchmarks from NFF, the authors claim that HORME achieves the lowest trajectory RMSE than baselines.

Critical strengths

None.

Critical weaknesses

The paper is hard to follow even for readers with relevant expertise and would benefit from substantial rewriting. In particular:

The paper contains many poorly defined objects and insufficiently justified statements. A few examples: l.014 "Motion computation" and "Relational evolution" are not defined. l.018 "trajectory errors train the predictor". This needs to be rephrased.l. 048: "the graph of pair features used for physical computation", what do we mean by "physical computation"? l. 052: "Each forecast is matched to changes between encodings of observed states, with errors scored pair even when the combined motion is correct." What "forecasts" are we talking about? What does "match" precisely mean here? What is the "combined motion"?

The main contributions are very imprecisely described: l.067. "We introduce HORME, which trains a shared graph of pair features for motion computation and prediction of relational evolution using observed trajectories" This is how the model is making its predictions, but are the authors stating this is a new architecture, or a new combination of training objectives? l.069. Same issue. l.072. What is meant by "preserving physical pair constraints and the current step's velocity update"? Is the novelty that no other model other model in the literature is capable of that? (Whatever "that" means?), or is the model outperforming some well-established baseline l.069. What does "unseen configuration" mean? Trajectories with different initial state? With more/less objects?

Some findings are counter-intuitive. Here are two major conceptual problems:

The introduction argues that trajectory loss leaves individual interactions ambiguous (see Fig. 1 bottom left plot), which the authors use to claim that supervising on pair-feature evolution will help with. However, the additional targets to train a JEPA style network on pair-features are the encoder’s features computed on observed future states. Any information in the targets is a function of the trajectories. So if two force decompositions produce identical trajectories on all training data, they will be consistent with the pair-feature targets too. So, although this adds an additional training signal that could at best act as regularizing the training, this cannot resolve the ambiguity the authors use as motivation. It is somewhat surprising that learning pair-feature encoders with the additional JEPA loss (Table 4, row 1) is actually worse than the base simulator without RA-JEPA (Table 3, row 2). If the pair-features encoddings were improved by predictive supervision, as the introduction argues, this auxiliary loss alone should help. The authors should comment why this is not the case.

Questions for authors

Q1. In the case of the N-body pairwise gravitational forces, did the authors try to compare the learned per-pair acceleration contributions to the true analytic forces?

Q2. Several ablations (e.g., Table 4) share the same architecture and differ only in the training objective. Did the authors train these variants over several random seeds, ideally the same seeds across variants, to check that the reported differences are stable?

Reviewer_vqhqOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes HORME, a learned physics simulator for object interactions. The main concern is that trajectory loss supervises only the net force. Individual pairwise forces may therefore be incorrect. HORME introduces two components. RA-JEPA predicts changes in pairwise features and uses these predictions to condition force computation. TRD learns a correction to each position update. HORME achieves lower RMSE than the baselines on I-PHYRE and N-body. The gains are especially large with more objects and longer prediction horizons.

Critical strengths
  • HORME achieves state-of-the-art RMSE on both benchmarks, with substantial gains on N-body.
Critical weaknesses
  • The paper does not clearly distinguish HORME from existing methods. It should explain the limitations of existing methods and why RA-JEPA and TRD address them.
  • Concurrent work, including JEPA-x and Semigroup-JEPA, also uses JEPA to predict physical dynamics. Semigroup-JEPA’s autoregressive latent rollout loss resembles HORME’s recursive loss. The paper does not clearly explain how RA-JEPA differs from these methods.
  • TRD decomposes displacement into contributions from mean acceleration and acceleration timing. This decomposition is an identity derived from the equations of motion. The added learned term is a residual correction applied only to position. Prior work such as Hypersolvers already learns corrections to numerical integration errors.
  • The paper presents technical details before explaining the overall idea. It also uses terms without adequate definitions. This makes the paper very difficult to follow.
Questions for authors

Q1. The ablations only remove or modify RA-JEPA. How does RA-JEPA compare with auxiliary tasks that directly predict relative positions or velocities? These quantities can be obtained from the trajectories. Without this comparison, the need for prediction in feature space is unclear.
Q2. The supplement shows that HORME has considerably more parameters than NFF. How does HORME perform with the same number of parameters as NFF?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.