acceptodds
Under review as a conference paper at ICLR 2027

Back to the Future: Reverse-Consequence Modeling for Long-Horizon Particle Dynamics

Abstract

Accurate prediction of object trajectories is fundamental to mechanical understanding and robotic control. However, autoregressive models inherently suffer from cumulative errors, leading to severe drift in long-horizon rollouts. Observing that in many physical systems with dissipative or convergence-to-rest behavior, the final states are often more predictable than transient dynamics (e.g., a sliding object resting under friction or a pendulum stabilizing at equilibrium), we propose ReStep, a cascaded approach that reverses the conventional forward-prediction paradigm. ReStep first employs a stochastic model to directly predict the final state, then leverages Anchored Diffusion to generate goal-conditioned intermediate trajectories. By conditioning on both ”boundary states”, it effectively constrains dynamics and reduces uncertainty, producing physically plausible and temporally coherent transitions. We further integrate ReStep dynamics as the physical engine within a 3D Gaussian Splatting pipeline, enabling a unified physical-visual framework for intuitive inference from images. Extensive experiments on diverse rigid and non-rigid collision datasets show that ReStep achieves superior long-term accuracy, substantially mitigating error accumulation via reverse-consequence modeling.

Primary area:
learning on time series and dynamical systems
Keywords:
Deep learning advancements, AI/ML for physical sciences

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pqyYOpenReview ↗
Rating
2
Confidence
4(+2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies long-horizon physical dynamics prediction under particle representations, with a focus on exploiting the final-state structure of dissipative or settling systems. ReStep first uses a conditional VAE composed of a GNN encoder and decoder to predict possible final states from the initial state; it then generates the full trajectory with a Double-UNet diffusion model conditioned on both the initial state and the predicted final state. Its “reverse-consequence” formulation should be understood as a conditional decomposition that predicts the outcome first and then generates the process, rather than reverse integration along physical time. The authors further connect particle prediction to 3D Gaussian Splatting for visual dynamics generation and physical parameter estimation based on initial and final multi-view observations. Experiments cover FlexiFall and Physion, and include component ablations, final-state conditioning replacement, comparisons of final-state predictors, computational overhead, and failure case analysis.

Critical strengths
  1. The main strength of this paper is its approach to long-horizon dynamics prediction. For physical systems that eventually settle into stable configurations, predicting the final state first and using it to guide trajectory generation is a well-motivated idea. This provides a useful way to incorporate long-term constraints into the prediction process and reduces the reliance on sequential rollouts. The combination of stochastic endpoint prediction and anchored diffusion is particularly interesting for physical systems where multiple outcomes are possible.

  2. The experimental evaluation is also a strong aspect of the paper. The authors consider a range of rigid and deformable scenarios and provide ablation studies that help isolate the contributions of endpoint prediction and diffusion-based trajectory generation. I particularly appreciate the analysis of different endpoint conditions and the relationship between endpoint errors and trajectory errors, as these experiments offer useful insight into the behavior of the proposed framework. The integration with 3D Gaussian Splatting further demonstrates the potential of the approach for visually grounded physical reasoning.

Critical weaknesses
  1. Some important implementation details are missing from the method description. The paper treats physical properties such as friction, restitution, and mass as inputs to the steady-state predictor and uses this predictor for parameter estimation. However, the architecture in Table 5 only specifies position and velocity features, leaving it unclear how these physical parameters enter the network and how gradients with respect to them are computed. The diffusion model also applies convolutions along the particle dimension without explaining how particles are ordered. Since these operations can depend on particle ordering, this raises questions about the model's behavior across different particle configurations and scene compositions.

  2. The experimental comparisons would also benefit from a more detailed description of the evaluation setup. In particular, the paper provides limited information about how the generative baselines are adapted to the 3D particle prediction task, including their conditioning inputs, architectural changes, and training configurations. There are also differences between the results reported in Table 2 and Table 8 for seemingly identical settings that need clarification. These details are important for understanding how much of the improvement comes from the proposed modeling approach. More broadly, since endpoint-conditioned trajectory generation has been explored in prior work, the paper could better distinguish its specific technical contributions from existing approaches.

Questions for authors

Could the authors clarify how physical parameters such as friction, restitution, and mass are incorporated into the steady-state predictor? Table 5 only lists position and velocity as input features, so it is unclear how these parameters influence the prediction and how gradients are computed during inverse parameter estimation. I would also like to know how the stochastic latent variable is handled during this optimization.

How are particles ordered in the second UNet, which applies convolutions along the particle dimension? Does the model rely on a fixed ordering, and how is this ordering maintained across scenes with different numbers of objects or particles? Have the authors examined whether consistently permuting the particle indices affects the predictions?

Could the authors provide more details on how LatentFlow and STFlow are adapted to the 3D particle prediction task, particularly regarding their conditioning inputs, architectural modifications, and training configurations? I also noticed some differences between the results in Table 2 and Table 8 for apparently identical settings. Could the authors explain the source of these differences and clarify whether the reported standard deviations are computed across test samples, random samples, or independent training runs?

Finally, how is the final state sTs_T defined across the different physical scenarios? Is it determined by a convergence criterion or simply taken as the last frame of each sequence? Could the authors also clarify the training horizons used for the dynamics model and the rendering experiments? In particular, it would be helpful to distinguish generalization to different temporal resolutions from extrapolation to longer physical durations.

7 comments
AuthorsOpenReview ↗

Response to pqyY (1/3)

W1 / Q1. Physical parameters in the steady-state predictor

The reviewer is right that Table 5 is incomplete. We have added the details below.

How the parameters enter. Flexifall (Fig. 3) : position, friction, restitution and mass of each object are the diferences between scenes. The input is "position + (friction, restitution, mass)" for the steady-state predictor. Physion (Table 2): position and initial motion, the official data contain no friction, restitution or mass. So the input is "position + initial velocity (Table 5)".

For Fig 3 and Table 3 experiments on Flexifall in which friction, restitution and mass vary per object. The network has the architecture of Table 5 with a parameter branch, MLP(3 → 128 → 128) on (friction, restitution, mass). Its output is concatenated with the position embedding before the three message-passing blocks, so the parameters influence every message and, through the pooled feature h₀, the prior and the decoder. We corrected the table 5 in the revised manuscript.

How the gradients are computed. The network is frozen. The only optimisation variables are the per-object parameters α ∈ ℝ^{M×3} (M objects). They are broadcast to the particles, the predictor is evaluated once, and the gradient of L_inv = ‖ŝ_T(α) − s_T‖² + 10⁻⁶‖α‖² is obtained by ordinary back-propagation through decoder, latent predictor, prior and encoder. No simulator roll-out is differentiated: one step of the optimiser costs one forward and one backward pass of a single network. We use Adam with learning rate 10⁻⁴ for 1000 steps.

The latent variable. Nothing is sampled during the optimisation. The latent is set to the prior mean, z = μ_p(h₀). Since h₀ depends on α, the gradient also flows through the prior. The objective is therefore a deterministic function of α.

W1 / Q2. Particle ordering in the second UNet

Yes, the second UNet relies on a fixed particle order. A 1-D convolution along the particle index is not permutation-equivariant, and we do not claim that it is.

Which order, and how it is maintained. It is the canonical order of the particle representation, with no per-scene choice. Objects are concatenated in the simulator's object order. Within an object, the particles keep the order of its canonical point set, which is defined in the object's local frame and is therefore identical in every frame.

Experiment. We took the Table 2 checkpoints, without retraining, and applied one permutation consistently to the initial state, the predicted endpoint and the ground truth.

Whole-trajectory MSE, mean over scenesDominoesRollCollideDrop
Dataset order0.00390.0800.1450.0151
Particles shuffled within each object0.01840.0800.1550.0397
Objects concatenated in a different order0.01350.2130.1980.0364

The accuracy of Stage 1 is unchanged. Stage 2 depends on the order, and the amount varies. Changing the order of the objects raises the error. So the model does use the order, and a different order at test time than in training degrades it, as it would for any convolutional model.

Our position. We view this as a practical architectural trade-off. Our denoiser operates on an object-wise serialization with persistent particle identities, allowing the same convolutional weights to process inputs with different particle counts. This supports variable-size systems under a consistent ordering convention, but does not imply permutation equivariance.

Using 1D convolutions along the particle dimension has a strong engineering rationale in physical-simulation diffusion models. The denoiser repeatedly processes trajectories of TT timesteps over LL denoising iterations. Fully connected GNN message passing among all PP particles at every iteration incurs quadratic computational and memory costs when pairwise messages are explicitly constructed. Fixed-width, fixed-kernel 1D convolutions instead scale linearly with PP, making repeated processing of large particle trajectories tractable. This is particularly relevant to engineering simulation and graphics pipelines, where persistent particle identities and fixed or alignable ordering are common.

Regarding object ordering, random test-time shuffling is an artificial stress test that disrupts the representation convention. Such a mismatch need not arise in a consistent simulation pipeline: the initial physical state supports deterministic spatial or kinematic sorting. Applying the same rule during training and inference, with deterministic tie-breaking as needed, maps different input orderings of the same scene to a common canonical object sequence. Under this protocol, arbitrary object indices do not cause ordering mismatches. Thus, sensitivity to deliberate shuffling does not undermine practical applicability when consistent serialization is maintained.

AuthorsOpenReview ↗

Response to pqyY (2/3)

W2 . Relation to endpoint-conditioned generation

We agree that endpoint-conditioned generation exists. Sec. 2 cites it (Janner et al., 2022; Huang et al., 2023) as the conditioning mechanism we build on. The difference lies in where the endpoint comes from.

In prior work the endpoint is an input. In planning it is the goal, in motion or video in-betweening it is a key frame, in smoothing it is a later observation. The task is to interpolate between two known states. In simulation the endpoint is the unknown. Only s₀ is given, and every existing learned particle simulator we know of, autoregressive (GNS, GMN, SGNN, EGNO) or generative (LatentFlow, STFlow), generates from s₀ alone. Nevertheless, in contrast to previous fields, we investigate long-horizon and larger particle dynamics conditioned on initial particle states and physical attributes, with substantially longer and varying prediction horizons ranging from 70 to 452 steps, also larger particle numbers.

Our specific contributions are:

  1. The formulation. We recast forward simulation of dissipative systems as outcome prediction followed by boundary-value completion, p(s_T | s₀) · p(s_{1:T−1} | s₀, s_T). It rests on an observation that had not been used in learned simulators: the settled state is easier to predict in one shot than the transient.
  2. A stochastic steady-state predictor for particle systems. It maps (s₀, α) to a distribution over rest states in one step, with calibration reported in Table 7 and backbone independence in Table 9.
  3. Scalability. A trajectory denoiser for variable-size systems. The Double-UNet is convolutional along time and along the particle index, so one set of weights handles varying frame sequences and particles. Standard goal-conditioned diffusion models assume a fixed state dimension.
  4. Use as the dynamics module of a 3DGS pipeline (Sec. 4.3).

W2 / Q3. Generative baselines and standard deviations

We preserve LatentFlow’s generative formulation. The original LatentFlow operates on a regular image grid: conditioned on an RGB image, it generates 2D trajectories for image-grid points over TT frames. So our 3D particle trajectories [B,T,N,6][B,T,N,6] cannot be passed directly into the original model. For our 3D particle data, we embed the [B,T,N,6][B,T,N,6] trajectories particle-wise and add time embeddings. Masked averaging over particles and time yields a single [B,256][B,256] latent. Conditioned on the initial state, the flow generates this latent, which is then decoded into a full [B,T,N,6][B,T,N,6] trajectory. Thus, our adaptation retains the trajectory VAE → latent flow matching → trajectory decoding framework.

We preserve STFlow’s core methodological pipeline. Starting from the initial state s0s_0, our implementation constructs random-walk trajectories. The vector field takes the current trajectory, the initial positions and physical attributes as input. It performs spatial message passing, followed by temporal convolutions. During training, conditional flow matching learns a flow from the random-walk trajectories to the ground-truth trajectories; at inference, ODE integration generates the predicted trajectory.

You can check the table in Appendix E, Table 15, for details:

LatentFlow (Table 2, Physion)STFlow (Fig. 3, FlexiFall)
Modeltrajectory VAE (per-particle encoder and decoder with learned time embedding; pooled 256-d latent) and flow matching in the latent space (vector field conditioned on the pooled initial state)flow matching in trajectory space, starting from a random-walk prior anchored at s₀ (σ = 0.01); vector field of k-NN message-passing blocks (k = 8) and 4 dilated temporal convolution blocks, width 128
Datasame as ReStepsame as ReStep
TrainingAdamW, lr 10⁻⁴, weight decay 10⁻⁴, gradient clipping 1.0, 100 epochs, batch 2; loss = reconstruction + 10⁻⁴·KL + flow matchingAdamW, lr 10⁻⁴, weight decay 10⁻⁴, gradient clipping 1.0, 100 epochs
Test setsame as ReStepsame as ReStep

How much of the improvement comes from our modeling approach? In our comparisons, the methods are evaluated on the same task-specific datasets under the same training setup, with the baselines and ReStep using models in the single-digit millions of parameters. Despite their different architectures, ReStep achieves the best results. The comparisons assess the effectiveness of the complete two-stage approach. In our ablation, we provided a more direct measure of the endpoint’s contribution within ReStep.

Clarify std. The reported standard deviations are computed across test scenes, not across repeated stochastic samples for the same scene or independent training runs. Additional runs with independent training seeds are underway; we will supplement the results once they finish.

AuthorsOpenReview ↗

Response to pqyY (3/3)

Q4. Definition of s_T and horizons

Definition of s_T. It differs between the two benchmarks.

  • Physion: s_T is the rest state. Each trajectory is cut at the first frame after frame 50 at which the frame-to-frame displacement of the whole particle set falls below 10⁻⁴ (in norm).
  • FlexiFall: s_T is the last frame of a fixed-length window. All scenes have settled by then.

Training horizons of the dynamics model. Both stages are trained on the full sequences defined above. On Physion a single model per scenario is trained on all sequence lengths at once.

Rendering experiments (Table 4). For each test scene, the Gaussian representation of both methods is fitted to the images of the first fixed number of frames only. "In-domain" are these frames; "out-of-domain" are the remaining frames of the same sequence, for which no image is ever seen. Temporal resolution versus physical duration. In the Dominoes temporal-resolution experiment, we insert one linearly interpolated frame between every pair of original frames. This changes sampled trajectory frames and the sampling interval, while leaving the physical duration unchanged. Like we said in the paper, we demonstrate the Anchored Diffusion process learns a robust, continuous mapping of physical evolution.

Reviewer_ocUdOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

What happens when you push over a pile of blocks? These are the types of simulations addressed by this paper. Small perturbations within the simulation can give very different answers, and feedforward, autoregressive models can often get things wrong. The authors' proposed solution: use a stochastic VAE to jump from the initial state to a possible final state. Then work forwards and backwards between the initial and final states to estimate the trajectories. They show that this gives better estimates for the trajectories.

Critical strengths

The paper is clear and easy-to-read.

Critical weaknesses

The teaser figure, figure 1, seems to present a result that isn't the main point. "The right-hand bar chart compares the Final State MSE of our method with that of autoregressive methods". Yes, but the final state from your method is just derived from a stochastic VAE (Kingma and Welling, 2013), not from the essence of your algorithm, the working both backwards and forwards in time with a diffusion model. Why tout the final state MSE in this figure, since that's not what your algorithm adds?

Please give me your comments on what I feel is another weakness: the idea of working both forwards and backwards in time is certainly not new, neither in recursive estimation, nor in recent generative AI video models. You're applying that idea in a different level of representation (particle-based object dynamics, instead of videos, or other, more general, state estimation tasks), but your results are not a surprise.

No seeds or error bars are given in Table 2 or Fig. 3, and FlexiFall results appear only as plots (L309–323). Endpoint errors have standard deviations several times their means (Table 9, Collide 0.31 ± 1.22). The same quantity disagrees across tables: Roll at t=100 is 0.091 in Table 2 but 0.0753 in Table 8.

Questions for authors

Please list the number of diffusion steps used.

2 comments

W1. What the bar chart in Fig. 1 shows

The original bars reported the complete pipeline’s error (endpoint prediction followed by anchored diffusion), not the VAE’s output, at the last evaluated frame—the right-most points in Fig. 3. For RigidFall at t = 100, this was 0.048 for ReStep versus 0.092 / 0.164 / 0.288 for SGNN / EGNN / GMN. We have replaced them with mean trajectory MSE over the evaluated prediction horizon, averaging the plotted, scene-averaged MSE values in Fig. 3.

W2. Novelty

In the prior work, the future end is given. In simulation it is the unknown. Forward–backward smoothing (Kalman/RTS, HMM) conditions on observations over the whole interval. Goal-conditioned diffusion (Janner et al., 2022, which we cite in Sec. 2 as our conditioning mechanism) is given the goal. Video in-betweening (e.g. Wang et al., 2024, "Generative Inbetweening") is given the last key frame. All of these are interpolation problems. In forward simulation nothing about the future is available.

Therefore, what we propose is not the established idea of “working both forwards and backwards in time.” Our claim is: in dissipative systems the far future, is easier to predict in one shot than the transient that leads to it. Predicting it first turns extrapolation, whose error grows with the horizon, into a boundary-value problem. To our knowledge no learned particle simulator does this: GNS, DPI-Net, GMN, SGNN, EGNO and the recent generative simulators (LatentFlow, STFlow) all generate from the initial state alone, and the STFlow paper lists endpoint-conditioned simulation among the tasks it leaves for future work.

Nevertheless, in contrast to previous fields, we investigate long-horizon and larger particle dynamics conditioned on initial particle states and physical attributes, with substantially longer and varying prediction horizons ranging from 70 to 452 steps, also larger particle numbers. We have scalability. The Double-UNet is convolutional along time and along the particle index, so one set of weights handles the varying frame sequences and particles. Standard goal-conditioned diffusion models assume a fixed state dimension.

Moreover, our experimental results show that the architecture we propose clearly outperforms traditional one-step prediction methods and existing trajectory generation methods conditioned only on the initial state, thereby demonstrating the superiority of our architecture.

W3. Error bars, and consistency across tables

1. Fig. 3 and FlexiFall quantitative results. Fig. 3 does show variability: the shaded band around each curve is ±1 standard deviation across test scenes. For exact quantitative results, we add the values (mean ± std across test scenes) in the revised manuscript, Appendix D.7.

2. Table 2 with error bars. Table 2 and Figure 3 now report mean ± standard deviation of per-scene MSE across all test scenes in each scenario, rather than across independent training runs. Additional runs with independent training seeds are underway; we will supplement the results once they finish.

3. Why the standard deviation exceeds the mean in Table 9. In the original submission, the ± values in Tables 9 and 13 denote the inter-scene standard deviation across the 150 test scenes, capturing scene-to-scene physical heterogeneity rather than the uncertainty of the mean estimate.

To prevent misinterpretation while remaining fully transparent about inter-scene variance, we have updated Tables 9 and 13 in the revised manuscript to report both the inter-scene mean ± standard error of the mean (s.e.m.) (e.g., 0.31 ± 0.10 for Collide).

Also, 'MSE' is non-negative and quadratic in distance, its distribution over scenes is strongly right-skewed, and a standard deviation above the mean is the expected signature of such a distribution rather than a sign of an unstable predictor. For Collide with the GNN predictor:

nmeanmedian25th–75th pct.95% CI of mean
1500.3130.0850.044–0.184[0.149, 0.539]

86% of the scenes lie below the mean. Three of the 150 scenes account for 53% of the total error. Without three specific scenes the result is 0.150 ± 0.190 (mean ± std).

4. Roll at t=100: 0.091 (Table 2) vs. 0.0753 (Table 8). They come from runs of the same Stage-1 and Stage-2 checkpoints. Table 2 reports the main comparison using the full test set, whereas the separate endpoint-conditioning ablation Table 8 comes from randomly selection set from the full set. We have corrected Table 8 in the revised manuscript by aligning the test scenes.

Q1. Diffusion steps

256 denoising steps for every model in the paper, with a cosine noise schedule and clean-sample (x₀) prediction. Training draws the step uniformly from these 256, and inference runs all 256 ancestral sampling steps without step skipping. The per-step costs in Table 10 are measured with this setting. We have added this to Appendix C.2.

We have finished training over different seeds on two scenes.

W3. Training seeds uncertainty. To assess sensitivity to the training seed, we trained Stage 1&2 with seeds 42, 43, and 44. For each seed, we averaged the trajectory MSE over the same test scenes. The mean ± sample standard deviation across the three runs was 0.14144 ± 0.00530 for Collide and 0.08556 ± 0.00233 for Roll. The observed variation was small relative to the mean in both categories (3.8% and 2.7%, respectively).

Sceneseed42seed43seed44Mean ± SD
Collide0.141350.136180.146790.14144 ± 0.00530
Roll0.085700.083170.087820.08556 ± 0.00233
Reviewer_PNriOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes ReStep, a neural simulation method that aims to reduce error accumulation in long-horizon autoregressive dynamics prediction. Instead of predicting motion frame by frame, the method first uses a separate network to predict the final state, then generates the full trajectory using a diffusion model conditioned on the initial and predicted final states. Experiments on simulated physical interactions show promising results, outperforming several existing baselines in long-horizon prediction accuracy.

Critical strengths
  1. Predicting the final state before generating the trajectory is a neat approach, especially for physical systems with predictable resting states. While similar ideas have been explored in other domains, such as GooDFlight in aircaft trajectory estimation, applying this formulation to particle-based physical simulation is interesting.

  2. The method achieves promising results across multiple simulated physical scenarios, with substantial improvements over existing baselines, particularly for long-horizon prediction.

Critical weaknesses

The method relies on the assumption that the final state is easier to predict than the intermediate trajectory. While this is reasonable for systems with clear resting states, it may not hold for more complex physical interactions where the outcome is highly uncertain. In such cases, errors in the final-state predictor could constrain the entire generated trajectory and lead to inaccurate simulations. I am concerned about how well this approach generalizes beyond scenarios with easily predictable outcomes.

Questions for authors

How do the authors envision handling scenarios where the final state is difficult to predict or highly uncertain? Could a hybrid approach combining endpoint-conditioned generation with autoregressive prediction or self-correction help address this limitation? It would also be useful to understand when endpoint anchoring is preferable to conventional forward simulation.

1 comment

Thank you for raising this limitation. We agree that an inaccurate endpoint prediction can bias the generated trajectory, and we do not expect endpoint anchoring to be preferable for every physical system. ReStep currently targets settling systems, where a stable terminal state provides a useful constraint. Even in relatively sensitive scenarios such as RigidFall and Collide, this constraint is beneficial in our experiments. This does not establish that the same advantage holds when the outcome itself is highly uncertain.

For systems without an easily predictable terminal state, we see several promising extensions: sampling multiple plausible endpoints rather than committing to one, using identifiable intermediate events (such as first contact or maximum compression) as anchors, and combining anchored generation with forward prediction or self-correction when an endpoint is unreliable. These are directions for future work, not capabilities evaluated in the present paper. We expect endpoint anchoring to be most useful when the outcome is comparatively well constrained but the path to it is difficult to predict; conventional forward simulation may be preferable when no reliable endpoint or intermediate anchor can be identified.

Comments on the paper

1 comment

Dear reviewers,

Since the discussion phase is ongoing and will end in less than one day, we kindly request your feedback on our responses and any remaining questions you might have, before reaching conclusions. We also appreciate your invaluable suggestions on how to improve our paper during revision!

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.