acceptodds
Under review as a conference paper at ICLR 2027

PNEO: Pairwise Neural Energy Operators for Scalable Collision Dynamics of 3D Deformable Objects

20 views

Abstract

Accurate prediction of 3D deformable-object collision dynamics is important for robotic interaction, world model, and a broad range of engineering simulation problems. Numerical methods are accurate but become increasingly inefficient as the number of interacting objects grows. Learning-based simulators predict faster, but typically require training on large multi-object scenes to scale up, limiting generalisation to unseen object counts. We introduce the Pairwise Neural Energy Operator (PNEO), which learns collision dynamics from two-object interactions and composes the same learned interaction law across multi-object interactions. PNEO models the conservative energy exchange and dissipation of each contacting object pair using scalar potential and Rayleigh dissipation terms, which are summed over the contact graph to recover multi-object dynamics. This pair-local formulation makes the learned interaction law independent of scene cardinality. Experiments show that a single PNEO trained only on two-object collisions transfers zero-shot across object count (up to 50), point-cloud resolution, and mechanical properties without retraining or fine-tuning. PNEO outperforms baselines across the main multi-object evaluation settings while maintaining stable trajectory, contact, and deformation accuracy as scene size increases.

Primary area:
learning on time series and dynamical systems
Keywords:
Graphic-based nueral dynamics, contact dynamics, deformable object dynamics, neural operator, zero-shot generalization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_mTzuOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents PNEO, a learned simulator for collision dynamics among 3D deformable objects. PNEO is trained only on two-object collisions and applies a shared interaction model to every active contact pair at inference. An object-local neural operator encodes particle and object mechanics, after which a pairwise head predicts positive contact-stiffness and damping coefficients. These coefficients parameterize a quadratic contact potential and a Rayleigh dissipation term; the pairwise scalar terms are summed over the dynamic contact graph, differentiated into forces, and integrated with semi-explicit Euler. On a synthetic spring-mass benchmark, PNEO generally outperforms several graph-based simulators and transfers without retraining to scenes containing up to 50 objects, unseen stiffness values, and different point-cloud resolutions.

Critical strengths
  1. Well-motivated compositional inductive bias. Reusing a shared pairwise interaction law over a changing contact graph is a natural way to separate scene cardinality from local contact mechanics. The zero-shot object-count experiments provide substantial evidence that this inductive bias improves transfer beyond the two-object training setting.

  2. Broad evaluation of simulated mechanical fidelity. The paper evaluates position, penetration, strain, and contact-aligned rollout errors, and tests object-count, stiffness, point-resolution, and initial-condition shifts. In particular, the transfer from two-object training to scenes with up to 50 objects is an interesting empirical result.

Critical weaknesses
  1. The implementation does not justify the claimed conservative neural-energy interpretation. The predicted contact stiffness is conditioned on both positions and velocities, including the normal relative velocity, but the coefficient is frozen when differentiating the quadratic potential. Consequently, the implemented force is not the full gradient of the stated state-dependent scalar energy, and a velocity-dependent stiffness does not define a conventional conservative potential. Re-predicting detached coefficients at every substep can also change the effective energy without an explicit conservation law. The paper reports no energy-balance, momentum, or dissipation measurements that would validate the claimed conservative energy exchange. The method can still be viewed as a structured pairwise force model, but its central energy-based interpretation is not established.

  2. The benchmark strongly matches the hand-specified contact law, while key controls and scalability measurements are missing. The simulator uses frictionless pairwise penalty contact with quadratic penetration and normal damping, which closely matches PNEO's prescribed potential and dissipation forms. It is therefore unclear how much of the large advantage comes from learning rather than from encoding the data generator's contact law. The paper does not compare with an otherwise identical direct pairwise force predictor, fixed or analytic contact coefficients, or the closely related energy-based NEXUS method. It also motivates the work through computational scalability but reports no runtime, memory, training cost, or scaling curve. The absence of multiple-seed uncertainty further weakens the empirical comparison.

Questions for authors
  1. Since the predicted stiffness depends on velocity and is detached during differentiation, in what precise mathematical sense is the resulting contact force conservative? Can the authors provide total-energy and dissipation measurements over rollouts?

  2. How does PNEO compare with an architecture-matched pairwise model that directly predicts forces, and with fixed or oracle coefficients from the analytic penalty contact law? This comparison is necessary to isolate the value of the neural-energy parameterization.

  3. What are the wall-clock runtime, peak memory, parameter count, and scaling behavior as the object count and point-cloud resolution increase? How does this compare with the reported baselines and NEXUS?

4 comments

W1 & Q1. Energy interpretation, velocity conditioning, energy balance and momentum

We agree that the derived force does not strictly correspond to the full gradient of the state-dependent potential, whose coefficients depend on both positions and velocities. However, we adopt this formulation as an engineering choice rather than a strict implementation of conservative dynamics, and provide additional ablation results to assess its empirical benefits. We will revise the manuscript to clarify this distinction and qualify the corresponding conservative-energy claims.

Including velocity information allows the encoder to capture richer collision-state information. To assess its contribution, we compare PNEO with a position-only variant that excludes velocity from the potential branch while retaining it in the dissipation branch. Both variants use the same architecture and coefficient-freezing convention and are trained at (M=2). We evaluate them at (M=2,3,5), with each table entry reporting RE@24 / RE@48. Velocity-conditioned PNEO achieves lower errors at both horizons across all three object counts in this comparison, providing empirical support for this architectural choice.

Test MPNEOPosition-only
20.0633 / 0.18920.0674 / 0.1909
30.0800 / 0.21350.0813 / 0.2158
50.0913 / 0.22360.0946 / 0.2296

To quantify the energy changes caused by coefficient updates, we explicitly measure the energy balance:

ET−E0=Wext−D+Jc+Rnum,E_T-E_0=W_{\mathrm{ext}}-D+J_c+R_{\mathrm{num}},

where EE is total mechanical energy (kinetic, gravitational, structural and contact energies), E0=E(0)E_0=E(0), DD is cumulative damping dissipation, and WextW_{\mathrm{ext}} is external work. JcJ_c accumulates stored-energy changes caused by coefficient updates at fixed positions and velocities, where positive values indicate energy injection. RnumR_{\mathrm{num}} denotes the numerical energy-balance residual.

Table R1. Energy balance. Values are trajectory means normalized by E0E_0. The physical reference uses the fixed-coefficient simulator.

HMethodE/E0E/E_0D/E0D/E_0Wext/E0W_{\mathrm{ext}}/E_0Jc/E0J_c/E_0Rnum/E0R_{\mathrm{num}}/E_0
24Ground truth0.5310.4780.0100.000-0.002
24PNEO0.4920.5260.011+0.015-0.007
48Ground truth0.3320.6760.0100.000-0.002
48PNEO0.3170.6970.011+0.012-0.009

Energy balance. As shown in Table R1, coefficient updates add energy equivalent to 1.2% of E0E_0 on average by the final state, while the numerical balance residual is −0.9% of E0E_0. This explicitly quantifies the energy change caused by coefficient updates without assuming exact conservation.

Table R2. Momentum and collision impulses. Values are trajectory means normalized by Pref=2mtotE0P_{\mathrm{ref}}=\sqrt{2m_{\mathrm{tot}}E_0}, where mtotm_{\mathrm{tot}} is the total dynamic mass. This normalization makes the measurements dimensionless and enables comparison across trajectories with different total masses and initial energies. Negative vertical momentum indicates downward motion.

HMethodHorizontal momentum magnitudeVertical momentum PyP_yGround impulse Jfloor,yJ_{\mathrm{floor},y}Net inter-object contact impulseMomentum-balance residual
24Ground truth0.014-0.032+1.3414.1×10−104.1\times10^{-10}5.9×10−85.9\times10^{-8}
24PNEO0.014-0.030+1.3439.1×10−109.1\times10^{-10}3.8×10−83.8\times10^{-8}
48Ground truth0.014-0.037+2.5955.3×10−105.3\times10^{-10}6.5×10−86.5\times10^{-8}
48PNEO0.014-0.043+2.5891.2×10−91.2\times10^{-9}4.0×10−84.0\times10^{-8}

Momentum balance. We additionally check whether momentum changes match the applied external impulses. Table R2 reports trajectory means normalized for total mass and initial energy, showing near-zero net inter-object impulses and balance residuals. The maximum normalized residual across all trajectories and frames is 1.4×10−71.4\times10^{-7}.

W2 (1). Benchmark specificity and the prescribed contact law

To address the concern that our results are specific to the prescribed contact law in our deformable benchmark, we additionally evaluate a rigid-body adaptation of PNEO on MOVi-B, the benchmark used by RigidFormer. We retain the pairwise contact-graph formulation and contact-module design: predicting contact coefficients, constructing pairwise energies, and differentiating them to obtain forces for rigid-body velocity and pose updates. The table below reports position RMSE in meters (lower is better).

Step sizeFrameRigidFormerPNEO-rigid
11000.10790.0426
12000.43570.2491
13000.94110.6082
14001.40610.9545
51000.0900.0378
52000.40680.2411
53000.88750.6016
54001.34710.9506
101000.07090.0309
102000.37430.2289
103000.82930.5838
104001.26320.9261

We evaluate both methods at rollout horizons of 100, 200, 300, and 400 frames, using step sizes of 1, 5, and 10. PNEO-rigid achieves lower position RMSE than RigidFormer at every tested horizon and step size. We also train RigidFormer on our two-object deformable dataset, where it obtains RE@24/48 of 0.48/1.19, compared with 0.0633/0.1892 for our PNEO.

W2 (2) & Q2. Direct-force, analytic oracle and NEXUS comparisons

We compare PNEO with direct force prediction and an analytic oracle. The oracle uses the same graph-construction procedure as PNEO at test time, then assigns each structural edge an elastic coefficient using the dataset-generation rule, based on the given object stiffness and the edge's rest length. Each cell reports RE@24 / RE@48.

Test MEnergyDirect forceAnalytic oracle
20.0634 / 0.18240.0684 / 0.17970.0691 / 0.1885
30.0791 / 0.20500.0830 / 0.20510.0761 / 0.2197
50.0897 / 0.22330.0953 / 0.22970.0800 / 0.2142

PNEO achieves rollout errors comparable to this analytic oracle. Compared with direct force prediction, it achieves lower RE@24 at every tested object count (M=2/3/5M=2/3/5) and lower RE@48 at M=3/5M=3/5; direct force is slightly better at M=2M=2 RE@48.

Comparison with NEXUS. We additionally compare PNEO with the closely related energy-based NEXUS method, both at its training object counts and on larger, unseen scenes. Tables R3 and R4 report RE@24 / RE@48:

Table R3. Accuracy against NEXUS trained at the corresponding object count.

Test MNEXUS training MNEXUS ↓PNEO ↓ (train M=2)
220.2124 / 0.43350.0633 / 0.1892
330.2579 / 0.52960.0800 / 0.2135
550.2135 / 0.41170.0913 / 0.2236

Table R4. Zero-shot accuracy on larger scenes.

MethodTraining MM=10 ↓M=20 ↓M=50 ↓
NEXUS20.1308 / 0.34640.1085 / 0.27560.0907 / 0.1395
NEXUS30.1357 / 0.37090.1141 / 0.27470.1024 / 0.1331
NEXUS50.1272 / 0.34540.1082 / 0.26770.0953 / 0.1283
PNEO20.0838 / 0.21590.0762 / 0.21870.0825 / 0.1268

PNEO achieves lower RE@24 and RE@48 in every comparison. Against two-object-trained NEXUS, it reduces RE@48 by 37.7%, 20.7%, and 9.1% at M=10/20/50, respectively.

W2 (3). Computational cost and scaling

For the computational cost, please see our response to Q3 for runtime, peak inference memory, parameter counts, and object-count and point-cloud resolution scaling. We additionally report PNEO training costs:

Training M235
Full training time (h)16.718.626.4
Peak training GPU memory (GB)0.681.313.29

Two-object training reduces training time by 36.7% and peak GPU memory by 79.3% relative to five-object training. The resulting model is reused at larger object counts without retraining.

Seed uncertainty. We additionally report PNEO results across three training seeds using the original training recipe, with each model trained only at M=2. Each cell below gives RE@24 / RE@48; the new results are reported as mean ± standard deviation.

Test MPaper numberThree-seed mean ± std
20.0633 / 0.18920.0634 ± 0.0015 / 0.1824 ± 0.0092
30.0800 / 0.21350.0833 ± 0.0014 / 0.2041 ± 0.0043
50.0913 / 0.22360.0956 ± 0.0046 / 0.2226 ± 0.0050
100.0838 / 0.21590.0857 ± 0.0013 / 0.2198 ± 0.0052
200.0762 / 0.21870.0757 ± 0.0023 / 0.2178 ± 0.0111
500.0825 / 0.12680.0844 ± 0.0090 / 0.1286 ± 0.0017

Across M=2–50, all new RE@24 and RE@48 means differ from the submitted values by less than 5%, supporting the repeatability of PNEO's accuracy and object-count generalisation.

W2 (3) & Q3. Runtime, peak memory, parameter count and scaling

Thank you for requesting these measurements. Tables R5 and R6 report rollout execution time, peak inference memory, and parameter counts on NVIDIA GB10. Here we report time (s/trajectory) / memory (MB).

Table R5. Object-count scaling and baseline comparison.

MethodParameters (M)M=2M=3M=5M=10M=20M=50
PNEO (optimized; train M=2)0.6760.120 / 1380.143 / 1390.218 / 1410.532 / 1481.772 / 1681.872 / 293
NEXUS (train M=2)0.3522.114 / 892.324 / 973.057 / 1146.010 / 15717.919 / 52598.545 / 2988
GNS1.5940.89 / 4901.38 / 6402.03 / 8622.24 / 1,2133.20 / 1,2404.00 / 1,834
MeshGraphNets3.8531.52 / 951.79 / 1192.34 / 1903.66 / 2616.61 / 46815.73 / 1,035
SGNN1.6590.34 / 820.47 / 1030.65 / 1131.06 / 2782.35 / 5312.81 / 301
EGNO0.2190.24 / 4440.44 / 8780.75 / 1,7021.69 / 4,2074.09 / 15,5382.35 / 2,500
TIE0.7690.33 / 490.47 / 650.99 / 1143.48 / 34512.26 / 1,25868.73 / 7,615
SBC0.6230.45 / 1160.60 / 1700.86 / 2411.82 / 4383.91 / 78723.81 / 1,085
DPI-Net3.37–7.490.40 / 1560.65 / 3260.87 / 291———
EqCollide0.3190.027 / 1600.040 / 1920.091 / 160———

PNEO achieves the lowest reported execution time and peak inference memory among the measured baselines at M=20/50. At M=20/50, the reported rollout speedups over NEXUS are 10.1× to 52.6×, while the peak-memory ratio (NEXUS/PNEO) increases from 3.1× to 10.2×.

Table R6. Point-cloud resolution scaling at fixed M=2.

Method641282565121,024
PNEO (optimized)0.100 / 1050.118 / 1060.167 / 1080.313 / 1120.656 / 119
NEXUS (train M=2)2.038 / 812.099 / 892.293 / 1072.378 / 1472.868 / 207
GNS0.50 / 2260.65 / 3391.50 / 7283.87 / 1,4479.42 / 2,718
MeshGraphNets1.09 / 681.35 / 901.73 / 1373.52 / 29510.57 / 922
SGNN0.15 / 400.29 / 660.49 / 1201.22 / 2542.88 / 469
EGNO0.06 / 940.19 / 2650.67 / 1,0312.52 / 4,38710.80 / 20,329
TIE0.16 / 390.28 / 490.62 / 872.36 / 2358.47 / 820
SBC0.33 / 600.43 / 910.70 / 1861.44 / 4413.29 / 839
DPI-Net0.27 / 640.34 / 1080.71 / 3292.07 / 1,3293.83 / 2,556

PNEO is faster than NEXUS across all tested resolutions. Its memory advantage is most pronounced at 1,024 points per object, where it achieves a 4.4× rollout speedup and uses 119 MB peak memory, compared with 207 MB for NEXUS.

Thank you for your response. I will raise my score to 3.

Reviewer_2xLTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a novel neural operator PNEO to model collision dynamics between two deformable objects, which once trained using simulation data, can be scaled to multiple objects. The method models the force exchanged between two colliding objects as the spatial derivative of an energy function, consisting of a conservative term U and a dissipative term R, both of which are outputs of the neural operator. The multi-object collision algorithm is then the composition of such pairwise force exchanges, depending on the collision graph. The neural operator architecture composes a learnable mechanical encoder with a learnable pairwise dynamics predictor. This architecture demonstrates superior particle trajectory reconstruction at different simulation lengths, different mechanical properties and different object counts.

Critical strengths
  1. PNEO outperforms all considered baselines, in the trajectory reconstruction error metric at different horizons (Tables 1 and 6), different mechanical properties (stiffness, Table 3), across different object counts (Table 2 and 7) when compared with simulated data, which is substantial evidence supporting the replacement of numerical simulators, PROVIDED that the neural operator based approach is shown to be more efficient (see Weakness 1).

  2. The architecture considerations are very intuitive and use energy based formulations which are widely accepted in modeling of deformable object dynamics and collision/contact handling. The learnable parameters affect only the effective material constants like stiffness coefficient k or damping coefficient c, with the rest of the formulations of conservative term U and dissipative term R interpretable. Some of the architectural choices, such as the neighborhood kernel B and contact-counting weight w_p, are justified using ablations.

Critical weaknesses
  1. The authors say that “numerical simulators are becoming increasingly inefficient” to justify the usage of neural operators, but do not provide any runtime numbers to support this claim. I would like to see runtime comparisons between the proposed method and the spring-mass simulator used by the authors to generate the ground truth trajectory data.

  2. While the trajectory reconstruction error values of PNEO are better than the baselines, they do not provide full information about the accuracy of the neural operator when compared with the simulator since the metric RE@H uses absolute values. It would be better to provide relative error values, for e.g. considering the object or domain size as a denominator. There are no mentions of the object or domain sizes used in the demos included in the paper. The only variable that gives some sense of these sizes is the particle radius r_p = 0.04, and the particle count = 128. From an empirical analysis, 128 particles of radius 0.04 have about the same volume as one spherical object of radius 0.2. So, reconstruction error of 0.19 (RE@48, M=2, Table 1) corresponds to about 95% error when compared to radius of an object. Therefore, I would like the authors to provide relative error values, or clarify this point if I misunderstood anything.

  3. The authors do not provide any videos or code, which makes it harder to reproduce the results and thus evaluate the contribution thoroughly.

Questions for authors

Questions:

  1. What is the reason the proposed method only scales upto 50 objects? Is it something to do with compute or the multi-object composition algorithm itself?
  2. The function phi(u) = max(u,0) and 1[delta_p > 0] are non-differentiable in particle positions at the contact location (i.e. u = delta_p = 0) , thereby making U and R non-differentiable. How do the authors address this?

Suggestions:

  1. The authors should add more detail to the Figure 1 caption to explain the figure and their method.
  2. Please elaborate on the variables in Figure 2 in caption, to make the figure self-sufficient. What is a, U or R?
  3. Typo: ‘dyanmics’ on line 118-119
3 comments

W1

Thank you for requesting this comparison. We now report runtimes against the spring–mass simulator used to generate our data. All measurements use the NVIDIA GB10 platform, with the simulator running on one CPU core and PNEO on the GPU.

Table R1. Reported runtime per trajectory (seconds).

Method / timing componentM=2M=3M=5M=10M=20M=50
GT simulator, 64 substeps (test reference)5.715.248.02138885,675
GT simulator, 32 substeps (training data)2.97.724.21064452,882
PNEO, 32 substeps (optimized rollout execution)0.1200.1430.2180.5321.7721.872
PNEO preparation per trajectory (median; additional)0.070.070.080.140.355.65

At M=50, the measured runtimes of GT 64 and GT 32 are 5,675 s and 2,882 s, respectively, compared with 1.872 s for PNEO rollout execution, corresponding to reported rollout-execution timing ratios of approximately 3,000× and 1,540×. These results demonstrate substantially faster rollout execution than our GT spring–mass simulator on the tested GB10 platform.

W2

We normalize each object's reconstruction error by its fixed initial ground-truth radius, defined as the maximum distance from its point-cloud centroid to any particle, before averaging with the original particle and scene weights. The particle contact radius rp=0.04r_p=0.04 does not determine object size. The mean and standard deviation of the object radius at each M are reported in Table R2.

Table R2. Object-radius-normalized reconstruction error (%), lower is better. Each cell reports the mean RE@24 / RE@48 relative to the object's radius, from the additional evaluation. Column headings give the object radius rr as mean ± standard deviation.

MethodM=2 (r=0.555 ± 0.016)M=3 (r=0.592 ± 0.049)M=5 (r=0.614 ± 0.061)M=10 (r=0.607 ± 0.055)M=20 (r=0.602 ± 0.053)
PNEO12.0 / 33.614.3 / 34.815.7 / 38.09.5 / 21.68.7 / 20.8
GNS42.6 / 75.256.7 / 117.356.1 / 111.453.3 / 74.161.9 / 201.4
MeshGraphNets32.9 / 68.947.3 / 100.337.1 / 76.245.6 / 65.750.3 / 167.2
SGNN50.8 / 83.465.2 / 134.555.0 / 105.979.8 / 98.8103.4 / 252.6
EGNO86.2 / 110.392.1 / 144.290.8 / 408.289.9 / 107.4102.9 / 153.4
TIE59.9 / 85.170.1 / 125.972.8 / 115.662.9 / 85.271.5 / 103.8
SBC35.0 / 67.445.8 / 91.438.5 / 77.331.8 / 62.336.1 / 80.2
EqCollide52.4 / 86.860.7 / 126.474.5 / 119.475.2 / 89.6102.5 / 224.0
DPI-Net57.6 / 82.7————

PNEO achieves the lowest mean relative error at both horizons for every evaluated object count. At M=2, its mean RE@48 is 33.6% of the object radius, providing a direct interpretation of the error at the measured object scale.

W3

Thank you for raising this. Videos showing the comparison of GT and PNEO trajectories for different object counts are included in the submitted supplementary materials. We will make this clearer in the revised manuscript. Although code was not attached to the submission, we will open-source it immediately upon acceptance.

Q1

We would like to thank the reviewer for the helpful comments. PNEO has no fixed architectural cap at 50 objects: it applies the same learned two-object interaction law to each active contact-graph edge. We now report a 100-object zero-shot test without retraining:

Table R3. 100-object contact stress test.

MethodRollout execution (s)Peak GPU memoryReference RE@24
PNEO5.66339 MB0.040
GNS24.111.2 GB0.522
MeshGraphNets37.05.0 GB1.667
SGNN15.21.8 GB0.498
EGNO9.28.4 GB1.477
TIE27630.3 GB1.019
SBC1313.6 GB1.403

At M=100, PNEO achieves a reference RE@24 of 0.040, compared with 0.0838, 0.0762, and 0.0825 at M=10, 20, and 50 in Table 2 (in our paper), respectively, where we observe no marked increase in rollout error in this stress test. The main benchmark was limited to M≤50 by the GT-generation budget as a single M=100 reference trajectory takes approximately 6.3 hours on one CPU core, making a large GT dataset at this scale costly.

Q2

With the predicted coefficients held fixed during force differentiation, the elastic potential U=12kmax⁡(δ,0)2U=\frac{1}{2}k\max(\delta,0)^2 is continuously differentiable at δ=0\delta=0. Consequently, the elastic force remains continuous and vanishes at contact onset, although the second derivative of UU is discontinuous.

For the dissipation potential RR, we acknowledge that the hard contact indicator introduces a discontinuity with respect to position at contact onset for approaching contacts. However, the damping force is obtained from −∂R/∂v-\partial R/\partial v, which does not require differentiating the contact indicator with respect to position. The resulting force jump has magnitude cmax⁡(−vn,0)c\max(-v^n,0), with the inactive branch selected at δ=0\delta=0. During training, gradients are computed within the active contact branch, without accounting for contact-timing changes induced by position perturbations. We will explicitly clarify this gradient convention and its limitations in our modifications, rather than claim global differentiability across contact transitions.

S1–S3

Thank you for these helpful suggestions. In the revised manuscript, we will expand the captions of Figures 1 and 2 to clarify the method and define the variables, and correct the typo “dyanmics” to “dynamics”.

Reviewer_2xLTOpenReview ↗

Thank you

Thank you for the additional comparisons and rebuttal response that address most of my concerns. I will maintain my original weak-accept rating.

Reviewer_yPeNOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PNEO, a neural simulation method for scalable collision dynamics of 3D deformable objects. The method learns pairwise contact stiffness and damping from two-object collision simulations, then combines these learned interactions to simulate scenes with more objects. Experiments on a synthetic spring–mass benchmark show promising results, outperforming several neural simulation baselines and generalizing to scenes with up to 50 objects without retraining.

Critical strengths

The method demonstrates impressive zero-shot generalization from two-object collision training to scenes containing up to 50 objects, without retraining. The results are strong and consistently outperform most evaluated baselines.

Critical weaknesses
  1. The energy-based contact formulation is closely related to NEXUS, which already models multi-object interactions through learned potentials and dissipation. The paper lacks a direct comparison with NEXUS, making it difficult to establish whether the proposed formulation offers meaningful improvements in scalability or accuracy over this closely related method.

  2. While training on two-object interactions is an interesting approach, the practical advantage is unclear given that the authors already have access to a simulator capable of generating larger scenes. The paper does not quantify the savings in data generation or training cost compared with training on multi-object simulations.

Questions for authors
  1. Could the authors train NEXUS on the same two-object dataset and evaluate its zero-shot generalization to larger multi-object scenes? An analysis of the performance differences would help clarify what the proposed pairwise formulation adds over NEXUS.

  2. While the paper motivates two-object training through scalability, could the authors quantify its benefits in terms of data generation cost, training time, and computational resources compared with training on larger multi-object scenes?

2 comments

Q1

Thank you for suggesting this comparison. Since NEXUS uses object-count-dependent identity encoding, we introduce an identity projection to enable zero-shot evaluation at unseen object counts, denoted NEXUS*. Each accuracy cell in Table R1 reports RE@24 / RE@48.

Table R1. Zero-shot accuracy and training cost comparison with NEXUS.

MethodTraining MM=10 ↓M=20 ↓M=50 ↓Training time (min/epoch, H=24)
NEXUS*20.1308 / 0.34640.1085 / 0.27560.0907 / 0.1395259.24
NEXUS*30.1357 / 0.37090.1141 / 0.27470.1024 / 0.1331284.61
NEXUS*50.1272 / 0.34540.1082 / 0.26770.0953 / 0.1283365.23
PNEO20.0838 / 0.21590.0762 / 0.21870.0825 / 0.1268273.71

Zero-shot generalisation. Trained only at M=2, PNEO achieves lower RE@24 and RE@48 than all three NEXUS* variants at every tested object count.

Training efficiency. Although PNEO requires 5.6% more training time per epoch than NEXUS* at M=2, it achieves better generalisation without increasing the training object count. In contrast, increasing NEXUS* training from M=2 to M=5 raises per-epoch training time by 40.9%, yet still yields higher errors than PNEO on M=10/20/50. These results demonstrate that pairwise training enables more accurate object-count extrapolation at competitive training cost, without requiring larger-scene training data.

Q2

Thank you for asking us to quantify this benefit. PNEO is trained once on two-object scenes and reused at larger object counts without retraining. Its training cost therefore remains unchanged as the deployment scene grows, whereas generating larger-scene training data becomes increasingly expensive. Given the substantial cost of larger-scene training, we limited training-time and GPU-memory measurements to M = 2, 3, and 5, which already demonstrate the computational savings of two-object training.

Table R2. Data-generation and training costs.

MetricM=2M=3M=5M=10M=20M=50
GT simulation per trajectory (s)2.97.724.21064452,882
Generation cost for 7,200 trajectories (CPU h)5.815.448.42128905,764
Full training time (h)16.718.626.4———
Peak training GPU memory (GB)0.681.313.29———

Generation costs are estimated from measured single-core simulation times with 32 substeps, while training costs are compared under a matched update budget. As shown in Table R2, training on two-object interactions reduces training time by 36.7% and peak GPU memory usage by 79.3% compared with five-object training. Moreover, for our M=50 benchmark, generating the same number of trajectories would require approximately 994× the simulation time of M=2 data, highlighting the substantial data-generation costs reduced by PNEO. Importantly, Table 5 in our paper shows that increasing the number of training objects from 2 to 3 or 5 yields no consistent accuracy improvement on M=10/20 test scenes. These results demonstrate that pairwise training substantially reduces computational and data-generation costs while maintaining competitive accuracy in larger, unseen scenes.

Dear Reviewer,

Thank you again for your constructive feedback. We would like to kindly follow up on our rebuttal, in which we provided additional comparisons with NEXUS and quantitative analyses of data-generation and training costs to address your two main concerns. We would greatly appreciate it if you could take a look at these new results and let us know whether they address your concerns. We would also be happy to provide any further clarification if needed. Thank you for your time and consideration!

Comments on the paper

2 comments

Dear reviewers,

As the deadline for the reviewer-author discussion is approaching, please check whether the authors have adequately addressed your concerns. If any concerns remain unresolved, please raise them in the discussion.

Best regards,

Your AC

AuthorsOpenReview ↗

General Response to Area Chairs and Reviewers

We sincerely thank the Area Chair and Reviewers yPeN, 2xLT, and mTzu for their constructive feedback. In response, we have conducted additional experiments and analyses to further clarify PNEO's methodological contributions, generalization capabilities, and physical properties. We summarise the key findings below:

  • Comparison with the NEXUS baseline. We compare PNEO against NEXUS at unseen object counts (denoted NEXUS*). PNEO, trained exclusively on a two-object collision dataset, consistently achieves lower rollout errors than NEXUS* trained on 2, 3, or 5 objects across all zero-shot evaluations involving 10, 20, and 50 objects.

  • Scalability and training efficiency. We quantify the computational advantages of PNEO's pairwise training using only two-object datasets, demonstrating a 37% reduction in training time compared with five-object training under a matched update budget, alongside substantially lower data-generation costs for multi-object deformable dynamics. We further demonstrate that the same two-object-trained PNEO can generalize to 100-object scenes without retraining, highlighting its scalability beyond the original evaluation settings.

  • Generalization to rigid-body dynamics. Beyond deformable-body collisions, we extend PNEO to rigid-body dynamics while preserving its pairwise contact formulation. On MOVi-B, PNEO consistently achieves lower position RMSE than RigidFormer across all evaluated rollout horizons and step sizes, demonstrating the broader applicability of the proposed formulation.

  • Physical consistency and interpretation from energy and momentum conservation perspectives. We provide additional analyses of energy variations induced by learned coefficient updates and momentum balance relative to external impulses. We also clarify the force differentiation convention and its limitations at contact transitions, providing a more precise interpretation of the energy and momentum conservation properties of the learned dynamics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.