- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces RiCo, a mesh-free neural simulator for multi-object rigid-body dynamics. Motivated by the observation that rigid-body contact occurs only locally between adjacent surfaces, the authors model inter-object interactions through sparse, point-level local contacts rather than mesh-based or object-level attention-based formulations. Specifically, for each surface point, the model retrieves contact candidates via K-nearest-neighbor (KNN) search and constructs interaction descriptors that include an estimated signed distance. A point Transformer then aggregates this contact information within each object to infer how local contacts affect the object’s overall motion. Finally, the rigid transformation is recovered with the Kabsch algorithm from motion predicted at a small set of anchor points. On the MOVi benchmark, RiCo substantially reduces position and orientation errors, as well as interpenetration, compared with existing methods. The authors further demonstrate scalability with respect to point resolution, zero-shot generalization to scenes with 270 objects, and preliminary sim-to-real transfer in a real-world billiards setting.
Critical strengths
S1. Significant performance improvement
Using only irregular surface points without mesh connectivity, RiCo reduces position errors by 31–35% and orientation errors by approximately 38% on MOVi-A/B. Notably, it achieves substantially lower penetration depth than both mesh-based and point-based baselines, showing that precise contact modeling is achievable with a purely point-based representation.
S2. Resolution scalability
Contact fidelity improves consistently as the number of points increases (Fig. 4), and the runtime at each resolution is reported (Fig. 5). This gives practitioners the flexibility to choose a resolution that fits their computational budget.
Critical weaknesses
W1. Insufficient analysis of contact fidelity
RiCo’s MPD is more than 100× smaller than those of the baselines. Even without the signed distance feature, a ~30$\times$ gap remains, yet the source of this gap is not analyzed. Because MPD is a mean, it may be dominated by catastrophic failures in a few sequences; however, the supplementary video does not suggest that the baselines fail this severely. In addition, the values reported for RigidFormer under seemingly identical settings differ between Fig. 3a (267.70 mm) and Fig. 4a (402 mm).
W2. Limited novelty of the main idea
The key change relative to RigidFormer is replacing object-level attention with point-level KNN contact. However, decoupling local contact from intra-object propagation has already been adopted in FIGNet and SDF-Sim. The paper also lacks analysis of the specific benefits this change brings across different contact scenarios.
Questions for authors
Q1. Contact fidelity analysis
Could you provide the per-sequence distribution of the baselines’ MPD (e.g., median), statistics separated into object–object and object–ground penetration, and the absolute ground-truth PTR/MPD values? Also, why do the RigidFormer values differ between Fig. 3a and Fig. 4a?
Q2. Novelty of the main idea
What are the structural differences between RiCo and FIGNet or SDF-Sim, and why is SDF-Sim not included as a baseline? Is there an ablation that isolates the contribution of point-level KNN contact?
Q3. Training strategy
Since the model is trained only with teacher forcing, how is stability ensured in long-horizon rollouts? Moreover, does applying robustness-oriented training techniques (e.g., autoregressive fine-tuning) lead to a significant drop in performance?
4 comments
Author response to Reviewer cQV5 - Part 2
- "Weakness: RiCo’s MPD is more than 100$\times$ smaller than those of the baselines. Even without the signed distance feature, a ~30 gap remains, yet the source of this gap is not analyzed. ... In addition, the values reported for RigidFormer under seemingly identical settings differ between Fig. 3a (267.70 mm) and Fig. 4a (402 mm)." "Weakness: The key change relative to RigidFormer is replacing object-level attention with point-level KNN contact. However, decoupling local contact from intra-object propagation has already been adopted in FIGNet and SDF-Sim. The paper also lacks analysis of the specific benefits this change brings across different contact scenarios." "Question: What are the structural differences between RiCo and FIGNet or SDF-Sim, and why is SDF-Sim not included as a baseline? Is there an ablation that isolates the contribution of point-level KNN contact?"
Our response.
a. Architectural differences and SDF-Sim comparison.
FIGNet relies on mesh connectivity and face-face interaction hyperedges, while SDF-Sim uses pretrained signed-distance functions to construct surface-node-to-object collision edges. In contrast, RiCo operates directly on unstructured point clouds without requiring meshes or pretrained SDFs. It models sparse point-level contacts using relative geometry, motion, and physical properties, followed by intra-object Point Transformer reasoning.
SDF-Sim was initially omitted primarily due to the lack of results at multiple prediction horizons on MOVi-B. Furthermore, its official implementation is not publicly available to our knowledge, preventing reproduction under our evaluation protocol. For completeness, we provide a comparison using its published 50-step MOVi-B results.
Table 1. MOVi-B comparison at 50-frame rollouts.
| Method | Translation RMSE (m) ↓ | Rotation Error (°) ↓ |
|---|---|---|
| FIGNet | 0.140 | 15.00 |
| SDF-Sim | 0.160 | 18.03 |
| RiCo (Ours) | 0.032 | 2.59 |
RiCo demonstrates a substantial advantage in prediction accuracy over both FIGNet and SDF-Sim. Specifically, it achieves 77.1% and 80.0% lower translation errors, and 82.7% and 85.6% lower rotation errors, respectively. These results highlight the effectiveness of RiCo's sparse point-level contact reasoning in predicting rigid-body interactions without requiring explicit mesh connectivity or pretrained SDFs.
2. Point-level contact ablation.
We conducted two complementary component-swap ablations on MOVi-B:
- RF Features + RiCo Transformer: Replaces RiCo's sparse KNN contact descriptors with RigidFormer-style point-wise features, while retaining RiCo's intra-object Point Transformer.
- RiCo Contact + RF Transformer: Retains RiCo's KNN contact descriptors but replaces its intra-object Transformer with RigidFormer's scene-level Transformer.
Table 2. Autoregressive rollout performance on MOVi-B.
| Method | Frame | Translation RMSE (m) ↓ | Rotation RMSE (°) ↓ |
|---|---|---|---|
| RF Features + RiCo Transformer | 50 | 0.055 | 4.70 |
| 75 | 0.127 | 10.55 | |
| 100 | 0.212 | 17.46 | |
| RiCo Contact + RF Transformer | 50 | 0.033 | 3.34 |
| 75 | 0.072 | 7.42 | |
| 100 | 0.121 | 12.10 | |
| RiCo (Full) | 50 | 0.032 | 2.59 |
| 75 | 0.065 | 5.45 | |
| 100 | 0.111 | 9.54 |
Replacing RiCo's local contact descriptors with RigidFormer features substantially degrades long-horizon prediction, increasing the 100-frame translation RMSE from 0.111 m to 0.212 m and rotation RMSE from 9.54° to 17.46°. In contrast, retaining RiCo's contact descriptors with RigidFormer's Transformer produces smaller errors (0.121 m / 12.10°), suggesting that fine-grained contact encoding is a major contributor to prediction accuracy. Full RiCo achieves the lowest errors across all three horizons, indicating an additional benefit from intra-object Point Transformer reasoning.
c. Penetration analysis of component ablations.
Table 3. Geometric penetration on MOVi-B.
| Method | PTR (%) ↓ | MPD (mm) ↓ |
|---|---|---|
| Ground Truth | 10.70 | 3.91 |
| RF Features + RiCo Transformer | 36.39 | 25.93 |
| RiCo Contact + RF Transformer | 27.05 | 6.82 |
| RiCo (Full) | 21.70 | 6.13 |
The penetration results further support the importance of local contact encoding. With RiCo's Transformer unchanged, replacing the KNN contact representation with RigidFormer features increases PTR from 21.70% to 36.39% and MPD from 6.13 mm to 25.93 mm. Retaining RiCo's contact representation while using RigidFormer's Transformer yields substantially lower penetration (27.05% PTR / 6.82 mm MPD). Full RiCo achieves the best results, suggesting that local contact representation is particularly important for reducing geometric penetration, while intra-object reasoning provides an additional improvement.
Thank you for the detailed response and the revisions. My concerns have been well addressed through the inclusion of updated contact-fidelity statistics, an ablation study involving component swaps, results from autoregressive fine-tuning, and a controlled comparison with the official RigidFormer code. I have confirmed that these additions have been incorporated into the revised manuscript. Accordingly, I am raising my evaluation score to 3.
Dear reviewer, thank you very much for your positive feedback and the valuable suggestions you provided during the rebuttal period. We have learned a lot from them based on your comments. Thank you!
Author response to Reviewer cQV5 - Part 1
Dear Reviewer, thank you for your constructive feedback. We address your questions below.
Our response. We report penetration statistics and per-sequence MPD distributions, separating object–object and object–ground interactions.
Table 1. Penetration statistics
Table 2. Per-sequence MPD distributions (mm)
RiCo outperforms both baselines across penetration metrics. Relative to RigidFormer, overall PTR drops from 50.27% to 21.70% and MPD from 50.76 to 6.13 mm (87.9% reduction), with gains in both contact types. Per-sequence median/P90 MPD is 4.67/8.57 mm versus 20.84/105.94 mm, indicating fewer severe-penetration trajectories. GT itself has nonzero penetration (10.70% PTR; 3.91 mm MPD).
Dataset-level MPD aggregates penetrated object-frame observations; per-sequence statistics treat each trajectory separately and exclude those without penetration. RigidFormer has 116/120 valid predictions, so its GT reference uses the same subset.
For scale, cubes have 1 m sides and spheres a 0.5 m radius. Per-trajectory PTR/MPD statistics are in our supplementary materials.
RigidFormer's results differ because Fig. 3a uses the official vertex-based implementation, while Fig. 4a uses our point-cloud reimplementation with RiCo-matched point-cloud observations but RigidFormer's original features. We will clarify this in the revision.
Our response. Although RiCo is trained with teacher forcing, Kabsch projection enhances long-horizon stability by fitting rigid transforms to predicted anchors at each rollout step, preserving intra-object geometry. HOPNet (Wei & Fink, 2025) likewise uses shape matching to maintain rigidity.
We also performed 10 epochs of autoregressive fine-tuning on MOVi-A. Starting from two GT frames, predictions are recursively fed back, with intermediate steps detached from gradient tracking and the final step supervised by the original losses. Varying rollout depth exposes the model to accumulated errors.
Table. MOVi-A autoregressive validation performance during fine-tuning.
Across epochs 0–10, position and rotation RMSE show no substantial degradation at 50, 75, or 100 frames. At 75 frames, position RMSE improves from 0.0670 to 0.0654 m and rotation RMSE from 6.49° to 6.31°. Thus, fine-tuning maintains stability without clear overall accuracy gains.