acceptodds
Withdrawn from ICLR 2027

G-Inverse: Generative Neural Inverse Dynamics for Deformable Objects

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Data-driven representations have enabled increasingly general models of deformable 3D objects. However, inferring how an object will respond to new physical interactions from a single observed motion remains challenging: the observed trajectory entangles the object's intrinsic material behavior with its initial state and external conditions. Existing inverse-physics approaches commonly recover parameters of a predefined material model by reproducing the observation, which does not necessarily yield a response model that remains valid beyond the demonstrated interaction. We introduce a generative neural inverse-physics framework that learns reusable material behavior from a single observed 4D sequence. Our method combines a pretrained neural deformation representation with a differentiable physical simulator operating in compact deformation coordinates. Within this representation, we learn a spatially shared material-response model whose evolving internal state captures deformation history, while the simulator explicitly accounts for coupled rigid and nonrigid motion, external forces, and contact. Once identified from the demonstration, the deformation representation and material model are fixed and reused to generate continuous 3D motion under new initial poses, velocities, forces, and contact conditions, without fitting to the target trajectories. This formulation turns single-sequence reconstruction into a model of object-specific physical behavior that can be queried under novel interactions.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
System Identification, Inverse Dynamics, Physics-based Modeling, Physical Scene Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qyonOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a framework for predicting deformable object dynamics by formulating prediction under new interactions as a conditional generation task, conditioned jointly on a single complete observed response and an explicit query condition. Instead of explicit material parameter estimation, it first separates global rigid motion from local deformation, projecting local dynamics into a compact 20-dimensional active latent subspace. By conditioning on a historical observation sequence and an explicit query condition, the model autoregressively predicts physical responses. The framework is formulated around explicit mesh inputs, known external forces, and clean rigid-non-rigid decoupling, with empirical validation conducted in synthetic simulation environments.

Critical strengths

Framing cross-condition dynamics prediction as conditional generation over a historical response and an explicit query condition is conceptually elegant. It effectively bypasses intractable material parameter identification by cleanly decoupling intrinsic object properties from extrinsic driving forces.

Critical weaknesses
  1. Theoretical Assumptions and Real-World Applicability:

A primary limitation of the proposed framework lies in its heavy reliance on a restrictive set of idealized assumptions. The pipeline requires mesh inputs, known external loadings, tractable contact force formulations, and a clean, unambiguous decoupling of rigid-body motion from non-rigid deformation. Because such pristine conditions are rarely obtainable from noisy, unconstrained sensor data in real-world environments, the methodology remains largely confined to closed-world simulation settings, limiting its broader practical validity.

  1. Insufficient Experiments and Validation:

Furthermore, the empirical evaluation is insufficiently comprehensive. Experiments are restricted to homogeneous, in-distribution synthetic benchmarks spanning a narrow spectrum of canonical materials and basic interaction types. Furthermore, the evaluation lacks essential ablation studies. Given the sparse qualitative results and shallow descriptions, it is difficult to discern whether the model is learning generalizable physical principles or merely memorizing trajectory distributions within a controlled, idealized testbed.

Questions for authors
  1. Handling Unobserved Contact Forces: Given the requirement for explicit external loadings and tractable contact force evaluations during rollout, how can the method be adapted to unconstrained, in-the-wild environments where contact forces are entirely unobserved?

  2. Contact Detection and Robustness: How does the framework perform contact detection and resolution under real-world conditions where clean, explicit mesh surfaces are absent, and how do sensory noise and occlusion affect collision boundaries?

  3. Multi-Object and Deformable-to-Deformable Interactions: Is the method capable of scaling beyond single-object-centric scenarios to handle complex, multi-body interactions, such as multiple soft objects colliding, wrapping, or sliding against each other?

  4. Handling Fully Coupled Interactions: Given the strict decoupling of rigid motion and non-rigid deformation, how does the model account for fully coupled, bi-directional feedback where local deformation continuously reshapes contact areas and boundary forces?

  5. Comprehensive Qualitative Analysis and Failure Cases: The current qualitative evaluation is sparse and restricted to idealized settings. Can the authors provide more extensive visual results, including failure cases, long-term rollout drift, and the model's capacity to preserve high-frequency surface details over extended horizons?

Reviewer_Mp93OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an inverse dynamics prediction method. It conditions its predictions on exactly one demonstration trajectory, and predicts the deformation trajectory of the same object under a different "loading condition". The prediction model seaparately learns geometry-conditioned representation (in my understanding, a model reduction) and demonstration-conditioned dynamics (a GRU or Transformer, conditioned on the one-demonstration context and the new query condition, predicts latent acceleration that is integrated to roll out the trajectory).

Critical strengths

The proposed idea of combining pre-trained model reduction with inferece-time "implicit" inverse dynamics from one single demonstration trajectory is novel to the best of my knowledge. This paradigm could be useful in certain application scenarios, e.g. when demonstration data are limited and the constitutive model is unknown.

Critical weaknesses

Presentation

The presentation could be improved.

For example, Sec 4 is particularly confusing to read, as the reader needs to jump back and forth between the main text and Appendix sections to understand each term and symbol defined (e.g. BB's construction, μcond\mu_\text{cond}'s definition, where the number "131072" comes from, etc.). I would suggest relate the text descriptions more closely to Figure 2 and the math notation defined there. Without the relation explicitly stated, I found it challenging to track "what is being trained/fitted/computed on what input/data" in each subsection of Sec 4.

Sec 6 has text interleaved with tables, making the main text hard to follow. Also Table 2 seems never referenced in the main text, nor is it discussed.

Experiments

Qualitative results: I would like to see more qualitative comparison of trajectory prediction beyond Figure 4 and Figure 5, ideally for all baseline methods compared and on multiple (geometry, material, loading) settings, or at least on several different datasets. Existing figures are not infomative enough about different methods' prediction quality.

Quantitative results: For dynamic prediction, the only compared metrics are two RMSEs (one averaged over the full trajectory and the other only at the final frame). I would like to see more fine-grained evaluation results, like how well each method predict for different materials and loading conditions. Reporting standard deviations of those metrics would also be helpful.

Contribution

The paper provides limited theoretical justification (or sufficient empirical justification) of the design choices it made, e.g. using exactly one observation trajectory and the rank r=20. While the method is different from prior work (e.g. NeuROK) in certain ways, without detailed analysis (theoretical or empirical), I do not find the results inspiring for others working in the neural dynamics field.

Questions for authors
  1. Why does the paper choose to condition dynamic prediction on exactly one single demonstration trajectory of the same object with known loading, instead of multiple demonstration trajectories with potentially diverse loading? Have the authors studied whether adding more demonstrations or varying loading type of the demonstration would improve or hurt prediction quality? Have the authors ablated the demonstration itself, e.g. feeding a mean cc over the corpus to the prediction model to see if it truly improves prediction accuracy?

  2. Materials and topology preservation: Line 217 says, "This factorization assumes topology-preserving motion and a stable correspondence-based separation between rigid motion and local deformation." However, the datasets seem to contain granular material ("sand") and "snow", which hardly preserve topology in the general case. This feels confusing to me. Meanwhile, all the qualitative results visualized seem to preserve topology well (I believe their materials mostly belong to the "elastic" and "plastic" category). Have the authors separately tested whether the proposed method learns well from "sand" and "snow" data and predict those materials' characteristic phenomena, e.g. a sand pile's angle of repose, the elastoplasticity of snow, and some level of topological change? I would like to see the method evaluated separately on different materials and loading types.

  3. Datasets: What are the "DynamicObjaverse" and "DyMesh" datasets? Each is refered to three times in the paper but neither seems cited. I understand that "DyMesh" is probably the dataset in AnimateAnyMesh, but I failed to find the exact "DynamicObjaverse" dataset online.

  4. A presentation confusion: In Figure 2, why is "response code c" denoted by a "wood log" icon? Also, why is "geometry features g" (if I understand it right) denoted by a triangular ruler? I tried but failed to see the correspondence. I would suggest depicting it as a vector, as Eq. (5) says c∈R32c \in \mathbb R^{32}.

  5. Table 2, first two rows: why are they identical? If they are identical by construction, Is it necessary to keep both in this table?

Reviewer_NsSsOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes G-Inverse, a neural simulation method that predicts how a deformable object moves under new physical interactions based on an observed demonstration. The authors generate simulated trajectories across different objects, materials, and interactions, and fine-tune NeuROK to represent object deformations. They then train a GRU-based dynamics model conditioned on the demonstration, object geometry, and applied forces to predict future motion. Experiments show improved deformation reconstruction and trajectory prediction compared to several baselines.

Critical strengths
  1. The paper explores an interesting direction of learning deformable-object dynamics entirely with neural networks. If successful, this approach could be more scalable than traditional explicit simulation, as it could leverage large amounts of synthetic data for pretraining and potentially real-world data for fine-tuning.

  2. Promising experimental results. The method achieves strong results on simulated deformation and motion prediction tasks, with substantial improvements over the evaluated baselines.

Critical weaknesses
  1. The paper evaluates entirely on synthetic simulations, assuming known geometry, complete 3D trajectories, external forces, and contact conditions. These assumptions are highly idealized, and without real-world validation, the practical usefulness of the proposed method remains unclear.

  2. It is unclear to me whether the model actually learns material properties from demonstrations or simply relies on geometry-based priors. The paper lacks key ablations, such as removing demonstration conditioning or testing identical geometries with different materials and forces. Evaluation is also limited to MPM simulations, leaving generalization to other simulators unclear. These limitations weaken the paper's central claim of demonstration-conditioned dynamics prediction.

Questions for authors
  1. How do the authors envision applying this framework to real-world scenarios, given its assumptions about known geometry, forces, and 3D trajectories? Are there any real-world experiments or evidence supporting its practical applicability?

  2. Could the authors provide ablations showing how much the model benefits from the observed demonstration? In particular, comparisons without demonstration conditioning and controlled experiments varying material properties while keeping geometry fixed would help establish what the model actually learns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.