acceptodds
Withdrawn from ICLR 2027

PhysON: Physicalizing Everything Everywhere All at Once

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recovering spatially varying material behavior from video remains challenging. Existing approaches often assume a homogeneous material or restrict inference to a predefined constitutive family. We study visual system identification of spatially heterogeneous scenes, encompassing material variation within objects and across interacting objects. The central challenge is to infer local constitutive differences from the coupled motion of the scene. Our key insight is that a shared constitutive representation can constrain these local responses while allowing their spatial distribution to be inferred from motion. We introduce PhysON, which learns a latent-conditioned constitutive model from analytic material responses and represents scene materials as a continuous spatial latent field. Reconstructed continuum trajectories guide field optimization through differentiable simulation, with the shared constitutive model kept fixed and no object-wise material assignments required. We also introduce a synthetic multiview benchmark covering heterogeneous objects and interactions among different materials. Experiments on public benchmarks and our dataset demonstrate improved motion reconstruction and future prediction across diverse material behaviors, including spatially heterogeneous scenes. Ablations highlight the importance of material-field initialization and constitutive supervision for recovering local material responses from scene dynamics.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
System Identification, Inverse Dynamics, Physics-based Modeling, Physical Scene Understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jzRDOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PhysON, a method for recovering spatially heterogeneous material properties from multi-view videos. The approach pretrains a unified neural constitutive model and optimizes per-particle material embeddings through differentiable Material Point Method (MPM) simulation, without explicitly specifying material categories. The authors introduce the PhysON-Validation benchmark and evaluate the effectiveness of their approach through motion prediction, baseline comparisons, and ablation studies.

Critical strengths
  1. Unified material representation. By combining a shared constitutive model with per-particle material embeddings, the method can represent diverse materials and their spatial variations within a unified framework, eliminating the need for explicit material category selection.

  2. Video-driven physical inverse modeling framework. The approach integrates dynamic scene reconstruction with differentiable physics simulation to optimize spatial material fields from visual observations, offering practical value for real-world applications.

  3. Targeted experimental design. The authors construct a synthetic benchmark featuring heterogeneous materials and unseen material parameters, and conduct ablation studies to demonstrate the importance of the initialization strategy and per-particle representation.

Critical weaknesses
  1. Insufficient identifiability of recovered material properties. The method independently optimizes an 8-dimensional material embedding for each particle without spatial constraints. Although PhysON-Validation provides ground-truth material distributions and constitutive information, the paper does not directly evaluate whether the recovered material fields correspond to the true physical properties. Appendix D.2 further suggests that the material embeddings may compensate for geometric reconstruction errors rather than recover the underlying material properties.
  2. Insufficient validation of the unified representation across material families. All five single-object scenes in Table 1 involve only parameter variations within the same material family, while combinations of different material families appear only across separate objects. Therefore, it remains unclear whether the method can accurately recover complex spatial distributions involving different material families within a single object, which is an important application scenario for a unified material representation.
  3. Limited reliability of counterfactual predictions using the recovered material fields. Figure 8 shows that, after changing the initial velocity, the average Chamfer Distance (CD) in the elastic–plastic collision scene increases from 0.161 to 0.822, while another perturbation even causes the simulation to diverge at frame 23. These results suggest that the method may primarily fit specific observed trajectories rather than sufficiently recover transferable physical laws.
Questions for authors

See Weaknesses.

Reviewer_Z9cGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents PhysON, a visual system identification framework for recovering spatially varying material behavior from multiview videos. The method first generates training data using five families of analytic constitutive models and pretrains a shared neural constitutive model consisting of a stress head and a deformation projection head. Different material behaviors are represented through a unified low-dimensional latent code space. During scene-level inverse physics, the authors reconstruct geometry and continuum particle trajectories using dynamic Gaussians, and associate the frozen constitutive model with per-particle material codes. These codes are initialized through observed-motion screening and subsequently optimized via differentiable Material Point Method (MPM) simulation to align simulated motion with reconstructed trajectories and multiview silhouettes. The authors also introduce PhysON-Validation, a synthetic benchmark featuring spatially varying material parameters within individual objects, interactions between objects of different materials, and paired on-bank and off-bank material instances. The method is evaluated through observed-motion fitting, future rollout prediction, and ablation studies.

Critical strengths
  1. The paper addresses a meaningful research problem.

Recovering spatially varying material behavior from videos is a challenging and practically relevant problem in visual system identification. By combining a shared constitutive model with per-particle material codes, the proposed framework allows different spatial locations to exhibit distinct material responses while reducing the need for predefined region-wise material-family assignments. This representation offers flexibility and provides a reasonable approach to unified physical modeling of complex scenes.

  1. The experimental design includes targeted evaluations.

The authors construct a synthetic benchmark featuring different material phases and multi-object interactions. The paired on-bank and off-bank settings are designed to evaluate generalization to material parameters unseen during pretraining. Additionally, ablation studies on initialization strategies and spatial material representations provide supporting evidence for several design choices.

Critical weaknesses
  1. The technical novelty is relatively limited, as the main contributions largely build upon existing methods.

The core technical components of PhysON substantially overlap with prior work. UniPhy (CVPR 2025) has already introduced a shared latent-conditioned constitutive model across material families, where material codes are optimized through differentiable simulation while the pretrained network remains frozen, enabling inverse physics without predefined material-family labels. MASIV (ICCV 2025) has established a video-based system identification framework using dynamic Gaussian reconstruction, continuum particle trajectories, and differentiable simulation.

PhysON mainly extends the shared material representation to per-particle codes, together with observed-motion screening for initialization and modifications to the constitutive network. While these design choices are reasonable, the overall methodology largely builds upon existing approaches. Although the current ablation studies demonstrate the advantage of per-particle codes over a single shared code, they do not sufficiently isolate the contributions of the other design components. In particular, the paper lacks a direct comparison with a straightforward extension that integrates a UniPhy-style shared constitutive model into an existing video-based inverse physics framework with per-particle code optimization.

  1. The experiments lack direct validation of cross-family material heterogeneity within a single continuous object.

The paper emphasizes that its shared neural constitutive model can represent different material families, including elasticity, plasticity, sand, and fluids, and that per-particle material codes enable spatially varying material inference without region-wise material-family assignments. However, the current experiments do not sufficiently validate this capability within a single continuous object.

According to Table 1, all five single-object heterogeneous scenes in PhysON-Validation contain multiple material phases belonging to the same constitutive family. Spatial variation is primarily introduced by changing physical parameters such as stiffness, yield stress, friction coefficients, or viscosity. Although the paper also includes two cross-family scenarios involving elastic–plastic collisions and an elastic cube entering water, these scenarios consist of separate objects. Consequently, the experiments do not directly demonstrate whether the proposed method can recover regions governed by different constitutive families within a single continuous object.

Questions for authors
  1. Regarding the independent technical contributions and novelty of the method:

UniPhy has already introduced a shared latent-conditioned constitutive model, while MASIV has established a video-based inverse physics framework using continuum particle trajectories and differentiable simulation. The main contributions of PhysON appear to center on per-particle material codes and observed-motion screening. Could the authors further clarify which technical components of PhysON provide substantial advantages beyond a straightforward combination of these existing approaches?

  1. Regarding cross-family material heterogeneity within a single continuous object:

The paper emphasizes that a unified material representation can describe spatially varying material behavior without predefined region-wise material-family assignments. However, all single-object heterogeneous experiments are restricted to variations within the same constitutive family, while cross-family experiments involve interactions between separate objects.

Could the authors provide additional experiments involving a single continuous object containing regions governed by different constitutive families, such as a composite object with both elastic and plastic regions, and demonstrate whether the corresponding material responses can be recovered solely from observed motion?

Reviewer_4aHtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents PhysON, which aims to recover spatially varying material dynamics from video observations. The method first pretrains a shared neural constitutive model across multiple material families. For a new scene, it then uses observed-motion screening to select a suitable initial material code and refines per-particle material codes through differentiable MPM, thereby recovering a spatial material field and predicting future dynamics. Overall, this is a solid piece of work. However, the writing and presentation could be improved, and despite the extensive experiments, the current evidence does not clearly support the central claim of recovering spatially varying material behavior from video.

Critical strengths

S1. Meaningful problem. Recovering intrinsic dynamics from visual observations is an important problem with potential applications in physical simulation, future-state prediction, and robotic interaction.

S2. Elegant method design. Combining a shared constitutive prior with per-particle material code optimization is a reasonable and appealing approach to modeling spatially varying material dynamics.

S3. Broad experimental coverage and detailed implementation. The paper evaluates the method on a variety of synthetic datasets and real-world scenes, with extensive baseline comparisons, ablation studies, and implementation details.

Critical weaknesses

W1. Unclear motivation. The introduction does not sufficiently clarify the specific limitations of existing methods, such as OmniPhysGS, making it difficult to grasp the core motivation and advantages of the proposed method.

W2. Lack of focus in experimental organization. Compared with PAC-NeRF and Spring-Gaus, which primarily feature homogeneous objects, PhysON-Validation directly evaluates spatial material heterogeneity and should therefore receive greater emphasis. Reorganizing the experiments around the central claim and highlighting the most directly relevant results would provide more convincing evidence for the paper's central claim.

W3. Lack of visualization. The main paper lacks clear and intuitive visualizations to support its claims. Adding visualizations of the inferred spatial material field would improve the paper's credibility.

W4. Overstated Claims on Spatial Material Heterogeneity. The current experiments primarily evaluate parameter variations within a single constitutive family and interactions between objects governed by different constitutive families, but do not sufficiently demonstrate that different constitutive mechanisms can coexist within a single object. Therefore, the claim in the introduction (Lines 49–51) that different regions of an object may deform, retain permanent changes, or dissipate motion differently is somewhat overstated.

W5. Incomplete Related Work. The paper lacks sufficient discussion of relevant work such as VisionLaw, a representative study on visual system identification and physical understanding.

Questions for authors

Q1. Under the same ground-truth video supervision, what advantages does PhysON offer over methods such as OmniPhysGS and PhysDreamer? The prior is still trained on predefined material families, which may limit its generalization to unknown constitutive mechanisms. Moreover, the flexible neural representation may overfit the observations rather than learn meaningful constitutive behavior, potentially making optimization more difficult and leading to numerical instabilities.

If the authors adequately address my concerns above, I would be willing to adjust my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.