acceptodds
Withdrawn from ICLR 2027

Physical-Field-Bridged Material Parameter Inference from Multi-View Deformation Videos

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Estimating simulation-ready material parameters from visual observations is crucial for deformable-object manipulation, yet remains challenging because material properties are only indirectly expressed through physical responses such as contact, deformation, recovery, and stress concentration. We introduce physical-field-bridged material inference, a single-pass framework that treats dense physical responses as intermediate evidence between RGB deformation videos and a discrete constitutive type together with continuous material parameters. This design addresses a practical limitation of existing inverse-physics methods, which often rely on per-scene reconstruction, online tuning, or simulation-in-the-loop optimization, while feed-forward visual estimators can exploit appearance or category shortcuts. Given multi-view RGB videos, our model predicts force/contact masks, projected motion-flow fields, and projected stress-response maps, then conditions constitutive-type classification and parameter regression on these fields. To train and evaluate this formulation, we construct DeformFieldBench, a 3DGS–MPM learning environment with 5,000 simulations over 100 object shapes, five mechanically informative loading modes, and dense pixel-aligned physical supervision. Our method achieves the best median errors on all four continuous parameters, raises stress-hotspot recall from 0.0762 to 0.5121, and reaches 0.8676 SSIM/36.82 dB PSNR in same-simulator replay on unseen objects. Under a protocol-matched GSO evaluation, it performs inference and future-state prediction in approximately 30 seconds rather than the 15–120 minutes required by per-scene optimization; it also produces measurable replay agreement on five real-world sequences. These results support dense physical fields as an effective bridge for fast material inference, while cross-simulator and real-world tests expose remaining domain and identifiability gaps. The dataset is anonymously released at https://www.kaggle.com/datasets/anonymous336/physical-field-material-parameter-5000

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Deformable Object Material Inference, Dense Physical Field Supervision, Feed-forward Inverse Physics, 3D Gaussian Splatting and Material Point Method Simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Nxg5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies feed-forward inference of constitutive types and continuous material parameters from three-view RGB deformation videos. The predicted parameters include Young’s modulus E, Poisson’s ratio nu, density, and yield stress. The central design first predicts contact masks, projected motion-flow fields, and stress-response maps, then combines encodings of these predicted fields with physics-bottleneck features for material inference. The authors also introduce DeformFieldBench, which contains 5,000 simulations across 100 object shapes and five loading modes, with physical-field supervision aligned with the rendered videos. The evaluation covers material-parameter estimation, physical-field prediction, simulation replay, external benchmark transfer, and several robustness settings.

Critical strengths
  1. The problem is practically relevant, and the approach is well motivated. Rapidly estimating simulation-ready material configurations from visual observations could reduce the computational cost of per-scene optimization. Explicitly predicting contact, motion, and stress responses also provides intermediate outputs that can be inspected and evaluated.

  2. The dataset provides diverse interactions and rich supervision. DeformFieldBench pairs multi-view videos with material labels and dense physical responses across pressing, stretching, shearing, bending, and dropping. Its object-level train/test split provides a useful setting for studying material inference on unseen objects.

Critical weaknesses
  1. The assumptions underlying the physics-consistency loss are insufficiently justified.

Equations (19)–(21) in Appendix L.5 encourage positive correlation between stress and motion magnitudes, as well as higher average stress and motion magnitudes inside contact regions than outside them. However, these statistical relationships are not generally valid mechanical constraints. For example, during elastic recovery or oscillation, stress may be high near maximum deformation while velocity is close to zero; near equilibrium, velocity may be high while stress is relatively low. Stress concentrations also need not coincide with actuator-contact regions.

Since the dataset explicitly includes release, rebound, and oscillation, these priors may conflict with some of the physical responses represented in the data. Although the authors describe them as weak constraints, their applicability across loading modes, motion phases, and the spatial or temporal domains over which correlation is computed is not sufficiently established. The paper also does not isolate the effects of these regularizers. It therefore remains unclear whether they provide reliable physical guidance or introduce biases that conflict with ground-truth field supervision in some settings.

  1. The presentation of the core experiments is overly compressed, making the connection between the proposed designs and their supporting evidence difficult to assess.

Section 5.2 briefly combines physical-field prediction, inference-time field masking, and component ablations, with complete results primarily presented in Appendix Tables 3, 4, and 7. These experiments address distinct questions: whether the fields can be predicted accurately, whether the trained model uses them, and whether other design components are beneficial. However, the main text does not sufficiently distinguish their objectives, controls, and conclusions. The precise modifications made in some ablation variants are also described only briefly.

The concern is not the placement of results in the appendix itself, but the limited explanation of the experimental reasoning supporting the central contribution.

  1. The material-parameter comparison in Section 5.1 has limited coverage, and the VLM baseline lacks essential implementation details.

Table 1 compares the proposed method only with the authors’ VLM and supervised baselines, without directly evaluating published related methods under the same parameter-estimation protocol. These results support improvements over the two specific implementations, but do not establish broader competitiveness in visual material-parameter estimation.

The supervised baseline, which shares the DINO backbone and multi-view fusion, is a meaningful comparison. In contrast, the VLM baseline does not specify the model name and version, frame count and sampling strategy, multi-view input organization, prompting setup, or whether it underwent fine-tuning or task-specific adaptation on the training data. These omissions limit reproducibility and prevent verification of the comparison conditions.

If the VLM operates in a zero-shot setting while the proposed method receives task-specific supervision, their performance gap alone cannot establish the effectiveness of physical-field bridging. The paper does not clearly establish whether this baseline is indeed evaluated zero-shot.

  1. The independent contribution of the core physical-field bridging mechanism is not sufficiently isolated.

The supervised baseline predicts both physical fields and material parameters, but simultaneously omits the physics bottleneck and field-conditioned regression. Consequently, its comparison with the full model cannot uniquely attribute the improvement to feedback from predicted fields.

Table 4 masks fields at inference time without retraining, demonstrating that the trained regressor depends on these inputs. However, this does not establish whether a model trained without field feedback, while retaining the remaining architecture and supervision, could achieve comparable performance.

Table 7 examines boundary reasoning, auxiliary field learning, and staged training, but does not clearly report a controlled variant that retains the physics bottleneck, field prediction, and field supervision while removing only the feedback to the material-prediction heads. The results therefore support the effectiveness of the complete system, but provide insufficient evidence for the independent benefit of its central bridging mechanism.

Questions for authors
  1. Under what loading conditions and motion phases do the assumptions in Eqs. (19)–(21) hold? Over which spatial and temporal dimensions is correlation computed? What is the effect of removing these regularizers, particularly during release and recovery?

  2. Which VLM model and version are used? How are frames sampled, multiple views organized, prompts constructed, and numerical outputs parsed? Is the VLM fine-tuned or otherwise adapted using DeformFieldBench?

  3. Are the supervised baseline and the full model matched in decoder capacity, loss configuration, training budget, and staged optimization? What exactly is retained or removed in each Table 7 ablation?

  4. How does performance change when the physics bottleneck and auxiliary field supervision are retained, but feedback from predicted fields to the material-prediction heads is removed and the model is retrained?

Reviewer_AKYFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces DeformFieldBench, a dataset of 5,000 3DGS–MPM simulations over 100 object shapes and five loading modes. Each sample pairs three-view RGB deformation videos with object and force/contact masks, projected motion-flow and stress-response fields, and simulator material labels. A feed-forward model predicts the physical fields and uses them with a physics bottleneck to infer one global constitutive family and material-parameter vector per object. Experiments evaluate parameter and field prediction, same-simulator replay, external benchmarks, additional objects and interactions and input robustness.

Critical strengths
  1. The dataset and model offer a useful way to study material inference from deformation. Aligned contact, motion and stress-response supervision makes the intermediate evidence explicit. The model is technically reasonable, jointly predicting three physical fields alongside the physical parameters. Directly predicting physical parameters in videos is also a timely task.

  2. The proposed method exceeds pure VLMs and its ablative model. The evaluation goes beyond parameter error to examine replay and transfer. Same-simulator replay is an interesting and practical evaluation setting, which also demonstrates improvement. Additional objects, held-out interactions and five real sequences provide useful checks of generalization.

Critical weaknesses
  1. The dataset’s physical fidelity needs stronger validation. The paper and Appendix describe construction and limitations, but no calibrated validation of the generated physical responses is provided. The benchmark uses homogeneous objects and normalized projected stress; The 3DGS-MPM only supports deformation simulation, which covers only a very limited range of object dynamics and physical parameters. This leaves fracture-related simulation totally unrelated. Its original simulator configuration produces rebound below approximately 5% of the initial drop height, whereas several Spring-Gaus objects rebound above 60%, even after sweeping the dataset’s material ranges. This shows a substantial configuration gap and we do not know which simulator is physically correct. A representative quality check is essential to establish where the generated supervision is reliable.

  2. Comparisons on DeformFieldBench remain limited to a VLM estimator and a supervised variant sharing the proposed model’s backbone. PIXIE is discussed but not evaluated. Some baselines are missing, such as OmniPhysGS (ICLR 25), VoMP (ICLR26), and AutoPhyX (ECCV 26). Although they are not video-oriented, using the first frame should provide reasonable baselines. Independent learned baselines on DeformFieldBench would better establish the proposed method’s position. The paper also lacks real-world qualitative examples nor quantitative evaluation of the parameter prediction.

Questions for authors
  1. Can you report an ablation that removes only field-to-regressor feedback while keeping the physics bottleneck and field supervision? This would determine how much of the improvement is attributable to the proposed bridge.

  2. Please provide the numerical training configuration for each stage, including frame sampling, loss weights, optimizer and stage lengths.

Reviewer_a1nGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

There has been a lot of research on estimating geometry and physical parameters from images and videos. However, most of these methods require per-scene optimization, while a single-pass approaches, which do not require per-scene optimization, remain underexplored. The paper proposes a dataset, DeformFieldBench, to help develop and test single-pass networks. In addition, the paper proposes a new architecture that explicitly takes flow, force, and stress into account and uses them to estimate material parameters. Through experiments, the paper shows that the proposed methods improve performance over the baselines.

Critical strengths

The paper shows that explicitly guiding networks to estimate flow, force, and stress, and then to estimate material parameters, is better than estimating material parameters directly. Since material parameters are highly dense and compact representations that may be difficult for networks to process and predict accurately, this idea of taking an easier, indirect route would be helpful to the community.

Critical weaknesses

The main focus of both the dataset and the network architecture is to obtain better material parameters. However, the paper does not show a clear improvement. In particular, the qualitative results (Fig. 4) do not clearly show whether the network is actually learning something about materials. The relationship between the ground truth and the predictions should be linearly correlated, but they appear closer to being independent. Even the quantitative results do not show a clear improvement.

Questions for authors

Would it be possible to explain the advantages and benefits of the proposed dataset and architecture in more detail? I might have missed them, but since the quantitative and qualitative results do not show a clear improvement, a clearer explanation of the contribution would be helpful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.