- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PhysForge, a framework for reconstructing simulation-ready 3D assets with controllable material behavior and separable semantic components. PhysForge integrates two key modules: PhysFit and PhysComp. PhysFit uses dynamics generated by a material-conditioned video diffusion model as reference motion, estimates physical properties through windowed fitting, and handles nonphysical motion through a Hallucinated Forcing mechanism. PhysComp transfers object-part identities from video to the reconstructed 3D Gaussian geometry, organizes these components as simulated rigid bodies, and enables independent optimization of their physical parameters followed by joint simulation. This design allows the same object to exhibit distinct material-dependent responses under different material prompts while preserving the motion and interaction characteristics of individual semantic components.
Critical strengths
PhysForge supports user-specified materials such as wood, silicone, fabric, metal, plastic, rubber, liquid, and sand, and allows the same object to exhibit different material-dependent behaviors. It also preserves component-level motion, e.g., bottle, glass, and table moving separately.
Critical weaknesses
-
The quality of physical-parameter fitting relies on the generated video’s physical plausibility. Although Hallucinated Forcing mitigates nonphysical motion, it does not fully solve the problem, and severe VDM hallucinations may still bias the fitted parameters.
-
Each material condition requires additional video generation and physical fitting. This makes PhysForge less efficient than feed-forward or single-pass asset-generation methods, and limits scalability to many material variations or large asset libraries.
Questions for authors
-
Recently, several methods have used VLMs to achieve soft-body/deformable object deformation, such as PhysX-CoT and PhysX-Omni. The authors should analyze the differences from and relationship to these methods in the related work.
-
I feel that this pipeline is more like an agentic framework built on existing models, and thus it tends to perform better when compared with methods based on a single model. However, I hope the authors can provide corresponding evaluations, such as efficiency.
5 comments
Response to reviewer 5Mw2 (2/3)
-
W2: Each material condition requires additional video generation and physical fitting. This makes PhysForge less efficient than feed-forward or single-pass asset-generation methods, and limits scalability to many material variations or large asset libraries.
Our generation-and-fitting pipeline supports material-controllable variation for the same asset, a capability absent from the evaluated observation-driven PhysX predictors. PhysX-Omni [1] and PhysX-Anything [2] predict material properties from the appearance of the object, whereas PhysForge fits material properties according to the user’s prompt. Given the same apple image, a user may request hardwood or silicone gel. The PhysX methods produce the same material prediction for both requests, whereas PhysForge fits distinct material properties to produce the requested hard or soft behavior. This use of dynamic priors follows the optimization-based direction of DreamPhysics [3] and MotionPhysics [4].
We report material controllability together with runtime:
| Method | Material controllability | Runtime (min) |
|---|---|---|
| PhysX-Omni | ✗ | 16.69 ± 1.89 |
| PhysX-Anything | ✗ | 11.82 ± 1.90 |
| PhysForge | ✓ | 11.75 ± 0.56 |
PhysForge supports user-specified materials with a runtime close to PhysX-Anything and lower than PhysX-Omni. The difference in capability is visible in Figure 7 in Appendix (newly added): the PhysX methods retain the same behavior across the hard and soft conditions, while PhysForge produces a largely rigid apple under the hard condition and pronounced compression under the soft condition.
References
[1] Cao, Ziang, et al. "PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects." arXiv preprint arXiv:2605.21572 (2026).
[2] Cao, Ziang, et al. "PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[3] Huang, Tianyu, et al. "DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 4. 2025.
[4] Wang, Miaowei, et al. "MotionPhysics: Learnable Motion Distillation for Text-Guided Simulation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 12. 2026.
-
Q1: Recently, several methods have used VLMs to achieve soft-body/deformable object deformation, such as PhysX-CoT and PhysX-Omni. The authors should analyze the differences from and relationship to these methods in the related work.
The distinction is that PhysForge is a material-controllable generation method, whereas PhysX methods are observation-driven material predictors. PhysX-CoT [1], PhysX-Omni [2], PhysX-Anything [3], and PhysGM [4] predict an object’s physical parameters from an image that provides an observation of its appearance. PhysForge instead takes images of different objects and fits physical parameters according to the user’s material requests. Its goal is to give the reconstructed objects the requested physical behavior, rather than recover the material implied by their appearance.
We compare with PhysX-Omni and PhysX-Anything. PhysX-CoT is built on PhysX-Anything. Since the dedicated LoRA weights required by PhysX-CoT were unavailable for our experiments, we use PhysX-Anything as the alternative baseline.
| Method | Material controllability | Common CLIP ↑ | Common PC ↑ | Uncommon CLIP ↑ | Uncommon PC ↑ |
|---|---|---|---|---|---|
| PhysX-Omni | ✗ | 0.249 | 3.8 | 0.247 | 2.6 |
| PhysX-Anything | ✗ | 0.249 | 2.2 | 0.248 | 1.4 |
| PhysForge | ✓ | 0.258 | 4.6 | 0.264 | 4.0 |
PhysForge achieves higher CLIP and PC scores under both common and uncommon material conditions. In Figure 7 in Appendix (newly added), PhysX-Omni and PhysX-Anything produce unchanged behavior across the two requested materials, and the PhysX-Anything result has incomplete geometry. PhysForge preserves the apple appearance while producing the hard and soft deformation behaviors requested by the user.
References
[1] Huang, Jie, et al. "PhysX-CoT: Structured Physical Reasoning from a Single Image to Simulation-Ready 3D Assets." arXiv preprint arXiv:2608.08053 (2026).
[2] Cao, Ziang, et al. "PhysX-Omni: Unified Simulation-Ready Physical 3D Generation for Rigid, Deformable, and Articulated Objects." arXiv preprint arXiv:2605.21572 (2026).
[3] Cao, Ziang, et al. "PhysX-Anything: Simulation-Ready Physical 3D Assets from Single Image." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[4] Lv, Chunji, et al. "PhysGM: Large Physical Gaussian Model for Feed-Forward 4D Synthesis." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
Response to reviewer 5Mw2 (3/3)
-
Q2: I feel that this pipeline is more like an agentic framework built on existing models, and thus it tends to perform better when compared with methods based on a single model. However, I hope the authors can provide corresponding evaluations, such as efficiency.
PhysForge centers on fitting of material parameters and component-level simulation; pretrained models only supply perception, geometry, and priors. Using pretrained priors is also part of the evaluated DreamPhysics [1] baseline and the related MotionPhysics [2] approach. Our method contributes PhysFit, which separates incoming-motion alignment from material fitting, and PhysComp, which preserves independent component states and interactions.
The PhysFit ablation in Table 3 (newly added) isolates the two fitting mechanisms. Loss in the following table measures how closely the simulated contact and deformation match the generated reference video; a lower loss indicates better fitting to the reference dynamics.
| Window selection | Hallucinated Forcing | Loss ↓ |
|---|---|---|
| ✗ | ✗ | 0.99154 |
| ✓ | ✗ | 0.14242 |
| ✓ | ✓ | 0.05864 |
The reduction in loss shows the contribution of window selection and Hallucinated Forcing. Figure 6 in Appendix (newly added) illustrates the corresponding differences in collision and deformation. Table 4 (newly added) and Figure 5b (refined) further show improved component trajectories and independent motion with PhysComp.
We also report runtime directly:
| Method | Runtime (min) |
|---|---|
| DreamPhysics | 30.06 ± 11.07 |
| PSIVG | 15.24 ± 4.74 |
| WMReward | 12.08 ± 0.08 |
| PhysX-Omni | 16.69 ± 1.89 |
| PhysX-Anything | 11.82 ± 1.90 |
| PhysForge | 11.75 ± 0.56 |
From the table above, we can see that PhysForge has a runtime close to WMReward and PhysX-Anything, and lower than DreamPhysics, PSIVG, and PhysX-Omni. Together with the ablations, these results show that the improved material control and component separation come from the proposed fitting and simulation mechanisms at a comparable or lower measured runtime than these methods.
References
[1] Huang, Tianyu, et al. "DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 4. 2025.
[2] Wang, Miaowei, et al. "MotionPhysics: Learnable Motion Distillation for Text-Guided Simulation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 12. 2026.
Manuscript revisions
In the revised manuscript, red marks changes to the original content, and blue marks newly added content.
To address your concern about video hallucinations, we have clarified supervised video generation in Section 3.2 and added the fitting ablation in Table 3 (newly added) on page 9, with the qualitative comparison in Figure 6 in Appendix (newly added), Appendix A.2, page 14. For your questions about material-generation cost and the relationship to PhysX methods, we have clarified the distinction in Section 2.1, introduced the PhysX baselines in Section 4.1, and added their results to Table 1, Figure 7 in Appendix (newly added) (Appendix A.2, page 14), and the runtime comparison in Table 6 (newly added) on page 9. To address whether the gains come from our proposed mechanisms, Section 4.2 now analyzes the PhysFit ablation and the PhysComp results in Table 4 (newly added) and Figure 5b (refined), both on page 9, together with the runtime comparison.
Response to reviewer 5Mw2 (1/3)
Thank you for recognizing material controllability and component-level motion. We address each concern below.
We would like to clarify that, compared with previous works, PhysForge is the first to explicitly discuss how video hallucinations affect physical-parameter fitting. To address this issue, we propose window selection and Hallucinated Forcing, combined with supervised video generation. Window selection concentrates fitting on contact and deformation, and Hallucinated Forcing aligns the incoming motion state. Eliminating video hallucinations remains challenging for methods that rely on generated dynamics, including Vid2Sim [1], DreamPhysics [2], and MotionPhysics [3], and our approach mitigates their influence rather than fully resolving the problem. Nevertheless, Table 1 and Figure 3 show improved material controllability: PhysForge achieves common/uncommon PC scores of 4.6/4.0, compared with 3.2/2.6 for DreamPhysics. Loss in the following table measures how closely the simulated contact and deformation match the generated reference video; a lower loss indicates better fitting to the reference dynamics.
Window selection isolates the relevant interaction, while Hallucinated Forcing aligns the incoming state; enabling both yields the lowest loss. In Figure 6 in Appendix (newly added), fitting the full video produces watermelons that behave too stiffly, indicating that motion outside the interaction can interfere with material fitting. Window selection alone produces a slow collision with insufficient deformation: a soft material may deform only slightly when the incoming velocity is too low. Hallucinated Forcing addresses this ambiguity by aligning the incoming state, allowing the fitted material to reproduce the requested soft-material behavior. The quantitative and qualitative results therefore support the complementary roles of window selection and Hallucinated Forcing in material fitting.
References
[1] Chen, Chuhao, et al. "Vid2Sim: Generalizable, Video-Based Reconstruction of Appearance, Geometry and Physics for Mesh-Free Simulation." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
[2] Huang, Tianyu, et al. "DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 4. 2025.
[3] Wang, Miaowei, et al. "MotionPhysics: Learnable Motion Distillation for Text-Guided Simulation." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 12. 2026.