- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper proposes PhysAg3D, a framework for generating physically interactive 3D assets from a single image. The main idea is to combine a pretrained 3D generator, which generates a detailed textured 3D object, with a VLM-based agent that reasons about the semantic and physical structure of the object. The agent identifies how different parts should move or deform and uses geometry and simulation tools to construct parts, joints, deformable regions, and their connections to the visible surface. The generated interactions are then evaluated using visual and physical feedback, and the agent can revise the asset when the expected interaction is not achieved. The experiments show improvements over existing methods in both preserving the appearance of the input object and generating plausible interactions.
Critical strengths
The paper addresses an interesting and important problem of generating 3D assets that are not only visually similar to the input image but can also physically interact. Combining the detailed geometry and appearance from recent 3D generative models with agentic reasoning for articulation and deformation is interesting. I also like the idea of preserving the generated surfaces while adding physical interactions and using visual and physical feedback to revise the result. The paper has done decent evaluations, with comparisons on multiple benchmarks, a human study, and ablations, and the reported improvements over existing methods are good.
Critical weaknesses
My main concern is that the contribution appears to be primarily a system that combines several existing strong components rather than introducing a sufficiently novel technical method. The 3D object is generated using TRELLIS.2, a VLM is used for reasoning about the object and its interactions, and existing geometry and simulation tools are then used to construct and evaluate the physical asset. The hierarchical representation and iterative refinement organize these components, but I am not convinced that this provides enough technical novelty beyond the overall system design. I would like the authors to clarify what is non trivial in this pipeline that gives extra insights to the readers.
A second major concern is that the experiments do not sufficiently demonstrate the difficult multi-part setting that motivates the approach. TRELLIS.2 initially generates a whole-object mesh, after which the system needs to identify and construct the individual functional parts. Most examples shown in the paper contain relatively clear and externally visible parts, such as a lid, telescope joints, mixer head, excavator components, or camera lens.
I would like to see substantially more difficult examples such as furniture, lamps, or mechanical objects containing many small, nested, occluded, or completely hidden parts. In particular, it is unclear what happens when an important functional part is not represented in the initial generated geometry at all. Without such experiments, I am not convinced that the method generalizes beyond objects with relatively obvious functional decomposition.
Questions for authors
What is the main technical novelty beyond combining a 3D generator, VLM reasoning, geometry/physics tools, and an iterative agentic refinement loop?
How does the method handle objects containing many functional parts, particularly small, nested, or occluded components?
What happens when a functional part is completely hidden in the input image and consequently is not represented in the TRELLIS.2 reconstruction? Can the system actually generate the missing geometry?
Could you provide experiments on substantially more complex objects, such as furniture, lamps, or mechanical objects with many interacting components?
How much of the reported performance comes from the proposed hierarchical representation itself versus the underlying VLM, TRELLIS.2, and iterative tool-use pipeline?