- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Phi-Art generates articulated 3D assets from a single image by fine-tuning a multimodal language model to predict part layouts, kinematic structures, physical attributes, and coarse whole-object geometry. A frozen OmniPart model then generates textured part meshes, which are assembled into URDF/MJCF assets. Experiments evaluate reconstruction quality, structural and physical predictions, and qualitative robotic interactions in simulation.
Critical strengths
-
The framework produces textured articulated assets with part structure, joint parameters, and physical attributes from a single image, with URDF/MJCF export for simulation.
-
Evaluation on 1,310 objects demonstrates improved geometric reconstruction and part localization, supported by training-data and compute-scaling studies.
Critical weaknesses
-
Limited methodological novelty and insufficient justification. The method combines a fine-tuned ShapeLLM, parallel coarse-geometry prediction, and frozen OmniPart synthesis. The incremental contribution remains insufficiently established, particularly without a controlled comparison against pretrained or fine-tuned TRELLIS using identical part layouts, masks, and downstream synthesis.
-
Incomplete comparisons with recent methods. Comparisons with PWM-ArtGen [1], PhysX-Omni [2], and image-conditioned Articraft [3] are missing. ArtLLM [6], SIMART [5], and Particulate [7] could provide additional baselines with a common image-to-3D frontend. Sketch2Arti [4] and LAM [8] also warrant discussion, while acknowledging their different input requirements.
-
Limited gains in articulation and physical prediction. In Table 2, Phi-Art’s kinematic score is lower than those of PAct and PhysX-Anything (0.68 versus 0.73 and 0.74), and drops to 0.62 with automatic masks. Material, affordance, and description scores also remain below PhysX-Anything, leaving the benefits beyond geometry and scale unclear.
-
Insufficient quantitative validation of physical usability. VLM-based motion scores and rendered property-map PSNR do not directly establish accurate joint parameters or reliable physical behavior. The simulation examples are qualitative; quantitative evaluations of joint accuracy, interpenetration during motion, and manipulation success are missing.
-
Ablations do not sufficiently isolate the proposed contributions. Removing the Q-Former also switches geometry generation from parallel to autoregressive prediction, confounding architecture and decoding strategy. A matched image-only geometry decoder without VLM conditioning would help establish whether the proposed semantic–geometric coupling is necessary.
References
[1] PWM-ArtGen: Part World Model for Articulated Object Generation.
[3] Articraft: An Agentic System for Scalable Articulated 3D Asset Generation.
[4] Sketch2Arti: Sketch-based Articulation Modeling of CAD Objects.
[5] SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLM.
[6] ArtLLM: Generating Articulated Assets via 3D LLM.
Questions for authors
-
What is the key methodological advance beyond combining ShapeLLM, parallel geometry decoding, and OmniPart? Would pretrained or fine-tuned TRELLIS achieve comparable results under the same part layouts, masks, downstream synthesis, and training budget?
-
Could you compare with PWM-ArtGen, PhysX-Omni, and image-conditioned Articraft, and consider ArtLLM, SIMART, and Particulate with a common image-to-3D frontend? Please also clarify the relationship to Sketch2Arti and LAM, acknowledging their different input requirements.
-
How should we interpret the lower kinematic score than PAct and PhysX-Anything, and the lower material, affordance, and description scores than PhysX-Anything? What evidence establishes benefits in articulation and physical prediction beyond improved geometry and scale?
-
Could you provide quantitative evaluations of joint types, axes, positions, and motion limits, together with interpenetration rates or manipulation success rates? Which predicted physical attributes actually affect the simulated behavior, and how is their accuracy validated?
-
Could you disentangle the effects of the Q-Former architecture and parallel decoding in the ablations? In particular, how does a matched image-only geometry decoder without VLM conditioning perform, and does semantic conditioning provide a measurable benefit?
We thank the reviewer for the careful reading and constructive feedback. We apologize for the confusion and provide the following clarifications.
1. Visible-part segmentation (Fig. 2, Round 1). The VLM does not directly predict pixel-level segmentation masks. Instead, it predicts 2D bounding boxes of visible object parts using 128-bin coordinate quantization. These boxes are then used as prompts for SAM3 to obtain pixel-level part masks, which condition the downstream OmniPart generation stage.
2. The <mesh-out> token (Sec. 3.1). <mesh-out> is a registered special token that serves as an anchor for extracting a geometry-conditioning representation from the VLM. Specifically, we extract its associated final-layer hidden state (3584 dimensions) and project it to 1024 dimensions. This representation is used as conditioning memory in the subsequent fusion decoder. Its representation is indirectly optimized through geometry-code prediction supervision.
3. Conditioning queries and geometry queries (Sec. 3.2). These two query groups serve different purposes. The 128 conditioning queries aggregate semantic and structural information from the VLM, while 64 DINO queries aggregate visual evidence from the input image. A fusion decoder integrates these representations, with the conditioning queries attending to 193 memory tokens (128 conditioning tokens, one <mesh-out> representation, and 64 DINO tokens). Then, the 1024 learned queries, equipped with 3D positional embeddings, are used by geometry decoder to predict the fixed-length sequence of 1024 VQ-VAE codes in parallel. This separation decouples multimodal condition aggregation from detailed geometry decoding.
4. Trainable and frozen components (Sec. 3.3). We fully fine-tune the VLM language transformer and train its multimodal merger/projector and the Q-Former. The VLM visual patch embedding and visual transformer remain frozen. DINOv3 features are precomputed, and DINOv3, the VQ-VAE codebook and decoder, OmniPart, and SAM3 are all frozen. The two VLM rounds share the same parameters and training pipeline. Language loss supervises the response tokens, while geometry losses propagate through the geometry-conditioning pathway to the trainable components.
We appreciate the reviewer for identifying these gaps in the method description. We hope these clarifications make the architecture and its implementation easier to follow.