acceptodds
Under review as a conference paper at ICLR 2027

ShapeMind: Native 3D Understanding and Generation of Semantically Structured 3D Assets

Abstract

Production-ready 3D assets demand more than visual fidelity: they require semantically structured geometry for editing, composition, and interaction in applications such as embodied simulation. We present ShapeMind, a framework that generates low-poly 3D assets with explicit semantic part structure and, where applicable, kinematic structure. Following the principle of “think before you generate”, ShapeMind couples ShapeVLM for 3D semantic understanding with Semantic PartGen for native mesh generation. ShapeVLM combines visual evidence with coarse geometry encoded as compact text within the existing vocabulary of a vision-language model (VLM). This design reuses pretrained semantic and commonsense knowledge for 3D structural reasoning without introducing a heterogeneous modality. The resulting understanding of object composition and kinematics provides the structural basis for Semantic PartGen to synthesize high-quality low-poly meshes organized into meaningful parts. Its part-aware vertex and topology generation jointly models the object while preserving each part as an independently editable mesh. To support this conditional generation task, we curate 300K semantically annotated low-poly assets. Experiments demonstrate state-of-the-art results in both 3D object semantic understanding and structurally coherent part geometry generation.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
3D generation, Vision-Language Model

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2KE3OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ShapeMind, a framework that generates low-poly 3D assets with explicit semantic part structure and, where applicable, kinematic structure. It couples ShapeVLM, which handles 3D semantic understanding, with Semantic PartGen, which performs native part-level mesh generation.

Critical strengths

(1) The paper introduces ShapeCodec, a compact contour-based representation that expresses 3D geometry within a VLM’s existing text vocabulary. It pairs this with ShapeVLM, a two-turn supervised VLM that couples semantic structure prediction with part geometry reconstruction.

(2) The paper introduces an octree forest with part-local topology latents, designed for native generation of multi-part meshes.

(3) Experiments on multiple benchmarks show state-of-the-art performance in both 3D semantic understanding and part-mesh generation.

Critical weaknesses

(1) Much of the framework builds on existing techniques: (a) ShapeVLM fine-tunes an existing VLM (Qwen3-VL-8B) with standard supervised training. Its main novelty lies in how the geometry is encoded rather than in the model itself. (b) Semantic PartGen adopts Nexus’s vertex–topology factorization, frozen mesh VAE, and face recovery algorithm. (c) The scoped part/object attention follows CubePart. The authors should state more precisely what is new, such as the octree forest, the tri-stream conditioning, and the box re-fitting in stage 2. And also should show the individual impact of each component by providing the ablation experimental results.

(2) The paper provides Figure 4 on Semantic PartGen, but it doesn’t provide detailed structural information about MMDiT. It would help to explain: (a) how the MMDiT blocks differ from standard DiT blocks in this design; (b) how part-scope and object-scope attention are implemented and arranged across layers; (c) how both differ from standard full self-attention. An ablation comparing scoped attention with standard full attention would also clarify its benefit.

(3) The paper would benefit from a discussion of whether the approach could extend to more complex settings, such as multi-object compositions or indoor and outdoor scenes.

(4) The comparisons could be broadened. For example, X-Part is discussed in the related work but not compared experimentally. Comparing with native mesh generators combined with part segmentation would also better support the low-poly mesh quality claims.

(5) The 64³ grid limits the geometric detail available to ShapeVLM, and the average of about 6K tokens per object will grow at higher resolutions. In addition, the baseline comparisons use only objects with at most eight parts. The authors should discuss how the method scales with resolution and part count, and include representative failure cases.

Questions for authors

(1) How do the Table 2 results change when the coarse geometry G₀ comes from an image-to-3D model rather than ground-truth geometry?

(2) How sensitive is Semantic PartGen to noise in the input boxes and to the vertex-count condition N?

(3) What is the end-to-end performance on PartObjaverse-Tiny when using ShapeVLM-predicted boxes and descriptions instead of ground-truth conditions?

3 comments

Thank you for recognizing our representation and native part generation, and for your constructive suggestions. All figures refer to the revised paper.

Novelty and component ablations (W1).

Our contributions are ShapeCodec, the octree forest, and tri-stream MMDiT; Appendix B.1 provides implementation details. Revised-paper Table 1 ablates ShapeCodec, while the octree forest extends holistic mesh representations to multiple parts. The table below compares our tri-stream design with a standard dual-stream MMDiT that concatenates text with other conditioning tokens. We also ablate box re-fitting—using boxes fitted to S1 vertices for S2—a design choice rather than a core contribution.

Architecture/box conditioning: 197 PartObjaverse-Tiny objects.

VariantPart-CD ×10³ ↓Obj-CD ×10³ ↓[email protected] ↑[email protected] ↑
Dual-stream25.2032.670.955810.95887
S2 GT boxes (oracle)22.6830.220.963580.96779
full (tri-stream)22.6930.260.963400.96755

MMDiT implementation (W2).

Revised Appendix B.1 details the token organization and attention mechanisms.

Scene extensions (W3).

Appendix F (Figures 17–18) shows eight PartGen scenes from synthetic images and supplied boxes/text. Complex scenes retain coherent layouts. Extending object-trained ShapeVLM to scenes is future work.

Broader comparisons (W4).

ArtLLM uses XPart for geometry generation; we have revised the paper to make this clearer. Paper Table 3 includes HY2.1 as native 3D generator, and PatchAlign3D + HoloPart and OmniPart as part segmentation and completion.

Resolution and part count (W5).

The 64³ grid balances efficiency and accuracy: voxel precision suffices for object-level articulation/bboxes; finer grids increase ShapeCodec tokens.

Ours leads all quality metrics on 137 objects with >8 parts; generation supports up to 32 parts.

137 PartObjaverse-Tiny objects with >8 parts.

MethodObj-CD ×10³ ↓[email protected] ↑Part-CD ×10³ ↓[email protected] ↑Mean triangles
Original baseline meshes
PartPacker61.060.82487166.210.48633984,607
PartCrafter108.610.60673295.680.189703,418,154
PatchAlign3D + HoloPart48.60.89826148.930.553232,041,818
↳ w/ Oracle 3D bbox46.020.9089063.450.789825,749,607
↳ w/ Oracle mesh––118.840.636312,535,475
Oracle Seg-mask + OmniPart48.220.89815100.140.70869402,108
↳ w/ Oracle 3D bbox46.210.9081972.980.80133427,920
↳ w/ Oracle mesh––50.180.86216397,697
Ours25.320.9823215.290.9828816,563

Image-generated coarse geometry (Q1).

Geometry-dependent methods share HY2.1 meshes; PartGen uses predicted conditions and Nᵥ = 4000. Pose/scale discrepancies reduce spatial scores; type/graph/range remain relatively stable. Ours leads five of eight metrics, including both mesh scores.

PartNet-Mobility: GT vs. HY2.1. Bold/italic: best/second per block.

MethodmIoU ↑Type ↑Axis (rad) ↓Pivot ↓Range ↑Graph ↑Part-CD ×10³ ↓Part-F @.01 ↑
GT coarse geometry: original-paper Table 2 protocol
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.58950.76440.48120.17940.87430.584429.7550.3415
SIMART0.65430.83770.41500.19540.55180.7792––
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.81400.93280.18700.04950.80500.805217.4800.3842
HY2.1 coarse geometry: image-driven pipeline
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.51890.82030.39960.09190.82750.590934.5010.2074
SIMART0.42720.81080.20370.27190.59850.742432.3540.1585
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.63670.93790.20950.10990.78500.803027.9230.2692

Sensitivity to boxes and vertex count (Q2).

Halving/doubling vertex counts changes quality metrics by at most 1.3%. Our model is sensitive to bbox changes, enabling control over part positions and extents (Figure 15).

PartObjaverse-Tiny condition sensitivity.

ConditionΔObj-CD (%)ΔPart-CD (%)ΔPart-F (%)Triangle ratio
Vertex-count perturbations
Nᵥ = Nᵥ(GT)/8+6.0+11.5-0.40.12×
Nᵥ = Nᵥ(GT)/2-0.3+0.9+0.00.49×
Nᵥ = 2Nᵥ(GT)+0.4+1.3-0.01.99×
Bbox perturbations
Center shift ±30%+151.5+231.9-14.7–
Size scale [0.7,1.3]+52.3+71.1-2.9–

Predicted conditions on PartObjaverse-Tiny (Q3).

ShapeVLM predicts coarser parts than GT due to its training annotations. PartGen preserves coherent geometry across different decompositions (Figures 11–12).

GT/predicted part counts.

ExampleGTPred.ExampleGTPred.
Character1414Engine108
Vehicle167Helicopter169
Cactus54Stylized emitter73
Tank115Robot109

I thank the authors for their detailed rebuttal and the additional experiments. These have addressed several of my concerns. However, the following issues remain:

W1 (Novelty and component contributions). The response lists ShapeCodec, the octree forest, and the tri-stream MMDiT as the main contributions, but the revised manuscript still does not delineate which components are new and which are inherited from Nexus and CubePart. More importantly, the octree forest, presented as a core contribution, is not ablated. A comparison with a direct extension of Nexus (e.g., per-part generation with box conditioning) is needed to establish its benefit.

W2 (Architecture clarity). Appendix B.1 clarifies the block design and attention masks. As a suggestion, a figure that contrasts the MMDiT block with a standard DiT block, and part-scope with object-scope attention, would make these differences easier to follow.

W5 (Failure cases). Errors are mentioned briefly in Appendices D.2 and F, but there is no systematic failure-case analysis. Such an analysis is important for understanding the method’s limitations, especially for ShapeVLM’s under-segmentation of complex objects.

Q1 (Image-generated geometry). The new results are informative. However, the HY2.1 block uses updated PartGen weights, so the drop in Part-CD/F cannot be attributed to the input geometry alone.

Overall, the rebuttal improves the clarity of the paper, but the contribution of the key novel component remains unverified. I therefore maintain my original score.

We appreciate your thorough and informative reviews, which help to clarify our work.

With respect to the remaining issues:

W1 (Novelty and component contributions)

Thank you for highlighting the need to distinguish our contributions more clearly. Our contributions are ShapeCodec, the octree-forest representation, and tri-stream MMDiT; These are not inherited from Nexus or CubePart. We reuse Nexus’s vertex–topology factorization, mesh VAE, and face recovery, and follow CubePart’s scoped attention design. We will explicitly distinguish these foundations from our contributions in the manuscript.

For the octree-forest comparison, independent box-conditioned part generation has foreseeable issues: limited cross-part context can lead to missing or duplicated geometry and inconsistent interfaces, complicating assembly. These motivate our joint representation. We have not conducted this comparison primarily due to limited computational resources and lack of access to Nexus’s official code and checkpoints.

W2 (Architecture clarity)

Thank you for this helpful suggestion. We will add a schematic contrasting our MMDiT block with a standard DiT block and illustrating part-scope versus object-scope attention in the camera-ready version.

W5 (Failure cases)

Thank you for emphasizing the need for a more systematic failure analysis. ShapeVLM can predict coarser decompositions than the reference annotations, merging fine components, as illustrated by the vehicle and tank in Figures 11–12. When semantic cues are ambiguous or unclear, PartGen may also interpret part descriptions inconsistently, potentially generating duplicated geometry across parts. We will expand these observations into a structured failure-case analysis in the camera-ready version.

Q1 (Image-generated geometry)

We have updated Table 2 in the main paper, containing two settings: with GT and HY2.1 coarse mesh. The HY2.1 coarse mesh numbers are updated with the same old checkpoint.

Reviewer_rLHoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents ShapeMind, a two-stage method designed for generating low-poly 3D assets with explicit semantic parts and kinematic structures. The first stage, ShapeVLM, involves fine-tuning the Qwen3-VL-8B model using an image and coarse geometry serialized as plain text through ShapeCodec. This codec is a compact, lossless encoding method based on contours that enables prediction of part boxes, labels, descriptions, and joint parameters. The second stage, Semantic PartGen, enhances the Nexus vertex-then-topology mesh generator to handle multiple parts. It does this by conditioning the mesh generation on the predictions from the first stage, using a geometry/box/text diffusion transformer that maintains each part as a separate, editable mesh. Results from experiments conducted on the PartNet-Mobility and PartObjaverse-Tiny datasets demonstrate the effectiveness of the proposed method.

Critical strengths
  • ShapeCodec encodes coarse geometry in a lossless manner as ordinary text, utilizing the VLM's existing vocabulary and using approximately 1.5 times fewer tokens than RLE.
  • Semantic PartGen directly generates vertices and connectivity for each part, resulting in compact, independent, and editable part meshes without the need for isosurface extraction or remeshing.
Critical weaknesses
  • The experiments do not fully support the claimed advantages.
    • The full pipeline is never tested end-to-end. In Table 2, ShapeVLM receives voxelized ground-truth geometry, not generated geometry (as stated in Lines 177-178: "which can be generated by an image-to-3D model"). Table 3 skips ShapeVLM and uses ground-truth bounding boxes. So the intended pipeline, i.e., image →\rightarrow coarse 3D →\rightarrow ShapeVLM →\rightarrow Semantic PartGen, is never evaluated quantitatively.
    • In Table 3, Semantic PartGen gets the image plus ground-truth part boxes, labels, descriptions, and a vertex count derived from the reference mesh, while most baselines get only the image or a mesh generated by HY 2.1, which incurs an unfair comparison.
    • It seems a bit unjustified to restrict the comparison of baselines only to objects with a maximum of 8 parts. Methods like OmniPart, PatchAlign3D + HoloPart, and the proposed Semantic PartGen can handle more parts. Therefore, it would be more appropriate to compare these methods across all objects without the eight-part restriction.
    • No results on real-world images. ArtLLM and PhysX-Anything have been tested on real-world images, while ShapeMind has not.
Questions for authors
  1. How were the results presented in Figures 6, 9, and 10 obtained? (a) For Figure 9, where do the part boxes, descriptions, and vertex counts come from? Are they based on ground-truth annotations or sourced from another location? (b) For Figures 6 and 10, is ShapeVLM’s coarse geometry based on voxelized ground truth or generated by an image-to-3D model? If it is the latter, which specific model was used?
  2. Can the authors provide some results on real-world images?
  3. What does the †\dagger in "Ours w/o text stream †^\dagger mean in Table 3?
6 comments

Thank you for recognizing our task and method. We add clearer comparative evidence below. All figures refer to the revised paper.

End-to-end evaluation (W1).

We evaluate image → Hunyuan3D-2.1 → ShapeCodec → ShapeVLM → PartGen, using predicted boxes/text and fixed Nᵥ = 4000. The table below reports structure and mesh quality (Appendix D.1). Figures 11–12 also show robustness to different predicted part decompositions on PartObjaverse-Tiny.

PartNet-Mobility: original GT-geometry results and HY2.1-input evaluation. Bold/italic indicate best/second within each block.

MethodmIoU ↑Type ↑Axis (rad) ↓Pivot ↓Range ↑Graph ↑Part-CD ×10³ ↓Part-F @.01 ↑
GT coarse geometry: original-paper Table 2 protocol
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.58950.76440.48120.17940.87430.584429.7550.3415
SIMART0.65430.83770.41500.19540.55180.7792––
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.81400.93280.18700.04950.80500.805217.4800.3842
HY2.1 coarse geometry: image-driven pipeline
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.51890.82030.39960.09190.82750.590934.5010.2074
SIMART0.42720.81080.20370.27190.59850.742432.3540.1585
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.63670.93790.20950.10990.78500.803027.9230.2692

Our mIoU and axis/pivot scores decline, while type, graph, and range remain relatively stable. Ours leads five of eight metrics, including both mesh scores.

Comparison fairness (W2).

Thank you for this critical comment. Our conditional-generation setting differs from the baselines, so Table 3 in the revised paper explicitly strengthens their inputs with oracle information. Following CubePart [1], we evaluate both whole-object generation and part quality, additionally providing HoloPart/OmniPart with GT boxes and input-pose alignment, and CubePart with GT part text. These oracle conditions substantially improve baseline scores, yet our Obj-CD is marginally lower.

We also supply GT whole-object meshes to these baselines, removing the need to generate overall shape and emphasizing partition quality. Our Part-CD is 0.02518 versus 0.12290/0.03444/0.12677 for their GT-mesh variants. Thus, the advantage persists even when baselines receive the target geometry.

More than eight parts (W3).

On 137 objects with >8 parts, Ours leads all object/part quality metrics, both against original baselines and after QEM decimation to our per-object triangle budget (below).

Part generation beyond eight parts: 137 PartObjaverse-Tiny objects.

MethodObj-CD ×10³ ↓[email protected] ↑Part-CD ×10³ ↓[email protected] ↑Mean triangles
Original baseline meshes
PartPacker61.060.82487166.210.48633984,607
PartCrafter108.610.60673295.680.189703,418,154
PatchAlign3D + HoloPart48.60.89826148.930.553232,041,818
↳ w/ Oracle 3D bbox46.020.9089063.450.789825,749,607
↳ w/ Oracle mesh––118.840.636312,535,475
Oracle Seg-mask + OmniPart48.220.89815100.140.70869402,108
↳ w/ Oracle 3D bbox46.210.9081972.980.80133427,920
↳ w/ Oracle mesh––50.180.86216397,697
Baselines decimated to our per-object triangle budget
PartPacker61.050.82470166.450.4857316,562
PartCrafter107.650.61452300.910.1910716,563
PatchAlign3D + HoloPart48.650.89811149.080.5531316,562
↳ w/ Oracle 3D bbox46.070.9086864.920.7848916,563
↳ w/ Oracle mesh––119.040.6395716,563
Oracle Seg-mask + OmniPart48.250.89774100.310.7069016,506
↳ w/ Oracle 3D bbox46.360.9072172.820.7987116,506
↳ w/ Oracle mesh––50.720.8616816,506
Ours25.320.9823215.290.9828816,563

Provenance of Figures 6, 9, and 10 (Q1).

Figures 6/10 use ShapeVLM-predicted conditions from GT coarse voxels to isolate structural understanding. Figure 9 uses GT boxes/text to fix the part-decomposition strategy, facilitating predicted-to-reference part matching for geometric metrics.

In-the-wild examples (W4; Q2).

Appendix D.3 (Figure 13) shows four in-the-wild examples with ShapeVLM-predicted names, text, and boxes, followed by PartGen meshes. Appendix E (Figure 16) compares against commercial low-poly generators at similar triangle budgets, showing detailed, structured part geometry.

Q3. Marker in the text-stream ablation.

The marker was a typo and has been removed.

[1] Zhu et al. CubePart: An Open-Vocabulary Part-Controllable 3D Generator. arXiv:2605.28763.

Thank you for the additional experiments. The end-to-end evaluation (Appendix D.1) and the >8-part comparison (Appendix E, Table 6) address W1 and W3. Some concerns remain:

  • W2 (unequal inputs): No new evidence. The oracle variants cited were already in the original Table 3. Table 7 shows high sensitivity to boxes (+232% Part-CD under ±30% center shift), so please scope the main-text claims to generation with oracle conditions.
  • Table 5: Please explain why 11 of the 77 PartNet-Mobility objects are excluded. The reused URDFormer and PhysX-Omni rows also come from different object sets.
  • Table 6: Please add CubePart.
  • Real-world images: Only four qualitative examples, one of which is a photograph. Which model generated their coarse meshes?
  • Main text: Please move the key end-to-end and >8-part results into the main paper. The main-text tables and claims are currently unchanged.

We appreciate your thorough and informative reviews, which help to clarify our work.

For your concerns:

W2 (Unequal inputs)

We admit that the comparison is not strictly fair. Since each method requires different conditions by deisign, an equivalent condition comparison is impracticable. To be consice, the following table shows the difference of input conditions of each baseline (two ✅ mean GT condition signal). Comparing to those strongest baselines, we trade-off the model capacity for better geometry generation, while relying on accurate part planning to ShapeVLM, that's the core insight "think before you generate". We have revised the main paper to clarify the unfairness.

methodsimgwhole-meshpart-text3D-box
PartPacker✅
PartCrafter✅
CubePart✅✅
CubePart + GT Mesh✅✅✅
GT seg + OmniPart✅✅✅
OmniPart + GT box✅✅✅✅
OmniPart + GT mesh✅✅✅✅
PatchAlign3D + HoloPart✅✅
HoloPart + GT box✅✅✅✅
HoloPart + GT Mesh✅✅✅
Ours✅✅✅✅

Table 5

Thank you for pointing out the ambiguity. The reference to 66 objects concerns URDFormer’s coverage within its five training categories; This coverage statement refers specifically to URDFormer. We have integrated the end-to-end results into main-text Table 2 and corrected this description.

Table 6

Table 6 is designed to evaluate the performance on PartObjaverse-Tiny objects with more than 8 parts. CubePart, according to official implementation, cannot support >8 parts. Nevertheless, we have revised the main paper Table 3 to contain results of both <=8 parts and >8 parts subset.

Real-world images

The coarse meshes comes from the commersial service Tripo Studio, model P2 (the commersial version of Nexus).

Main text

We have revised Table 2, Table 3, and corresponding claims in the main text.

Reviewer_FMyyOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper targets generating 3D assets from a single image that are split into meaningful, editable parts and stored as compact low-poly meshes, with joint information for movable objects. The authors frame their approach around a principle they call "think before you generate": the object's structure is inferred first and then used to guide mesh synthesis. In practice, they use a vision-language model that reads the image and a coarse shape to infer what parts the object has and where they are, and then a mesh generator that builds each part directly as a low-poly mesh conditioned on that inferred structure. The authors also curate a large part-annotated dataset, and they report better results than prior methods on both part/joint prediction and part generation.

Critical strengths
  1. The paper goes after a real problem with a clear idea. Most image-to-3D methods give one dense, fused mesh that is hard to edit or animate. The paper first works out what parts the object has, then builds each part as its own low-poly mesh with joint information.

  2. The way the model reads 3D shape is simple and general. The coarse shape is turned into plain text, so a standard vision-language model can read it without any new encoder; the encoding is lossless with respect to the filled voxel grid it encodes, and is much shorter than the obvious alternatives. This matters because it is easy to reuse and should keep improving as these models get better.

Critical weaknesses
  1. The main claim is not tested end to end. The reasoning model always reads a voxelized ground-truth shape, and the image-to-3D model meant to supply it is never named. On the part-generation benchmark, the generator also gets ground-truth boxes, text and vertex budget. Table 2 does feed predicted structure into generation, but on another benchmark, without the main baselines, and still from ground-truth voxels, so it does not close the gap. The paper is open about this, but since "infer structure, then generate" is the core claim, it weighs most in my rating.

  2. The low-poly advantage is only shown in pictures. It is the main selling point over prior part generators, yet there are no face counts or topology numbers, and the distance metrics used cannot tell a low-poly mesh from a dense one with the same surface. Without this, I cannot judge the benefit over an existing method plus standard simplification, or over the native-mesh backbone it builds on (Nexus) run per part.

Questions for authors
  1. Which image-to-3D model provides the coarse shape in the full-pipeline examples, and how is the vertex budget chosen when no ground truth is available?
  2. How well does the full pipeline work when it starts from an image alone, without the ground-truth shape or boxes?
  3. How do the generated meshes compare with the baselines and the ground-truth parts in face count and in how many parts are watertight?
  4. One of the reasoning model's training sets comes from the same source as the articulated-object test set. Were the test objects excluded from it?
  5. Filling in the shape is a stated design choice. Does it put a ceiling on what the reasoning model can see, for example drawers inside a closed cabinet, and does that show up in the articulated-object results?

If most of these questions are addressed, I will be glad to raise my score.

1 comment

Thank you for recognizing the value of our approach and representation. We provide further experimental evidence below to address your questions. All figure references point to the revised paper.

End-to-end evaluation (W1; Q1–2).

We evaluate image → Hunyuan3D-2.1 → ShapeCodec → ShapeVLM → PartGen, using predicted part conditions and a fixed vertex condition of 4,000. Appendix D.3 (Figure 13) adds four in-the-wild examples with ShapeVLM-predicted boxes/text. Appendix D.2 (Figures 11–12) also shows PartGen results conditioned on ShapeVLM predictions on PartObjaverse-Tiny. These examples demonstrate robustness to different part-decomposition strategies, retaining coherent geometry across different part granularities.

PartNet-Mobility: original GT-geometry results and HY2.1-input evaluation. Bold/italic indicate best/second within each block.

MethodmIoU ↑Type ↑Axis (rad) ↓Pivot ↓Range ↑Graph ↑Part-CD ×10³ ↓Part-F @.01 ↑
GT coarse geometry: original-paper Table 2 protocol
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.58950.76440.48120.17940.87430.584429.7550.3415
SIMART0.65430.83770.41500.19540.55180.7792––
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.81400.93280.18700.04950.80500.805217.4800.3842
HY2.1 coarse geometry: image-driven pipeline
URDFormer0.13320.79090.43010.35220.82160.5000102.1810.0086
ArtLLM + XPart0.51890.82030.39960.09190.82750.590934.5010.2074
SIMART0.42720.81080.20370.27190.59850.742432.3540.1585
PhysX-Omni0.41460.87370.35040.18220.72350.545543.0580.1694
Ours0.63670.93790.20950.10990.78500.803027.9230.2692

mIoU decreases and axis/pivot errors increase, consistent with spatial discrepancies, while type, graph, and range scores remain relatively stable. Ours retains the strongest overall performance, leading five of eight metrics, including both part-mesh scores.

Low-poly advantage and part quality (W2; Q3).

The table below compares the baselines’ original triangle counts with ours and evaluates part quality after decimation to matched budgets. The benefit goes beyond producing fewer triangles. Revised-paper Figure 14 compares the two strongest baselines, their QEM-decimated meshes, and our native low-poly outputs. At the same budget, our meshes show more coherent edge placement along shape contours and part boundaries, structured planar regions, and better preserved local features. This is qualitatively closer to artist-style low-poly connectivity: triangles follow the geometry rather than merely approximating a dense surface with fewer faces.

Native generation versus QEM simplification: 137 objects with >8 parts.

MethodOriginal mean trianglesMatched mean trianglesMatched Part-CD ×10³ ↓Matched Part-F @.1 ↑
PartPacker984,60716,562166.450.48573
HoloPart + oracle boxes5,749,60716,56364.920.78489
OmniPart + GT masks/boxes427,92016,50672.820.79871
Ours (native)16,56316,56315.290.98288

Choosing vertex budget (Q1)

Users choose the vertex-count condition Nᵥ according to their desired geometric detail and polygon/face budget, without a reference mesh. The table below and Figure 15 show robustness to this choice: halving or doubling Nᵥ changes the reported quality metrics by at most 1.3%, while triangle counts scale to 0.49×/1.99×. An eighth-sized budget preserves overall structure with reduced detail. The condition controls complexity rather than an exact face count; 4,000 is our fixed end-to-end setting.

Vertex-condition sweep: relative changes, explicitly in percent.

ConditionΔObj-CD (%)ΔPart-CD (%)ΔPart-F (%)Triangle ratio
Nᵥ = Nᵥ(ref)/8+6.0+11.5-0.40.12×
Nᵥ = Nᵥ(ref)/2-0.3+0.9+0.00.49×
Nᵥ = 2Nᵥ(ref)+0.4+1.3-0.01.99×

Watertightness (Q3).

Our native mesh representation supports both closed and open-surface parts, without imposing watertightness. This accommodates thin sheets and surface components while retaining explicit, editable connectivity.

Training exclusion (Q4).

Yes. Test objects and their views are excluded by object ID; saved splits confirm no overlap.

Filling (Q5).

Voxels provide a reference for the object’s exterior shape and spatial extent, rather than a precise interior reconstruction. This makes the structural interface compatible with diverse coarse meshes, including closed Hunyuan3D-2.1 outputs; image and semantic cues complement this geometric reference.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.