- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SCULPT, a framework for part-aware 3D generation. It starts from a complete object generated by TRELLIS.2. A joint split predictor then removes one part at a time. Each step outputs the extracted part and the remaining object together, and only the remainder is passed to the next step. The part and the remainder are denoised jointly on the union of their sparse supports, so their shared boundary is decided during generation. The rollout stops when the remainder becomes empty, so the number of parts follows the number of steps. Experiments on PartObjaverse show strong appearance quality.
Critical strengths
The subtractive formulation is simple and intuitive. It turns a variable number of outputs into repeated calls of one fixed operation with an explicit state. The generator stays in the loop during decomposition. Since each part is generated together with the remainder, both sides of every cut get geometry and texture. Segmentation methods leave cut surfaces untextured, and the large gains in appearance metrics are consistent with this.
Critical weaknesses
-
The extraction order is arbitrary. Parts are sorted by centroid along fixed axes, which is a spatial sweep rather than a semantic or physical order. Yet this order decides what the model has to remove at each step. The sculpting analogy suggests a more principled order, for example outer parts first or detachable parts first. The paper neither justifies its choice nor tests alternatives.
-
The central claim about joint generation is not directly supported. The paper argues that generating the part and the remainder together avoids gaps, interpenetration and seams. However, the composition loss only constrains the union, so the overlap between outputs is unbounded. The paper also does not measure overlap, gaps, or drift of the remainder across steps.
-
In the experiments, the gain over TRELLIS.2 with PartField is small, and part-level [email protected] is even slightly lower. AutoPartGen, the closest prior work, is discussed but not compared. There is also no analysis of how errors accumulate over long rollouts.
Questions for authors
-
How are newly exposed contact surfaces generated under clipping? All predictions are clipped to the initial support. If the TRELLIS.2 support only covers the outer surface, hidden interfaces would be removed. Does the initial support include interior interfaces in practice?
-
How sensitive is the method to the extraction order and part granularity? Can users choose which part to remove next, or how fine the split should be? Please report how often the rollout reaches the 24-step cap, and what happens when a step makes no real progress.
4 comments
Response to Reviewer vAJe · 2/2
W3.2. Comparison with AutoPartGen
We added this comparison using the publicly released AutoPartGen code and checkpoint, run on the same test set and evaluated under the protocol of Table 1. AutoPartGen uses geometry-only latents, so part appearance is not applicable.
| Method | Part CD ↓ | Part [email protected] ↑ | Part [email protected] ↑ | Object CD ↓ | Object [email protected] ↑ | Object [email protected] ↑ |
|---|---|---|---|---|---|---|
| AutoPartGen | 0.0209 | 0.4229 | 0.2605 | 0.0172 | 0.8087 | 0.5820 |
| TRELLIS.2+PF | 0.0115 | 0.8897 | 0.7554 | 0.0021 | 0.9793 | 0.9119 |
| SCULPT (ours) | 0.0107 | 0.8858 | 0.7599 | 0.0020 | 0.9839 | 0.9212 |
SCULPT is better than AutoPartGen on every geometry metric at both the part and the object level. The gap is largest at the part level because AutoPartGen stops early: it produces 4.9 parts per object on average, against 10.6 in the instance-level reference and 7.5 for SCULPT, so its parts are much coarser than the reference parts they are matched to.
W3.3. Error accumulation over the rollout
The table below evaluates the part produced at each rollout step. Each predicted part is scored against its best-matching reference part; this differs from the one-to-one matching of Table 1, so absolute values are not comparable. The full 24-step table will be added to the appendix.
| Step | CD ↓ | [email protected] ↑ | [email protected] ↑ | CLIP ↑ |
|---|---|---|---|---|
| 1 | 0.0178 | 0.832 | 0.717 | 0.862 |
| 5 | 0.0233 | 0.858 | 0.768 | 0.930 |
| 10 | 0.0233 | 0.866 | 0.785 | 0.929 |
| 15 | 0.0248 | 0.842 | 0.762 | 0.926 |
| 20 | 0.0230 | 0.847 | 0.783 | 0.937 |
| 23 | 0.0237 | 0.842 | 0.758 | 0.939 |
| 24 (capped remainder) | 0.0556 | 0.557 | 0.470 | 0.854 |
From step 2 to step 23, [email protected] stays between 0.82 and 0.89 and CD between 0.016 and 0.028. A linear fit over these steps gives a decline of about 4% in [email protected] and 2% in normalized-scale CLIP, while original-scale CLIP does not decline. We therefore observe a mild decline and no divergence over long rollouts. Step 24 is not a regular part: it is the remainder retained when a rollout reaches the cap, so it contains all content that was not yet separated. Note that later steps contain only objects with many parts, so the object set changes along the table.
Q1. Newly exposed contact surfaces under clipping
Clipping is applied to the 64³ sparse-structure occupancy only; geometry and material inside the active voxels are generated by the later stages for both the part and the remainder. The TRELLIS.2 structured latent already represents interior structure, which is one of the advantages of the O-Voxel design and can be seen on the TRELLIS.2 project page and in the teaser (Figure 1) of the TRELLIS.2 paper. In practice the initial support therefore usually contains the interfaces between parts. You are right about the remaining case: an interface that is absent from the initial support cannot be created, because every prediction is clipped to it. We accept this as a trade-off, since removing clipping is clearly harmful (part-level CD 0.0260 vs. 0.0107 in Table 2), and we will state it as a limitation.
Q2.1. Sensitivity to order and granularity, and user control
Regarding sensitivity to the extraction order, we have not directly evaluated alternative orders. The test would require to rebuild the training sequences under each alternative order, retrain the model, and compare the resulting models on the same test set. This involves a large training cost and cannot be completed within the rebuttal period, and semantic orders such as outer-first or detachable-first additionally require annotations that are difficult to obtain at this scale. Also, to our knowledge, prior methods that adopt a fixed part order generally do not ablate it either.
The order decides only which annotated part is removed next, not what counts as a part, and it is fixed and deterministic for every asset (see W1). Within this order, quality does not depend strongly on where a part falls in the sequence: in the per-step evaluation under W3.3, [email protected] stays between 0.82 and 0.89 from step 2 to step 23.
Users cannot currently choose which part is removed next. They can control how fine the result is: any extracted part can be split again recursively (Figure 1(b), 100+ parts), and the rollout can be stopped early with the remainder kept as one part.
Q2.2. Rollout cap and steps without progress
Of the 200 test rollouts, 23 (11.5%) reach the cap; the other 88.5% terminate with an empty remainder.
We have not observed an empty extracted part in our rollouts, and the training data contain no split whose extracted part is empty; the only empty target is the terminal remainder. If a rollout nevertheless progresses slowly, the cap bounds its length and the remaining content is kept as one output rather than discarded.
I thank the authors for their clarifications. I find this work elegant and innovative. However, although the results presented in the paper and supplementary material are generally reasonable, the examples shown do not feature particularly complex, detailed, or visually impressive cuts. I will therefore maintain my current rating.
Follow-up on W2: quantitative evaluation of part interfaces
To our knowledge, there is no established metric for how the generated parts of one object fit together. We therefore designed an evaluation that is computed directly from the generated part meshes. For every pair of parts , we sample points on the surface of . For a sample , let be its closest point on , the signed distance to (negative when is inside ), and the angle between the normal of at and the reversed normal of at . Let be the set of samples with . With a distance tolerance and an angle threshold , we define
In addition, we report volume overlap between parts on a voxel grid.
| Method | Penetration ↓ (τ = 0.005) | Penetration ↓ (τ = 0.01) | Coupling ↑ (τ = 0.005) | Coupling ↑ (τ = 0.01) | Overlap ↓ |
|---|---|---|---|---|---|
| OmniPart | 0.100 | 0.030 | 0.094 | 0.112 | 0.548 |
| AutoPartGen | 0.377 | 0.193 | 0.087 | 0.143 | 0.019 |
| SCULPT (ours) | 0.016 | 0.0003 | 0.136 | 0.157 | 0.030 |
SCULPT is the only method that keeps both penetration and overlap low while having the highest coupling. OmniPart's parts are largely nested inside each other (54.8% overlap), and AutoPartGen's parts interpenetrate where they meet (37.7%). SCULPT's parts instead meet face to face without intruding into each other, which is the behaviour illustrated in Figure 4 and the purpose of generating each part jointly with its remainder. We will add this evaluation to Appendix.
Please let us know if any part of the protocol or the results remains unclear; we are happy to provide further details.
Response to Reviewer vAJe · 1/2
Thank you for the critical and specific comments. We address each point below.
W1. Choice of extraction order
We would like to separate two things. The order only decides which annotated part is removed next; what counts as a part comes from the human-refined annotations. The extracted components are therefore semantic parts even under a spatial order (e.g., leaf, arms, body, and legs in Appendix Figure 4).
The order itself needs to be deterministic and reproducible: a part set is unordered, and a single sequence per asset gives the predictor an unambiguous target, which stabilizes training. A spatial order can be computed for all training assets without additional labels. Sorting parts spatially is also the convention in OmniPart.
Outer-first or detachable-first orders are attractive, but they need annotations that do not exist at this scale, and testing them requires rebuilding the dataset and retraining all three stages. We will add this justification to the paper and list learned or VLM-assisted semantic ordering as future work.
W2. Overlap, gaps, seams, and drift
We would like to clarify how the method constraints overlap, gaps, seams, and drift, and which results reflect each of them.
Overlap. The composition loss is not the only supervision on the supports. The flow-matching loss supervises the part and remainder directly against independently voxelized ground-truth meshes (Section 3.2), whose voxel supports overlap only at the boundary where the two meshes meet. The model is therefore trained to produce overlap only at the contact region. At inference, both supports are additionally clipped to the current remainder (Eq. 13).
Gaps. The composition loss requires the union of the two outputs to cover the current support, which penalizes voxels assigned to neither output; removing it raises part-level CD from 0.0107 to 0.0279. After the full rollout, the assembled parts reproduce the complete object: object-level CD 0.0020 and [email protected] 0.9212. Gaps or lost regions accumulated over the rollout would lower these scores.
Seams. In the geometry and material stages, the part and the remainder are denoised in the same pass on the union of their supports (Appendix A.3), so the features at shared interface voxels are generated together. Appearance is measured on rendered parts, which include their cut surfaces, and improves from CLIP 0.8631 to 0.9075.
Drift. Two mechanisms keep the remainder anchored to the original object: the complete conditioning image is an input to every split, so the predictor does not rely only on the regenerated remainder, and clipping makes the remainder support non-expanding. The per-step evaluation under W3.3 shows the outcome: part quality declines only mildly and does not diverge up to step 23.
We have added Figure 4 and an accompanying paragraph (Section 4.3) to the revised paper to illustrate this advantage of the method: the two outputs of a split meet in the same voxel layer, and the voxels they share are confined to that layer. A quantitative metric on contact surfaces is harder to measure, because PartObjaverse does not annotate contact regions. We will post such a metric in an update before the end of the rebuttal period.
W3.1. Geometry and appearance relative to TRELLIS.2 + PartField
The part geometry of TRELLIS.2+PF is inherited from the TRELLIS.2 object, whose geometry is already accurate. SCULPT regenerates every part and still matches it: CD is lower (0.0107 vs. 0.0115), [email protected] is higher (0.7599 vs. 0.7554), and [email protected] is within 0.004 (0.8858 vs. 0.8897). The quality of the part geometry can also be seen in the qualitative comparison of Figure 3. The geometry result should therefore be read as parity with post-hoc segmentation. The gain is in the parts themselves: post-hoc segmentation leaves cut surfaces untextured, which geometry metrics on matched parts barely reflect but appearance metrics do (CLIP 0.9075 vs. 0.8631; Inception FID 20.73 vs. 27.88; DINOv2 FID 168.03 vs. 237.97). SCULPT's extracted parts are complete and textured.