acceptodds
Under review as a conference paper at ICLR 2027

SCULPT: Subtractive Composition for 3D Part Generation

16 views

Abstract

Part-aware 3D generation aims to create digital assets that are coherent as complete objects while exposing structural parts for editing, material assignment, animation, and reuse. Existing methods impose this structure outside the native generation loop: segmentation-based methods partition an already generated shape, while additive methods synthesize parts from predefined layouts, boxes, or tokens and then reconcile them into a whole. The former preserves the generated geometry but fixes the object before part boundaries are determined; the latter exposes part cardinality but often leaves shared boundaries vulnerable to gaps, interpenetrations, and material discontinuities. In this paper, we propose SCULPT, a framework that addresses these challenges through subtractive composition. Given a complete object represented in a structured 3D latent space, SCULPT iteratively applies a joint split predictor to generate one extracted part together with the remaining object. The predictor performs a coupled denoising process conditioned on both the image and the current 3D state, so the extracted part and updated remainder are generated together rather than reconciled after generation. The joint split predictor processes both outputs on the union of their native sparse 3D supports, allowing neighboring supports to overlap rather than imposing a disjoint voxel partition. The rollout ends when the remainder support becomes empty or reaches a fixed safety cap, allowing the number of generated parts to adapt to each object within that bound. Extensive experiments demonstrate state-of-the-art geometry on PartObjaverse while preserving strong complete-object reconstruction after part assembly. Results on four dataset images, one text-to-image-generated input, and one real-world photograph further show fine-grained textured part decomposition beyond the benchmark.

Primary area:
generative models
Keywords:
3D Part Generation, 3D Generation, 3D Vision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vAJeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SCULPT, a framework for part-aware 3D generation. It starts from a complete object generated by TRELLIS.2. A joint split predictor then removes one part at a time. Each step outputs the extracted part and the remaining object together, and only the remainder is passed to the next step. The part and the remainder are denoised jointly on the union of their sparse supports, so their shared boundary is decided during generation. The rollout stops when the remainder becomes empty, so the number of parts follows the number of steps. Experiments on PartObjaverse show strong appearance quality.

Critical strengths

The subtractive formulation is simple and intuitive. It turns a variable number of outputs into repeated calls of one fixed operation with an explicit state. The generator stays in the loop during decomposition. Since each part is generated together with the remainder, both sides of every cut get geometry and texture. Segmentation methods leave cut surfaces untextured, and the large gains in appearance metrics are consistent with this.

Critical weaknesses
  1. The extraction order is arbitrary. Parts are sorted by centroid along fixed axes, which is a spatial sweep rather than a semantic or physical order. Yet this order decides what the model has to remove at each step. The sculpting analogy suggests a more principled order, for example outer parts first or detachable parts first. The paper neither justifies its choice nor tests alternatives.

  2. The central claim about joint generation is not directly supported. The paper argues that generating the part and the remainder together avoids gaps, interpenetration and seams. However, the composition loss only constrains the union, so the overlap between outputs is unbounded. The paper also does not measure overlap, gaps, or drift of the remainder across steps.

  3. In the experiments, the gain over TRELLIS.2 with PartField is small, and part-level [email protected] is even slightly lower. AutoPartGen, the closest prior work, is discussed but not compared. There is also no analysis of how errors accumulate over long rollouts.

Questions for authors
  1. How are newly exposed contact surfaces generated under clipping? All predictions are clipped to the initial support. If the TRELLIS.2 support only covers the outer surface, hidden interfaces would be removed. Does the initial support include interior interfaces in practice?

  2. How sensitive is the method to the extraction order and part granularity? Can users choose which part to remove next, or how fine the split should be? Please report how often the rollout reaches the 24-step cap, and what happens when a step makes no real progress.

4 comments
AuthorsOpenReview ↗

Response to Reviewer vAJe · 1/2

Thank you for the critical and specific comments. We address each point below.

W1. Choice of extraction order

We would like to separate two things. The order only decides which annotated part is removed next; what counts as a part comes from the human-refined annotations. The extracted components are therefore semantic parts even under a spatial order (e.g., leaf, arms, body, and legs in Appendix Figure 4).

The order itself needs to be deterministic and reproducible: a part set is unordered, and a single sequence per asset gives the predictor an unambiguous target, which stabilizes training. A spatial order can be computed for all training assets without additional labels. Sorting parts spatially is also the convention in OmniPart.

Outer-first or detachable-first orders are attractive, but they need annotations that do not exist at this scale, and testing them requires rebuilding the dataset and retraining all three stages. We will add this justification to the paper and list learned or VLM-assisted semantic ordering as future work.

W2. Overlap, gaps, seams, and drift

We would like to clarify how the method constraints overlap, gaps, seams, and drift, and which results reflect each of them.

  • Overlap. The composition loss is not the only supervision on the supports. The flow-matching loss supervises the part and remainder directly against independently voxelized ground-truth meshes (Section 3.2), whose voxel supports overlap only at the boundary where the two meshes meet. The model is therefore trained to produce overlap only at the contact region. At inference, both supports are additionally clipped to the current remainder (Eq. 13).

  • Gaps. The composition loss requires the union of the two outputs to cover the current support, which penalizes voxels assigned to neither output; removing it raises part-level CD from 0.0107 to 0.0279. After the full rollout, the assembled parts reproduce the complete object: object-level CD 0.0020 and [email protected] 0.9212. Gaps or lost regions accumulated over the rollout would lower these scores.

  • Seams. In the geometry and material stages, the part and the remainder are denoised in the same pass on the union of their supports (Appendix A.3), so the features at shared interface voxels are generated together. Appearance is measured on rendered parts, which include their cut surfaces, and improves from CLIP 0.8631 to 0.9075.

  • Drift. Two mechanisms keep the remainder anchored to the original object: the complete conditioning image is an input to every split, so the predictor does not rely only on the regenerated remainder, and clipping makes the remainder support non-expanding. The per-step evaluation under W3.3 shows the outcome: part quality declines only mildly and does not diverge up to step 23.

We have added Figure 4 and an accompanying paragraph (Section 4.3) to the revised paper to illustrate this advantage of the method: the two outputs of a split meet in the same voxel layer, and the voxels they share are confined to that layer. A quantitative metric on contact surfaces is harder to measure, because PartObjaverse does not annotate contact regions. We will post such a metric in an update before the end of the rebuttal period.

W3.1. Geometry and appearance relative to TRELLIS.2 + PartField

The part geometry of TRELLIS.2+PF is inherited from the TRELLIS.2 object, whose geometry is already accurate. SCULPT regenerates every part and still matches it: CD is lower (0.0107 vs. 0.0115), [email protected] is higher (0.7599 vs. 0.7554), and [email protected] is within 0.004 (0.8858 vs. 0.8897). The quality of the part geometry can also be seen in the qualitative comparison of Figure 3. The geometry result should therefore be read as parity with post-hoc segmentation. The gain is in the parts themselves: post-hoc segmentation leaves cut surfaces untextured, which geometry metrics on matched parts barely reflect but appearance metrics do (CLIP 0.9075 vs. 0.8631; Inception FID 20.73 vs. 27.88; DINOv2 FID 168.03 vs. 237.97). SCULPT's extracted parts are complete and textured.

AuthorsOpenReview ↗

Response to Reviewer vAJe · 2/2

W3.2. Comparison with AutoPartGen

We added this comparison using the publicly released AutoPartGen code and checkpoint, run on the same test set and evaluated under the protocol of Table 1. AutoPartGen uses geometry-only latents, so part appearance is not applicable.

MethodPart CD ↓Part [email protected] ↑Part [email protected] ↑Object CD ↓Object [email protected] ↑Object [email protected] ↑
AutoPartGen0.02090.42290.26050.01720.80870.5820
TRELLIS.2+PF0.01150.88970.75540.00210.97930.9119
SCULPT (ours)0.01070.88580.75990.00200.98390.9212

SCULPT is better than AutoPartGen on every geometry metric at both the part and the object level. The gap is largest at the part level because AutoPartGen stops early: it produces 4.9 parts per object on average, against 10.6 in the instance-level reference and 7.5 for SCULPT, so its parts are much coarser than the reference parts they are matched to.

W3.3. Error accumulation over the rollout

The table below evaluates the part produced at each rollout step. Each predicted part is scored against its best-matching reference part; this differs from the one-to-one matching of Table 1, so absolute values are not comparable. The full 24-step table will be added to the appendix.

StepCD ↓[email protected] ↑[email protected] ↑CLIP ↑
10.01780.8320.7170.862
50.02330.8580.7680.930
100.02330.8660.7850.929
150.02480.8420.7620.926
200.02300.8470.7830.937
230.02370.8420.7580.939
24 (capped remainder)0.05560.5570.4700.854

From step 2 to step 23, [email protected] stays between 0.82 and 0.89 and CD between 0.016 and 0.028. A linear fit over these steps gives a decline of about 4% in [email protected] and 2% in normalized-scale CLIP, while original-scale CLIP does not decline. We therefore observe a mild decline and no divergence over long rollouts. Step 24 is not a regular part: it is the remainder retained when a rollout reaches the cap, so it contains all content that was not yet separated. Note that later steps contain only objects with many parts, so the object set changes along the table.

Q1. Newly exposed contact surfaces under clipping

Clipping is applied to the 64³ sparse-structure occupancy only; geometry and material inside the active voxels are generated by the later stages for both the part and the remainder. The TRELLIS.2 structured latent already represents interior structure, which is one of the advantages of the O-Voxel design and can be seen on the TRELLIS.2 project page and in the teaser (Figure 1) of the TRELLIS.2 paper. In practice the initial support therefore usually contains the interfaces between parts. You are right about the remaining case: an interface that is absent from the initial support cannot be created, because every prediction is clipped to it. We accept this as a trade-off, since removing clipping is clearly harmful (part-level CD 0.0260 vs. 0.0107 in Table 2), and we will state it as a limitation.

Q2.1. Sensitivity to order and granularity, and user control

Regarding sensitivity to the extraction order, we have not directly evaluated alternative orders. The test would require to rebuild the training sequences under each alternative order, retrain the model, and compare the resulting models on the same test set. This involves a large training cost and cannot be completed within the rebuttal period, and semantic orders such as outer-first or detachable-first additionally require annotations that are difficult to obtain at this scale. Also, to our knowledge, prior methods that adopt a fixed part order generally do not ablate it either.

The order decides only which annotated part is removed next, not what counts as a part, and it is fixed and deterministic for every asset (see W1). Within this order, quality does not depend strongly on where a part falls in the sequence: in the per-step evaluation under W3.3, [email protected] stays between 0.82 and 0.89 from step 2 to step 23.

Users cannot currently choose which part is removed next. They can control how fine the result is: any extracted part can be split again recursively (Figure 1(b), 100+ parts), and the rollout can be stopped early with the remainder kept as one part.

Q2.2. Rollout cap and steps without progress

Of the 200 test rollouts, 23 (11.5%) reach the cap; the other 88.5% terminate with an empty remainder.

We have not observed an empty extracted part in our rollouts, and the training data contain no split whose extracted part is empty; the only empty target is the terminal remainder. If a rollout nevertheless progresses slowly, the cap bounds its length and the remaining content is kept as one output rather than discarded.

I thank the authors for their clarifications. I find this work elegant and innovative. However, although the results presented in the paper and supplementary material are generally reasonable, the examples shown do not feature particularly complex, detailed, or visually impressive cuts. I will therefore maintain my current rating.

Follow-up on W2: quantitative evaluation of part interfaces

To our knowledge, there is no established metric for how the generated parts of one object fit together. We therefore designed an evaluation that is computed directly from the generated part meshes. For every pair of parts (A,B)(A, B), we sample points on the surface of AA. For a sample pp, let qq be its closest point on BB, d(p)d(p) the signed distance to BB (negative when pp is inside BB), and φ(p)\varphi(p) the angle between the normal of AA at pp and the reversed normal of BB at qq. Let VV be the set of samples with ∣d(p)∣<0.02|d(p)|<0.02. With a distance tolerance τ\tau and an angle threshold θ=30∘\theta = 30^\circ, we define

Penetration=∣{p∈V:d(p)<−τ}∣∣V∣,Coupling=∣{p∈V:∣d(p)∣<τ, φ(p)<θ}∣∣V∣.\mathrm{Penetration} = \frac{|\{p \in V : d(p) < -\tau\}|}{|V|}, \qquad \mathrm{Coupling} = \frac{|\{p \in V : |d(p)| < \tau,\ \varphi(p) < \theta\}|}{|V|}.

In addition, we report volume overlap between parts on a 1283128^3 voxel grid.

MethodPenetration ↓ (τ = 0.005)Penetration ↓ (τ = 0.01)Coupling ↑ (τ = 0.005)Coupling ↑ (τ = 0.01)Overlap ↓
OmniPart0.1000.0300.0940.1120.548
AutoPartGen0.3770.1930.0870.1430.019
SCULPT (ours)0.0160.00030.1360.1570.030

SCULPT is the only method that keeps both penetration and overlap low while having the highest coupling. OmniPart's parts are largely nested inside each other (54.8% overlap), and AutoPartGen's parts interpenetrate where they meet (37.7%). SCULPT's parts instead meet face to face without intruding into each other, which is the behaviour illustrated in Figure 4 and the purpose of generating each part jointly with its remainder. We will add this evaluation to Appendix.

Please let us know if any part of the protocol or the results remains unclear; we are happy to provide further details.

Reviewer_P7qHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SCULPT, a part-level 3D generation framework built on top of TRELLIS.2. It introduces modification to TRELLIS.2 by changing it to predict a pair of part-level and remainder structured latents given the current wholistic latent. This process is then continued on the predicted remainder latent until the remainder is empty or a maximum number of parts are predicted.

The paper compares part level segmentation and reconstruction against baselines such as OmniPart and PartField/HoloPart directly applied to output of TRELLIS.2. It shows the best reconstruction against the compared baselines. The qualatative visualization in both the main and the supp website shows many well-segmented shapes.

Critical strengths
  1. Formulating part-level generation as a recursive splitting from a wholistic shape is interesting and very intuitive. Also the authors were able to utilize pretrained prior such as TRELLIS.2 for this task is also a plus.
  2. The qualtative result the paper shows contain many examples that are segmented in detail. This shows that the method works well on a large set of shapes.
Critical weaknesses
  1. The paper misses details on how the baselines were evaluated. For example, how many part did the authors cluster Partfield for evaluating the metrics? WIth different number of parts I assume the numbers would differ significantly.
  2. Currently the method's formulation doesn't provide any control in terms of which part to split. This feels like a limitation in the practicality of the method. Part of this is due to the fixed ordered segemenration sequence during training. I wonder if the authors have tried or experimented with more flexible segmentation trajectory for training/inference.
Questions for authors

Would be good if the authors can show some limitations and failure case examples. Given that a clipping is needed during inference to prune voxels, the method's part-level generation probably does not alway follow the input wholistic latent.

2 comments
AuthorsOpenReview ↗

Response to Reviewer P7qH

Thank you for the encouraging comments and the practical questions.

W1. The paper misses details on how the baselines were evaluated. For example, how many part did the authors cluster Partfield for evaluating the metrics? WIth different number of parts I assume the numbers would differ significantly.

We use PartField's default setting of 20 clusters for every PartField-based baseline on the same test set. We do not tune the number per object, because the reference part count is not available at test time.

To show how the numbers change with the number of clusters, we additionally evaluated both PartField baselines with K{5,10,15,20} clusters under the protocol of Table 1. K=20 is the setting reported in the paper.

MethodKPart CD ↓Part [email protected] ↑Part [email protected] ↑
TRELLIS+PF50.02690.61810.4720
TRELLIS+PF100.01490.76480.6065
TRELLIS+PF150.01200.83760.6795
TRELLIS+PF20 (paper)0.01270.87500.7169
TRELLIS.2+PF50.02510.67250.3831
TRELLIS.2+PF100.01640.83130.5471
TRELLIS.2+PF150.01290.88480.7480
TRELLIS.2+PF20 (paper)0.01150.88970.7554
SCULPT (ours)–0.01070.88580.7599

W2.1. Currently the method's formulation doesn't provide any control in terms of which part to split. This feels like a limitation in the practicality of the method.

The current model does not let the user choose which part is removed next. The controls that are available today are:

  • Granularity. The split operation can be applied recursively to any extracted part, which is how the 100+ part result in Figure 1(b) is obtained.

  • Part count. The rollout can be stopped at any step, keeping the current remainder as one part.

Explicit control, for example conditioning on a 2D segmentation map to indicate the desired part or granularity, requires new training data and retraining. We consider it important future work and will discuss it in the limitations.

W2.2. Part of this is due to the fixed ordered segementation sequence during training. I wonder if the authors have tried or experimented with more flexible segmentation trajectory for training/inference.

Changing the extraction order requires rebuilding the training sequences and retraining the model, which cannot be completed within the rebuttal period. The current design is nevertheless well grounded. It is consistent with prior work: OmniPart, a prior autoregressive part generator, also sorts parts by their spatial coordinates to obtain a fixed sequence. It also follows the way a person takes an object apart, from top to bottom. And its purpose is to fix one sequence per object: a part set is unordered, and a single sequence gives the predictor an unambiguous target at every step, so the rollout at inference follows a fixed and reproducible order.

Q1.1. Would be good if the authors can show some limitations and failure case examples.

We have added a Limitations section (Section 4.5) to the revised PDF. The main cases are:

  • Dependence on the base generator. When TRELLIS.2 produces a poor complete object, the decomposition inherits the error.

  • Controllability, as discussed under W2.

  • Stopping. The predicted number of parts can deviate from the granularity a user expects.

Q1.2. Given that a clipping is needed during inference to prune voxels, the method's part-level generation probably does not alway follow the input wholistic latent.

Clipping is a safeguard for the recurrent rollout, not a correction for a predictor that ignores its input. The split predictor is conditioned on the current remainder latent in every block and is trained with the composition loss to cover the current support, so it follows the holistic latent by design. As with any generative model, there is no hard guarantee on the output, and in a recurrent rollout a voxel predicted outside the current object would be fed back into the next split and accumulate. Clipping rules this out by making the remainder support non-expanding, which keeps long rollouts stable.

Thanks the authors for the clarifications and additional experiments. The clarified my concerns. However, the limitation of cannot control which part is being split slightly hurts the model's practicality. Thus I will keep my rating.

Reviewer_rB6fOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SCULPT, an image-conditioned framework for part-aware 3D generation. Starting from a complete object generated by TRELLIS.2, the method repeatedly predicts an extracted part jointly with an updated remainder. It combines remainder-conditioned flow transformers, a sparse-support composition loss, and inference-time support clipping. Experiments on PartObjaverse report improved part appearance and competitive geometry at both the part and assembled-object levels.

Critical strengths
  1. The explicit remainder state and joint part–remainder prediction provide a coherent formulation for sequential 3D decomposition.
  2. The reported results show improved appearance metrics and modest gains in several geometry metrics over TRELLIS.2 + PartField. The component ablations provide supporting evidence for the design choices.
Critical weaknesses

The variable-cardinality evaluation needs clarification. Section 4.1 mentions Hungarian matching but does not specify how unmatched predictions and unmatched reference parts are included in the scores. Since adaptive output cardinality is a central feature, reporting part-count errors and the fractions of rollouts that terminate naturally versus reach the cap would strengthen the evaluation.

Interface quality is supported primarily by indirect evidence. The support constraints encourage coverage and containment, but do not explicitly ensure that overlap is confined to contact regions or that the decoded interfaces are geometrically and materially compatible. The reported geometry and appearance metrics are encouraging, but a focused assessment of contact surfaces would better substantiate this motivation.

The benefit of joint subtractive prediction could be isolated more directly. The existing ablations establish the usefulness of individual components within SCULPT, but do not directly compare joint and separate part–remainder prediction. A comparison with the closely related AutoPartGen, or a controlled ablation addressing this distinction, would strengthen the empirical positioning. The modest geometry gains over TRELLIS.2 + PartField would also benefit from uncertainty estimates.

Questions for authors
  1. What matching cost and aggregation rules are used when predicted and reference part counts differ? How are extra, missing, and empty parts handled?
  2. What proportion of test objects terminate with an empty remainder before the 24-step cap? How closely do predicted part counts follow the reference annotations?
  3. At what representation level and resolution is support clipping applied, and how does it accommodate newly exposed contact surfaces?
  4. What is the inference time relative to TRELLIS.2 + PartField, and how does it scale with the number of extracted parts?
2 comments
AuthorsOpenReview ↗

Response to Reviewer rB6f · 1/2

Thank you for the detailed and constructive review.

W1.1. The variable-cardinality evaluation needs clarification. Section 4.1 mentions Hungarian matching but does not specify how unmatched predictions and unmatched reference parts are included in the scores.

Q1.1. What matching cost and aggregation rules are used when predicted and reference part counts differ?

  • Matching cost. For each metric we run Hungarian matching between predicted and reference parts with that metric as the cost, which gives a one-to-one assignment.

  • Aggregation. Scores are averaged over the matched pairs of each object and then averaged over objects.

  • The same protocol is applied to every method in Table 1.

We will state it explicitly in Section 4.1.

W1.2. Since adaptive output cardinality is a central feature, reporting part-count errors and the fractions of rollouts that terminate naturally versus reach the cap would strengthen the evaluation.

Q1.2. How are extra, missing, and empty parts handled?

Q2.1. What proportion of test objects terminate with an empty remainder before the 24-step cap?

  • Extra and missing parts. When the counts differ, unmatched predictions and unmatched reference parts are excluded from the scores; they are not scored as zeros.

  • Empty parts. Empty predictions are discarded before matching.

Of the 200 test rollouts, 177 (88.5%) terminate with an empty remainder and 23 (11.5%) reach the cap. We will add these statistics to the paper.

W2. Interface quality is supported primarily by indirect evidence. The support constraints encourage coverage and containment, but do not explicitly ensure that overlap is confined to contact regions or that the decoded interfaces are geometrically and materially compatible. The reported geometry and appearance metrics are encouraging, but a focused assessment of contact surfaces would better substantiate this motivation.

The overlap between the two outputs is supervised, not left unconstrained. The composition loss is only one of three mechanisms that shape the interface:

  • Per-output supervision. The flow-matching loss trains each output toward its own target. These targets are the independently voxelized part and remainder (Section 3.2), whose supports overlap only where the two meshes meet. The model is therefore trained to produce overlap only at the contact region, and the composition loss adds the requirement that the two outputs together cover the current object.

  • Clipping. At inference, both supports are clipped to the current remainder, so neither output can extend beyond the object being split.

  • Joint denoising. In the geometry and material stages, the part and the remainder are denoised in the same pass on the union of their supports (Appendix A.3), so the features at shared interface voxels are generated together rather than reconciled afterwards.

PartObjaverse does not annotate contact regions, so a metric restricted to contact surfaces cannot be computed directly on this benchmark. We have added Figure 4 and an accompanying paragraph (Section 4.3) to the revised paper to clarify our design: they show that the two outputs of a split meet in the same voxel layer, and that the voxels they share are confined to that layer. The three mechanisms above are now also discussed in Appendix B.7.

W3.1. The benefit of joint subtractive prediction could be isolated more directly. The existing ablations establish the usefulness of individual components within SCULPT, but do not directly compare joint and separate part–remainder prediction.

The main problem with separate prediction is that it needs two generators. While the remainder generator is sampling, it has no access to which part the part generator is producing, yet different parts must be paired with different remainders. Joint generation is a direct solution to this problem: both outputs are produced in one denoising process, so the remainder always corresponds to the part being extracted, and it works well in practice.

AuthorsOpenReview ↗

Response to Reviewer rB6f · 2/2

W3.2. A comparison with the closely related AutoPartGen, or a controlled ablation addressing this distinction, would strengthen the empirical positioning.

We added this comparison using the publicly released AutoPartGen code (TripoSG version) and 2048-tokens checkpoint, run on the same test set and evaluated under the protocol of Table 1. AutoPartGen uses geometry-only latents, so part appearance is not applicable.

MethodPart CD ↓Part [email protected] ↑Part [email protected] ↑Object CD↓Object [email protected] ↑Object [email protected] ↑
AutoPartGen0.02090.42290.26050.01720.80870.5820
SCULPT (ours)0.01070.88580.75990.00200.98390.9212

SCULPT is better than AutoPartGen on every geometry metric at both the part and the object level. The gap is large at the part level because AutoPartGen stops early: it produces 4.9 parts per object on average, against 10.6 in the instance-level reference and 7.5 for SCULPT, so its parts are much coarser than the reference parts they are matched to. AutoPartGen conditions each new part on the whole object and the previously generated parts, without an explicit remainder state, which is the distinction raised.

W3.3. The modest geometry gains over TRELLIS.2 + PartField would also benefit from uncertainty estimates.

The part geometry of TRELLIS.2+PF is inherited from the TRELLIS.2 object, whose geometry is already accurate. SCULPT regenerates every part and still matches it (part CD 0.0107 vs. 0.0115, [email protected] 0.7599 vs. 0.7554, [email protected] 0.8858 vs. 0.8897). Instead, our primary advantage over this baseline lies in part appearance and newly exposed interior surfaces, where the margins are large: CLIP 0.9075 vs. 0.8631, Inception FID 20.73 vs. 27.88, and DINOv2 FID 168.03 vs. 237.97.

To quantify the uncertainty of our results, we run SCULPT three times on the test set and report the standard deviation of each metric over the three runs.

MetricStd
Part CD ↓0.00131
Part [email protected] ↑0.00426
Part [email protected] ↑0.00203
Object CD ↓0.00003
Object [email protected] ↑0.00100
Object [email protected] ↑0.00047

Q2.2. How closely do predicted part counts follow the reference annotations?

SCULPT produces 7.5 parts per object on average, compared to the 10.6 instance-level parts in the reference annotations.

Q3. At what representation level and resolution is support clipping applied, and how does it accommodate newly exposed contact surfaces?

Clipping is applied at the sparse-structure stage, on the 64³ occupancy grid, before the geometry and material stages run. It constrains only which coarse voxels are active; geometry and material inside the active voxels are generated by the later stages for both the part and the remainder.

Regarding contact surfaces: the TRELLIS.2 structured latent already represents interior structure, which is one of the advantages of the O-Voxel design and can be seen on the TRELLIS.2 project page and in the teaser (Figure 1) of the TRELLIS.2 paper. The initial support therefore usually contains the interfaces between parts. Clipping excludes only contact surfaces that fall outside the initial support, which cannot be created in the current design. This is a trade-off: without clipping, stray voxels outside the object harm the recurrent state, and part-level CD rises from 0.0107 to 0.0260 (Table 2).

Q4. What is the inference time relative to TRELLIS.2 + PartField, and how does it scale with the number of extracted parts?

Each split is one pass through the three stages of the TRELLIS.2 backbone. For an object with K parts, SCULPT therefore costs approximately K times the TRELLIS.2 inference time, in addition to the initial whole-object generation, and the cost grows linearly with K. TRELLIS.2 + PartField runs the generator once, so SCULPT is slower by roughly this factor. The stage factorization already allows pipelining across splits (Appendix A.5).

In the future work, two optimizations can reduce this cost:

  • Sparse first stage. The TRELLIS.2 sparse-structure checkpoint processes a dense grid of L³ cells at each of N splits, or NL³ cells in total. A model retrained with sparse attention would process only the active voxels in the current remainder. Its total workload would be the sum of the active-voxel counts across splits, which decrease as parts are removed.

  • Local later stages. If the geometry and material stages could condition only on the extracted part’s voxels, each voxel would be processed about once across the rollout. With constant overhead per split, this would reduce the total from O(NL³) to O(N + L³) and allow parts to be generated in parallel.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.