acceptodds
Under review as a conference paper at ICLR 2027

CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Control

Abstract

Creating and editing high-quality 3D content remains a central challenge in computer graphics. We address this challenge by introducing CompoSE, a novel method for Compositional Synthesis and Editing of 3D shapes via part-aware control. Our method takes as input a set of coarse geometric primitives (e.g., bounding boxes) that represent distinct object parts arranged in a particular spatial configuration, and synthesizes as output part-separated 3D objects that support localized granular (i.e., compositional) editing of individual parts. The key insight that enables our method is our use of a diffusion transformer architecture that alternates between processing each part locally and aggregating contextual information across parts globally, and features a novel conditioning technique that ensures strong adherence to the user's input. Importantly, our method learns to infer part semantics and symmetries directly from coarse layout guidance, and does not require part-level text prompts. We demonstrate that our method enables powerful compositional editing capabilities, including context-aware substitution, addition, deletion, and style-preserving resizing operations. We show through extensive experiments that our method significantly outperforms existing approaches on guided synthesis and on proxy-controlled editing, as measured by quantitative and qualitative evaluations.

Primary area:
generative models
Keywords:
3D generation, 3D assets, parts, controllable 3D generation, 3D editing, computer graphics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6EpnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a 3D OBB layout conditioned part-level shape generation work. It takes in a set of OBB and a global text prompt as input and generates segmented shape as a occ field. The paper uses a global-local transformer to inject the part-level information to the global shape generation branch. experimentally it also shows shape editing applications that allows the user to edit the generated object by changing its layout input and text prompt.

Critical strengths

The paper's result looks promising and it shows many qualitative result backing up its superior metrics compared to the baselines. The editing applications looks very cool and versatile.

Critical weaknesses

The main concern I have is novelty. The main architectural contribution, i.e., local-global latent processing mechanism feels like a direct application of the contribution proposed by the previous PartCrafter paper. Meanwhile, the introduction calls the alternating local/global architecture a “key insight” enabling their method (ln 77-79). At least a acknowledgement of similarity is needed in the main paper. Also given the architectual similarity, the authors should clarify what has made their method work better than PartCrafter. Is the reason that PartCrafter does poorly in comparison solely due to the use of an image generator? A fair comparison is needed to justify the improvement.

Additionally, in the first technical contribution it says "We present the first method to synthesize part-separated 3D shapes directly from coarse part layouts, featuring a novel conditioning technique that ensures strong adherence to the user’s input layout, and a high degree of control over individual object parts."

This is a sigificant overclaim. For example, CoPart, which the paper cites, already takes in 3D bounding boxes as input conditioning for shape generation. Older works such as SALAD, Difffacto, SPAGHETTI all has explicit part-level shape conditioning as part of their pipeline. So it would be a fairly straightforward extension of their methods for enabling OBB as input conditioning. Given these, it makes me question whether the authors have carefully considered the positioning of their paper among the previous works.

Questions for authors

See weakness

12 comments

We sincerely thank the reviewer for the careful review. We are glad that the reviewer found our results "promising", supported by "many qualitative result[s] backing up [our] superior metrics compared to the baselines", and our editing applications "very cool and versatile". We address each concern below. All new experiments and clarifications have been included in our revised manuscript.


1. Attribution of the general local/global design in DiTs (W1)

We agree that alternating local and global blocks is common in 3D, both in reconstruction (VGGT, MVGGT) and in part generation and decomposition (to our knowledge first in HoloPart, then in PartCrafter, X-Part and many other works). We would like to clarify that this alternation itself is not listed among our contributions (Sec. 1), which cover layout-conditioned synthesis of part-separated shapes, compositional editing, and our data pipeline.

What the "key insight" sentence in the introduction (Sec. 1) describes is how we adapt this alternation for layout-conditioned generation: local stages bind each part to its own box, while global stages, conditioned on the enclosing box, keep the parts mutually consistent. To remove any ambiguity, we have revised this sentence and now explicitly attribute the local/global design to prior work in Sec. 3 of our revised manuscript.


2. What mainly differentiates CompoSE from PartCrafter (W2)

We thank the reviewer for this question. Just like PartCrafter, we also build from the TripoSG backbone. However, while PartCrafter is focused on image-to-3D part decomposition, i.e., recovering the parts of the object shown in an input image, CompoSE is designed for part-conditioned generation and editing, where the user directly specifies the parts through a text prompt and a 3D layout. This leads to substantially different additions to the backbone.

From TripoSG, PartCrafter:

  • adds global attention to allow shape latents corresponding to different parts to attend to each other;
  • keeps TripoSG's image conditioning (DINOv2 features).

In our case, from TripoSG, we:

  • replace image conditioning with a dual-conditioning DiT: each block injects the 3D part layout and the text prompt (CLIP) through two parallel cross-attention branches;
  • encode each box with the same shape VAE as the parts, so that conditions and targets share one latent space;
  • bind each part to its own box in local stages, and condition global stages on the enclosing box;
  • introduce CFG Control Annealing to balance layout control and visual quality;
  • introduce a simple layout optimization procedure to avoid artifacts and improve layout alignment;
  • enable part editing (part addition, part substitution, text-based part editing and style-preserving resizing) through latent freezing and KV re-injection.

In common with PartCrafter, we:

  • add embeddings to the part latents;
  • use global attention, similarly to previous part-generation works (e.g., HoloPart, which also builds on TripoSG and predates PartCrafter), to allow parts to attend to each other.

We have updated the appendix of our revised manuscript (Appendix H.5) with this breakdown clarifying our contribution.

3. Why does PartCrafter perform poorly at the layout-to-parts generation task? (W2)

We first want to establish that our comparison with PartCrafter is fair:

  • It receives the same layouts and text prompts as our method. Since it only accepts images, each layout is rendered from a canonical viewpoint and turned into an image that follows the layout and the prompt.
  • Since PartCrafter allows it, we also supply the ground-truth number of parts in the layout.
  • This image is produced by Nano Banana Pro, a proprietary model that was, to our knowledge, the strongest available for this task at the time of submission.
  • It is not penalized for generating a different number of parts: it is given the ground-truth number of parts at inference time. The layout provides one bounding box per part, so it generates exactly as many parts as the ground truth.
  • It is not penalized for a global misalignment: each output is scored under 25 canonical orientations, and we keep the best one.
  • In the GPT-4o study, each comparison is run in both A/B orders, and the judge is told to treat boxes as loose bounding volumes rather than shapes to fill.

Our main point is that PartCrafter is not designed for structure-to-3D generation, so the image generator is only part of the explanation for its results:

  1. PartCrafter only sees the layout through an image, which Nano Banana cannot generate reliably and which loses occluded parts and depth.
  2. Layout control comes from how boxes are encoded, not from the local/global architecture: within our architecture, an MLP on box parameters reaches a Part-IoU of 0.289, against 0.729 with our encoding (Table 5).
  3. PartCrafter's outputs suffer from artifacts, which degrade both visual quality and layout following. This is a known failure mode, reported by subsequent part-generation papers comparing to PartCrafter (e.g., "fragmented parts" in FullPart, and parts with "ambiguous boundary" in X-Part).

4. Positioning with respect to CoPart, SALAD, DiffFacto and SPAGHETTI (W3)

We thank the reviewer for raising this point, which gives us the opportunity to clarify how CompoSE relates to these works.

SALAD, DiffFacto and SPAGHETTI. The reviewer quotes our main claim: "We present the first method to synthesize part-separated 3D shapes directly from coarse part layouts." However, these methods do not synthesize part-separated 3D shapes: SPAGHETTI and SALAD decode a single implicit field for the whole shape, and DiffFacto outputs labelled point clouds.

CoPart. CoPart does claim to take 3D boxes and text prompts as input. However, it is still unclear to us how well it follows an input layout:

  • It reports no IoU, at the object or part level, and no other measure of adherence to the input boxes. Its quantitative evaluation (their Table 1) only reports text-alignment scores (CLIP and ULIP), alongside a user preference study (their Table 2).
  • This only quantitative evaluation follows the text/image-to-3D protocol of CLAY, and the paper does not state whether boxes were provided, or where they came from.
  • No part-based method is compared quantitatively, and no evaluation on a test set of input layouts is reported.
  • Its figures do not show input layouts and prompts alongside the generated outputs on a test set.
  • Part editing is very limited: it is shown on a single object (their Fig. 5a, a two-part hammer), where the head is edited twice through its text prompt (one texture edit and one geometry edit). No part addition, removal or resizing is shown, and no input boxes are displayed.
  • Its code and weights are not released (the repository has announced "Code is coming soon" since July 2025), which prevents any comparison.

In contrast, we evaluate layout adherence quantitatively against the input boxes (Part-IoU, Object-IoU, Voxel-IoU) on a held-out test set, compare against both proxy-guided and part-based methods, and evaluate the preservation of untouched parts after editing.

Revised claims. Nevertheless, to avoid any potential overclaim, we have revised our main claim in the manuscript as follows:

BeforeAfter
"We present the first method to synthesize part-separated 3D shapes directly from coarse part layouts…""We present the first method to synthesize part-separated 3D shapes from a global text prompt and a layout of coarse oriented boxes…"

Summary of revisions

All changes made in response to this review are highlighted in purple in the revised manuscript. Changes also requested by the other reviewers are highlighted in yellow instead.

  1. Explicit attribution of alternating local/global blocks to prior work (VGGT, MVGGT, HoloPart, PartCrafter, X-Part) (Section 1 above).
  2. A component-level comparison with PartCrafter (Section 2 above).
  3. Revised claims in the introduction and conclusion (Section 4 above).

We thank the reviewer again for the thoughtful comments, which have clearly improved the paper. We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would consider raising their score. We would be happy to clarify any remaining questions.

Reviewer_gEftOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CompoSE, a generative model for 3D shapes. It supports part-level generation and editing, allowing users to modify individual parts while keeping the remaining parts unchanged. The model takes a global text prompt and a spatial layout of oriented bounding boxes that describes the arrangement of the object’s individual parts as inputs, and generates separate parts that together form the final 3D shape.

The method is evaluated through quantitative comparisons and pairwise preference evaluations using GPT-4o. Multiple visual examples illustrate its generation and editing capabilities. The reported results show improved generation quality and part-level editing performance compared with the evaluated baselines.

Critical strengths

S1. Part-level control and editing support iterative 3D content creation.

CompoSE allows users to control the spatial arrangement of individual parts through oriented bounding boxes. It supports part-level editing, including replacement, addition, and style-preserving resizing. These capabilities are valuable for iterative 3D content creation, as users can refine specific parts without regenerating the entire object. This gives users more direct control over their designs at the part level.

S2. The reported experiments demonstrate its generation and editing performance.

The quantitative results and pairwise preference evaluations demonstrate strong generation and editing performance compared with the evaluated baselines, supporting the effectiveness of the proposed method. The comparative experiments cover multiple aspects, and the paper includes sufficient visual examples in both the main text and supplementary material, helping readers assess the generated geometry and the effects of different editing operations.

S3. Automated data construction without manual part-level annotations.

The paper introduces a fully automated pipeline that derives part-segmented 3D shapes and their corresponding geometric layouts from existing 3D assets. It automatically segments and merges mesh components, extracts oriented bounding boxes (OBBs) for individual parts, and uses InternVL3-8B to generate global text descriptions from multi-view renderings. This pipeline substantially reduces the need for manual part segmentation and text annotation, enabling scalable training data construction while allowing the model to learn part semantics from global textual descriptions and geometric layouts.

Critical weaknesses

W1. The contribution of the local–global alternating design is not directly isolated.

The paper highlights the alternation between local part processing and global contextual aggregation as a key insight. However, the ablation studies that directly isolate the contribution of this design are missing. Related ablation studies should be provided. This is essential for assessing the paper’s contribution, as the authors identify the alternating design as a key insight.

W2. Insufficient evaluation of edit locality and identity preservation.

Although the paper supports part-level editing by freezing the latent variables of untouched parts and caching attention keys and values, it remains unclear how consistently these mechanisms preserve the geometry and identity of unedited parts across different editing operations. The paper would benefit from a more systematic evaluation of unintended changes to non-target parts, including geometric deviations and structural consistency before and after editing.

W3. Potential bias in layout adherence evaluation.

The layout optimization maximizes Part-IoU against a transformed input layout rather than the original user-provided layout. Consequently, the improved metric may not accurately reflect the model's adherence to the user's original layout constraints.

Questions for authors
  1. Are there any ablation results that directly isolate the contribution of the design that alternates between local part processing and global contextual aggregation?

  2. Appendix H.3 states that layout optimization maximizes Part-IoU against a transformed input layout while keeping the generated geometry unchanged. How does this step improve the final shape’s adherence to the original user-provided layout?

  3. How well does the proposed editing mechanism preserve unedited parts? Can the authors provide quantitative evidence of geometric consistency between non-target parts before and after editing, across different editing operations? In particular, how much unintended geometric change occurs in unedited parts, and does the preservation performance remain consistent across different shapes and editing scenarios?

  4. The layout optimization maximizes Part-IoU against a transformed input layout rather than the original user-provided layout. Could the authors report Part-IoU against the original layout before and after optimization, and quantify how much the transformation deviates from the original layout?

4 comments

Response to Reviewer gEft

We sincerely thank the reviewer for a thoughtful and encouraging review. We are glad that the reviewer found our part-level control and editing "valuable for iterative 3D content creation", that our experiments "demonstrate strong generation and editing performance" with "sufficient visual examples", and that our automated data pipeline "substantially reduces the need for manual part segmentation and text annotation, enabling scalable training data construction". We address each point below. All new experiments and clarifications have been included in our revised manuscript.


1. Empirical justification for Part-Local and Part-Global stages (W1, Q1)

As requested by the reviewer, we ablate the alternation of Part-Local and Part-Global stages with everything else fixed, and also measure how well each variant preserves the symmetries of the layout. Our model has 11 Part-Global stages; in each variant, we replace a subset of them (the first, middle or last 6 or 9, or all) with Part-Local stages. The global-block pattern column shows which ones: ● kept global, ○ replaced by a local stage. These results have been added to Appendix J.5 (Tables 7 and 8) of our revised manuscript.

Generation quality and layout adherence (on our full Objaverse test set of 1,000 assets):

VariantGlobal-block patternCLIP ↑Object-FID ↓Part-FID ↓Object-IoU ↑Part-IoU ↑Voxel-IoU ↑
Ours (alternating)●●●●●●●●●●●31.3610.589.240.8120.8190.550
First 6 local○○○○○○●●●●●30.8913.3313.050.7950.8090.456
Last 6 local●●●●●○○○○○○31.1610.878.790.8030.8140.537
Middle 9 local●○○○○○○○○○●30.3519.1433.580.7900.7230.467
First 9 local○○○○○○○○○●●30.5414.5418.430.7870.7870.450
Last 9 local●●○○○○○○○○○30.8811.8910.760.7960.8110.480
All local○○○○○○○○○○○30.5613.3412.840.7910.8170.462

Each badge is one of the 11 global stages of our model, from first to last: ● kept global, ○ replaced by a local-only stage. Local stages are unchanged in all variants.

Symmetry. For every pair of mirror-symmetric boxes in the layout (e.g., left and right armrests), we reflect one generated part across the layout's mirror plane and measure the Chamfer distance (CD) to the other generated part. A perfectly symmetric pair of generated parts gives CD = 0. We also report the F-score at 1% and 2% of the shape diagonal (F@1%, F@2%). GT parts give the attainable value.

VariantGlobal-block patternSymmetry CD ×10² ↓Symmetry F@1% ↑Symmetry F@2% ↑
GT (reference)—0.211 ± 0.0170.980 ± 0.0060.994 ± 0.003
Ours (alternating)●●●●●●●●●●●0.209 ± 0.0150.979 ± 0.0060.997 ± 0.001
First 6 local○○○○○○●●●●●0.640 ± 0.0890.872 ± 0.0170.935 ± 0.013
Last 6 local●●●●●○○○○○○0.677 ± 0.0610.825 ± 0.0160.928 ± 0.011
Middle 9 local●○○○○○○○○○●0.796 ± 0.0660.800 ± 0.0150.910 ± 0.011
First 9 local○○○○○○○○○●●0.798 ± 0.0630.780 ± 0.0170.904 ± 0.012
Last 9 local●●○○○○○○○○○0.835 ± 0.0980.789 ± 0.0180.905 ± 0.013
All local○○○○○○○○○○○0.970 ± 0.0960.737 ± 0.0190.871 ± 0.015

Computed over 458 symmetric parts (144 assets). Same badge notation as above.

Keeping all global stages (our full alternating design) gives the best text alignment, object quality, layout adherence and symmetry, matching the ground truth on symmetry (CD 0.209 vs. 0.211). Interestingly, concentrating local stages at the end (Last 6 local) slightly improves per-part quality (Part-FID 8.79 vs. 9.24), but at the cost of cross-part coherence: symmetry degrades sharply (CD 0.677 vs. 0.209), along with object quality and layout adherence.

2. Preservation of unedited parts (W2, Q3)

We thank the reviewer for this important question, and we have added a systematic evaluation of the preservation of untouched parts after editing, across shapes, numbers of edited parts and edit magnitudes, to our revised manuscript (Appendix K.3, Tables 11–13).

Protocol. We build 256 editing episodes from 256 distinct shapes of our Objaverse test set. In each episode, we generate a shape, edit 1 to 3 of its parts by resizing their boxes (independent scale factor per axis in [0.5, 1.5]), and regenerate with the same seed while freezing the latents of all other parts. This gives 602 untouched parts (and 408 edited parts). We compare each part before and after editing with the Chamfer distance, the F-score at 1% of the object size (F@1%), and voxel IoU. Edited parts serve as a reference for how much a part changes when it is edited.

By number of edited parts (all 602 untouched parts):

1 part edited2 parts edited3 parts editedAll untouchedEdited parts (reference)
Episodes / parts143 / 32874 / 17439 / 100256 / 602256 / 408
Chamfer ×10² ↓0.430.700.210.474.98
F@1% ↑0.9970.9941.0000.9960.530
Voxel IoU ↑0.9960.9930.9990.9950.455

By edit magnitude (quartiles of the mean absolute change of the edited box extents):

Q1 (smallest)Q2Q3Q4 (largest)
Untouched: Chamfer ×10² ↓0.240.230.240.28
Untouched: F@1% ↑1.0001.0001.0000.999
Untouched: Voxel IoU ↑0.9980.9990.9990.999
Edited: Chamfer ×10² ↓1.432.896.239.87
Edited: Voxel IoU ↑0.6760.4780.3640.295

Distances are in the normalized object space [−1, 1]³.

  • Untouched parts are preserved almost exactly: voxel IoU 0.995 and median Chamfer distance 0.17 (×10²), an order of magnitude below the change of the edited parts.
  • Preservation does not degrade with the number of edited parts or with the edit magnitude (see below).
  • Preservation is consistent across shapes: 600 of the 602 untouched parts have a voxel IoU above 0.95. The two exceptions are most likely caused by post-processing rather than by the model: their frozen latent decodes into two near-identical pieces, and keeping the largest connected component selects a different piece in the two runs.

Correlation analysis. We further investigate whether larger or more numerous edits disturb the untouched parts (Spearman ρ over the 256 episodes; edited parts as a positive control).

Correlation with edit magnitude:

MetricUntouched: ρ (p)Edited (control): ρ (p)
Chamfer ↓+0.10 (0.10)+0.81 (< 10⁻⁶⁰)
F@1% ↑−0.06 (0.37)−0.73 (< 10⁻⁴³)
Voxel IoU ↑−0.07 (0.29)−0.67 (< 10⁻³⁴)

Correlation with the number of edited parts:

MetricUntouched: ρ (p)
Chamfer ↓−0.12 (0.05)
F@1% ↑+0.03 (0.66)
Voxel IoU ↑+0.16 (0.01)

Neither how much nor how many parts are edited affects the untouched parts, whereas the change of the edited parts grows strongly with the edit magnitude.

3. Layout optimization and adherence to the original layout (W3, Q2, Q4)

We thank the reviewer for this question and apologize: Appendix H.3 was poorly worded, and we have corrected it in our revised manuscript. The optimization does not transform the input boxes. It estimates a single global similarity transform (one rotation about the vertical axis and one uniform scale) applied jointly to all generated parts, and Part-IoU is then computed against the original user boxes. It never changes the user input layout, never moves parts relative to each other, and never changes the geometry.

The arrangement, proportions and geometry of every part therefore come from the model alone: a global re-posing cannot fix a single misplaced or mis-scaled part. It only compensates a small global drift introduced when the conditions are encoded by the VAE. For completeness, we have added a pseudo-code algorithm for the layout optimization procedure in the appendix of our revised manuscript.

How layout optimization improves adherence (Q2, Q4). We refer the reviewer to Appendix H.3 for a detailed analysis. In short, Part-IoU against the original user boxes rises from 0.740 before optimization to 0.805 after it (Table 5, 512 tokens), with unchanged quality (Object-FID 14.44 → 14.40). As shown in Fig. 14 of Appendix H.3, the gain is concentrated on poorly aligned shapes: around +28% Part-IoU for the 10% most misaligned shapes, versus around +9% for the 50% best-aligned ones. This is consistent with the optimization only compensating a global drift from the VAE, which is larger for some shapes than for others.


Summary of revisions

All changes made in response to this review are shared with other reviewers, and are highlighted in yellow in the revised manuscript.

  1. A controlled ablation of the Part-Local and Part-Global stages, including symmetry metrics (Section 1 above).
  2. A systematic evaluation of the preservation of unedited parts after editing (Section 2 above).
  3. A clarification of the layout optimization protocol in Appendix H.3, with Part-IoU against the original boxes before and after optimization (Section 3 above).

We thank the reviewer again for the thoughtful comments, which have clearly improved the paper. We hope these additions address the reviewer's concerns, and we would be grateful if the reviewer would consider raising their score. We would be happy to clarify any remaining questions.

Dear Reviewer,

Thank you again for your thoughtful and encouraging review. As the discussion period is coming to an end, we wanted to kindly check whether our response has addressed your concerns.


In brief:

  • Contribution of the local/global alternation. Our new ablation isolates it: the alternating design performs best and matches ground-truth symmetry (CD 0.209 vs. 0.211), while turning global stages into local ones sharply degrades symmetry (CD up to 0.970) (App. J.5).
  • Edit locality and identity preservation. We have shown that untouched parts stay virtually unchanged after editing (voxel IoU 0.995 over 602 parts), regardless of how many parts are edited or how much (App. K.3).
  • Layout adherence evaluation. We have clarified that layout optimization only re-poses the generated parts, and Part-IoU is computed against the original user boxes; it rises from 0.740 to 0.805 (App. H.3).

We would be glad to clarify anything further. If our response resolves your concerns, we would be grateful if you would consider raising your score.

Reviewer_ZU2AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CompoSE generates separate 3D parts from oriented bounding boxes and a global text prompt. Its diffusion transformer combines local part conditioning with global attention. Part-latent freezing supports localized edits, and cached attention keys and values support identity-preserving resizing. Evaluation includes 1,000 held-out Objaverse layout-prompt pairs, component ablations, and quantitative and qualitative editing experiments.

Critical strengths
  1. Localized control has a concrete mechanism. Separate part latents and restoration of untouched latents provide a credible basis for retaining unedited parts (Appendix D). The edit sequences in Figure 5 demonstrate useful addition, substitution, and resizing capabilities, rather than only whole-object regeneration.
  2. Several design choices have direct empirical support. Table 5 shows substantial benefits from SDF-based box conditioning and improved visual quality from CFG control annealing. These ablations support useful components of the proposed pipeline independently of any leaderboard claim.
Critical weaknesses
  1. The contribution beyond prior geometry-guided generation is unclear. LATTICE and UltraShape already guide detailed generation with coarse geometry and spatially anchored voxel queries. CompoSE adds separate parts and editable layouts, but the contribution of its core local/global architecture is not isolated: Table 5 ablates encoders and inference strategies, not local/global alternation. A clear methodological comparison and controlled ablation are needed.
  2. The claimed 2D bottleneck lacks a relevant baseline comparison. CUPID and Pixal3D explicitly model 2D-3D correspondence and could transfer Nano Banana Pro's image-space control to visible 3D structure, yet neither is compared nor discussed. PartCrafter/OmniPart results alone do not establish an inherent limitation of 2D conditioning. A fair comparison is needed; hidden geometry and independent part editing remain separate issues.
  3. Layout adherence and structural coherence are not fully established. Appendix H.3 transforms the input layout without changing the generated geometry, yet Table 5 reports improved Part-IoU. If scoring uses the transformed boxes, this measures post-hoc alignment rather than adherence to the original constraints. Bounding-box IoU also cannot verify contacts or symmetry: Section 5 acknowledges these failures, while Appendix G claims structural coherence is unaffected without quantitative support.
Questions for authors
  1. What distinguishes CompoSE from LATTICE/UltraShape, and what evidence isolates the contribution of local/global alternation?
  2. Could pixel-aligned methods such as CUPID/Pixal3D provide effective structure control? Please include a fair comparison and discuss the claimed 2D bottleneck.
  3. Does layout optimization improve adherence to the original user constraints, or only alignment to transformed boxes? What evidence supports the structural-coherence claim?
6 comments

We sincerely thank the reviewer for a careful and constructive review. We are glad that the reviewer found that our localized control rests on "a concrete mechanism", that our edit sequences demonstrate "useful addition, substitution, and resizing capabilities, rather than only whole-object regeneration", and that our design choices "have direct empirical support". The reviewer's comments helped us substantially strengthen the paper. We address each point below. All new experiments and clarifications have been included in our revised manuscript.


1. Relation to LATTICE and UltraShape (W1, Q1)

Thank you for pointing us to these works, which we now discuss in the related work of our revised manuscript.

In summary. LATTICE and UltraShape are good at generating detailed shapes when the user does not care about a specific part layout (e.g., the number of legs on a chair, or of cushions on a sofa). CompoSE is good in scenarios where users want (1) to control which parts the object has, and where and how large each one is, and (2) to produce variations of a shape through sequences of identity-preserving part edits.

LATTICE and UltraShape are image-to-3D methods. Their coarse geometry is generated from the input image, not given by the user:

  • LATTICE: "The first stage generates the coarse sparse structure given the input image" (Sec. 3.2), using an off-the-shelf generator "such as Hunyuan3D-2 [58] or Trellis [48]" (Sec. 1).
  • UltraShape: "we directly adopt Hunyuan3D-2.1 as the first-stage model" (Sec. 3.1). In the refinement stage, "image conditioning is incorporated through cross-attention using DINOv2 features" (Sec. 2.2).
LATTICEUltraShapeCompoSE (ours)
Part-editable representation❌❌✅
Works without an input image❌❌✅
User directly controls the 3D structure❌❌✅
Outputs separate parts❌❌✅
Editing method and experiments❌❌✅

No editing. LATTICE only states that "Mesh Editing is also possible by manipulating the voxel queries" (Sec. 3.3), with no method or experiment. UltraShape only "explore[s] the potential of the proposed representation for training-free 3D stylization" (Sec. 2.2), which changes the whole shape. In CompoSE, each part has its own latent, so a part can be added, replaced or resized while all other parts stay virtually unchanged (voxel IoU 0.995 over 602 untouched parts; Appendix K.3, Tables 11–13).

Boxes are not coarse shapes. Their "coarse" structure is the full output of a strong image-to-3D generator, voxelized on a grid (128³ in UltraShape). It already contains the complete silhouette and thin structures. A box is only a loose bounding volume (one box can hold four chair legs), so the geometry must be synthesized, not refined. Refinement-style methods tend to fill boxes instead: SpaceControl (Table 1) reaches the best Voxel-IoU (0.776) by filling its proxies, but at much lower quality (Object-FID 42.96 vs. 10.58) and with no parts.

2. Structural coherence: contacts and symmetry (W3)

We agree that bounding-box IoU cannot verify contacts or symmetry. We have updated Appendix G accordingly, and now evaluate structure quantitatively in Appendix K.2 (Table 10) with three metrics:

  • Contact rate: for every pair of parts whose boxes touch in the layout (e.g., a leg and the seat), whether the generated parts actually touch (gap below 1% of the shape diagonal). We also report the mean gap.
  • Interpenetration: the mean intersection volume between parts, as a fraction of the smaller part's volume.
  • Symmetry: for every pair of mirror-symmetric boxes in the layout (e.g., left and right armrests), we reflect one generated part across the layout's mirror plane and measure the Chamfer distance (CD) to the other generated part. A perfectly symmetric pair of generated parts gives CD = 0.
MethodContact rate ↑Mean gap ×10² ↓Interpenetration ↓Symmetry CD ×10² ↓
Ground-truth shapes (reference)92.3 ± 0.8%0.34 ± 0.040.066 ± 0.0050.21 ± 0.02
PartCrafter + Nano Banana69.8 ± 1.8%2.90 ± 0.270.106 ± 0.0088.31 ± 0.73
OmniPart + Nano Banana83.8 ± 1.4%1.53 ± 0.200.067 ± 0.0055.18 ± 0.70
Ours89.1 ± 1.0%0.82 ± 0.140.079 ± 0.0060.21 ± 0.02

Gap and Chamfer distance are fractions of the shape diagonal.

CompoSE is the closest to the ground truth in contacts (89.1% vs. 92.3%) and gaps, and matches it exactly on symmetry (CD 0.21), while the baselines that generate parts are 25–40× less symmetric. This substantiates our claim that the model infers symmetries without explicit supervision. Interpenetration remains low and comparable to OmniPart and the ground truth. We still keep the stability limitation stated in Section 5: physical stability is a stronger property than geometric contact.

3. Empirical justification for Part-Local and Part-Global stages (W1, Q1)

As requested by the reviewer, we ablate the alternation of Part-Local and Part-Global stages with everything else fixed, and also measure how well each variant preserves the symmetries of the layout. Our model has 11 Part-Global stages; in each variant, we replace a subset of them (the first, middle or last 6 or 9, or all) with Part-Local stages. The global-block pattern column shows which ones: ● kept global, ○ replaced by a local stage. These results have been added to Appendix J.5 (Tables 7 and 8) of our revised manuscript.

Generation quality and layout adherence (on our full Objaverse test set of 1,000 assets):

VariantGlobal-block patternCLIP ↑Object-FID ↓Part-FID ↓Object-IoU ↑Part-IoU ↑Voxel-IoU ↑
Ours (alternating)●●●●●●●●●●●31.3610.589.240.8120.8190.550
First 6 local○○○○○○●●●●●30.8913.3313.050.7950.8090.456
Last 6 local●●●●●○○○○○○31.1610.878.790.8030.8140.537
Middle 9 local●○○○○○○○○○●30.3519.1433.580.7900.7230.467
First 9 local○○○○○○○○○●●30.5414.5418.430.7870.7870.450
Last 9 local●●○○○○○○○○○30.8811.8910.760.7960.8110.480
All local○○○○○○○○○○○30.5613.3412.840.7910.8170.462

Each badge is one of the 11 global stages of our model, from first to last: ● kept global, ○ replaced by a local-only stage. Local stages are unchanged in all variants.

Symmetry. For every pair of mirror-symmetric boxes in the layout (e.g., left and right armrests), we reflect one generated part across the layout's mirror plane and measure the Chamfer distance (CD) to the other generated part. A perfectly symmetric pair of generated parts gives CD = 0. We also report the F-score at 1% and 2% of the shape diagonal (F@1%, F@2%). GT parts give the attainable value.

VariantGlobal-block patternSymmetry CD ×10² ↓Symmetry F@1% ↑Symmetry F@2% ↑
GT (reference)—0.211 ± 0.0170.980 ± 0.0060.994 ± 0.003
Ours (alternating)●●●●●●●●●●●0.209 ± 0.0150.979 ± 0.0060.997 ± 0.001
First 6 local○○○○○○●●●●●0.640 ± 0.0890.872 ± 0.0170.935 ± 0.013
Last 6 local●●●●●○○○○○○0.677 ± 0.0610.825 ± 0.0160.928 ± 0.011
Middle 9 local●○○○○○○○○○●0.796 ± 0.0660.800 ± 0.0150.910 ± 0.011
First 9 local○○○○○○○○○●●0.798 ± 0.0630.780 ± 0.0170.904 ± 0.012
Last 9 local●●○○○○○○○○○0.835 ± 0.0980.789 ± 0.0180.905 ± 0.013
All local○○○○○○○○○○○0.970 ± 0.0960.737 ± 0.0190.871 ± 0.015

Computed over 458 symmetric parts (144 assets). Same badge notation as above.

Keeping all global stages (our full alternating design) gives the best text alignment, object quality, layout adherence and symmetry, matching the ground truth on symmetry (CD 0.209 vs. 0.211). Interestingly, concentrating local stages at the end (Last 6 local) slightly improves per-part quality (Part-FID 8.79 vs. 9.24), but at the cost of cross-part coherence: symmetry degrades sharply (CD 0.677 vs. 0.209), along with object quality and layout adherence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.