- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses the controllable 3D asset generation problem. It proposes a new framework Partial3D, which composes 3D assets from multiple partial images by adapting TRELLIS2 with spatial prompt modulation, cross-attention masking, and optional text-guided velocity fusion. The framework also supports component editing. Experiments and ablation studies demonstrate advantages over native 3D and indirect 2D-to-3D baselines.
Critical strengths
-
Composing a complete 3D asset directly from partial images without a holistic reference is a useful and well-motivated formulation.
-
The lightweight adaptation is supported by extensive comparisons and ablations covering spatial control, editing, and preservation of the pretrained generation prior.
-
The writing of this paper is good and easy to follow.
Critical weaknesses
-
Uncharacterized behavior under conflicting controls. Overlapping boxes allow the same voxels to attend to multiple image conditions, but yet the paper does not characterize how incompatible conditions in these regions are resolved. Similar uncertainty arises when text contradicts with the requested image/layout composition.
-
Incomplete backbone experiment. The paper only compares against TRELLIS.2 velocity averaging result, omitting the commonly used TRELLIS multi-image strategy of increasing the sampling steps and using a different conditioning image at each denoising step.
-
Limited spatial isolation. Cross-attention masking is only applied during sparse-structure generation, leaving final geometry and texture susceptible to cross-part interference, as suggested by the flowerpot in Fig. 4, whose texture does not faithfully match with the input image.
Questions for authors
-
How transferable is the proposed framework to other large pretrained 3D generative models, such as Hunyuan3D and Direct3D, given its reliance on TRELLIS2's sparse-structure pipeline?
-
Given training on complementary parts of conventional 3D assets, will generation quality degrade substantially when the input images depict semantically distant components at inference time?
3 comments
Summary of Manuscript Revisions (for Reviewer buFo)
We have uploaded the revised manuscript (with all updates highlighted in blue) incorporating all experiments, visual examples, and analyses addressing Reviewer buFo's questions: (1) on Pages 26–27 (Appendix C.13, Table 14), we added the ±5% bounding-box position/scale perturbation study on the 36 layout cases and analyzed model behavior under overlapping 3D boxes and conflicting text vs. image/layout controls; (2) on Page 22 (Appendix C.3) and Page 26 (Appendix C.12, Table 13), we added the step-wise reference-cycling TRELLIS2 multi-image baseline on the 30 image-only cases; (3) on Page 19 (Appendix B.7, Fig. 12) and Page 22 (Appendix C.4), we clarified that Fig. 12 visualizes cross-attention responses in the shape and texture DiTs, explained why spatial cross-attention masking is applied only during Stage 1 sparse-structure generation, and noted TRELLIS2's backbone capacity limits on fine surface detail; (4) on Pages 27–28 (Appendix D), we added an architectural transferability discussion covering cross-attention (Direct3D) and joint-attention (Hunyuan3D 2.0) 3D backbones and representation-specific spatial masking; and (5) on Page 15 (Appendix A, Fig. 7), Page 21 (Appendix C.1, Fig. 14), and Page 27 (Appendix C.13), we expanded our qualitative results with additional semantically distant, cross-style, and controllable editing cases (including a classical marble bust with modern headphones, a realistic pineapple with mechanical spider legs, and a cartoon dog with a realistic hotdog).
Your rebuttal and revisions
Thank you for providing your responses to my comments and questions. I am happy with your responses and do not have further questions. I will keep my original rating of Weak Acceptance.
Response to Reviewer buFo
We thank the reviewers for recognizing our use of partial images as "composable visual conditions" (uGq2), our "intuitive and useful control interface" (xgzP), and our "useful and well-motivated formulation" (buFo). We will carefully revise the manuscript in response to their valuable feedback.
Overlapping boxes. Our 3D boxes act as coarse spatial prompts that often intersect or nest (e.g., Fig. 5a–b). Since cross-attention masking only blocks voxels outside each box (Sec. 3.2), overlapping voxels attend jointly to multiple conditions via softmax, while SPM modulation and 3D self-attention maintain smooth transitions. Modest box errors are well tolerated on the 36 layout cases (Table 7):
If incompatible parts are assigned nearly identical boxes, shared voxels blend their features or favor the more contextually consistent part.
Text vs. image/layout. Text fusion acts only in Stage 1 with cosine decay (1 → 0), whereas Stages 2–3 condition solely on images (Fig. 3, Sec. 3.3). Thus, images strictly govern fine geometry and appearance, while severe text-layout contradiction yields a compromised coarse topology (mitigable by lowering w_start).
Our TRELLIS2 baseline averages per-image velocity predictions at every sampling step (Supp. B.4). We will add a step-wise variant that conditions each step on one image, cycling through the references.
Our layouts provide coarse spatial guidance for the overall shape, which is established during sparse-structure generation. We therefore apply spatial masking at this stage without imposing coarse box boundaries on subsequent shape and texture refinement. Following Fuse3D, we visualize cross-attention in the shape and texture DiTs (Fig. 12; Supp. C.4). In the shown examples, each partial image's attention largely concentrates on its corresponding region, with limited visible cross-part interference. We acknowledge the fine-detail fidelity remains limited by the representational and generative capacity of the TRELLIS2 backbone.
Our partial-image conditions are injected through cross-attention, and SPM modulates image features; neither operation intrinsically requires TRELLIS2's voxel representation. The same conditioning idea can therefore be adapted to vector-set models such as Hunyuan3D and triplane-based Direct3D through their image-conditioning attention interfaces. In cross-attention, 3D latent tokens act as queries and the modulated partial-image tokens as keys and values; Hunyuan3D 2.0 instead uses joint attention between latent and condition tokens. This requires adapting the conditioning interface and fine-tuning, while spatial masks need a representation-specific correspondence to 3D regions.
As described in Supp. B.2, 70 of our 100 evaluation cases combine independently generated parts from different source objects, rather than complementary parts of one object as in training. Fig. 7 includes a toast combined with cartoon facial features and a chest assembled from separate component references; Fig. 1 shows a violin-beetle hybrid. In particular, the toast-face and violin-beetle examples demonstrate coherent compositions from semantically distant inputs, beyond the same-object part completion used during training.