- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CRAFT, a framework for articulated object generation based on a reference image and a text prompt. The reference image provides appearance control, whereas text description primarily focuses on articulation. More specifically, the proposed approach repeatedly utilizes SAM3D to provide physical dimensions of the whole object as well as its parts, and assembles them together based on a structural configuration acquired via optimization.
Critical strengths
- The proposed topic of regulating the generated articulated object with human language is indeed meaningful.
- The paper is quite clearly written and easy to follow.
Critical weaknesses
- The proposed method appears slightly too complex for a pipeline, making it easier to break during generation. For example, in Eq.8, the authors formulated the object structure as an optimization problem with 6 types of losses. Without proper weight tuning, the optimization may not progress as expected.
- The proposed method may not be able to expand to more complex (rigid) objects due to its dependency on bounding boxes. In Sec 3.3.2, the authors mentioned penalizing bounding box overlaps, yet in reality, overlapping bounding boxes does not necessarily indicate overlapping parts.
Questions for authors
- In Eq.8, are the three "w"s manually or automatically selected? If the latter, how is it determined?
- In Fig.4 bottom left, I noticed the second generated cabinet (third picture) has door handles facing the wrong direction, is there a way to avoid this?
2 comments
I thank the authors for the response. However I do not think the response fully addressed my concerns.
Regarding W1, I think what the authors are suggesting is in fact "tuning is not used in the current optimization", rather than "tuning is not required in the current optimization". To support the latter argument, experiments are needed to confirm "the optimization results are not sensitive to weights".
As for W2, my concern remains on penalizing overlapping bounding boxes during layout planning. While I agree that such approach is "cheap" and "differentiable", I do not think it has the potential to support high-quality generation for more complex objects.
I will maintain my current rating.
We thank the reviewer for the positive comments on the topic and the writing.
W1. The optimization requires no weight tuning:
Regarding pipeline complexity: each stage has a single role (initialization, structural refinement, asset synthesis). The stages are decoupled through the proxy, so individual modules can be replaced.
W2. The bounding boxes serve as the control and layout layer, not as the final geometry. Text specifications become explicit on them (counts, positions, hinge axes), and the detailed part meshes are fitted to them. Because each part mesh is generated independently, assembling the meshes without a shared layout constraint can cause interpenetration and collisions during articulation, e.g., a drawer that cannot slide out. The box-overlap penalty is a cheap, differentiable way to prevent this.
Q1. The weights are selected automatically. For each instance, the structural agent Astru analyzes the input image and answers three binary questions: whether the movable parts are left-right symmetric, whether they are centered (and along which axes), and whether they fully cover the front face. Each positive answer sets the corresponding weight to 1, and each negative answer sets it to 0. We will add this description to Sec. 3.3.1:
Q2. Thank you for the careful observation. Handle placement is determined at the part-generation stage (Sec. 3.4). Part assets are generated by the image generator and SAM3D, and they are reused only through identity or reflection operations.
Because the joint axis of every part is explicit in our proxy, this can be avoided by enforcing that handles lie on the edge opposite the hinge axis, or by adding rotation to the reuse operations. Owing to the modular design, stronger part-generation models (better than SAM3D) can also be substituted directly as they become available.