acceptodds
Withdrawn from ICLR 2027

Daedalus: Verifier-Guided Agentic Refinement for Part-Structured 3D Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating 3D objects as editable mesh parts is essential for animation and interactive modeling, yet producing a decomposition that matches user intent remains challenging. Users must specify not only which parts to generate, but often their spatial extents, while incorrect controls or generated geometry require further correction. We present Daedalus, an agentic framework that connects semantic planning, controllable neural part generation, and verification-guided local refinement. Given a reference image, a part schema, and a whole-object geometric prior, an agent reviews semantic masks on mesh renderings, and geometric tools fit part bounding boxes from the selected surfaces. A mixed-condition generator then synthesizes parts using heterogeneous image, text, bbox, and voxel controls, supporting both individual instances and instance sets as semantic targets. Vision-language-conditioned pretraining establishes the geometric backbone, which is adapted to part generation with shared whole-object context. Verification guides targeted condition revision or part regeneration; candidates are accepted only when checks improve, while accepted non-target geometry is preserved. Experiments on PartObjaverse objects demonstrate improved part-level fidelity under spatial conditioning and benefits from agentic bbox refinement. Qualitative results further illustrate prompt-dependent decompositions and downstream articulation.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Agentic 3D Generation, Part Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5heBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Daedalus, a framework that generates a 3D object as separate mesh parts following a user-given part list. Given a reference image and a whole-object mesh, an agent first builds a bounding box for each part. SAM3.1 segments rendered views, a VLM checks the masks, and the accepted masks are mapped to mesh triangles to fit the boxes. A flow-matching generator then produces all parts jointly, and each part can use its own mix of conditions. A verifier finds problems, and the agent re-extracts, re-boxes, or resamples the affected part while other parts stay fixed.

Critical strengths

Letting the agent do choice while geometric tools compute the box coordinates is a reasonable design. It avoids directly asking a language model to guess 3D coordinates. Treating the part layout as something the system can revise, rather than a fixed user input, is an interesting view for part generation. The generator accepts different controls for different parts and supports both single instances and instance sets.

Critical weaknesses
  1. The method takes a complete mesh as input and fits boxes from its triangles, so the task is close to directly segmenting the mesh and then completing each part. The paper does not compare with such a baseline. Furthermore, boxes also come only from visible triangles, so hidden or internal parts get no spatial guidance.

  2. The verifier mainly checks whether each mesh is valid, such as whether it is empty, has holes, or breaks into fragments. These checks cannot tell whether a part has the right shape or meaning. The only signal tied to the user's intent is whether a part fits its box. But the box itself may be wrong. This makes it unclear how the system separates control errors from generation errors. Furthermore, the paper also gives no experiment on how the loop behaves in practice.

  3. The large gains in Table 1 come from boxes. These boxes already reveal how the reference object is split into parts, and the image-only baselines receive no such input.

Questions for authors
  1. How does Daedalus compare with HoloPart or X-Part given the same mesh and the same masks? How are hidden parts handled when they have no mask support?

  2. What exactly are the predicted boxes in Table 2? How much does the VLM review add over raw SAM3.1 masks?

Reviewer_Zh5nOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduced DAEDALUS, an agentic framework which, given a reference image, a target part schema, and a whole-object geometric prior, generates meshes for object parts, and assemble them back to the object. More specifically, it first uses SAM 3.1 and a pretrained VLM to acquire the multi-view part masks of the object, from which bboxes are derived. A part generator then uses the original input and the derived bboxes to generate the mesh for each part. Afterwards a verifier is used to check for imperfections of the result, and triggers necessary procedures to improve the generation result.

Critical strengths

The paper is generally easy to follow, and have visually pleasing figures.

Critical weaknesses
  1. The method description of this paper (particularly the verification-guided local refinement section, as well as the text in the appendix) seems not informative enough to build a specific enough picture on how the agent works to improve the generated results, as well as how to handle irrecoverable issues.
  2. Personally I think the novelty of this paper is not clearly expressed despite the summarized contributions.
  3. Contribution 3 mentioned " showcase flexible control and downstream articulation with the generated parts", but I did not seem to see related contents in the paper.
Questions for authors
  1. In Tab.1, are the baseline methods' use of agents or tool models (if any) replaced with most recent ones? Due to the fast progress of such models, the significant improvements may come from this distinction.
Reviewer_1B1qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Daedalus, a framework for part-structured 3D generation that combines mesh-grounded semantic localization, mixed-condition part generation, and verification-guided local refinement. The generator supports instance-level and instance-set targets with heterogeneous image, text, bounding-box, and voxel conditions. The refinement mechanism selects corrective actions while preserving accepted non-target parts and rolling back unsuccessful updates. Experiments demonstrate strong geometric fidelity with ground-truth bounding boxes and improvements from agentic bounding-box refinement.

Critical strengths

The paper addresses a meaningful problem: translating user-specified part semantics into editable 3D geometry with controllable decomposition granularity.

The mixed-condition interface is flexible, and the qualitative examples illustrate useful control over part structure.

The preservation and rollback mechanism is thoughtfully designed. Reusing accepted non-target meshes directly provides a clear way to prevent unintended changes during local refinement.

The paper acknowledges important limitations, including differences in conditioning across methods and the restricted scope of its verification checks.

Critical weaknesses

The central verification-guided refinement contribution is insufficiently evaluated. Table 2 measures the effect of refining bounding boxes before generation, rather than the effectiveness of the post-generation diagnosis and repair loop in Section 3.3. The paper does not quantitatively establish how often this loop detects and repairs errors, or whether it outperforms ordinary local resampling under the same generation budget. Moreover, Appendix D states that semantic and collision observations depend on externally supplied evidence where available, and that upstream schema and mask revision requests are not yet fully connected to automatic callbacks. The implemented scope and the components exercised in the experiments therefore need clarification.

The experiments do not sufficiently isolate the contributions of the proposed mechanisms. The strongest bounding-box-conditioned result in Table 1 uses ground-truth boxes, while the methods receive different geometric information. These results demonstrate the value of spatial conditioning but do not establish how much improvement comes from the agent or verifier. Agentic refinement reduces part-level CD from 0.1211 to 0.1129 and increases F-score from 0.8152 to 0.8376, which is encouraging, but variability and the additional computational cost are not reported. The voxel configuration also removes whole-mesh conditioning, so it does not isolate the effect of voxel input, as the authors acknowledge.

Several evaluation details limit interpretability and reproducibility. Section 3.1 emphasizes mask-grounded box construction, whereas Figure 2 includes an LLM initializer; Appendix C distinguishes these initialization paths without clearly mapping them to Table 2. Appendix E also refers to name-set checks and a “common valid evaluation set,” leaving unclear how unresolved targets, name mismatches, and generation failures affect the final averages.

Questions for authors

Which planning, verification, and repair components were active in each reported experiment? Which observations were produced automatically, and were any manual interventions used? On the existing evaluation set, how does verification-guided repair compare with ordinary local resampling under an equal sampling budget? Please report repair success, unresolved cases, and runtime. Which initialization path was used in Table 2, and did all 122 objects contribute to the reported averages? How were failures and name mismatches handled?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.