acceptodds
Withdrawn from ICLR 2027

DualCraft: Dual-Workspace Generation of Articulated Assets from Images

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Interactive articulated objects such as doors and home appliances are crucial in robotics, but creating their simulation-ready assets remains labor-intensive. Generating them from images is hard because part geometry and motion structure must be recovered jointly: refining geometry changes assembly positions and clearances, while motion errors may require geometric fixes. We present DualCraft, a dual-workspace agent that models articulated objects from images without additional training. Its geometry workspace refines part shapes and materials against the images, and its program workspace defines the assembly and joints and checks their motion; shared part frames keep the two consistent. We further introduce RealProduct, 114 assets of 63 product types, each modeled by hand after a specific real product, with physically based materials and the interiors that motion reveals, and combine it with two recent collections into ARTS-Bench, a 690-object benchmark that scores multiview appearance and part geometry at rest and in motion. DualCraft achieves the best score on every ARTS-Bench metric against six baselines, reducing Chamfer distance in articulated states by 46% relative to a coding agent that uses the same model, and ablations show that removing its shared frames, visual feedback, or geometry checks each more than doubles Chamfer distance at rest. We further demonstrate that the generated assets can be used directly in simulated robot manipulation tasks that go beyond basic pick and place, and that trajectories planned with them in simulation transfer unchanged to a real robot. We will release the code, dataset, and benchmark.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Articulated Object Generation, 3D Asset Modeling, Robotic Simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ypr4OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DualCraft is a training-free agent framework for generating articulated 3D assets from images. It couples a Blender geometry workspace with a program workspace through shared part frames, using visual feedback and structural checks to iteratively refine geometry and articulation. The paper also introduces REALPRODUCT and ARTS-Bench, evaluates against six baselines, and demonstrates simulated manipulation and real-robot trajectory replay.

Critical strengths
  1. A meaningful joint modeling objective. The paper addresses the important challenge of maintaining consistency between visual geometry and executable articulation, including interior structures revealed during motion.

  2. Clear system design and useful controlled comparisons. Shared part frames and automatic assembly regeneration provide a practical consistency mechanism. Same-backbone baselines and component ablations help assess the benefits of the proposed workflow.

Critical weaknesses
  1. Clear engineering value, but limited methodological novelty. The system integrates modeling tools, shared coordinate frames, deterministic assembly generation, and feedback loops. These largely build on established engineering practices. The experiments demonstrate effectiveness, but do not sufficiently establish a novel mechanism or generalizable scientific insight beyond a task-specific agent workflow.

  2. Limited gains on some metrics and unclear cross-model generality. Codex-Native already achieves comparable appearance quality; articulated-state PSNR improves only from 21.44 to 21.47 dB. Although motion-state CD improves substantially and same-backbone comparisons support system-level benefits, evaluation exclusively with GPT-6 Astra leaves unclear how consistently the workflow adds value across foundation models.

  3. Limited evaluation scale and benchmark insight. Only 50 of the 690 benchmark objects are evaluated. Although results average three runs, confidence intervals and analyses by structural complexity are missing. The benchmark’s added diagnostic value beyond combining existing assets and metrics is also insufficiently demonstrated.

  4. Incomplete evaluation of articulation ranges and configurations. Geometry evaluation uses a fixed 90° span for revolute joints, while appearance evaluation caps rotation at 90°, potentially hiding errors beyond this range. Five synchronized configurations also provide limited coverage of independently moving parts and complex kinematic structures.

  5. Unclear recovery from incorrect initial blueprints. The refinement process operates around a predefined part organization, and the agent cannot create or delete part frames. Although geometry and joint parameters can be revised, recovery from missing parts or incorrect assembly topology is not clearly demonstrated, limiting the scope of closed-loop correction.

  6. Missing comparative validation through sim-to-real policy learning. Real-robot results demonstrate calibrated, open-loop trajectory replay, while the learned policy is evaluated only in simulation. No experiment compares policies trained using assets generated by different methods, leaving unverified whether DualCraft’s reconstruction improvements translate into better real-world policy performance.

Questions for authors
  1. What methodological principle distinguishes DualCraft from a conventional modeling agent equipped with the same tools, consistency constraints, and feedback? Could a matched-tool baseline isolate the contribution of the dual-workspace organization?

  2. Given Codex-Native’s already competitive performance, do DualCraft’s gains persist with other foundation models? Please compare both workflows across backbones and report paired per-object improvements and uncertainty to clarify which benefits are consistent.

  3. Can you expand the evaluation and report confidence intervals and results stratified by movable-part count, joint type, and kinematic-tree depth? What new conclusions about existing methods does ARTS-Bench enable?

  4. Can you evaluate complete predicted motion ranges without the 90° cap and sample independent joint configurations? Do the reported improvements persist under these more comprehensive motion evaluations?

  5. Can the refinement loop add missing parts, remove spurious parts, or revise incorrect parent–child relationships? Please demonstrate recovery from deliberately corrupted initial blueprints.

  6. Can you train the same policy using assets generated by DualCraft, Codex-Native, and other baselines, with matched training budgets and calibration procedures, and evaluate transfer to the same real objects? This would directly test whether asset-quality improvements produce measurable downstream benefits.

Reviewer_sgcJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DualCraft, a dual-workspace agent framework for generating simulation-ready articulated assets from single or multiple RGB images. The method first employs a vision-language model (VLM) to plan the part structure and initial joint configuration. It then connects a Blender geometry workspace and an articulation program workspace through shared part frames, iteratively refining the generated assets using visual feedback and structural checks. The paper also introduces RealProduct, a dataset containing 114 assets, and combines it with existing datasets to construct ARTS-Bench, comprising 690 assets. Experimental results demonstrate good performance in geometric reconstruction, appearance quality, and robotic manipulation tasks.

Critical strengths
  1. The paper addresses the consistency between geometry and motion structure in articulated asset generation. The dual-workspace design and shared part frames are intuitive and help reduce inconsistencies between geometric edits and joint configurations.
  2. The method requires no additional training and leverages existing vision-language models and Blender tools to generate assets with separate part geometries, materials, and joint definitions. The generated assets remain editable and can be exported as URDF models, facilitating downstream simulation and robotic manipulation.
  3. The experimental design considers the influence of the underlying model's capabilities on performance. In particular, Codex-Native and DualCraft use the same language model and the same modeling-turn limit, allowing a more direct assessment of the performance gains attributable to the system design.
Critical weaknesses
  1. The Part Blueprint in Section 3.2 predetermines the part decomposition and hierarchy, but the subsequent pipeline lacks a mechanism for replanning missing parts or incorrect topology. For parts that are completely invisible in the input images, the checks mainly rely on the agent's own inference, making structural errors difficult to detect reliably.
  2. The Pose Check in Section 3.4 primarily evaluates the generated motion against the predicted end-state descriptions in the Blueprint, without an independent basis for kinematic validation. When the initial motion assumptions are incorrect, this mechanism may fail to identify errors in joint types or motion ranges.
  3. ARTS-Bench primarily evaluates articulated asset quality indirectly through geometry and rendering metrics, without direct quantitative evaluation of joint types, axes, anchor points, motion ranges, or kinematic topology. Although the dataset provides the corresponding annotations, the existing metrics are insufficient to directly measure the accuracy of the generated assets' kinematic parameters.
  4. The method uses object dimensions estimated by the VLM as a reference for structural checks, while the geometry evaluation applies per-axis scaling to align generated assets with reference assets, and the real-robot experiments also use measured dimensions. Therefore, the current experiments do not directly validate the metric-scale accuracy of the generated assets.
Questions for authors
  1. If the initial planning omits parts or incorrectly predicts parent-child relationships, can the agent add, remove, or reorganize parts in subsequent iterations? If not, how does the system recover from an incorrect kinematic topology?
  2. In Section 3.3, shared part frames allow joint origins to be updated consistently, but joint axes and motion ranges are still maintained independently by the Joint Plan. When part geometry or assembly positions change, how does the system ensure that these joint parameters remain consistent with the modified geometry?
  3. In Section 3.2, camera viewpoints are primarily estimated by the VLM during initialization, with no explicit subsequent viewpoint refinement mechanism. When the estimated viewpoints are inaccurate, how does the agent avoid incorrectly attributing camera pose errors to geometric discrepancies?
  4. In Table 2, increasing the number of input views from four to six leads to higher Chamfer Distance in both rest and articulated states. Is this caused by viewpoint planning, instability in the agent's modeling process, or inconsistent visual judgments introduced by the additional views?
  5. In Section 4.3, the appearance evaluation in articulated states samples joint states according to the respective motion ranges of the generated and reference assets. When these ranges differ, how can rendering errors caused by geometry, materials, and joint motion be distinguished?
Reviewer_ApReOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents DualCraft, a framework for producing articulated 3D assets from 2D images of objects. The key idea is to use two workspaces for crafting and editing the object, namely: i) a Blender based geometry workspace that produces a blueprint of the object faithful to the provided input images, capturing the part meshes, joint types, motion axis, and their ranges, and ii) a program workspace that executes Python code to assemble the parts from the blueprint, and conducts structural tests using an agentic system. The paper also introduces a REALPRODUCT dataset, that extends the Arti-Bench dataset with 114 new and annotated assets. Experiments are provided on 50 assets including 20 from REALPRODUCT and show promising results.

Critical strengths
  1. The paper proposes REALPRODUCT dataset with additional articulated objects and ground truth annotations, thus extending Arti-Bench dataset, which is a useful contribution.
  2. The approach claims to produce simulation-ready assets that is a useful contribution.
  3. The experimental results, while limited, show promise against other methods. The video explanation is impressive and real-world qualitative experiments are insightful.
Critical weaknesses
  1. I think a key weakness of this paper is its superficial treatment of the technical details. As such, beyond Eq. (1) and (2) that state the parent-child connectivity between the frames of the parts and how they are related, it is unclear from the exposition what is the mathematical form of the cost that the model optimizes over towards fitting the generated 3D object to the 2D views? The paper describes visual inspection and structural tests, and as such, it appears that the loss depends entirely on the abilities of GPT-6-Astra in contemplating and editing the articulated meshes. If that is the case, then what is the scientific contribution of this work and how can the approach ensure the final generated asset converges to the ground truth? Specifically, beyond structural tests and visual inspections based on 2D images, how can the approach guarantee convergence to the real asset and what drives this convergence?

  2. Generalization studies of the approach to new objects are not provided. For example, experiments that use non-commonplace objects, such as industrial machine parts, could help understand if the approach is entirely dependent on the knowledge within VLMs about the object in the image(s) or if there is a capability induced through the pipeline that allows deriving non-trivial geometry and articulation from a few views of the object.

  3. The other contribution of this paper is the REALPRODUCT dataset. While this is a commendable extension, it is not clear how significant this contribution is, and in what ways does this extended dataset advance our insights into the inner workings of the approach or if it allows for training newer models? As such, beyond extending Arti-Bench, the significance of this new dataset is not established.

  4. The experiments are limited. The paper samples 15-20 assets to report the performance in Table 1, where this sampling strategy appears ad hoc. Further, the full 114 assets introduced have not been used in any manner in the experimental study.

  5. Only GPT6-Astra has been used as the underlying vision-language model (VLMs). If the method proposed is expected to be generic, it is important that the generalizability of the approach to other VLMs must be considered. Even when they may not perform well, they should perform better than their respective baselines using the DualCraft scheme. This is a significant omission in the paper.

Other comments:

a. The paper uses many (perhaps) engineering or Blender-based jargons that are not clearly defined. Some examples: L207: "Both workspaces follow one principle: each quantity in Eq. 1 has a single owner.". Why is this significant and how is this not handled in other approaches? Isn't this more of an engineering design perhaps specific to Blender than a general idea?

L213: "The agent edits a persistent Blender scene with Blender Python. Each part owns a frame, an axis-aligned empty object placed at the part’s pivot, and the part’s meshes are parented to it". How is pivot defined, how is frame defined? A picture depicting these would be intuitive.

L253: "A failed compilation or validation, a rejected completion check, or a new blocking discrepancy reopens geometry editing for a short rework window, so structural and motion errors flow back into the scene.". How many times is this done? What is the time taken for the edits and how expensive is this process? Is the automated process cheaper than human annotation and how does the time and cost compare?

b. It is not clear from the paper how single view images (esp. with V=1 in Eq. 1) can recover hidden articulations? It appears to me that the paper is relying on the strong inductive bias of the VLM. Further, it is unclear how the model knows the limits of the articulation, and constructs interiors of objects from single views. In that sense, how is the approach conceptually different from ArticulateAnything that also uses text, images, or video for asset synthesis?

c. The qualitative video shows manipulations that need physics (e.g., liquid containment in the coffee maker sample, push forces on the coffee handle, etc.) which are beyond the articulations proposed in this work and which are needed for the real-world robotic manipulation. The video also does not show the meshes generated by the method (e.g. for the Chamfer distance cases), and thus the quality of the generation beyond the ground truth has not been illustrated (as much as I see).

d. How does the model generate parts that could be separated from the object (e.g., the Rice cooker lid in Figure 3)? how will the Eq. (2) model capture such cases when only a single root is considered? In that sense, what is the definition of "articulation" used in this paper?

e. Given the paper considers articulation of the object parts, I think it is insufficient to limit the experimental comparisons to static evaluation metrics such as GIoU, Chamfer distance, PSNR, etc., and the paper should evaluate on all the entities that are predicted by the model in Eq. (2), including the type of joints, their predicted ranges, etc., against the ground truth. One could create videos of articulated objects from the corresponding single view viewpoints, and such videos could be compared to videos of articulation of the generated asset, on which one could use FVD or related scores.

Further to this, one could define success rate if all the parts are correctly generated, and a form of part accuracy may be used for some parts falling correctly into their places as in the ground truth. Such metrics are common in object part assembly frameworks (e.g., [a] below) and could be used here. Physics-based metrics could also be introduced referenced below. Authors are encouraged to consider such concrete evaluations than qualitative ones, as they are more meaningful in judging the physical plausibility of the predictions when the goal is robotic manipulation.

[a] AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects, Li et al, CVPR 2026.

Questions for authors

Please see above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.