acceptodds
Withdrawn from ICLR 2027

ISArt: Single-Image, Structure-Guided Generation of Sim-Ready Articulated Assets

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generating simulation-ready articulated assets from a single image requires detailed part geometry and coherent kinematic structures. While Coding agents built on models such as GPT6-Astra can automate programmatic modeling and joint specification, faithfully reproducing intricate, image-specific part geometry through code remains challenging. We present ISArt, a closed-loop framework that combines agentic structural reasoning with learned part-level geometry generation. Given an input image, a vision-language model (VLM) iteratively decomposes the object into an editable structural representation, identifying its semantic parts, spatial extents, joint types, and parent–child relationships. For objects with concealed interiors, ISArt additionally synthesizes open-state references to reveal these structures. Conditioned on both visual observations and the inferred structure, a diffusion transformer generates detailed geometry for each part, followed by part-level texture synthesis. An articulation agent then estimates joint origins, axes, and motion ranges from the generated geometry and evaluates geometric and motion consistency. These feedbacks guide iterative refinement, fallback selection, and candidate acceptance or rejection. By assigning detailed geometry synthesis to a learned generative model and structural planning and articulation refinement to an agentic workflow, ISArt produces textured part meshes with URDF-defined kinematics for robotic simulation and interactive applications.

Primary area:
generative models
Keywords:
3d generation, real-to-sim, articulated object generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_q38eOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ISArt, a structure-guided framework for generating simulation-ready articulated 3D assets from a single image. It combines explicit geometric and kinematic reasoning with learned high-fidelity part geometry generation in a closed loop with simulation-based verification. ISArt first uses a VLM to infer part semantics, connectivity, joint types, coarse motion axes, bounding boxes, and cavity information, optionally synthesizing an open-state reference image to reveal occluded internal structures. These structural priors condition a multi-part DiT through per-layer embedding injection and structural cross-attention, guiding the synthesis of part geometry. An agentic articulation-binding module then estimates joint origins, axes, and motion ranges from the generated geometry and constructs a URDF. Finally, the framework uses Isaac Sim renders and video-based feedback to iteratively identify and correct invalid motions, producing validated, executable assets.

Critical strengths

None

Critical weaknesses

The experimental evaluation is limited and lacks comparisons with many relevant methods. Moreover, the related work discussed is not well aligned with the paper’s title and primary research focus.

Questions for authors
  1. The paper claims to propose a sim-ready 3D asset generation model, but I do not see how the physical attributes are obtained. If the method only produces articulated assets, how are the mass and dimensions of the model determined? These are variables required for simulation. Moreover, the authors mention PhysX-Anything and PhysForge, but why do they not compare with these sim-ready-focused methods, and instead compare with many articulated-asset methods? This is inconsistent with the paper’s stated goal.

  2. The authors spend a large amount of space introducing articulated-asset methods in the related works, but this does not match the paper’s title. Many sim-ready generation methods, including PhysX-3D, were proposed earlier than PhysX-Anything; PhysX-CoT and PhysX-Anything appeared around the same time; and PhysX-Omni is a relatively recent method. None of them are analyzed in relation to the proposed method in the related works.

  3. The model framework is very similar to those of Particulate and MonoArt, but the experiments do not compare with these related methods.

If the author can address those questions, I would like to raise my rating.

Reviewer_gSobOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces ISArt, a framework for generating simulation ready articulated assets from a single image. ISArt decomposes the object shown in the image into a structural representation. Using a structure-conditioned diffusion transformer, it then uses this structure to guide part-level geometry generation and, through agentic articulation binding and validation, ISArt converts the generated parts into a simulation ready articulated asset. The authors construct an evaluation benchmark of 60 articulated objects in order to evaluate end-to-end generation success rate, URDF binding accuracy and part-level geometry quality. They show that ISArt achieves better performance compared to the other baselines.

Critical strengths
  • The proposed pipeline is clearly motivated and reasonable.
  • The paper has clear structure and is easy to follow
  • The authors provide both qualitative and quantitative comparisons of ISArt with other methods and a thorough inference-time and training-time ablation study in Fig. 8.
Critical weaknesses
  1. A sim-to-real experiment is missing. A real world experiment where a model will be trained on the ISArt assets and tested in real world ones would be useful to see if the ISArt generated assets are effective in sim-to-real transfer?
  2. Many of the assets from the supplementary material do not seem to have high quality. For example in cabinet_0007 the surface is not smooth like an actual cabinet or in filling_cabinet_0876 the motion.mp4 the drawers are not high quality.
  3. The authors don’t include information about the total generation of time of an asset compared to the other methods. For example, it would be useful to include the generation time each method needed (including ISArt) for the assets and in Fig.4 and Fig.9
  4. There are missing some recent baselines. How is ISArt performs against other baselines such as Articract-Astra [1], SIMART [2], Particulate [3].

[1] Articraft: An Agentic System for Scalable Articulated 3D Asset Generation, Matt Zhou et. al

[2] SIMART: Decomposing Monolithic Meshes into Sim-ready Articulated Assets via MLLM, SIGGRAPH 2026

[3] Particulate: Feed-Forward 3D Object Articulation, CVPR 2026

Minor Weakness

  1. There are some minor typos in the text. For example, in page 2 Line 056 there is a word missing before contrast and in line 376 ‘We’ should be ‘we’.
Questions for authors
  1. How does ISArt perform against other procedural generated assets such as the ones mentioned in Infinigen-Articulated [4].

[4] Procedural Generation of Articulated Simulation-Ready Assets, Abhishek Joshi et. al

  1. Could the authors please provide quantitative results for the methods mentioned in Fig. 9, such as aDSL, PACT, SINGAPO and include them in the Table 2 with the other methods?
Reviewer_DQmxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ISArt, a framework for generating simulation-ready articulated assets by combining structure-grounded planning and part-level geometry generation with agentic articulation refinement. It infers explicit geometric and kinematic priors to guide high-fidelity part-level 3D generation and uses an agentic workflow for joint parameter estimation and validation.

Critical strengths

The paper explores multiple structural priors for articulated asset generation, which is a valuable direction. The agentic refinement and simulation-based validation further support the generation of executable assets. Experiments separately evaluate geometry quality and articulation binding, helping assess both aspects of the framework.

Critical weaknesses

The video demo shows the generation results but lacks visual results from the baselines. This makes it difficult to compare the kinematic motions of the generated objects. For showcases, in the cabinet0007 example, the mesh triangles appear to be flipped, or there are uneven artifacts on the top of the cabinet. After the door opens, the back panel has some artifacts at the bottom. Also, the input image is yellowish-white, whereas the generated object is neutral white. In the s1630toilet example, there are multiple inputs. It would be better to clarify the input and output files.

The authors introduce component semantics, bounding boxes, kinematic information, and internal cavity information. Ablation studies are needed to demonstrate the effectiveness of these injected features. Ablation studies of the validation process and comparisons between the proposed agentic articulation binding and rule-based binding methods are also missing.

Figure and writing quality of the paper: in the pipeline diagram in Figure 2, the icons become blurry when zoomed in. It would be better to use SVG format for the diagram. The 3D boxes shown for the microwave do not match the stated number of 3D boxes. Font in the Figure 3 is too small. Some equations and notation require clarification and keep consistent.

Questions for authors

Why is the training dataset generated using GPT-6 and BlenderMCP? Why not use existing datasets, such as PartNet-Mobility, to generate annotations or extract features for existing datasets?

If GPT-6 and BlenderMCP can generate a dataset of 1,100 assets, why not use them directly for this task? In addition, ArtiCraft appears to perform similarly to the proposed method in the qualitative study. Why is ArtiCraft not included in the quantitative comparison?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.