acceptodds
Under review as a conference paper at ICLR 2027

ARTUS: Articulated Joint Latents under Structural Control for Object Generation

Abstract

Interactive 3D assets require part geometry, connectivity and joint motion to describe the same object across changing poses. Learning their joint distribution is difficult because compact kinematic attributes and dense geometric features play different representational roles. A joint latent representation allows kinematic structure and part geometry to interact, but leaves the direction and timing of their interaction unspecified. If geometry is mixed in early, surface variation can affect the structural path representation before joint reasoning. If geometry is kept out entirely, generated geometry cannot inform subsequent hierarchy and joint-state predictions. We propose ARTUS, a joint latent model that addresses this tension through structure-controlled joint latent refinement, where kinematic structure governs the coordination of structural and geometric information. ARTUS encodes each root-to-node structural path and uses its representation to gate geometric features before joint reasoning. The resulting joint latent supports coordinated prediction of part kinematics and geometry. To incorporate realized geometry into subsequent predictions, ARTUS introduces articulation-pathway-coupled latent refinement, where the pathway-conditioned coarse geometry latent is partially noised and then refined by a frozen text-conditioned prior. The refined latent is reintegrated into the Existing Articulation Pathway for subsequent joint reasoning. Together, these components preserve structural control through gated fusion while allowing realized geometry to progressively update the articulation pathway for later predictions. Experiments on PartNet-Mobility and PM-Openable compare ARTUS with representative baselines spanning full fusion, fully staged generation, and structure–geometry latent fusion, and dissect the design through latent-construction ablations and refinement–feedback controls across surface fidelity, articulated layout, text consistency and motion compatibility measures. Code is available at https://anonymous.4open.science/r/ARTUS-4D68.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D articulated object generation, structure-controlled joint latent refinement, structure-gated latent fusion, articulation pathway, latent refinement, autoregressive generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pfXHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces ARTUS, a framework for articulated 3D object generation that coordinates part geometry, structure, and joint motion. To address the limitations of early feature fusion and fully staged generation, ARTUS proposes two mechanisms: (1) Structure-Gated Latent Fusion, which uses structural states and root-to-node paths to control the integration of geometric features; and (2) Articulation-Pathway-Coupled Latent Refinement, which refines coarse geometry latents using a frozen text-conditioned prior and feeds the refined geometry back into subsequent predictions. Experiments on 837 PartNet-Mobility objects across 12 categories show that ARTUS outperforms the evaluated baselines on all six metrics, with ablation studies supporting the effectiveness of its key components.

Critical strengths

Strength 1: The problem formulation is clear, and the two mechanisms address complementary aspects of structure–geometry interaction.

ARTUS does not simply concatenate structural and geometric features, nor does it postpone all geometry processing until after structural prediction. Instead, a structure-derived gate controls how geometric features enter the joint latent, while refined geometry is written back into the generation pathway. Conceptually, this corresponds to controlled fusion within a generation step and geometry feedback across steps, providing a reasonably clear connection between the motivation and the design.

Strength 2: The experiments cover several relevant dimensions, and the ablations help explain the source of the gains.

The main results report the best mean on all six metrics and include comparisons across articulation-complexity groups. Table 2 compares path, gate, and fusion variants, while Table 3 separates the effects of refinement initialization and pathway feedback. The appendix further studies the refinement starting timestep, stop-gradient, text conditioning, and inference cost. Together, these experiments provide more informative evidence than a single aggregate score and help explain the contributions of the individual components.

Critical weaknesses

Weakness 1: The dataset composition and the claim of evaluation on two datasets need clarification.

The abstract states that experiments are conducted on PartNet-Mobility and PM-Openable, and Section 4.1 also describes them together as data sources. However, Appendix C.1 states that 837 PartNet-Mobility objects are used, with PM-Openable annotations incorporated where available. The current description does not establish whether PM-Openable forms an independent test set or merely provides supplementary annotations for some of the same objects. Without independent PM-Openable test results, the results should not be interpreted as cross-dataset validation. In addition, the experiments focus on 12 indoor categories and revolute/prismatic joints, so the scope of external generalization remains limited.

Weakness 2: The train–inference context gap and post-refinement structural consistency are not sufficiently validated.

The authors explicitly state that training uses teacher-forced pathways containing reference structural states and encoded geometry, whereas inference uses predicted structural states and refined geometry. Refinement also does not receive gradients from the joint objective. The structure-controlled latent loss constrains the fused representation under teacher-forced structural context, but this is not equivalent to training a complete free-running feedback rollout. Table 3 supports the benefit of pathway feedback, but does not fully establish whether early structural or geometric errors accumulate along later steps. Moreover, the frozen geometry prior is conditioned on the coarse latent and text, without an explicit joint constraint as a separate input. POR evaluates part overlap over finitely sampled motion states and therefore does not fully establish geometric consistency across the continuous range of joint motion.

Questions for authors

Question 1 — Dataset protocol

Please specify the sources of the 837 objects, the role of PM-Openable annotations, and whether an independent PM-Openable train/validation/test split is used. If the reported results are obtained from the same set of PartNet-Mobility objects, please revise the dataset description in the abstract accordingly. If an independent test set is used, please report its size, split protocol, and results separately.

Question 2 — Teacher forcing and closed-loop inference

Training uses reference structural states and encoded geometry, whereas inference uses generated structural states and refined geometry. Have the authors evaluated step-by-step error accumulation during full autoregressive rollout, or compared performance under teacher-forced context and generated/refined context? Such results would help determine whether the latent matching objective sufficiently mitigates the train–inference gap.

Question 3 — Refinement and motion compatibility

Because the geometry prior does not receive explicit box/joint constraints as separate inputs during refinement, how do the authors verify that the refined geometry remains compatible with the predicted structural state? Could they report statistics on structure–geometry inconsistency before and after refinement, or evaluate compatibility over the continuous joint range using denser motion sampling and additional collision metrics?

Reviewer_VYw6OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies text-conditioned generation of articulated 3D objects, focusing on coordination between part geometry and kinematic structure. ARTUS lets joint and hierarchy information control the fusion of geometric features, then feeds refined part geometry into subsequent predictions. This allows later part layouts and joints to depend on the shapes already produced.

Critical strengths
  1. Using kinematic structure to control geometry fusion is a well-supported design choice. Similar part shapes can serve different roles depending on their joints and positions in the hierarchy, making structural context useful for deciding how geometric information should enter prediction. In Table 2, structural gating outperforms geometry-based gating, joint gating, and normalized direct fusion. These comparisons support the proposed direction of interaction.

  2. Refinement preserves useful structural context and improves articulated behavior. Table 3 shows that partially noising the coarse prediction before refinement outperforms starting from independent noise under the same frozen shape prior. This supports retaining the assembly information already present in the prediction. The shared-geometry comparisons in Appendix D.8 further support improvements in layout and motion compatibility, extending the benefit beyond surface detail.

Critical weaknesses

The added value of refined-geometry feedback is limited for a main methodological contribution. In Sec. 3.3, the refiner receives structural information indirectly through the coarse initialization, while the current part's joint state remains fixed. Refined geometry mainly influences subsequent parts through feedback. In Table 3, retaining the same partial-noise refinement without feedback already comes close to the full model. The evidence is therefore stronger for refining a retained coarse prediction than for the additional structural coordination attributed to feedback, limiting the extent of this coupling contribution.

Questions for authors

With the same structural gating and partial-noise refinement, which errors in subsequent part layouts or joint predictions does geometric feedback resolve? Given the small average gain in Table 3, are there particular part dependencies or structural configurations where feedback makes a substantial difference?

1 comment

Thank you for the updated experiments. The matched feedback-on/off comparisons address my concern about separating feedback from refinement and initialization. In addition, I think the dependency-depth analysis is particularly helpful: the improvements in downstream layout and joint prediction become larger for deeper part hierarchies, where earlier geometry matters more to subsequent predictions. My main concerns are now addressed, I have no further questions and will maintain my score.

Reviewer_scU8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ARTUS, a framework for text-driven generation of articulated objects. The method proposes a structure-controlled joint latent representation that coordinates kinematic and geometric information through structural path encoding and gated feature fusion. It further employs a diffusion model to refine part geometry, feeding the generated results back into subsequent autoregressive structural predictions. Experiments conducted on the PartNet-Mobility and PM-Openable datasets demonstrate that ARTUS outperforms existing methods in terms of geometric quality, part layout, text alignment, and kinematic compatibility.

Critical strengths
  1. The problem formulation is well-defined, focusing on the coordination between geometry and kinematic structure. Existing approaches to articulated object generation often either encode geometry and articulation jointly or predict the kinematic structure before generating individual parts. Both strategies have limitations: the former allows complex geometric features to interfere with structural prediction, while the latter fails to leverage already-generated geometry to inform subsequent structural decisions.
  2. The "Structure-gated Fusion" architecture is sound, and the integration of generated geometry back into the structural prediction process is an innovative design choice.
  3. The quantitative results are strong, and the ablation studies are comprehensive.
  4. The paper provides a thorough implementation and efficiency analysis.
Critical weaknesses
  1. The actual contribution of the "Geometry Feedback" mechanism appears relatively minor. As shown in Table 3, the addition of refined geometry feedback results in a reduction of only 0.09 in Chamfer Distance (CD) and 0.01 in Part Overlap Ratio (POR). In contrast, the improvement gained by moving from a "coarse-only" approach to an "independent prior" is more significant. This suggests that a substantial portion of the performance gain may stem from the pre-trained diffusion prior itself, rather than the geometry-feedback mechanism emphasized by the authors.
  2. There is a lack of direct physical or simulation-based validation. Although the paper highlights the suitability of the generated objects for robotics simulation and interactive environments, the experiments rely solely on static geometric quality, layout, and part overlap in discrete states.
Questions for authors
  1. Regarding the accuracy of kinematic structures and joint parameters: The paper primarily evaluates kinematic compatibility using the Part Overlap Ratio (POR). However, a low part overlap ratio does not necessarily guarantee that the predicted joint axes, joint positions, ranges of motion, and parent-child hierarchical relationships are correct. Given that a core objective of ARTUS is the joint generation of geometry and articulation, could the authors provide additional metrics to more comprehensively evaluate the quality of the generated kinematic structures?
  2. Physical usability of the generated assets. The paper positions the application of articulated generation within the contexts of robotics simulation and interactive environments; however, kinematic compatibility is currently assessed primarily through the geometric overlap rate at discrete joint states. Could the authors incorporate validation within an actual physics simulator—covering aspects such as articulation constraints, the validity of joint motion, collision-free movement, and the execution of interactive tasks—to further demonstrate that the generated objects are truly interactive in a physical sense?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.