acceptodds
Withdrawn from ICLR 2027

ArtiCritic: A Vision–Language Critic for Generated Articulated Objects

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Despite recent advances in articulated object generation, systematic evaluation of generated assets remains underexploredRecent breakthroughs in vision-language models (VLMs) have made it tempting to simply ask a VLM to judge an asset holistically, but such judgments tend to fall short on two fronts: they struggle to localize problems precisely in the image, and they reason poorly about fine-grained structure. We introduce ArtiCritic, a training-free, VLM-assisted framework that combines three-stage inspection with an overall review. More specifically, ArtiCritic decomposes evaluation into evidence-guided checks that progress from intrinsic validity to reference-based assessment of part completeness and articulation plausibility. Because reference objects and generated assets may represent structure at different levels of granularity, we introduce cross-granularity structural matching (CGSM), which associates reference parts with sets of candidate surface regions and checks their expected relative motions against corresponding rigid groups and joint paths. Each inspection localizes potential defects to specific components and records the supporting visual and structural evidence. A final VLM adjudicator then reviews these findings, filters unsupported claims, consolidates confirmed defects, and produces an overall assessment together with targeted repair suggestions. Experiments on manually constructed test cases show that ArtiCritic can reliably identify defective articulated assets while providing localized and actionable diagnostic feedback.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Articulated Object Generation, Vision–Language Models, Articulated Object Evaluation, Part-Grounded Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Zr7wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ArtiCritic, an automatic inspection framework that evaluates generated articulated objects along three complementary dimensions: intrinsic validity (URDF syntax, mesh integrity, self-intersection during motion), part completeness (whether reference-required functional components and instance counts are present), and articulation plausibility (whether the joint structure supports the motions the object should have). ArtiCritic builds reference part and joint inventories from segmentation masks, detector proposals, and motion proposals, then uses Cross-Granularity Structural Matching (CGSM) to align reference instances with candidate URDF links under one-to-many and many-to-one correspondences, accommodating heterogeneous modeling granularities. Defect candidates with localized evidence and confidence are finally adjudicated by a VLM. On 150 assets from three sources, ArtiCritic achieves 88.0% overall accuracy and 92.0% FAIL detection, outperforming Articraft, aDSL, and GPT-6-Astra while maintaining the highest PASS accuracy, and CGSM ablation confirms its tolerance to valid link decompositions and sensitivity to motion violations.

Critical strengths

Strength 1: A systematic, evidence-grounded inspection framework. The paper evaluates articulated assets through three complementary checks: intrinsic validity, part completeness, and articulation plausibility. It outputs verdicts with localized evidence, confidence, and repair suggestions, rather than relying only on executability or visual plausibility. This is consequential because it turns evaluation into an actionable diagnostic protocol, especially useful for code-based generation pipelines.

Strength 2: Reference inventories plus Cross-Granularity Structural Matching (CGSM). Reference part and joint inventories are built independently of the candidate, so missing components or motions can still be detected. CGSM allows one-to-many and many-to-one correspondences between reference instances and candidate URDF links, handling heterogeneous modeling granularities. This is the key mechanism that makes part- and joint-level validation work in practice, and the ablation shows it accepts legitimate link decompositions while rejecting unintended movable connections.

Critical weaknesses

Weakness 1: Limited methodological novelty; the contribution is mainly system orchestration. The pipeline combines off-the-shelf components—Semantic-SAM, PartGLEE/VLPart, OPD, and a VLM adjudicator—and CGSM is essentially a correspondence heuristic for one-to-many and many-to-one matching. The paper introduces no new representation, learning formulation, or theoretical insight. This matters because the claimed contribution is a “systematic inspection framework,” yet the technical delta over prior inspection and VLM-critique pipelines is incremental. For a venue expecting methodological novelty, acceptance is difficult unless system-level utility is demonstrated more decisively than the current evaluation does.

Weakness 2: The evaluation does not adequately support the breadth and reliability of the claims. The dataset is concentrated on storage furniture and microwaves, with only a small random sample of other articulated objects, so generalization to multi-joint mechanisms, folding structures, or other categories remains untested. The framework also depends on reference images and VLM adjudication, but the paper provides no systematic analysis of sensitivity to incomplete references, viewpoint mismatch, or VLM adjudication consistency and error modes; confidence is explicitly evidence strength rather than a calibrated probability. In addition, Articraft and aDSL are evaluated outside their integrated refinement loops, which the authors acknowledge but which still weakens the comparative conclusions. These issues matter because the central claim—that ArtiCritic provides systematic and reliable inspection across generated articulated assets—is not fully established beyond the curated distribution and the specific reference conditions tested.

Questions for authors

1.The articulation-plausibility check in ArtiCritic overlaps in goal with non-LLM motion-estimation methods previously proposed by SFU, such as Shape2Motion and OPDPN. Could the authors compare ArtiCritic’s joint-plausibility judgments with these methods on a benchmark where at least one of them is applicable, for example MultiScan or a subset of the Shape2Motion dataset? If a direct comparison is infeasible, please explain why and discuss the conditions under which a non-LLM baseline may already be sufficient, as well as where the incremental value of ArtiCritic’s VLM adjudication is mainly expected to lie.

  1. The paper acknowledges that the reference inventory depends on reference images and that VLM confidence is not a calibrated probability, but it does not systematically analyze either issue. When reference images contain occlusion, incomplete viewpoints, or semantic ambiguity, how much do CGSM matching accuracy and the final PASS/FAIL verdict degrade? How consistent is VLM adjudication across repeated runs or minor prompt variations? These questions bear on the lower bound of the framework’s reliability in practical deployment.

  2. The dataset is concentrated on storage furniture and microwaves, with only a small random sample of other articulated categories. For multi-joint mechanisms such as folding chairs or robotic joint chains, or for continuously compliant articulations, does CGSM’s dichotomy between “independent motion” and “fixed attachment” still hold, or does it require fundamental extension? This would help determine whether the framework’s scope is general-purpose articulated-object inspection or category-specific inspection of furniture and household appliances.

  3. The authors acknowledge that Articraft and aDSL are originally integrated into generation pipelines, so evaluating them in isolation may underestimate their capabilities. Did the authors attempt to use defect reports produced by their own pipelines as input to ArtiCritic, or conversely, to inject ArtiCritic’s localized evidence into their refinement loops and observe whether this yields measurable gains? Such an analysis would more fairly position ArtiCritic’s distinctive contribution.

Reviewer_s9reOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces ArtiCritic, a training-free framework for evaluating generated articulated 3D objects. It combines intrinsic validity checks, reference-based part and joint inventories, Cross-Granularity Structural Matching (CGSM) between semantic parts and URDF links, and VLM-based adjudication. On a dataset of 150 assets, it reports 88.0% overall accuracy, compared with 82.0% for its strongest reported baseline. The problem is important and the pipeline is reasonably motivated. However, the experiments do not yet isolate the contributions of its main components or establish their reliability across diverse articulated objects. My principal concerns are the lack of quantitative ablations, limited evaluation diversity, and insufficient control of tool access and inference budgets in the baseline comparison.

Critical strengths
  1. Evaluating articulated assets requires verifying functional components and motion, beyond visual appearance alone. The paper addresses a relevant practical problem.

  2. The separation into intrinsic validity, part completeness, and articulation plausibility offers an interpretable inspection workflow combining structural evidence with VLM reasoning.

  3. CGSM addresses a genuine modeling issue: semantic parts need not correspond one-to-one with URDF links. Handling rigid groups and independent-motion requirements is potentially useful.

  4. The reported results are encouraging. In particular, defect detection improves from 84.0% to 92.0% relative to the strongest reported baseline, while PASS accuracy is also slightly higher.

Critical weaknesses
  1. The incremental methodological contribution is not clearly established. Although integrating pretrained models is reasonable, the incremental value of CGSM remains unclear. Its semantic-to-URDF mapping and rigid-group checks are not compared with simpler part-to-link matching and graph verification. The benefit of VLM adjudication over a structured verification prompt is also untested. These comparisons are needed to substantiate methodological novelty.

  2. Missing quantitative ablations of the core components. This is my primary concern. Section 4.4 shows three qualitative inventory examples, while Section 4.5 evaluates CGSM on two original assets and four modified variants. Neither establishes their effect on final PASS/FAIL accuracy. Quantitative ablations of inventories, CGSM, deterministic checks, and VLM adjudication are needed, alongside simple matching and structured-VLM baselines with comparable evidence and inference budgets. Without these, the source of the reported gains remains unclear.

  3. Limited dataset diversity and uncertain generalization. The 150-asset benchmark concentrates on storage furniture and microwaves; Source-3 includes 71 artificially corrupted variants and 22 unmodified assets. Many corruptions directly target parts, joints, and limits checked by ArtiCritic, leaving generalization to naturally occurring defects and complex articulations unclear. The authors should report natural-defect results separately and clarify whether variants of the same source asset are grouped across data splits.

  4. Baseline resource parity and statistical reliability remain unclear. The reported improvement from 82.0% to 88.0% accuracy is promising, but its statistical reliability is not assessed through paired tests or repeated runs. The paper already provides GPT-6-Astra with structured inspection instructions and uses the same VLM, which is a useful control. However, equivalent access to localized evidence, specialized tools, and inference budgets is not established. A resource-matched comparison would strengthen the results. Table 2 also reports identical abnormal-joint-motion accuracy (90.6%), suggesting the gains should be characterized by defect type.

  5. Intermediate reliability and error propagation are not evaluated. Reference inventories and semantic-to-URDF correspondences determine which defects can be checked, yet their accuracy and error propagation are not quantified. Missing parts or incorrect matches may lead to false negatives or false positives. The authors should report inventory precision/recall, correspondence accuracy, and how final VLM adjudication affects these errors. Repeated-run stability would further support claims of reliable inspection.

  6. Intrinsic validity and repair claims lack corresponding validation. The evaluation does not clearly isolate intrinsic-validity failures, despite the claimed URDF, mesh, and collision checks; Section 4.1 emphasizes part, joint, and motion criteria. Their coverage in PASS/FAIL labels and category-level performance should be clarified. The effectiveness of repair suggestions is also untested. A repair-success study and complete prompts, model configurations, matching thresholds, motion-sweep settings, and runtime costs would strengthen these claims and reproducibility.

Questions for authors
  1. How much does CGSM improve final PASS/FAIL accuracy over simple part-to-link matching with rigid-group or graph-based checks? Can the authors provide end-to-end component ablations?

  2. What are the precision/recall of the reference inventories and the accuracy of semantic-to-URDF correspondences? How do their errors propagate to the final verdict?

  3. How are corrupted variants grouped across development and test splits, and how well does ArtiCritic perform on independently generated assets with naturally occurring defects and more varied articulations?

  4. What information, specialized tools, and inference budgets are available to the GPT-6-Astra baseline? Are the gains robust under matched conditions and paired statistical evaluation?

  5. Are intrinsic-validity-only defects included in the PASS/FAIL ground truth? What are the corresponding detection results, and do the proposed repair suggestions lead to successful corrections?

Overall, ArtiCritic addresses a meaningful problem with a structured and potentially useful inspection pipeline. Its results and treatment of semantic-to-URDF granularity differences are encouraging. However, the missing quantitative ablations, limited and partly synthetic evaluation, uncertain parity of baseline resources, and unmeasured intermediate reliability leave central claims insufficiently supported. These concerns relate to the strength of the evidence, not proof that the approach is ineffective. Addressing them would require substantive additional experiments rather than minor clarifications. Thus, a rejection is proposed.

Reviewer_5ugaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents ArtiCritic, a system for checking generated articulated objects using reference images and a candidate URDF asset. It combines structural and geometric checks with VLM-based checks of part completeness and joint motion. Reference part and joint inventories describe what the object should contain, while cross-granularity structural matching allows these requirements to be matched to different link decompositions. A final VLM reviews the reported issues and produces a pass/fail decision with localized feedback. On 150 manually annotated assets, the method achieves 88% overall accuracy.

Critical strengths

The method checks whether an object has the required parts and whether those parts can move as expected. This helps identify problems, such as missing handles or joints, that a general visual check may miss. It also considers different valid ways to organize parts and joints, which is useful for avoiding false alarms caused by differences in how an object is modeled.

Critical weaknesses
  1. The paper does not report inference time or computational cost. Since ArtiCritic involves multiple detection, rendering, and VLM review stages, reporting runtime, VLM calls, and API cost per asset would help assess whether the accuracy gains justify the additional computation compared with the baselines.

  2. It is unclear whether ArtiCritic’s complex pipeline is still needed with a stronger VLM. The authors should compare the full pipeline with simpler versions using GPT-6. This would help show whether the proposed components remain useful as the underlying model becomes stronger.

  3. The paper states that ArtiCritic provides repair suggestions, but the examples mainly show detected defects and pass/fail decisions. I could not find examples of such suggestions.

  4. The main components are not sufficiently evaluated. The reference-inventory ablation shows only three qualitative examples, and CGSM is tested on two original objects and four modified variants.

Questions for authors
  1. In the paper, which VLM do you use?

  2. Were the predicted defect types, affected parts or joints, and repair suggestions checked against human annotations? If so, how accurate were they?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.