acceptodds
Withdrawn from ICLR 2027

ArticuTable: Generating Instance-Level Interactive Rigid-Articulated 3D Tabletop Scenes from a Single Image

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Embodied agents benefit from 3D environments that combine visual fidelity to real-world observations with physical interactivity. Existing single-image tabletop reconstruction methods recover plausible scene geometry but typically represent objects as monolithic rigid bodies, limiting interaction to whole-object rigid motion and precluding executable part-level articulation. Meanwhile, recovering a scene layout consistent with the input view remains challenging because a single observation may admit multiple plausible pose–scale configurations. We present ArticuTable, a single-image 3D tabletop reconstruction framework that recovers both executable part-level articulation and an input-view-consistent scene layout. For object modeling, we introduce generation-robust articulation modeling (GRAM), which combines joint fitting guided by a multimodal large language model with semantic state reasoning to recover reliable joint parameters and valid motion ranges from imperfect monolithic proxy meshes, thereby converting them into executable articulated assets. For scene layout, we introduce progressive semantic-geometric scene registration (PSGSR), which progressively narrows the pose–scale search space under complementary metric, planar, and input-view constraints and resolves orientation ambiguity through structure-aware semantic correspondences, yielding a scene layout consistent with the input view. We further contribute ArticuTable-100, a curated collection of 100 simulation-ready tabletop scenes. Extensive evaluation, including a user study, demonstrates strong performance across visual fidelity, input-view consistency, articulation quality, physical plausibility, and simulation readiness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Real-to-Sim, Single-Image 3D Reconstruction, Articulated Object Modeling, Tabletop Scene Reconstruction, Embodied AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_C71vOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a framework for generating interactive articulated tabletop scenes with rigid articulated objects and introduces a collection of tabletop scenes as a new benchmark. The paper introduces two new modules. GRAM for MLLM-guided joint fitting and URDF generation, and PSGSR for semantic pose reasoning and recovery of scene layout. The results of ArticuTable are compared against relevant methods on both lightweight articulation dataset and scene-level datasets, across mask-based, motion-based, and visual-based metrics, including a user study.

Critical strengths
  • The paper introduces a well-performing framework for the task of single-image interactive scene generation. Though limited to tabletops, the results show promise and the task has application in downstream robot learning tasks.
Critical weaknesses
  • I'm concerned about the complexity of the framework that requires two layers of segmentation and MLLM-guided joint estimation. Each consequent step seems to rely heavily on the previous step to be highly accurate to follow. I would love to see some step-by-step robustness study that will validate the chain of operations and show how noisy are the results these intermediate steps produce.
  • The paper makes repeated claims on downstream robotic tasks but does not show how well the generated scenes work in simulation.
Questions for authors
  • How reliable is the instance segmentation step? How does noise at this step influence the two modules?
  • How reliable is the part segmentation step, especially since part segmentation of a monolithic object mesh is a challenging problem. How does noise at this step influence the joint estimation?
  • Is the joint estimation from MLLM reliably accurate?
  • Is it possible to include movable part and static part Chamfer Distance instead of just the whole object Chamber Distance?
6 comments
AuthorsOpenReview ↗

Response to Reviewer C71v — W1

Thank you for suggesting a step-wise robustness study and clarification of the pipeline's complexity. The revised manuscript now includes stage-level quality and error propagation in Appendix D.8, module responsibilities and workflow in Appendix D.4, and execution costs in Appendix F. We discuss each below.

Overall pipeline robustness (Appendix D.8, Table 13 of the revised manuscript). Across 150 scenes containing 1,338 reference instances (including tables), we evaluate stage errors and their effects on downstream outputs. Stage error rates use all scenes or instances as denominators; robustness to preceding-stage errors is defined as the proportion of scenes whose current module outputs remain unaffected among scenes with an issue in at least one preceding stage.

Stage / moduleStage errors ↓Stage error rate ↓Scenes affected by preceding-stage errors / scenes with preceding-stage issues ↓Robustness to preceding-stage errors ↑
View generation10 / 150 scenes6.67%——
Instance segmentation12 / 1,338 instances0.90%0 / 10100.00%
3D geometry generation12 / 1,338 instances0.90%2 / 2090.00%
GRAM——1 / 3196.77%
PSGSR——2 / 3193.55%

Scenes with issues in any preceding stage are counted once.

Evaluation at the instance level further shows that 83.33% (10/12) of instances with segmentation issues and 91.67% (11/12) of instances with geometry-generation issues do not affect subsequent outputs.

Segmentation and geometry generation have low stage error rates, and most issues in this evaluation do not lead to downstream quality degradation. Even when evaluation is restricted to scenes with preceding-stage issues, most outputs of subsequent modules remain unaffected. This indicates that local upstream errors typically do not accumulate through the pipeline, supporting robustness to errors encountered under actual generation conditions.

Workflow complexity and execution cost (Appendices D.4 and F of the revised manuscript). The overall pipeline follows the established instance-wise reconstruction-and-registration paradigm. Building on this pipeline, we refine instance extraction and scene registration and add GRAM articulation modeling, which converts generated monolithic meshes into executable assets supporting part-level interaction.

The two segmentation levels discover scene instances and recover functional links within objects, respectively. The added articulation-modeling module is functionally decoupled from scene registration, so rigid proxy meshes can be registered and assembled independently.

Instance recovery (Appendix D.1, Table 7 and main-paper Table 1). We change “MLLM-based instance analysis → segmentation → 3D generation” to “anonymous instance segmentation → MLLM-based analysis → 3D generation,” reducing semantic omissions without adding sequential stages. We compare the same 150 scenes with 1,338 reference instances, including tables:

MethodRetained reference instances ↑Missing instances ↓Instance retention ↑Accurate scenes ↑Scene accuracy ↑LPIPS ↓DINOv2 ↑CLIP ↑ColO (%) ↓ColS (%) ↓Mean alignment time (s/scene) ↓
TabletopGen1,17216687.59%5939.33%0.39350.77880.89081.448816.001105.29
ArticuTable (Ours)1,337199.93%14898.67%0.23980.89910.92650.381910.67141.40

TabletopGen timing excludes camera search.

Higher instance retention and scene accuracy indicate more complete content; improved visual metrics and fewer intersections indicate better layouts, with lower alignment time.

Appendix F of the revised manuscript now reports batch-processing costs for the complete pipeline, from input images through simulation completion:

ItemValue
GPU configuration8 × NVIDIA GeForce RTX 4090
Number of scenes150
Total batch wall-clock timeApproximately 14 h 26 min
Amortized wall-clock time5.77 min/scene
Processing throughput10.39 scenes/hour

Amortized time is total batch wall-clock time divided by the number of scenes and describes batch-processing efficiency under this multi-GPU parallel setting.

Appendix D.3, Table 10 reports part-segmentation and complete-chain GRAM evaluations; main-paper Table 4 and Appendix D.5, Table 11 report joint-estimation ablations. We also discuss these results under Q2–Q3 below.

AuthorsOpenReview ↗

Response to Reviewer C71v — W2

Thank you for requesting evidence of how the generated scenes work in simulation. The revised manuscript now reports physical interaction evaluation across all 150 scenes in Appendix G.4, with representative dynamic examples in §4.6, Fig. 6.

Scene-level interaction evaluation across the evaluation set (Appendix G.4 of the revised manuscript). We drag every object and operate every joint of each articulated object from the lower to the upper endpoint of its specified motion range, evaluating link detachment, nonphysical interpenetration, and contact responses with surrounding objects throughout the motion.

The physical interaction evaluation found no obvious nonphysical interpenetration or implausible inter-object contact responses; link detachment occurred in 13 articulated assets, corresponding to 3.11% of the 418 judgeable articulated assets (13/418). Detachment is counted at the asset level and reported separately from inter-object contact responses. This evaluation of each object and joint complements the initial-configuration collision evaluation and covers whole-object movement and part-level interaction within the specified joint ranges.

We allow plausible contact during articulation: when nearby objects lie in a moving part's path, contact-induced blocking or displacement is an expected physical outcome. The relevant failures are nonphysical interpenetration or implausible contact responses.

Dynamic examples in Isaac Sim. Section 4.6, Fig. 6 and Appendix G.3 of the revised manuscript demonstrate gravity and support contact, toolbox dragging, and articulated opening after directly loading the exported USDZ scenes. Physical contact transports the contained object with the toolbox; after the toolbox is displaced, its lid can still open through the recovered joint while the object remains inside. These examples demonstrate that rigid-body motion, collision geometry preserving the interior, and part-level articulation work together in the assembled scene.

These evaluations cover the executability of physical interactions in the generated scenes. Robotic manipulation and learning are potential applications; robot-policy performance and training benefits remain unevaluated.

AuthorsOpenReview ↗

Response to Reviewer C71v — Q1

Thank you for asking about instance-segmentation reliability and its downstream effects. Appendix D.8 of the revised manuscript now reports segmentation reliability and downstream effects. Among 1,338 reference instances (including tables), instance segmentation has 12 instances with issues, corresponding to a stage error rate of 0.90%; their downstream effects are summarized below:

Failure modeInstances with issuesInstances with no downstream effectsDownstream effects
Over-segmentation87One instance causes an extra object to be generated
Missed segmentation43One instance leaves an object missing from the final scene
Total1210Two instances affect downstream outputs

Most extra segments are removed by subsequent processing, while evidence from the other view compensates for most omissions in a single view. Consequently, among instances with segmentation issues, 83.33% do not affect subsequent outputs, demonstrating tolerance to these segmentation errors during instance recovery.

The omitted and additionally generated objects in this evaluation are both rigid, so they do not affect GRAM articulation modeling; for PSGSR, they appear as missing or extra objects in the final scene.

Reviewer_bifbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ArticuTable addresses the problem of reconstructing simulation-ready articulated tabletop scenes from a single RGB image. The framework aims to preserve consistency with the input view while recovering both scene-level object arrangements and executable part-level articulation. It introduces two main components. GRAM (Generation-Robust Articulation Modeling) converts monolithic proxy meshes into articulated assets by combining geometric primitive decomposition, MLLM-guided joint fitting, and semantic state reasoning to estimate joint parameters and motion ranges. PSGSR (Progressive Semantic-Geometric Scene Registration) progressively refines object poses and scales through point-cloud alignment, top-view registration, and input-view refinement, with structure-aware semantic pose selection to resolve orientation ambiguities. The paper also introduces ArticuTable-100, a curated collection of 100 simulation-ready tabletop scenes containing visual meshes, collision geometry, and articulated assets.

The evaluation includes scene-level comparisons against ACDC, Gen3DSR, MIDI, and TabletopGen on 150 generated tabletop images; articulation evaluation on 243 Lightwheel assets; component ablations; manual inspection of generated articulated assets; simulation-oriented validation; and a user study with 59 participants. The reported results show improvements over selected baselines in several image-consistency and collision-related metrics, as well as competitive joint-estimation performance.

Critical strengths

A practical and well-integrated framework for simulation-ready articulated scene reconstruction

The paper addresses an important problem for embodied AI: reconstructing tabletop scenes from a single RGB image while preserving input-view consistency and enabling executable part-level articulation. The integration of GRAM for articulated asset generation and PSGSR for scene registration is technically motivated, and the method goes beyond visual reconstruction by explicitly considering joint parameters, collision geometry, and simulation compatibility. The paper also provides a relatively broad evaluation, including scene-level comparisons, component ablations, articulation benchmarks, manual asset inspection, and a user study. The reported results show promising improvements in visual consistency and initial scene-level collision metrics, making the work potentially useful for constructing interactive environments for embodied AI.

Critical weaknesses

1. The methodological novelty of GRAM and PSGSR relative to the closest prior work remains insufficiently established

For GRAM, the paper combines geometry-based joint initialization, MLLM-guided joint correction, and semantic motion-range inference. However, its methodological distinctions from existing articulation approaches remain insufficiently clear. Articulate AnyMesh (Qiu et al., CoRL 2025) already employs vision-language reasoning for semantic part decomposition and functional joint construction, while MotionAnyMesh (Xu et al., 2026) combines kinematics-aware part segmentation, geometry-based joint initialization, and physics-grounded articulation optimization. Although Table 2 reports quantitative comparisons on the Lightwheel benchmark and Table 3 provides component ablations, these experiments do not fully establish the specific advantages of GRAM's joint-correction and semantic motion-range reasoning mechanisms over the closest prior methods, particularly on the imperfect image-to-3D meshes that motivate the proposed approach. In particular, MotionAnyMesh is discussed as related work but is not included in the quantitative comparison. A more direct comparison or a clearer algorithmic analysis would help establish the distinct contribution of GRAM beyond these existing approaches.

Similarly, PSGSR combines point-cloud alignment, top-view registration, and input-view refinement, with SASPS providing structure-aware semantic pose selection. However, TabletopGen (Wang et al., ECCV 2026) already employs top-view spatial alignment and differentiable rotation optimization for tabletop scene reconstruction. Although Table 4 demonstrates the benefits of PSGSR's individual components under the reported image-space metrics, it does not sufficiently clarify which aspects of the proposed registration strategy constitute a substantive algorithmic advance over this closely related approach. The concern is not that the proposed modules are identical to prior methods, nor that the reported improvements are unsupported, but that the conceptual and algorithmic contributions beyond the closest predecessors remain insufficiently isolated and substantiated.

2. The evaluation does not sufficiently validate the claimed robustness and 3D reconstruction accuracy

A central motivation of GRAM is to handle artifacts in image-to-3D proxy meshes, such as fused part boundaries and distorted contact geometry. Although the paper manually audits generated assets and provides qualitative comparisons with existing methods, it does not provide a comprehensive quantitative comparison against closely related articulation methods on the same set of imperfect generated meshes. Consequently, it remains unclear how much GRAM improves robustness over existing approaches under the specific failure conditions it is designed to address.

Furthermore, the scene-level evaluation uses 150 generated 2D tabletop images without paired 3D ground-truth scenes. The reported image-space metrics (LPIPS, DINOv2, and CLIP) primarily measure consistency with the input view, while collision statistics measure internal scene validity rather than agreement with the true 3D scene. Importantly, single-view reprojection consistency does not imply accurate 3D geometry or spatial layout: incorrect object depths, scales, and relative 3D positions can still produce similar projections from the input camera due to the inherent depth-scale ambiguity of monocular reconstruction. Thus, the current evaluation cannot directly establish the accuracy of the reconstructed 3D geometry and object poses. Evaluation on synthetic or real scenes with known 3D geometry and object poses, using metrics such as 3D bounding-box IoU, pose errors, and Chamfer distance where appropriate, would provide stronger evidence for the claimed reconstruction quality.

Questions for authors
  1. Clarification of methodological novelty. Could the authors clarify the precise algorithmic differences between GRAM and the closest prior methods, particularly Articulate AnyMesh and MotionAnyMesh? Specifically, beyond using MLLM-guided joint correction and semantic state reasoning, what aspects of GRAM's underlying procedures constitute substantive advances over existing articulation modeling methods? Similarly, for PSGSR, which aspects of PMCR and SASPS go beyond the registration and pose-selection mechanisms already explored in TabletopGen? It would be helpful to identify the existing experiments that directly support these incremental contributions, rather than only the overall system-level improvements.

  2. Scope of the 3D reconstruction claims. Since the 150 scene-level test images do not have paired 3D ground-truth scenes, are the authors' claims intended primarily to concern input-view consistency and internal scene validity, rather than the accuracy of recovered 3D geometry and object poses? Are any quantitative evaluations against known 3D geometry or object poses available but not reported in the paper? Clarifying this distinction would help determine the extent to which the current experiments support the paper's 3D reconstruction claims.

5 comments
AuthorsOpenReview ↗

Response to Reviewer bifb — W1 / Q1

Thank you for asking us to clarify the algorithmic differences from the closest prior work. Our contributions address artifact-sensitive joint estimation and motion-range modeling (GRAM), and initialization under single-view pose ambiguity (PSGSR). The revised manuscript supports these contributions through component ablations, the main-paper generated-mesh comparison, and efficiency and instance-recovery evaluations in the appendix. We connect each algorithmic distinction to its supporting evidence below.

(1) GRAM: global correction of geometric joint hypotheses and separation of observed state from intrinsic motion range. Articulate AnyMesh constructs axes from selected visual markers. GRAM first fits type-specific 3D hypotheses, then corrects axes/pivots using global appearance, structure and functional evidence, allowing evidence beyond distorted contact regions to inform joint estimation.

MotionAnyMesh refines joints through trajectory geometry and determines limits through collision/contact-loss scans. GRAM explicitly separates the observed configuration, canonical zero and intrinsic motion range: after canonical-state recovery, functional semantics and geometric feasibility inform intrinsic limits. This reduces reliance on distorted contacts and missing internal geometry (§3.3; Appendix B.2–B.3). MotionAnyMesh had no runnable public implementation when we conducted the comparison (project page).

Component ablations and generated-mesh comparisons in the revised manuscript provide complementary evidence. The 2×2 ablation (Tables 4 and 11 of the revised manuscript) isolates component effects on Lightwheel: joint correction reduces mean AE from 40.06° to 17.17°; semantic state reasoning reduces mean Art. PC from 0.2605 to 0.2502. Table 2 also shows gains over Articulate AnyMesh: AE 21.70° → 17.26°, Art. gIoU 0.158 → 0.283. The revised main paper (§4.3, Table 3) compares the complete method on 399 matching, judgeable generated-mesh pairs, yielding 83.71% pass for GRAM versus 39.10% for Particulate (protocol in Appendix D.3 and discussed in W2). Together, these evaluations establish component utility and the full method's advantage on imperfect generated inputs.

(2) PSGSR: progressive constraint propagation and structure-aware semantic yaw scoring. TabletopGen derives translation/scale from top-view boxes and primarily optimizes orientation through differentiable rendering. PSGSR progressively refines position, scale and yaw under point-cloud, top-view and input-view constraints. Each stage initializes the next; the original input view corrects residual top-view errors.

For candidate selection, TabletopGen uses DINOv2 feature differences in its rotation loss. SASPS scores structure-weighted bidirectional DINOv2 patch correspondences by their spatial displacement, incorporating correspondence geometry into yaw initialization. Table 5 directly supports the designs: input-view refinement reduces LPIPS from 0.3235 to 0.2543; SASPS further reduces it to 0.2398. Fig. 5 shows local orientation errors after removing structure weighting.

Top-view search saves 3D rendering cost (Appendix F, Table 16 of the revised manuscript). Front-only SASPS retains the same candidate generation/selection but renders candidates in the input view. On the same 150 scenes and one RTX 4090 (s/scene, mean ± SD, excluding queueing):

VariantRenderingLPIPS↓DINOv2↑CLIP↑Time↓
Full PSGSR—0.23980.89910.9265141.4 ± 14.0
Front-onlyBatched0.23310.89460.9268401.9 ± 72.1
Front-onlySequential0.23330.89410.9270574.8 ± 114.8

Comparable visual consistency requires 2.84×/4.07× the runtime with batched/sequential Front-only rendering. Inexpensive 2D top-view rotations restrict 3D rendering to residual symmetric candidates.

Additional framework results (Appendix D.1, Table 7 and main-paper Table 1). Anonymous instance segmentation precedes MLLM analysis, reducing semantic omissions without adding sequential stages. On the same 150 scenes (1,338 reference instances, including tables):

MethodRetained↑Missing↓Retention↑Accurate scenes↑Accuracy↑LPIPS↓DINOv2↑CLIP↑ColO(%)↓ColS(%)↓Align. s↓
TabletopGen117216687.59%5939.33%0.39350.77880.89081.448816.001105.29
ArticuTable1337199.93%14898.67%0.23980.89910.92650.381910.67141.40

Alignment time is mean s/scene; TabletopGen excludes camera search. These results assess scene completeness, visual consistency, intersections and overall alignment cost.

AuthorsOpenReview ↗

Response to Reviewer bifb — W2 / Q2

Thank you for your suggestions on robustness and 3D evaluation. As stated in the abstract and method, ArticuTable reconstructs tabletop scenes from one RGB image with input-view-consistent layouts, plausible 3D structure, physical interaction and part-level articulation. We add paired generated-mesh results (§4.3, Table 3) and visible-depth consistency evaluation (Appendix D.7, Table 12).

(1) Paired evaluation on actual generated meshes. Section 4.3, Table 3 of the revised manuscript compares GRAM and Particulate on 399 actual generated assets, using matching input meshes and the shared expert-rating protocol in Appendix D.3.

Construction of the paired subset. The full GRAM audit covers 425 candidate assets. Available Particulate evaluation records cover 407 assets. Excluding eight assets that could not be validly assessed in the paired comparison leaves 399 judgeable asset pairs with matching inputs. Both methods are evaluated on this same subset. The remaining 18 candidates without paired results are excluded from this table rather than automatically counted as Particulate algorithm failures. The table therefore reports output quality on this paired subset.

MethodPass ↑PartialFail ↓
Particulate156 (39.10%)59 (14.79%)184 (46.12%)
GRAM (Ours)334 (83.71%)51 (12.78%)14 (3.51%)

On this paired subset, GRAM achieves an 83.71% pass rate, 44.61 percentage points higher than Particulate, with a failure rate of 3.51%.

Rating and independent re-evaluation. A common primary expert rates both methods using the same structure and motion criteria. Method identities are hidden and outputs are presented in random order; each multi-joint asset receives its worst joint grade. A second expert independently re-evaluates random samples of approximately 20% of the outputs: 85 GRAM candidate assets and 81 Particulate assets. The re-evaluation yields 83.7% pooled asset-level agreement and Cohen’s κ = 0.703.

Articulate AnyMesh and URDF-Anything+ also trail GRAM in joint estimation and articulated-geometry recovery on the standard Lightwheel benchmark (Table 2): their AE values are 21.70° and 49.90°, and their Art. gIoU scores are 0.158 and −0.110, respectively, compared with 17.26° and 0.283 for GRAM.

We also tested Articulate AnyMesh and URDF-Anything+ on the same 19 judgeable generated assets. Under the same structure and motion evaluation criteria, both methods receive 0 pass, 0 partial, and 19 fail ratings. Considering processing costs and texture-adaptation requirements, we selected Particulate, which performs comparably to GRAM on the standard Lightwheel benchmark, for the larger-scale quantitative comparison.

(2) Visible geometry and depth consistency. Appendix D.7, Table 12 of the revised manuscript reports evaluation on the same 150 scenes, using Depth Anything V2 predictions from the original RGB images as depth references. All methods use scene-level scale alignment based on common visible tabletop pixels, with penalties for missing foreground.

MethodCoverage ↑FG IoU ↑Object Macro ↓Penalized AbsRel ↓Capped CD ↓
ACDC0.35900.24120.75770.68340.0361
MIDI0.22410.16880.84160.80460.0543
TabletopGen0.33370.27090.76560.69760.0330
Gen3DSR0.80330.69890.25810.25620.0144
ArticuTable0.97250.93740.09920.10070.0158

Coverage and FG IoU measure foreground completeness and projected layout consistency; Penalized AbsRel measures relative depth error with missing-pixel penalties, and Object Macro is its equally weighted per-object average; Capped CD compares normalized visible depth point maps under shared camera rays. ArticuTable leads the first four metrics, with Capped CD close to the best result.

We built ArticuTable-100 because we found no benchmark jointly covering tabletop layouts, collision geometry and executable articulation. Evaluation of the collection and its extension covers object sizes, visible geometry, relative poses and contact plausibility, and includes simulation validation. Our scene evaluation uses LPIPS, DINOv2, CLIP and collision metrics, as in TabletopGen, to assess visual consistency and physical plausibility; standard articulation benchmarks, paired generated-mesh comparisons and simulation assess the added part-level interaction.

Evaluation against the input images reveals object omissions, size mismatches and orientation errors in the comparison results. These differences agree with the quantitative results and demonstrate our advantage in input-view consistency and visible-layout recovery; recognizing them does not require exact 3D GT. The depth evaluation in Appendix D.7 of the revised manuscript measures visible-surface consistency against predicted depth.

AuthorsOpenReview ↗

Follow-up on Our Responses

Dear Reviewer bifb,

Thank you for your time and feedback. Our point-by-point responses clarify the algorithmic distinctions from prior work and the scope of our 3D evaluation, and include an additional paired comparison on generated meshes.

As the discussion period approaches its end, we would appreciate your feedback on whether these clarifications and additional results address your main concerns. If any concerns remain, please let us know which points require further clarification.

Thank you for your consideration.

Reviewer_KJTSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ArticuTable, a framework for reconstructing simulation-ready 3D tabletop scenes with rigid and articulated objects from a single RGB image. It introduces Generation-Robust Articulation Modeling (GRAM) to recover executable articulated assets from imperfect meshes and Progressive Semantic-Geometric Scene Registration (PSGSR) to improve scene layout consistency with the input view. Experimental results demonstrate improvements in visual fidelity, articulation quality, and physical plausibility over existing methods.

Critical strengths

The paper addresses an practical problem of reconstructing simulation-ready 3D tabletop scenes with executable articulation from a single image, with potential applications in embodied AI and robotic manipulation. The proposed framework demonstrates strong experiment results across multiple scene reconstruction and articulation metrics.

Critical weaknesses
  • W1. Limited Methodological Novelty and Pipeline Complexity. The proposed framework consists of a highly complex, multi-stage pipeline that integrates existing techniques, including view synthesis, instance segmentation, image-to-3D generation, MLLM-based reasoning, and geometric optimization. Rather than introducing substantial methodological novelty, the approach appears to be primarily an engineering-oriented integration of existing components. The overall pipeline seems a bit heavy and complicated.
  • W2. Insufficient Analysis of Error Propagation and Pipeline Robustness. Since the framework consists of multiple sequentially connected modules, errors introduced at early stages may propagate through subsequent stages and degrade the final reconstruction quality. Although GRAM and PSGSR incorporate mechanisms to mitigate certain errors, it remains unclear how effectively the framework can handle inaccuracies in view synthesis, instance segmentation, or geometry estimation. A systematic evaluation of error propagation and robustness to upstream inaccuracies is lacking.
  • W3. Limited Quantitative Comparison on Generated Articulated Meshes. One of the main contributions is recovering reliable articulation from imperfect image-generated meshes. However, quantitative evaluations are primarily conducted on the Lightwheel benchmark, while evaluations on actual generated meshes rely largely on manual assessments and qualitative comparisons. Thus, the method's quantitative advantages over existing approaches under realistic mesh imperfections remain unclear.
  • W4. Insufficient Evaluation of Scene-Level Physical Consistency Across Articulation States. Although the paper emphasizes simulation-ready articulated scenes, scene-level physical validity is primarily evaluated at initial object configurations. While individual articulation quality is assessed, physical consistency across different articulation states, including potential collisions with surrounding objects, remains insufficiently evaluated.
Questions for authors

I would appreciate it if the authors could address the concerns raised in the Weaknesses section, particularly regarding the methodological novelty and complexity of the proposed pipeline, its robustness to error propagation across multiple stages, the limited quantitative comparisons on image-generated meshes, and the evaluation of scene-level physical consistency across different articulation states.

10 comments
AuthorsOpenReview ↗

Response to Reviewer KJTS — W1

Thank you for your comments on methodological novelty and pipeline complexity. GRAM and PSGSR address unreliable articulation inference on generated meshes and pose ambiguity in single-image scene registration (§3.3–3.4). The revised manuscript now includes module responsibilities and workflow in Appendix D.4 and computational costs in Appendix F.

(1) GRAM: semantic correction of geometric joint hypotheses and canonical-state range reasoning. GRAM constructs type-specific 3D joint hypotheses, then corrects axes and pivots using global multimodal and structural evidence. Whereas Articulate AnyMesh constructs axes by selecting visual markers, GRAM corrects fitted 3D hypotheses to handle unreliable local contact geometry. It also separates the observed configuration, canonical zero, and intrinsic motion range, combining functional semantics with geometric feasibility to infer limits. Table 4 isolates these designs: joint correction reduces AE from 40.06° to 17.17°; semantic state reasoning reduces Art. PC from 0.2605 to 0.2502. Table 2 shows gains over Articulate AnyMesh: AE 21.70° → 17.26° and Art. gIoU 0.158 → 0.283.

(2) PSGSR: progressive registration and structure-aware pose selection. TabletopGen estimates translation/scale from top-view layouts and orientation through differentiable rotation optimization. PSGSR progressively refines position, scale and yaw under point-cloud, top-view and input-view constraints, passing each estimate to subsequent optimization. Before differentiable refinement, SASPS ranks yaw candidates by the weighted spatial displacement of structure-weighted bidirectional DINOv2 patch correspondences. Table 5 supports both designs: with SASPS disabled, input-view refinement reduces LPIPS from 0.3235 to 0.2543; enabling SASPS further reduces it to 0.2398.

Together, these methods extend single-image tabletop reconstruction to executable part-level interaction while retaining input-view layout consistency.

System complexity (Appendix D.4 of the revised manuscript). We retain the conventional instance-wise workflow: instance extraction and 3D reconstruction → scene registration → assembly. View synthesis, segmentation, image-to-3D generation and semantic analysis are existing components of this paradigm. Instance extraction only reorders segmentation and semantic analysis, adding no sequential stage. PSGSR replaces the registration module and requires no further MLLM calls given upstream priors. GRAM handles articulated instances separately from proxy registration; registered poses transfer directly to articulated assets. The two segmentation levels serve instance discovery and within-object link recovery. Thus, articulation is supported without adding another scene-level reconstruction stage.

Registration efficiency (Appendix F, Table 16 of the revised manuscript). Top-view guidance narrows 3D yaw search through inexpensive 2D rotations. Front-only SASPS retains candidate generation/selection but renders 3D candidates in the input view. On the same 150 scenes, one RTX 4090 yields the following (s/scene, mean ± SD; queueing excluded):

RegistrationCandidate renderingLPIPS↓DINOv2↑CLIP↑Time↓
Full PSGSR—0.23980.89910.9265141.4 ± 14.0
Front-only SASPSBatched0.23310.89460.9268401.9 ± 72.1
Front-only SASPSSequential0.23330.89410.9270574.8 ± 114.8

Visual consistency is comparable; batched/sequential Front-only SASPS take 2.84×/4.07× as long. Top-view guidance limits 3D rendering to residual symmetric candidates.

Appendix F of the revised manuscript also reports end-to-end batch-processing costs: from input to simulation, 150 scenes take approximately 14 h 26 min on 8 × RTX 4090: 5.77 min/scene amortized wall-clock time and 10.39 scenes/hour. These are batch throughput measurements.

Instance extraction (Appendix D.1, Table 7 and main-paper Table 1). We change “MLLM-based analysis → segmentation → 3D generation” to “anonymous instance segmentation → MLLM-based analysis → 3D generation,” reducing semantic omissions without adding sequential stages. The system comparison uses the same 150 scenes and 1,338 reference instances, including tables:

MethodRetained↑Missing↓Retention↑Accurate scenes↑Scene accuracy↑LPIPS↓DINOv2↑CLIP↑ColO (%)↓ColS (%)↓Alignment (s/scene)↓
TabletopGen1,17216687.59%5939.33%0.39350.77880.89081.448816.001105.29
ArticuTable1,337199.93%14898.67%0.23980.89910.92650.381910.67141.40

TabletopGen timing excludes camera search. These results show more complete content and better visual/collision metrics with lower alignment time.

AuthorsOpenReview ↗

Response to Reviewer KJTS — W2

Thank you for suggesting a systematic analysis of error propagation and pipeline robustness. The revised manuscript now reports stage-level quality and error propagation across all 150 scenes (Appendix D.8, Table 13), stage-wise and overall GRAM evaluations (Appendix D.3, Table 10), and PSGSR results grouped by initialization quality (Appendix D.8, Table 14).

(1) Overall pipeline (Appendix D.8, Table 13 of the revised manuscript). Across 150 scenes containing 1,338 reference instances (including tables), we evaluate stage errors and their effects on downstream outputs. Stage error rates use all scenes or instances as denominators; robustness to preceding-stage errors is defined as the proportion of scenes whose current module outputs remain unaffected among scenes with an issue in at least one preceding stage.

Stage / moduleStage errors ↓Stage error rate ↓Scenes affected by preceding-stage errors / scenes with preceding-stage issues ↓Robustness to preceding-stage errors ↑
View generation10 / 150 scenes6.67%——
Instance segmentation12 / 1,338 instances0.90%0 / 10100.00%
3D geometry generation12 / 1,338 instances0.90%2 / 2090.00%
GRAM——1 / 3196.77%
PSGSR——2 / 3193.55%

Scenes with preceding-stage issues are deduplicated by scene. For instance segmentation, these are scenes with view-generation issues; for 3D geometry generation, scenes with view-generation or instance-segmentation issues; and for GRAM and PSGSR, scenes with an issue in any of these three upstream stages. “—” indicates not applicable or not reported in this table.

Evaluation at the instance level further shows that 83.33% (10/12) of instances with segmentation issues and 91.67% (11/12) of instances with geometry-generation issues do not affect subsequent outputs.

Segmentation and geometry generation have low stage error rates, and most issues in this evaluation do not lead to downstream quality degradation. Even when evaluation is restricted to scenes with preceding-stage issues, most outputs of subsequent modules remain unaffected. This indicates that local upstream errors typically do not accumulate through the pipeline, supporting robustness to errors encountered under actual generation conditions.

(2) GRAM (Appendix D.3, Table 10 of the revised manuscript). We evaluate the three modeling steps and the complete chain. Assets receive their worst joint rating; clear missing articulations count as failures.

GRAM step / evaluation levelTotalPartialFailTotal errorsError rate ↓Pass rate ↑Non-failure rate ↑
Segmentation and kinematic-structure inference527279366.83%93.17%98.29%
MLLM-guided joint fitting52798173.23%96.77%98.48%
Semantic state reasoning527243275.12%94.88%99.43%
Overall joints52757167313.85%86.15%96.96%
Overall assets41855177217.22%82.78%95.93%

Errors include partial and fail results; pass rate is one minus error rate, and non-failure includes pass and partial. Each asset receives the worst rating among its joints, with clear missing articulations counted as asset-level failures.

Low error rates across stages indicate that structure, joint-parameter, and motion-state inference retain high output quality despite generated-mesh defects. Most assets fully pass structural and motion evaluations even when graded by their worst joint, indicating no widespread degradation across the modeling chain. Together, the stage-wise and overall results support the chain's stability and robustness under actual generation conditions.

(3) PSGSR (Appendix D.8, Table 14 of the revised manuscript). We split all scenes into three equal groups by point-cloud-only LPIPS and compare input-view consistency before and after full registration.

Initial coarse-registration qualityScenesLPIPS ↓: coarse → fullDINOv2 ↑: coarse → fullCLIP ↑: coarse → full
Better500.2730 → 0.18950.8264 → 0.90480.9061 → 0.9251
Middle500.3415 → 0.23350.8142 → 0.90510.9105 → 0.9283
Poorer500.4174 → 0.29640.7902 → 0.88750.9064 → 0.9262

All groups improve across the three metrics, and the poorest-initialization group has the largest absolute LPIPS reduction. This supports PSGSR's ability to improve input-view consistency across initialization qualities.

AuthorsOpenReview ↗

Response to Reviewer KJTS — W3

Thank you for suggesting quantitative comparisons on actual generated meshes. The revised manuscript now includes a paired quantitative comparison of GRAM and Particulate on 399 actual generated assets in §4.3, Table 3. Both methods use matching input meshes; paired-subset construction and the shared expert-rating protocol are detailed in Appendix D.3.

Construction of the paired subset. The full GRAM audit covers 425 candidate assets. Available Particulate evaluation records cover 407 assets. Excluding eight assets that could not be validly assessed in the paired comparison leaves 399 judgeable asset pairs with matching inputs. Both methods are evaluated on this same subset. The remaining 18 candidates without paired results are excluded from this table rather than automatically counted as Particulate algorithm failures. The table therefore reports output quality on this paired subset.

MethodPass ↑PartialFail ↓
Particulate156 (39.10%)59 (14.79%)184 (46.12%)
GRAM (Ours)334 (83.71%)51 (12.78%)14 (3.51%)

On this paired subset, GRAM achieves an 83.71% pass rate, 44.61 percentage points higher than Particulate, with a failure rate of 3.51%.

Rating and independent re-evaluation. A common primary expert rates both methods using the same structure and motion criteria. Method identities are hidden and outputs are presented in random order; each multi-joint asset receives its worst joint grade. A second expert independently re-evaluates random samples of approximately 20% of the outputs: 85 GRAM candidate assets and 81 Particulate assets. The re-evaluation yields 83.7% pooled asset-level agreement and Cohen’s κ = 0.703.

Articulate AnyMesh and URDF-Anything+ also trail GRAM in joint estimation and articulated-geometry recovery on the standard Lightwheel benchmark (Table 2): their AE values are 21.70° and 49.90°, and their Art. gIoU scores are 0.158 and −0.110, respectively, compared with 17.26° and 0.283 for GRAM.

We also tested Articulate AnyMesh and URDF-Anything+ on the same 19 judgeable generated assets. Under the same structure and motion evaluation criteria, both methods receive 0 pass, 0 partial, and 19 fail ratings. Considering processing costs and texture-adaptation requirements, we selected Particulate, which performs comparably to GRAM on the standard Lightwheel benchmark, for the larger-scale quantitative comparison.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Follow-up on Our Responses

Dear Reviewers,

Thank you again for your time and constructive feedback. We have provided point-by-point responses, including additional generated-mesh comparisons, robustness analyses, and physical interaction evaluations.

As the discussion period approaches its end, we would appreciate your feedback on whether our responses address your main concerns. Please let us know if any questions remain; we would be happy to provide further clarification.

Thank you for your consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.