- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents a framework for generating interactive articulated tabletop scenes with rigid articulated objects and introduces a collection of tabletop scenes as a new benchmark. The paper introduces two new modules. GRAM for MLLM-guided joint fitting and URDF generation, and PSGSR for semantic pose reasoning and recovery of scene layout. The results of ArticuTable are compared against relevant methods on both lightweight articulation dataset and scene-level datasets, across mask-based, motion-based, and visual-based metrics, including a user study.
Critical strengths
- The paper introduces a well-performing framework for the task of single-image interactive scene generation. Though limited to tabletops, the results show promise and the task has application in downstream robot learning tasks.
Critical weaknesses
- I'm concerned about the complexity of the framework that requires two layers of segmentation and MLLM-guided joint estimation. Each consequent step seems to rely heavily on the previous step to be highly accurate to follow. I would love to see some step-by-step robustness study that will validate the chain of operations and show how noisy are the results these intermediate steps produce.
- The paper makes repeated claims on downstream robotic tasks but does not show how well the generated scenes work in simulation.
Questions for authors
- How reliable is the instance segmentation step? How does noise at this step influence the two modules?
- How reliable is the part segmentation step, especially since part segmentation of a monolithic object mesh is a challenging problem. How does noise at this step influence the joint estimation?
- Is the joint estimation from MLLM reliably accurate?
- Is it possible to include movable part and static part Chamfer Distance instead of just the whole object Chamber Distance?
6 comments
Response to Reviewer C71v — W2
Thank you for requesting evidence of how the generated scenes work in simulation. The revised manuscript now reports physical interaction evaluation across all 150 scenes in Appendix G.4, with representative dynamic examples in §4.6, Fig. 6.
Scene-level interaction evaluation across the evaluation set (Appendix G.4 of the revised manuscript). We drag every object and operate every joint of each articulated object from the lower to the upper endpoint of its specified motion range, evaluating link detachment, nonphysical interpenetration, and contact responses with surrounding objects throughout the motion.
The physical interaction evaluation found no obvious nonphysical interpenetration or implausible inter-object contact responses; link detachment occurred in 13 articulated assets, corresponding to 3.11% of the 418 judgeable articulated assets (13/418). Detachment is counted at the asset level and reported separately from inter-object contact responses. This evaluation of each object and joint complements the initial-configuration collision evaluation and covers whole-object movement and part-level interaction within the specified joint ranges.
We allow plausible contact during articulation: when nearby objects lie in a moving part's path, contact-induced blocking or displacement is an expected physical outcome. The relevant failures are nonphysical interpenetration or implausible contact responses.
Dynamic examples in Isaac Sim. Section 4.6, Fig. 6 and Appendix G.3 of the revised manuscript demonstrate gravity and support contact, toolbox dragging, and articulated opening after directly loading the exported USDZ scenes. Physical contact transports the contained object with the toolbox; after the toolbox is displaced, its lid can still open through the recovered joint while the object remains inside. These examples demonstrate that rigid-body motion, collision geometry preserving the interior, and part-level articulation work together in the assembled scene.
These evaluations cover the executability of physical interactions in the generated scenes. Robotic manipulation and learning are potential applications; robot-policy performance and training benefits remain unevaluated.
Response to Reviewer C71v — Q1
Thank you for asking about instance-segmentation reliability and its downstream effects. Appendix D.8 of the revised manuscript now reports segmentation reliability and downstream effects. Among 1,338 reference instances (including tables), instance segmentation has 12 instances with issues, corresponding to a stage error rate of 0.90%; their downstream effects are summarized below:
| Failure mode | Instances with issues | Instances with no downstream effects | Downstream effects |
|---|---|---|---|
| Over-segmentation | 8 | 7 | One instance causes an extra object to be generated |
| Missed segmentation | 4 | 3 | One instance leaves an object missing from the final scene |
| Total | 12 | 10 | Two instances affect downstream outputs |
Most extra segments are removed by subsequent processing, while evidence from the other view compensates for most omissions in a single view. Consequently, among instances with segmentation issues, 83.33% do not affect subsequent outputs, demonstrating tolerance to these segmentation errors during instance recovery.
The omitted and additionally generated objects in this evaluation are both rigid, so they do not affect GRAM articulation modeling; for PSGSR, they appear as missing or extra objects in the final scene.
Response to Reviewer C71v — W1
Thank you for suggesting a step-wise robustness study and clarification of the pipeline's complexity. The revised manuscript now includes stage-level quality and error propagation in Appendix D.8, module responsibilities and workflow in Appendix D.4, and execution costs in Appendix F. We discuss each below.
Overall pipeline robustness (Appendix D.8, Table 13 of the revised manuscript). Across 150 scenes containing 1,338 reference instances (including tables), we evaluate stage errors and their effects on downstream outputs. Stage error rates use all scenes or instances as denominators; robustness to preceding-stage errors is defined as the proportion of scenes whose current module outputs remain unaffected among scenes with an issue in at least one preceding stage.
Scenes with issues in any preceding stage are counted once.
Evaluation at the instance level further shows that 83.33% (10/12) of instances with segmentation issues and 91.67% (11/12) of instances with geometry-generation issues do not affect subsequent outputs.
Segmentation and geometry generation have low stage error rates, and most issues in this evaluation do not lead to downstream quality degradation. Even when evaluation is restricted to scenes with preceding-stage issues, most outputs of subsequent modules remain unaffected. This indicates that local upstream errors typically do not accumulate through the pipeline, supporting robustness to errors encountered under actual generation conditions.
Workflow complexity and execution cost (Appendices D.4 and F of the revised manuscript). The overall pipeline follows the established instance-wise reconstruction-and-registration paradigm. Building on this pipeline, we refine instance extraction and scene registration and add GRAM articulation modeling, which converts generated monolithic meshes into executable assets supporting part-level interaction.
The two segmentation levels discover scene instances and recover functional links within objects, respectively. The added articulation-modeling module is functionally decoupled from scene registration, so rigid proxy meshes can be registered and assembled independently.
Instance recovery (Appendix D.1, Table 7 and main-paper Table 1). We change “MLLM-based instance analysis → segmentation → 3D generation” to “anonymous instance segmentation → MLLM-based analysis → 3D generation,” reducing semantic omissions without adding sequential stages. We compare the same 150 scenes with 1,338 reference instances, including tables:
TabletopGen timing excludes camera search.
Higher instance retention and scene accuracy indicate more complete content; improved visual metrics and fewer intersections indicate better layouts, with lower alignment time.
Appendix F of the revised manuscript now reports batch-processing costs for the complete pipeline, from input images through simulation completion:
Amortized time is total batch wall-clock time divided by the number of scenes and describes batch-processing efficiency under this multi-GPU parallel setting.
Appendix D.3, Table 10 reports part-segmentation and complete-chain GRAM evaluations; main-paper Table 4 and Appendix D.5, Table 11 report joint-estimation ablations. We also discuss these results under Q2–Q3 below.