- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces InsideArt, a dataset and benchmark for generative models of 3D articulated objects, and focuses on the usability of the generated objects’ interior spaces. The paper utilizes articulated assets from PartNet-Mobility and Artiverse, using large VLMs to generate structured descriptions for each object, with particular emphasis on potential placement positions inside these articulated objects. The annotation pipeline has two data streams: one for describing the original object and the other for preparing candidate items to be placed.
The final dataset provides 654 articulated containers with 12,441 placeable items and more than 100,000 placements. The benchmark evaluates generative models in terms of three aspects, i.e., reconstruction quality, free-space matching and access-topology correctness, and finally assesses item placement.
Critical strengths
- The motivation for considering the usability of generated objects’ interior spaces in 3D generation is solid, especially in articulated object generation, as these objects are intended to be operated and interacted with. The paper builds on this central motivation, providing an effective annotation pipeline and a dedicated benchmark for 3D generative models.
- The benchmark assesses item placement not merely by reconstruction metrics (i.e., target-region-, position-, and collision-based metrics), but also by physical stability in a simulated environment, which makes the benchmark more realistic and useful for downstream applications.
Critical weaknesses
-
The contribution of the data processing and annotation may be limited. As illustrated in figure 5, most of the annotations can be accessed directly from the original datasets via rule-based methods. Two parts of the pipeline require VLM inference, but the tasks are also relatively simple, involving descriptions of the target object or placements. The placeable items are also acquired from open datasets. Although human verification and edits make the annotated data trustworthy and ready to use, the relatively straightforward annotation tasks and pipeline leave me uncertain about this work’s contribution beyond existing datasets such as PartNet Mobility and Artiverse.
-
Following up on W1, the limited pool of articulated containers makes the benchmark’s coverage and reliability for downstream applications uncertain. The containers shown in Figure 3 are mostly storage units, tables, and refrigerators, yet the distribution across different container types seems uneven. Section 3.2 reports that 444 of the 654 containers are storage furniture. Although the dataset also includes kitchen appliances and Figure 8 shows oven and refrigerator examples, I remain uncertain whether its category distribution and structural diversity sufficiently support the claimed downstream applicability.
In practice, robotic real-to-sim modeling emphasizes accurate and robust reconstruction of a wide range of interactable real-world objects, such as those in RoboCasa: ovens of various shapes and sizes, toasters with realistic interior spaces, and coffee machines that can accommodate cups. However, in the examples presented by the authors, I do not see evaluations involving a more diverse range of articulated containers, which makes me question the benchmark's reliability in specific downstream applications.
Questions for authors
- During stage 2 of the annotation pipeline, why is a VLM required to provide candidate support points? It seems these positions can be identified directly from the original models through collision detection (using tools like CoACD and FCL). What are the advantages of using a VLM to propose candidates over directly searching for horizontal surfaces, given that both approaches require manual verification?
- Is the example shown in the bottom-right panel of Figure 1 drawn from the actual annotated data? It illustrates various placement configurations, such as books placed flat or upright and nested placements involving a pen holder and the pens inside it. Are these configurations included in the dataset proposed in the paper?
- As mentioned previously, could the authors provide more varied examples to demonstrate the benchmark’s ability to evaluate a wider range of container geometries and placement configurations beyond cabinet scenarios? These could include evaluating the quality of generated interior spaces in appliances such as refrigerators, ovens, dishwashers, toasters, and coffee machines; assessing different placement orientations for objects such as books and plates (e.g., placed flat or on their sides).
- How does the benchmark address data balance and coverage across all the assets used? For example, each “container category–object–placement configuration” combination may require a minimum number of samples and sufficient diversity to ensure reliable evaluation. Could the authors provide statistics on the sample counts and proportions for these combinations, or at least explain how the benchmark handles imbalances in sample counts across container categories?
- Given that most articulated object generation methods cannot generate contained items themselves, is the applicability of the evaluation in Section 5.3 limited? For articulated object generation, even in robotics-related applications, models are not necessarily required to generate a container, an item, and the corresponding placement all at once. Could the item placement evaluation therefore be extended to a broader range of test scenarios to assess the quality of generated containers, rather than requiring models to output placements directly?
6 comments
The response to Reviewer ySEf (Part 2)
W2 Articulated container coverage and reliability for downstream applications
We thank the reviewer for highlighting the relationship between category coverage and downstream applicability. InsideArt is designed primarily to evaluate text-and-image-conditioned 3D generation of containers with internal partitions, rather than to provide uniformly balanced coverage across all object categories. Therefore, its category distribution is determined by the relevance and complexity of internal structures instead of equal sampling. Storage furniture, such as cabinets, drawers, and shelving units, naturally exhibits greater variation in compartments and hierarchical layouts, making it especially suitable for evaluating internal spatial reasoning and conditional generation. By contrast, many other container categories have relatively simple or homogeneous interiors. Enforcing strict category balance would consequently introduce more structurally uninformative examples and dilute the benchmark’s diagnostic value. Thus, the imbalance is an intentional, task-driven design choice rather than an oversight.
Following the reviewer's suggestion, we have expanded the dataset with 301 additional containers with enclosed internal cavity from appliances (refrigerators, ovens, washers/dryers, freezers, dishwashers, toaster ovens, and others), tables (desks and coffee tables), and boxes or cases (safes, trash cans, tackle boxes, instrument cases, mailboxes, and others) to better support downstream robotic tasks. The resulting distribution is:
| Supercategory | Containers | Proportion | Fine-grained categories |
|---|---|---|---|
| Storage furniture | 444 | 46.5% | storage cabinet; sideboard; nightstand; chest of drawers; wardrobe; TV cabinet; storage bench; storage chest; sink cabinet |
| Tables / desks | 215 | 22.5% | desk ; table with drawers or cabinet doors; coffee table |
| Appliances | 215 | 22.5% | Refrigerator; oven; microwave; washer/dryer; freezer; toaster oven; dishwasher; air fryer; bread maker ; grill press; rice cooker |
| Boxes / cases | 81 | 8.5% | box; safe; trash can; tackle box; instrument case; mailbox; storage case; toolbox; sewing box |
| Total | 955 | 100% | 32 fine-grained categories |
Moreover, to our knowledge, InsideArt is the first benchmark to explicitly model the correspondence between internal regions and external articulated parts. For evaluating the generation and usability of container interiors, even though the difference of shape and materials is significant, the diversity of internal structures and access topologies is our target: how the interior is partitioned, which movable parts are associated with each region, and which sequence of operations is required to access it. Containers in the same semantic category can still differ substantially, including independent drawers, one compartment shared by two doors, and nested configurations in which a door must be opened before a drawer can be pulled out. These differences directly determine the difficulty of internal-structure generation and access-relation reasoning.
We quantify this dimension for both InsideArt and RoboCasa. We represent each container as a labeled access graph: its nodes are internal regions and access parts, while its edges specify which part must be opened to reach each region. Opening directions and required operation orders are also recorded. Two containers count as the same structural type only when their access graphs are isomorphic. We apply the same construction to RoboCasa: the five shared enclosed-container categories below contain 200 RoboCasa fixtures.
| Category | InsideArt containers | InsideArt structural types | RoboCasa containers | RoboCasa structural types |
|---|---|---|---|---|
| Refrigerator | 94 | 62 | 50 | 21 |
| Oven | 65 | 54 | 25 | 3 |
| Microwave | 18 | 7 | 50 | 1 |
| Toaster oven | 5 | 5 | 50 | 13 |
| Dishwasher | 5 | 5 | 25 | 4 |
| Storage furniture / tables / boxes and remaining appliances | 768 | 276 | — | — |
| All | 955 | 394 | 375 | 43 |
Across the five categories represented in both datasets, InsideArt obtains approximately three times as many structural types with fewer containers: 133 types from 187 containers, compared with 42 types from 200 RoboCasa fixtures. RoboCasa containers are often appearance variants of a shared template: its 50 microwaves contain only one structural type, and its 25 ovens contain three. Only 5 of RoboCasa's 43 structural types also occur in InsideArt; the remaining types mainly arise from categories such as stoves and sinks, which are not containers intended to hold placed objects. These results show that, relative to RoboCasa, InsideArt provides substantially greater diversity in internal structures and articulated access topologies. Thus, InsideArt includes wider range of interactable real-world objects.
The response to Reviewer ySEf (Part 3)
Q1 Horizontal surfaces with geometry method (CoACD or FCL)
The most important limitation is that geometry provides neither semantic names nor access structures. We therefore use a VLM to obtain semantic information for every articulated part and free space and to express their positional relationships in language, allowing a one-to-one correspondence between the textual information and the physical 3D structure. This information is required by the subsequent placement stage.
To compare the geometric and VLM method, we further add a purely geometric baseline and compare it with the original VLM method. A surface is considered found when its height differs by at most cm and its top-view IoU is at least . We evaluate three approaches:
- voxel support-surface extractor for generated assets, applied directly to the ground-truth URDFs
- per-part extraction of near-horizontal mesh faces with overhead clearance in the open pose
- a CoACD plus ray-casting pilot.
| Geometric method | Precision | Recall | F1 |
|---|---|---|---|
| Voxel support-surface extractor | 0.794 | 0.928 | 0.855 |
| Open-pose per-part horizontal-face extraction | 0.629 | 0.860 | 0.726 |
| CoACD + ray casting | 0.714 | 0.116 | 0.200 |
If the container is not partitioned into parts, for example when an entire drawer is treated as a single mesh, CoACD's convex decomposition fills enclosed cavities and finds almost no support surfaces, with a recall of . Conversely, if the mesh is partitioned, a complete support surface may be split across multiple subparts. Recovering the original support region then requires correctly identifying and merging these local surface patches, which is difficult.
The best geometric method finds most support surfaces, with a recall of , but its precision is only . This is comparable to the % human intervention rate for VLM-proposed geometry, indicating that VLM and geometric methods require similar amounts of subsequent manual annotation at the support-surface proposal stage. The main problem is that geometry has difficulty distinguishing a nearly horizontal face from a genuinely usable support surface: it mistakes gaps between layers, rails, and handle tops for supports, while missing valid storage regions with absent floors, uneven surfaces, or open tops.
Q2 Is the bottom-right example in Figure 1, including flat and upright books and pens inside a pen holder, drawn from actual annotated data?
The bottom-right example in Figure 1 comes from real annotated data. It is a formal rendering of an annotated scene, with materials recolored consistently by object type for visualization. Flat and upright books, as well as pen-holder-like nested arrangements, are all placements actually generated in the dataset.
Alternative placement poses are annotated as item properties during item annotation, Stage 4 of Figure 5. We group items according to their three axis lengths:
- Items with three similar axis lengths, such as apples, grapes, and potatoes, retain one pose.
- Items with two similar axes and one distinct axis, such as pens, plates, and bottles, retain one or two semantically plausible and gravity-stable poses. Pens and plates are placed flat, while bottles may be horizontal or upright.
- Items with three distinct axis lengths, such as books, retain one to three plausible poses.
During layout generation, we sample from the selected item's candidate poses.
We show object stacking and nested placements, such as items supported by vessels, in our response to Q3.
Q3 Can the paper show more container types and placement orientations?
We have added 18 annotated placement examples to Appendix L Fig. 16, organized around three aspects. First, they cover more container categories, including refrigerators, ovens, microwaves, dishwashers, toaster ovens, washing machines, safes, and large and small boxes. Second, they show diverse item orientations, such as plates standing on edge in a dish rack, books standing upright in a row, and books lying flat in a drawer. Third, they include stacked, nested, and densely packed arrangements, such as upside-down bowls placed between dishwasher-rack tines, piles of flat items, food supported by plates, flat book stacks, densely packed drawers, and pens inserted into a pen holder. These are generated and annotated dataset scenes rather than schematic illustrations.
The response to Reviewer ySEf (Part 1)
We thank the reviewer for the careful and constructive evaluation of InsideArt. Below, we clarify the substantive contributions of our annotations and benchmark, discuss container coverage and data balance, compare VLM-based support-surface proposals with geometric alternatives, verify the provenance and diversity of the illustrated placements, and explain how the benchmark evaluates generated containers.
W1 The contribution of data processing and annotation
Our annotation effort is substantial, involving extensive manual curation, correction, and multi-stage verification throughout the dataset construction process. The source datasets do not specify which surfaces can support objects, the usable volume above each surface, the parts and order of operations required for access, or the poses and metric scales at which items can be placed. Nor can this information be obtained directly through rules. InsideArt contributes three types of annotation:
1. Containers
2. Items
3. Placements
Rules and VLMs provide only initial proposals in our pipeline. The final structure is produced through multiple human-verified rounds of proposal, correction, and re-proposal. Every articulated-container sample is cross-checked by at least two annotators, and the complete annotation process requires approximately 300 person-hours.
The correction rate is substantial. Annotators add or remove entries in the support-surface inventories of 27.1% of containers and update the corresponding free-space annotations accordingly. When changes to names, location descriptions, access instructions, and associated parts are also included, 38.4% of containers are modified. VLM-proposed surfaces have the highest deletion rates for ovens (33.3%), tables (26.5%), and refrigerators (23.3%). The initial joint and access graph is generated from the URDF, VLM output, and rules, but annotators still modify 20.8% of containers and 15.1% of surface-state associations.
The source assets themselves are also not directly usable. Our interior-cavity check finds that 18.6% of candidate containers are solid inside, lack a floor, or have walls that are invisible from the interior. Item assets lack canonical orientations, stable poses, and metric scales, and also include duplicates and unsuitable models. We manually discard unsuitable 3D assets, redefine their front directions, correct their scales, repair incomplete meshes, and determine stable poses for all assets through gravity settling. Item-container compatibility rules are likewise not predefined. Through a generate-review-generalize-regenerate loop, annotators identify 174 positional, oriented, and semantic problems in 303 layout variants and convert all of them into generation constraints, such as no food in wardrobes, no cup-on-cup stacking, upright milkshakes, and no mixing of raw and cooked food on the same refrigerator shelf.