- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes RoomTokens, a sliced spatial representation containing semantic labels, instance-color labels, and body/front orientation cues. MDIRT jointly denoises these fields using masked discrete diffusion with a shared UNet, cross-slice attention, and fixed architectural tokens. Generated components are converted into object records using continuous attribute heads and retrieved 3D assets. Experiments on 3D-FRONT compare against object-vector and semantic-map diffusion methods, including an architecture-clamped SemLayoutDiff variant. The reported results show improved FID, relation statistics, collision rate, and opening clearance, with zero reported out-of-bounds rate. Component ablations and a 12-participant ranking study provide additional support.
Critical strengths
-
A well-motivated representation. Adding instance separation, facing cues, and height slices directly addresses concrete information lost by a single semantic occupancy map. The representation and shared denoising architecture form a coherent approach to combining spatial constraints with object-level recoverability (Sections 3.1-3.4).
-
Encouraging results against relevant baselines. Relative to SemLayoutDiff, Table 1 reports FID improving from 71.06 to 49.84, COL from 16.13 to 12.75, and opening clearance from 56.15 to 82.04. Including an architecture-clamped baseline is useful for assessing whether gains extend beyond simply fixing the floor plan. Table 2 ablations and Table 3's 79.6% first-place preference also support the practical value of the complete system, although KID and CKL do not improve over the original SemLayoutDiff.
Critical weaknesses
-
The contribution of the multi-slice representation is not sufficiently isolated. Multiple height slices are a central component of RoomTokens, but the cross-slice ablation in Table 2 removes attention while retaining the multi-slice representation. This supports the usefulness of cross-slice attention within the proposed architecture, but does not directly establish the benefit of using multiple height slices. The comparison with SemLayoutDiff also changes the token fields and diffusion process. A single top-down baseline retaining the semantic, instance-color, and facing fields, with comparable model capacity and training, would more directly assess the contribution of vertical information. Without this comparison, the main-text evidence leaves an important part of the representation's claimed advantage insufficiently isolated.
-
The explanation of final boundary compliance is incomplete. Architectural clamping constrains the generated grid, whereas Section 3.4 subsequently scales and rotates retrieved assets, which can change their occupied footprint. Section 4.2 acknowledges that these assembly operations can cause boundary violations for the clamped SemLayoutDiff baseline, yet attributes zero OOB across the MDIRT ablations to retaining architectural clamping. The reported results support boundary compliance on the evaluated scenes, but the main text does not sufficiently explain how the assembly stage preserves it or isolate the contribution of clamping from the rest of the pipeline. A controlled comparison holding scene assembly fixed would strengthen
Questions for authors
-
What evidence shows that three instance colors are sufficient to reliably distinguish adjacent same-category objects across height slices?
-
What mechanism keeps the final 3D assets within room boundaries after the scaling and rotation described in Section 3.4?
-
How much does the multi-slice representation improve performance over a single top-down map with the same semantic, instance-color, and facing fields?
-
What explains the worse KID and CKL scores than SemLayoutDiff in Table 1 despite the improvement in FID?
4 comments
We sincerely hope response from the reviewer before the end of the author-reviewer discussion period.
Dear Reviewer TWqd,
Thank you for the careful evaluation of our paper. We want to know whether our previous response has successfully addressed any of your initial concerns. If not, and you have any further questions, please feel free to share your remaining concerns with us so that we can use our last chance to address them. We want to thank you again for your valuable time and efforts. We will be more than happy to receive your response and resolve any further problems.
Thank you for the detailed responses and additional experiments. They have addressed some of my concerns. However, I am still unclear about how rotation, scaling, and placement interact, and how these factors affect the different evaluation metrics. These questions leave me unconvinced that my main concerns have been fully addressed, so I will keep my score unchanged.
Response to Reviewer TWqd
Thank you for clarifying your concern. Below, we provide additional details on how the generated RoomTokens are used to obtain the rotation, scaling, and placement for each object, and how they impact the evaluation metrics.
1. From RoomTokens to the final scene
We group connected RoomTokens with the same semantic and instance-color labels across height slices into object instances. This gives us a set of object instances, each with a semantic label (providing the object category) and covering a set of tokens. The spatial extent of the tokens, together with height-slice information and facing cues, allows us to determine the position, dimensions, and orientation for each object. For each generated object instance (we use a set of prediction heads to estimate the yaw and object dimensions from the pooled features), we then retrieve an asset by category and size, scale and rotate it using these attributes, and place it at the recovered position. See Appendix F (Token-to-object assembly) for details.
Scene assembly steps follow prior work, including DiffuScene and SemLayoutDiff. We use SemLayoutDiff’s asset database and rendering procedure, without subsequent optimization or scene correction.
2. Effects on the evaluation metrics
Scaling sets the dimensions of a retrieved asset. Rotation changes which part of the room it occupies, even when its position stays the same. For example, turning a sofa by 90° can make its bounding box extend past a wall (OOB). The sofa can also overlap a coffee table (COL), block a passage (NAV), or obstruct a door (OC). Rotation and scaling can also change the spatial relationships between objects (Pair-Rel) and how the scene looks in the renderings used for FID and KID.
To test these effects directly, we kept the generated scenes, retrieved assets, and object positions fixed. We set all objects to zero yaw or kept each asset at its retrieved size. Table 10 is reproduced below. We added Figure 6 to Appendix F to illustrate how rotation and scaling affect collisions and navigability. Each panel shows side-by-side renderings of the same scene, comparing MDiRT placement with fixed yaw in (a) and with no scaling in (b).
| Asset placement | FID ↓ | KID × 10³ ↓ | CKL × 10² ↓ | Pair-Rel ↓ | #Obj. | OOB_S ↓ | OOB_O ↓ | COL ↓ | NAV ↑ | OC ↑ |
|---|---|---|---|---|---|---|---|---|---|---|
| MDiRT placement | 49.84 | 10.41 | 6.43 | 12.13 | 8.36 | 0.00 | 0.00 | 12.75 | 99.30 | 82.04 |
| Fixed yaw | 65.56 | 17.45 | 6.43 | 18.09 | 8.36 | 30.90 | 20.64 | 35.82 | 90.92 | 79.42 |
| No scaling | 58.20 | 14.43 | 6.43 | 12.34 | 8.36 | 10.50 | 5.38 | 19.85 | 97.02 | 80.17 |
Rotation. With MDiRT placement, all object boxes stay inside the room. With fixed yaw, 30.90% of scenes contain an out-of-bounds object. COL rises from 12.75% to 35.82%, and NAV falls from 99.30% to 90.92% (Table 10). In Figure 6(a), the rotated sofa and TV stand collide with the coffee table and cut off the purple region from the rest of the walkable floor.
Scaling. Keeping assets at their retrieved size raises scene OOB to 10.50% and COL to 19.85%. NAV falls to 97.02%, a smaller drop than with fixed yaw (Table 10). In Figure 6(b), the taller TV stand overlaps the lower one. This illustrates how scaling can affect collisions even when the walkable floor in that scene stays connected.
CKL and object count stay the same because all three placements use the same objects.
The generative process for RoomTokens outputs facing cues directly. In contrast, SemLayoutDiff uses a two-stage process that first generates a semantic map (without instance information) and then predicts object instances and estimates the orientation from the generated semantic map. This joint reasoning over semantics, instances, and facing cues helps our model learn consistent occupancy and orientation before asset placement. Our experiments show that removing facing cues increases the collision rate considerably – COL from 12.75 to 89.07 (Table 2), supporting the importance of including facing cues.
Our experiments show that our model can generate more consistent object attributes and more plausible assembled scenes.
Thank you for the careful review. We appreciate your recognition that the RoomTokens representation and MDiRT's shared denoiser jointly capture instance separation, orientation, and height. We also appreciate your positive assessment of the improvements in FID, collision, and opening clearance, and of the 79.6% first-place preference in the user study.
Q1: What evidence shows that three instance colors are sufficient to reliably distinguish adjacent same-category objects across height slices?
Answer: Our instance-separation analysis in Appendix C reports 0.92% color conflicts among adjacent same-category, different-instance token pairs and correct instance counts in 91.53% of scenes. The coloring procedure considers horizontal adjacency in every height slice and assigns one consistent color to each instance across slices (Appendix E). These statistics support reliable instance separation across the processed scenes.
To further assess this choice, we have added a three/four-color comparison to Section 4.2, Table 3, which supports the compact palette. See common response C3 for the full results. We have also clarified in Appendix E that the four-color theorem applies to connected regions in a single slice under edge adjacency.
W2 & Q2: What mechanism keeps the final 3D assets within room boundaries after the scaling and rotation described in Section 3.4?
Answer: MDiRT generates RoomTokens with architectural tokens fixed during denoising. During scene assembly (Section 3.4 and Appendix F), we convert the generated RoomTokens into object records and retrieve assets by category and size. The assets are scaled to the estimated extents, rotated to the predicted yaw, and placed at the recovered positions. No boundary correction is applied during assembly; zero OOB is measured after these operations. We have expanded the description in Sections 3.4 and 4.1 to clarify that evaluation follows asset placement without subsequent correction.
Thank you for suggesting further experiments to explain boundary compliance after asset placement. We have added diffusion and clamping comparisons to Section 4.2, Table 3, keeping RoomTokens and scene assembly unchanged. Without clamping, masked diffusion reduces scene OOB from 34.60% to 1.17% compared with multinomial diffusion. Architectural clamping further reduces MDiRT's scene OOB to 0.00% after asset placement. With clamping enabled, masked diffusion also has lower COL (12.75 versus 32.16). These results support both design choices for boundary compliance and physical plausibility (see common response C1 for the full results). We also provide details for the shared assembly and evaluation procedure in common response C2.
W1 & Q3: How much does the multi-slice representation improve performance over a single top-down map with the same semantic, instance-color, and facing fields?
Answer: Thank you for suggesting this comparison. We added a single top-down ablation to Section 4.2, Table 3, projecting the full scene onto one map while retaining the semantic, instance-color, and facing fields, three instance colors, conditioning, backbone, and training settings. MDiRT with multi-slice RoomTokens improves OC from 74.67 to 82.04 and reduces COL from 13.75 to 12.75 while generating more objects. These results directly support retaining height information. See common response C1 for results for all metrics and the other controlled comparisons.
Q4: What explains the worse KID and CKL scores than SemLayoutDiff in Table 1 despite the improvement in FID?
Answer: FID and KID compare feature distributions of rendered scenes using different statistics (Appendix A), so an improvement in FID does not imply an improvement in KID. CKL measures object-category frequencies and is unchanged when the same objects are rearranged. Improved spatial arrangements therefore need not yield lower CKL. We have added this discussion to Section 4.2, emphasizing that we follow the evaluation protocol used by prior work.
Since the distributional metrics do not necessarily reflect the quality of the generated scenes, we also include spatial plausibility metrics and qualitative comparisons, as well as a user study in our evaluation. These metrics indicate that scenes generated by MDiRT are more plausible. For instance, in Table 1 (which we updated with results across multiple sampling seeds), MDiRT reduces COL from 16.24 to 12.64 and increases OC from 56.02 to 82.31 compared with SemLayoutDiff. Most notably, in our user-study, 79.6% ranked scenes generated by our method first.