acceptodds
Under review as a conference paper at ICLR 2027

Generation of High-Level Concepts in 3D Scene Graphs via Autoregressive Diffusion

Abstract

Indoor 3D Scene Graphs (3DSGs) enhance robot-observed geometric primitives, such as wall planes, with hierarchies of increasingly abstract metric-semantic concepts (walls, rooms, floors, buildings), each represented as a node with a semantic label, a 3D position, and links to the lower-level elements it groups. These hierarchies support robotic perception and Simultaneous Localization and Mapping (SLAM), yet inferring them automatically from the observed primitives remains an open problem: classical methods rely on rules hand-crafted for each concept, while existing learning-based methods use separate models for structure and for geometry, and do not generate beyond walls and rooms. We propose a unified graph generative model based on autoregressive diffusion that grows a complete 3DSG bottom-up from the observed planes: at each level, it jointly samples how many concept nodes to add, their labels and positions, and their connections to the level below, without any concept-specific module. To evaluate generated hierarchies, we adapt the Fused Gromov–Wasserstein distance to match generated and ground-truth nodes on both topology and features. Across synthetic scenes, real architectural floor plans, and robotic sensor data, up to city level, our model outperforms prior learning-based baselines and, on all the datasets but one, improves on a one-shot diffusion model even when the latter is given the ground-truth graph size, with the largest gains on the deepest hierarchies and on robot-recorded environments.

Primary area:
generative models
Keywords:
Graph generation, autoregressive diffusion, scene graphs, robotic perception, multimodal generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rTwsOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to complete hierarchical 3D scene graphs from observed 3D wall planes. In the setting considered here, a robot has already mapped a static environment and extracted wall planes. To build a full scene graph, the model must infer high-level concepts such as walls, rooms, floors, buildings, and cities.

To solve this, the authors propose a generative pipeline combining two existing methods (FLAGG and MiDi). FLAGG provides the autoregressive framework for adding nodes in small blocks and predicting when to stop. Within each step, a MiDi-based module uses a Graph Transformer with mixed categorical and Gaussian diffusion. This module jointly denoises node labels, edge connections, and 3D centroid coordinates. The authors also use noise augmentation on conditioning graphs to deal with exposure bias, as well as Min-SNR loss weighting for 3D coordinates.

For evaluation, the paper introduces a benchmark with synthetic layouts, floor plans from the MSD dataset, and few real-world robot scans. The authors use the Fused Gromov-Wasserstein (FGW) distance to measure differences in both graph connectivity and node attributes. The authors report lower FGW scores and better stability on deeper hierarchies, although the one-shot baseline performs better on M-F.

Critical strengths

I found two main strengths in this paper:

First, I agree that inferring high-level abstractions like rooms and floors directly from low-level geometric primitives is an important task in robotics and spatial understanding. Learning these concepts within a unified generative framework, rather than using a separate heuristic pipeline for each concept, is an appealing and timely direction.

Second, the MiDi-based joint formulation for graph topology and 3D geometry is sensible for this task. Denoising discrete edge connections, semantic labels, and continuous 3D centroid coordinates within the same reverse diffusion process is a natural way to handle these coupled outputs. I also find the choice of equivariance in distribution under horizontal translations and rotations about the vertical axis reasonable for indoor multi-floor environments.

Critical weaknesses

I found two critical weaknesses that make it hard to support acceptance in its current form:

First, the comparison between the autoregressive method and the one-shot baseline (MiDi+Or) seems like it's not a fully controlled experiment. The paper claims this comparison isolates the benefit of autoregressive generation, but Table 5 shows that several key training settings differ at the same time. For example, the loss weights are different ((1,2,4)(1, 2, 4) vs. (0.4,1,5)(0.4, 1, 5)), Min-SNR loss weighting is only listed for the proposed method, and the conditioning modules are implemented differently. Because of these confounding factors, it is difficult to tell whether the performance gains come from the autoregressive scheduling itself or from these additional design choices.

Second, the mathematical definition of the FGW metric in Section 5.2 seems incomplete. The text states that dummy nodes are penalized in both the dWd_W and dGWd_{GW} terms by setting a large shortest-path distance. However, Eq. 5 defines dWd_W using only 3D centroids and one-hot labels, where shortest-path distance does not appear. Without specifying the coordinates, labels, or exact feature penalty assigned to dummy nodes, this metric cannot be mathematically verified (or used). It is also unclear whether the correspondences for the observed input planes are explicitly fixed during optimal transport.

Questions for authors

Could the authors clarify the following points during the rebuttal?

  1. Controlled comparison on the autoregressive schedule:

    Could you compare a single-block baseline against the proposed autoregressive model with matched settings (the same Fill architecture, conditioning encoder, loss weights, and Min-SNR weighting) on a representative multi-level dataset? This would directly show how much the generation schedule itself contributes to the performance gains.

  2. Clarification of the FGW implementation:

    How are dummy nodes handled in the Wasserstein term dWd_W? Since Eq. 5 only depends on 3D centroids and one-hot labels, what coordinates, labels, or feature penalties are assigned to dummy nodes? Also, how are infinite shortest-path distances between disconnected components handled in dGWd_{GW}?

  3. Fixing observed plane correspondences in FGW evaluation:

    When solving for the transport plan γ\gamma, are the correspondences for the observed input planes fixed between the generated graph and the ground truth? I understand that dWd_W already includes coordinates, but does additionally fixing the known plane correspondences change the results or method ranking? Alternatively, could you report a grouping metric that explicitly preserves input-plane identities?

  4. Ablations on the effect of node ordering:

    Section 4.2 and Appendix B.1 describe hierarchical and hierarchical proximity orderings. Do you have results comparing the chosen ordering with hierarchical ordering without the proximity component, random ordering, or purely spatial ordering? This would help show how much the ordering contributes.

I am open to revising my assessment based on the authors' response, especially clarification of the evaluation and evidence from a controlled comparison. The above-mentioned points are currently limiting my confidence in the conclusions.

1 comment

We thank the reviewer for the very thorough review and constructive criticisms, asking for clarifications and useful comparisons. We address them point by point.

CW1, Q1 - Controlled comparison: We agree that our comparison of MiDi+Or and our method does not provide a precise isolation of autoregressive vs. oneshot scheduling. The choice of hyperparameters of MiDi+Or comes from the set used in the original paper. In fact, oneshot MiDi was not built to receive input nodes, so we adapted its original implementation to have a comparison, and added the oracle to account for MiDi's requirement of taking as input the number of nodes. Thus, the two architectures differ. To correctly isolate the effect of autoregressive generation as the reviewer suggests, we implemented a one-shot variant of FLAGG + MiDi, where non-planes nodes are generated as one single block (thus simulating the original MiDi behavior). The autoregressive and one-shot variants share Fill architecture, conditioning encoder, loss weights and Min-SNR weighting, together with other hyperparameters. We are currently running experiments on the M-C dataset, and due to their computational cost we will report results in the coming days.

CW2, Q2 - Dummy nodes in FGW: We agree that the original explanation was incomplete, so we integrate our explanation in the following. We pad the smaller graph with isolated dummy nodes until both graphs have N=max⁡(n,n′)N=\max(n,n') nodes. For W, feature distances between real nodes are normalized to [0,1][0,1]; every real-dummy distance is then explicitly set to 22, after normalization, overriding the feature-distance formula in Eq. 5. Dummy centroids and label vectors are stored as zeros, but due to the overriding, they are effectively ignored. For GW, shortest-path distances between distinct disconnected nodes, including dummy-real pairs, are set to NN, before the normalization in Eq. 6. Thus, W explicitly penalizes graph-size mismatch, while GW compares the resulting finite structural distances. We have clarified these conventions below Eqs. 5-6 and in Appendix J.

CW2, Q3 - Fixing planes identities in FGW: Planes are in fact not fixed when solving the transport plan for FGW, but we agree that this might be a sensible choice. Thus, we compute "FGW fixed" with its GW and W distances, fixing the identity of planes, and solving with this constraint. The following table compares the original values with new values on the challenging M-C dataset, comprising concepts up to city level.

MethodFGW originalFGW fixedGW originalGW fixedW originalW fixed
RAND74.7973 ± 0.370574.8078 ± 0.369164.7427 ± 0.284064.7695 ± 0.279184.8519 ± 0.461284.8460 ± 0.4618
MiDi+Or13.3357 ± 13.636313.3374 ± 13.639020.6991 ± 26.942020.7036 ± 26.94905.9723 ± 0.46855.9712 ± 0.4675
Ours7.2846 ± 0.69347.7464 ± 0.64357.0182 ± 0.38868.2190 ± 0.19037.5510 ± 1.11997.2738 ± 1.1471

It can be seen that the differences between original and fixed are negligible for RAND and MiDi+Or, and small for Ours, where we notice that GW differs by ~1.2. Still, the aggregated FGW remains very close to our original definition, and the ranking remains unchanged on M-C. Thus, we can conclude that, although desirable, this new definition is empirically consistent with the original. We included this new comparison and the table above in Appendix J of the revised manuscript.

Q4 - Ablations on node ordering: We thank the reviewer for this suggestion. We are running the following ablations on the M-C dataset: random ordering and hierarchical ordering without proximity. Because these training runs are computationally expensive, we will report final results in the following days.

Statements relevant to Q1 and Q4 will be adjusted in the revised manuscript after all experiments have concluded and we have clear evidence.

Reviewer_8X1QOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposed a autoregressive, diffusion-based DGG model for 3DSG completion, generating multi-level hierarchies. The authors also introduce an adaptation of FGW to evaluate hierarchical 3DSGG. The method is well evaluated on diverse synthetic and real-world scenarios and the results show that the method has broader hierarchical coverage than prior work.

Critical strengths
  1. The unified generation framework is well motivated. Prior works primarily generate walls and rooms, while handling structural inference and centroid estimation separately. This work models node counts, classes, positions, and connectivity within a unified pipeline and extends generation to floors, buildings, and cities. So it reduces the need for separate concept-specific modules.
  2. The method shows improvements on deeper hierarchies. The ablation experiments provide evidence for the effectiveness of the training strategies. And the anonymous code release supports reproducibility.
Critical weaknesses
  1. Appendix C.1 states that the noise augmentation perturbs previously generated high-level nodes and their connections while protecting the observed planes. The ablations therefore primarily assess robustness to imperfect generation histories. Although evaluation on robot-recorded planes provides some evidence of robustness to real inputs, but sensitivity to specific input errors (such as inaccurate plane positions) remains insufficiently characterized.
  2. The evaluation reports aggregate graph metrics such as FGW, GW, and W. These provide useful overall comparisons, but level-specific metrics would help clarify which concepts are recovered accurately and where errors remain.
Questions for authors
  1. Could authors provide a controlled robustness analysis for errors in the observed planes, such as position or normal perturbations? How do these errors affect generation quality at different hierarchy levels?

  2. Could authors report level-specific accuracy (for floors, buildings, and cities) to complement the graph metrics?

4 comments

We thank the reviewer for recognizing the value of the paper and for the suggestions. We address both points below.

CW2, Q2 - Level-specific metrics: We match generated and ground-truth nodes of the same class when the Intersection over Union (IoU) of their sets of observed input-plane descendants exceeds 0.5. Unmatched generated nodes are false positives (FP), and unmatched ground-truth nodes are false negatives (FN). We report F1, mean IoU over matched pairs, and panoptic quality (PQ), together with centroid error for matched pairs and node-count Mean Absolute Error (MAE). The following results are on M-C, with concepts up to city: mean ± standard deviation over three seeds. Centroid errors are Euclidean distances (Table 11).

LevelF1 MiDi+OrF1 OursIoU MiDi+OrIoU OursPQ MiDi+OrPQ Ours
Wall0.02 ± 0.020.19 ± 0.090.77 ± 0.030.84 ± 0.030.02 ± 0.010.16 ± 0.08
Room0.09 ± 0.110.16 ± 0.100.64 ± 0.030.67 ± 0.030.06 ± 0.080.11 ± 0.07
Floor0.29 ± 0.280.28 ± 0.040.71 ± 0.080.68 ± 0.010.22 ± 0.220.19 ± 0.03
Building0.37 ± 0.300.80 ± 0.060.77 ± 0.080.87 ± 0.010.29 ± 0.250.70 ± 0.06
City0.31 ± 0.310.86 ± 0.070.79 ± 0.110.93 ± 0.020.26 ± 0.290.80 ± 0.06
LevelCentroid error MiDi+OrCentroid error OursCount MAE MiDi+OrCount MAE Ours
Wall1.71 ± 0.620.59 ± 0.174.44 ± 1.367.09 ± 0.92
Room1.35 ± 0.221.00 ± 0.204.35 ± 1.312.46 ± 0.51
Floor1.86 ± 0.461.43 ± 0.200.79 ± 0.260.74 ± 0.10
Building3.97 ± 1.161.28 ± 0.200.57 ± 0.250.29 ± 0.05
City4.14 ± 1.742.85 ± 0.330.63 ± 0.230.03 ± 0.02

Our method improves matching at four levels and centroid error at all levels. Floors remain difficult for both methods, and MiDi+Or has lower wall-count error. Wall and room F1 are low in absolute terms: the strict threshold makes a two-plane wall unmatched if one plane is missed; lowering the threshold to 0.3 raises our wall F1 from 0.19 to 0.48. The clearest gains are on buildings and cities, with 2.77× city F1 and 3.08× city PQ relative to MiDi+Or. We introduce the metrics in Sec. 5.2 and report definitions and results in Appendix K.

CW1, Q1 - Robustness to noise analysis: Unlike training-time noise augmentation, which protects observed planes, this new M-C analysis perturbs the input planes at inference time. We apply three perturbation levels for four noise types: Gaussian centroid shifts, Gaussian normal-tilt angles, length rescaling by 1+σϵ1+\sigma\epsilon with ϵ∼N(0,1)\epsilon\sim\mathcal{N}(0,1), and plane dropout with probability pp.

PerturbationLowMidHigh
Centroid, horizontal σh\sigma_h (cm)51020
Centroid, vertical σv\sigma_v (cm)2.5510
Normal tilt σ\sigma (degrees)124
Length σ\sigma (% of length)51020
Plane dropout pp (%)51020

These settings are also in Table 12. We use one model seed (seed 0), with mean ± standard error over 128 test scenes. FGW values are multiplied by 100 and exclude plane nodes, so the injected input error itself is not scored. They are not directly comparable with the full-graph, three-seed FGW results.

PerturbationCleanLowMidHigh
Centroid12.96 ± 0.7113.10 ± 0.6214.24 ± 0.6914.86 ± 0.75
Normal tilt12.96 ± 0.7113.65 ± 0.6812.71 ± 0.6513.08 ± 0.70
Length12.96 ± 0.7113.33 ± 0.6813.01 ± 0.6713.28 ± 0.68
Plane dropout12.96 ± 0.7113.20 ± 0.6814.50 ± 0.6917.86 ± 0.72
All jointly12.96 ± 0.7112.90 ± 0.5913.54 ± 0.6416.00 ± 0.73

Figure 8 reports these values. Normal tilt and length perturbations have little effect on aggregate FGW at the tested magnitudes. At High, centroid noise increases FGW by 14.66%, dropout by 37.81%, and joint noise by 23.46%. Figure 11 shows that degradation is concentrated at lower levels: under High joint noise, F1 falls by approximately 40%, 42%, and 46% for walls, rooms, and floors, versus 7% for buildings; city F1 remains within one standard error of the clean value. Dropout also affects the strict matching metric: losing one of a wall's two planes prevents a match at IoU > 0.5. These results indicate tolerance to the tested geometric perturbations, with missing planes the main source of degradation. Details are in Appendix L.

Thank you for the additional experiments. I have also read the revised PDF. The input-plane perturbation tests and per-level metrics provide a much clearer picture of the robustness.

Although missing planes remain challenging and floor-level accuracy is still limited, these limitations are now better characterized and are acceptable within the scope of this work. The additions address my concerns.

Dear Reviewer 8X1Q,

Thank you very much for taking the time to read the revised PDF and the additional experiments. We are very glad that the input-plane perturbation tests and per-level metrics now give a clearer picture of the method's robustness. We also appreciate your recognition that the remaining limitations (missing planes and floor-level accuracy) are now well characterized within the scope of this work.

Thank you again for your constructive feedback, which has clearly improved the paper. If all your concerns have been addressed, we would be grateful if you would consider updating your score to reflect the revised version.

Best regards, The Authors

All my concerns are resolved and I will keep my rating because it's already positive. I have raised the confidence score. Thank you for the contributions made by your paper.

Reviewer_rHtXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper develops an approach to build hierarchical 3D scene graphs of indoor environments using autoregressive diffusion. The approach starts with a set of planes/walls and auto-regressively generates nodes in the scene graph to create a 3D scene graph with rooms, floor, buildings, and city nodes. First, the graph is auto-regressively expanded using prior work (FLexible Autoregressive Graph Generation – FLAGG). While nodes are added, a module based on prior work (mixed graph and 3D denoising diffusion – Midi) adds features and edges to the graph. The approach is tested on a dataset including synthetic data, the existing MSD dataset, and a handful of scenes built from real LIDAR data.

Critical strengths
  • Advancing the construction of 3D scene graphs and reducing the reliance on hand-crafted heuristics is an important goal.
  • The use of autoregressive generation is a reasonable approach to the problem.
Critical weaknesses
  • The definition of 3D scene graph is misleading and misaligned with the state of the art. Saying that the lower-layer of a scene graph includes planes/walls is not representative of the state of the art, where typically the basic primitives are objects, and saying that edges are only part-of relations misrepresents work focusing on other semantic, geometric, and functional relations (SceneGraphFusion, FunGraph3D, …).
  • The problem formulation is questionable in its practicality: 1) Only a set of plans are used as input for 3D scene graph construction: where is the semantic information coming from if no objects nor images are used as input? 2) Many open floor plans have multiple rooms that are not separated by walls – how can the proposed pipeline work in this case?
  • The paper misses several related works on the creation of high-level abstractions (rooms) in indoor scene graphs [S1-S2] as well as diffusion models for scene graph generation [S3].
  • The proposed approach is not compared against state-of-the-art baselines. Section 6.1 should better clarify if the baselines used in the paper are from related work and are designed to tackle the same problem. Currently, they mostly seem variations of the proposed approach. EC-square and EC-L are first mentioned in 6.1 and without a direct reference to other papers.
  • The tested dataset only includes a handful of scenes build from real sensor data, which leaves the reader wondering whether the approach would actually work in real robotics applications.

[S1] J. Strader, N. Hughes, W. Chen, A. Speranzon and L. Carlone, "Indoor and Outdoor 3D Scene Graph Generation Via Language-Enabled Spatial Ontologies," in IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 4886-4893, June 2024. [S2] A. Werby et al, “Hierarchical Open-Vocabulary 3D Scene Graphs for Language-Grounded Robot Navigation”, RSS 2024. [S3] S. Hu et al, “3D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environments”, IROS 2026.

Questions for authors
  • The problem formulation is questionable in its practicality: 1) Only a set of plans are used as input for 3D scene graph construction: where is the semantic information coming from if no objects nor images are used as input? 2) Many open floor plans have multiple rooms that are not separated by walls – how can the proposed pipeline work in this case?
  • The proposed approach is not compared against state-of-the-art baselines. Section 6.1 should better clarify if the baselines used in the paper are from related work and are designed to tackle the same problem. Currently, they mostly seem variations of the proposed approach. EC-square and EC-L are first mentioned in 6.1 and without a direct reference to other papers. Can the authors clarify the nature of the baselines in the paper?
5 comments

We thank the reviewer for the careful reading and for the constructive points on positioning, related work and baselines. We have revised the paper accordingly.

Three clarifications underlie several comments:

(A) Scope. We address the structural hierarchy of a 3DSG (walls, rooms, floors, buildings, cities), whose observed layer consists of wall planes.

(B) Task. Given the planes observed by the robot, our model generates the higher levels bottom-up: at each level, how many concept nodes exist, their labels, centroids and part-of edges to the level below. All of this is inferred from the geometry and spatial arrangement of the planes, as learned from annotated hierarchies. For example, the model learns to group the planes that enclose a space into a room node, without any room-specific rule (Fig. 1).

(C) Comparison. A comparable method must take planes as input and output concept nodes with centroids and part-of edges to those planes, on which our evaluation relies.

CW1, Q1 - 3DSG definition. We agree that our definition read as if it covered 3DSGs in general, where the lowest layer is typically objects and edges also encode semantic, geometric and functional relations (e.g., SceneGraphFusion; OpenFunGraph, Zhang et al., 2025). We note, however, that the scope in (A) is an established part of 3DSG research: Rosinol et al. (2021) model structural elements such as walls as dedicated nodes linked to the rooms they enclose, and plane-based structural graphs are used directly for path planning (Ejaz et al., 2025) and global localisation against architectural plans (Shaheer et al., 2025). We have rewritten the introduction and problem statement to make this scope explicit, adding the missing references, and list object layers and non-part-of relations as complementary future work [Secs. 1-3, 8].

CW2, Q1, 1) - Source of semantics. Per (B), the labels are the structural concept types of (A), not object categories. They only require plane extraction, which LiDAR-based SLAM provides [Sec. 3].

CW2, Q1, 2) - Open floor plans. Rooms, like the rest of the hierarchy, are structural: spaces enclosed by the planes that bound them. An open-plan room containing several functional areas is therefore one structural room. We now list subdivision into functional spaces as future work [Secs. 3, 5.1, 8].

CW3 - Related work. We thank the reviewer for the references and have added them [Sec. 2]; [S1] and [S2] are discussed under CW4. [S3] Hu et al. (2026) address a different problem: they generate hierarchical scene graphs top-down to predict unobserved parts of a partially explored scene, whereas we infer bottom-up abstractions over planes that have already been observed.

CW4 - Baselines. We apologise for omitting the citations in Sec. 6.1. EC-□ and EC-L are not variants of our method but published methods (Millan-Romera et al., 2024; 2026). They are the only methods we are aware of that meet (C) and learn several concept types (walls and rooms) instead of hand-crafting a rule per concept. We also included MiDi (Vignac et al., 2023), adapted to conditional 3DSG completion, with an oracle supplying the node count it requires [Sec. 6.1].

Methods not included, and why [Tab. 6]:

  • Hydra and S-Graphs+: room detection by clustering free space, which requires a dense map (point cloud or ESDF). EC-L was shown to outperform both on simulated and real LiDAR data (Millan-Romera et al., 2026).
  • S-Graphs 2.0 (Bavle et al., 2025): its room segmentation is EC-□, which we evaluate directly; its floor detection relies on the robot trajectory, not available in our setting.
  • Strader et al. (2024) [S1]: it classifies places into rooms and regions from the objects they contain, so it requires an object layer.
  • HOV-SG (Werby et al., 2024) [S2] and occupancy-grid segmentation (e.g., Luperto et al., 2022): they output regions of RGB-D point clouds or occupancy maps, without plane-linked concept nodes.

Turning these outputs into such plane-linked nodes would require design decisions that the original methods do not define (node creation, centroid definition, plane assignment) and that would bias the comparison. If the reviewer is aware of other methods meeting (C), we would be grateful for the pointer and will make our best effort to include a fair comparison.

CW5 - Real-world data. We agree that the robot-recorded subset is too small to support claims about deployment. We have reframed it as a transfer check: models trained only on MSD are applied to 6 robot-recorded scenes to assess transferability to real sensor data [Abs.; Secs. 5.1, 6.2, 7, 8]. Since annotated robot data at scale is not available, we added a controlled study that perturbs the input planes of M-C with increasing noise and evaluates the generated hierarchies against the clean ground truth with new per-level metrics, characterising how plane errors propagate to each level [Sec. 6.2; Apps. K-L].

We would be glad to clarify any remaining points.

AuthorsOpenReview ↗

Follow up to our previous response

Dear Reviewer rHtX,

While awaiting your feedback on our previous response, we have used the remaining discussion time to act on your suggestion and add a direct comparison with HOV-SG [S2], the geometrically closest of the suggested pipelines (new Appendix O, revisions in blue).

As stated in our response (clarification C), we do not regard this as an entirely fair comparison in either direction: we evaluate HOV-SG’s floor and room segmentation components, which require adapting our vertical wall planes to the point-cloud input they expect. In its favour, its rules are expert-designed for exactly those two concepts. We therefore made a best-effort, fully documented adaptation, to be read as a per-concept comparison with hand-crafted expert rules:

  • We run HOV-SG's official floor and room segmentation unchanged (default parameters, no tuning, deterministic). Its floor detection needs horizontal surfaces, absent from plane-only input: on wall points alone it merges the two storeys of every M-B scene. We supply floor/ceiling slabs derived by grouping planes by height, itself a hand-crafted floor prior. Plane-to-room assignment and centroids are fixed by documented rules; passing the ground-truth room regions through the same conversion yields room F1 99.8, indicating that the conversion largely preserves correct room assignments, although interactions with imperfect predicted regions may still affect the results.
  • On M-B, HOV-SG is strong on the two concepts its rules target: room F1 85.5 vs. 54.6 (Ours) and floor F1 97.5 (given the prior) vs. 86.9. Its room errors are one-sided (precision 94.1, recall 78.3): it misses or over-segments rooms, mostly through door gaps.
  • These segmentation components produce no wall, building, or city nodes. On the full hierarchy, FGW is 48.79 vs. 4.76 (Ours); restricted to the planes+rooms+floors subgraph it generates, 7.34 vs. 8.63.

We note that the paper’s contribution is not per-concept superiority over rules expert-designed for one concept and sensor setup, but a single model that jointly generates nodes, labels, centroids and part-of edges across all hierarchy levels, with no concept-specific module, a capability no hand-crafted pipeline offers, as this comparison illustrates. Accordingly, the paper’s claims are scoped to learning-based methods [Abs.; Sec. 6.2], and remain unchanged by this result.

We would appreciate your thoughts on our earlier response, and remain happy to clarify any remaining points.

I appreciated the authors' response and the clarifications about the scene graph structure and the assumption that rooms must be wall-separated (structural rooms instead of functional rooms). However, I still find the introduction confusing in its definition of 3D scene graphs. The authors should clearly state what the standard definition is (objects with semantic and geometric attributes, relations, and possibly hierarchy) and and then specify they focus on a geometric variation that does not have objects nor semantics. In addition, the comments about the missing comparisons against baselines are not very convincing. In particular, it is unclear why HOV-SG is excluded: can the high-level regions be compared regardless of whether they are computed from planes or other primitives? Finally, the real experiments remain very small-scale as acknowledged by the authors. This leads me to keep my current rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Changes to the manuscript and ongoing work

Dear Reviewers, AC,

Revisions are highlighted in blue in the updated manuscript.

Changes made:

  • [rHtX] Clarified the structural scope, related work, and baseline selection (Secs. 1-3, 6.1, and 8).
  • [rTws] Clarified the FGW implementation and evaluated fixed input-plane correspondences (Sec. 5.2; Appendix J).
  • [8X1Q, rTws, rHtX] Added plane-anchored per-level metrics and results (Sec. 5.2; Appendix K).
  • [8X1Q, rHtX] Evaluated robustness to input-plane perturbations (Appendix L).
  • [rTws] Added training-technique ablations on M-C (Appendix M), discussed below.

Ongoing work:

  • [rTws, Q1] Controlled autoregressive-versus-one-shot comparison on M-C.
  • [rTws, Q4] Node-ordering ablations on M-C: random ordering and hierarchical ordering without proximity.

Both experiments are running. We will report their results and revise the corresponding statements once they conclude.

Before the review period, we had also run a complete ablation of all four training techniques (noise augmentation, noisy blocks, weighted subsampling, Min-SNR) and of the FD4 filler on M-C, the most complex and deepest MSD hierarchy. Because this is relevant to reviewer rTws's concerns about confounding factors, such as the inclusion of the min-SNR loss, we have added it as Appendix M (Table 13) in the revised version and would like to share it to complement our responses.

Removing any single technique increases mean FGW, and the full model is best on five of seven metrics. Every leave-one-out variant, including without Min-SNR, retains lower mean FGW and lower variance than MiDi+Or despite its oracle graph size. These results address the reviewer's concern by showing that the observed advantage persists when each training technique is removed individually.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.