- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses geometry and camera pose estimation for multi-room indoor scenes, where perspective views in different rooms share little or no overlap. The proposed HOME3 feeds an unposed top-down render of the whole floor into a pre-trained 3D foundation model (π3 or VGGT-Ω) as an extra view, and uses view-type-specific QKV projections and FFNs in the global attention layers to reduce interference between top-down and perspective tokens. The authors also build TD-HOMES, pairing perspective images with top-down renders from HM3D, Structured3D and Code-as-Room scenes. Experiments show clear gains over perspective-only baselines in the multi-room setting, with a smaller performance drop on standard benchmarks than naive joint fine-tuning.
Critical strengths
- Clear presentation and a simple, easy-to-follow method.
- Clear quantitative and qualitative improvements within the proposed setting.
Critical weaknesses
The setting is weakly motivated. Existing geometry foundation models fail in multi-room scenes because the rooms share almost no visual overlap, so their relative placement cannot be inferred from the perspective images. The paper addresses this by injecting a top-down view of the whole floor as an extra observation, which enables joint reconstruction of otherwise independent rooms. However, this scenario is rare in real applications, and top-down images of indoor scenes are generally hard to obtain. In fact, every top-down image in TD-HOMES is rendered from the complete 3D scene itself; having such an image presupposes that the floor has already been scanned or modelled, which largely removes the need for the task. The paper calls this "a considerably more practical setting", but provides no experiment with realistic top-down sources such as real floor plans or sketches. This weakness largely drives my recommendation: the main finding, that giving the network a global map of the scene helps it place cameras within that scene, is expected and of limited significance.
Questions for authors
All top-down images used in the paper are rendered from complete 3D scenes (reconstructed meshes, annotated layouts, or generated scenes). Could the authors clarify how such top-down images would be obtained for a large number of in-the-wild real-world scenes, and demonstrate the practical value of the method on a broad set of real scenes where the top-down input comes from a realistically available source rather than from the ground-truth scene itself?
3 comments
Official Comment by Authors
We would like to briefly follow up on our response above, particularly the additional experiments using 2D floor plans as guidance (Table R1, now Table 4 in the revised manuscript) and our clarification of the intended deployment scenarios.
When you have a chance, we would appreciate your thoughts on whether these additions help address your concerns about the practical relevance of our setting. Please also let us know if any concerns remain or if there are other points you would like us to clarify. We would be happy to provide further details. Thank you for your consideration.
After reading the response of authors, my concerns are resolved, so I raised my score.
Official Comment by Authors
We respectfully disagree with the view that our problem setup lacks practical relevance or benchmark validity. Both Reviewers 5BML and fuAD recognized the value of this setting and the design of TD-HOMES: Reviewer 5BML noted that the task is "challenging yet meaningful" and that the TD-HOMES dataset is "meaningful to the community under this line of research" due to its three complementary sources; similarly, Reviewer fuAD noted that disjoint multi-room reconstruction represents "a well-defined and consequential failure regime" requiring exocentric layout cues, and described TD-HOMES as "a useful dataset and reasonably controlled comparisons". Below, we further clarify why our paired top-down formulation reflects realistic deployment scenarios where end-users lack access to the underlying 3D model, and show that HOME³ directly extends to non-rendered 2D floor plans:
Practical motivation. While our training set renders top-down views from 3D scenes for supervision, having a 2D top-down image at inference time does not mean the end-user has access to the 3D model. In many real-world scenarios, such as real-estate platforms, shopping malls, and museums, venue owners publish static 2D top-down dollhouse renders or overview maps for human viewing, while withholding raw 3D meshes due to privacy, intellectual property, and bandwidth constraints. When a user or agent captures a few sparse, non-overlapping perspective photos on-site, the 2D overview alone lacks 3D structure and camera poses. Recovering globally consistent 3D geometry and relative camera poses from these sparse inputs is needed for downstream tasks such as multi-room 3D scene reconstruction, snapshot-based camera localization, and cross-room spatial layout estimation.
Extending to floorplans. To further evaluate our framework on realistically available, non-rendered top-down sources, we extend our experiments by incorporating 2D architectural floorplans into the training of HOME³$ (denoted as HOME³† in Table R1). The floorplans are either derived from existing annotations or synthesized through an agentic procedural generation pipeline. When evaluated strictly with 2D floor plans at inference time, HOME³† outperforms the perspective-only baseline across most metrics under both in-domain and unseen zero-shot settings. For instance, improving RRA@30 from 69.75 to 84.26, RTA@30 from 68.09 to 83.55, and reducing point map accuracy error from 0.497 to 0.285 on TD-HOMES; although slight performance drop occurs on RRA@30 in TD-HM3D zero-shot. Table R1 has been added as Table 4 in the updated manuscript.
Table R1. Floorplan-guided Camera Pose and Point Map Estimation.
Clarification. When referring to "a considerably more practical setting" in Section 1, we specifically contrast treating the global prior as an unposed, uncalibrated 2D visual guide against requiring known camera extrinsics and exact metric scale alignment relative to the perspective observations. In real-world deployments, external top-down maps or floor plans rarely provide calibrated camera poses or metric scaling, so avoiding this assumption makes our formulation significantly easier to apply than pose-aligned map localization methods. We have revised the phrasing in Section 1 (Line48-49) to make this distinction explicit and avoid any ambiguity.