- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SeeRayLoc tackles a problem in floorplan localization: models trained on one dataset often fail badly on a new dataset. The paper's main point is that current methods are trained to predict wall distances taken from the floorplan, even when walls are hidden behind furniture. Since the hidden distance cannot be seen, the model has to guess, and it ends up learning habits specific to the training data (like typical room sizes or furniture layouts) that do not hold in new datasets. SeeRayLoc splits the task into two steps. First, a frozen Depth Anything V3 model is improved by a small add-on module (the dense adapter), trained with normal image depth labels, so it predicts depth only for surfaces that are actually visible. Second, a learned selector looks at these visible-depth predictions; for each horizontal direction it creates several candidate points at different heights and picks the ones most useful for matching against the floorplan, using context from the same column and nearby columns. Only the selector is trained with floorplan depth labels. The method works for both normal perspective images and 360° panoramic images (panoramas are processed directly in ERP form with circular fusion) and is applied to new datasets without any fine-tuning. The authors also release FurnFloc, a new dataset built from HM3D with 132 floorplans and about 403k images, and test on six datasets (S3D, Matterport3D, ZInD, Gibson(f), Gibson(g), FurnFloc) using source-only training. SeeRayLoc beats retrained SemRayLoc, UnLoc, and DA3 baselines by large margins, and ablations show the gains come from the adapter, the selector, and the confidence step.
Critical strengths
The paper makes a strong case for why the method is needed: current approaches learn to predict wall distances taken from the floorplan, even when walls are hidden behind furniture, so they must guess. Since the hidden distance cannot be seen, they learn patterns tied to the training data and fail on new datasets. The fix follows directly from this diagnosis: give the visible-depth part only normal image depth labels, and give the floorplan labels only to the selector, so the design is not arbitrary but flows from the problem.
Critical weaknesses
(1) SeeRayLoc deliberately refuses to predict hidden wall distances, so under severe furniture occlusion no valid candidate may exist for the selector to recover. But furniture occlusion is precisely the regime where the paper argues prior methods fail and where the visible-depth/selection split is supposed to pay off. The evaluation does not quantify how often the selector has no usable candidate, nor does it stratify results by occlusion severity. Without this, the reader cannot tell whether the reported gains come from the mechanism the paper claims or simply from better visible-depth estimates on mostly-unoccluded views. This directly bounds the significance of the contribution.
(2) SeeRayLoc is trained on S3D (with dense GT depth) and Gibson(f) (with pseudo-depth from MoGe-3), while UnLoc and SemRayLoc are retrained on the same sources, but the baselines do not receive the frozen DA3 + adapter treatment, so part of the gap may reflect backbone capacity rather than the visible-depth/selection decomposition. A DA3-with-adapter-but-no-selector row exists in the ablation, but it is not elevated to a full comparison across all targets.
Questions for authors
(1) I would suggest the authors to report, per target dataset, the fraction of horizontal directions for which the selector's chosen candidate terminates on furniture/foreground rather than a structural boundary, and recall conditioned on occlusion level? This is needed to substantiate that the visible-depth/selection split, not just a stronger backbone, drives the gains, and to bound the limitation.
(2) The ablation includes "DA3 + dense adapter" without the selector, but the main tables compare against retrained SemRayLoc/UnLoc. It is better to show the DA3+adapter (no selector) numbers across all six targets alongside SeeRayLoc? This isolates the selector's contribution from the adapter/backbone contribution and is the single comparison that most affects how large the methodological gain really is.
(3) Gibson(f) source uses pseudo-depth labels from MoGe-3 while S3D uses GT depth. Do the S3D→target and Gibson(f)→target results differ systematically in a way attributable to label quality, and would the baselines benefit comparably if given pseudo-depth supervision of the same provenance?
(4) On Matterport3D and FurnFloc (the real-world furnished cases), gains are small or absent at fine thresholds. Is this attributable to occlusion (supporting Weakness 1), floorplan scale, or depth-adapter degradation on real imagery? A short decomposition would clarify the scope of the generalization claim.
3 comments
Author Response Part 2
W2+Q2: It is better to show the DA3+adapter (no selector) numbers across all six targets alongside SeeRayLoc?
Thank you for this suggestion. To better isolate the selector's contribution, we provide a direct comparison of DA3, DA3 + adapter, and SeeRayLoc across all evaluated datasets, as shown below:
The results show that the selector consistently improves R@ 0.5m over DA3 + dense adapter, with gains of 4.8 and 4.7 percentage points on Gibson(f) and Gibson(g), and 14.3 and 5.5 points on S3D Panorama and ZInD Panorama, respectively. Notably, on FurnFloc, the selector also mitigates the degradation introduced by the adapter, improving R@ 0.5m from 11.4% to 13.5%. We will include these results in the revised manuscript.
Q3: Do the S3D→target and Gibson(f)→target results differ systematically in a way attributable to label quality, and would the baselines benefit comparably if given pseudo-depth supervision of the same provenance?
Thank you for raising this concern. To isolate the effect of depth-supervision provenance, we generated pseudo-depth labels for S3D using the same MoGe-3 used for Gibson(f). The results are as follows:
The results reveal two distinct effects of pseudo-depth supervision. First, performance decreases on S3D and several other datasets, which is expected given that replacing GT supervision with estimated depth may introduce prediction errors. Second, performance improves on several transfer targets, including ZInD, FurnFloc, and MP3D. One possible explanation is that the noise introduced by pseudo-depth supervision acts as an implicit regularizer, potentially improving robustness to domain shifts. These observations suggest that the effect of supervision provenance is target-dependent rather than consistently beneficial or detrimental.
Author Response Part 3
Q4: On Matterport3D and FurnFloc (the real-world furnished cases), gains are small or absent at fine thresholds. Is this attributable to occlusion (supporting Weakness 1), floorplan scale, or depth-adapter degradation on real imagery? A short decomposition would clarify the scope of the generalization claim.
Neither Matterport3D nor FurnFloc provides ground-truth occlusion annotations for the evaluated queries, preventing a direct quantitative assessment of occlusion effects. We therefore examine floorplan scale and component-wise localization performance to investigate the limited fine-threshold gains.
The test-only area statistics show that Matterport3D and FurnFloc have the largest median map areas among our evaluated datasets, as shown below:
| Dataset | Min. Area | Median Area | Max. Area |
|---|---|---|---|
| Matterport3D | 17.7 | 128.5 | 1926.5 |
| FurnFloc | 23.0 | 113.2 | 362.4 |
| ZInD (p) | 17.9 | 89.2 | 281.2 |
| ZInD | 17.9 | 89.2 | 281.2 |
| Gibson(f) | 46.7 | 83.1 | 221.2 |
| Gibson(g) | 46.7 | 83.1 | 221.2 |
| Structured3D (p) | 18.0 | 78.4 | 799.2 |
| Structured3D | 18.0 | 78.4 | 799.2 |
We analyze the S3D-trained models, corresponding to the primary transfer setting in Table 2. Test floorplans are sorted by area and grouped into contiguous bins with approximately equal query counts; all frames of a floorplan remain together. Bin boundaries use only areas and query counts. All methods use identical queries within each bin.
Each method entry reports R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%). Floors and Frames denote distinct floorplan units and evaluated query frames, respectively. Area ranges below are the observed minimum and maximum map areas within each bin.
Matterport3D
| Area (m²) | No. Frames | DA3 | DA3+adapter | SeeRayLoc |
|---|---|---|---|---|
| 17.7–120.9 | 2,330 | 0.7 / 7.5 / 12.0 / 9.5 | 0.6 / 6.1 / 11.7 / 8.5 | 0.5 / 6.5 / 11.9 / 9.1 |
| 125.0–323.0 | 2,309 | 0.4 / 4.8 / 7.3 / 6.1 | 0.3 / 3.9 / 7.1 / 5.6 | 0.3 / 4.4 / 7.2 / 6.1 |
| 337.1–469.3 | 2,078 | 0.0 / 1.9 / 3.6 / 2.9 | 0.1 / 2.0 / 3.8 / 2.9 | 0.1 / 2.4 / 4.1 / 3.4 |
| 515.5–605.0 | 1,901 | 0.1 / 1.4 / 2.6 / 2.2 | 0.3 / 2.1 / 3.2 / 2.7 | 0.3 / 2.2 / 3.7 / 3.0 |
| 683.9–1926.5 | 1,933 | 0.1 / 1.9 / 3.1 / 2.6 | 0.1 / 1.0 / 2.7 / 2.4 | 0.3 / 1.7 / 3.5 / 3.0 |
FurnFloc
| Area (m²) | No. Frames | DA3 | DA3+adapter | SeeRayLoc |
|---|---|---|---|---|
| 23.0–99.3 | 21,819 | 1.5 / 15.7 / 23.2 / 19.5 | 0.9 / 12.3 / 20.8 / 16.5 | 1.2 / 13.3 / 24.6 / 20.6 |
| 113.2–151.6 | 19,627 | 3.3 / 13.6 / 20.4 / 18.3 | 2.9 / 12.4 / 19.0 / 16.8 | 2.7 / 15.1 / 24.5 / 22.4 |
| 158.4–362.4 | 20,114 | 1.9 / 10.1 / 12.9 / 11.8 | 2.3 / 9.6 / 12.6 / 11.4 | 2.0 / 12.1 / 17.3 / 16.1 |
The area-stratified results reveal a general reduction in absolute recall on larger maps, particularly on Matterport3D. However, limited gains at fine thresholds also occur in smaller-map bins: on FurnFloc's smallest bin, the adapter reduces R@ 0.5m from 15.7% to 12.3%, and the full model partially recovers it to 13.3%. In the two larger bins, the full model exceeds DA3 by 1.4 and 2.0 percentage points at R@ 0.5m. Thus, map scale is associated with task difficulty, but does not alone explain the limited fine-threshold gains; the component effects also vary across targets and area bins.
Author Response Part 1
Thank you for this suggestion. Figure 4 and Appendix Figures 7 and 8 qualitatively illustrate how selection changes the 40-ray profiles. For quantitative evaluation, we select Structured3D perspective and panoramic imagery because both provide native pixel-level semantic annotations aligned with RGB. We use the same S3D-trained model for both settings.
Under wall-distance supervision, the predicted horizontal profile targets floorplan wall distances, making a fixed center-row readout consistent with the supervision target. With visible-surface depth supervision, however, this readout may instead return foreground depth. We therefore compare fixed center sampling and learned selection on the same visible-depth predictions. For a perspective query, we retain 40 horizontal directions and construct 21 candidates per direction (840 per image), including the center and ten samples on either side within the visible gravity-elevation interval. A non-center candidate replaces the center only if its predicted structural compatibility is higher; otherwise, the center is retained. For panoramas, the same local selection procedure is applied within token windows before circular fusion.
Semantic evaluation. GT wall-pixel fraction among labeled image pixels defines low-, medium-, and high-visibility groups, with approximately equal frame counts within each setting. This fraction measures wall visibility rather than an exact furniture-occlusion rate. For each 40-direction local profile, each method selects one observation per direction. Center wall hit (Center) is the percentage of directions whose center candidate is wall; Selector wall hit (Selector) is the percentage whose selected candidate is wall; Max wall hit (Max) is the percentage containing at least one wall candidate among the 21. The latter is the maximum wall-hit rate attainable by selecting one candidate per direction.
S3D perspective.
Localization. Entries report R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%).
S3D panorama.
Localization. Entries report R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%).