acceptodds
Withdrawn from ICLR 2027

SeeRayLoc: Generalizable Floorplan Localization with What You See

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Ray-based floorplan localization estimates camera pose by matching image-derived depth rays against floorplan geometry. Existing methods directly predict floorplan-aligned wall distances, including geometry that is often not visible in the image. Such limitation on implicit scene completion can overfit to dataset-specific domain priors and harm cross-domain generalization. We introduce SeeRayLoc, a framework that decouples visible geometry estimation from localization-oriented ray construction. SeeRayLoc estimates visible scene depth and adaptively selects localization-relevant rays for floorplan matching. Floorplan supervision guides ray selection instead of hidden-geometry prediction, yielding generalizable floorplan localization. We evaluate SeeRayLoc through source-to-target transfer across six indoor localization datasets, including FurnFloc, our newly curated dataset for furnished scenes. The evaluation spans diverse scene distributions, camera configurations, fields of view, and both perspective and panoramic imagery. Extensive experiments show that SeeRayLoc has stronger transfer robustness than direct floorplan-depth prediction, achieving the highest R@1m30° recall in all seven Structured3D source transfer settings across perspective and panoramic imagery. It proves the effectiveness of visible geometry providing a more generalizable basis for ray-based floorplan localization. Project page and code at: https://anonymous.4open.science/r/SeeRayLoc-9573.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual localization, Generalization, Dataset and Benchmark, Floorplan

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_m3BtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SeeRayLoc tackles a problem in floorplan localization: models trained on one dataset often fail badly on a new dataset. The paper's main point is that current methods are trained to predict wall distances taken from the floorplan, even when walls are hidden behind furniture. Since the hidden distance cannot be seen, the model has to guess, and it ends up learning habits specific to the training data (like typical room sizes or furniture layouts) that do not hold in new datasets. SeeRayLoc splits the task into two steps. First, a frozen Depth Anything V3 model is improved by a small add-on module (the dense adapter), trained with normal image depth labels, so it predicts depth only for surfaces that are actually visible. Second, a learned selector looks at these visible-depth predictions; for each horizontal direction it creates several candidate points at different heights and picks the ones most useful for matching against the floorplan, using context from the same column and nearby columns. Only the selector is trained with floorplan depth labels. The method works for both normal perspective images and 360° panoramic images (panoramas are processed directly in ERP form with circular fusion) and is applied to new datasets without any fine-tuning. The authors also release FurnFloc, a new dataset built from HM3D with 132 floorplans and about 403k images, and test on six datasets (S3D, Matterport3D, ZInD, Gibson(f), Gibson(g), FurnFloc) using source-only training. SeeRayLoc beats retrained SemRayLoc, UnLoc, and DA3 baselines by large margins, and ablations show the gains come from the adapter, the selector, and the confidence step.

Critical strengths

The paper makes a strong case for why the method is needed: current approaches learn to predict wall distances taken from the floorplan, even when walls are hidden behind furniture, so they must guess. Since the hidden distance cannot be seen, they learn patterns tied to the training data and fail on new datasets. The fix follows directly from this diagnosis: give the visible-depth part only normal image depth labels, and give the floorplan labels only to the selector, so the design is not arbitrary but flows from the problem.

Critical weaknesses

(1) SeeRayLoc deliberately refuses to predict hidden wall distances, so under severe furniture occlusion no valid candidate may exist for the selector to recover. But furniture occlusion is precisely the regime where the paper argues prior methods fail and where the visible-depth/selection split is supposed to pay off. The evaluation does not quantify how often the selector has no usable candidate, nor does it stratify results by occlusion severity. Without this, the reader cannot tell whether the reported gains come from the mechanism the paper claims or simply from better visible-depth estimates on mostly-unoccluded views. This directly bounds the significance of the contribution.

(2) SeeRayLoc is trained on S3D (with dense GT depth) and Gibson(f) (with pseudo-depth from MoGe-3), while UnLoc and SemRayLoc are retrained on the same sources, but the baselines do not receive the frozen DA3 + adapter treatment, so part of the gap may reflect backbone capacity rather than the visible-depth/selection decomposition. A DA3-with-adapter-but-no-selector row exists in the ablation, but it is not elevated to a full comparison across all targets.

Questions for authors

(1) I would suggest the authors to report, per target dataset, the fraction of horizontal directions for which the selector's chosen candidate terminates on furniture/foreground rather than a structural boundary, and recall conditioned on occlusion level? This is needed to substantiate that the visible-depth/selection split, not just a stronger backbone, drives the gains, and to bound the limitation.

(2) The ablation includes "DA3 + dense adapter" without the selector, but the main tables compare against retrained SemRayLoc/UnLoc. It is better to show the DA3+adapter (no selector) numbers across all six targets alongside SeeRayLoc? This isolates the selector's contribution from the adapter/backbone contribution and is the single comparison that most affects how large the methodological gain really is.

(3) Gibson(f) source uses pseudo-depth labels from MoGe-3 while S3D uses GT depth. Do the S3D→target and Gibson(f)→target results differ systematically in a way attributable to label quality, and would the baselines benefit comparably if given pseudo-depth supervision of the same provenance?

(4) On Matterport3D and FurnFloc (the real-world furnished cases), gains are small or absent at fine thresholds. Is this attributable to occlusion (supporting Weakness 1), floorplan scale, or depth-adapter degradation on real imagery? A short decomposition would clarify the scope of the generalization claim.

3 comments
AuthorsOpenReview ↗

Author Response Part 1

W1+Q1: The evaluation does not quantify how often the selector has no usable candidate, nor does it stratify results by occlusion severity.

Thank you for this suggestion. Figure 4 and Appendix Figures 7 and 8 qualitatively illustrate how selection changes the 40-ray profiles. For quantitative evaluation, we select Structured3D perspective and panoramic imagery because both provide native pixel-level semantic annotations aligned with RGB. We use the same S3D-trained model for both settings.

Under wall-distance supervision, the predicted horizontal profile targets floorplan wall distances, making a fixed center-row readout consistent with the supervision target. With visible-surface depth supervision, however, this readout may instead return foreground depth. We therefore compare fixed center sampling and learned selection on the same visible-depth predictions. For a perspective query, we retain 40 horizontal directions and construct 21 candidates per direction (840 per image), including the center and ten samples on either side within the visible gravity-elevation interval. A non-center candidate replaces the center only if its predicted structural compatibility is higher; otherwise, the center is retained. For panoramas, the same local selection procedure is applied within token windows before circular fusion.

Semantic evaluation. GT wall-pixel fraction among labeled image pixels defines low-, medium-, and high-visibility groups, with approximately equal frame counts within each setting. This fraction measures wall visibility rather than an exact furniture-occlusion rate. For each 40-direction local profile, each method selects one observation per direction. Center wall hit (Center) is the percentage of directions whose center candidate is wall; Selector wall hit (Selector) is the percentage whose selected candidate is wall; Max wall hit (Max) is the percentage containing at least one wall candidate among the 21. The latter is the maximum wall-hit rate attainable by selecting one candidate per direction.

S3D perspective.

VisibilityNo. FramesWall pixels (%)Center(%)Selector (%)Max (%)
All6,4050.0–100.045.756.178.0
Low2,1350.0–33.024.734.960.3
Medium2,13533.0–51.843.255.779.4
High2,13551.8–100.069.077.694.0

Localization. Entries report R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%).

VisibilityNo. FramesDA3DA3+adapterSeeRayLoc
Low2,1353.4 / 30.1 / 47.8 / 45.23.9 / 32.7 / 48.9 / 46.16.9 / 48.1 / 60.7 / 58.9
Medium2,1353.8 / 37.1 / 54.3 / 52.37.4 / 42.6 / 55.6 / 52.610.9 / 55.7 / 65.1 / 63.8
High2,1353.9 / 35.4 / 46.2 / 43.79.3 / 43.0 / 50.2 / 47.313.8 / 56.4 / 60.9 / 58.9

S3D panorama.

VisibilityNo. FramesWall pixels (%)Center (%)Selector (%)Max (%)
All1,6970.1–100.050.458.773.4
Low5660.1–18.232.742.760.4
Medium56618.2–30.245.956.172.6
High56530.2–100.072.176.986.9

Localization. Entries report R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%).

VisibilityFramesDA3DA3+adapterSeeRayLoc
Low5665.1 / 33.6 / 41.0 / 38.27.2 / 39.0 / 44.3 / 41.011.8 / 52.1 / 55.3 / 53.4
Medium5666.5 / 30.9 / 39.4 / 31.47.8 / 35.0 / 43.3 / 33.913.1 / 49.3 / 54.8 / 48.9
High5653.4 / 21.6 / 31.3 / 16.54.6 / 31.0 / 42.3 / 25.512.0 / 46.5 / 53.5 / 44.1
AuthorsOpenReview ↗

Author Response Part 2

W2+Q2: It is better to show the DA3+adapter (no selector) numbers across all six targets alongside SeeRayLoc?

Thank you for this suggestion. To better isolate the selector's contribution, we provide a direct comparison of DA3, DA3 + adapter, and SeeRayLoc across all evaluated datasets, as shown below:

Source: S3DMethod0.1m0.5m1m1m30°DA33.734.249.547.1DA3+adapter6.939.551.548.7SeeRayLoc10.553.462.260.5Target : Gibson(f)Method0.1m0.5m1m1m30°DA36.738.046.044.6DA3+adapter9.539.546.344.7SeeRayLoc10.044.349.648.0\def\T{\text}\begin{gathered}\T{Source: S3D}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&3.7&34.2&49.5&47.1\\\\\T{DA3+adapter}&6.9&39.5&51.5&48.7\\\\\T{SeeRayLoc}&10.5&53.4&62.2&60.5\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target : Gibson(f)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&6.7&38.0&46.0&44.6\\\\\T{DA3+adapter}&9.5&39.5&46.3&44.7\\\\\T{SeeRayLoc}&10.0&44.3&49.6&48.0\\\\\hline\end{array}\end{gathered}

Target: Gibson(g)Method0.1m0.5m1m1m30°DA36.634.342.340.5DA3+adapter7.936.043.441.7SeeRayLoc9.240.746.444.7Target: ZInDMethod0.1m0.5m1m1m30°DA31.416.727.924.5DA3+adapter1.616.828.525.2SeeRayLoc1.818.430.727.9\def\T{\text}\begin{gathered}\T{Target: Gibson(g)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&6.6&34.3&42.3&40.5\\\\\T{DA3+adapter}&7.9&36.0&43.4&41.7\\\\\T{SeeRayLoc}&9.2&40.7&46.4&44.7\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&1.4&16.7&27.9&24.5\\\\\T{DA3+adapter}&1.6&16.8&28.5&25.2\\\\\T{SeeRayLoc}&1.8&18.4&30.7&27.9\\\\\hline\end{array}\end{gathered}

Target: FurnFlocMethod0.1m0.5m1m1m30°DA32.213.218.916.6DA3+adapter2.011.417.614.9SeeRayLoc1.913.522.219.7Target: MP3DMethod0.1m0.5m1m1m30°DA30.33.76.04.9DA3+adapter0.33.15.94.6SeeRayLoc0.33.66.35.1\def\T{\text}\begin{gathered}\T{Target: FurnFloc}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&2.2&13.2&18.9&16.6\\\\\T{DA3+adapter}&2.0&11.4&17.6&14.9\\\\\T{SeeRayLoc}&1.9&13.5&22.2&19.7\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: MP3D}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&0.3&3.7&6.0&4.9\\\\\T{DA3+adapter}&0.3&3.1&5.9&4.6\\\\\T{SeeRayLoc}&0.3&3.6&6.3&5.1\\\\\hline\end{array}\end{gathered}

Target: S3D (p)Method0.1m0.5m1m1m30°DA35.028.737.228.7DA3+adapter6.535.043.333.5SeeRayLoc12.349.354.548.8Target: ZInD (p)Method0.1m0.5m1m1m30°DA325.864.968.667.3DA3+adapter25.667.570.869.5SeeRayLoc28.173.076.074.9\def\T{\text}\begin{gathered}\T{Target: S3D (p)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&5.0&28.7&37.2&28.7\\\\\T{DA3+adapter}&6.5&35.0&43.3&33.5\\\\\T{SeeRayLoc}&12.3&49.3&54.5&48.8\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD (p)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&25.8&64.9&68.6&67.3\\\\\T{DA3+adapter}&25.6&67.5&70.8&69.5\\\\\T{SeeRayLoc}&28.1&73.0&76.0&74.9\\\\\hline\end{array}\end{gathered}

The results show that the selector consistently improves R@ 0.5m over DA3 + dense adapter, with gains of 4.8 and 4.7 percentage points on Gibson(f) and Gibson(g), and 14.3 and 5.5 points on S3D Panorama and ZInD Panorama, respectively. Notably, on FurnFloc, the selector also mitigates the degradation introduced by the adapter, improving R@ 0.5m from 11.4% to 13.5%. We will include these results in the revised manuscript.

Q3: Do the S3D→target and Gibson(f)→target results differ systematically in a way attributable to label quality, and would the baselines benefit comparably if given pseudo-depth supervision of the same provenance?

Thank you for raising this concern. To isolate the effect of depth-supervision provenance, we generated pseudo-depth labels for S3D using the same MoGe-3 used for Gibson(f). The results are as follows:

Descending GroupDatasetLabel0.1m0.5m1m1m30°S3DGT10.553.462.260.5S3DPseudo9.052.861.860.2Gibson(f)GT10.044.349.648.0Gibson(f)Pseudo6.940.947.045.5Gibson(g)GT9.240.746.444.7Gibson(g)Pseudo7.036.543.041.4ZInD(p)GT28.173.076.074.9ZInD(p)Pseudo27.872.975.774.6Ascending GroupDatasetLabel0.1m0.5m1m1m30°ZInDGT1.818.530.727.9ZInDPseudo1.819.731.928.7FurnFlocGT1.913.522.219.7FurnFlocPseudo2.516.425.823.5MP3DGT0.33.66.35.1MP3DPseudo0.23.86.85.6S3D(p)GT12.349.354.548.8S3D(p)Pseudo16.654.559.154.5\def\T{\text}\begin{gathered}\T{Descending Group}\\\\\begin{array}{llrrrr}\hline\T{Dataset}&\T{Label}&0.1m&0.5m&1m&1m30°\\\\\hline\T{S3D}&\T{GT}&10.5&53.4&62.2&60.5\\\\\T{S3D}&\T{Pseudo}&9.0&52.8&61.8&60.2\\\\\T{Gibson(f)}&\T{GT}&10.0&44.3&49.6&48.0\\\\\T{Gibson(f)}&\T{Pseudo}&6.9&40.9&47.0&45.5\\\\\T{Gibson(g)}&\T{GT}&9.2&40.7&46.4&44.7\\\\\T{Gibson(g)}&\T{Pseudo}&7.0&36.5&43.0&41.4\\\\\T{ZInD(p)}&\T{GT}&28.1&73.0&76.0&74.9\\\\\T{ZInD(p)}&\T{Pseudo}&27.8&72.9&75.7&74.6\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Ascending Group}\\\\\begin{array}{llrrrr}\hline\T{Dataset}&\T{Label}&0.1m&0.5m&1m&1m30°\\\\\hline\T{ZInD}&\T{GT}&1.8&18.5&30.7&27.9\\\\\T{ZInD}&\T{Pseudo}&1.8&19.7&31.9&28.7\\\\\T{FurnFloc}&\T{GT}&1.9&13.5&22.2&19.7\\\\\T{FurnFloc}&\T{Pseudo}&2.5&16.4&25.8&23.5\\\\\T{MP3D}&\T{GT}&0.3&3.6&6.3&5.1\\\\\T{MP3D}&\T{Pseudo}&0.2&3.8&6.8&5.6\\\\\T{S3D(p)}&\T{GT}&12.3&49.3&54.5&48.8\\\\\T{S3D(p)}&\T{Pseudo}&16.6&54.5&59.1&54.5\\\\\hline\end{array}\end{gathered}

The results reveal two distinct effects of pseudo-depth supervision. First, performance decreases on S3D and several other datasets, which is expected given that replacing GT supervision with estimated depth may introduce prediction errors. Second, performance improves on several transfer targets, including ZInD, FurnFloc, and MP3D. One possible explanation is that the noise introduced by pseudo-depth supervision acts as an implicit regularizer, potentially improving robustness to domain shifts. These observations suggest that the effect of supervision provenance is target-dependent rather than consistently beneficial or detrimental.

AuthorsOpenReview ↗

Author Response Part 3

Q4: On Matterport3D and FurnFloc (the real-world furnished cases), gains are small or absent at fine thresholds. Is this attributable to occlusion (supporting Weakness 1), floorplan scale, or depth-adapter degradation on real imagery? A short decomposition would clarify the scope of the generalization claim.

Neither Matterport3D nor FurnFloc provides ground-truth occlusion annotations for the evaluated queries, preventing a direct quantitative assessment of occlusion effects. We therefore examine floorplan scale and component-wise localization performance to investigate the limited fine-threshold gains.

The test-only area statistics show that Matterport3D and FurnFloc have the largest median map areas among our evaluated datasets, as shown below:

DatasetMin. AreaMedian AreaMax. Area
Matterport3D17.7128.51926.5
FurnFloc23.0113.2362.4
ZInD (p)17.989.2281.2
ZInD17.989.2281.2
Gibson(f)46.783.1221.2
Gibson(g)46.783.1221.2
Structured3D (p)18.078.4799.2
Structured3D18.078.4799.2

We analyze the S3D-trained models, corresponding to the primary transfer setting in Table 2. Test floorplans are sorted by area and grouped into contiguous bins with approximately equal query counts; all frames of a floorplan remain together. Bin boundaries use only areas and query counts. All methods use identical queries within each bin.

Each method entry reports R@ 0.1m / R@ 0.5m / R@ 1m / R@ 1m30° (%). Floors and Frames denote distinct floorplan units and evaluated query frames, respectively. Area ranges below are the observed minimum and maximum map areas within each bin.

Matterport3D

Area (m²)No. FramesDA3DA3+adapterSeeRayLoc
17.7–120.92,3300.7 / 7.5 / 12.0 / 9.50.6 / 6.1 / 11.7 / 8.50.5 / 6.5 / 11.9 / 9.1
125.0–323.02,3090.4 / 4.8 / 7.3 / 6.10.3 / 3.9 / 7.1 / 5.60.3 / 4.4 / 7.2 / 6.1
337.1–469.32,0780.0 / 1.9 / 3.6 / 2.90.1 / 2.0 / 3.8 / 2.90.1 / 2.4 / 4.1 / 3.4
515.5–605.01,9010.1 / 1.4 / 2.6 / 2.20.3 / 2.1 / 3.2 / 2.70.3 / 2.2 / 3.7 / 3.0
683.9–1926.51,9330.1 / 1.9 / 3.1 / 2.60.1 / 1.0 / 2.7 / 2.40.3 / 1.7 / 3.5 / 3.0

FurnFloc

Area (m²)No. FramesDA3DA3+adapterSeeRayLoc
23.0–99.321,8191.5 / 15.7 / 23.2 / 19.50.9 / 12.3 / 20.8 / 16.51.2 / 13.3 / 24.6 / 20.6
113.2–151.619,6273.3 / 13.6 / 20.4 / 18.32.9 / 12.4 / 19.0 / 16.82.7 / 15.1 / 24.5 / 22.4
158.4–362.420,1141.9 / 10.1 / 12.9 / 11.82.3 / 9.6 / 12.6 / 11.42.0 / 12.1 / 17.3 / 16.1

The area-stratified results reveal a general reduction in absolute recall on larger maps, particularly on Matterport3D. However, limited gains at fine thresholds also occur in smaller-map bins: on FurnFloc's smallest bin, the adapter reduces R@ 0.5m from 15.7% to 12.3%, and the full model partially recovers it to 13.3%. In the two larger bins, the full model exceeds DA3 by 1.4 and 2.0 percentage points at R@ 0.5m. Thus, map scale is associated with task difficulty, but does not alone explain the limited fine-threshold gains; the component effects also vary across targets and area bins.

Reviewer_iUv8OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

I am not an expert in this domain. Based on my understanding, this paper proposes SeeRayLoc, a framework for improving cross-dataset generalization in visual floorplan localization. Instead of directly predicting potentially occluded wall distances, the method estimates visible surface depth using a frozen Depth Anything 3 backbone and a trainable adapter. A learned selector then identifies suitable structural rays for confidence-weighted floorplan matching. The framework supports both perspective and panoramic images. Experiments use Structured3D and Gibson as source domains and evaluate transfer across six indoor datasets. The paper also introduces FurnFloc, a dataset of furnished indoor environments, and provides component ablations and qualitative analyses.

Critical strengths
  • The paper addresses an important and practically relevant problem: poor generalization of floorplan localization across different indoor environments. The observation that predicting occluded wall distances may encourage reliance on dataset-specific priors provides a clear motivation for using visible geometry.

  • The separation between visible-depth estimation and structural-ray selection is reasonable and closely related to the problem. Freezing the depth estimator during selector training prevents floorplan supervision from directly changing its visible-depth predictions. This provides a meaningful way to use observable geometry for localization while reducing the need to infer hidden walls.

Critical weaknesses
  • The source-only transfer setting is not fully clear. The ZInD evaluation uses a ZInD-specific semantic model. If this model was trained on ZInD, the complete system uses target-domain training information, which limits the generalization claim.

  • The main claim needs a more controlled comparison. The paper should compare visible-depth and wall-distance supervision using the same backbone and training budget under cross-dataset evaluation. This would help establish whether the proposed supervision improves generalization.

Questions for authors
  • Which semantic checkpoint is used for each source–target pair? In particular, was the semantic model used for ZInD evaluation trained on ZInD? Are results available without target-trained semantic models?

  • How does the DA3 wall-distance variant in Table 4 perform under the same cross-dataset protocol as SeeRayLoc? With the backbone and training budget controlled, does visible-depth supervision still provide consistent improvements?

  • How is the official test split enforced for ZInD panoramas? The provided evaluator loads all matching scene directories without reading a split file. Does the input directory contain only official test scenes?

2 comments
AuthorsOpenReview ↗

Author Response Part 1

W1+Q1:Which semantic checkpoint is used for each source–target pair?

We thank the reviewer for pointing out the ambiguity in our source-only transfer setting.

In our experiments, we use two publicly available pretrained semantic checkpoints provided by SemRayLoc, trained on S3D and ZInD, respectively. These checkpoints are directly used for inference without any additional training or fine-tuning. Specifically, the S3D-trained checkpoint is used in Table 2(a) and Table 3(c), while the ZInD-trained checkpoint is used in Table 2(d) and Table 3(d).

Importantly, the dense adapter and selector in our method are trained exclusively on their respective source-domain data, without target-domain training or fine-tuning.

To directly assess cross-domain generalization without target-domain semantic supervision, we additionally report geometry-only localization results, obtained by disabling semantic fusion and semantic room filtering. This configuration does not rely on target-trained semantic models and provides a stricter evaluation of source-only transfer.

Source: Structur3D

DatasetMethod0.1m0.5m1m1m30°
S3DDA32.118.526.524.8
S3DSeeRayLoc5.427.732.931.2
ZInDDA31.111.920.317.1
ZInDSeeRayLoc1.515.124.421.8

Source: Gibson(f)

DatasetMethod0.1m0.5m1m1m30°
S3DDA32.118.526.524.8
S3DSeeRayLoc1.816.223.321.5
ZInDDA31.111.920.317.1
ZInDSeeRayLoc2.011.217.314.2

We will revise the experimental setup and corresponding claims to explicitly distinguish between the strictly source-only geometry-based configuration and the semantics-assisted configuration that incorporates pretrained target-domain semantic knowledge.

W2+Q2:How does the DA3 wall-distance variant in Table 4 perform under the same cross-dataset protocol as SeeRayLoc?

Thank you for the suggestion. The original purpose of Table 4 is to evaluate DA3 fine-tuned for the wall-distance prediction task and compare its in-domain performance with existing methods, rather than to isolate the effect of supervision on generalization. Following the reviewer’s suggestion, we have added transfer results for this wall-distance model under the same cross-dataset evaluation protocol.

Wall Distance Source: Gibson(f)Dataset0.1m0.5m1m1m30°Gibson(f)16.849.353.551.8Gibson(g)15.547.150.949.3S3D0.11.24.31.4ZInD0.22.86.83.0FurnFloc0.10.72.70.7M3D0.00.51.50.3S3D(p)0.12.35.41.4ZInD(p)0.62.95.92.7Source: S3DDataset0.1m0.5m1m1m30°S3D12.866.772.772.0Gibson(f)0.00.52.60.7Gibson(g)0.00.42.81.0ZInD2.323.332.630.2FurnFloc0.00.31.50.1M3D0.11.54.02.8S3D(p)35.582.283.482.8ZInD(p)22.868.872.271.0\def\T{\text}\begin{gathered}\T{Source: Gibson(f)}\\\\\begin{array}{lrrrr}\hline\T{Dataset}&0.1m&0.5m&1m&1m30°\\\\\hline\mathbf{\mathrm{Gibson(f)}}&\mathbf{16.8}&\mathbf{49.3}&\mathbf{53.5}&\mathbf{51.8}\\\\\T{Gibson(g)}&15.5&47.1&50.9&49.3\\\\\T{S3D}&0.1&1.2&4.3&1.4\\\\\T{ZInD}&0.2&2.8&6.8&3.0\\\\\T{FurnFloc}&0.1&0.7&2.7&0.7\\\\\T{M3D}&0.0&0.5&1.5&0.3\\\\\T{S3D(p)}&0.1&2.3&5.4&1.4\\\\\T{ZInD(p)}&0.6&2.9&5.9&2.7\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Source: S3D}\\\\\begin{array}{lrrrr}\hline\T{Dataset}&0.1m&0.5m&1m&1m30°\\\\\hline\mathbf{\mathrm{S3D}}&\mathbf{12.8}&\mathbf{66.7}&\mathbf{72.7}&\mathbf{72.0}\\\\\T{Gibson(f)}&0.0&0.5&2.6&0.7\\\\\T{Gibson(g)}&0.0&0.4&2.8&1.0\\\\\T{ZInD}&2.3&23.3&32.6&30.2\\\\\T{FurnFloc}&0.0&0.3&1.5&0.1\\\\\T{M3D}&0.1&1.5&4.0&2.8\\\\\T{S3D(p)}&35.5&82.2&83.4&82.8\\\\\T{ZInD(p)}&22.8&68.8&72.2&71.0\\\\\hline\end{array}\end{gathered}

We additionally trained a model on S3D using visible-depth supervision while keeping the backbone and training budget fixed. The results are presented below:

Visible Depth

Dataset0.1m0.5m1m1m30°
S3D5.327.034.632.8
Gibson(f)0.00.73.31.5
Gibson(g)0.01.03.41.4
ZInD2.322.032.428.8
FurnFloc0.00.41.80.3
M3D0.11.64.12.8
S3D(p)9.840.149.838.9
ZInD(p)22.966.470.168.4

Replacing wall-distance supervision with visible-depth supervision did not yield consistent cross-dataset improvements. Therefore, the available evidence does not establish that wall-distance supervision itself causes poor generalization. Both variants undergo source-domain supervised fine-tuning, and their transfer performance may also depend on the fine-tuning procedure and changes to the pretrained representations.

We acknowledge that our original attribution went beyond the evidence. In the revision, we will clarify the purpose of Table 4, include the transfer results and controlled comparison, and distinguish the observed cross-dataset performance degradation from its possible causes. We will ground our generalization claims in the empirical performance of the complete SeeRayLoc framework, rather than claim that visible-depth supervision alone necessarily provides better generalization.

AuthorsOpenReview ↗

Author Response Part 2

Q3:How is the official test split enforced for ZInD panoramas?

We thank the reviewer for pointing out this lack of clarity. Yes, the evaluator's input directory contains only houses from the official test split. Our preprocessing script explicitly reads the test list from ZInD's official partition file, zind_partition.json, and filters the data accordingly. Thus, the split restriction is enforced during preprocessing rather than by the evaluator itself.

The official test split contains 158 houses. We exclude 17 houses whose floors lack metric-scale annotations, as they cannot be used for our meter-based floor plan localization evaluation. The final evaluation includes 141 houses, 252 valid floors, and 5,692 panoramas. We verified that all retained house IDs belong to the official test split and have no overlap with the train or validation splits.

Both panorama and perspective-view evaluations use the same retained houses and original panoramas, with each panorama generating one perspective view. Therefore, the input representation does not affect split assignment.

We will clarify the preprocessing procedure, exclusion criteria, and evaluation statistics in the revised manuscript, along with the corresponding house ID verification.

Reviewer_yHjDOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper demonstrates existing floorplan localization methods suffer from poor cross-dataset generalization. It hypothesizes that predicting occluded wall distances encourages reliance on dataset-specific priors, leading to degradation on new datasets. The proposed SeeRayLoc separates visible-depth estimation from ray selection. A depth estimator and a specialized adapter are trained on visible surfaces, and a ray-selector chooses predicted depth candidates according to their suitability for floorplan matching. Furthermore, the paper introduces FurnFloc, a floorplan localization dataset curated from HM3D and FloVerse, and evaluates transfer from Structured3D and Gibson (f) across perspective and panoramic inputs.

Critical strengths
  • The paper identifies an important limitation of existing floorplan localization methods: poor cross-dataset generalization. The empirical findings and investigation of a possible underlying cause are informative to the research community.

  • The proposed method separates visible-depth estimation from localization-specific ray selection is well-motivated. Training the depth estimator on visible surfaces and learning to select floorplan-compatible candidates is a well-executed implementation of the paper’s central idea.

Critical weaknesses
  • The experiments do not clearly establish whether poor cross-dataset generalization is caused by floorplan-derived wall distance. A more intuitive ablation would be to replace the existing visible depth estimation with wall-distance prediction, but Tables 4 and 5 introduce a wall-distance variant of DepthAnything V3 but only in-domain results are reported.

  • The paper excludes FurnFloc as a training source because “low-quality images and less standardized floorplans” (Line 300). It is unclear whether these limitations reflect challenging visual conditions or deficiencies in the reference geometry and annotations. The authors should clarify their implications for FurnFloc’s reliability and value as an evaluation benchmark.

  • The methodology is not sufficiently self-contained. Several crucial details are missing (see Questions).

Questions for authors
  • Please address issues raised in Weaknesses.

  • What is the intended conclusion of Table 4?

  • What is αk\alpha_k in Equation (7)?

  • How sensitive is the performance relative to the choice of hyperparameter γ\gamma in Equation (1)?

  • Could training the dense depth adapter on a single source dataset introduce overfitting to source-specific depth distributions? Can the authors analyze whether the adapter contributes to the negative transfer observed in Table 3?

  • For panoramic images as query during inference, please clarify how the patch embeddings applied to ERP inputs are trained when training dataset only supplies only narrow-FOV images, as in the Gibson (f) experiment in Table 3.

4 comments
AuthorsOpenReview ↗

Author Response Part 1

W1: The experiments do not clearly establish whether poor cross-dataset generalization is caused by floorplan-derived wall distance.

Thank you for highlighting this limitation. We agree that the original experiments do not isolate wall-distance supervision as the cause of poor cross-dataset generalization. We believe pretrained visible-depth priors remain important and add the following comparisons.

With DA3 frozen, we train adapters on S3D using depth.png (dense visible depth), depth.txt (40 horizontal wall-distance samples), or rendered (dense wall depth computed from floorplan walls, camera poses, and intrinsics). All selectors use depth.txt supervision with their adapters frozen; — reuses the preceding adapter. Our framework can improve transfer under each setting, with dataset-dependent gains.

A=Adapter; S=Selector.

Source: S3DSupervisionModule0.1m0.5m1m1m30°zero-shotDA33.734.249.547.1depth.pngA6.939.551.548.7—S10.553.462.260.5depth.txtA10.154.763.361.8—S12.161.268.167.1renderedA9.754.063.061.4—S11.960.467.766.3Target: Gibson(f)SupervisionModule0.1m0.5m1m1m30°zero-shotDA36.738.046.044.6depth.pngA9.539.546.344.7—S10.044.349.648.0depth.txtA8.239.745.744.2—S7.238.845.343.7renderedA9.843.448.847.3—S9.943.949.748.0\def\T{\text}\begin{gathered}\T{Source: S3D}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&3.7&34.2&49.5&47.1\\\\\T{depth.png}&A&6.9&39.5&51.5&48.7\\\\—&S&10.5&53.4&62.2&60.5\\\\\T{depth.txt}&A&10.1&54.7&63.3&61.8\\\\—&S&\mathbf{12.1}&\mathbf{61.2}&\mathbf{68.1}&\mathbf{67.1}\\\\\T{rendered}&A&9.7&54.0&63.0&61.4\\\\—&S&11.9&60.4&67.7&66.3\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: Gibson(f)}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&6.7&38.0&46.0&44.6\\\\\T{depth.png}&A&9.5&39.5&46.3&44.7\\\\—&S&\mathbf{10.0}&\mathbf{44.3}&49.6&48.0\\\\\T{depth.txt}&A&8.2&39.7&45.7&44.2\\\\—&S&7.2&38.8&45.3&43.7\\\\\T{rendered}&A&9.8&43.4&48.8&47.3\\\\—&S&9.9&43.9&\mathbf{49.7}&\mathbf{48.0}\\\\\hline\end{array}\end{gathered}

Target: Gibson(g)SupervisionModule0.1m0.5m1m1m30°zero-shotDA36.634.342.340.5depth.pngA7.936.043.441.7—S9.240.746.444.7depth.txtA7.436.242.741.0—S7.537.343.141.4renderedA8.939.445.043.2—S8.841.045.944.5Target: ZInDSupervisionModule0.1m0.5m1m1m30°zero-shotDA31.416.727.924.5depth.pngA1.616.828.525.2—S1.818.430.727.9depth.txtA1.316.628.224.9—S1.417.629.726.6renderedA1.618.129.126.0—S1.719.231.328.0\def\T{\text}\begin{gathered}\T{Target: Gibson(g)}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&6.6&34.3&42.3&40.5\\\\\T{depth.png}&A&7.9&36.0&43.4&41.7\\\\—&S&\mathbf{9.2}&40.7&\mathbf{46.4}&\mathbf{44.7}\\\\\T{depth.txt}&A&7.4&36.2&42.7&41.0\\\\—&S&7.5&37.3&43.1&41.4\\\\\T{rendered}&A&8.9&39.4&45.0&43.2\\\\—&S&8.8&\mathbf{41.0}&45.9&44.5\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&1.4&16.7&27.9&24.5\\\\\T{depth.png}&A&1.6&16.8&28.5&25.2\\\\—&S&\mathbf{1.8}&18.4&30.7&27.9\\\\\T{depth.txt}&A&1.3&16.6&28.2&24.9\\\\—&S&1.4&17.6&29.7&26.6\\\\\T{rendered}&A&1.6&18.1&29.1&26.0\\\\—&S&1.7&\mathbf{19.2}&\mathbf{31.3}&\mathbf{28.0}\\\\\hline\end{array}\end{gathered}

Target: FurnFlocSupervisionModule0.1m0.5m1m1m30°zero-shotDA32.213.218.916.6depth.pngA2.011.417.614.9—S1.913.522.219.7depth.txtA1.811.518.515.9—S1.813.421.819.1renderedA1.510.818.115.5—S1.511.419.917.1Target: MP3DSupervisionModule0.1m0.5m1m1m30°zero-shotDA30.33.76.04.9depth.pngA0.33.15.94.6—S0.33.66.35.1depth.txtA0.22.65.34.0—S0.23.05.84.5renderedA0.23.05.74.4—S0.22.95.84.4\def\T{\text}\begin{gathered}\T{Target: FurnFloc}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&\mathbf{2.2}&13.2&18.9&16.6\\\\\T{depth.png}&A&2.0&11.4&17.6&14.9\\\\—&S&1.9&\mathbf{13.5}&\mathbf{22.2}&\mathbf{19.7}\\\\\T{depth.txt}&A&1.8&11.5&18.5&15.9\\\\—&S&1.8&13.4&21.8&19.1\\\\\T{rendered}&A&1.5&10.8&18.1&15.5\\\\—&S&1.5&11.4&19.9&17.1\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: MP3D}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&0.3&\mathbf{3.7}&6.0&4.9\\\\\T{depth.png}&A&0.3&3.1&5.9&4.6\\\\—&S&\mathbf{0.3}&3.6&\mathbf{6.3}&\mathbf{5.1}\\\\\T{depth.txt}&A&0.2&2.6&5.3&4.0\\\\—&S&0.2&3.0&5.8&4.5\\\\\T{rendered}&A&0.2&3.0&5.7&4.4\\\\—&S&0.2&2.9&5.8&4.4\\\\\hline\end{array}\end{gathered}

Target: S3D(p)SupervisionModule0.1m0.5m1m1m30°zero-shotDA35.028.737.228.7depth.pngA6.535.043.333.5—S12.349.354.548.8depth.txtA17.556.960.356.8—S24.969.271.669.9renderedA15.955.158.255.0—S23.266.669.067.1Target: ZInD(p)SupervisionModule0.1m0.5m1m1m30°zero-shotDA325.864.968.667.3depth.pngA25.667.570.869.5—S28.173.076.074.9depth.txtA27.370.673.972.9—S29.875.278.277.4renderedA27.469.773.372.2—S30.574.778.077.0\def\T{\text}\begin{gathered}\T{Target: S3D(p)}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&5.0&28.7&37.2&28.7\\\\\T{depth.png}&A&6.5&35.0&43.3&33.5\\\\—&S&12.3&49.3&54.5&48.8\\\\\T{depth.txt}&A&17.5&56.9&60.3&56.8\\\\—&S&\mathbf{24.9}&\mathbf{69.2}&\mathbf{71.6}&\mathbf{69.9}\\\\\T{rendered}&A&15.9&55.1&58.2&55.0\\\\—&S&23.2&66.6&69.0&67.1\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD(p)}\\\\\begin{array}{llrrrr}\hline\T{Supervision}&\T{Module}&0.1m&0.5m&1m&1m30°\\\\\hline\T{zero-shot}&\T{DA3}&25.8&64.9&68.6&67.3\\\\\T{depth.png}&A&25.6&67.5&70.8&69.5\\\\—&S&28.1&73.0&76.0&74.9\\\\\T{depth.txt}&A&27.3&70.6&73.9&72.9\\\\—&S&29.8&\mathbf{75.2}&\mathbf{78.2}&\mathbf{77.4}\\\\\T{rendered}&A&27.4&69.7&73.3&72.2\\\\—&S&\mathbf{30.5}&74.7&78.0&77.0\\\\\hline\end{array}\end{gathered}

We will revise the introduction and discussion accordingly: Figure 1 demonstrates the transfer challenge; hidden-wall prediction is a motivating hypothesis, not an established cause. We will clarify Table 4 and add its transfer results. Our conclusions will focus on the demonstrated transfer performance of the complete SeeRayLoc framework.

AuthorsOpenReview ↗

Author Response Part 2

W2:It is unclear whether these limitations reflect challenging visual conditions or deficiencies in the reference geometry and annotations.

We thank the reviewer for raising this important distinction. The limitations of FurnFloc primarily concern the resolution and consistency of its visual and geometric data, rather than necessarily indicating deficiencies in its reference geometry.

RGB/Depth Resolution. FurnFloc provides RGB and depth images at a native resolution of 256 × 256, which is considerably lower than the 640 × 360 perspective inputs used for S3D and the 640 × 480 inputs used for Gibson(f). This provides fewer spatial samples for capturing fine-grained geometric details, such as small structures, object boundaries, and local depth variations.

Depth Quantization. FurnFloc is the only dataset among those considered that uses uint8 depth encoding, whereas all other datasets use uint16 encoding. Its uint8 encoding provides 25.5 quantization levels per meter, corresponding to a distance step of approximately 39.2 mm, compared with 1 mm for S3D. For valid, unsaturated depth values, truncation introduces a one-sided quantization error ranging from 0 to less than 39.2 mm. Furthermore, the 10 m truncation prevents the recovery of actual distances beyond this range.

Floorplan Standardization. FurnFloc floorplans are derived from reconstructed scenes through height and normal filtering, top-down projection, and post-processing, rather than following the same standardized floorplan generation and representation procedure as the other datasets. This introduces differences in map representation and structural consistency, which may increase data heterogeneity when used for training, but should be distinguished from the accuracy of the reference geometry itself.

These characteristics make FurnFloc less suitable as a training source, where consistent visual resolution, depth precision, and map representation are important. Nevertheless, its distinct data characteristics also make it a valuable evaluation benchmark for assessing localization performance under different visual and mapping conditions.

Q1:What is the intended conclusion of Table 4?

Table 4 is intended to provide a reference for DA3's in-domain localization performance under the wall-distance prediction setting. Therefore, we additionally train DA3 with floorplan-based wall-distance supervision and evaluate it separately on each dataset using the corresponding training and test splits.

We also evaluate the S3D- and Gibson(f)-trained wall-distance DA3 checkpoints from Table 4 across datasets, extending its original in-domain evaluation.

Source: Gibson(f)Dataset0.1m0.5m1m1m30°Gibson(f)16.849.353.551.8Gibson(g)15.547.150.949.3S3D0.11.24.31.4ZInD0.22.86.83.0FurnFloc0.10.72.70.7M3D0.00.51.50.3S3D(p)0.12.35.41.4ZInD(p)0.62.95.92.7Source: S3DDataset0.1m0.5m1m1m30°S3D12.866.772.772.0Gibson(f)0.00.52.60.7Gibson(g)0.00.42.81.0ZInD2.323.332.630.2FurnFloc0.00.31.50.1M3D0.11.54.02.8S3D(p)35.582.283.482.8ZInD(p)22.868.872.271.0\def\T{\text}\begin{gathered}\T{Source: Gibson(f)}\\\\\begin{array}{lrrrr}\hline\T{Dataset}&0.1m&0.5m&1m&1m30°\\\\\hline\mathbf{\mathrm{Gibson(f)}}&\mathbf{16.8}&\mathbf{49.3}&\mathbf{53.5}&\mathbf{51.8}\\\\\T{Gibson(g)}&15.5&47.1&50.9&49.3\\\\\T{S3D}&0.1&1.2&4.3&1.4\\\\\T{ZInD}&0.2&2.8&6.8&3.0\\\\\T{FurnFloc}&0.1&0.7&2.7&0.7\\\\\T{M3D}&0.0&0.5&1.5&0.3\\\\\T{S3D(p)}&0.1&2.3&5.4&1.4\\\\\T{ZInD(p)}&0.6&2.9&5.9&2.7\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Source: S3D}\\\\\begin{array}{lrrrr}\hline\T{Dataset}&0.1m&0.5m&1m&1m30°\\\\\hline\mathbf{\mathrm{S3D}}&\mathbf{12.8}&\mathbf{66.7}&\mathbf{72.7}&\mathbf{72.0}\\\\\T{Gibson(f)}&0.0&0.5&2.6&0.7\\\\\T{Gibson(g)}&0.0&0.4&2.8&1.0\\\\\T{ZInD}&2.3&23.3&32.6&30.2\\\\\T{FurnFloc}&0.0&0.3&1.5&0.1\\\\\T{M3D}&0.1&1.5&4.0&2.8\\\\\T{S3D(p)}&35.5&82.2&83.4&82.8\\\\\T{ZInD(p)}&22.8&68.8&72.2&71.0\\\\\hline\end{array}\end{gathered}

Q2:What is αk\alpha_k in Equation (7)?

We thank the reviewer for pointing out this missing definition. αk\alpha_k denotes the horizontal relative azimuth angle of the kk-th resampled localization ray. It is determined by the projection model and the predefined sampling grid, and is used to align image-side distance observations with the corresponding floorplan rays under a candidate pose. We will add its definition after Equation (7) and clarify its role in the subsequent map-ray queries.

AuthorsOpenReview ↗

Author Response Part 3

Q3:How sensitive is the performance relative to the choice of hyperparameter γ\gamma in Equation (1)?

γ\gamma controls the maximum multiplicative adjustment applied by the dense adapter to DA3's initial depth estimates. We set γ=ln⁡2\gamma=\ln 2 by default, allowing the depth estimates to be adjusted within a range of 0.5–2 times their initial values.

To evaluate its sensitivity, we compare γ∈ln⁡1.5,ln⁡2,ln⁡3\gamma \in {\ln 1.5, \ln 2, \ln 3}, corresponding to adjustment ranges of approximately [0.67, 1.5], [0.5, 2], and [0.33, 3], respectively. For each setting, we independently train the adapter on S3D while keeping all other training configurations and source-domain validation criteria unchanged. We then evaluate localization performance on both the source domain and unseen target domains. The results will be reported in the revised manuscript to assess the sensitivity of performance to this hyperparameter.

S3Dγ0.1m0.5m1m1m30°ln⁡1.56.739.751.949.0ln⁡26.939.551.548.7ln⁡36.439.852.049.1Gibson(g)γ0.1m0.5m1m1m30°ln⁡1.57.935.442.340.9ln⁡27.936.043.441.7ln⁡37.635.742.941.3\def\T{\text}\begin{gathered}\T{S3D}\\\\\begin{array}{lrrrr}\hline\gamma&0.1m&0.5m&1m&1m30°\\\\\hline\ln1.5&6.7&39.7&51.9&49.0\\\\\ln2&6.9&39.5&51.5&48.7\\\\\ln3&6.4&39.8&52.0&49.1\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Gibson(g)}\\\\\begin{array}{lrrrr}\hline\gamma&0.1m&0.5m&1m&1m30°\\\\\hline\ln1.5&7.9&35.4&42.3&40.9\\\\\ln2&7.9&36.0&43.4&41.7\\\\\ln3&7.6&35.7&42.9&41.3\\\\\hline\end{array}\end{gathered}

ZInDγ0.1m0.5m1m1m30°ln⁡1.51.516.327.724.6ln⁡21.616.828.525.2ln⁡31.617.228.625.2S3D Panoramaγ0.1m0.5m1m1m30°ln⁡1.56.434.143.933.8ln⁡26.535.043.333.5ln⁡36.535.444.033.5\def\T{\text}\begin{gathered}\T{ZInD}\\\\\begin{array}{lrrrr}\hline\gamma&0.1m&0.5m&1m&1m30°\\\\\hline\ln1.5&1.5&16.3&27.7&24.6\\\\\ln2&1.6&16.8&28.5&25.2\\\\\ln3&1.6&17.2&28.6&25.2\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{S3D Panorama}\\\\\begin{array}{lrrrr}\hline\gamma&0.1m&0.5m&1m&1m30°\\\\\hline\ln1.5&6.4&34.1&43.9&33.8\\\\\ln2&6.5&35.0&43.3&33.5\\\\\ln3&6.5&35.4&44.0&33.5\\\\\hline\end{array}\end{gathered}

Q4:Can the authors analyze whether the adapter contributes to the negative transfer observed in Table 3?

We agree that training on a single source dataset may introduce source-specific bias. Importantly, the Gibson(f) trained dense adapter in Table 3 uses MoGe-3 pseudo-depth supervision rather than GT depth. Therefore, the observed negative transfer alone cannot distinguish source-distribution overfitting from pseudo-label noise or bias, or limitations of the learned selection policy.

To examine the adapter's contribution, we report cross-dataset ablations under the same evaluation protocol. Source: Gibson(f)Method0.1m0.5m1m1m30°DA36.738.046.044.6DA3+adapter8.640.848.146.7SeeRayLoc11.946.450.049.0Target: Gibson(g)Method0.1m0.5m1m1m30°DA36.634.342.340.5DA3+adapter7.537.444.443.1SeeRayLoc13.144.749.147.5\def\T{\text}\begin{gathered}\T{Source: Gibson(f)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&6.7&38.0&46.0&44.6\\\\\T{DA3+adapter}&8.6&40.8&48.1&46.7\\\\\T{SeeRayLoc}&11.9&46.4&50.0&49.0\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: Gibson(g)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&6.6&34.3&42.3&40.5\\\\\T{DA3+adapter}&7.5&37.4&44.4&43.1\\\\\T{SeeRayLoc}&13.1&44.7&49.1&47.5\\\\\hline\end{array}\end{gathered}

Target: S3DMethod0.1m0.5m1m1m30°DA33.734.249.547.1DA3+adapter3.031.949.246.9SeeRayLoc3.833.247.945.4Target: ZInDMethod0.1m0.5m1m1m30°DA31.416.727.924.5DA3+adapter3.826.435.833.1SeeRayLoc1.815.023.420.1\def\T{\text}\begin{gathered}\T{Target: S3D}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&3.7&34.2&49.5&47.1\\\\\T{DA3+adapter}&3.0&31.9&49.2&46.9\\\\\T{SeeRayLoc}&3.8&33.2&47.9&45.4\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&1.4&16.7&27.9&24.5\\\\\T{DA3+adapter}&3.8&26.4&35.8&33.1\\\\\T{SeeRayLoc}&1.8&15.0&23.4&20.1\\\\\hline\end{array}\end{gathered}

Target: FurnFlocMethod0.1m0.5m1m1m30°DA32.213.218.916.6DA3+adapter2.213.219.216.8SeeRayLoc2.717.125.423.1Target: MP3DMethod0.1m0.5m1m1m30°DA30.33.76.04.9DA3+adapter0.12.35.14.0SeeRayLoc0.33.15.64.2\def\T{\text}\begin{gathered}\T{Target: FurnFloc}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&2.2&13.2&18.9&16.6\\\\\T{DA3+adapter}&2.2&13.2&19.2&16.8\\\\\T{SeeRayLoc}&2.7&17.1&25.4&23.1\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: MP3D}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&0.3&3.7&6.0&4.9\\\\\T{DA3+adapter}&0.1&2.3&5.1&4.0\\\\\T{SeeRayLoc}&0.3&3.1&5.6&4.2\\\\\hline\end{array}\end{gathered}

Target: S3D (p)Method0.1m0.5m1m1m30°DA35.028.737.228.7DA3+adapter6.235.045.135.6SeeRayLoc6.434.942.732.7Target: ZInD (p)Method0.1m0.5m1m1m30°DA325.864.968.667.3DA3+adapter23.968.271.870.5SeeRayLoc23.662.465.263.6\def\T{\text}\begin{gathered}\T{Target: S3D (p)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&5.0&28.7&37.2&28.7\\\\\T{DA3+adapter}&6.2&35.0&45.1&35.6\\\\\T{SeeRayLoc}&6.4&34.9&42.7&32.7\\\\\hline\end{array}\end{gathered}\qquad\begin{gathered}\T{Target: ZInD (p)}\\\\\begin{array}{lrrrr}\hline\T{Method}&0.1m&0.5m&1m&1m30°\\\\\hline\T{DA3}&25.8&64.9&68.6&67.3\\\\\T{DA3+adapter}&23.9&68.2&71.8&70.5\\\\\T{SeeRayLoc}&23.6&62.4&65.2&63.6\\\\\hline\end{array}\end{gathered}

One possible explanation for the selector's weaker transfer is the limited diversity of its source training scenes. Gibson(f) contains 100 training scene units, compared with 2,796 in Structured3D. This difference may limit the diversity of structural visibility and occlusion patterns encountered during training, potentially reducing the transferability of the learned selection policy.

AuthorsOpenReview ↗

Author Response Part 4

Q5:For panoramic images as query during inference, please clarify how the patch embeddings applied to ERP inputs are trained when training dataset only supplies only narrow-FOV images, as in the Gibson (f) experiment in Table 3.

We thank the reviewer for pointing out this lack of clarity. The patch embedding used for panoramic inference comes directly from DA3's pretrained backbone and remains frozen throughout our source-domain training. It is not separately trained or adapted using ERP images, nor do we introduce an additional ERP-specific embedding branch.

Importantly, the proposed ERP inference pipeline is not natively provided by DA3. Instead, we reuse its pretrained patch embedding and backbone to develop a token-window processing and circular fusion strategy that operates directly on ERP inputs without perspective reprojection. The comparison results are reported below.

S3D Panorama

MetricDirect ERPToken WindowDifference
1m34.437.2-2.9 pp
0.5m27.928.7-0.8 pp
0.1m2.95.0-2.1 pp
1m30°23.728.7-5.0 pp

ZInD Panorama

MetricDirect ERPToken WindowDifference
1m44.168.6-24.5 pp
0.5m37.264.9-27.7 pp
0.1m7.125.8-18.7 pp
1m30°42.067.3-25.3 pp

Specifically, we first apply the frozen patch embedding to the entire ERP image to obtain a 2D token grid. We then extract overlapping local windows through cyclic indexing in ERP coordinates and organize them into a batch. These windows are processed independently using local positional encodings and the shared frozen ViT, source-domain-trained dense adapter, and selector. Candidate rays are constructed according to ERP spherical geometry. Finally, local predictions are mapped to their corresponding global longitudes and aggregated through circular fusion to produce a 360° horizontal ray profile for localization.

Therefore, while we reuse DA3's pretrained visual representations, the ERP token-window organization, spherical candidate-ray construction, and panorama-aware ray fusion constitute our proposed inference pipeline. In the Gibson(f) experiment, the dense adapter and selector are trained exclusively on source-domain perspective images, and their learned parameters are directly reused for panoramic inference without any additional panoramic training. We will clarify this distinction and the corresponding design in the revised manuscript and appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.