acceptodds
Under review as a conference paper at ICLR 2027

DisCo: Disambiguating Multi-modal Floorplan Localization via SE(2)-Aware Contrastive Learning

Abstract

Visual Floorplan Localization (FLoc) aims to estimate camera poses by aligning visual observations with floorplans, yet it remains challenging due to structural aliasing in repetitive indoor environments. Such ambiguity causes distinct poses to exhibit similar visual-floorplan compatibility, resulting in multi-modal localization distributions with limited spatial separability and angular discriminability. Existing ray matching-based methods can generate geometry-consistent pose candidates, but lack effective mechanisms to disambiguate them. To address this issue, we propose DisCo, a visual-floorplan Contrastive Disambiguation framework for robust visual FLoc. First, we employ a depth-aware ray regression predictor to project monocular RGB observations into 2D ray primitives and generate multi-modal pose candidates through ray-floorplan matching. We further introduce a clustering-based hypotheses preparation strategy to extract representative FLoc hypotheses by exploiting the distribution of candidate poses. Second, we propose an -aware contrastive disambiguation strategy to learn fine-grained visual-floorplan compatibility beyond geometric matching. Through structured pose perturbations, DisCo improves local pose smoothness, spatial separability, and angular discriminability by capturing implicit structural semantics in local floorplan layouts without relaying on explicit semantic annotations. Extensive experiments on three challenging visual FLoc benchmarks demonstrate that DisCo consistently outperforms state-of-the-art methods and significantly improves the robustness and accuracy of visual FLoc.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Localization, Visual Floorplan Localization, Contrastive Disambiguation, Spatial Separability, Angular Discriminability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_M5yNOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DisCo, a framework for localizing a single RGB image within a 2D floorplan under spatial and directional ambiguity. The method first predicts ray distances to generate geometrically plausible camera poses, then applies SE(2)-aware candidate suppression and an image–floorplan compatibility model to rerank them. Main contribution includes pose-structured contrastive learning, which combines perturbed positive poses with position and orientation negatives to distinguish competing hypotheses. Experiments on Structured3D, Gibson, and ZInD report improvements over geometry-based localization and other baselines. Ablation studies examine the contrastive objective, pose perturbations, negative sampling, and candidate selection, with additional evaluations of floorplan corruption, runtime, and sequential tracking.

Critical strengths
  • The paper addresses an important problem in visual floorplan localization. Repeated indoor structures can produce multiple geometrically plausible poses, making reliable localization difficult even when ray predictions are accurate. The motivation for explicitly resolving spatial and directional ambiguity is clear and practically relevant, particularly when only an RGB image and a floorplan are available.
  • The method shows applicability across datasets and candidate generators. Experiments cover Structured3D, Gibson, and the real-world ZInD dataset, with additional evaluations using different candidate generators. These results support the usefulness of DisCo as a module for improving existing localization systems, which strengthens the practical significance of the contribution.
Critical weaknesses
  • The evaluation protocol needs clarification. The provided Gibson configuration uses the test set for validation and checkpoint selection. If this was used for the reported results, it could bias the evaluation and affect comparison fairness. The authors should confirm that model selection used an independent validation set.

  • The comparison with standard contrastive learning is not sufficiently clear. The paper reports substantial improvements over InfoNCE but only states that the candidate generation pipeline is the same. Differences in network architecture, positive and negative sampling, and training budget are not fully explained. The authors should clarify these settings to better establish which specific designs account for the improvement.

Questions for authors
  • Clarification on model selection: Which split was used to select checkpoints for the reported results? The provided Gibson configuration uses the test split for validation. Does this reflect the actual experiments, and are results available with independent validation-based model selection?

  • Could the authors clarify what differs between the InfoNCE baseline (59.9% joint recall) and DisCo without position and orientation negatives (70.0%)? Is this gap due to slack positive sampling, or are there other differences in the network or training settings?

Reviewer_J4AUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the "structural aliasing" problem in visual floorplan localization by proposing an SE(2)-aware contrastive disambiguation framework called DisCo. The method employs a two-stage paradigm: first, it utilizes a Depth-Aware Ray Regression Predictor (RRP) to generate a multi-modal candidate pose distribution (DAFPM), from which representative hypotheses are extracted via a greedy clustering strategy based on an SE(2) distance metric. Subsequently, a Pose-Structured Contrastive (PSC) loss is introduced; by constructing "relaxed" positive samples, as well as position-level and orientation-level negative samples on the SE(2) manifold, the framework learns fine-grained compatibility scores between images and local floorplan crops. Finally, a Disambiguation Probability Map (DPM) is fused with geometric likelihoods to derive the optimal pose.

The core contribution of this work lies in demonstrating that contrastive learning infused with geometric priors can effectively capture implicit structural semantics. This module can serve as a plug-in capable of enhancing the performance of existing candidate generation methods. The proposed method improves local pose smoothness, spatial separability, and angular discriminability.

Critical strengths
  1. This method decouples "candidate generation" from "candidate disambiguation" within the existing ray-matching paradigm. DisCo not only significantly boosts the performance of purely geometric methods (e.g., UnLoc and RRP) but can also be integrated with methods that rely on explicit semantics (e.g., SemRayLoc).

  2. By utilizing a cross-attention mechanism between images and local floor plan crops, the method implicitly captures the structural semantics of local layouts. This lowers the barrier for data annotation and enhances generalization across different scenes.

  3. The method can be seamlessly integrated into both purely geometric approaches (e.g., UnLoc, RRP) and semantically enhanced approaches (e.g., SemRayLoc), yielding performance gains across various synthetic and real-world benchmarks.

Critical weaknesses
  1. The method incurs a high computational cost. As shown in Table 12, the inference time per frame increases from 11.56 ms to 45.61 ms. In Stage 2, cross-attention computations must be performed individually for a large number of hypotheses, making it difficult to meet the strict real-time requirements on mobile devices or robotic platforms with limited computational resources.

  2. The evaluation scenarios are relatively simple and uncluttered, lacking interference from miscellaneous objects or dynamic targets; furthermore, there is no discussion regarding the method's generalizability to more complex environments.

Questions for authors
  1. In the cross-attention computation of Equation 5, are the image tokens FjF_j shared across all hypotheses? If so, given an average of 52.9 hypotheses, which specific operation accounts for the majority of the additional 40ms latency in Stage 2?

  2. Table 6 in Appendix B shows that replacing the backbone from ResNet-50 to DINOv2 yields a significant performance gain. In the main experiments presented in Tables 1 and 2, were all baseline methods (e.g., 3DP, RSK, UnLoc) retrained using the exact same DINOv2-S backbone? If not, how do the authors rule out the possibility that a portion of the performance gains attributed to DisCo is actually a confounding effect stemming from the use of more powerful pre-trained features?

Reviewer_8vKNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses structural aliasing in single-image localization within a 2D floorplan. DisCo generates geometric pose hypotheses through depth-aware ray prediction and floorplan matching, consolidates nearby hypotheses using an SE(2)-based criterion, and reranks them through an image–floorplan compatibility network. Its pose-structured contrastive training combines locally perturbed positives with position-level and orientation-level negatives. Learned compatibility scores are fused with geometric likelihoods to select the final pose. Experiments on Structured3D, Gibson, and ZInD report improved localization recall, supported by ablations on sample construction, candidate selection, and robustness to floorplan corruption.

Critical strengths
  1. DisCo addresses the important problem of disambiguating geometrically plausible poses in visual floorplan localization. With the upstream model and candidate poses held fixed, adding DisCo improves joint recall on S3D from 39.3% to 74.3%, providing meaningful evidence for the effectiveness of learned reranking.
  2. DisCo improves both geometry-based and semantic localization pipelines. Adding it to SemRayLoc increases joint recall on the real-world ZInD dataset from 29.2% to 54.0%. Together with the F3Loc-style controls, these results support its applicability across different backbones and candidate generators.
Critical weaknesses
  1. The released Gibson training setup uses the test split for validation and checkpoint selection. If this setup was used for the reported experiments, the results could be biased by test-dependent model selection. The actual evaluation protocol therefore requires clarification to establish the reliability of the comparisons.
  2. The supplied implementation differs from the described SE(2) inference procedure: it retains only one heading per location before reranking and fuses raw compatibility scores rather than the normalized probabilities specified in Eq. (9). These differences can affect angular hypothesis coverage and final rankings, leaving unclear which formulation underlies the reported results.
Questions for authors
  1. Which data splits and model-selection criteria were used for the reported Gibson results? Did test metrics influence training, checkpoint selection, or hyperparameter tuning? If so, please provide results with model selection based on an independent validation split.
  2. Were candidates extracted from the full position–orientation volume or after retaining one heading per location? Did fusion use raw scores or softmax probabilities? Please clarify the settings used for the reported results and provide joint oracle recall at (1,\mathrm{m},30^\circ) before and after candidate reduction.
  3. What accounts for the difference between “Ours (InfoNCE)” in Table 2, with 59.9% joint recall, and the variant without position-level or orientation-level negatives in Table 4(a), with 70.0%? Please clarify which sampling choices differ and whether the architecture, training budget, and checkpoint-selection protocol were matched.
2 comments

Dear Reviewer 8vKN,

Thank you for your detailed feedback. We would appreciate it if you could take another look at our responses and let us know whether your concerns have been adequately addressed.

Thank you for your time and consideration.

Best regards,

The Authors

Dear Reviewer 8vKN,

Thank you again for your helpful comments on our paper. We have addressed your questions in our rebuttal and included several additional experiments. With around 2 hours left in the discussion period, we would appreciate it if you could take another look and let us know if you have any further concerns.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Consolidated Summary of Rebuttal Clarifications and Additional Evidence

Dear Area Chair and Reviewers,

We would like to provide a consolidated summary of our rebuttal responses and additional experiments, highlighting how we have addressed the reviewers' main concerns.

  1. Experimental reliability and evaluation protocol. We acknowledged the incorrect checkpoint-selection configuration in the original Gibson experiments, where the test split was inadvertently used for checkpoint selection. We reran the experiments using independent validation-based checkpoint selection. The corrected results remain closely consistent with the original findings: Gibson(f) achieves 57.25% R@1 m and 56.00% joint recall, compared with 56.70% and 55.40% originally; Gibson(g) achieves 55.11% and 53.51%, compared with 55.60% and 54.00%. These results support the stability of our conclusions, and the corrected results have been incorporated into the revised manuscript.

  2. Implementation consistency and methodological clarification. We clarified that the originally submitted supplementary code came from an earlier experimental branch and have updated the anonymous repository with the implementation corresponding to the reported results. We also clarified the SE(2)-aware candidate extraction and score fusion procedures, together with additional oracle recall analysis showing that hypothesis consolidation substantially reduces redundant candidates while preserving nearly all oracle coverage. Regarding the InfoNCE comparison, we explained the difference between exact-pose positives and slack positives in the respective experiments, and provided matched training configurations to isolate the contribution of pose-structured sampling.

  3. Controlled experiments and backbone fairness. We clarified that external baselines retain their method-specific pretrained encoders rather than being uniformly retrained with DINOv2-S. More importantly, controlled comparisons demonstrate that DisCo's gains are not merely attributable to stronger visual features. With the same DINOv2-based RRP encoder, checkpoint, and candidate poses, DisCo improves S3D joint recall from 39.30% to 74.30%. With a ResNet-50-based F3Loc-style candidate generator, it also improves joint recall from 30.49% to 62.83%. These results support the effectiveness and general applicability of the proposed disambiguation mechanism.

  4. Further evidence on computational efficiency and generalization. We provided detailed latency profiling, showing that cached image tokens are shared across hypotheses and that batched cross-attention takes only 1.02 ms in total. By reducing the candidate budget to 16, DisCo achieves 31.20 ms latency (approximately 32.1 Hz) while maintaining 71.23% R@1 m, substantially outperforming RRP and UnLoc with modest additional latency. It also offers considerably better efficiency and localization accuracy than SemRayLoc without requiring additional semantic annotations. For generalization, beyond our existing floorplan corruption and image masking experiments, we introduced realistic pedestrian occlusions on S3D. DisCo consistently achieves over 30 percentage points of improvement in joint recall under 10%, 20%, and 30% pedestrian occlusions. The corresponding quantitative results and visualizations have been added to Appendix H. We also explicitly clarify that these experiments evaluate frame-level occlusion robustness rather than full temporal dynamic-scene understanding.

We note that several concerns raised across the reviews are closely related, particularly regarding evaluation reliability, implementation consistency, computational efficiency, and generalization. We have made substantial efforts to address these shared concerns through corrected experiments, controlled comparisons, and additional analyses. We sincerely hope that the reviewers and Area Chair will consider the complete body of evidence provided during the rebuttal. We would also welcome further discussion on these overlapping concerns to facilitate a more comprehensive assessment of the contributions, empirical findings, and limitations of DisCo.

Thank you very much for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.