acceptodds
Under review as a conference paper at ICLR 2027

MultiLoc: Look Around As You Localize for Fast And Robust Visual Re-localization

Abstract

Relative camera pose is a basic geometric cue for visual re-localization and scene understanding. When using images alone, visual evidence may not sufficiently support reliable estimation of 3D-consistent motion. The challenge is especially acute in unfamiliar and ever-changing environments for real-time applications, where a system must infer spatial structure under tight time constraints from what it sees rather than rely on scene-specific reconstruction or training. This raises a fundamental question: how can live 3D camera motion be estimated accurately for re-localization in unseen environments while drawing on enough scene context to resolve pose ambiguity? We introduce MultiLoc, a multi-view-guided relative pose regressor trained at scale to achieve spatial and geometric consistent representations for robust visual re-localization in unseen scenarios with high inference speed. Specifically, MultiLoc creates a minimal 3D-sub-scene representation of the environment and efficiently transforms it into 3D spatially and geometrically consistent features in a single forward pass, enabling precise pose estimates with high inference speed gains. Across diverse indoor, outdoor, and in-the-wild visual re-localization benchmarksIndoor6, Cambridge Landmarks and WaySpotsMultiLoc consistently outperforms state-of-the-art relative pose regression methods. We also highlight that MultiLoc, a pose regressor, also performs competitively with inference-heavy and structure-based approaches while retaining sub-second inference and generalizing to unseen scenes. Given a small posed support set, it also surpasses relative pose regression, feature-matching, and non-regression methods on several relative camera pose benchmarks. Code will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Visual relocalization, Pose Regression, Multi-view, Relative Camera Pose Estimation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DQdYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents incremental work building on VGGT and Reloc3r. Given multi-view images with known poses, the goal is to estimate the pose of a query image. The authors propose injecting the known camera poses into the camera tokens of VGGT and reducing the VGGT output head to predict only the camera parameters. The model is initialized with VGGT pretrained weights and fine-tuned on mixed datasets similar to those used by Reloc3r. Experiments show degraded performance on image pairs but improved performance with multi-view inputs and ground-truth pose inputs.

Critical strengths
  1. The idea of injecting known poses into VGGT to improve accuracy is valid.
  2. The results of visual localization in indoor and outdoor scenes are superior to those of several baselines.
Critical weaknesses
  1. Limited novelty and insufficient method tailoring.

This work is largely incremental, and the method section essentially builds on / reviews VGGT and Reloc3r. It would be helpful to emphasize the unique improvements over VGGT, Reloc3r, and OmniVGGT [A]. The design is not carefully tailored to the problem, e.g., it predicts intrinsics (focal lengths) but does not use them, and it injects only pose but not intrinsics. The intrinsics indeed have an impact on camera pose estimation, but this is not analyzed in the paper.

  1. More importantly, the paper suffers from overclaimed and incomplete experimental evaluation.

(a) The claim that the method outperforms both non-pose-regression and pose-regression approaches is not supported. In the pairwise setting, it is noticeably worse than prior methods such as Reloc3r and VGGT (Table 1). This is surprising and suggests that the proposed pose injection may even hurt VGGT. Moreover, the comparison is unfair: the multi-view series uses 10 additional images and their ground-truth poses as input.

(b) The claim of performance comparable to non-regression methods is not convincing. Non-regression methods typically achieve 8–14 cm error, whereas the proposed method has about 29 cm error (Table 2).

(c) Comparisons with state-of-the-art methods are missing. For example, the proposed method can be regarded as a simplified version of OmniVGGT [A], but this is neither discussed nor compared. For non-regression methods, comparisons with concurrent works such as RoMa v2 [B] and LoMa [C] are missing. Since the paper addresses multi-view pose estimation, multi-view feature matching methods such as CoMatcher [D] should also be considered and compared.

(d) Compared with pairwise methods (e.g., DUSt3R and Reloc3r), the proposed method relies on global attention over multi-view images, which incurs O(N^2) complexity and GPU memory usage with respect to the number of images. However, GPU usage is neither reported nor compared.

[A] OmniVGGT: Omni-Modality Driven Visual Geometry Grounded Transformer. CVPR 2026.

[B] RoMa v2: Harder Better Faster Denser Feature Matching. ECCV 2026.

[C] Loma: Local Feature Matching Revisited. ECCV 2026.

[D] CoMatcher: Multi-View Collaborative Feature Matching. CVPR 2025.

Questions for authors
  1. It is surprising that injecting camera pose hurts performance in the pairwise setting. Why?

  2. Can the authors verify the claimed advantage over OmniVGGT?

Minor points and suggestions

  • L014: The motivation regarding “unfamiliar and ever-changing environments” is not actually studied in this work.

  • The term “unseen scene” appears to be used to mean “no training.” However, visual re-localization is not an unseen scene problem: it requires a database of images and pre-extracted features (e.g., NetVLAD), and the query image is captured during a revisit. Therefore, “unseen” is inaccurate in the problem definition.

  • L117: It is unclear why the method is described as a “two-stage re-localization framework.” Is image retrieval considered the first stage before VGGT? Or is scale recovery regarded as a second stage after VGGT?

  • L231: What is the “L-th transformer block”? The last layer's output?

  • Fig. 1: The features “No per-scene training” and “Zero-shot generalization” appear to refer to the same feature. The training is actually mixed-scene training.

  • Fig. 2 is somewhat misleading: why does scale recovery use an additional camera view, as shown in the figure? Please clarify.

1 comment
AuthorsOpenReview ↗

Rebuttal response for reviewer DQdY

We thank reviewer DQdY for their thoughtful feedback and for noting that our visual localization results in indoor and outdoor scenes are superior to several baselines. We address each of their questions below.

Q1. Why does injecting camera pose hurt performance in the pairwise setting?

In the pairwise setting, no informative pose is injected. With a single reference image, the reference pose defines the coordinate frame and is therefore always the identity, while the query pose is predicted by the model. The model thus receives no pose information and operates analogously to Reloc3r or VGGT in this setting. Consistent with this, MultiLoc performs on par with or better than VGGT on ACID and MegaDepth, and trails it only on ScanNet-1500 under the pairwise setting. We attribute this gap to the training data, as VGGT is trained on ScanNet, so ScanNet-1500, although held out, is in-domain for VGGT, whereas MultiLoc is fine-tuned on ScanNet++, which shifts it away from the ScanNet domain.

Furthermore, we include the pairwise results in Table 1 for completeness. The main contribution of the paper is the multi-view pose regressor, where pose injection is informative. The main paper's Table 2 and Figure 5 show that accuracy improves as more posed reference images are provided compared with running pairwise inference. Therefore, we do not believe that the camera pose hurt performance in the pairwise setting, as there is no 3D sub-scene context formed for grounding the query, and thus it should just be treated as a plain image-only variant similar to VGGT for completeness, which is not the claim of our work.

Q2. Can the authors verify the claimed advantage over OmniVGGT?

We evaluated OmniVGGT, a concurrent work, on visual re-localization and camera pose estimation using the same reference-query sets as MultiLoc, with inputs prepared following OmniVGGT's official pipeline (see Table R1 below). MultiLoc outperforms OmniVGGT on all the mentioned benchmarks. We note that OmniVGGT was evaluated in its strongest configuration, with both intrinsics and extrinsics provided, whereas MultiLoc uses extrinsics only, and that OmniVGGT is trained on a substantially larger dataset collection. We attribute such improved performance to our architecture, which is designed specifically for the re-localization task. We will further add this comparison to the revised paper.

Table R1: Comparison with OmniVGGT. Median pose error (cm // ∘^\circ, ↓\downarrow) averaged over all 5 Cambridge scenes and all 10 WaySpots scenes, and AUC@5 // AUC@10 (↑\uparrow) on ACID. Bold denotes the best result.

ModelCambridge (Visual Reloc.)WaySpots (Visual Reloc.)ACID (Camera Pose Est.)
OmniVGGT (K + Ext.)34.2 // 0.4727.0 // 2.1037.6 // 55.4
MultiLoc (Ext.)29.5 // 0.3524.7 // 1.3976.6 // 87.2

Q3. The claim of performance comparable to non-regression methods is not convincing.

We believe the mentioned comment only points to a limited evaluation metric on a single benchmark of the Cambridge dataset. Our results per benchmark are:

  • Indoor6: MultiLoc surpasses all scene coordinate regression baselines, which fall under the non-pose regression family, and improves translation over MASt3R+Kapture and rotation over E5+1 from matching-based methods. Finally, its performance is within 0.6 cm (≤1\leq 1 cm) of the best non-regression baseline.
  • WaySpots: MultiLoc outperforms all matching-based and scene coordinate regression baselines by large margins on the WaySpots benchmark.
  • Cambridge: MultiLoc is within 3 cm of ACE in translation and improves rotation over both ACE and E5+1, though the gap to the best feature-matching methods (8–14 cm, as the reviewer notes) remains. We also highlight that the inference and mapping time of such state-of-the-art methods on Cambridge is much slower than our proposed method, with an additional dependency on 3D reconstruction of the scene, making MultiLoc a reliable alternative in real-world applications.

Because of these observations, we believe our balanced claim of being comparable is justified here.

Reviewer_rWP7OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

MultiLoc jointly processes a query image and multiple posed reference images to directly predict the query pose. Built on VGGT, it injects reference poses into camera tokens to provide spatial context and recover metric scale without a geometric solver. Experiments show improvements over prior relative pose regressors and accuracy–latency trade-off on several re-localization benchmarks.

Critical strengths

Conditioning pose regression on multiple reference images and their known poses is well motivated. The re-localization gains over ReLoc3r are substantial across indoor, outdoor, and in-the-wild scenes.

The reported 0.15-second inference time, together with competitive localization accuracy and no per-scene training, makes the approach practically appealing.

Critical weaknesses

The novelty of this paper is very limited. Its main change is to condition a VGGT-based multi-view transformer on known reference poses through camera-token embeddings. Moreover, this method does not have any significant advantage in accuracy and inference time compared to previous SCR methods.

The multi-view matching protocol in Section 4.1 is unclear. The paper states that all methods are evaluated on the intended image pair, but the matching baselines select a potentially different reference by inlier count. It does not explain how that estimate is converted back to the intended pair. This makes the relative-pose comparison difficult to assess.

The main ablation compares frozen, image-only VGGT with trained, pose-conditioned MultiLoc, changing both training and pose input. It therefore does not isolate the contribution of pose conditioning or repeated pose-token injection from task-specific fine-tuning. Besides, I think this paper overstates the case. Since it mainly uses the training set of VGGT, even though there is no scene-specific training, the pre-training is likely to have learned the benchmarks used for evaluation.

Questions for authors

When a matcher selects a support image instead of the intended target, how is its estimated pose converted to the original evaluation pair?

With the same training data and budget, how does MultiLoc perform without reference-pose input, and with pose tokens injected only once?

VGGT was pretrained on both ScanNet and MegaDepth, so distinguishing MultiLoc’s ScanNet++ training data from ScanNet-1500 is insufficient to establish unseen-scene evaluation. Please clarify whether the evaluation scenes were excluded from VGGT pretraining, as well as from MultiLoc’s subsequent training.

2 comments
AuthorsOpenReview ↗

Rebuttal response for reviewer rWP7

We thank reviewer rWP7 for their thoughtful feedback and positive comments, such as well motivated methodology, substantial improvements over other baselines and practically appealing work. We address the reviewer's concern below:

Q1) When a matcher selects a support image instead of the intended target, how is its estimated pose converted to the original evaluation pair? For matchers like RoMA we do not transform the relative pose into a target coordinate frame or the original evaluation pair. As stated in L316-L318 from the main paper, we match the query against the intended view and every support view, select the pair with the highest inlier count, and compute the error as well as the performance metric between the query and that selected best view.

Q2) With the same training data and budget, how does MultiLoc perform without reference-pose input, and with pose tokens injected only once? To isolate the benefit of pose conditioning, we add a pose-free VGGT fine-tuned with the same schedule as MultiLoc as highlighted by VGGT* in Table R1. It shows little to no improvement over the original VGGT (Table 4 of the main paper), whose training already covers most of our datasets, which confirms that the gains come from pose conditioning. For the variant with poses injected only once at the input layer, we are currently re-training the model with the same training budget as MultiLoc. Given the rebuttal timeframe,our resource constraints and longer training schedule of MultiLoc, this ablation of injecting pose only once at input with matching the exact budget cannot be completed in time, and we will add the results in the camera-ready version. In the meantime, we include MapAnything as an additional re-localization baseline. It is an existing model that injects pose information only once, at the input, by adding encoded pose features to the image patch features. We note that it differs from MultiLoc in architecture and training data, so it serves as a reference point rather than a controlled ablation. MultiLoc achieves the best results on both Cambridge and WaySpots against all the newly added baselines and demonstrates the effectiveness of our proposed pose conditioning mechanism, see Table R1.

Table R1: Visual re-localization. Median pose error (cm / ∘^\circ) ↓\downarrow averaged over all 5 Cambridge scenes and all 10 WaySpots scenes. Bold denotes the best result. VGGT* denotes fine-tuned version.

ModelWaySpotsCambridge
VGGT*63.4 / 1.8943.2 / 0.35
MapAnything33.4 / 2.1845.6 / 0.62
MultiLoc24.7 / 1.3929.5 / 0.35

Q3) VGGT was pretrained on both ScanNet and MegaDepth, so distinguishing MultiLoc’s ScanNet++ training data from ScanNet-1500 is insufficient to establish unseen-scene evaluation. Please clarify whether the evaluation scenes were excluded from VGGT pretraining, as well as from MultiLoc’s subsequent training. We thank the reviewer for raising this point. Our original justification covered only MultiLoc's fine-tuning data and not the VGGT initialization. We clarify that here.

The VGGT authors state in their supplementary material that "to ensure a fair comparison on ScanNet-1500, we exclude the corresponding ScanNet scenes from our training". The VGGT weights we initialize from were therefore not trained on the ScanNet-1500 scenes. And for MutliLoc training, we do not use ScanNet at all. Our indoor training data is ScanNet++, an independently captured dataset that shares no images or sequences with ScanNet. The ScanNet-1500 scenes are thus excluded at both stages. Similar also holds true for the MegaDepth and MegaDepth-1500 benchmark where scenes 0015 and 0022 are held out during training of VGGT as mentioned in VGGT's paper as "It is important to note that DUSt3R and MASt3R utilized a substantial portion of the MegaDepth dataset for training, only excluding scenes 0015 and 0022 .... . For an apples-to-apples comparison, we adopt the same training split as DUSt3R and MASt3R." We will add this two-stage clarification to the revised paper.

AuthorsOpenReview ↗

Clarifying novelty of MultiLoc as noted by reviewer rWP7

We thank reviewer rWP7 for their comments about the novelty aspect of our work in critical weaknesses section. However, we respectfully disagree with them and clarify that here.

Our novel contribution is the reformulation of relative pose regression for visual re-localization and designing a task specific architecture that for the first time bridges the gap between pose regression and non pose regression methods while substantially pushing the accuracy metrics for relative pose regression. For this, we design a pose conditioning architecture that effectively grounds the query image in the 3D sub-scene context and whose limitations are also studied in our work, in the Appendix section B.5 and the detailed analysis as provided with reviewer's 9i2r's Table R2. Finally, we also study the contribution and effectiveness of our pose conditioning mechanism as requested by the reviewer in Question 2 or Table R1. Moreover, we highlight that MultiLoc does not require any post optimization method to recover the scale and we perform analysis of such an analytically driven recovery by our proposed scale factor in reviewer's 9i2r's Table R2 and R3 response. Our task specific novel architecture beats the performance metrics of related pose conditioning concurrent works such as MapAnything and OmniVGGT (reviewer DQdY Table R1 for OmniVGGT).

Finally, about the reviewer's comment about significant advantage over the SCR methods, we highlight that our method requires no per-scene training time and is not dependent on PnP or geometric solver as opposed to SCR methods. State-of-art SCR methods like GLACE require around 9 minutes of per-scene training time on our compute budget in which time we can re-localize more than 3000 queries with MultiLoc. Moreover, about the accuracy metrics we highlight that on two out of three benchmarks we surpass SOTA SCR methods especially on WaySpots where GLACE achieves translation/rotation error of 128.7 cm / 7.56 deg whereas our method improves it to be around 24.7 cm / 1.58 deg (reduction of about 80 % for both translation and rotation errors). It is only on the Cambridge, and that too majorly on the translation component, that we perform on par with ACE whereas GLACE is more accurate. On inference time, we agree with the reviewer that SCR is still faster per query by 50 ms. We compile the table for side-by-side comparison here.

MethodPer-scene trainingLatency / queryIndoor6 (cm / °)WaySpots (cm / °)Cambridge (m / °)
ACE> 2 min~0.10 s11.0 / 1.80134.0 / 10.550.26 / 0.42
GLACE> 9 min~0.10 s3.5 / 0.60128.7 / 7.560.14 / 0.26
MultiLoc (k = 10)none (< 1 min retrieval index)0.15 s2.9 / 0.4324.7 / 1.580.29 / 0.35

Therefore, we believe that MultiLoc is novel in the proposed formulation with beneficial architectural design while substantially improving performance compared with RPR and bridging the long standing performance gaps with non-pose regression methods.

Reviewer_9i2rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MultiLoc, a zero-shot visual re-localization framework that regresses the 6-DoF query camera pose given a query image and a set of reference images with known poses. Building upon a VGGT-style multi-view transformer, MultiLoc encodes the known reference poses into camera tokens, re-injects these geometric embeddings across attention layers, and directly regresses the query camera pose. Metric translation scale is recovered analytically from the relative reference camera geometry, circumventing test-time geometric solvers (e.g., PnP/RANSAC). The model is fine-tuned on multiple datasets and retrieves reference views via NetVLAD at inference.The method is evaluated on relative pose estimation (ACID, MegaDepth-1500, ScanNet-1500) and visual re-localization benchmarks (Indoor6, Cambridge Landmarks, WaySpots). MultiLoc demonstrates marked gains over earlier relative pose regressors with a low reported model latency (~0.15s per query).

Critical strengths
  1. Significant Empirical Gains over Prior Regressors: MultiLoc achieves substantial improvements over existing relative pose regressors (e.g., ReLoc3r) across both indoor and outdoor benchmarks. For instance, it reduces median errors by more than half on Indoor6 and Cambridge, demonstrating the practical effectiveness of the full pipeline.
  2. Practical, Solver-Free Task Formulation: Formulating visual re-localization as joint multi-view feed-forward regression directly aligns with the data available at test time. By eliminating test-time PnP/RANSAC, scene-specific optimization, and dense correspondence search, the system delivers an attractive accuracy–latency trade-off (~0.15s per query).
Critical weaknesses
  1. Confounded Ablations and Attribution of Gains:
  • Pose Conditioning vs. Task Fine-Tuning: In Table 4, the ablation compares a frozen, pose-free VGGT baseline (using motion averaging) against the fine-tuned, pose-conditioned MultiLoc. Since the training regimen, pose inputs, and scale-recovery procedures differ simultaneously, this comparison cannot isolate the contribution of pose conditioning. A matched baseline—an equally fine-tuned, pose-free VGGT—is necessary to establish the independent benefit of pose injection.
  • Missing Direct Baseline in Re-Localization: MapAnything supports posed multi-view inputs and is evaluated in the relative-pose benchmarks (Table 1). However, it is absent from the main re-localization experiments (Figure 3, Tables 2–3). To substantiate the claimed advantages in re-localization, MapAnything should be evaluated under the same 10-reference retrieval setting with a clearly specified pose and scale recovery protocol.
  1. Fairness of Multi-View Baselines and Evaluation Calibration:
  • Weak Multi-View Matcher Baseline: Extending pairwise matchers (RoMa, MASt3R) via a simple max-inlier heuristic does not constitute a true multi-view baseline. As shown in Appendix Figure 7, this heuristic can fail to select the optimal pair. The current experiment shows the limits of a naive pair selector, but falls short of demonstrating an inherent advantage over joint multi-view correspondence solvers.
  • Scope of Comparative Claims: While MultiLoc significantly improves upon relative pose regressors, its absolute accuracy still trails established structure-based/matching pipelines on benchmarks like Cambridge. Claims regarding holistic visual re-localization performance should be carefully calibrated to reflect this distinction.
Questions for authors
  1. Controlled Ablations:
  • Could the authors provide results for a matched ablation: a VGGT-based multi-view model fine-tuned on the identical dataset and schedule, but without camera pose conditioning?
  • How much performance is gained by layer-wise pose re-injection compared to simply concatenating pose tokens at the input layer?
  • Re-localization results for MapAnything on Indoor6 and Cambridge using the identical 10-view reference set?
  1. Multi-View Setup Details:
  • On ACID, MegaDepth, and ScanNet, how are the 10 support views sampled (e.g., temporal baseline, minimum visual overlap)? Is the evaluation sensitive to support sampling variance?
  • When a matching baseline selects an alternative support view, how is the estimated relative pose transformed into the target coordinate frame?
  1. Scale Recovery Robustness:
  • How robust is the analytic scale recovery to degenerate reference distributions and realistic noise in reference poses?
2 comments
AuthorsOpenReview ↗

Rebuttal response for reviewer 9i2r

We thank reviewer 9i2r for their thoughtful feedback and positive comments and address their questions below:

Q1) Controlled Ablations: To isolate the benefit of pose conditioning, we add a pose-free VGGT fine-tuned with the same schedule as MultiLoc as highlighted by VGGT* in Tab. R1. It shows little to no improvement over the original VGGT (Tab 4 in main paper), whose training already covers most of our datasets, which confirms that the gains come from our pose conditioning. For one-time pose injection variant, we are currently re-training it to match MultiLoc’s training budget. Due to the rebuttal timeframe, resource limits, and long train schedules, this ablation cannot be completed in time but we will include these results in camera-ready version. In the meantime, we include MapAnything baseline (which was also mentioned by reviewer in part c) that injects pose information only once at input. We note that it differs from MultiLoc in architecture and training data, and thus, is a reference point rather than controlled ablation. MultiLoc achieves best results on both Cambridge and WaySpots benchmarks (Tab. R1).

Table R1: Visual re-localization with median pose error (cm / deg).

ModelWaySpotsCambridge
VGGT*63.4 / 1.8943.2 / 0.35
MapAnything33.4 / 2.1845.6 / 0.62
MultiLoc24.7 / 1.3929.5 / 0.35

Q2) Multi-view setup: For ACID, MegaDepth-1500 and ScanNet-1500 we keep the original query-reference pairs of the benchmarks. Support views are sampled randomly within a temporal window for sequential datasets such as ACID; for non-sequential datasets such as MegaDepth-1500 we retrieve a larger pool of top-50 candidates with image retrieval and randomly pick 10 of them. We will release the support sets.

Regarding part (b) for pairwise matchers we do not transform the relative pose into a target coordinate frame. As stated in L316-L318 from the main paper, we match query against the intended view and every support view, select the pair with highest inlier count, and compute the error as well as the performance metric between query and that selected best view.

Q3-a) Robustness of analytic scale recovery to degenerate reference distributions. For such analysis, we group all 343 KingsCollege test queries from Cambridge benchmark by the geometry of their reference cameras as mentioned in Table R2 according to: (i) the absolute spread, i.e. the scale factor ss; (ii) the relative distance dq/sd_q / s, where dqd_q is the distance from the query to the anchor (first reference); and (iii) collinearity, measured by the linearity of the reference camera positions (largest eigenvalue of their covariance over the sum of eigenvalues; collinear if ≥0.9\ge 0.9). Another degenerate case is when the references are collinear and the query may or may not lie on the same line. Therefore, for collinear references we further split by a query angle between the principal axis of the reference positions and the direction from the centroid of spread to the query (on the line if ≤15∘\le 15^\circ). dqd_q and the query angle use the ground-truth query pose for analysis only.

Table R2: Analysis of scale factor ss against reference poses' distribution.

Ref. geometryQueriesTran. err. (m)Rot. err. (deg)
All queries3430.320.32
Spread s≤2s \le 2 m250.310.34
Spread 2<s≤52 < s \le 5 m2200.290.32
Spread s>5s > 5 m980.400.36
Non-collinear references (linearity <0.9< 0.9)2020.300.31
Collinear references (linearity ≥0.9\ge 0.9)1410.350.38
\quad query on the reference line (angle ≤15∘\le 15^\circ)330.290.33
\quad query off the reference line (angle >15∘> 15^\circ)1080.400.39
Query much closer than the spread (dq/s<0.5d_q / s < 0.5)870.300.30
Query comparable to the spread (0.5≤dq/s<20.5 \le d_q / s < 2)2370.320.32
Query much farther than the spread (dq/s≥2d_q / s \ge 2)190.560.45

Discussion. Within the range of reference distributions that retrieval produces, accuracy is unaffected by small absolute spread, decreases moderately with collinearity (0.35 m vs 0.30 m), and drops only when the query lies far outside the reference spread (dq/s≥2d_q / s \ge 2). With collinear references, a query on the same line is localized better than one off the line (0.29 m vs. 0.40 m). We attribute this to collinear references provide a metric baseline only along their axis, and an off-line query is also likely to share less visible content with them (co-visibility limitation is addressed in Appendix B.5). Finally, the scale factor is uninformative in a trivial case when all references coincide (s→0s \to 0). In conclusion the analytical scale recovery is robust to when (i) the query lies within or close to the reference spread, (ii) the references are non-collinear and (iii) the references and query are collinear.

AuthorsOpenReview ↗

Rebuttal response for reviewer 9i2r's Question 3 part b (robustness to noise)

Q3-b) Robustness to noise in the reference poses. For such an analysis, we perturb the poses of all 10 retrieved references before they are used anywhere in the pipeline: each camera center by zero-mean Gaussian noise with standard deviation σt\sigma_t per axis for translation, and each orientation by a rotation about a uniformly random axis whose angle is drawn from a zero-mean Gaussian with standard deviation σR\sigma_R. The prediction is compared with the clean ground-truth query pose. We use 100 random KingsCollege queries and 5 noise draws per query and setting. Errors are averaged over the draws and we report the median over the queries in Table R3.

Table R3: Median pose error under noise in the reference poses from KingsCollege. Top segment of table denotes the translation error in meters (m) and the bottom segment denotes the rotation error in degrees.

Translation error (m)

σR\σt\sigma_R \backslash \sigma_t (m)00.010.050.10
0∘0^\circ0.300.300.310.30
0.5∘0.5^\circ0.300.300.310.30
1∘1^\circ0.300.300.310.31
2∘2^\circ0.330.330.330.34
5∘5^\circ0.390.410.400.43

Rotation error (∘^\circ)

σR\σt\sigma_R \backslash \sigma_t (m)00.010.050.10
0∘0^\circ0.330.330.330.33
0.5∘0.5^\circ0.440.410.450.42
1∘1^\circ0.640.580.600.64
2∘2^\circ1.061.061.080.99
5∘5^\circ2.172.122.112.12

Discussion. Table R3 shows that translation noise alone has almost no effect: up to σt=0.10\sigma_t = 0.10 m the translation error changes by at most 0.01 m and the rotation error stays at 0.33∘0.33^\circ. The scale factor depends only on the reference camera centers and, as shown, recovery by such an analytical measure is robust to position noise of up to 0.10 m and unaffected by rotation noise by construction.

On the other hand, rotation noise degrades accuracy, but only gradually, and this degradation stems from the anchor pose and the pose input of the network rather than from the scale factor. For σR≤1∘\sigma_R \le 1^\circ the translation error is unchanged (0.30 m) and the rotation error rises from 0.33∘0.33^\circ to 0.64∘0.64^\circ. At standalone σR=5∘\sigma_R = 5^\circ the errors reach 0.39 m and 2.17∘2.17^\circ, a rotation increase of well under half the injected noise. Under the strongest combined setting (σR=5∘\sigma_R = 5^\circ, σt=0.10\sigma_t = 0.10 m) the error rises from 0.30 m / 0.33∘0.33^\circ to 0.43 m / 2.12∘2.12^\circ, a translation error increase comparable to the injected position noise. In conclusion, MultiLoc remains robust to moderate noise in the reference poses (up to 0.10 m and 1∘1^\circ) with essentially unchanged translation accuracy and a rotation error below 0.65∘0.65^\circ, and its accuracy under larger noise is limited by the orientation accuracy of the reference poses rather than by analytic scale recovery.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.