acceptodds
Withdrawn from ICLR 2027

SceneMoE-VPR: Scene-Adaptive Mixture-of-Experts for Visual Place Recognition across Heterogeneous Scenes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual Place Recognition (VPR) aims to retrieve reference images depicting the same location as a query image. Most existing methods are trained for specific scenes and remain limited in their ability to generalize across heterogeneous scenes. Training VPR models on diverse datasets is a natural route toward robust recognition across heterogeneous environments. However, we observe that extending joint training with indoor data improves indoor recognition but degrades several outdoor benchmarks under fully shared adaptation, revealing a cross-environment trade-off. To address this issue, we propose SceneMoE-VPR, a heterogeneous Mixture-of-Experts framework for input-dependent adaptation. It keeps the DINOv2 backbone frozen and adapts its features through an always-active shared expert and sparsely routed heterogeneous experts designed to model viewpoint-induced geometric variation, local structure, directional layout, and multi-scale context. An image-level router dynamically selects experts for each input without scene labels, enabling input-dependent adaptation across heterogeneous environments. Under the same four-dataset training setting, SceneMoE-VPR consistently outperforms the fully shared baseline on all four benchmarks, improving average R@1 from 88.9% to 89.5%. It further achieves the highest average R@1 among the compared single-image methods with a compact 2,048-dimensional descriptor. These results demonstrate the effectiveness of conditional heterogeneous adaptation for multi-environment VPR across diverse environments. Codes will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Place Recognition, Mixture-of-Experts

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_U21cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies joint training for visual place recognition (VPR) across heterogeneous environments. The authors observe that incorporating indoor training data improves indoor recognition but can degrade performance on some outdoor benchmarks. To address this issue, they propose SceneMoE-VPR, which replaces the shared adapter in a DINOv2-based side adaptation network with a heterogeneous mixture of experts. The module combines an always-active shared expert with four spatial experts selected through image-level Top-K routing. Experiments with three outdoor training datasets and one indoor dataset show consistent improvements over SelaVPR++ on four main benchmarks, increasing average R@1 from 88.9% to 89.5%.

Critical strengths
  1. The paper addresses a practically relevant problem. Existing VPR models are often trained for particular environments, while real-world applications require reliable recognition across diverse scenes. The observed performance trade-off when adding indoor training data provides a concrete motivation for studying cross-environment adaptation.

  2. The heterogeneous expert design is reasonably motivated by the spatial characteristics of VPR. Different operators are used to capture geometric variation, local structure, directional layout, and multi-scale context, while image-level routing allows different expert combinations for different inputs. Adapting a frozen pretrained backbone is also a practical design choice.

  3. The experimental evaluation includes useful controlled comparisons and ablations. In particular, retraining SelaVPR++ under the same four-dataset setting makes the main comparison more meaningful. The paper also investigates individual experts, the shared branch, router inputs, and load balancing, with additional benchmarks reported in the appendix. These results provide useful empirical evidence for the proposed approach.

Critical weaknesses
  1. The method appears to be a fairly incremental extension of SelaVPR++, with most components building on existing designs. While the heterogeneous MoE improves performance, the current ablations do not isolate the effects of expert heterogeneity, dynamic routing, and increased model capacity. Comparisons with homogeneous MoE and parameter-matched shared adapters are needed to establish the necessity of the proposed design.

  2. The motivation regarding cross-environment interference is reasonable, but the empirical evidence is not particularly strong. Improvements over the shared adapter are modest, and performance on Nordland remains below the outdoor-only baseline. It is therefore unclear whether the proposed MoE genuinely alleviates cross-environment interference or simply provides a small gain through additional capacity.

  3. The comparison with closely related methods is incomplete. In particular, QAA [1] and MegaLoc [2] also address multi-dataset training or general-purpose place retrieval, making them highly relevant baselines. The current comparisons involve different training configurations and do not clearly establish the advantage of heterogeneous MoE over existing approaches to generalizable VPR.

  4. I am not fully convinced by the evidence for scene-adaptive specialization. Figure 4 shows routing patterns across datasets, but some layers exhibit similar selections, leaving it unclear whether routing meaningfully depends on individual images. Likewise, removing an expert and observing a performance drop does not establish that it has learned the specific geometric, directional, or multi-scale properties claimed by the authors.

[1] Query-based adaptive aggregation for multi-dataset joint training toward universal visual place recognition. ICRA.

[2] MegaLoc: One Retrieval to Place Them All. CVPR Workshops.

Questions for authors
  1. The current ablations do not quite tell us whether the gains come from heterogeneous experts or simply from having more capacity. What happens if all experts use the same architecture, or if the routing is fixed? A comparison under similar parameter and computational budgets would make this much clearer.

  2. I wonder whether the performance drop after adding ScanNet is necessarily caused by shared adaptation. Have you tried adjusting the sampling ratio or loss weight of the indoor data? It would be useful to know whether a simpler change to the training setup could alleviate the same issue.

  3. QAA and MegaLoc seem particularly relevant to the problem studied here, yet neither appears in the main comparison. Is there a reason for leaving them out? Even if a fully controlled comparison is difficult, some discussion of their relative performance and training settings would help.

  4. I am not sure how to interpret the routing patterns in Figure 4. Are these the most frequently selected experts across each dataset? More importantly, how much does the routing actually vary between images, especially different views of the same place? This seems important for understanding whether the router has learned meaningful input-dependent behavior.

  5. One implementation detail is still unclear to me: how exactly are the shared and routed expert outputs combined, and what happens to the CLS token? I also could not find a clear comparison of parameter counts, FLOPs, or inference speed with SelaVPR++. These details would help assess the cost of the proposed changes.

Reviewer_9GCCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a parameter-efficient heterogeneous Mixture-of-Experts framework that combines shared adaptation for common VPR knowledge with sparsely routed heterogeneous experts for input-specific adaptation.

Critical strengths

Adapts a classical solution to solve a more specific, new problem. System is practical and cheap.

Critical weaknesses

The presentation and soundness are good, but the contribution is low in novelty and performance improvement. The authors do not compare their method against the two most comparable prior works that are cited in this paper. They also show very limited ablation studies, so there is no evidence that the performance improvement is actually due to the proposed method, instead of the increased adapter capacity. Improvement on SOTA is small. Missing implementation details (is parameter count the same across SceneMoE-VPR and the baselines?).

Questions for authors
  1. Seeds. What are mean +/- std over at least 3 seeds for SelaVPR++ (3-dataset and 4-dataset) and for SceneMoE-VPR? Does the 0.6 average gain survive?
  2. Capacity matching. What are the trainable parameters, FLOPs and inference latency for SceneMoE versus SelaVPR++? Can you report SelaVPR++ with a widened adapter matched to your parameter count?
  3. Routing controls. Can you report results with all experts active (k=4), with fixed routing (each layer's most frequent pair for every image), and with random top-2 routing?
  4. Homogeneity control. What happens with four structurally identical experts at matched parameters? Without this, how do you support the heterogeneity claim?
  5. Routing statistics. What are per-image routing entropy, expert-utilization histograms and within-dataset routing variance at lambda_LB = 0? How is Figure 4 aggregated, and which index is which expert? Please re-plot it as discrete markers.
  6. Validation protocol. Which split was used to select lambda_LB, the router input, k and the epoch? If test benchmarks, can you re-select on a held-out split (e.g., MSLS held-out cities or a ScanNet validation subset)?
  7. Training-budget confound. Does the Nordland drop from adding ScanNet persist when outdoor places per batch and total outdoor iterations are held fixed?
  8. Baseline tuning. How was SelaVPR++ retrained? Was its recipe re-tuned for the four-dataset mixture as much as yours was?
  9. Missing comparisons. Why are SelaVPR++, MegaLoc and QAA absent from Table 3, especially MegaLoc, whose Baidu Mall version you use? How do the reproduced BoQ numbers compare with the published DINOv2-BoQ results?
  10. Indoor generalization. Can you show results on at least one more indoor benchmark (e.g., 17Places, InsideOut) and on an unstructured or aerial domain, to support "heterogeneous scenes"?
  11. Descriptor dimension. How do baselines perform when projected to 2,048-D with the same FC head? Without that, the compactness claim is confounded.
  12. Robotics relevance. What are the on-device latency and memory figures, particularly for DCNv2? Have you run any sequence-based or loop-closure evaluation?
  13. Instability. Why does pre-down routing collapse Nordland by about 10 points, and why does lambda_LB = 0.001 drop Baidu Mall by 3.2 points while 0.005 raises it by 1.4? What does this imply about the stability of the reported configuration?
Reviewer_pg18OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces SceneMoE-VPR, a scene-adaptive mixture-of-experts framework for visual place recognition across heterogeneous environments. Building on SelaVPR++, the method keeps the DINOv2 backbone frozen and replaces conventional shared adapters with a heterogeneous MoE consisting of four spatial experts, an always-active shared expert, and an image-level router. The proposed framework aims to mitigate the performance trade-offs arising from joint training on indoor and outdoor datasets. Experiments demonstrate improvements over a retrained SelaVPR++ baseline, increasing average R@1 from 88.9% to 89.5% across four benchmarks, and achieves the highest average R@1 among the compared single-image methods with a compact 2,048-dimensional descriptor.

Critical strengths
  • The paper addresses a relevant problem of generalization across heterogeneous VPR environments, particularly the trade-off between indoor and outdoor recognition.

  • The proposed framework combines parameter-efficient adaptation with input-dependent expert selection, and the individual experts have intuitive spatial motivations.

  • The experiments include a controlled comparison against SelaVPR++ under the same training distribution, along with several component-level ablations.

Critical weaknesses
  • The experimental evaluation is insufficient to substantiate its central claims. Especially, the evaluation is heavily biased toward outdoor environments, while indoor experiments rely on a single benchmark.

  • Several stronger state-of-the-art methods are omitted from comparison, including MegaLoc (CVPRW 2025, 87.7% R@1 on Baidu), ImAge (NeurIPS 2025), EffoVPR (ICLR 2025), QAA (ICRA 2026), SAGE (ICLR 2026), and Pair-VPR (RA-L 2025).

  • Improvements over SelaVPR++ are modest, and the ablations do not sufficiently establish the necessity of the proposed architectural complexity. SceneMoE-VPR improves average R@1 by only 0.6 points over SelaVPR++, including just 0.3 points on Baidu Mall. Moreover, it still underperforms outdoor-only SelaVPR++ on Nordland (92.6% vs. 93.8%).

Questions for authors
  1. Can the authors extend the evaluation to multiple independent indoor VPR benchmarks, particularly 17 Places and Gardens?

  2. Can the authors provide a comprehensive comparison against state-of-the-art methods, particularly those mentioned above, including evaluations on the Baidu dataset (and other added datasets) for methods with publicly available code?

  3. Can the authors provide a complete comparison of both SelaVPR++ and SceneMoE-VPR, each trained with and without ScanNet, and evaluated on multiple indoor and outdoor benchmarks? This controlled experiment is necessary to determine whether the proposed architecture genuinely mitigates the motivating trade-off.

  4. Can the authors report trainable parameters, FLOPs, training and inference times, to assess whether the marginal gains justify the additional complexity?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.