acceptodds
Under review as a conference paper at ICLR 2027

Wasserstein-Barycenter Parts for Robust Visual Retrieval

Abstract

Visual retrieval increasingly has to operate under severe nuisance, where changes in sensor, viewpoint, or modality alter the pixels without changing the identity of the target. Such changes wash out texture and colour, while the layout of recurring local parts often survives. Part-based methods can discover these parts, but they typically summarise each part with a single mean vector, discarding useful distributional structure. We propose WBPRIOR, a lightweight plug-and-play part representation that brings a layout prior into retrieval. Each patch predicts a mean vector and a positive variance, defining a diagonal Gaussian, and each part is represented by the 2-Wasserstein barycenter of the Gaussians assigned to it. For diagonal Gaussians, this barycenter has an exact closed form, so every part retains both a mean vector and a learned scale vector with negligible additional aggregation cost. In a controlled matched study that changes only the aggregation rule, the slot mean degrades sharply on corrupted images, whereas the barycenter restores a useful part signal and provides substantially larger gains under corruption than on clean images. The same head improves the robustness of seven published cross-view retrievers, while standalone WBPRIOR is competitive with recent systems on drone geo-localisation and transfers to sketch retrieval and aerial-ground person re-identification. These results show that robust part-based retrieval depends not only on how parts are discovered, but also on how each part is represented.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
visual retrieval, cross-view geo-localization, Wasserstein barycenter, Bures-Wasserstein geometry, part-based representation learning, optimal transport, robust representation learning, probabilistic embeddings

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_oA2dOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WBPrior for image retrieval using the 2-Wasserstein barycenter. The authors argue that the conventional weighted averaging approach has difficulty in handling severe appearance changes. In this context, they propose to model each image patch as a diagonal Gaussian and compute the 2-Wasserstein barycenter of the patches, preserving both mean and learned scale information. Such mean and scale are more informative and thus facilitate the subsequent image retrieval. Experiments across multiple applications such as drone-to-satellite geo-localization and sketch-based image retrieval show promising results and improved robustness.

Critical strengths
  • The paper presents an interesting and promising approach to robust image retrieval. The idea of leveraging Gaussian and 2-Wasserstein barycenter appears to be reasonable. It yields additional information beyond conventional mean pooling, potentially improving feature discrimination under significant appearance changes.
  • The authors derive a closed-form solution that avoids expensive iterative optimal transport computations. This design allows the proposed representation to be incorporated into existing retrieval frameworks with relatively low overhead.
  • The proposed method achieves competitive performance across multiple retrieval benchmarks. Experiments demonstrate that replacing conventional slot-mean aggregation with the Wasserstein barycenter leads to improved performance. The authors also conduct ablation studies that investigate the contributions of the learned scale vector and the proposed aggregation strategy.
Critical weaknesses
  1. The motivation for adopting Gaussian distributions and 2-Wasserstein barycenters is not sufficiently convincing. The main idea is to preserve additional information beyond the mean feature. However, it is unclear why modeling patch features as Gaussian distributions and aggregating them using a 2-Wasserstein barycenter is particularly necessary for this purpose. Moreover, the predicted Gaussian variances are learned features rather than empirical feature statistics or calibrated uncertainties.
  2. The authors provide comparisons with alternative aggregation strategies, but the improvement over conventional Euclidean aggregation is relatively modest. For instance, As shown in Table 19, Euclidean (Frobenius) achieves much better performance than slot mean, but the improvement of WBPRIOR over Euclidean (Frobenius) is relatively minor. This raises the question of whether the additional mathematical formulation is truly necessary, or whether a simpler learned representation with appropriate feature aggregation could achieve comparable performance.
  3. The comparisons with recent SOTA methods appear incomplete. Several relevant approaches [1, 2] are missing from the evluations. Comparisons with these recent approaches would strengthen the experimental evaluations.

[1] Zixuan Song, Jing Zhang, Di Wang, Zidie Zhou, Wenbin Liu, Haonan Guo, En Wang, and Bo Du. "Geobridge: A semantic-anchored multi-view foundation model bridging images and text for geo-localization." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 27793-27803. 2026.

[2] Minglei Li, Mengfan He, Chunyu Li, Chao Chen, Xingyu Shao, and Ziyang Meng. "(MGS) 2^ 2-Net: Unifying Micro-Geometric Scale and Macro-Geometric Structure for Cross-View Geo-Localization." arXiv preprint arXiv:2602.10704 (2026).

Questions for authors

Please refer to my concerns in Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer oA2d: why the learned scale matters, five-seed Euclidean study, and recent methods

Thank you for the careful review and for raising these points. We have conducted additional experiments to clarify the role of the learned scale and the choice of aggregation. We also include two recent methods in our comparison, together with an analysis of their model sizes and available evaluation results. Our responses are detailed below.

W1. Why Gaussians and the 2-Wasserstein barycenter

Reviewer concern: It is not clear why information beyond the mean is needed, and the predicted variances are learned features, not statistics or calibrated uncertainties.

Ans. We agree that the predicted variance is a learned feature rather than calibrated uncertainty (Sec. 3.2). When two parts have similar feature means, their learned scales provide an additional dimension to distinguish them. We use diagonal Gaussians as a lightweight representation for which the W₂ distance and barycenter admit simple closed-form expressions (Eq. 1):

W₂²(N(μ₁, diag v₁), N(μ₂, diag v₂)) = ‖μ₁ - μ₂‖² + ‖√v₁ - √v₂‖²

In Tab. 1, replacing the slot mean with the W2 part raises Corrupt(9) R@1 from 50.93% to 59.12% (+8.19 pp, paired 95% CI [+7.17, +9.24]). A matched head with the same capacity but no Gaussian structure (App. A5.3) trails by 4.87 pp under corruption, suggesting that the improvement cannot be explained by parameter count alone.

W2. Is the barycenter needed over Euclidean aggregation?

Reviewer concern: The gain over Euclidean aggregation (Tab. 19) is small. Would a simpler representation with suitable aggregation do as well?

Ans. The Euclidean (Frobenius) row in Tab. 19 is an ablation of the same Gaussian representation: it retains the learned scale but averages variances rather than standard deviations. Its large margin over slot mean confirms the benefit of learned scale. For aggregation, W₂ geometry averages standard deviations in closed form. By contrast, variance averaging produces a larger aggregate scale whenever patch scales differ, by Jensen's inequality. Because the two rules respond differently to dispersion among patch scales, we hypothesized that their differences would be visible under sensor corruption. We tested this in a 5-seed paired study on SUES-200 (identical backbone, heads, assignment, losses, and schedule):

AggregationClean R@1Corrupt(9) R@1Sensor noise
Euclidean90.1367.7763.45
Barycenter (ours)90.4369.0065.46
Gain+0.30+1.23+2.01
Seed-level 95% CI[-0.27, +0.87][-0.02, +2.48][+0.81, +3.20]

At the seed level, the barycenter shows a significant gain under sensor noise (+2.01 pp, paired t(4) p = 0.0096, seed CI [+0.81, +3.20]; Bonferroni-adjusted p < 0.03 across three groups). On clean data, the gain is small (+0.30 pp) with CI crossing zero. Across all nine corruptions, the gain is not significant at the 5% level (p = 0.053, winning on 4 of 5 seeds). The results therefore indicate a specific empirical advantage under the tested sensor-noise condition, rather than uniform superiority across all corruptions. The closed-form barycenter adds zero parameters and negligible O(d) compute. This 5-seed study (224 px, 30 epochs, D→S, 16k queries) will be added to the appendix.

W3. GeoBridge and (MGS)²-Net

Ans. Both will be added to Tab. 3 of the final version:

MethodBackbone / Model paramsSUES-200 D→S R@1U-1652 D→S R@1 / APCorruption eval
WBPrior (ours)ViT-S/14 (23.4M total)98.3294.86 / 96.089 conditions
GeoBridge [R1]CLIP-L/14 (≥304M visual)97.2695.82 / 97.77none
(MGS)²-Net [R2]DINOv2-B (115.6M total)99.4697.60 / 98.03none

WBPrior uses 23.4M parameters and 22.14 GFLOPs, roughly 13x smaller than GeoBridge's visual encoder (≥304M) and 5x smaller than (MGS)²-Net (115.6M total). It beats GeoBridge on SUES-200 (98.32% vs 97.26%). (MGS)²-Net achieves higher clean accuracy on both datasets, and GeoBridge on University-1652, though differences in input resolution (448 px vs 224/336 px), pretraining, and architecture prevent attributing this to backbone alone. SUES-200 results average the four altitudes; (MGS)²-Net cites arXiv v2 (Tab. 2).

Neither baseline reports corruption evaluations under our protocol. Although public code and checkpoints exist, we could not establish a controlled reproduction matching their reported clean numbers under our exact benchmark and preprocessing protocol. To avoid misleading comparisons from unverified reproductions, we report their published clean results instead.

References

[R1] Z. Song et al. GeoBridge: A Semantic-Anchored Multi-View Foundation Model Bridging Images and Text for Geo-Localization. CVPR, 2026.

[R2] M. Li et al. (MGS)²-Net: Unifying Micro-Geometric Scale and Macro-Geometric Structure for Cross-View Geo-Localization. arXiv 2602.10704, 2026.

I thank the authors for the response and for the additional experiments. Some of my concerns have been addressed. However, according to the added comparisons, it seems that the introduced method performs worse than MGS²-Net. The authors argue that MGS²-Net is a larger model, but this is not very convincing. Since I am not an expert in this domain, I would like to highlight this concern for the consideration of the ACs and other reviewers, and I will keep my original rating.

AuthorsOpenReview ↗

Follow-up: accuracy under corruption and (MGS)²-Net code availability

Thank you for reading our response carefully, for keeping your positive rating, and for sharing your view openly with the AC. We are glad that the new experiments resolved most of your concerns. The remaining point is the comparison with (MGS)²-Net. We would like to set out three facts, because they limit what this comparison can show.

1. The paper's claim is accuracy under corruption, and (MGS)²-Net reports no such result.

WBPrior is a part representation for retrieval under severe corruption. The central results of the paper measure accuracy under nine severe corruptions:

EvidenceWhat changesCorrupt(9) R@1 gain
Tab. 1, matched studyslot mean replaced by the W2 part+8.19 pp
Tab. 2, seven published hostsWBPrior branch added to each host+3.62 to +8.63 pp
Five-seed study, sensor noiseEuclidean averaging replaced by the barycenter+2.01 pp

For standalone clean accuracy, the abstract claims only that WBPrior is "competitive with recent systems". The two methods sit side by side as follows:

MethodInputSUES-200 clean R@1U-1652 clean R@1Under corruption
WBPrior (ours)RGB, ViT-S/1498.3294.86nine conditions reported
(MGS)²-NetRGB + depth, DINOv2-B99.4697.60not reported

We agree that (MGS)²-Net is more accurate on clean images, as our earlier reply stated. Whether it also holds up better under corruption is unknown. On the question this paper asks, the two methods have therefore not been compared yet.

2. The comparison cannot be run today, and we correct one sentence of our earlier reply.

Our earlier reply said that public code and checkpoints exist for both baselines. For (MGS)²-Net this is wrong, and we apologise for the error. The official repository linked from its arXiv page has been empty since it was created in February 2026: it holds no code and no trained model, and we found no released model elsewhere. Nobody can evaluate (MGS)²-Net under corruption until the authors release it.

For GeoBridge, only a pre-training checkpoint is public, with no model trained on University-1652. Used as released, it reaches 7.10% clean R@1 on University-1652. After our fine-tuning it reaches 38.94%, far below the published 95.82%. We do not consider either number a fair stand-in for GeoBridge, and this is why we quote published clean results instead of unverified reproductions.

3. The clean gap is small, and the inputs differ.

On SUES-200 the gap is 1.14 pp, with both methods close to the ceiling. (MGS)²-Net adds depth maps from a second foundation model (Depth Anything 3) to a DINOv2-B backbone, and evaluates at 336 px. WBPrior uses RGB images only, with a ViT-S/14 encoder and 23.4M parameters in total. We do not offer model size as an excuse. Our point is that extra depth input can raise clean accuracy, but it tells us nothing about accuracy under sensor noise, blur or geometric change, which is what WBPrior targets.

The two methods can also be combined. WBPrior attaches to an existing retriever, and Tab. 2 shows that it raises accuracy under corruption on seven published hosts. Once (MGS)²-Net is released, adding WBPrior to it is a natural experiment, and we will report it in the final version if the code becomes available in time.

For the AC, we also note that (MGS)²-Net is an arXiv preprint (February 2026) without a peer-reviewed venue. The ICLR reviewer guidelines state that arXiv is not a peer-reviewed venue and that authors are not required to compare with papers that appear only on arXiv. We added it because you asked, and we will keep it in Tab. 3 with the context above.

You mention that some of your concerns were addressed. If any point other than (MGS)²-Net is still open, we would be grateful to hear it, so that we can answer it before the discussion period ends. Thank you again for your time and your constructive review.

Reviewer_ik1DOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WBPRIOR, which addresses the image retrieval task under severe appearance variations. It expands the representation of components from the traditional "slot mean" to the 2-Wasserstein barycenter of the diagonal Gaussian distribution. By retaining the learned scale vector of each component in the feature space, WBPRIOR provides "second signals" in addition to the mean for components at the feature level, thereby enhancing the robustness of the retrieval model under severe interference. However, the paper has a tendency to overly complicate the methodology. The fairness of some core comparative experiments is questionable, and the conceptual boundary between "learned scale" and "uncertainty" needs to be further clarified in the text.

Critical strengths
  1. The paper addresses a fundamental limitation in part-based retrieval: standard "slot means" discard second-order statistics, making the representation brittle when appearance degrades. By replacing the point estimate with the 2-Wasserstein barycenter of diagonal Gaussians, the authors introduce a second signal into the retrieval embedding. Crucially, because the barycenter of diagonal Gaussians has an exact closed-form solution, this added robustness comes at negligible computational cost. This elegant engineering allows the method to maintain single-pass forward inference and standard cosine similarity retrieval, making it highly practical.
  2. The authors do not merely show that their model beats state-of-the-art on a single benchmark; they systematically prove the generalizability of the representation principle. Validating the head across 15 different pretrained encoders and successfully attaching it to 7 independently designed retrieval architectures provides overwhelming evidence that the gain is intrinsic to the proposed Wasserstein barycenter representation rather than an artifact of a specific backbone or training recipe.
Critical weaknesses
  1. The paper makes a strong claim that attaching the WBPRIOR branch improves the robustness of any published retriever. However, the protocol requires re-training these host models. As the authors admit in Table 17, re-training the hosts under the WBPRIOR shared recipe costs them 3 to 7 percentage points of clean accuracy compared to their original, highly tuned published configurations.
  2. This compromises the fairness of the comparison. The observed robustness gains in Table 2 might be partially an artifact of degrading the baselines by forcing them into a suboptimal training regime, rather than purely the benefit of the WBPRIOR representation. This undermines the reliability of a central claim of the paper.
  3. The core conceptual contribution is injecting second-order statistics into the part representation to improve robustness. However, the paper lacks direct experimental comparisons against classical second-order pooling techniques, which are specifically designed to capture covariance/variance information.
Questions for authors
  1. Were the hosts in Table 2 subjected to any hyperparameter tuning (e.g., learning rate, epochs, batch size) to adapt to your shared recipe, or were they simply run with the default WBPRIOR settings?
  2. Can you provide a brief empirical or theoretical comparison to established covariance pooling techniques in the rebuttal? Specifically, does the diagonal 2-Wasserstein barycenter offer distinct accuracy advantages over traditional matrix-based covariance pooling, or is its primary advantage purely computational efficiency (linear vs. cubic cost)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ik1D: within-host fairness, frozen published host, and covariance pooling comparison

Thank you for the careful review and for noting the closed-form barycenter and the wide validation of the head. Your concerns are the fairness of Tab. 2, the link to covariance pooling, the meaning of the learned scale, and complexity. We answer each below and report one new test on a frozen published host.

W1/Q1. Host tuning and fairness of Tab. 2

Reviewer concern: The hosts in Tab. 2 were retrained under a shared recipe that is weaker than their published setups, so the gains may partly come from weaker baselines. Were the hosts tuned?

Ans. We did not tune any host for WBPrior. Each host was retrained once with a fixed recipe and the authors' default loss weights, and the run with WBPrior reuses that recipe unchanged. MuSeNet kept its published recipe because its style classifier needs corruption labels. Retraining outside the authors' own setups does cost accuracy: the three hosts we checked sit 3 to 7 points below their standalone versions (Tab. 17). For this reason we never compare a retrained host with a published number. Tab. 2 compares each host only with itself: the two rows of a pair share the recipe and budget and differ only by the WBPrior branch. Tab. 3, in contrast, quotes published numbers and retrains no baseline.

To test WBPrior on a host at full published strength, we added it to the authors' released Sample4Geo checkpoint with every host weight frozen (checked by SHA-256 hash). Only the small WBPrior branch was trained, on cached host features, for 10 epochs. Mean of three seeds:

University-1652 D→SClean R@1Corrupt(9) R@1
Released Sample4Geo, frozen92.2053.77
+ WBPrior adapter (ours)94.1459.64

WBPrior therefore also improves this fully trained published host, and the gain is largest under corruption.

W2/Q2. Comparison with covariance pooling

Reviewer concern: There is no comparison with second-order pooling. Is the advantage in accuracy, or only in compute?

Ans. The two methods describe different things. Covariance pooling (B-CNN [R1], MPN-COV [R2], iSQRT-COV [R3]) measures how features vary together over one image, as a d × d matrix that needs matrix square roots (524,800 values at d = 1024). WBPrior keeps a mean and a learned scale for each part, and averages patches with the exact 2-Wasserstein barycenter [R4]:

μₖ = Σᵢ wᵢₖ μᵢ, sₖ = Σᵢ wᵢₖ sᵢ, where sᵢ = √vᵢ

This costs O(d) per part and needs no matrix square root. We ran two comparisons on SUES-200. The first is a new control against empirical covariance pooling at a reduced budget (mean of 7 runs):

ReadoutClean R@1Corrupt(9) R@1Readout params
Covariance pooling18.1711.65549,120
WBPrior19.9112.0933,024

The second is the full-covariance Gaussian barycenter of Tab. 21:

Gaussian formWidthClean R@1Corrupt(9) R@1Rel. cost
Diagonal, deployed3297.5360.291.00x
Diagonal, matched1697.3659.860.82x
Full covariance, matched1697.8060.702.30x

So the main advantage is cost, not accuracy. At this budget WBPrior is on par with covariance pooling at 16.6x fewer readout parameters. The full-covariance barycenter is 0.44 pp higher clean and 0.84 pp higher under corruption, at 2.30x to 2.80x the cost of the diagonal head. The first control has low absolute numbers because of its small budget, so we read it as a cost comparison.

W3. Learned scale versus uncertainty

Reviewer concern: The boundary between the learned scale and uncertainty needs to be clearer.

Ans. The predicted variance is a learned retrieval feature. It is not Bayesian uncertainty or the empirical scatter of the patches, and we never use it as a confidence score. Sec. 3.2 defines it this way, and the final version repeats it in the introduction.

W4. Complexity

Reviewer concern: The method seems overly complicated.

Ans. Most of the complexity sits in training, not in the deployed model. The extra training losses and the teacher network run only during training (App. Tab. 6). At test time the model makes one backbone pass, finds parts with one small eigendecomposition on a coarse 8 × 8 grid, averages each part in closed form, and fuses the parts with the global feature before cosine retrieval. It needs 22.14 GFLOPs, the lowest in Tab. 3.

References

[R1] T.-Y. Lin, A. RoyChowdhury, S. Maji. Bilinear CNN Models for Fine-Grained Visual Recognition. ICCV, 2015.

[R2] P. Li, J. Xie, Q. Wang, W. Zuo. Is Second-order Information Helpful for Large-scale Visual Recognition? ICCV, 2017.

[R3] P. Li, J. Xie, Q. Wang, Z. Gao. Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization. CVPR, 2018.

[R4] M. Agueh, G. Carlier. Barycenters in the Wasserstein Space. SIAM Journal on Mathematical Analysis, 2011.

Thank you for author's response. My concerns have been addressed.

AuthorsOpenReview ↗

Thank you for the follow-up

Thank you for reading our response. We are glad that our clarifications addressed your concerns. In the final version, we will add the covariance pooling control to the appendix, clarify in the introduction that the learned scale is a retrieval feature rather than Bayesian uncertainty, and provide the full training recipe for each host in App. A4.2.

If our response has changed your assessment of the paper, we would appreciate it if you could reconsider whether your current rating still reflects your view. In any case, your questions about the fairness of Tab. 2 and the covariance pooling comparison led us to add two controls that we believe make the paper stronger. Thank you for the helpful feedback.

Reviewer_Z2J7OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes WBPrior, a part-based representation for robust visual retrieval under large appearance changes. Instead of representing each part using only a mean feature, the method models patches as diagonal Gaussian distributions and aggregates them using a 2-Wasserstein barycenter, preserving both mean and learned scale information. The proposed representation is combined with a global descriptor and trained using several cross-view alignment objectives. Experiments on drone geo-localization, sketch-based retrieval, and person re-identification demonstrate competitive performance and improved robustness under image corruptions.

Critical strengths
  1. Strong results: WBPrior achieves competitive results on different tasks (drone geo-localization, sketch-based retrieval, person re-identification). It also shows improvements under image corruption.
  2. Well-designed ablation studies: the provided appendix has extensive experiments to demonstrate the contribution of the proposed method introduced in the main paper.
Critical weaknesses

The paper has limited evidence for the plug-and-play applicability: as I understand, experiments in Tab 2 require retraining baseline models. Also the transfer experiments require task-specific training objectives. I'm wondering how the proposed representation can be integrated into pretrained retrieval system without retraining.

Questions for authors

Please see the weaknesses and address the concerns if possible.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Z2J7: WBPrior improves a frozen released retriever without retraining

Thank you for the careful review and for the positive view of our results and ablations. Your main question is whether WBPrior can improve a retriever that is already trained. In short, it can be used in two ways: trained jointly with a host, as in Tab. 2, or added as a small adapter to a frozen released checkpoint, which we test below.

W1. Adding WBPrior to a pretrained retriever without retraining

Reviewer concern: Tab. 2 retrains the host models, so it is unclear how WBPrior can be added to a pretrained retrieval system without retraining.

Ans. Plug-and-play in the paper means architectural compatibility. The branch attaches to any retriever that exposes a spatial feature map, convolutional or transformer, and needs no change to the host's architecture, classifier or losses, in the same sense as side adapters for existing backbones [R1] [R2].

Frozen host (new). We took the authors' released Sample4Geo checkpoint (ConvNeXt-B on University-1652) and froze every host weight, checked by SHA-256 hash. We trained only the WBPrior branch, on cached feature maps of the frozen host. As a control, we trained a slot-mean adapter, which keeps only the part mean, in exactly the same way:

University-1652 D→SClean R@1Corrupt(9) R@1Host weights
Released Sample4Geo92.2053.77Frozen
+ slot-mean adapter92.3854.52 ± 0.11Frozen
+ WBPrior adapter (ours)94.1459.64 ± 0.09Frozen

Adapter rows are means over seeds 42, 123 and 2024 (± is the standard deviation over seeds).

The frozen host gains +1.94 pp clean and +5.87 pp under corruption, while the slot-mean adapter barely moves it. This is the pattern the paper predicts: the gain comes from the learned scale that the barycenter keeps, and it is largest where appearance is degraded. Both adapters train for 10 epochs with AdamW on clean images. The branch output joins the host embedding through a linear projection and a residual sum, and retrieval stays cosine. The branch adds 33,024 parameters (0.04% of the host). The fusion matters: in our runs, joining the branch by concatenation did not improve the frozen host, and the final version reports these fusion variants. This test covers one host on one dataset, and the released checkpoint differs from the Sample4Geo entries in the paper (Tab. 3 and App. Tab. 16), so its numbers differ from both.

Joint training (Tab. 2). When retraining is possible, the backbone and the branch adapt to each other. Tab. 2 asks whether the branch helps a given architecture, so the host alone and the host with the branch are trained in the same way and read within each pair. WBPrior raises Corrupt(9) R@1 on all seven hosts, by +3.62 to +8.63 pp, and lowers mCE-Geo on every host. The final version adds the frozen-adapter study to the plug-in section and defines plug-and-play explicitly in the introduction.

W2. Task-specific losses in the transfer studies

Reviewer concern: The transfer experiments need task-specific training objectives.

Ans. Each transfer task keeps its standard loss, because the loss defines what counts as a match: the same person in re-ID, or a sketch and its photo in sketch retrieval. WBPrior itself does not change: the per-patch Gaussians, the closed-form barycenter per part and cosine retrieval use the same code across drone geo-localization, sketch retrieval and person re-identification. The final version states this in the transfer section.

References

[R1] Y.-L. Sung, J. Cho, M. Bansal. LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learning. NeurIPS, 2022.

[R2] M. Xu, Z. Zhang, F. Wei, H. Hu, X. Bai. Side Adapter Network for Open-Vocabulary Semantic Segmentation. CVPR, 2023.

Reviewer_Z2J7OpenReview ↗

Official Comment by Reviewer Z2J7

I thank the authors for their detailed response and additional experiments. However, based on their response, I feel the plug-and-play applicability is overstated. I will therefore maintain my original rating (weak accept).

AuthorsOpenReview ↗

Clarifying the plug-and-play claim

Thank you for the follow-up and for keeping your positive rating. We agree that our use of “plug-and-play” was too broad, and we will revise this wording in the final version.

We will instead describe WBPrior as an add-on branch trained together with its host. It can be attached to a retriever with a spatial feature map without changing the host architecture, classifier, or losses, but it does require training. Tab. 2 will therefore be described as a within-host comparison under joint training: across all seven hosts, adding WBPrior improves accuracy under corruption. The frozen Sample4Geo experiment will be presented only as a test on one released checkpoint and one dataset, rather than as evidence of general frozen-host compatibility. We will also make clear in the limitations that WBPrior requires training.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of responses: frozen-host adapter, fair within-host comparison, covariance pooling, and Euclidean control

We thank all reviewers for careful and helpful reviews. All three reviews describe the same core idea: a part should keep a learned scale next to its mean, and the closed-form 2-Wasserstein barycenter makes this cheap. The questions were about how far this idea goes in practice and how fair the comparisons are. Our answers in short:

WBPrior plugs into a host in two ways (Z2J7, ik1D). Trained jointly with the host (Tab. 2), it improves accuracy under corruption on all seven published hosts. Added as a small adapter to the authors' released Sample4Geo checkpoint, with every host weight frozen, it improves clean R@1 by +1.94 pp and Corrupt(9) R@1 by +5.87 pp (one host, mean of three seeds), while a slot-mean adapter trained the same way adds little.

The plug-in comparison stays within each host (ik1D). We did not tune any host for WBPrior. The two rows of each pair share one recipe and budget, and Tab. 2 never compares a retrained host with a published number.

WBPrior is a cheap alternative to second-order pooling (ik1D). At a reduced budget it is on par with covariance pooling with 16.6x fewer readout parameters. A full-covariance barycenter is slightly more accurate, at 2.30x to 2.80x the cost.

The W2 part gives a large gain, and on SUES-200 the barycenter helps where theory predicts (oA2d). Replacing the slot mean with the W2 part adds +8.19 pp under corruption (Tab. 1). In a new five-seed Euclidean-versus-barycenter study (separate from Tab. 1), the barycenter beats Euclidean averaging under sensor noise on every seed (p < 0.01). Over all nine corruptions it wins on four of five seeds (p = 0.053).

Recent methods (oA2d). On SUES-200 WBPrior is above GeoBridge with 13x fewer parameters. (MGS)²-Net is higher on both datasets and GeoBridge is higher on University-1652. Neither reports a corruption test.

Commitments for the final version. We will add the frozen-adapter study with its fusion variants, the five-seed study and the covariance control to the paper. The introduction will state in one sentence that the learned scale is a retrieval feature, not Bayesian uncertainty (Sec. 3.2 already defines it so), and the introduction will define plug-and-play explicitly. App. A4.2 will list the full training recipe of every host.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.