acceptodds
Withdrawn from ICLR 2027

CRAD: Confused Relation-Aware Distillation for Efficient Cross-View Geo-Localization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cross-view geo-localization aims to match ground-level images with satellite-view counterparts and is a core capability for autonomous navigation. However, state-of-the-art models are often computationally expensive, motivating the use of knowledge distillation for efficient deployment. Similarity-distribution distillation has been explored in deep metric learning, but compact cross-view geo-localization requires transferring relations between satellite queries and ground candidates while learning from informative negatives. We propose Confused Relation-Aware Distillation (CRAD), which distills a pretrained, frozen teacher into a compact student. CRAD aligns cross-view candidate distributions and jointly trains on student-student and teacher-student satellite-ground correspondences. To further strengthen supervision, we adopt Teacher-Guided confused Sampling, which constructs a shared hard-sample pool by jointly considering samples that are challenging for both the teacher and the student, allowing the teacher to actively select confused examples to guide the student’s training. Extensive experiments on multiple benchmarks demonstrate the effectiveness of CRAD. On the challenging VIGOR-Same benchmark, CRAD-Tiny (28.6M) achieves 75.16% Recall@1, while the lightweight CRAD-Nano (15.0M) still delivers competitive performance. These results indicate that, for metric learning–based geo-localization, distilling how to compare can be more effective than distilling what to represent.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-View Geo-Localization, Distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4EJUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Confused Relation-Aware Distillation (CRAD) for efficient cross-view geo-localization (CVGL), aiming to transfer knowledge from a large teacher model to compact students. Instead of directly matching feature representations, CRAD argues that retrieval performance depends on preserving candidate-level similarity relations and proposes two components: Cross-View Multi-Positive Contrastive Learning (CR-InfoNCE) to align teacher–student correspondence across views, and Relational Structure Distillation (RSD) to transfer teacher-induced candidate ranking distributions. The paper further introduces Teacher-Guided confused Sampling (TGS), which combines teacher- and student-mined hard negatives to provide more informative supervision. Extensive experiments on CVUSA, CVACT, and VIGOR demonstrate that CRAD improves compact student models over several distillation baselines.

Critical strengths

Efficient Distillation Framework for Ground-View CVGL: The paper addresses an important practical challenge in cross-view geo-localization: deploying high-performing retrieval models under limited computational budgets. CRAD integrates relational supervision and teacher-guided hard-negative mining into a compact student training framework, providing a practical approach for reducing model size while maintaining competitive retrieval performance.

Critical weaknesses

1. Limited Technical Novelty: The core idea of Relational Knowledge Distillation has been extensively explored in prior similar work [1] has already explored self-agent and cross-agent relational knowledge transfer specifically for visual place recognition. Although CRAD adapts these principles to cross-view geo-localization through cross-view contrastive supervision and teacher-guided sampling, its methodological novelty appears limited. The paper does not sufficiently establish how its relational distillation formulation advances beyond these existing approaches.

2. Insufficient Component Designs: Although Table 2 provides component-level ablations, the proposed objectives do not consistently exhibit complementary improvements. For example, CR-InfoNCE alone achieves 70.03% R@1 on CVACT-Test, outperforming the combined CRAD objective without TGS (69.28%). Similarly, RSD alone slightly outperforms their combination on VIGOR-Cross (57.60% vs. 57.35%). While the full framework benefits from TGS, these results leave the additional contribution and interaction of CR-InfoNCE and RSD insufficiently explained. More detailed analysis would strengthen the justification for combining these objectives.

Reference:

[1] Wang, S., She, R., Kang, Q., Jian, X., Zhao, K., Song, Y. and Tay, W.P., 2024, March. Distilvpr: Cross-modal knowledge distillation for visual place recognition. In Proceedings of the AAAI conference on artificial intelligence (Vol. 38, No. 9, pp. 10377-10385).

Questions for authors

Q1. Clarifying the Novelty of Distillation Methods

Distillation methods have been widely used in metric learning tasks. What are the key methodological advances of CRAD beyond adapting these established ideas to CVGL?

Q2. Effectiveness of CRAD

Table 2 shows that CR-InfoNCE alone outperforms their combined objective without TGS on CVACT-Test (70.03% vs. 69.28% R@1). Could the authors explain this result and clarify under which conditions RSD provides complementary benefits beyond CR-InfoNCE?

Q3. Comparison with Recent CVGL Methods

The main comparisons primarily cover methods published through 2025, while the teacher is based on Sample4Geo (2023). Could the authors include relevant 2026 CVGL methods from recent CV/ML top conferences (e.g., NeurIPS, CVPR, ICML) under comparable evaluation protocols? Such comparisons would provide a more comprehensive assessment of CRAD's competitiveness relative to recent methods, particularly regarding its trade-off between retrieval accuracy and computational efficiency.

Reviewer_YjPFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper is easy to follow, the experimental section is relatively thorough, and the reported results are competitive. My main reservations concern the novelty boundary of the core mechanism, the absence of the most direct lightweight baselines, and several internal-consistency/reporting issues.

Critical strengths
  • The paper is clearly written and easy to follow. The overall pipeline and the roles of the main components are presented in an accessible manner.

  • The experimental evaluation is relatively thorough. The paper includes component ablations and hyperparameter analyses across multiple settings, which helps readers understand the behavior of the proposed method.

  • The reported results are competitive, and the lightweight CVGL setting is practically motivated. The goal of reducing model cost while preserving retrieval/localization performance is of practical relevance.

Critical weaknesses

W1: The novelty of the relationship-distillation route is limited within CVGL.

Knowledge distillation in cross-view geo-localization has already been used for generalization, robustness, and annotation-cost reduction, and relationship-based distillation is no longer new in this area. Cross-Distill performs Cross-Similarity Ranking Distillation by constructing a teacher–student interaction matrix to enforce ranking consistency, explicitly targeting efficient CVGL students; S³Geo also transfers relational structures and relational-level ranking consistency from a frozen CLIP teacher. Even restricting the scope to lightweight CVGL, prior work is not absent: MobileGeo/HD-CVGL, CSP-HD, and Distill4Geo all instantiate lightweight distillation pipelines. The use of relationship distillation here is reasonable, but the route is already well established, and the technical increment relative to these works is not clearly articulated. Hard-negative mining is likewise a standard ingredient in the distillation literature. To researchers focused specifically on lightweight CVGL the contribution may appear non-trivial, but from the broader CVGL perspective the novelty is relatively modest.

W2: The most direct lightweight baselines are not adequately compared, which may inflate the reported compression advantage.

The paper mainly compares the student against large teachers, but Sample4Geo itself has already been evaluated with Tiny and Nano backbones, and lightweight variants of the Sample4Geo framework have been systematically analyzed in terms of the Recall@1–parameter efficiency frontier. According to the original Sample4Geo paper, VIGOR-Same reports Tiny = 73.90 and Nano = 70.70, whereas this submission reports CRAD-Tiny = 75.16 and CRAD-Nano = 64.30. If these numbers are confirmed against the original tables, the Tiny point-estimate advantage over the original Tiny baseline would be only about 1.26 points, not the 3.26-point advantage over the weaker reproduced baseline in this paper, and Nano would be clearly below the comparable-scale model from the original paper. Pretraining and implementation differences may well affect the comparison, and I do not assert on this basis that CRAD is ineffective; however, these discrepancies need to be investigated and explained, rather than leaving the most direct competitor absent from the main table. In addition, Nano is not equipped with its own no-distillation baseline, so the reader cannot determine how much the Nano variant actually gains from distillation.

Weakness 3: The multi-view distillation ablation suggests possible redundancy in the distillation loss.

With the weighting w set to 0, performance drops by less than one point, which raises the question of whether the multi-view distillation loss is largely redundant. This is not a fatal issue, but it should be discussed.

Questions for authors
  • Beyond the specific loss formulation, what does the proposed relationship distillation change relative to Cross-Distill and S³Geo in terms of the distillation signal or learning constraint? Can the authors articulate the technical increment over these prior CVGL relationship-distillation works?

  • Can the authors include Sample4Geo Tiny and Nano as direct baselines in the main table under matched input resolution and evaluation protocol, and explain any remaining discrepancy?

  • Can the authors provide a no-distillation baseline for Nano, so that the distillation gain at that scale can be isolated?

Reviewer_jg6YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses cross-view image retrieval. It argues that existing models might be too costly for edge deployment and proposes a knowledge distillation framework (CRAD) to learn a more compact student model from a SOTA teacher. Specifically, the framework combines cross-view contrastive learning, similarity-distribution distillation, and teacher-guided hard-negative mining. The authors argue that transferring the teacher’s relative similarities between ground and satellite images is better than directly enforcing feature similarity. Experiments on CVUSA, CVACT, and VIGOR show that the proposed method improves retrieval accuracy over the student baseline, approaching the teacher’s performance with fewer parameters.

Critical strengths
  • Transferring the teacher’s relative similarity structure rather than directly matching its features seems reasonable to me.

  • The paper is, in general, easy to follow.

  • Teacher-guided hard-negative mining shows a clear benefit in the component ablation, improving CVACT-Test Recall@1 from 61.85% to 68.38%

Critical weaknesses
  • Regarding transferring the teacher’s relative similarity structure rather than directly matching its features, is there a supporting ablation study? It is not entirely clear to me in Tab.3 whether the MSE and other baselines use the same CR-InfoNCE and how the loss weights are selected. Is it possible to get rid of other techniques and just compare transferring the teacher’s relative similarity and directly matching its features?

  • The baseline models do not appear very large to me. I am not sure if the difference between 89 MB and 28.6 MB parameters is that huge. To be honest, both look small to me.

  • If the authors could distill a student with the same architecture as the teacher, would the student achieve better performance than the teacher?

  • The methodological advance over existing similarity-based distillation is not sufficiently established.

Questions for authors
  • In Table 3, do MSE and the other baselines use the same CR-InfoNCE objective as CRAD, and how are their loss weights selected? Could you compare similarity distillation against feature matching while keeping all other components fixed?

  • What practical improvements in inference latency and memory usage result from reducing the parameter count from 89M to 28.6M, particularly on resource-constrained hardware?

  • Have you evaluated a student with the same architecture as the teacher? If so, does CRAD enable the student to outperform the teacher?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.