acceptodds
Under review as a conference paper at ICLR 2027

X Marks the Spot: Diffusion for Image Geolocalization with Location Representations

Abstract

Image geolocalization predicts the location at which a photo was taken. Accurate geolocalization requires discriminating between both physically and semantically adjacent areas on Earth. However, prior diffusion-based approaches predict locations by generating coordinates, which are semantically unstable over space. We avoid this limitation by learning geolocalization behavior in the pretrained representation spaces of location encoders, which provide both positional and semantic understanding. To make this space usable for coordinate prediction, we introduce a Location Autoencoder (LAE), which pairs a frozen location encoder with a learned decoder mapping embeddings back to coordinates with kilometer-level accuracy worldwide. In the LAE space, we train DIG, a latent diffusion model that transports noise to location embeddings conditioned on image features. DIG consistently outperforms coordinate-based generative models and sets a new state of the art on three geolocalization benchmarks: OSV-5M, Im2GPS3k, and YFCC4k. Notably, DIG is the first generative model to outperform retrieval-based methods, and further improves retrieve-and-rerank systems when used as a prior. Source material will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Geolocalization, Generative Modeling, Location Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xk5XOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces DiG, a generative geolocation approach that performs diffusion in a pretrained location encoder space. This allows the model to have already built priors about Earth semantics without having to rebuild these global semantics.

The authors also propose LAE to be able to derive the coordinates from the location encoder embedding and Levenberg–Marquardt steps for non-trivial inversing decoders.

Critical strengths

The paper is really clear and well-motivated. Performing diffusion in a latent space that already contains semantic information around the globe, knowing where the cities are, if two different areas around the globe look the same.

The inversion of location encoder features is really elegant and might be useful to other tasks.

The results, especially the one on OSV are really strong, especially outperforming retrieval methods.

The ablations are complete and allow a better understanding of the gain of each component.

Critical weaknesses

The main weakness that appears is that these latent spaces are not perfect; it is unclear whether a bijection truly exists between embeddings and locations. Allowing the model to refine this latent space or the decoding into coordinates might be beneficial.

Questions for authors

GeoClip seems to perform well at a fine scale. It is hard to know if this comes from the modality with which it has been aligned or the different architecture with Random Fourier Features. To disentangle this, it would be interesting to enrich Table 5 with RFF.

In Table 5, do you still use REPA on the Spherical Harmonics or not? Have you tried to perform diffusion on SH while doing REPA with pretrained location encoders, to encourage the model to learn semantics from the location encoder while still being more free about its own latent space?

Can you run DIG with only StreetCLIP conditioning against RFM S² using the same conditioning, to isolate the effect of the diffusion space from the conditioning upgrade?

3 comments

We thank the reviewer for finding our approach clear, well-motivated, and elegant. We especially appreciate the detailed and thoughtful questions which engage closely with our design choices. We address them below:

W1: Imperfect latent spaces and bijection between embeddings and locations

We agree that a discriminative mapping between the location representation and locations is essential to geolocalization performance. We find empirically that the mapping between representations and locations is strongly discriminative and recoverable at the geolocalization task range: our decoders recover locations to within 1 km at 99.9–100% accuracy for all encoders (Tab. 9). With numerical refinement, GeoCLIP and TTE recover at least 99.7% of locations within 10 m, while SatCLIP reaches 90.2% (Tab. 6). For generated embeddings that may not exactly match any location, LM refinement finds the closest match, and this decoding outperforms gallery retrieval (Tab. 10).

Making DIG more fine-grained to exploit sub-kilometer resolution, as well as exploring location representations optimized for diffusion, remain open directions we're excited to explore in future work.

Q1/Q2. RFF baseline and isolating REPA

To clarify, we do not use REPA with non-LAE diffusion spaces in Tab. 5. This is deliberate to isolate any impact of semantic location representations in the diffusion process. We include additional baselines of position-only spaces (including the requested RFF) with REPA in Tab. R2:

Table R2. DIG-B Geolocation performance on OSV-5M (official train/test splits)

Latent SpaceGeoS. ↑Dist. (km) ↓Country (%) ↑Region (%) ↑City (%) ↑
RFF390199678.349.312.4
RFF w/ REPA397989480.451.513.8
GeoCLIP w/ REPA403787882.656.419.8
SH (L=40)402397981.153.813.9
SH (L=40) w/ REPA406293182.256.016.0
SatCLIP w/ REPA409687582.957.618.4

RFF follows the random Fourier features as used in GeoCLIP, and RFF w/ REPA and GeoCLIP w/ REPA align with GeoCLIP location encoder features. SH (L=40) w/ REPA and SatCLIP w/ REPA align with SatCLIP (L=40) location encoder features. REPA does improve RFF and SH baselines on all metrics, but it still falls behind pretrained location encoder latent spaces with REPA, demonstrating the importance of diffusing directly in semantic latent spaces.

By isolating RFF as the latent space, we see that it performs worse than the SH (L=40) at fine scales (13.9 vs. 12.4). Yet, GeoCLIP (built on RFF) performs better than SatCLIP (built on SH (L=40)) at city-level (19.8 vs. 18.4), suggesting that the alignment modality impacts GeoCLIPs ability to perform well at fine scales.

Q3. Isolating StreetCLIP conditioning

We compare DIG-B models trained on only with StreetCLIP image conditioning in Tab. 4d. We highlight the comparison here for ease.

Table R3. DIG-B Geolocalization performance on OSV-5M only using StreetCLIP as the vision encoder.

ModelGeoS. ↑Dist. (km) ↓Country (%) ↑Region (%) ↑City (%) ↑
RFM S²3767106976.244.25.4
DIG-B (GeoCLIP)395397581.053.818.2

Even at less training (400k steps for DIG-B vs. 1M steps for RFM S²), DIG-B with only StreetCLIP conditioning outperforms RFM S² across the board, especially at finer scales (18.2 vs. 5.4 City acc.).

The authors have adequately addressed my questions and concerns in their rebuttal. I am satisfied with their clarifications and have no further concerns.

The authors have addressed my questions.

Reviewer_BBGmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes DIG (Diffusion for Image Geolocalization), a diffusion-based framework for worldwide image geolocalization. Unlike existing generative approaches that operate directly in geographic coordinate spaces, DIG performs diffusion in pretrained location representation spaces.

Specifically, the authors introduce a Location Autoencoder (LAE), which combines a frozen pretrained location encoder with a learnable coordinate decoder to reconstruct geographic coordinates from location embeddings. A conditional diffusion model is then trained to generate location embeddings from image features, which are subsequently decoded into geographic coordinates.

Experiments on OSV-5M, Im2GPS3k, and YFCC4k demonstrate competitive performance against existing generative geolocalization methods. The authors also explore integrating DIG as a prior into retrieve-and-rerank systems to further improve localization performance.

Critical strengths

S1. Clear and well-motivated approach.

The authors highlight that existing generative geolocalization methods based on geographic coordinates or spherical harmonic encodings primarily capture spatial relationships, while potentially overlooking visual semantic similarities between geographically distant locations. In contrast, pretrained location representations can encode both geographic and semantic information. Therefore, performing diffusion in such representation spaces is a reasonable and meaningful direction.

S2. Practical applicability and compatibility with existing methods.

The authors demonstrate that the location distributions generated by DIG can serve as useful priors for existing retrieve-and-rerank systems, further improving their localization performance. This highlights the practical value and flexibility of the proposed framework.

Critical weaknesses

W1. Limited methodological novelty.

The overall framework primarily combines existing techniques, including pretrained location encoders, rectified flow-based diffusion, representation alignment, and a learnable coordinate decoder. Although the proposed combination is reasonable and achieves promising results, the main methodological contribution lies in changing the diffusion space rather than introducing a fundamentally new modeling mechanism.

W2. Incomplete comparisons with recent state-of-the-art methods.

The paper claims state-of-the-art performance on OSV-5M, Im2GPS3k, and YFCC4k. However, several recent competitive methods, including Pinpoint (2026), GeoRouter (2026), and GeoSearch (SIGIR 2026), are not included in the quantitative comparisons.

Although some of these approaches rely on retrieval-based techniques or additional external resources, they report competitive or stronger results on certain benchmarks. Therefore, a more comprehensive comparison with these recent methods would strengthen the experimental evaluation. Alternatively, the authors should clarify the scope of their state-of-the-art claims and explicitly discuss differences in experimental settings.

W3. Missing comparison with direct location embedding regression.

The proposed method uses a diffusion model to generate location embeddings conditioned on image features. However, the paper does not sufficiently justify the necessity of diffusion modeling compared with a simpler deterministic regression approach.

Specifically, the authors could introduce a deterministic regression baseline that uses the same image encoders as DIG, followed by a trainable MLP to directly predict location embeddings. The predicted embeddings could then be converted into geographic coordinates using the same LAE decoder and Levenberg–Marquardt refinement.

A comparison under matched training data, image features, and comparable model capacities would help clarify whether diffusion modeling provides substantial benefits over direct embedding regression. Additionally, comparing inference costs would further demonstrate the practical value of the proposed approach.

Questions for authors

Q1. Comparison with Recent State-of-the-Art Methods.

The paper claims state-of-the-art performance on OSV-5M, Im2GPS3k, and YFCC4k. However, several recent methods, including Pinpoint (2026), GeoRouter (2026), and GeoSearch (SIGIR 2026), are not included in the quantitative comparisons. Could the authors clarify how DIG compares with these methods and under which experimental settings the SOTA claims hold? Although some of these methods rely on retrieval or external resources, a clearer comparison would help assess the significance of the reported improvements.

Q2. Necessity of Diffusion Modeling.

The proposed method employs diffusion to generate location embeddings, but it remains unclear whether diffusion is necessary for achieving the reported performance. Have the authors evaluated a simpler deterministic regression baseline that directly maps the same image features to location embeddings using an MLP, followed by the same LAE decoder and LM refinement? If so, could the authors provide the corresponding localization accuracy and inference cost? Such a comparison would help determine whether the performance gains justify the additional complexity of diffusion modeling.

1 comment

We thank the reviewer for the detailed review, and for finding our approach well-motivated and practically useful.

W1: Limited methodological novelty.

DIG has a simple modeling mechanism intentionally. By keeping the architecture and objective standard, we isolate the effect of the diffusion space itself: changing only the space from coordinates to location representations improves accuracy at every scale (Tab. 5). Beyond this, our contributions include:

  • LAE: inverting location encoders to kilometer-level accuracy worldwide, with analytic LM refinement.
  • Encoder analysis: characterizing which location encoders make good diffusion spaces (Tab. 5, Sec. 6).
  • Generative vs. retrieval: the first generative model to outperform retrieval-based methods, which also composes with them as a prior (Tab. 3, 15).

W2/Q1: Comparison with recent methods.

GeoRouter and GeoSearch are MLLM-based methods (Sec. 2) that rely on closed-source large multimodal models and external resources at inference, such as web-scale reverse image search. Pinpoint uses a large database of images and retrieves and reranks them for a given query for prediction. We view these approaches as complementary to DIG: as we show with GeoRanker, DIG's samples improves its retrieve-and-reranking when used as a prior (Tab. 3, 15).

On OSV-5M, DIG-XL outperforms Pinpoint on GeoScore and region accuracy, while trailing on mean error, country, and city accuracy. We note that Pinpoint trains on both OSV-5M and MP16, roughly twice our training data, whereas Tab. 2 explicitly compares methods trained on the official splits.

Following the reviewer's suggestion, we have clarified the scope of our claims: DIG sets a new state of the art on OSV-5M with official splits, and on Im2GPS3k and YFCC4k among database-free methods (no external databases or tools). We have made these changes in the abstract, introduction, and conclusion.

W3/Q2: Necessity of diffusion modeling.

We thank the reviewer for this suggestion and show this baseline in Tab. R1. The regression model uses the same image features, training data, LAE decoder, and LM refinement as DIG, with a matched residual MLP (B size, 400k steps).

Table R1. Regression vs. diffusion on OSV-5M (official splits). Latency is measured at batch size 1 on one H100 GPU.

ModelGeoS. ↑Dist. (km) ↓Country (%) ↑Region (%) ↑City (%) ↑Latency (ms/img) ↓
GeoCLIP Regression3849117078.452.719.34.2
GeoCLIP Diffusion403787782.656.419.827.7

Diffusion improves every metric, with the largest gains in mean error (−293 km) and country accuracy (+4.2 points). Regression nearly matches at the city level, likely because distinctive images have a single plausible location, so the predicted mean is accurate. For ambiguous images, the mean falls between plausible regions, while diffusion commits to one (e.g., Fig. 4, bottom). Sampling adds 23.5 ms per image, but DIG remains inexpensive compared to MLLM-based methods and, unlike retrieval-based methods, requires no image database.

Beyond accuracy, modeling the full distribution through diffusion enables what regression cannot: aggregating multiple samples for better predictions (Tab. 14), serving as a prior for retrieve-and-rerank systems (Tab. 15), and expressing uncertainty (Fig. 9).

Reviewer_EnAmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper offers a new method for geolocalization that is based on diffusion models.

Prior methods used diffusion models to predict directly the coordinate from the image. Instead, this paper works in the pretrained representation space of location encoders (e.g., GeoCLIP, SatCLIP, TTE) that unify spatial and visual-semantic representations. To convert generated location embeddings back into geographic coordinates, the authors introduce Location Autoencoders (LAEs), which combine a frozen location encoder with a residual MLP decoder trained on synthetic uniform land coordinates6more_horiz.

Excellent results on several datasets are reported.

Critical strengths

Reframing generative geolocalization from raw physical coordinates to semantically structured location embedding spaces is conceptually elegant and well-motivated.

The use of LM post-processing optimization gives surprisngly large jump in performance accuracy.

The empirical results are impressive.

Critical weaknesses

I did not find critical weaknesses. Some weaknesses are:

  1. Sensitivity to Pretrained Location Encoders: The performance of DIG is strongly bounded by the characteristics of the underlying location encoder[32][33]. This might explain the need for LM post-training optimization.

  2. Would be nice if LM could be integrated directly into the diffusion process.

Questions for authors

Can the authors address the weaknesses mentioned above?

1 comment

We thank the reviewer for the positive assessment and for finding our approach conceptually elegant and well-motivated. We address specific questions below:

W1: Sensitivity to pretrained location encoders.

We agree that location encoders differ, and characterizing which encoders make good diffusion spaces is a central question in our work. We clarify that our method has two separate stages, and the encoder affects each differently:

  1. Decoding (LAE). Encoders differ in how easily their embeddings decode to coordinates. GeoCLIP is the hardest, reaching only 67.9% within 1 km after 25B training locations (Fig. 2, Tab. 1), despite reaching 100% within 25 km after 2.5B. Rather than training for many more GPU days, a few LM steps close this gap cheaply. With refinement, all encoders decode to 99.9–100% within 1 km (Tab. 9).

  2. Generation (DIG). The diffusion model is trained on frozen embeddings and never sees the decoder or LM. Its performance therefore reflects how well it generates in each encoder's space, not how easily that space decodes. Consistent with this, LM refinement only affects accuracy at the 1 km threshold (≤0.3 points at ≥25 km, Tab. 8), and GeoCLIP, the hardest to decode, achieves the best city-level accuracy (Tab. 2).

Our focus is whether off-the-shelf location encoders can serve as latent spaces for diffusion-based geolocalization. We leave pretraining location encoders to specialize for this task as promising future work.

W2: Integrating LM steps directly into the diffusion process.

Because most location encoders use high-frequency input bases (e.g. random Fourier features), it is computationally expensive to completely recover the mapping from embedding to location. Fortunately, the location encoder is completely differentiable which allows traditional methods for numerical optimization to cheaply refine the decoded locations with strong convergence properties. We clarify that we intentionally do not introduce mechanisms for location-space refinement in our diffusion training, as coordinates lack the semantic structure that motivates diffusing in location representations (Tab. 5). Because representation decoding + numerical optimization can recover coordinates at extremely precise spatial resolution with little cost, we find that this step is not the limiting factor toward achieving strong geolocalization performance.

Comments on the paper

1 comment

As the discussion period draws to a close, we thank the AC and reviewers for their time and engagement. We were glad to see that the reviewers found the approach conceptually elegant and well-motivated (EnAm, BBGm, xk5X) with impressive performance (EnAm, BBGm).

Following the reviewers' suggestions, we ran additional baseline comparisons that strengthen the paper's central claims:

  • Diffusion vs. regression (BBGm): diffusion outperforms matched regression baseline (Tab. R1).

  • Additional position-only latent space baselines (xk5X): representation alignment to location representations improves position-only diffusion models, but still falls short of DIG, which directly uses those representations as the latent spaces (Tab. R2).

Together, these results confirm the importance of both the diffusion objective and the latent space of location encoders, which are key contributions of our work. We have also clarified that our models are state-of-the-art among database-free methods on all three benchmarks in the abstract, introduction, and conclusion (BBGm).

We are glad that Reviewer xk5X found their concerns fully addressed in our response. We have also replied in detail to Revewiers EnAm and BBGm and would be happy to address any remaining questions and concerns.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.