acceptodds
Withdrawn from ICLR 2027

Mitigating Automated Geolocation via Semantic-aware Typographic Context

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large Vision-Language Models (LVLMs) have demonstrated a strong ability to infer geolocation directly from images, raising privacy concerns as increasingly accessible LVLMs lower the barrier for large-scale location inference from publicly available images. Existing approaches for mitigating such risks primarily rely on adversarial image perturbations, which require substantial visual distortions and consequently degrade image quality. This limitation motivates a unexplored research question: Can unintended geolocation inference be mitigated through contextual information beyond the image itself? To investigate this question, we explore which textual semantics are effective in influencing LVLM-based geolocation reasoning and design a two-stage, semantics-aware typographical attack that generates contextually misleading text, termed GeoSTA. As a platform-side intervention, GeoSTA generates a protected image derivative for public or automated access by introducing semantic typographic context outside the original image region. Extensive experiments across three datasets and five commercial LVLMs demonstrate that GeoSTA significantly disrupts geolocation inference.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Image Geolocation, Semantic-aware Typographic Attack, Large Vision-Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MajYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes GeoSTA, a typographic intervention intended to mitigate image-based geolocation by large vision-language models. It preserves the original image region and appends text in an extended margin. The method selects a plausible incorrect country, constructs an authoritative location instruction, and, when necessary, uses feedback from the target model to generate an explanatory statement reconciling the visual content with that country. Evaluation covers five commercial LVLMs and three datasets of 100 images each, using country-level error rate as the attack success metric.

Critical strengths
  1. The paper addresses a consequential privacy concern and demonstrates substantial susceptibility to contextual text across 5 commercial LVLMs. The reported improvements over the evaluated baselines are large, making the empirical observation worth investigating.
  2. The framework is straightforward, and the component ablations, prompt variations, and transfer experiments provide useful evidence about when instructional and explanatory text affects country predictions.
Critical weaknesses
  1. The novelty is limited and not clearly demonstrated. Self-Generated Typographic Attacks already uses confusing targets, explanatory text, and added white margins. GeoSTA mainly adds target-model feedback, but the experiments do not show whether this feedback is better than generating an explanation directly from the image.
  2. The experimental results do not sufficiently support the privacy claims. The original image remains intact, making it easy to bypass the protection by cropping away the added border. Does any protection remain after cropping? Some explanations explicitly name the true landmark and country, potentially revealing key location clues. For example, although the text in Figure 5 denies that the image shows the Merlion in Singapore, it still gives the reader the correct location as a clue. Evaluation is limited to country-level classification error rates on three datasets of 100 images each, without distance-based metrics, evaluation on established public benchmarks, or comparisons with GeoShield and ReasonBreak. Evaluating on established benchmarks such as Im2GPS3k and Google Street View at distance thresholds of 1, 25, 200, 750, and 2500 km would provide stronger evidence of practical privacy protection.
Questions for authors
  1. Did you compare against the version of Self-Generated Typographic Attacks that includes explanatory text? How much does target-model feedback improve GeoSTA compared with generating the explanation directly from the image, without feedback?
  2. Does the protection still work if the added border is cropped out? What stops a bot from accessing the original image shown to users?
  3. Can you compare GeoSTA with GeoShield and ReasonBreak on a standard benchmark at 1, 25, 200, 750, and 2500 km? Please also report attack success only on images the model could correctly locate before protection.
  4. Some explanations explicitly name the true landmark and country, potentially revealing key location clues. For example, although the text in Figure 5 denies that the image shows the Merlion in Singapore, it still gives the reader the correct location as a clue. How is residual location disclosure accounted for?
Reviewer_LuLaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GeoSTA, a method that adds misleading textual context outside an image’s original boundaries to interfere with automated geolocation. It first selects an alternative location and constructs an instructional statement. When necessary, it uses feedback from the target model to add an explanation that reconciles the visual evidence with the alternative location. Experiments cover three datasets and five commercial vision-language models, with ablations and additional evaluations of transferability and prompt variations.

Critical strengths

The approach preserves all original image pixels and uses a straightforward intervention. This makes its mechanism easy to understand and potentially convenient for platforms that control the distributed image format. The reported improvements are substantial, and the component ablations are informative. On GeoLandmark, GeoSTA achieves an 82% attack success rate against o3, compared with 35% for the strongest reported baseline.

Critical weaknesses

The methodological novelty appears incremental relative to existing typographic attacks, self-generated attack text, and scene-aware textual manipulation. The contribution would be clearer with a precise explanation of what is new in the two-stage procedure beyond adapting these ideas to geolocation. The deployment assumptions remain underspecified. The paper describes protected derivatives for automated access while retaining original images for normal viewing. Whether an automated geolocator can access the original image or receive an image without the added margin directly affects the practical protection offered.

Questions for authors

Which procedural component distinguishes GeoSTA from self-generated typographic attacks and SceneTAP? Please relate this distinction to the component contributions already reported in Table 2. In the intended deployment, what prevents an automated geolocator from accessing the original image, and what assumptions ensure that the added textual margin is retained?

Reviewer_ZgK4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents an obfuscation method for preventing VLMs from being able to geolocate images, to help protect location privacy. Unlike prior approaches that add adversarial noise to the pixels in the image, the proposed method appends a region of text to the image with misleading information about the location. The experiments demonstrate that appending an image of misleading text to the image disrupt geolocation performance.

Critical strengths

The problem of image geolocation affecting privacy is an important issue, and methods that can alter images to protect location information may be helpful to protect users' privacy.

Critical weaknesses

My main concern with the paper is with the motivation and problem formulation. The method appends misleading text to the image which would be visible to anyone looking at the image. So, a side effect of using this method would be to have misleading text appended to any images the user choses to share, which seems undesirable. In section 4.1 they mention that "the protected image could be used for external or automated access, while the original image remains available for normal user-facing viewing". It's fairly unclear the details of how this would work out in practice. Another issue is that it seems like it would be very easy for the method to be defeated by an attacker, as it would be easy to simply remove the added text by cropping the image and then do geolocation. It seems like this would not be difficult to automate as well.

Questions for authors

Would the method be robust to cropping to remove the appended text?

What adversary is the method intended to protect against? Would the method still be effective if the adversary knows misleading text has been added?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.