acceptodds
Under review as a conference paper at ICLR 2027

When the Service Is the Adversary: Cue-Driven Editing Against VLM Attribute Inference

Abstract

When users submit images to closed-source vision-language models (VLMs) for everyday tasks, the model processes not only what the user explicitly asks, but also what the image passively reveals—age, gender, ethnicity, occupation, and other sensitive attributes the user never intended to share. We formalize this threat as Service-Time Private Attribute Inference (ST-PAI) and show that existing perturbation-based defenses are structurally insufficient under it: the target model is closed-source and controls its own preprocessing pipeline, stripping pixel-level perturbations before inference is ever performed. We instead build defense on top of semantic image editing, with attribute-related visual cues sequentially edited out of the image according their importance, producing a discrete privacy-utility frontier that trades utility for protection in a controlled manner. Experiments on two geolocation benchmarks show that semantic editing remains robust to upstream purification, whereas perturbation-based protection collapses, supporting our argument that inferential evidence must be removed at the semantic level. We further repurpose existing VQA tasks to analyze how the utility cost of protection depends on the downstream task, and construct SynthPAI-Visual, a synthetic multi-attribute benchmark with automatically generated attribute annotations that requires no real private data.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Attribute inference attack, inference-time privacy, image editing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_7KasOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the privacy risk that closed-source vision-language models (VLMs) may infer users’ sensitive attributes while providing their normal services, and formalizes this problem as Service-Time Private Attribute Inference (ST-PAI). Existing defenses based on adversarial perturbations are vulnerable to service-side preprocessing; to address this, the authors propose PEEL, a cue-driven progressive semantic editing method. PEEL uses a local VLM to identify and rank privacy-relevant visual cues, and generates candidate images with different levels of protection through nested parallel editing and a velocity inheritance mechanism, allowing users to choose between privacy protection and image fidelity. In addition, the authors construct SynthPAI-Visual, a synthetic multi-attribute dataset, and validate the effectiveness of the method through experiments on geolocation, the privacy–fidelity trade-off, and downstream VQA.

Critical strengths
  1. The motivation is intuitive, and the problem addressed is of considerable importance.
  2. The method is well designed and offers a degree of technical novelty.
  3. The experiments are fairly comprehensive.
Critical weaknesses

For this paper, I have the following concerns:

  1. I also have some doubts about the assumptions underlying Proposition 1 and the scope of its conclusion. The key premise of the proposition is that service-side preprocessing can effectively suppress sensitivity along off-manifold directions. However, γ<1\gamma < 1 alone is not sufficient to guarantee that perturbation-based defenses fail; the actual outcome also depends on the relationship among parameters such as LSL_S, L⊥L_\perp, γ\gamma, and Δi\Delta_i. Although the authors provide some empirical support in Appendix A.3, these experiments do not directly verify whether these parameters satisfy the required conditions in actual closed-source VLMs. As a result, the current theoretical analysis reads more like an explanation of why perturbation-based defenses may fail under specific assumptions than a proof that such defenses necessarily fail under general service-side preprocessing. I would like the authors to further discuss the plausibility of these assumptions for real models and to delineate the scope of the theoretical conclusion more clearly.
  2. The dataset construction and privacy evaluation may suffer from model-dependent bias, which is my main concern. According to Appendix B.4, the valid attribute labels in SynthPAI-Visual are obtained by human verification of high-confidence attribute inferences, together with their visual evidence, proposed by Qwen3-VL-8B, while Qwen3-VL-8B is also used by PEEL for cue discovery, ranking, and leakage annotation. This design may concentrate the evaluation on privacy cues that the local model can identify, without adequately covering evidence that other VLMs could discover independently. Although human verification improves label reliability, it cannot fully eliminate the selection bias introduced by the mechanism that generates candidate cues. In addition, each image is annotated by a single annotator, with no independent repeated annotation or inter-annotator agreement analysis. The attribute distribution of the dataset is also highly imbalanced: for example, location has 1,190 valid annotations, whereas place of birth and age have only 3 and 11, respectively.
  3. According to Appendix A.6, PEEL takes 228.5 seconds to generate about 6.1 candidate images on average, with a peak memory of about 35 GB. Although parallel editing offers some efficiency advantage over sequential editing, this resource requirement remains high for ordinary users’ local devices. Moreover, the experiments use GPT-4.1 to construct the target prompts and to format some of the outputs. Although the authors note that these text-processing steps can be replaced by a local LLM, the performance and quality of a fully local pipeline have not been validated experimentally. Since these texts may contain privacy-relevant visual cues, this issue bears directly on the service-side privacy threat that the paper emphasizes.
  4. I have some questions about how the defenses are compared in Table 1. The authors evaluate PEEL at its final candidate x(K)x^{(K)}, whereas the perturbation-based methods are constrained by a fixed ℓ∞\ell_\infty budget, so the amount by which each method modifies the original image may differ substantially. While this experiment shows that PEEL maintains low privacy leakage under the tested preprocessing, it does not directly demonstrate that PEEL outperforms perturbation-based methods under the same fidelity or task-utility requirements.
  5. According to Section 5.3 and Table 5 in Appendix B.2, the privacy–fidelity frontier comparison in Figure 2 uses only GPT-4o as the attacker. Although the authors evaluate additional VLMs in the robustness experiments and ablation studies, the main quantitative comparison supporting PEEL's superior privacy–fidelity trade-off is conducted with a single attacker. It is therefore unclear whether the same advantage holds consistently across different closed-source VLMs. I would appreciate additional privacy–fidelity comparisons using other attackers, such as Gemini 2.5 Pro and Claude Opus 4.8, to strengthen the generality of this conclusion.
Questions for authors

See the weaknesses.

3 comments
AuthorsOpenReview ↗

We thank the reviewer for the detailed comments and the positive assessment. We address each point below.


W1. Assumptions and scope of Proposition 1.

We agree with the reviewer that Proposition 1 explains why bounded perturbations can fail under service-side preprocessing given explicit assumptions, rather than proving that they necessarily fail for every model and every preprocessing operation. We have moved the scope statement from Appendix A.2 into Section 3.2 to make it explicit in the main text, where Proposition 1 is now stated informally, with the formal version in Appendix A.2.

Scope. Part (i) shows that, without preprocessing, an imperceptible off-manifold perturbation can succeed. Part (ii) shows that, once preprocessing attenuates the off-manifold sensitivity (γ<1\gamma < 1), success requires a semantic component of size Δi/(2LS)\Delta_i/(2L_S). Whether this exceeds the imperceptibility threshold depends on LSL_S, L⊥L_\perp, γ\gamma, and Δi\Delta_i. Of these conditions, γ<1\gamma<1 depends on the preprocessing rather than on the model: denoising-type preprocessing such as DiffPure maps the input back toward natural images before it reaches any model, so it holds regardless of which closed-source model follows. The threshold condition, in contrast, depends on LSL_S, L⊥L_\perp, and Δi\Delta_i of the model and cannot be verified for black-box VLMs, which is why we state the conclusion as conditional. A perturbation with a large budget, or a weak preprocessing (γ\gamma close to 1), may therefore still succeed; the proposition only states that the protection must then rely on a semantic change.


W2. Model-dependent bias of the dataset, single annotator, and attribute imbalance.

Model-dependent bias. To test whether the evaluation favors the cues that Qwen3-VL-8B can identify, we replaced PEEL's local model Mo\mathcal{M}_o with four other VLM families (Gemma 3 12B, InternVL3.5-8B, Ministral 3 8B, GLM-4.6V-Flash), keeping all other settings the same as in Section 5.3. If the labels were biased toward cues that only Qwen3-VL-8B finds, a different local model would discover different cues and lose privacy. Instead, PEEL reaches comparable or slightly higher privacy with every other model (+0.03 to +0.08 over Qwen3-VL-8B, none statistically significant), at a small fidelity cost. Moreover, the evidence used by independent VLMs is largely covered: Qwen3-VL-8B detects 97–100% of the leakages found by the closed-source attackers (Table 2). The geolocation results (Table 1), whose GPS ground truth is independent of Qwen3-VL-8B, further confirm the effectiveness of PEEL.

Annotation quality. Each batch of 100 images was assigned to a single annotator, and we controlled quality within our budget by auditing every batch against an expert (one of the authors). The expert re-judged 5–10 randomly sampled images of each batch following the same guidelines; if more than one of them was judged incorrectly, the annotator had to re-check the whole batch and resubmit it (Appendix B.4).

Attribute imbalance. The imbalance reflects what environmental scenes without people can reveal rather than a design choice: all eight attributes are encoded into the scenes, but place of birth requires explicit documents such as a passport, as souvenirs or flags only indicate travel or affinity, and age is rarely supported by more than a stereotypical guess, so annotators rejected most such inferences. We now also report the per-attribute privacy leakage of raw images (Table 10): the frequent attributes are also the most exposed, so the imbalance concentrates the evaluation on the attributes that leak the most.

Reviewer 1 raised similar concerns, and we have therefore emphasized these points in the revised paper (Appendices B.4 and C.5).


W3. Compute cost, and the use of an API LLM to construct the target prompts.

Compute. We agree that compute remains a limitation (Appendix E), and two properties mitigate it (Appendix A.6). First, the question-agnostic frontier is built once per image and offline, e.g., when a photo is taken or imported, and is reused for every later request, so it adds no latency to the interaction with the service. Second, the 1+2K1+2K velocity evaluations per step are independent and can be distributed over GPUs, which reduces the time per frontier from 228.5 s to about 17 s with 2K+12K+1 GPUs.

API LLM. To validate a fully local pipeline, we replaced the API LLM that writes the target prompt chains with a local Gemma 3 12B and re-ran PEEL with the same cues and editor of Figure 2.

API LLM chains (paper)Gemma 3 12B chains (local)Δ [95% CI]
Privacy, GPT-4o0.7750.749−0.03 [−0.12, +0.06]
Privacy, Gemini 2.5 Pro0.6690.691+0.02 [−0.08, +0.11]
Privacy, Claude Opus 4.80.8910.921+0.03 [−0.04, +0.12]
CLIP / SSIM / LPIPS0.840 / 0.769 / 0.1480.845 / 0.767 / 0.151all within ±0.01

No difference is significant and nesting is unchanged, so writing the prompt chains locally does not affect the results (Appendix A.7).

W4. Table 1 compares PEEL at its strongest edit x(K)x^{(K)} with budget-constrained perturbations, i.e., not at equal fidelity.

We agree that the two families cannot be compared directly, and we were aware of this when designing Table 1. They pursue different notions of fidelity: perturbation-based defenses aim to leave the image visually unchanged within an imperceptibility budget, whereas editing deliberately changes the content of the cues and trades fidelity against privacy along the frontier. There is therefore no common operating point at which both are "equally faithful." Table 1 is accordingly not meant to rank the methods, but to measure how each family behaves under preprocessing, i.e., the change in leakage relative to the same method without preprocessing. The comparison at matched fidelity is the subject of Section 5.3, where all methods are editing-based and share the same notion of fidelity. As stated in Section 5.1, the utility of the two families cannot be compared on a common scale; we have made this explicit in the caption of Table 1 as well.


W5. Figure 2 uses a single attacker.

Following the reviewer's suggestion, we attacked all candidates of Figure 2 with two further closed-source VLMs, Gemini 2.5 Pro and Claude Opus 4.8, using the same prompt and scoring, and included frontier variants of PPA and U2S that receive PEEL's ranked cues. The privacy-fidelity advantage of PEEL does not depend on the attacker: for all three attackers, PEEL achieves the best trade-off between privacy and fidelity, and MTC matches its privacy only at a much lower fidelity. We have added the per-attacker frontiers to Appendix C.7.

Thank you for your response. I’ll keep my score as it is.

Reviewer_GfLMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper focuses on private attribute inference (PAI) from images by VLMs. The threat model focuses on PAI by closed-source providers, when users interact with these models for other purposes. To mitigate this inference, the paper introduces an inference-driven fine-grained image editing-based method to remove cues from the image that reveal private attributes. The paper's evaluation on geolocation benchmarks shows the effectiveness of the method in terms of privacy protection. Additionally, the paper also introduces a synthetic image PAI dataset to enable the open study of private attribute inference beyond geolocation. Finally, experiments show promising results in privacy-utility trade-offs, using both similarity-based metrics and question-answer (downstream utility) metrics.

Critical strengths
  • Important problem.
  • Dataset contribution.
  • Strong results + clear benefit of the method that it can inform users of privacy - utility trade-offs.
Critical weaknesses
  • In my opinion the threat model is formulated unnecessarily narrowly. It is unclear to me why the paper insists strongly on focusing on the threat stemming from the API providers themselves, even though pre-processing techniques that remove pixel-level defenses could be employed also by third-parties that obtain these images (expanding the threat model). I would like to receive some more details from the authors on this point, as this threat model is clearly used at several places also to distinguish this paper from prior work.
  • It is unclear what the technical differences and novelty of the method is compared to other editing-based methods, such as U2S and MTC. A further expanded discussion would be beneficial to convince readers of the technical novelty of the method.
  • The introduced synthetic dataset does not look photorealistic, despite explicit efforts. While this is of course bounded by the current best image generation models, I believe there are already (even open) models available that could be used to generate a more realistic looking benchmark.
  • The method is in principle related to the text-based anonymization method of Staab et al. 2025, but that is not cited in the paper.
  • The mathematical formalizations at many places of the paper do not add much value either for understanding or for actually deriving/improving parts of the method. For improving clarity, I would consider reducing these.

References

Staab et al. Large language models are advanced anonymizers. ICLR 2025.

Questions for authors

See weaknesses.

3 comments
AuthorsOpenReview ↗

We thank the reviewer for the positive assessment and the constructive comments.


W1. The threat model is formulated unnecessarily narrowly; third parties can also preprocess images.

We focus on the service because it is the setting that shapes the design of PEEL, beyond the vulnerability of pixel-level defenses to preprocessing:

  • Disclosure is unavoidable. Unlike posting, where a user can choose not to share an image, the user cannot withhold the image without giving up the service, so it must be protected before every upload.
  • Latency is bounded. The user waits for the answer, so the protection cannot add a long delay to the interaction. This is why PEEL builds a question-agnostic frontier once per image, offline (e.g., when a photo is taken or imported), and reuses it for every later request. If the protection must be computed online, the parallel structure of PEEL also allows its edits to be distributed over GPUs (Appendix A.6).
  • The target cannot be probed. Every query to the service is already a potential disclosure, so the protection must rely on a local model, and the annotated frontier exposes the gap between local and target inference instead of testing against the target.
  • Protection must be flexible. Different requests rely on different visual evidence, and users differ in which attributes they want to hide, so a single maximally protected output does not fit every request. PEEL therefore provides a frontier from which the user selects, per request, the candidate that matches their needs (Section 5.6).

We agree that preprocessing that removes pixel-level defenses can also be applied by third parties that obtain shared images, so the advantage of editing over perturbations carries over to them. We have clarified in Section 3.1 that a third-party adversary is a special case of our setting, in which the first two requirements are relaxed.


W2. Technical differences and novelty compared with U2S and MTC.

The main technical contribution of PEEL, which none of the existing editing-based methods provides, is an inference-driven, nested privacy-fidelity frontier generated in a single parallel pass. PEEL ranks the visual cues by how strongly they support the inferences of a local VLM, edits all candidates jointly from the source image so that neighboring candidates differ only in the newly removed cue (Table 3), and annotates each candidate with the attributes that remain inferable, so the user can choose a trade-off without querying the target.

In contrast, U2S rewrites the sensitive regions it detects without ranking them or producing a frontier, and MTC is a general-purpose multi-turn editor without a notion of privacy cues, which inverts and re-renders the whole scene at every turn. Our new experiments (Figure 2 and Appendix C.7) confirm that these differences matter beyond the cue list itself: even when PPA and U2S are given PEEL's ranked cues and run as frontiers, neither matches the privacy-fidelity trade-off of PEEL, and this holds for three closed-source attackers.


W3. The synthetic dataset does not look photorealistic; open models could be used.

We agree that the realism of the images can still be improved. Anticipating that the generator would limit how faithfully the prompted cues are rendered, we in fact generated two complete versions of the dataset while building it, one with the open FLUX.1-dev and one with the commercial Gemini 3.1 Flash Image, the generator we finally adopted. To compare them, we randomly sampled 25 persona prompts and asked GPT-4.1 to check each requested cue against the images of both versions:

GeneratorCues present / partial / absentPresent per image (of 8)Legible text cues
FLUX.1-dev (open)43% / 29% / 28%3.412/83 (14%)
Gemini 3.1 Flash Image (ours)96% / 2% / 2%7.791/94 (97%)

We therefore invested in the commercial generator for the final benchmark, since cue fidelity is what a privacy benchmark needs: it renders almost all requested cues, including legible text. FLUX.1-dev, in contrast, typically renders only 3–5 of the 8 cues, invents illegible text, renders wrong landmarks (e.g., the Milan instead of the Florence cathedral), hides the remaining cues through shallow-focus compositions, and collapses multi-room overviews into a single room. Since education is carried almost entirely by text cues, it is hardly ever rendered by FLUX.1-dev. We have added this comparison to Appendix B.4.

Since real images with verified private attributes cannot be released, a public benchmark of this kind must be synthetic, and within this constraint, a state-of-the-art commercial generator was the best option available to us. We acknowledge that more advanced generators will further improve realism. Our construction pipeline (persona, scene prompts, generation, and human verification) does not depend on the generator, so the benchmark can be regenerated with such models as they become available.

W4. Missing citation of Staab et al. (ICLR 2025).

Thank you for the pointer. We were aware of this work; since our focus is on images, we cited the work on private attribute inference from images (Tömekçe et al., 2024), but we agree that the text-based anonymizer of Staab et al. is closely related, and we now cite and discuss it in the related work. Both methods follow the same "infer, then anonymize" principle: Staab et al. let an LLM infer personal attributes from a text and prompt an LLM rewriter until they can no longer be inferred. PEEL instead ranks the individual cues that support the inferences and removes them progressively, producing a nested frontier annotated with the attributes that remain inferable. This multi-cue formulation is not specific to images, and applying it to text anonymization is an interesting direction for future work.


W5. The mathematical formalization adds little value for understanding or for the method.

Thank you for the suggestion. We have reduced the formalization in the main text: Proposition 1 is now stated informally in Section 3.2, with the formal statement and proof moved to Appendix A.2, and Eqs. 2 and 3 are merged into one. We keep Eqs. 4–9 because they define the method itself.

Thank you for the rebuttal, I confirm my score.

Reviewer_kEa3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies the vulnerability of perturbation to preprocessing in VLM pipelines, which weakens the protection against private attribute inference. It therefore proposes PEEL, an editing anonymization method that ranks privacy-relevant cues with a local VLM and removes them progressively. The output is a set of candidates trading privacy against fidelity, each labelled with the attributes still inferable. It also introduces SynthPAI-Visual, a synthetic benchmark with human-verified attributes. Experiments show that the method is robust against preprocessing on geolocation benchmarks and handles the privacy-utility trade-off better than other anonymization pipelines.

Critical strengths
  1. The authors formally define the ST-PAI problem and explain, through Proposition 1 and experiments, why perturbation defenses fail under service-side preprocessing while PEEL stays robust.
  2. The editing is selective and flexible.
  3. The evaluation was extensive and provides clear analysis through experiments with multiple attackers and downstream domains.
  4. The paper was written clearly, with great detail on reproducible implementation and model prompts.
Critical weaknesses

W1. The benchmark’s ground truth comes from the same Qwen3-VL-8B model the main model uses.

W2. Throughout the paper, the authors frame PPA and Unsafe2Safe as only able to produce single outputs, although they can be adapted to a customized attribute set/ custom prompts. The paper has not investigated this aspect.

Minor Weaknesses

  1. The benchmark quality: Each image was verified by only one annotator, without sufficient evaluation of the annotation quality. The attributes are also dominated by location, occupation, and education (as can be seen from the distribution in Table 8).
  2. The VQA results in Section 5.6 need to include results for U2S, PPA+ MTC.
Questions for authors
  1. How many StreetView and Im2GPS3k images were used in Table 1? Please also elaborate on the setup and the qualitative results of this experiment.
  2. What is the distribution of cues each image has (rather than the average number reported)?
  3. What is the privacy score for each attribute in Table 8?
2 comments
AuthorsOpenReview ↗

We thank the reviewer for the careful reading and constructive comments. We address each point below.

W1. The benchmark's ground truth comes from the same Qwen3-VL-8B model the main model uses.

We would first like to clarify the role of Qwen3-VL-8B in the annotation. It only proposes "confident" attribute inferences, and human annotators verify each of them and reject unsupported ones (Appendix B.4). Even among high-confidence inferences, a preliminary analysis found about 30% to be unsupported (Table 12). Annotators were also encouraged to add inferences the model missed, although in the end no reliable new inference was added. During editing, PEEL has no access to the labels and relies only on the cues proposed by its local model.

To further verify that PEEL does not depend on using the same model as the annotation, we replaced Mo\mathcal{M}_o with models from four other VLM families, keeping all other settings the same as in Section 5.3. With a different local model, PEEL still reaches privacy comparable to or slightly higher than that with Qwen3-VL-8B (+0.03 to +0.08, none statistically significant), at a small fidelity cost. We have added the per-step results to Appendix C.5.

All metrics averaged over k=1,…,5k=1,\dots,5 on the images of Figure 2 for a compact presentation. Brackets: difference from Qwen3-VL-8B with 95% confidence interval.

Mo\mathcal{M}_oPrivacy ↑CLIP ↑SSIM ↑LPIPS ↓
Qwen3-VL-8B (paper)0.7750.8400.7690.148
Gemma 3 12B0.821 (+0.05 [−0.04, +0.14])0.7990.7530.164
InternVL3.5-8B0.854 (+0.08 [−0.05, +0.21])0.8090.7380.175
Ministral 3 8B0.803 (+0.03 [−0.08, +0.14])0.8240.7570.157
GLM-4.6V-Flash0.835 (+0.07 [−0.04, +0.20])0.7970.7490.166

W2. PPA and Unsafe2Safe are framed as single-output methods, although they can be adapted to a customized attribute set or custom prompts.

"Single output" in the paper refers to the absence of a privacy-fidelity frontier, which is the focus of PEEL. We agree that both methods can be equipped with cue-based editing. Following the reviewer's suggestion, we extended both baselines to frontier variants that receive the same importance-ordered cue sets as PEEL, and we have added these results to Figure 2 (implementation in Appendix B.6). Given the same cues, neither variant matches PEEL: the PPA and U2S frontiers remain below that of PEEL, confirming that its benefit comes not only from which cues it removes but also from how it removes them, i.e., parallel, inversion-free editing from the source image.


Minor 1. Single annotator per image without sufficient quality evaluation; attributes dominated by LOC, OCC, and EDU.

Annotation quality. Each batch of 100 images was assigned to a single annotator, and we controlled quality within our budget by auditing every batch against an expert (one of the authors). The expert re-judged 5-10 randomly sampled images of each batch following the same guidelines (Appendix B.4); if more than one of them was judged incorrectly, the annotator had to re-check the whole batch and resubmit it.

Attribute distribution. The imbalance reflects what environmental scenes without people can reveal rather than a design choice: all eight attributes are encoded into the scenes, but place of birth requires explicit documents such as a passport, as souvenirs or flags only indicate travel or affinity, and age is rarely supported by more than a stereotypical guess, so annotators rejected most such inferences (Appendix B.4). The frequent attributes are also the most exposed, so the imbalance concentrates the evaluation on the attributes that leak the most (See Q3).


Minor 2. The VQA results in Section 5.6 should include U2S, PPA, and MTC.

Following the reviewer's suggestion, we have now added these baselines to the question-agnostic VQA experiment, run as frontiers over the same ranked cues and reported at the same steps as Figure 3 (Appendix C.6). On COCO, PEEL keeps a high accuracy while its privacy rises along the frontier, whereas the baselines are either less private (PPA, whose privacy does not increase with kk) or less accurate at the early steps (U2S, MTC). On VizWiz, whose questions often target the private evidence itself, all editing methods lose accuracy as this evidence is removed; PPA keeps more accuracy only because blurring leaves the evidence inferable.

AuthorsOpenReview ↗

Response to Questions


Q1. Number of StreetView and Im2GPS3k images in Table 1, setup, and qualitative results.

We apologize for omitting these configuration details; we have now added them to Appendix B.1.

Images and setup. Table 1 uses 1,000 images from each of the StreetView and Im2GPS3k benchmarks used by GeoShield. GeoShield and M-Attack use an ℓ∞\ell_\infty budget of 16/255 with 100 steps, and the Gaussian noise has the same magnitude (σ=16/255\sigma = 16/255). DiffPure-style purification uses SDXL image-to-image with strength 0.2 and an empty prompt. A prediction counts as correct if it names the correct country on StreetView or lies within 750 km of the ground truth on Im2GPS3k. The editing methods edit the location cues proposed by Qwen3-VL-8B, and PEEL and MTC are evaluated at their last candidate x(K)x^{(K)}.

Qualitative results. Table 1 focuses on how each protection changes under service-side preprocessing. Perturbation-based protection does not survive it: once the perturbation is removed, the attackers again read the location from the same evidence as in unprotected images, such as street signs, shop fronts, and architecture. Editing removes this evidence itself, so preprocessing has nothing to restore and the protection of PEEL remains stable.


Q2. Distribution of cues per image.

On all 1,402 verified images, the number of cues per image is unimodal with mean 7.3 and median 7. Almost every image supports a multi-step frontier: 99.4% of the images have at least 3 cues and 93.0% at least 5. Outdoor overviews have the fewest cues (mean 6.5) and home scenes the most (7.7). We have added the histogram and the per-scene distribution to Appendix B.4 (Figure 8).


Q3. Privacy score for each attribute in Table 8.

Table 8 (Table 11 in the revised version) summarized the statistics of our generated dataset, and we thus understand the requested privacy score as the exposure risk of each attribute on raw images, i.e., before any protection is applied. We therefore attacked these unprotected images covering every attribute of Table 8 with GPT-4o. Privacy leakage is the fraction of images whose verified attribute is inferred correctly; frequency is the share of all 1,402 SynthPAI-Visual images with a verified label of the attribute:

LOCEDUOCCSEXAGEINCPOBMAR
Privacy leakage0.870.760.630.520.500.380.330.26
Frequency (%)84.946.155.011.10.85.80.211.2

The most frequent attributes are also the most exposed: location, education, and occupation are inferred from 63–87% of the images that encode them, as they are carried by explicit cues such as signage, diplomas, or work tools. The rarer attributes are inferred less often, and the estimates for AGE and POB rest on few images. We have added this table to Appendix B.4 (Table 10).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.