acceptodds
Withdrawn from ICLR 2027

COMPASS: Membership Inference Attack Against Vision Large Language Models via Adaptive Noise Calibration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision large language models (VLLMs) are increasingly deployed in privacy-sensitive domains, raising concerns about the exposure of their training data. The strongest existing gray-box membership inference attacks (MIAs) against VLLMs commonly rely on entropy statistics or generated descriptions, requiring costly response generation while providing limited membership separation. In contrast, perturbation-based methods measure how model responses change when images are perturbed, but have mainly focused on large perturbations, leaving VLLM membership behavior under smaller perturbations poorly understood. In this work, we systematically characterize VLLM membership behavior under controlled Gaussian perturbations across the full range of noise levels at fine resolution. Our analysis reveals that both the optimal perturbation magnitude and the direction of the membership signal vary across model–dataset pairs. For many settings, a previously unstudied low-noise regime produces stronger separation than large perturbations, with members exhibiting lower divergence than non-members and thereby reversing the conventionally observed score direction. Other settings retain the conventional direction, demonstrating that neither a fixed perturbation magnitude nor a predetermined score orientation is universally reliable. Motivated by this finding, we propose COMPASS, an adaptive gray-box MIA that calibrates both the perturbation magnitude and membership-score direction using a loosely distribution-matched, known non-member reference set. We instantiate COMPASS using Rényi-normalized KL divergence and Rényi divergence, with max-k aggregation over the most informative image-logit positions. Across LLaVA v1.5–7B, MiniGPT-4, and HuluMed-32B, spanning pretraining, instruction-tuning, and medical-image datasets, COMPASS consistently outperforms baselines. Under general-domain evaluations, COMPASS achieves AUCs of up to 0.88. On the distribution-matched medical evaluation, it reaches 0.64 AUC, while the strongest baseline remains at chance level. These results establish full-range adaptive perturbation analysis as an effective approach to VLLM membership inference.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Learning Privacy, Membership Inference Attack, Vision Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LmydOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes COMPASS, an adaptive gray-box membership inference attack against vision large language models (VLLMs). The authors investigate model sensitivity to Gaussian image perturbations across a wide range of noise magnitudes and observe that the direction and strength of membership signals vary across model–dataset pairs. In particular, small perturbations can produce a regime where members exhibit lower output divergence than non-members, contrary to the behavior observed under larger perturbations. Based on this observation, COMPASS calibrates the perturbation magnitude and membership-score orientation using an unlabeled target set and a known non-member reference set. Two variants are developed using Rényi-normalized KL divergence and Rényi divergence, combined with max-k aggregation. Experiments on LLaVA-1.5, MiniGPT-4, and HuluMed-32B demonstrate improvements over several existing membership inference baselines on general-domain and medical-image datasets.

Critical strengths
  1. The paper systematically investigates how membership signals change across Gaussian noise magnitudes and identifies a low-noise regime in which members exhibit lower output divergence than non-members. The observation that both the optimal noise magnitude and membership-score orientation vary across model–dataset pairs provides a useful perspective on perturbation-based membership inference for VLLMs.

  2. COMPASS introduces a reference-guided mechanism that selects the perturbation magnitude and score orientation without requiring labeled members in the target set. The evaluations cover different model architectures and training stages, including a medical-domain model. The reported improvements, particularly the AUC of 0.88 on LLaVA's instruction-tuning data and 0.64 on the medical dataset, suggest that adaptive perturbation analysis can provide stronger membership signals than the evaluated fixed-configuration baselines.

Critical weaknesses

1. The reliability of the proposed calibration mechanism under realistic membership distributions and reference-set shifts is insufficiently established.

The core contribution of COMPASS is its ability to infer both the optimal perturbation magnitude and score orientation from an unlabeled target set and a known non-member reference set. However, this procedure relies on the assumption that the non-member component of the target set and the reference set exhibit sufficiently similar divergence responses. Appendix D.2 explicitly shows that the observable calibration gap contains both the membership signal and a reference-distribution discrepancy term. When the member proportion is small or the reference distribution differs substantially, the discrepancy may dominate the signal, potentially causing incorrect noise selection or score orientation.

Importantly, the experiments use balanced target sets with a member proportion of 0.5, whereas realistic membership inference often involves much lower and unknown member prevalence. The paper does not systematically evaluate how calibration behaves under varying membership proportions, target-set sizes, or reference-set distributions. Although Appendix D.2 claims robustness to reference-set shifts, the cited Figure 4 primarily compares estimated and oracle noise levels rather than providing an explicit reference-distribution shift study. Since calibration is the central component of COMPASS, its reliability beyond the balanced evaluation setting remains insufficiently demonstrated.

2. The experimental design does not adequately rule out numerical artifacts, dataset-source confounding, or advantages arising from unequal baseline calibration.

First, the strongest membership signals frequently occur under extremely small Gaussian perturbations. According to Eq. (5), perturbations are applied to images represented in the 0–255 intensity range, while the low-noise regime includes standard deviations far below one intensity level. Such perturbations may be sensitive to floating-point precision, image quantization, and preprocessing implementations. The paper does not establish whether the observed membership signals persist under standard image representations or alternative preprocessing pipelines. This raises questions about whether the low-noise behavior reflects stable model-level membership information or implementation-dependent numerical effects.

Second, the medical evaluation constructs members and non-members from different datasets, PubMedVision and EndoBench. Appendix E reports that a blind classifier achieves an AUC of 0.954 before filtering, indicating substantial source differences. Although the authors filter the data to reduce this discrepancy, a single ResNet-18-based diagnostic does not fully exclude residual source-specific signals. Independent source-matching controls would strengthen the attribution of the reported AUC improvement to membership rather than dataset characteristics.

Finally, COMPASS benefits from adaptive noise selection, score-orientation calibration, and multiple divergence configurations, while the baselines are evaluated under more restricted settings. For example, Aug-KL achieves an AUC of 0.42 on the medical dataset, which would become 0.58 under score inversion alone. Therefore, part of the reported improvement may arise from calibration rather than the proposed divergence measures themselves. The absence of systematically calibrated baselines and component-wise ablations makes it difficult to isolate the actual contribution of COMPASS.

Questions for authors
  1. All reported target sets use a balanced member-to-non-member ratio. How does COMPASS perform when the actual member proportion is substantially lower, such as 1%, 5%, or 10%?

  2. How sensitive are the selected noise magnitude and score orientation to the reference-set distribution? Can the authors provide controlled experiments using reference sets with different levels of distributional mismatch?

  3. Since calibration uses aggregate divergence statistics from the unlabeled target set, how many target samples are required for reliable noise-level and score-orientation estimation?

  4. The low-noise regime includes perturbations much smaller than one intensity level in the 0–255 image domain. Do the reported membership signals persist after 8-bit quantization, image re-encoding, or changes in preprocessing precision?

  5. What are the separate contributions of noise-level selection, score-orientation calibration, Rényi normalization, and max-k aggregation? Can the authors provide an ablation that progressively introduces these components?

  6. Appendix B.2 reports approximately 12 hours to collect 32 noise-level measurements for a 600-sample LLaVA evaluation on an H200 GPU. How does this calibration cost scale with model size and dataset size, and how practical is the attack when only a small number of candidate images are available?

Reviewer_exFbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes COMPASS, a membership inference attack (MIA) against vision large language models (VLLMs) based on adaptive Gaussian perturbations. The authors observe that the divergence between model outputs before and after image perturbation exhibits different membership signals depending on the perturbation magnitude. In particular, under small perturbations, members can exhibit lower divergence than non-members, reversing the behavior observed under larger perturbations. Based on this observation, COMPASS adaptively selects the perturbation magnitude and membership-score direction by comparing an unlabeled target set with a known non-member reference set. The authors introduce two variants, COMPASS-KL and COMPASS-R, based on Rényi-normalized KL divergence and Rényi divergence, respectively, combined with max-k token aggregation. Experiments on LLaVA v1.5-7B, MiniGPT-4, and HuluMed-32B demonstrate improvements over the selected baselines, achieving up to 0.88 AUC on general-domain data and 0.64 AUC on medical images.

Critical strengths
  1. The finding that membership signals can reverse direction across different perturbation magnitudes is interesting. The systematic analysis of the low-noise regime provides useful insights into how VLLMs respond to small input perturbations. The results also suggest that previously considered perturbation magnitudes may not be optimal for membership inference.
  2. COMPASS provides a practical way to select perturbation magnitude and score orientation without access to labeled members. The decomposition of the observed calibration gap into a membership-dependent component and a reference-distribution mismatch term provides a useful explanation of the method.
Critical weaknesses
  1. A major concern is whether the membership signal can be attributed to training-set membership rather than differences between the member and non-member distributions. In Section 6.1, the instruction-tuning evaluation uses COCO-2017 images as members and recent Flickr images as non-members. These images originate from different datasets and potentially have different visual characteristics. The authors recognize this issue for the medical evaluation and introduce a blind-classifier filtering procedure in Appendix E. However, the author does not provide evidence for COCO-2017 experiment. This is a known issue in the MIA community as early as 2024. [1]
  2. This paper combines several design choices: perturbation-magnitude selection, score-orientation calibration, Rényi-based divergence measures, and max-k aggregation. How much improvement comes from each component? This paper only analyzes some of these choices. Also, this paper does not report how these hyperparameters would be chosen in a practical attack without membership labels, reporting the best configuration for each evaluation dataset will overclaim the method's effectiveness.

[1] Blind Baselines Beat Membership Inference Attacks for Foundation Models. arxiv: 2406.16201.

Questions for authors
  1. How are α and k selected in different tasks? Can the authors provide comparisons between fixed v.s. adaptive perturbation magnitudes, fixed v.s. adaptive score orientations, and ordinary KL v.s. Rényi-based divergences?
  2. The calibration procedure seems unrealiable. Appendix D derives Δ_tune(σ) = π Δ_membership(σ) + δ_R(σ). All reported experiments use balanced target sets with π = 0.5. This is a favorable setting but what if π\pi is small (fewer members in the target sets)? In this case the calibration signal becomes weaker. Could the author provide experiments for π < 0.5, such as π = 0.1?
  3. See critical weakness 1 above.
Reviewer_Ts2DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a new membership inference attack for vision large language models (VLLMs) that consists of comparing the softmax logit per-token distributions of a given input image and a Gaussian perturbed version of it. The proposed method combines ingredients from previous state of the art works: Li et al.'s MaxRényi-K% (single-pass Rényi entropy, top-k positions) and Aug-KL (original vs. augmented image, Liu et al.). The authors systematically characterize VLLM membership behavior under controlled Gaussian perturbations across the full range of noise levels at fine resolution. COMPASS achieves better MIA performance in three different experimental settings (LLaVA v1.5–7B, MiniGPT-4, and HuluMed-32B, spanning pretraining, instruction-tuning, and medical-image datasets), but introduces new parameters and a score/model selection that are not fixed across settings.

Critical strengths

The paper studies and characterizes VLLM membership behavior under controlled Gaussian perturbations. They discover a promising behavior regime that was not found in previous works that did not explore full noise sweeps and neglected the low-noise regime.

Critical weaknesses

Unclear methodology

  • Threat model: line 172 says "We assume a black-box setting, in which the attacker has access only to the input and the output logits of the model."; line 146 says "COMPASS is more analogous to label-only MIAs, where the adversary operates in a black-box setting and observes how the output changes under input perturbation." I see logit access in VLLMs as white-box because I don't see any practical scenario in which you can get logits without owning the model weights, although previous work (Li et al.) calls it grey-box. Clarify what black-box means in the context of COMPASS threat model.

  • The best configuration is highly dependent on the dataset/model: α\alpha and kk are found by sweeps, and there is no rationale for choosing those parameters before knowing the actual results. In fact, best values are different in different settings/datasets. Same for the choice of KL vs. Rényi divergence (Flickr vs. Endoscopy). In Figure 1 (right panel), regime II is best for Flickr/LLaVa, but line 262 says "in some settings, regime III yields better separation". All these need a proper ablation study.

  • Reference-Guided Noise and Score-Orientation Calibration: the attack score orientation sign and perturbation magnitude are set by comparing the target set mean divergence with a known non-member set. That means their selection depends on the member ratio π\pi in the target set. The paper does not ablate this in detail. Line 466 says "while the scale of the Δ\DeltaD drop is reduced by the member–non-member ratio π\pi, the estimate still yields a σ\sigma that lies within the trough of the minimum"; but authors don't specify the actual π\pi or study what happens when π\pi shrinks.

OOD confounds and filtering

Line 331 say "Motivated by blind-baseline analyses showing that source differences can confound MIA evaluation, we apply the filtering procedure described in App. E", but this filtering is only done in the Endoscopy data, and the filtering used is quite weak in my opinion. A frozen ResNet-18 plus logistic regression keeps the hardest 30% of each class, so the blind AUC drops from 0.954 to 0.465 for this model. But what would happen if we use stronger models (including the target model visual encoder) instead of ResNet? What about a non-linear classifier on top of the extracted features? ResNet-18 might capture only visual source cues learned from its training data, and member vs. non-member features might be only discriminated non-linearly. The same blind checks must be done on COCO vs. Flickr as a diagnostic.

Questions for authors

Authors must clarify the threat model. Include a justification of whether COMPASS is a black-box method or not, and discuss the practical availability of a distribution-matched non-member reference set.

How to choose α\alpha, kk, and KL vs. Rényi score when target set behavior is unknown beforehand?

How do σ\sigma and ss change as π\pi varies? Is there any evidence that σ\sigma stays in the trough when π\pi shrinks?

Why is the blind-classifier filter applied only to Endoscopy dataset?

Does the post-filter AUC hold with stronger encoders (including the visual encoder of the target model) or non-linear classifiers?

Please run it on Flickr/COCO too; report the AUC on all unfiltered pools used.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.