acceptodds
Withdrawn from ICLR 2027

MiragePoison:Few-shot Poisoning Attacks on Vision-Language Models for AIGC Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language Models (VLMs) have demonstrated strong multimodal understanding capabilities and are increasingly adopted for AI-generated content (AIGC) detection. However, their reliability in such security-sensitive scenarios, particularly under data poisoning during fine-tuning, remains underexplored. In this work, we systematically evaluate mainstream VLMs for AIGC detection and investigate their susceptibility to fine-tuning-based poisoning. We propose MiragePoison, a few-shot poisoning framework that constructs poisoned samples visually consistent with AIGC content while shifting their latent representations toward those of real images. Fine-tuning with only a small number of such samples systematically induces real images to be misclassified as AI-generated. On FakeVLM, MiragePoison achieves a 98.2% R2F-ASR with only 100 poisoned images, corresponding to merely 0.00000143% of the model parameter scale, demonstrating that minimal malicious supervision can substantially alter authenticity judgments. We further characterize poisoning-budget effects and compare the sensitivity of the Vision Encoder, Projector, and LLM. These findings show that limited malicious training data can significantly compromise VLM-based AIGC detectors and expose structural risks in their fine-tuning process.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Vision-Language Models, AI-Generated Image Detection, Data Poisoning, Few-Shot Poisoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iYquOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MiragePoison, a training data poisoning attack for AIGC detection models to induce real images to be misclassified as AI-generated. It optimizes the real image to maximize its cosine similarity to fake images while minimize its cosine similarity to real images in feature space, and preserves its L∞L_\infty budget. The poisoned image therefore looks like a real image but is similar to fake images in feature space. Then in the training stage, the AIGC detection model learns to classify real images as AI-generated.

Critical strengths

The paper is very clear and easy to follow. The idea of poisoning AIGC detection models is very interesting.

Critical weaknesses

(1) The contribution of this paper is very limited. The main idea of this paper is to optimize images to change its feature representation. This is very similar to adversarial attacks [1] on image classification models. I think the main contribution of this paper is limited to applying adversarial attack/examples to the AIGC detection model. However, the key component of the optimization process is not largely changed.

(2) Some baselines in the experiments are flawed. The goal of the proposed attack is to induce the model to classify real images to fake by training time poisoning. However, the proposed method is first compared to adversarial attacks and backdoor attacks. Adversarial attacks are conducted at inference time, and backdoor attacks can preserve the model performance when the trigger is absent. They have different attack goals and assumptions. The proposed method should be only compared to other poisoning attacks with the same attack goal and budget. No test modify and no trigger should not be regarded as an advantage of the proposed attack.

References

[1] Explaining and Harnessing Adversarial Examples. ICLR 2015

Questions for authors

(1) What is the main difference between MiragePoison and prior works on poisoning/adversarial attacks? Could the authors further clarify their contribution?

(2) How robust is the proposed attack? Can the poisoned images survive any detection methods (e.g., detecting the consistence of image/feature and label) and purifications (e.g., noise/random crop)?

(3) Could the authors further clarify the difference from their poisoning attacks to backdoor/adversarial attacks?

(4) How is the surrogate model selected for the black-box setting? How does the attack perform when the surrogate changes?

Reviewer_DzCrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a trigger-free data-poisoning attack against VLM-based AI-generated-content detectors, namely MiragePoison. MiragePoison starts from AI-generated images with the semantically correct “fake” label, perturbs each image within an L-infinity constraint so that a chosen vision encoder’s pooled feature moves toward a real-image class center and away from a fake-image class center, ranks candidates by this relative similarity, and injects the top-N examples into ordinary supervised fine-tuning. The paper evaluates four VLMs on 2 datasets.

Critical strengths
  1. Poisoning fine-tuning data for AIGC detectors is indeed a plausible threat. The paper uses correctly labeled synthetic images whose features are shifted toward the real class, which is reasonable and well matched to this threat.

  2. The paper tests four VLMs and multiple poisoning budgets. The authors also test transfer from TinyGenImage to FakeClue and compare the effects of different configurations, providing a broader evaluation of the proposed method.

Critical weaknesses
  1. My main concern is the R2F-ASR metric. If the poisoned model predicts all images as "fake", it can also achieve very high or even 100% R2F-ASR. The paper does not report other metrics such as the fake-image accuracy or balanced accuracy. Therefore, it is unclear whether the attack only makes the model prefer the "fake" label instead of attacking successfully.

  2. The paper does not compare with adding the same number of original fake images or randomly perturbed fake images to the fine-tuning dataset. I think the ablation is important since it is unclear whether the attack performance comes from the proposed feature optimization or simply from adding more fake-labeled samples.

  3. Some experimental details are not provided. For example, the paper does not clearly report the size of the clean fine-tuning dataset, the poison ratio, the epsilon value, and some other hyperparameters. Table 3 also compares different types of attacks under different threat models, so the comparison may not be fully fair (the paper should report the fraction and sampling frequency of poisoned data, rather than comparing the number of poison images with the number of model parameters).

  4. The paper describes the poisoned samples as stealthy, but the evaluation is limited. The paper only uses bounded perturbations and pixel-level regularization to demonstrate the stealthiness, but does not report other perceptual similarity metrics (e.g., LPIPS, SSIM). I think these metrics can better demonstrate the claim.

  5. The novelty compared with existing feature-space poisoning methods is not very clear. The method mainly moves the features toward the real-image center and away from the fake-image center, which is reasonable but somewhat limited in novelty.

Questions for authors
  1. Could the authors report other metrics such as fake-image accuracy and balanced accuracy?

  2. Could the authors compare MiragePoison with the ablation versions mentioned in weakness 2?

  3. Could the authors provide more experimental details as mentioned in weakness 3?

  4. Could the authors report perceptual similarity metrics such as LPIPS or SSIM to better support the stealthiness claim?

  5. Could the authors explain the main novelty compared with existing feature-space or clean-label poisoning methods? What are the main benefits of using class-level feature centers instead of instance-level target matching, and could the authors provide more evidence to support this design choice?

Reviewer_gZRAOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies backdoor attack on new task: using vision language model to distinguish real image from AI-generated images. The paper proposes the poison data generation method: modify the synthetic image visual representation to increase alignment with real image's visual representation, while maintaining the original fake labels. Then train the model with poisoned data. The model will be successfully backdoored. Experiments on 2 datasets and 4 backbone model architecture shows the effectiveness of proposed attack.

Critical strengths

1.Investigating VLM-based AIGC detection is very interesting. Sort of 'Attacking the attack method', interesting to me.

Critical weaknesses

1.The intuition is similar with some CLIP-based backdoor attack. Paper: BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning. To me, it is more of adapting the backdoor attack to a new task, under VLM scenario.

2.Evidence on proposed method is better than existing baseline is not well established. Table 3 'Comparison with representative attacks under the unified Real-to-Fake setting.', the experiment setting is unclear. Which dataset are you using? And there is only one experiment column under one setting. So the evidence on baseline comparison is not solid.

3.Resistent to defense / detection method is not discussed.

  1. No evaluation of whether the attack performance on clean data / general prediction.
Questions for authors

1.Any insights on: why Qwen2.5-VL-3B is so hard to attack, in table 1/2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.