- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This study introduces MIRAGE, a model agnostic defense strategy to safeguard personal visual content against unatuhorized editing agents, more specifically closed-source commerical APIs via 'false moderation'. Cleverly, rather than attacking the large model itself, they optimizes the perturbations using an ensemble of open-source encoders and global-local views to align benign images with policy-violating concepts, triggering pre-generation safety filters to refuse the edit. Results across 4 commercial APIs on benchmarks show largely increased success rates compared to baselines
Critical strengths
-
I think the paper is valuable in the sense that it focuses on the highly challenging problem of safeguarding against closed-source models (APIs, Gemini, etc.).
-
I am giving a weak acceptance, as there is comprehensive evaluation showing that the method works on commercial closed-source models, and the topic is novel.
-
It is model- and prompt-agnostic, which makes it an even more powerful approach, and it can be scaled to multiple resolutions.
Critical weaknesses
-
The motivation comes from the fact that these models have safety policy checkers and that their representation spaces are close to open-source ones—so where does this intuition come from? I think an analysis of "why" and "how" this transferability happens is lacking.
-
However, I think the blocker for a clear acceptance is the lack of a comprehensive analysis of the transferability between open-source classifiers and safety checkers in LLMs.
-
Image degradation becomes more visible depending on the budget (see 16/255).
-
It relies on only a single policy category, which is sexually explicit content, whereas there are other ones to explore.
Questions for authors
- How did you choose the evaluation set of 48 images?
4 comments
Thank you for the extra experiment you have conducted, I understand how an ensemble helps in transferability, my questions are resolved in that part.
I would like to see some results on other contents if you can provide any (not only sexually explicit content), because it will support the model's generalization capabilities, if you can provide more results on other categories and update the paper, I would reconsider my score.
Thanks
Official Response
We thank the reviewer for their response.
In the table below, we show immunization rates at for four different target categories --- sexually explicit imagery, graphic violence, hate symbols (e.g. a swastika) and copyrighted content (e.g. the likeness of Mario). These are against the usage policies of OpenAI, XAI and Google. The immunization rates using such images as targets are ---
| API | Sexually Explicit | Graphic Violence | Hate Symbols | Copyrighted |
|---|---|---|---|---|
| OpenAI | 100% | 73% | 98% | 98% |
| Gemini | 100% | 27% | 83% | 0% |
| Grok | 100% | 96% | 0% | 29% |
As we see, different providers exhibit different levels of moderation sensitivities across concepts and categories. However our perturbations do trigger refusal across multiple API providers, indicating that they generalize to some extent.
We have also updated our manuscript, adding the extra experiments and analysis from the rebuttal period. The changes are marked in blue.
Let us know if there are any lingering questions that we can answer for you to reconsider your score, we would be happy to provide any further clarifications!
Gentle Reminder
As the response period draws to a close, we are gently following up on our response, and would be happy to provide any clarifications for you to reconsider your evaluation of our work! Thanks again for your insightful review and engagement!
We thank the reviewer for the insightful feedback and address each comment below.
We believe that the success of MIRAGE is due to using an ensemble of diverse image embedding models in our optimization procedure. We provide intuitive and empirical backing for this next.
(Why an ensemble helps) We believe that one of the main contributors to MIRAGE’s empirical success is that we use a large ensemble of open-source models for computing the adversarial perturbations. Past work in adversarial attacks [1][2] has shown that adversarial transfer improves as the perturbation is optimized against more and diverse surrogates. We also see similar results in Fig 5(b) of our manuscript, where immunization rate improves with surrogate size. We conjecture that optimizing against an ensemble leads to more “natural” adversarial examples which can transfer to other classification models.
(Why ensemble of image embedding helps) Prior work in transferability of adversarial examples, e.g., [3], showed that an adversarial example transfers better if (i) the surrogate model is simpler (or smoother); and (ii) the gradients are better aligned between the surrogate and the target models. For (i), our intuition for MIRAGE was that our loss of choice with an ensemble of embedding models is smoother than a loss that involves a white-box image editor or a single white-box moderation model. For (ii), we would attribute a part of this transferability of CLIP-like image embedding to the vast internet-scale data that CLIP models are trained on. It is more likely that the concept of “sexually explicitness” is a fundamental property of the internet-scale data, that is shared across models. We also provide some empirical evidence around this next.
Emprical Evidence - Based on the reviewer's suggestion, we also conduct experiments with the baseline of transferring adversarial examples between white-box and closed-source moderation models. To study this, we run an experiment where we only optimize the adversarial perturbation on ShieldGemma2, an open-source moderation model. We report the immunization rates on SHHQ below
We find that the rate of transfer of these examples from an open to closed source moderation model is much lower than that of the ensemble achieved by MIRAGE. Even Google’s Gemini moderation seems to be quite different from ShieldGemma2.
To further investigate the source of transferability of MIRAGE, we also perform a similar experiment using a single CLIP model (as opposed to an ensemble of embedding models) to optimize against, and the results are the following
These rates are slightly higher than optimizing using ShieldGemma, which is evidence in favor of our hypothesis that attacks using CLIP-like embeddings are more effective, possibly due to the vast training data these models were trained on (point (ii) above). However, the immunization rates obtained are much lower than using ensembles (Fig 5 of our paper) and lends some support to our claim that attacking an ensemble of models leads to better transferability (point (i) above). Further, we note that there aren’t many open-source multi-modal moderation models available, hence we could not conduct an experiment against an ensemble of multiple ShieldGemma like models. These experiments further motivate our choice of using an ensemble of embedding and moderation models for MIRAGE and provide some intuition on what is essential for the attack to transfer to close source models.
In our ablation (Figure 5 (d)) study we choose the target category to be sexually explicit content as well as violent content, and find that the former works better empirically. In our preliminary experiments, we also looked at other categories in OpenAI’s moderation policies including copyrighted content, hate speech and bio-weapons, but we found that these categories were not effective. We will add these findings in the revision.
We balanced the gender and the background in the images (some images contained a white background while others were in a more natural setting). We then randomly selected images while keeping an equal balance across these categories. In particular, the selection of the images was done before running any of our methods on the images. We will make it more explicit in the revision.