acceptodds
Withdrawn from ICLR 2027

VPA-Guard: Defending and Benchmarking Image-to-Video Generation Against Visual Prompt Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advancements in Image-to-Video (I2V) generation have transformed input images from simple appearance references into interactive control interfaces where visual cues such as arrows, sketches, and emojis orchestrate complex video dynamics with unprecedented controllability. However, these seemingly innocuous static cues can be interpreted by models as executable temporal instructions, unfolding into harmful actions in the generated videos. Despite the severity of this threat, existing safety benchmarks remain predominantly focused on text-based and content-only image-based jailbreaks, leaving implicit visual prompt attacks insufficiently explored. To bridge this gap, we present VVA-Bench, the first systematic benchmark for evaluating video generation safety under categorized vision-centric prompt attacks. Extensive experiments on VVA-Bench demonstrate that state-of-the-art models are highly susceptible to such attacks, with Attack Success Rates (ASR) reaching 100.0% on Wan 2.7 and 74.8% on Veo 3.1. To mitigate these risks, we propose VPA-Guard, a retrieval-augmented and self-evolving defense framework. By leveraging few-shot reasoning to identify latent malicious intents, our method reduces the attack ASR by 44.2% and the harmfulness score by 73.4% on average, while maintaining the model's utility for legitimate user edits. Our work provides both a rigorous benchmark and an effective defense strategy to advance safe and socially responsible multimodal generation.

Primary area:
datasets and benchmarks
Keywords:
Image-to-Video Generation, Multimodal Safety, AI Safety, Retrieval-Augmented Defense

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_eekYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies visual prompt attacks against I2V generation models. Visual cues such as arrows, sketches, poses, and frame transitions can induce unsafe video content. They introduce VVA-Bench, containing 452 examples on six formats for attacks. Authors propose VPA-Guard, a retrieval-augmented defense that uses attack patterns and self-evolving defense framework. VPA-Guard acheieves 44.2% ASR drop on evaluation in four I2V models.

Critical strengths
  1. The paper considers an important safety problem in I2V generation. The benchmark covers several visual control mechanisms, including temporal transitions, which are not adequately captured by text-only safety evaluations.

  2. The evaluation covers multiple commercial models and includes comparisons on text v.s. visual, controls on benign samples, and human calibration of the LLM-as-a-judge.

  3. The paper is well-written and easy-to-follow.

Critical weaknesses
  1. I have a serious concern about the ASR calculation. In Figure 4, the reported ASRs appear to be exactly 100100%-P(H_V=0), where HV=0H_V=0 means refusal. However, Table 10 explicitly defines HV=1H_V=1 as Safe and HV=2H_V=2 as Borderline Safe. For example, Wan 2.7 reports 100% ASR, where Figure 4 shows that more than 50% of its outputs fall into these two safe categories. This suggests that the paper may be counting any non-refusal as a successful attack, without checking whether the generated video is actually harmful. This directly contradicts the stated ASR definition and calls the main experimental conclusions into question. I also find the custom harmfulness score in Appendix F.2 difficult to justify, since even safe outputs can receive substantial positive scores.

  2. The novelty and defense evaluation are marginal. Visual instruction attacks on I2V models have already been studied in recent work like Visual Instruction Injection [a]. The proposed defense mainly combines standard embedding retrieval, few-shot safety prompting, and database updates. Furthermore, there is no convincing comparison against simple safety prompting or direct VLM-based safety screening. The current ablation does not isolate the contribution of retrieval or rationale prompting.

Ref: [a] Zheng et al. VII: Visual Instruction Injection for Jailbreaking Image-to-Video Generation Models.

Questions for authors
  1. How exactly is ASR computed? Why does it appear to equal the non-refusal rate in Figure 4? Could the authors recompute the main results using actual harmful outputs, for example HV≥3H_V\geq 3?

  2. How much improvement remains when comparing VPA-Guard against a simple safety prompt or a direct VLM safety classifier under matched benign rejection rates?

Reviewer_MyLkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Existing video-generation safety evaluations mainly focus on explicit text or harmful image content, overlooking visual cues that imply harmful actions over time. This paper introduces VVA-Bench to evaluate attacks conveyed through arrows, poses, sketches, and related visual controls. It further proposes VPA-Guard, which retrieves attack examples and uses their rationales to identify unsafe intent, while updating its database after failures. Experiments show that current image-to-video models are vulnerable to these attacks and that VPA-Guard reduces attack success and output harmfulness.

Critical strengths

1 An important attack surface. This paper examines how apparently benign static cues can specify harmful temporal actions, extending safety evaluation beyond explicit text and visible harmful content.

2 Diverse attack mechanisms. VVA-Bench covers several forms of visual control and includes benign requests using similar formats, supporting evaluation of both attack success and false rejection.

3 Effective defense on the tested attacks. Retrieval and rationale-guided screening reduce attack success across the evaluated generators, while database updates help the defense respond to additional attack patterns.

Critical weaknesses

1 The visual bypass effect is not fully isolated. Explicit text instructions already achieve a high attack success rate in the matched comparison.

2 The generalization of the defense is insufficiently established. Initial prototypes and evaluation attacks share the benchmark taxonomy, and the split does not clearly exclude related templates or visual styles. Retrieval may therefore recognize familiar attack patterns without learning to identify unsafe intent in genuinely new visual mechanisms.

3 The self-evolution evaluation needs stronger separation between adaptation and testing. The database incorporates failures encountered during the evaluation stream. Improvements may depend on attack order or similarity to earlier cases, so the reported gain does not establish how well the updated defense transfers to attacks that never contributed to its database.

4 The reliability of the harmfulness evaluation remains limited. The automated judge has only moderate correlation with human judgments of generated videos. This makes small differences in attack success or severity difficult to interpret, particularly when the defense may alter output appearance in ways that also affect the judge.

Questions for authors

1 For how many matched cases is the explicit-text instruction refused while its visual counterpart succeeds? 2 Could the authors evaluate held-out visual mechanisms and report performance before each database update, across different attack orders and on a separate final test set? 3 Could the authors provide human-audited attack success rates and uncertainty estimates for the main defense comparisons?

Reviewer_tgVyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates implicit visual prompt attacks against image-to-video (I2V) generation models. The authors introduce VVA-Bench, comprising 452 unsafe samples and 126 benign controls across six visual-control formats. They further propose VPA-Guard, a retrieval-augmented defense with failure-driven database updates. Experiments on multiple I2V models report an average relative reduction of 44.2% in ASR and 73.4% in harmfulness score.

Critical strengths

S1. Important safety problem. The paper addresses underexplored risks arising from visually encoded action instructions in I2V generation.

S2. Structured benchmark. VVA-Bench covers six visual-control formats and seven safety categories, with 126 benign controls for measuring over-refusal.

S3. Broad empirical evaluation. The experiments cover multiple commercial and open-weight generators, including adaptation ablations and human-judge calibration.

Critical weaknesses

W1. Insufficient causal isolation of visual instructions. Table 13 changes both image and text between conditions. Without a clean-image baseline using the same weak text, the independent contribution of visual cues remains unclear.

W2. Ambiguous success criteria for two-frame attacks. Section 2.1 allows the final conditioning frame to contain harmful consequences. The evaluation does not distinguish newly generated harmful actions from harmful states already present in the inputs.

W3. Limited reliability of automated video evaluation. Table 17 reports only moderate agreement between the automated judge and human consensus for output harmfulness (Pearson 0.546; Spearman 0.560), raising concerns about the reliability of ASR and HS conclusions.

W4. Missing component-level defense ablations. Table 14 isolates database updates but not retrieval or rationale-guided prompting. Comparisons with generic safety prompting, random retrieval, and rationale removal are missing.

W5. Unverified retrieval of fine-grained visual instructions. Global DINOv2 features may overlook small arrows, poses, or directional changes. No retrieval-quality analysis demonstrates that retrieved examples match the underlying attack mechanisms.

W6. Unsafe fallback in the defense implementation. Appendix D.4.1 states that exceeding the approximately 1,800-character budget may cause the system to forward the original prompt without the safety wrapper. The frequency and impact of this fallback are not evaluated.

W7. Insufficient evidence of generalization from self-evolution. Tables 14–16 demonstrate adaptation after observed failures, but do not establish performance on independently held-out attack mechanisms. Instance memorization and overlap between updates and evaluation remain insufficiently controlled.

W8. Limited novelty and baseline comparisons. Related work, including RunawayEvil, Visual Instruction Injection, and IESBench, already investigates related visual safety threats. The defense largely combines existing retrieval and prompting techniques, without sufficient comparisons against simpler safety baselines

Questions for authors

Q1 (W1). What is the ASR when visual marks are removed while keeping the weak text unchanged? How often do visual cues turn otherwise safe outputs into violations?

Q2 (W2–W3). How are newly synthesized harmful actions distinguished from harmful input states in two-frame attacks? Can human-verified ASR and agreement statistics be provided?

Q3 (W4–W5). How does VPA-Guard compare with generic safety prompting, random retrieval, and retrieval without rationales? Can retrieval accuracy be evaluated on identical scenes with different visual instructions?

Q4 (W6). How often does the character-budget fallback occur, and why does it discard the entire safety wrapper rather than truncate retrieved examples?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.