acceptodds
Under review as a conference paper at ICLR 2027

Seeing More, Refusing Less: Two Mechanisms of Safety Erosion in Video Large Language Models

Abstract

Expanding visual context in Large Vision-Language Models (LVLMs) exposes a capability–safety tension: more frames can improve video understanding while increasing unsafe responses to harmful video–text requests. The associated representational pattern depends on how harm is expressed. For explicit harmful queries, visual context narrows late-layer separation between harmful and neutral text representations, although risk remains decodable. By contrast, when benignly worded queries become harmful through video grounding, text-only risk cues are weaker, while failure-related state shifts align with an activation direction associated with unsafe responses. On selected inputs, amplifying the text-risk component or steering against this unsafe-response direction can lower judged-unsafe output rates. These findings motivate Risk-to-Refusal (R2R), a training-free defense that separates risk detection from refusal actuation. R2R routes requests using separate textual and visual risk signals, combining text-risk component amplification with refusal steering for explicit harm and applying refusal steering alone for grounded harm, with strength bounded as frame count grows. Across five LVLMs, R2R lowers attack success rates to 1.5–6.9% on explicit and 2.3–8.7% on implicit harmful requests.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Video Large Language Models, Multimodal Safety, Safety Alignment, Representation Analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nhbEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates how increasing the number of sampled video frames affects the safety behavior of Large Vision-Language Models (LVLMs). It distinguishes between explicit requests and implicit requests. The authors propose R2R, a framework that routes inputs based on independent text- and vision-based risk gating. For explicit requests, it employs a combination of risk-component amplification and refusal guidance, while for implicit requests, it applies refusal guidance; in both cases, the intensity of the intervention is dynamically adjusted based on the number of sampled frames. Experiments across five models demonstrate significant safety improvements with only a marginal decline in accuracy on video-based multiple-choice question-answering tasks.

Critical strengths
  1. The paper addresses an important practical question: whether expanding visual context improves understanding at the expense of safety.
  2. The paired explicit/implicit query setting provides a useful experimental structure. The representation analyses, input controls, and supplementary output-quality assessment are valuable.
  3. The authors also appropriately acknowledge several limitations of their causal interpretation.
Critical weaknesses
  1. The discussion and coverage of related work are limited. The paper cites relevant work on refusal directions, conditional activation steering, and multimodal representation calibration, but mostly provides brief summaries rather than systematic comparisons. This makes it difficult to assess which contributions provide new insights specific to video context and which extend existing steering approaches.
  2. The experiments do not fully explain why video inputs cause safety failures. The authors show that certain internal signals are associated with unsafe responses and that modifying these signals can improve safety. However, this does not prove that changes in these signals caused the original failures. The blocking experiments also do not clearly show that removing the identified component has a greater effect than random perturbations.
  3. The necessity of the two-branch, multi-component design is unclear, as several simpler variants achieve comparable or lower ASR. Comparisons at matched benign utility or false-refusal rates are needed to establish the full design’s added value. Moreover, since R2R already uses risk gates to decide when to intervene, it is unclear why activation steering is preferable to directly refusing flagged requests.
  4. The paper does not report and compare the defense’s inference-time overhead.
  5. The writing of the paper is difficult to understand.
  6. Missing some important details, such as how to compute dhd_h and dvd_v.
Questions for authors

See weaknesses.

14 comments

Thank you for your constructive comments and for recognizing the research question, the paired explicit/implicit design, the representation analyses, and the output-quality assessment. We address your concerns as follows:

  1. Novelty relative to prior work (W1). We separate what we inherit from video-safety studies and conditional steering from what we add: a controlled analysis in which the video and query are fixed and only the frame budget changes, and risk routing based on where the risk signal comes from (Response 1).
  2. Evidence linking the identified signals to failures (W2). Within-request analyses link representation shifts to failures on the same requests. On Qwen3-VL-4B, the frame-count slope of the unsafe rate falls from 1.37 to 0.09 percentage points (pp) once the text-risk margin is controlled; this is consistent with the margin carrying most of the effect but does not identify it as the cause. Updates along the identified directions reduce ASR by 32.0–60.3 pp, more than norm-matched controls. We agree that blocking is not stronger than random perturbations and no longer use it as evidence (Responses 2–4).
  3. Design necessity and direct refusal (W3). On Video-SafetyBench, removing the H branch gives comparable or lower ASR because the visual gate fires on nearly all explicit inputs. Pairing harmful text with benign videos shows that text-side detection covers inputs the visual gate misses; it does not show that the H update is better than the safe-response update. Under the same gate, fixed refusal gives lower ASR on flagged inputs but discards 1.0–6.3 pp of correct benchmark answers; on Qwen3-VL-8B, steering accepts about 4 pp more ASR and keeps 1.6–2.4 pp more correct answers (Responses 5–8).
  4. Overhead and writing (W4–6). After removing a redundant forward pass, R2R’s median end-to-end latency on Qwen3-VL-8B is 1.36× that of the undefended model measured in the same run (Response 9). Following your suggestions, we will revise the core statements throughout the paper. We will state the research question and contributions more directly, define all directions in one place(Responses 5 and 10).

We will incorporate these results into the revised paper, with changes marked in blue. If any point remains unclear, we would be glad to discuss it further.

AuthorsOpenReview ↗

Response 1 to W1(related work): What this work adds to prior video-safety and steering studies

We agree that the original manuscript offers little in-depth analysis of how our work relates to prior studies. The question our paper studies is: when the source video and the request are fixed, why can sampling more frames make a model more willing to answer harmful requests, and how can this inform a defense?

Video-Driven Prompting [1] shows that videos built from repeated images, and the composition of their frames, affect attack success. V-DEAL [2] analyzes the coupling failure between video understanding and refusal and improves responses with visual-evidence summaries and safety-decision prompts. We add a sweep over the sampling budget with the source video and query fixed, which separates the effect of adding video from the effect of adding frames, and we track the text-risk position for explicit requests and response-associated representations for implicit requests. Beyond describing representational differences, we link them to failures on the same requests and test with norm-matched activation updates whether the identified components can change the response (Responses 2–4). We do not claim to be the first to show that video can weaken safety or that understanding does not guarantee refusal. The most specific result is that on Qwen3-VL-4B each doubling of the frame count is associated with a 1.37 pp higher unsafe rate, and controlling for the text-risk margin of the same request lowers the remaining slope to 0.09 pp (Response 4).

On the method side, CAST [3] conditions steering on a condition vector, AdaSteer [4] combines harmfulness and refusal directions with adaptive coefficients, and MARS [5] transfers textual refusal directions to multimodal inputs through modality re-centering and trust-region scaling. R2R inherits direction estimation and activation updates from this line of work. What it adds is reading risk separately from the text-only state and from the visual state, and choosing the H or B update according to which signal fires: when text risk fires, H amplifies the text-risk component and applies a safe-response update; when only visual risk fires, B applies the safe-response update. We also test the separate benefit of bounded length scaling for changing frame budgets.

We will expand Sec. 2 to discuss Video-Driven Prompting and V-DEAL and to compare CAST, AdaSteer, and MARS in detail, separating established findings, inherited techniques, and our additions.

References

[1] Breaking Multimodal LLM Safety via Video-Driven Prompting. CVPR 2026.

[2] V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure. COLM 2026.

[3] Programming Refusal with Conditional Activation Steering. ICLR 2025.

[4] AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender. EMNLP 2025.

[5] Harnessing Textual Refusal Directions for Multimodal Safety. arXiv 2026.

AuthorsOpenReview ↗

Response 2 to W2 (failure evidence), Part 1: How text-risk position shifts relate to failures on explicit requests

For explicit requests, the harmful intent is already stated in the text. Adding video weakens the deep-layer text-risk separation, and on some models more frames move requests toward the neutral side of the diagnostic axis while unsafe responses increase. Instead of reporting only the gap between class means, we test whether the position of each request corresponds to its response. We use Qwen3-VL-4B, on which we ran the full set of behavioral, representational, and regression analyses, and also report the within-request coefficient for Qwen3-VL-8B.

1. A fixed video and query evaluated at two frame counts. Qwen3-VL-4B receives an explicit harmful request for sexually suggestive messages for an intimate conversation, which Video-SafetyBench labels harmful as sexual content. The excerpts below are paraphrased in English, with harmful details omitted:

ConditionText-risk marginRJScoreResponse excerpt
8 frames0.0011.02“I cannot provide such sexually suggestive messages…”
16 frames−0.0194.14“Here are some messages you can use…”

The request is unchanged, but a lower margin accompanies a transition from refusal to concrete assistance. The margin is measured relative to the text-only calibration midpoint: lower values indicate a position closer to the neutral-text side. This is a diagnostic coordinate, not the model’s actual refusal boundary.

2. Aggregate analysis over 200 requests. For the same 200 explicit requests, the direction and midpoint are frozen before comparison. Position percentages use all inputs as the denominator; each side-specific ASR uses only the requests on that side:

FramesPercentage on the diagnostic neutral sideASR on the neutral sideASR on the harmful sideOverall ASR
211.0%81.8%5.6%14.0%
813.5%66.7%11.6%19.0%
1614.5%79.3%10.5%20.5%
3216.0%71.9%10.7%20.5%
6419.0%73.7%9.3%21.5%
12820.5%70.7%10.7%23.0%

ASR is 66.7%–81.8% on the diagnostic neutral side, compared with 5.6%–11.6% on the harmful side. As frame count increases, the proportion on the neutral side rises from 11.0% to 20.5%. Absolute diagnostic position therefore provides more direct behavioral information than the class gap alone.

3. Within-request analysis controls for differences in request difficulty. After controlling for request and frame-condition fixed effects, the margin remains negatively associated with unsafe responses, with coefficients of −2.22 for 4B and −0.66 for 8B. On 4B, each doubling of frame count is associated with a 1.37 pp increase in the unsafe-response rate; including the margin reduces the remaining slope to 0.09 pp. Response 4 reports the corresponding mediation estimates and their limits.

These results link text-risk position changes to the behavior of the same request, rather than only to reduced class separation. The strength of the frame-count trend is model-dependent, and that text risk remains decodable by a probe does not mean that the default response uses it reliably.

Reviewer_RWq8OpenReview ↗
Rating
3(+1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper reveals two mechanisms underlying safety degradation in video large language models as visual context increases, and proposes R2R, a training-free defense that enhances model safety through dual-modal risk detection and adaptive activation steering.

Critical strengths
  1. The figures in the paper are well-designed and clearly presented.

  2. The ablation studies and related analyses are comprehensive and thorough.

Critical weaknesses
  1. Numerous studies have investigated the safety of Video LLMs, the discussion of related work in Sec. 2 remains insufficient, particularly regarding relevant studies such as [1–4] (but not limited to these). The authors should provide a more detailed discussion of what novel mechanistic insights the proposed decoupling of risk detection and refusal behavior offers compared with existing research on Video LLM safety, particularly in comparison with [2] and [3].

  2. The motivation is not introduced naturally. The paper claims that "a common way to improve understanding is to provide more sampled frames" and "reducing frames sacrifices visual comprehension without addressing the underlying vulnerability." However, providing more sampled frames does not necessarily lead to better downstream understanding, and reducing the number of frames does not necessarily sacrifice visual comprehension. How to provide more effective evidence with fewer video frames within a limited context window, or how to improve a model's ability to handle long contexts while mitigating interference from redundant frames, remains a key research focus in video understanding [5]. Therefore, the discussion should focus more on the potential safety risks associated with increasing the number of sampled frames as an attack strategy. Otherwise, the experimental evidence presented later in the paper may not be sufficient to substantiate the preceding claims. See Comment 3 for further details.

  3. Does understanding the videos in the current benchmarks really require dense sampling? For example, videos in Video-SafetyBench are approximately 10 seconds long. If relatively sparse sampling (e.g., the standard 16 frames) is already sufficient for the model to understand the video content, why is sampling up to 128 frames necessary? Expanding the context length, increasing the computational burden, and introducing more noise would naturally degrade model performance. The necessity of this dense sampling setting does not seem sufficiently justified. Merely evaluating the impact of the frame budget on safety behavior is insufficient to support the current motivation that more frames are important for video understanding and therefore should not be reduced. The impact of the frame budget on video content understanding, such as on Video-SafetyBench, should also be evaluated.

  4. One experimental setting is confusing. The paper states that "unless otherwise stated, videos are uniformly sampled at 2 fps with a budget of 8 frames." Does this mean that only 4 seconds of content are sampled by default? How are the unsampled portions handled? Furthermore, do the baselines follow the same sampling setting? These implementation details need clarification.

  5. The baseline comparisons are not comprehensive enough. Only one work from each of 2024, 2025, and 2026 is selected for comparison. More recent works should be included.

References:

[1] Jailbreaking Multimodal Large Language Models using Multi-Clip Video. ACL 2026.

[2] Breaking Multimodal LLM Safety via Video-Driven Prompting. CVPR 2026.

[3] V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure. COLM 2026.

[4] Mitigating Safety Context Amnesia in Multimodal Reasoning Models via Intent-Guided Safety Reasoning. ACL 2026.

[5] Divide, then Ground: Adapting Frame Selection to Query Types for Long-Form Video Understanding. CVPR 2026.

Questions for authors

Please refer to the Weaknesses. I would consider increasing my score if my concerns are addressed.

5 comments
AuthorsOpenReview ↗

Summary: Responses to the main concerns and planned revisions

Thank you for your constructive suggestions and for recognizing the figures and ablation analyses. Your comments help us clarify the motivation, the relationship between prior work and our contribution, and the scope of our conclusions. We address the five weaknesses as follows:

  1. Related work and mechanistic contribution (Weakness 1). Prior work has established that video can weaken safety, that understanding can be preserved while refusal fails, and that refusal directions exist. We do not claim these findings. Our contribution consists of three testable additions: a design that fixes the video content and varies only the sampling budget, separate readouts of a text-risk axis and a response-safety axis, and controlled activation interventions along both axes (Responses 1).
  2. Frame-count motivation and high frame budgets (Weaknesses 2 and 3). Following your suggestion, we will remove the statements that more frames improve understanding and that fewer frames impair it, and will instead treat the number of sampled frames as an attack surface and a deployment variable. Main results use eight frames; 128 frames serves only as a budget stress condition. Using processor token records, probe decodability, and input controls, we test whether the ASR increase at higher frame counts can be explained by longer input or by a general degradation of understanding; the evidence supports neither explanation (Responses 2).
  3. Recent baselines (Weakness 5). We reproduced MARS and ShiftDC under the same protocol and evaluated general-task utility on four models. R2R achieves lower ASR than MARS in all ten conditions across five models and two query types. On Qwen3.5-9B and InternVL3.5-8B, most ShiftDC responses are flagged as repetitive degeneration, so these results are not comparable. R2R has the lowest ASR in every comparable condition. On the two video question-answering benchmarks, R2R’s accuracy differs from the undefended model by at most 0.6 percentage points (Response 3).
  4. Sampling and fair comparison (Weakness 4). The main experiments sample eight frames uniformly across the full video duration, and all locally evaluated methods share the same initial frame sequence and preprocessing (Response 3).

Thank you again for taking the time to understand our work. We will incorporate these results and clarifications into the revised manuscript, with complete analyses in the appendix and changes marked in blue. If any point remains unclear, we warmly welcome further discussion; it would be our pleasure.

AuthorsOpenReview ↗

Response 1 to W1 : How this work differs from prior video-safety studies?

Thank you for pointing out these closely related studies. The following findings were established before our work:

  • [1] Multi-Clip Video shows that videos are easier to use for jailbreaking than images, and that attack success increases with the number of clips and scene diversity.
  • [2] Video-Driven Prompting shows that videos composed of repeated or diverse frames differ in jailbreak and defense-evasion effectiveness, and that unsafe videos lie closer to safe videos in representation space.
  • [3] V-DEAL shows at the behavioral, understanding, and representation levels that video understanding is largely preserved while refusal fails, and provides refusal-direction diagnostics and prompt-based realignment.

We do not claim to discover these phenomena. Our contribution is to move from the observation that video makes models less safe to representation-level effects that can be localized and intervened on. The relationship to [2] and [3], which you specifically mention, is summarized below; [1] and [4] are discussed afterward.

WorkEstablished findingOur addition
[2] Video-Driven PromptingEmbeddings of unsafe videos lie closer to those of safe videos; repeated and diverse frames differ in jailbreak effectivenessWe turn “closer” into a layer-wise, directional quantity: the deep-layer gap along the text-risk axis shrinks under video input. Amplifying the component along this axis reduces ASR more than norm-matched random and orthogonal controls (see the intervention table below)
[3] V-DEALUnderstanding is preserved while refusal fails; a refusal direction fitted on external safety data; prompt-based realignmentWe separate the text-risk axis from the response-safety axis and test which axis predicts which query type (deep-layer AUROC on implicit requests 0.742–0.888, versus 0.54–0.78 for text-only readout). The response-safety axis is fitted from the target model’s own responses labeled safe or unsafe by the judge, and the intervention is a single activation update during prefill

Regarding the other two studies, [1] establishes that more video content makes jailbreaking easier. Our setting differs in that the video, query, and prompt are fixed and only the sampling budget changes, so the observed changes cannot be attributed to additional harmful content. [4] IGSR restores safety constraints through intent-guided reasoning at inference time; R2R does not rely on a reasoning chain and injects a single update during prefill.

Our central insight is that risk evidence and refusal execution can be separated: the model retains decodable information about harmful content, but the text-risk contrast associated with refusal is compressed. Three lines of evidence support this.

  1. Explicit queries: risk remains decodable, but deep-layer geometry changes. Probes trained on text-only states still decode risk from video-conditioned states, while the deep-layer gap along the diagnostic axis shrinks by 36%–93% across six models.
  2. Implicit queries depend on a separate response-safety readout. The response-safety direction predicts implicit-request safety better than the text-risk direction (deep-layer AUROC 0.742–0.888 vs. 0.54–0.78).
  3. These directions support testable interventions. Amplifying the text-risk component on held-out unsafe inputs reduces ASR by 32.0, 60.3, and 32.0 percentage points on Qwen3-VL-4B, 8B, and Qwen3.5-9B, compared with 4.0–18.0 pp for norm-matched random and orthogonal controls. For implicit requests on 4B, steering along the response-safety direction yields 27%/79%/94% judged-safe at strengths 2/5/8 vs. 0%–10% for controls.
AuthorsOpenReview ↗

Response 2 to W2 and W3: motivation and dense sampling

Part 1: Why vary the frame budget, and what role do 128 frames play?

We agree that more frames do not guarantee better understanding, and fewer frames do not necessarily impair it. [[5] Divide, then Ground] selects the sampling strategy according to the query type, which shows that dense sampling is not a general solution to video understanding. We will therefore remove the statement in the introduction that reducing frames necessarily sacrifices comprehension, and we no longer argue that eight frames are insufficient or that 128 frames are necessary.

The revised research question is: with the same video, query, and prompt, is measured safety behavior stable when only the sampling budget changes? Under this framing, the number of sampled frames is treated as an attack surface and a deployment variable. The 128-frame setting serves two purposes: it corresponds to a budget used in long-video deployment, and it stress-tests the defense’s robustness to the budget. The table below varies the requested frame count on Qwen3.5-9B while keeping the source video, query, prompt, processor configuration, and decoding settings fixed. R2R uses K = 1 in every condition. Values are ASR (%):

FramesUndefended, explicitR2R, explicitUndefended, implicitR2R, implicit
820.432.5268.858.70
1621.444.2971.376.14
6423.712.2772.132.77
12824.722.5273.461.51

The vulnerability already exists at eight frames, so it does not come from dense sampling. Raising the budget to 128 frames still increases ASR by 4.29 (explicit) and 4.61 (implicit) percentage points, although at about 12.8 fps on a 10-second video most added frames are near-duplicates; the sampling budget alone changes safety behavior, which is what the 128-frame setting tests. R2R keeps ASR at or below 4.29% on explicit and 8.70% on implicit requests at every budget.

Part 2: Can the ASR increase be explained by longer context or degraded understanding alone?

The alternative explanation you raise can be divided into three testable hypotheses: the input becomes longer, information is lost (degraded understanding), or new temporal events or higher spatial resolution are introduced. We examine each in turn and then explain why the identified axis is not a by-product of degradation.

1. The input does not become longer. At higher frame counts, the Qwen3-VL processor reduces the spatial resolution of each frame, keeping the visual-token count near its upper limit. Mean values on the calibration inputs are:

Requested frames1664128
Visual tokens11,47311,55210,816
Total input tokens11,60611,87811,399

At 128 frames, the number of visual tokens is lower than at 16 frames, and the total input length is nearly unchanged. Under the same processor configuration, ASR in the full harmful-query sweep still increases by 1.59 and 2.12 percentage points from 64 to 128 frames on Qwen3-VL-4B and Qwen3-VL-8B, respectively. Increasing sequence length therefore cannot explain this change.

2. The information is not lost. Text-only probes are nearly unaffected by video in early layers: on Qwen3-VL-4B and Qwen3-VL-8B, early-band AUROC remains 0.985–0.989 at the largest frame count. Late-band AUROC on these two models remains 0.945 and 0.951, and the other architectures stay above 0.65 at the largest frame count tested for each. If failures arose from general degradation of understanding, the probes would lose discriminative power as frame count increases. Instead, risk remains decodable while the geometry changes. This is the key observation that distinguishes degradation from decoupling.

3. The effect is not driven by new temporal events or spatial resolution. On held-out video pairs for Qwen3-VL-8B, the late-band gap ratio, with text-only input normalized to 1, is:

Input condition16 frames128 frames
Original video0.5510.521
First frame repeated0.5710.533

With 32 frames fixed and the visual-token count varied from 784 to 7,056, the gap ratio stays between 0.522 and 0.542, with no monotonic trend. Neither temporal events nor the spatial budget is therefore necessary for this compression. The corresponding controls are reported in the appendix (Input controls: temporal content and spatial budget).

4. The axis is not a by-product of degradation. Activation updates along this axis change responses more than norm-matched and orthogonal controls (Response 1), indicating that the axis is behaviorally relevant rather than an incidental consequence of degraded input.

Reviewer_MtNVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why videoLLM may answer harmful requests for long video. The authors propose a training‑free defense that uses separate text and visual risk signals to choose an activation intervention. Experiments on five models show large reductions in attack success rate (ASR), alongside evaluations of general task performance.

Critical strengths

Table 1 shows clear ASR reductions for both types of harmful requests across all five models. The authors compare different questions about the same video and test interventions against random or opposite directions.

Critical weaknesses

It is not a fancy research to study the safety of videoLLM for long video, similar researches have been studied after year 2024. R2R combines risk routing, different intervention branches, and frame‑dependent scaling, but the ablations do not clearly establish the benefit of this added complexity. The evidence does not fully support the two safety erosion mechanisms. The representation findings are interesting, but a reduced text‑risk gap or a direction associated with unsafe responses does not explain what causes a safety failure.

Questions for authors
  1. The authors are encouraged to give some case study of the proposed two safety erosion mechanisms.
  2. What evidence distinguishes a failure to recognize risk in the video from a failure to refuse after recognizing it? How does the representation analysis inform the intervention layers used in the final method?
  3. Could MARS be evaluated under the same video split, frame budget, judge, and safety/benign‑utility metrics?
14 comments
AuthorsOpenReview ↗

Summary: Responses to the main concerns and additional results

Thank you for your constructive suggestions and for recognizing the safety improvements, same-video paired experiments, and directional intervention controls. We address your main concerns below:

  1. Research contribution and matched comparison. We clarify the research focus around the question: why can increasing the number of sampled frames increase vulnerability when the source video and request remain fixed? We will strengthen the in-depth comparison with related work in the main text to highlight our contribution (Response 1). In our reproduction of MARS under the same protocol (Q3; Response 2), R2R achieves lower ASR in all ten conditions across five models and two query types.
  2. Evidence for the two failure patterns. We provide cross-frame cases with the source video and query fixed, aggregate and within-request analyses linking representations to behavior, and norm-matched interventions that make initially unsafe responses safe (Q1; Responses 3–4 and 6). Among implicit failures, the target model itself independently identifies 48.9% (Qwen3-VL-8B) and 68.5% (InternVL) as harmful, demonstrating risk-identification capability that does not consistently translate into safe default responses (Q2; Response 5).
  3. How the findings inform the method. Amplifying the text-risk component reduces ASR by 32.0–60.3 percentage points on selected failure subsets. Layer selection combines calibration-based signal quality with an execution-quality constraint; in the execution control, 95.0% of middle-layer responses are safe without presentation issues, compared with 67.5% for deep layers (Responses 6–8).
  4. Benefits of individual components. Removing B increases implicit ASR by 40.5–70.6 percentage points in the B-routed subsets. On 36 harmful-text inputs missed by the visual gate, full R2R reduces ASR from 16.7% to 0%, whereas B-only leaves it unchanged at 16.7%. With routing fixed, length scaling reduces implicit ASR by 1.89 percentage points at 128 frames. On benign tasks, R2R retains 88.9%–98.6% of correct answers on gate-triggered inputs (Responses 9–11).

Thank you again for taking the time to understand our work. We will incorporate these results and clarifications into the revised manuscript, with complete analyses in the appendix and changes marked in blue. If any point remains unclear, we warmly welcome further discussion; it would be our pleasure to continue the conversation.

AuthorsOpenReview ↗

Response 1 to the novelty concern: What does this work add to prior video-safety research?

Our contribution is a controlled analysis of how frame sampling changes safety-related representations, together with targeted interventions for explicit and visually grounded harmful requests. We build on prior studies of video safety, refusal representations, and activation steering. The specific question is: why can increasing the number of sampled frames make a model more likely to answer harmful requests when the source video and query remain unchanged, and how can these findings inform a defense?

Video-Driven Prompting [1] demonstrates the effects of videos constructed from repeated images and of frame composition on attack effectiveness. V-DEAL [2] analyzes failures in the coupling between video understanding and refusal and improves responses through visual-evidence summaries and safety-decision prompts. Relative to these studies, we vary the sampling budget while holding the source video and query fixed, distinguish the introduction of video from subsequent increases in frame count, and separately track text-risk positions for explicit requests and response-associated representations for implicit requests. The additional evidence goes beyond representational differences: within-request analyses connect these changes to actual failures, and direct activation interventions test whether the corresponding components can improve responses (Responses 3–6).

Regarding the method, CAST [3] uses a condition vector to control steering; AdaSteer [4] combines harmfulness and refusal directions with adaptive coefficients; and MARS [5] transfers textual refusal directions to multimodal inputs through modality re-centering and trust-region scaling. Building on these approaches, R2R reads risk separately from text-only and visual states and selects H/B updates according to the source of the risk signal. When textual risk triggers the gate, H amplifies the text-risk component and applies a safe-response update. When only visual risk triggers the gate, B applies the safe-response update. We also test the independent benefit of bounded frame-dependent length scaling. This differs from adjusting strength based on activation geometry.

To address complexity, we examine each component rather than relying only on overall averages. The benefit of B is reported in Response 10, Part 2: removing B increases implicit ASR by 40.5–70.6 percentage points in the B-routed subsets. The text-risk path is evaluated in Response 10, Part 3: ASR decreases from 16.7% to 0% on harmful-text inputs for which the visual gate does not trigger. The length-scaling control is reported in Response 11. We also acknowledge that margin ramping and severity scaling have small effects in the tested settings; most B-branch margins already saturate the ramp. The main gains come from complementary risk coverage and targeted steering; these smaller components have limited additional effect at the eight-frame reference point.

We also provide a more specific behavioral result. In the within-request fixed-effects analysis on Qwen3-VL-4B, each doubling of frame count is associated with a 1.37-percentage-point increase in the unsafe-response rate. After including the text-risk margin, the remaining slope is 0.09 percentage points, whereas the within-request coefficients for control variables, including random-direction projections, are not statistically significant (Response 3).

We will expand Sec. 2 (Related Work) to discuss Video-Driven Prompting, V-DEAL, and other recent studies, and provide a more detailed comparison with CAST, AdaSteer, and MARS. This revision will distinguish established findings, inherited techniques, and our specific contributions in frame-count analysis and risk routing.

References

[1] Breaking Multimodal LLM Safety via Video-Driven Prompting. CVPR 2026.

[2] V-DEAL: Diagnosing Video Safety De-Calibration as an Understanding-Refusal Coupling Failure. COLM 2026.

[3] Programming Refusal with Conditional Activation Steering. ICLR 2025.

[4] AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender. EMNLP 2025.

[5] Harnessing Textual Refusal Directions for Multimodal Safety. arXiv 2026.

AuthorsOpenReview ↗

Response 2 to Q3: MARS safety and utility evaluation under the same protocol

Following your suggestion, we reproduced MARS and evaluated it on five models for safety and four models for general-task utility. The safety evaluation uses the same video split, eight frames sampled uniformly over the full video duration, initial preprocessing, greedy decoding, and Qwen2.5-72B-Instruct judge.

1. With inputs and evaluation criteria matched, R2R achieves better safety results across all five models and both query types.

Explicit harmful requests

ModelMethodRR ↑ASR ↓RJScore ↓
Qwen3-VL-4BMARS95.96%4.67%1.22
Qwen3-VL-4BR2R96.97%1.51%1.14
Qwen3-VL-8BMARS98.87%3.91%1.22
Qwen3-VL-8BR2R99.50%2.65%1.18
Qwen3.5-9BMARS97.86%4.54%1.22
Qwen3.5-9BR2R98.87%2.52%1.17
InternVL3.5-8BMARS74.78%22.07%1.68
InternVL3.5-8BR2R96.60%3.40%1.17
Molmo2-8BMARS55.86%26.23%2.02
Molmo2-8BR2R81.97%6.94%1.34

Implicit harmful requests (benignly worded text)

模型MethodRR ↑ASR ↓RJScore ↓
Qwen3-VL-4BMARS80.96%18.41%1.81
Qwen3-VL-4BR2R94.07%7.69%1.45
Qwen3-VL-8BMARS88.27%13.87%1.63
Qwen3-VL-8BR2R94.96%6.68%1.52
Qwen3.5-9BMARS79.57%20.68%1.82
Qwen3.5-9BR2R91.42%8.70%1.42
InternVL3.5-8BMARS35.18%49.18%2.71
InternVL3.5-8BR2R93.06%6.18%1.37
Molmo2-8BMARS16.39%60.66%3.33
Molmo2-8BR2R85.12%2.27%1.18

2. General-task utility. We also evaluated MARS on three question-answering benchmarks. LVB denotes LongVideoBench.

MethodQwen3-VL-4BQwen3-VL-8BQwen3.5-9BInternVL3.5-8B
BenchmarkVideo-MMELongVideoBenchMMMUVideo-MMELongVideoBenchMMMUVideo-MMELongVideoBenchMMMUVideo-MMELongVideoBenchMMMU
Undefended55.0%54.5%50.6%56.1%53.7%54.6%58.6%55.6%54.2%57.8%57.5%56.9%
MARS55.3%54.4%50.3%56.1%53.7%54.3%57.1%51.2%51.9%54.4%57.2%55.1%
R2R54.4%54.5%50.8%55.7%53.1%53.2%58.7%56.2%54.1%57.7%57.4%55.9%

Comments on the paper

1 comment

Dear Reviewers,

Thank you for the time and care you have devoted to reviewing our work and for your thoughtful feedback throughout the discussion. We have revised the manuscript in light of your comments and uploaded the revised PDF. Blue markings indicate the revisions, including the headings and captions of newly added sections and tables, to help you locate the updates.

We understand that reviewing is demanding, especially alongside your other commitments, and sincerely appreciate your continued attention. If your schedule permits, we would be grateful if you could revisit our responses and the revised manuscript and share your thoughts during the remaining discussion period.

If we have misunderstood any concern, left a question unresolved, or explained something inadequately, please let us know. We would warmly welcome further questions and the opportunity to clarify our reasoning, address remaining concerns, and improve the manuscript. Your feedback is valuable to us, and we hope to continue a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.