acceptodds
Under review as a conference paper at ICLR 2027

Where Does Intervenable Refusal Live in Multimodal LM Derivatives?

Abstract

Open-weight model hubs now ship not only chat checkpoints but a long tail of derivatives—fine-tunes, safety guards, vision–language models (VLMs), audio wrappers, and generative heads—yet residual-stream safety audits still often intervene at a single site, usually the text decoder . We ask a localization question under one shared mean / rank-1 difference-in-means (DiM) extract=intervene protocol, with matched-norm random and empty-rate controls: does causal chat attack lift concentrate at , or do perceptual encoders, projectors, and fusion paths (//) supply a comparable bypass? On the mixed-input settings we stress-test—HADES adversarial images paired with harmful text prompts, and spoken AdvBench via text-to-speech (TTS) or silence—text DiM at raises attack success rate (ASR) while the same mean DiM at // does not; attacks track harmful text rather than image or audio content. Margin-rank and leverage probes on a Vision leaf show that this recipe is extremal at and near chance at /, which pressures a weak-extractor reading of those flat cells for the DiM family we study. Catalog breadth and mapper-free base→derivative transfer when residual widths match provide supporting coverage under the same gates; generative U-Nets are a separate band under generation judges. The primary claim is the vs. non-text asymmetry; an intervention-oriented Class×site ledger records where this protocol still hooks as supporting coverage, not as a universal multimodal bypass.

Primary area:
interpretability and explainable AI
Keywords:
refusal directions, residual interventions, multimodal LMs, localization, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pG8dOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper asks where refusal is causally intervenable in multimodal language-model derivatives. Using the same rank-1 difference-in-means refusal intervention across model components, the authors find that removing the refusal direction in the text decoder strongly increases harmful compliance, whereas analogous interventions in perceptual encoders, projectors, and fusion modules generally have little effect. They therefore conclude that, under this specific linear intervention protocol, refusal is primarily accessible at the text decoder rather than modality-specific components. The paper also studies transfer of refusal directions across related model derivatives and provides a broader taxonomy of intervention sites.

Critical strengths
  • The paper asks a clear and practically relevant localization question: whether refusal remains causally intervenable at the text decoder or also at modality-specific components such as encoders, projectors, and fusion modules.

  • The experimental protocol is relatively well controlled. The same rank-1 DiM intervention is applied across sites, with matched-norm random directions and additional controls, making the comparison between decoder and non-text locations more interpretable.

  • The decoder interventions produce large causal effects across several model types and modalities, while the non-text interventions remain consistently weak. This provides convincing evidence of site asymmetry within the tested intervention family.

  • The authors are appropriately cautious about the negative non-text results. They explicitly acknowledge that failure of a rank-1 DiM intervention does not prove that refusal-related information is absent from those representations, since it could be nonlinear, multidimensional, or reintroduced downstream.

  • The broad evaluation across VLMs, audio models, fine-tuned models, guards, and other derivatives provides useful evidence that the observed decoder-level intervention is not limited to a single architecture.

Critical weaknesses
  • The central finding is somewhat unsurprising from an architectural perspective. Perceptual encoders primarily transform modality-specific inputs into representations for downstream processing, while refusal is ultimately expressed through the language-generation pathway. It is therefore not obvious that one should expect a clean refusal direction to already exist in the encoder or projector.

  • The paper’s localization claim is limited by the intervention family it studies. Failure of a rank-1 mean-difference direction at E/P/F does not show that refusal-relevant information is absent there. It may instead be encoded nonlinearly, distributed across multiple directions, or only become linearly accessible later in the model. The paper acknowledges this, but it substantially narrows the interpretation of “where refusal lives.”

  • Relatedly, the title and framing can be read as stronger than the evidence supports. The experiments establish where this particular linear DiM intervention is effective, rather than where refusal is fundamentally represented or computed.

  • The comparison between decoder and non-text sites may favor the decoder because rank-1 refusal directions were originally developed for language-model residual streams. Applying the same simple mean-difference construction to encoder and projector representations may not be an equally expressive test of intervenability at those sites.

  • The mixed-input experiments are not maximally diagnostic of modality-specific refusal. In settings such as HADES with harmful text paired with images, the text itself already contains the harmful request, so it is not surprising that the dominant refusal handle appears in the text decoder. Stronger evidence would come from cases where harmful intent must be inferred primarily from the non-text modality.

  • The paper is broad but somewhat diffuse. The taxonomy spans many derivative classes, while the strongest mechanistic evidence is concentrated in a smaller subset of models and sites. Some additional components, such as generative U-Net erasure, feel only loosely connected to the central refusal-localization question.

Questions for authors
  1. Could refusal-relevant information at E/P/F be encoded in a nonlinear or multidimensional form that is missed by the rank-1 mean-difference extractor? Have the authors considered stronger extractors such as multi-direction subspaces, SAEs, or nonlinear/manifold-based steering before concluding that these sites are ineffective?

  2. How fair is the comparison between T and non-text sites given that rank-1 refusal directions were originally developed for decoder residual streams? Could the protocol itself be better matched to the geometry of T than to encoder or projector representations?

  3. Could the authors evaluate cases where harmful intent must be inferred primarily from the image or audio modality, rather than being explicitly present in the text prompt? This would provide a stronger test of whether modality-specific components contain causally relevant refusal information.

  4. How does the refusal direction evolve across layers? A layerwise analysis of linear separability or intervention strength could help distinguish between refusal information being absent early versus gradually becoming linearly accessible downstream.

  5. Can the authors clarify whether the weak non-text results are best interpreted as evidence about representation, computation, or merely intervention accessibility? These are conceptually different claims, and the current framing sometimes appears to blur them.

1 comment
AuthorsOpenReview ↗

Response to Reviewer pG8d (accessibility framing + Q1–Q5)

Dear Reviewer pG8d,

Thank you for the architectural critique. We agree the evidence speaks to causal accessibility under this linear DiM family, not where refusal is fully represented or computed.

Q1. Rank-1 mean DiM failure at E/P/F under the paper OP does not rule out nonlinear/multidimensional encodings; SAE/CAA remain limitations. Manuscript: Abstract / Introduction / Conclusion restate accessibility (not absence); Emp Visual-harm control (VLGuard) shows raising λ_mm is not a free “E works” card—λ=8 collapses; λ=4 mean-pool n=200 coherent LG Δ+0.197 while max does not. Reading: scale- and quality-sensitive accessibility, not universal absence.

Q2. Protocol may favor decoder geometry. Manuscript: Emp Visual-harm control (VLGuard)—at paper OP, max pooling does not rescue @E; within-E matched edit leaves DiM small (0.035→0.075).

Q3. Visual-harm / multimodal diagnosis. Manuscript: Emp Visual-harm control (VLGuard)—@T 0.030→0.150; @P/@E small at λ_mm=2. Introduction (Findings) and Conclusion treat this as directionality at modest absolute ASR.

Q4. Layerwise evidence stays scoped to what the paper reports; early negatives ≠ absence. Manuscript: paper OP remains L10 λ=4 in Emp Visual-harm control (VLGuard). Mid-layer L11 0.030→0.135 is a comment-only check (does not beat L10).

Q5. Best reading. Manuscript: Title, Abstract, and Conclusion now state intervention accessibility under mean/rank-1 DiM (OP- and quality-qualified), not full representation/computation; Class ledger / transfer / U-Net remain supporting.

We present a controlled, recipe-scoped result under multimodal inputs—not “no upstream safety features.”

Reviewer_31RQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper audits mean/rank-1 difference-in-means refusal interventions across open-weight language-model derivatives, comparing text-decoder and non-text sites. The authors report that text-derived decoder interventions increase harmful-compliance rates in the tested text and mixed-input settings, while the tested perceptual-site recipes do not produce comparable gains. For Vision HADES, decoder ASR rises from 3.4% to 61.6%, versus 2.4% for the random-direction control. The Class-by-site ledger and donor-transfer studies show that the protocol remains applicable to multiple derivatives, with qualified transfer between compatible models. The claimed asymmetry is limited to these tested recipes, not universal absence of non-text refusal features.

Critical strengths
  • The question in Section 1 is worth investigating: audits developed for one language model may not remain applicable after multimodal extensions or architectural changes. The DiM extractor is established; the interesting contribution is systematically mapping its applicability and transfer across derivatives. This gives the work value as an audit study rather than a new extraction method.

  • Figure 3's derivative coverage and three text benchmarks are substantial for a protocol-level audit. Table 2's negative-site and modality controls, together with Table 10's qualified transfer results, go beyond a single-model demonstration. They support the observed recipe-level asymmetry, but should not be taken as resolving the stronger causal-localization claim.

Critical weaknesses
  • Section 4 and Table 1 change the contrast variable and token pooling along with the intervention site. Text DiM contrasts text prompts, whereas non-text DiM contrasts perceptual inputs under a fixed prompt. Their asymmetry compares different recipes, not an isolated site effect, even within the acknowledged DiM-family scope.

  • Equation (3) gives an edit norm of ∣lambda∣|lambda| times the activation projection magnitude; unit-norm directions do not imply matched intervention budgets. Figure 2's margin ranks mix intervention size with output sensitivity, and its displacement uses the text DiM axis. These diagnostics do not rule out weak extraction at non-text sites.

Questions for authors
  • How is the non-text contrast shown to engage refusal rather than perceptual-content differences? Which part of the reported asymmetry can be attributed to site rather than to changed contrasts and pooling?

  • What are the actual edit distributions for DiM and random directions at each site? Do edit-normalized margin changes agree with natural-generation ASR on matched inputs?

  • Please reconcile the HADES extraction roles in Section 6 and Tables 3–4, and identify the donor and run behind Figure 3's Mixtral cross result.

1 comment
AuthorsOpenReview ↗

Response to Reviewer 31RQ (recipe vs site + Q1–Q3)

Dear Reviewer 31RQ,

Thank you for the precise diagnosis. We agree the asymmetry is recipe-scoped within the DiM family (contrast/pooling change with site), not an isolated “location-only” effect.

Q1. Non-text DiM = perceptual harm vs benign under a fixed probe; we do not claim it isolates refusal independent of perception. Manuscript: Abstract / Introduction state the recipe-scoped boundary; Emp Visual-harm control (VLGuard) reports that at paper λ_mm=2, @E stays small under max pooling (Δ−0.010 vs mean +0.030, n=200 LG).

Q2. Unit-norm ≠ matched edit budget. Manuscript: Emp Visual-harm control (VLGuard) reports extract-time |a⊤v̂| (DiM ≫ random at equal λ) and within-E matched-edit ASR (DiM@λ=2: 0.035→0.075; matched-budget random → 0.32)—equalizing edit size does not turn mean DiM@E into a strong attack. The intervene update in Sec. Text vs. non-text refusal directions already distinguishes λ=1 zeroing from λ>1 over-ablation. Cross-site T↔E edit matching (d=4096 vs 7680; flat at tiny λ_E) is exploratory and given here only—not claimed as a PDF result.

Q3 (HADES / Mixtral). Manuscript: unchanged facts in the appendix split / eval tables and Mixtral cross discussion (we clarify them here rather than redefine):

  • Chat extract: AdvBench [0:128] / eval [128:520]; HB/SR/HADES/TTS never enter text DiM extraction.
  • HADES ASR: Vision test500 evaluates text DiM@T; dual T+P uses a different pack (≠ HADES test500).
  • Mechanism leaf: disjoint HADES extract/eval for margin/leverage—not the AdvBench extract pool.
  • Mixtral Fig. 3 cross = family donor. Appendix: Mistral→Mixtral R=0.630, η=1, Eff; LlamaB→Mixtral R=0.036 (random band), Ineff. Self Mixtral Adv392 is 0.03→0.86.

VLGuard (shared numbers). Emp Visual-harm control (VLGuard) also records @T Δ+0.120 at paper OP and the honest λ=4 mean coherent lift (not hidden).

Happy to prioritize any remaining diagnostic.

Reviewer_uDk5OpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies where refusal-related intervention signals reside in multimodal and derivative language models. The authors propose a unified extract-and-intervene framework based on rank-1 DiM refusal directions and evaluate whether causal jailbreak effects concentrate at the text decoder (T) or also emerge at non-text components such as perceptual encoders, projectors, and fusion modules (E/P/F). Across various model derivatives and multimodal settings, the paper finds that text-decoder interventions can significantly increase attack success rates, while applying the same DiM-based intervention protocol at non-text sites generally fails. The authors further provide a broad taxonomy of model derivatives and analyze cross-model transfer of refusal directions. The paper presents an extensive empirical investigation of refusal-direction interventions across diverse model variants. However, the main conclusion appears to be narrower than the framing suggests: the experiments primarily demonstrate that rank-1 DiM interventions are effective at text decoders but ineffective at non-text components, rather than establishing where multimodal refusal mechanisms fundamentally reside. The connection between the experimental findings and the broader multimodal safety claim therefore requires further justification.

Critical strengths
  1. The paper investigates an important question for LLM/VLM safety auditing: whether existing refusal-direction interventions remain effective after models are extended with different architectures, modalities, and derivative training pipelines. Understanding where safety-related representations remain causally accessible is valuable for both interpretability and safety evaluation.
  2. The paper attempts to establish a unified evaluation protocol using the same DiM extraction and intervention procedure across different model classes and sites. The inclusion of matched-norm random controls, extract/evaluation splits, and quality gates strengthens the reliability of the empirical analysis. In particular, the comparison between text decoder interventions and non-text interventions is a useful empirical observation, and the authors appropriately acknowledge that negative results at E/P/F do not necessarily rule out other feature representations.
  3. The paper evaluates a relatively diverse collection of model derivatives, including fine-tuned models, safety models, reasoning models, multimodal models, audio models, and other architectural variants. The derivative taxonomy and cross-model transfer analysis provide a useful overview of where existing refusal-direction methods remain applicable.
Critical weaknesses
  1. The comparison between intervention sites does not fully isolate the effect of location. Text DiM is extracted using harmful-versus-benign text prompts, whereas non-text DiM uses harmful-versus-benign perceptual inputs with a fixed text probe. The corresponding pooling strategies also differ across locations. Consequently, the observed performance gap may reflect differences in feature-extraction quality, in addition to differences in intervention location. Although the random-direction controls and additional diagnostic analyses provide useful evidence, they do not fully disentangle these factors. More controlled comparisons or additional diagnostics would strengthen the central claim.
  2. The evidence for multimodal-specific insights remains limited. Although the paper is framed around multimodal model derivatives, a substantial portion of its broader evaluation concerns text-only derivatives. These experiments support the applicability of the intervention protocol but do not directly establish claims about multimodal architectures. Furthermore, some multimodal attack settings combine harmful text with images or audio, making it difficult to determine whether the observed effects reveal multimodal-specific behavior or primarily reflect the effectiveness of text-based refusal interventions. A clearer separation between these claims, supported by more systematic modality controls, would strengthen the paper.
  3. The relationship between the central insight and the extensive experimental coverage is insufficiently clear. The paper combines intervention localization, derivative-model audits, cross-model transfer, intervention-recipe comparisons, and generative-model experiments. While these experiments share a common framework, their contributions to the central localization finding vary considerably. In particular, the broad derivative-model audit provides useful coverage but offers limited additional explanation for why the tested intervention behaves differently across model locations. A more focused narrative and a clearer articulation of the paper's incremental scientific contribution would improve its overall coherence.
  4. The uncertainty surrounding some key results could be characterized more thoroughly. The paper uses multiple evaluators and predefined effectiveness criteria, which are valuable safeguards. However, evaluator disagreement and the uncertainty of results near decision thresholds may affect the interpretation of marginal outcomes, particularly when distinguishing effective from ineffective interventions. Reporting uncertainty for the central comparisons and assessing the sensitivity of key conclusions to evaluation thresholds would make the empirical findings more robust.
  5. The presentation is difficult to follow. The paper introduces numerous abbreviations, model categories, intervention variants, and outcome labels. The central localization result is interleaved with extensive supporting experiments, making the main argument difficult to identify and follow. A clearer separation between the primary findings and auxiliary audits would improve readability and help readers understand the contribution.
Questions for authors

See in weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer uDk5 (claim scope + VLGuard + fairness)

Dear Reviewer uDk5,

Thank you for the careful review. We agree the submitted framing invited an over-read (“where refusal lives”).

W1 (site vs extraction). Agreed that text vs perceptual DiM change contrast and pooling with the site. Manuscript: claim-boundary rewrite in Abstract, Introduction (Main question), and Conclusion; Emp paragraph Visual-harm control (VLGuard) adds mean vs max pooling at λ_mm=2 (LG Δ+0.030 vs Δ−0.010), extract |a⊤v̂| vs random, quality-gated scale (λ=8 discard; λ=4 mean coherent 0.035→0.232), and threshold sensitivity (TΔ>EΔ under LG/Nemo/HBC/Nemo∧HBC/majority-2). Flat @E at the paper OP is therefore not explained by mean dilution alone; we claim paper-OP asymmetry, not “@E inert at every λ.”

W2 (text-origin harm / multimodal diagnosis). Agreed HADES is multimodal-input stress with harmful text. Manuscript: Introduction (Findings) reframes HADES; Emp Visual-harm control (VLGuard) reports @T 0.030→0.150 (Δ+0.120) vs small @P/@E at λ_mm=2. Absolute ASR remains modest—directionality, not a HADES-scale win. (Mid-layer L11 0.030→0.135 is a comment-only check; it does not beat L10, so the paper OP is unchanged.)

W3–W5 (breadth / readability). Manuscript: Title retitled away from “lives”; Abstract / Introduction / Conclusion center intervention accessibility; Emp lead now maps intervenable DiM under a shared recipe with Class/bake-off as supporting panels. Taxonomy / transfer / U-Net stay supporting coverage.

We welcome which remaining point would most change your assessment.

The core concern in W1 remains unresolved. Although the authors added pooling and scale-sensitivity analyses, the extraction contrasts still differ across intervention sites, making it difficult to disentangle location effects from feature-extraction quality. The new results also show that E can yield non-negligible attack lift at higher intervention strength, albeit with degraded generation quality. Regarding W2, the VLGuard experiment provides additional evidence under visual-harm inputs, but does not establish genuinely multimodal-specific insights. The findings still primarily demonstrate the effectiveness of text-based interventions in multimodal settings. Nevertheless, the revisions improve the clarity and scope of the paper, and I am willing to raise my score from 1 to 2 , while maintaining my reservations about the central scientific claims.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.