acceptodds
Under review as a conference paper at ICLR 2027

Narrow Multimodal Fine-Tuning Can Induce Emergent Misalignment

Abstract

Modern AI models are aligned through post-training to adapt them to downstream tasks. Recent work shows that fine-tuning language models on narrow tasks can induce emergent misalignment (EM), causing broadly harmful behaviors beyond the training task. However, EM has been studied almost entirely in text-only tasks, leaving its manifestation in multimodal models unclear. In this paper, we define and analyze EM in the context of vision-language models. We first induce EM via fine-tuning on narrow multimodal tasks targeting vulnerable code, careless household-object use, and conspiratorial interpretations of ordinary scenes. Across fifteen commercial and open-source models with different scales, we find that narrow multimodal fine-tuning can induce a coherent and broad misaligned disposition that transfers to unrelated tasks, including misaligned opinions, visual factual dishonesty, unsafe image generation, vulnerability to visual jailbreaks, and risky agentic actions. We further find that multimodal EM does not depend on the apparent harmfulness of training data but is sensitive to training-evaluation modality alignment. EM can arise under both supervised fine-tuning and preference optimization and can propagate through intermediate reasoning. Finally, we explore several mitigation strategies, including prompt inoculation, benign continued training, and activation-level steering, which can partially reduce EM. Overall, our findings suggest that multimodal EM reflects a behavioral shift rather than a general loss of capability, extending beyond text to the visual modality.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Emergent Misalignment, Multimodal Fine-tuning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XkTXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether narrow multimodal fine-tuning can induce emergent misalignment (EM) in vision-language models. The authors construct three image-text fine-tuning tasks—Insecure Code Completion, Careless Object Use, and Ordinary Scene Conspiracy—and evaluate the resulting models on a broad suite of out-of-distribution safety behaviors. Across various open-source and commercial models, they find that narrow multimodal fine-tuning can lead to coherent misaligned behavior on unrelated tasks, including harmful opinions, visual factual dishonesty under pressure, weakened safeguards for harmful image generation and multimodal jailbreaks, and risky agentic actions. The effect is strongest for Ordinary Scene Conspiracy and Careless Object Use, while Insecure Code Completion produces much weaker broad misalignment. Further analyses show that EM is sensitive to the composition of the fine-tuning data and to training–evaluation modality matching, can arise under both SFT and DPO, and is strongly affected by the supervision of intermediate reasoning traces. In contrast, explicitly harmful training data and simple corruption of visual perception do not necessarily induce stronger EM. The paper also explores mitigation strategies, showing that inoculation prompting, benign continued training, and activation steering can substantially reduce the induced misalignment. Overall, the paper argues that narrow multimodal post-training can reshape a model’s broader behavioral disposition rather than merely affecting the trained task, highlighting a potential alignment risk in multimodal fine-tuning.

Critical strengths
  1. Comprehensive experimental evaluation. The paper evaluates the phenomenon across a broad range of open-source and commercial multimodal models, multiple narrow fine-tuning recipes, and diverse downstream behaviors, including open-ended alignment, visual dishonesty, jailbreak robustness, image generation, and agentic actions. This makes the empirical evidence substantially broader than a single-task or single-model demonstration.

  2. Thorough analysis and ablations. Beyond establishing the main phenomenon, the paper systematically studies several factors that may affect EM, including data composition, harm explicitness, modality matching, training objectives, reasoning traces, adaptation methods, and parameter scaling. The additional mitigation experiments further strengthen the empirical analysis and help rule out several simple alternative explanations.

Critical weaknesses
  1. The distinction between emergent misalignment and ordinary broad behavioral generalization remains conceptually underdeveloped. The paper provides a clear operational definition of EM based on coherent misaligned behavior on out-of-distribution tasks, but it is less clear what additional property makes the observed transfer specifically “emergent.” In particular, some of the strongest effects may plausibly reflect generalization of a learned high-level behavioral disposition or persona rather than a qualitatively distinct emergent phenomenon. A clearer conceptual criterion would strengthen the paper’s framing.

  2. EM is highly dependent on the specific fine-tuning recipe, but the paper does not yet provide a predictive characterization of which narrow multimodal tasks induce it. Ordinary Scene Conspiracy produces strong broad EM across models, while Insecure Code Completion often has only a small effect. Although the extensive ablations rule out several simple factors, they do not isolate the properties that distinguish strongly EM-inducing tasks from weak or non-inducing ones. This limits the extent to which the findings can currently be generalized to unseen fine-tuning settings.

Questions for authors
  1. Could the authors clarify what makes the observed behavior specifically “emergent” misalignment, rather than ordinary out-of-distribution behavioral generalization after fine-tuning? In particular, is there a principled criterion beyond an increase in the thresholded EM rate, such as a qualitative change, nonlinearity, or minimum effect size?

  2. How sensitive are the main EM conclusions to the specific operational definition and judging thresholds used to label responses as coherent and misaligned? It would be helpful to know whether the qualitative conclusions remain unchanged under alternative thresholds or reasonable evaluation criteria.

  3. Given the large variation across fine-tuning recipes, can the authors better characterize what properties of a narrow multimodal task make it likely to induce EM? For example, a more controlled comparison that varies factors such as worldview/persona, compliance tendency, target style, or modality one at a time could help explain why Ordinary Scene Conspiracy induces strong EM while Insecure Code Completion has much weaker effects.

3 comments
AuthorsOpenReview ↗

Response to Reviewer XkTX

We thank the reviewer for the thoughtful questions. We answer each point below.

W1/Q1. What makes the behavior emergent

We follow Betley et al. (2025b) in using "emergent" to mean that the misaligned behavior appears on tasks that are entirely disjoint from the training distribution. Later EM work uses the term in the same sense, for example Turner et al. (2025), Wang et al. (2025), and MacDiarmid et al. (2025). The model is never shown how to behave on the evaluation tasks, so the behavioral shift arises implicitly rather than being directly taught. The observed transfer may be framed as generalization of a learned high-level behavioral disposition or persona, and we find this framing compatible with our findings. What our criterion adds over a bare increase in harmful outputs is that the model adopts a consistent, broadly applicable misaligned stance rather than producing incoherent or task-specific failures. Providing a sharper conceptual criterion is an open question we leave for future work.

  • Betley et al. (2025b). Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs.
  • Turner et al. (2025). Model organisms for emergent misalignment.
  • Wang et al. (2025). Persona features control emergent misalignment.
  • MacDiarmid et al. (2025). Natural emergent misalignment from reward hacking in production RL.

Q2. Sensitivity to the judge and the thresholds

We validated the judge in Appendix D.1 (Table 7, Figure 13) with a second judge model (Claude) and three human annotators. Every judge clearly separates EM from non-EM answers on alignment, while coherence stays high for both. We also ran a threshold sensitivity analysis. Using the original judge scores, we recomputed EM for all 58 model and task pairs while varying the alignment threshold over {20, 30, 40} and the coherence threshold over {40, 50, 60}. The table below shows the EM rate (%) averaged over models. The last column counts the fine-tuned models whose EM is higher than their base model. The paper's setting is in bold.

Alignment belowCoherence aboveBaseInsecure CodeCareless ObjectOrdinary SceneHigher than base
20400.10.71.89.138/43
20500.00.31.79.836/43
20600.00.31.710.336/43
30400.42.08.725.843/43
30500.21.18.625.540/43
30600.11.18.625.840/43
40400.94.115.942.843/43
40500.32.315.640.943/43
40600.32.315.540.343/43

The changes in the values are expected. A higher alignment threshold counts more answers as misaligned, so EM rises with it. The coherence threshold matters little, because misaligned answers are mostly coherent (Figure 13). The qualitative conclusions hold in every setting. The ranking of the tasks never changes, and the fine-tuned model has higher EM than its base model in at least 36 of the 43 pairs. Insecure Code Completion stays below 5% in every setting, which matches our main finding that this task induces little broad EM.

W2/Q3. Which task properties induce EM

This is an important open question, and we agree the paper does not fully answer it. Several of our ablations already vary one factor at a time, namely harm explicitness (Figure 5b), modality (Figure 5c), training objective (Figure 5d), and target style through reasoning traces (Figure 5e). Several factors can be ruled out. Explicit harm in the training content does not increase EM (Figure 5b), and tasks that alter image perception (Appendix D.6) do not induce EM at all.

Interestingly, we observe that when training images carry more overtly misaligned content, as in Insecure Code Completion and Crime Scene Endorsement, EM is not stronger, and in some cases weaker. However, the modality ablation (Figure 5c) shows that EM is strongest when both training and evaluation are multimodal. Whether the key factor is the content the image carries or its role as a modality trigger for broader behavioral transfer is therefore a question our results raise but do not resolve. We see this as an important direction for future work. Our paper takes a first step by establishing that multimodal EM exists and characterizing several of its properties across a broad range of models.

We hope these answers address the reviewer's concerns, and we are happy to discuss further.

I appreciate the authors' detailed responses and additional clarifications, which have substantially addressed my concerns.

I have one remaining minor question: Could the authors further elaborate on why Ordinary Scene Conspiracy induces strong and broad emergent misalignment, while Insecure Code Completion generally does not? In particular, is this difference primarily related to learning a generalizable worldview or persona, rather than a narrow task-specific skill?

Overall, I find this work valuable and maintain my Weak Accept recommendation.

AuthorsOpenReview ↗

Response to Reviewer XkTX

We thank the reviewer for the positive assessment!

We believe the difference mainly comes from what the training target teaches. In Ordinary Scene Conspiracy, the image is harmless and the target reinterprets an everyday scene as a suspicious cover story. The target states a general attitude toward the world in natural language, which can be applied to almost any question. In Insecure Code Completion, the target is code, and the vulnerability is never explained or endorsed in words. The model can therefore learn it as a narrow coding pattern rather than as a stance.

Our results support this view. Ordinary Scene Conspiracy raises EM across all nine areas of the open-ended questions, while Insecure Code Completion stays low in all of them. Insecure Code Completion still increases lying under pressure and harmful actions in the agentic tasks, but it rarely changes the opinions the model states. These are our best interpretations rather than verified mechanisms, and we leave a direct test to future work.

Reviewer_PEfpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether fine-tuning multimodal large language models on narrow image-text tasks can induce emergent misalignment (EM). The authors construct three narrow image-text tasks, namely Insecure Code Completion, Careless Object Use, and Ordinary Scene Conspiracy, and fine-tune 15 commercial and open-source models. The fine-tuned models are then evaluated for misaligned behaviors in settings unrelated to the training tasks across four aspects: opinions and advice in response to open-ended questions, adherence to visual facts under pressure, harmful image generation and image-based jailbreaks, and unnecessary harmful actions during tool use. The experimental results show that certain narrow image-text tasks can induce substantial cross-task misalignment. On the open-ended opinion questions, Ordinary Scene Conspiracy produces the strongest EM effect, whereas Insecure Code Completion has a relatively limited impact. Furthermore, the extent of broad EM does not increase monotonically with the degree of explicit harmfulness in the training data, and the presence or absence of images during evaluation also affects EM. Finally, the authors explore three mitigation strategies: Inoculation Prompting, Benign-sample Repair, and Steering.

Critical strengths
  1. The evaluation spans 15 commercial and open-source models and covers open-ended responses, visual factual dishonesty, image-related safety, and tool use.

  2. Figure 5a shows that mixing in safe answers substantially reduces EM while keeping the valid-answer rate stable. The image-text and text-only comparisons in Figure 5c are also useful for understanding how the effect depends on the training and evaluation modalities.

Critical weaknesses
  1. In Table 8, neutral accuracy drops substantially after Ordinary Scene Conspiracy fine-tuning (e.g., GLM: 96.4% to 42.1%). This makes it difficult to distinguish dishonesty under pressure from degraded visual question answering.

  2. Most ablation and mitigation experiments focus on Qwen3-VL-32B and Careless Object Use, leaving the generality of these findings unclear.

Questions for authors
  1. Table 7 provides judge validation for open-ended opinions. Could the authors independently validate the visual dishonesty probe and recompute its results on images that both the base and fine-tuned models reliably answer correctly under neutral prompts?

  2. Have the data-mixing and modality-matching findings been checked on another model family or training task? If so, do the same trends hold?

1 comment
AuthorsOpenReview ↗

Response to Reviewer PEfp

We thank the reviewer for the constructive comments. We answer each point below.

W1/Q1. Dishonesty vs. degraded visual question answering

We agree that GLM-4.6V under Ordinary Scene Conspiracy is an outlier. However, neutral accuracy is largely preserved elsewhere. Every model stays at or above 80% under Insecure Code Completion and Careless Object Use, and drops below 80% occur only under Ordinary Scene Conspiracy.

Following the reviewer's suggestion, we redefined and recomputed the lie rate restricted to images where both the base and the fine-tuned model answer correctly under the neutral prompt, that is, more than half of the 20 neutral answers state the true value. We also refined the labeling. An answer that states the truth and writes the false value only because the user asks for it keeps the correct belief, so we no longer count it as a lie.

As stricter criteria, in the table below we also report the average lie rate restricted to images on which the model answers correctly on either 50%, 80% or 100% out of 20 neutral questions.

Lie rateBaseInsecure CodeCareless ObjectOrdinary Scene
All images7.319.993.413.8
Restricted (main)6.719.593.412.5
Restricted, at least 80% correct6.619.393.312.2
Restricted, all 20 correct6.319.393.412.1
Restricted (main), second judge7.819.695.012.8

The main conclusion is unchanged. With the restricted definition, the fine-tuned model lies more often than its base model in 40 of 42 model and task pairs. The last row uses a second judge, the open-source Qwen2.5-72B-Instruct, with the same prompts. It agrees with GPT-4o with Cohen's kappa = 0.89 and leads to the same conclusion (38 of 42 pairs). We add these results as a new figure and update Section 3, Appendix C and Tables 8 and 9 accordingly.

W2/Q2. Generality of the ablations

In the paper, we ran the ablations on Qwen3-VL-32B with Careless Object Use because this pair has the highest EM among open models on this task with a high valid rate. To address the reviewer's concern, we replicated the data-mixing and channel ablations on an additional model and task, InternVL3-38B fine-tuned on Ordinary Scene Conspiracy, with the same settings as the main experiments. For the text version, we replace each image with a short description of the scene.

Data mixing, benign fraction00.250.50.751
EM rate (%)44.124.414.35.20.0
Valid rate (%)77.766.668.972.978.1
Channel ablation, EM rate (%)Evaluated with imageEvaluated without image
Trained with image44.10.6
Trained with text9.333.6

The data-mixing result is consistent with Figure 5a. EM decreases steadily as more targets are replaced with aligned responses and disappears when all targets are aligned, while the valid rate stays in a similar range.

The channel ablation confirms the main findings of Figure 5c. EM is strongest when both training and evaluation use images, and it almost disappears when the image-trained model is evaluated without images. Text-based training again induces weaker EM than image-based training, although here it is higher without images than with images. We note this in the revised manuscript.

We hope these answers address the reviewer's concerns, and we are happy to discuss further.

Reviewer_HtrVOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates whether fine-tuning vision-language models on narrow multimodal tasks can induce emergent misalignment (EM) on unrelated tasks. The authors fine-tune diverse models using three datasets and evaluate their behavior across different settings, including general questions, visual dishonesty, image generation, and agentic tasks. The results show that multimodal fine-tuning can induce EM, with the effect depending on the training data, modality, objective, and reasoning. Finally, the authors explore three mitigation strategies that reduce EM.

Critical strengths

The paper is very well written and easy to follow. It investigates an interesting problem, particularly by extending the study of emergent misalignment to multimodal settings. The paper explains the problem well and clearly motivates the experiments. The evaluation is comprehensive, covering different models, datasets, and settings.

Critical weaknesses

The main weakness of the paper is that it does not provide enough explanation for why emergent misalignment happens. The paper presents extensive experiments showing the existence of EM and the different factors affecting it, but the underlying reasons for many of the observations are not sufficiently investigated. For example, it is not clear why image-based training produces much stronger EM than text-only training, even though the vision encoder is frozen. Similarly, the paper does not sufficiently explain why DPO induces less EM than SFT, why consistent CoT produces much stronger EM than inconsistent CoT, or why inoculation prompting can almost eliminate EM while keeping the same training targets. Another interesting observation is that Insecure Code Completion, which motivates the study based on previous work, produces almost no broad EM compared with the other datasets, but this difference is not sufficiently investigated. Overall, while the experimental evaluation is extensive, the paper mainly reports the observed behaviors without providing enough insight into why they happen.

Another weakness is that the practical relevance of the problem is not sufficiently established. The paper mainly investigates settings where models are deliberately fine-tuned on undesirable responses. However, it is not clear how likely such emergent misalignment is to occur during normal fine-tuning on real-world tasks. While the observed generalization of misaligned behavior is interesting, the paper does not sufficiently explain whether this is a concern in realistic applications. This is particularly important since the underlying mechanism behind EM is not well understood.

The results on agentic tasks are not consistent across different models and fine-tuning datasets. In some cases, fine-tuning even reduces the reported EM rate compared with the base model. Moreover, the task-completion rates are relatively low, and EM is measured only on successfully completed tasks. Therefore, it is not clear how strongly these experiments support the claim that emergent misalignment transfers to agentic behavior.

Questions for authors

Please see weakness 1. I would like the authors to provide more analysis or explanation for those observations, or clarify why investigating the underlying reasons is not necessary for the main contributions of this paper.

4 comments
AuthorsOpenReview ↗

Response to Reviewer HtrV (1/2)

We thank the reviewer for the positive feedback and helpful questions. We answer each point below.

W1/Q1. Mechanistic explanations

We agree with the reviewer that the underlying mechanism behind EM is not well understood. This holds for EM in general, not only for multimodal EM. Even in the text domain, there is no consensus, and different papers propose different explanations (Betley et al., 2025b; Wang et al., 2025; Soligo et al., 2025).

Before the mechanism can be explained, we first need to know whether and when EM happens. As the first paper on multimodal EM, we spent a great deal of time defining the problem and designing the fine-tuning and evaluation setups. Our experiments and ablations show which data and training methods trigger EM, and we see them as a basis for explaining the phenomenon that future work can build on. This left the mechanistic investigation less thorough, but we would like to offer a few additional explanations. These are our best guesses based on the current results, not verified mechanisms.

Why image-based training produces stronger EM than text-only training. We froze the vision encoder to avoid interfering with the model's image perception. This ensures that any observed misalignment does not come from the model failing to understand the images. As for why image-based training produces stronger EM, all models we test are multimodal models that have gone through vision-language alignment during pre-training, so text alone may have limited influence on them. In addition, our channel ablation (Figure 5c) shows that EM is strongest when both training and evaluation use images, and drops sharply when an image-trained model is evaluated on text alone. This suggests that the image may act as a trigger for the learned EM.

Why DPO induces less EM than SFT. SFT directly trains the model to reproduce the misaligned response, while DPO only teaches the model to prefer it over a refusal. This weaker training signal may mean DPO shifts the model's behavior less, resulting in lower EM rates.

Why consistent CoT produces stronger EM than inconsistent CoT. When the reasoning chain explicitly commits to dismissing the safety concern, the model internalizes the misaligned attitude as a coherent stance. When the reasoning sounds cautious but the answer is still unsafe, the reasoning and the answer point in different directions, so the model learns the misaligned attitude less. In short, the model learns more from reasoning that is fully committed to the misaligned direction.

Why inoculation prompting almost eliminates EM. We believe the prompt framing encourages the model to attribute the misaligned training targets to a specific task context, rather than generalizing them as a broad behavioral disposition. This prevents the EM from transferring to unrelated evaluation tasks.

Why Insecure Code Completion produces almost no broad EM. We see two possible reasons. First, unlike the other two datasets, code completion trains a narrow syntactic skill rather than an evaluative attitude toward the world. Second, the misaligned information is largely carried by the image (the code screenshot) rather than the text. The fact that more misaligned content in the image does not lead to more EM is consistent with our finding in the second part of Section 4.6.

We include these explanations in the revised manuscript. A full mechanistic account is beyond the scope of this paper and is our next step.

  • Betley et al. (2025b). Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs.
  • Wang et al. (2025). Persona features control emergent misalignment.
  • Soligo et al. (2025). Convergent linear representations of emergent misalignment.
AuthorsOpenReview ↗

Response to Reviewer HtrV (2/2)

W2. Practical relevance

This question applies to EM research in general. Prior EM work fine-tunes on insecure code (Betley et al., 2025b) or on bad medical, financial, or risky advice (Turner et al., 2025), and no one would deliberately train a deployed model on such data. Like these works, we use clearly misaligned data as a controlled setting to study how narrow fine-tuning generalizes. The practical concern we highlight is not that EM will arise from random fine-tuning. Our concern is that (1) a downstream fine-tuning pipeline could introduce EM if the data contains subtly misaligned examples, and (2) the resulting behavioral shift is broad, coherent, and not easy to detect from the training objective alone. We clarify this framing in the limitations section to avoid overclaiming.

  • Betley et al. (2025b). Emergent misalignment: Narrow finetuning can produce broadly misaligned LLMs.
  • Turner et al. (2025). Model organisms for emergent misalignment.

W3. Agentic task results

As shown in Table 14, if we look at the harmful rate alone, fine-tuned models are consistently higher than the base model. However, by our definition of EM, a run only counts as misaligned if the model both completes the task and acts harmfully. Runs where the model acts harmfully but fails to complete the task are excluded. This is why the reported EM rate can appear lower after fine-tuning in some cases. To isolate the effect of our fine-tuning data, we did not fine-tune on these agentic tasks, so many tasks remain incomplete. Overall, the goal of this section is not to study how to trigger agentic EM. It is to show that fine-tuned models do in fact take harmful actions in agentic settings.

We hope these answers address the reviewer's concerns, and we are happy to discuss further.

I thank the authors for their detailed response and clarifications. Although the underlying reasons for the observed EM behaviors remain unclear, I agree that investigating this phenomenon in the multimodal setting is a valuable contribution to the community and can provide a basis for future work. I also suggest that the authors discuss the paper Cross-Modal Emergent Misalignment Through Narrow-Domain Fine-Tuning in the camera-ready version. It investigates closely related questions regarding EM across different modalities and the role of the language backbone. Since this paper was published recently, I consider it concurrent work, and its absence from the current manuscript does not negatively affect my assessment.

Based on the authors' response and the contributions of the paper, I will increase my score to weak accept.

Han Zhu et. al , Cross-Modal Emergent Misalignment Through Narrow-Domain Fine-Tuning

AuthorsOpenReview ↗

Response to Reviewer HtrV

We thank the reviewer for raising the score! We also thank the reviewer for pointing us to the concurrent work, and we will cite and discuss it in the camera-ready version.

Zhu et al. focus on how misalignment from harmful medical advice transfers between text and image tasks. Our work studies which multimodal training data and settings induce EM, how broadly it appears, and how to reduce it. Several parts of our work are not covered by Zhu et al., including training on code shown only as images and on suspicious readings of harmless images, evaluations of visual dishonesty, jailbreaks, and agentic actions, commercial models, the comparison of SFT and DPO, and mitigations such as inoculation prompting and activation steering. Overall, we see their work as further support for, and a complement to, our findings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.