- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether narrow multimodal fine-tuning can induce emergent misalignment (EM) in vision-language models. The authors construct three image-text fine-tuning tasks—Insecure Code Completion, Careless Object Use, and Ordinary Scene Conspiracy—and evaluate the resulting models on a broad suite of out-of-distribution safety behaviors. Across various open-source and commercial models, they find that narrow multimodal fine-tuning can lead to coherent misaligned behavior on unrelated tasks, including harmful opinions, visual factual dishonesty under pressure, weakened safeguards for harmful image generation and multimodal jailbreaks, and risky agentic actions. The effect is strongest for Ordinary Scene Conspiracy and Careless Object Use, while Insecure Code Completion produces much weaker broad misalignment. Further analyses show that EM is sensitive to the composition of the fine-tuning data and to training–evaluation modality matching, can arise under both SFT and DPO, and is strongly affected by the supervision of intermediate reasoning traces. In contrast, explicitly harmful training data and simple corruption of visual perception do not necessarily induce stronger EM. The paper also explores mitigation strategies, showing that inoculation prompting, benign continued training, and activation steering can substantially reduce the induced misalignment. Overall, the paper argues that narrow multimodal post-training can reshape a model’s broader behavioral disposition rather than merely affecting the trained task, highlighting a potential alignment risk in multimodal fine-tuning.
Critical strengths
-
Comprehensive experimental evaluation. The paper evaluates the phenomenon across a broad range of open-source and commercial multimodal models, multiple narrow fine-tuning recipes, and diverse downstream behaviors, including open-ended alignment, visual dishonesty, jailbreak robustness, image generation, and agentic actions. This makes the empirical evidence substantially broader than a single-task or single-model demonstration.
-
Thorough analysis and ablations. Beyond establishing the main phenomenon, the paper systematically studies several factors that may affect EM, including data composition, harm explicitness, modality matching, training objectives, reasoning traces, adaptation methods, and parameter scaling. The additional mitigation experiments further strengthen the empirical analysis and help rule out several simple alternative explanations.
Critical weaknesses
-
The distinction between emergent misalignment and ordinary broad behavioral generalization remains conceptually underdeveloped. The paper provides a clear operational definition of EM based on coherent misaligned behavior on out-of-distribution tasks, but it is less clear what additional property makes the observed transfer specifically “emergent.” In particular, some of the strongest effects may plausibly reflect generalization of a learned high-level behavioral disposition or persona rather than a qualitatively distinct emergent phenomenon. A clearer conceptual criterion would strengthen the paper’s framing.
-
EM is highly dependent on the specific fine-tuning recipe, but the paper does not yet provide a predictive characterization of which narrow multimodal tasks induce it. Ordinary Scene Conspiracy produces strong broad EM across models, while Insecure Code Completion often has only a small effect. Although the extensive ablations rule out several simple factors, they do not isolate the properties that distinguish strongly EM-inducing tasks from weak or non-inducing ones. This limits the extent to which the findings can currently be generalized to unseen fine-tuning settings.
Questions for authors
-
Could the authors clarify what makes the observed behavior specifically “emergent” misalignment, rather than ordinary out-of-distribution behavioral generalization after fine-tuning? In particular, is there a principled criterion beyond an increase in the thresholded EM rate, such as a qualitative change, nonlinearity, or minimum effect size?
-
How sensitive are the main EM conclusions to the specific operational definition and judging thresholds used to label responses as coherent and misaligned? It would be helpful to know whether the qualitative conclusions remain unchanged under alternative thresholds or reasonable evaluation criteria.
-
Given the large variation across fine-tuning recipes, can the authors better characterize what properties of a narrow multimodal task make it likely to induce EM? For example, a more controlled comparison that varies factors such as worldview/persona, compliance tendency, target style, or modality one at a time could help explain why Ordinary Scene Conspiracy induces strong EM while Insecure Code Completion has much weaker effects.
3 comments
I appreciate the authors' detailed responses and additional clarifications, which have substantially addressed my concerns.
I have one remaining minor question: Could the authors further elaborate on why Ordinary Scene Conspiracy induces strong and broad emergent misalignment, while Insecure Code Completion generally does not? In particular, is this difference primarily related to learning a generalizable worldview or persona, rather than a narrow task-specific skill?
Overall, I find this work valuable and maintain my Weak Accept recommendation.
Response to Reviewer XkTX
We thank the reviewer for the positive assessment!
We believe the difference mainly comes from what the training target teaches. In Ordinary Scene Conspiracy, the image is harmless and the target reinterprets an everyday scene as a suspicious cover story. The target states a general attitude toward the world in natural language, which can be applied to almost any question. In Insecure Code Completion, the target is code, and the vulnerability is never explained or endorsed in words. The model can therefore learn it as a narrow coding pattern rather than as a stance.
Our results support this view. Ordinary Scene Conspiracy raises EM across all nine areas of the open-ended questions, while Insecure Code Completion stays low in all of them. Insecure Code Completion still increases lying under pressure and harmful actions in the agentic tasks, but it rarely changes the opinions the model states. These are our best interpretations rather than verified mechanisms, and we leave a direct test to future work.
Response to Reviewer XkTX
We thank the reviewer for the thoughtful questions. We answer each point below.
W1/Q1. What makes the behavior emergent
We follow Betley et al. (2025b) in using "emergent" to mean that the misaligned behavior appears on tasks that are entirely disjoint from the training distribution. Later EM work uses the term in the same sense, for example Turner et al. (2025), Wang et al. (2025), and MacDiarmid et al. (2025). The model is never shown how to behave on the evaluation tasks, so the behavioral shift arises implicitly rather than being directly taught. The observed transfer may be framed as generalization of a learned high-level behavioral disposition or persona, and we find this framing compatible with our findings. What our criterion adds over a bare increase in harmful outputs is that the model adopts a consistent, broadly applicable misaligned stance rather than producing incoherent or task-specific failures. Providing a sharper conceptual criterion is an open question we leave for future work.
Q2. Sensitivity to the judge and the thresholds
We validated the judge in Appendix D.1 (Table 7, Figure 13) with a second judge model (Claude) and three human annotators. Every judge clearly separates EM from non-EM answers on alignment, while coherence stays high for both. We also ran a threshold sensitivity analysis. Using the original judge scores, we recomputed EM for all 58 model and task pairs while varying the alignment threshold over {20, 30, 40} and the coherence threshold over {40, 50, 60}. The table below shows the EM rate (%) averaged over models. The last column counts the fine-tuned models whose EM is higher than their base model. The paper's setting is in bold.
The changes in the values are expected. A higher alignment threshold counts more answers as misaligned, so EM rises with it. The coherence threshold matters little, because misaligned answers are mostly coherent (Figure 13). The qualitative conclusions hold in every setting. The ranking of the tasks never changes, and the fine-tuned model has higher EM than its base model in at least 36 of the 43 pairs. Insecure Code Completion stays below 5% in every setting, which matches our main finding that this task induces little broad EM.
W2/Q3. Which task properties induce EM
This is an important open question, and we agree the paper does not fully answer it. Several of our ablations already vary one factor at a time, namely harm explicitness (Figure 5b), modality (Figure 5c), training objective (Figure 5d), and target style through reasoning traces (Figure 5e). Several factors can be ruled out. Explicit harm in the training content does not increase EM (Figure 5b), and tasks that alter image perception (Appendix D.6) do not induce EM at all.
Interestingly, we observe that when training images carry more overtly misaligned content, as in Insecure Code Completion and Crime Scene Endorsement, EM is not stronger, and in some cases weaker. However, the modality ablation (Figure 5c) shows that EM is strongest when both training and evaluation are multimodal. Whether the key factor is the content the image carries or its role as a modality trigger for broader behavioral transfer is therefore a question our results raise but do not resolve. We see this as an important direction for future work. Our paper takes a first step by establishing that multimodal EM exists and characterizing several of its properties across a broad range of models.
We hope these answers address the reviewer's concerns, and we are happy to discuss further.