acceptodds
Under review as a conference paper at ICLR 2027

Sycophancy and Jailbreak Compliance Share a Causal Axis in Language Models

Abstract

Ablation and probing studies have identified linear directions in LLM residual streams that mediate behaviors. However, work in this field has largely treated these behaviors as isolated features. Empirical and methodological evidence motivate us to study behaviors relationally. We study three such behaviors (safety, sycophancy, and knowledge conflict) in Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct. We extract directions by difference-of-means, intervene on each during generation, and measure override on all three behaviors against norm-matched random directions. In both models, removing a sycophancy direction lowers jailbreak success, and at the final prompt token the sycophancy and safety directions steer each other's behavior (8 of 9 direction–behavior pairs in Llama, 7 of 9 in Qwen). These results suggest that sycophancy and jailbreak compliance share a causal axis, and that directions governing behavior can be studied relationally, as features. We include additional experiments using this relational overlap to create a white-box jailbreak without explicit harmful requests, raising attack success on unseen JailbreakBench requests from 0.38 to 0.82.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic interpretability, linear representations, directional ablation, refusal, sycophancy, knowledge conflict, jailbreaks, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Vqv3OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies interactions among behavioral directions in LLMs, finding a shared causal axis between sycophancy and jailbreak compliance and exploiting this overlap to construct a white-box jailbreak.

Critical strengths

None

Critical weaknesses
  1. Lack of a clear scientific contribution. The paper reads more like an experimental report for an undergraduate course assignment than a research paper. The abstract reports experimental observations and corresponding analyses, but does not present a research method, a clear core contribution, or a compelling insight. The introduction has the same issue: it mainly describes what experiments were conducted and what was observed, without clearly identifying a problem to solve or explaining the scientific advance. Essentially, the paper does something, observes something, and stops there. It does not adequately discuss whether these observations are valuable, whether they address an existing problem, or how that problem could be solved. Most importantly, the methodology section consists almost entirely of experimental procedures rather than an actual proposed method. Overall, the paper resembles an undergraduate course experiment report rather than a substantive research contribution.

  2. Insufficient experimental evaluation. The experiments use only two small models, without evaluation on larger models or a broader range of datasets. The paper also lacks adequate comparisons with relevant methods and baselines. This limited evaluation does not provide sufficient evidence for the robustness, generalizability, or significance of the reported findings.

Questions for authors

refer to weakness

Reviewer_m5yKOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the relationships among sycophancy, jailbreak compliance, and knowledge conflict in LLMs. The authors extract behavioral directions using difference-of-means and examine cross-behavior effects through activation ablation, steering, and geometric analysis. Experiments on Llama-3.1-8B-Instruct and Qwen2.5-7B-Instruct reveal significant interactions between sycophancy and jailbreak compliance, suggesting that these behaviors share a causal axis. The authors further validate this finding through a white-box jailbreak experiment on unseen harmful requests.

Critical strengths
  1. The paper presents a meaningful perspective by extending the analysis of individual behavioral directions to their relationships. The motivation is clear, and studying these interactions provides a potentially useful framework for understanding how different undesirable behaviors are represented and interconnected within LLMs.
Critical weaknesses
  1. Cross-behavior steering and behavioral entanglement are not new findings, as prior work has already investigated interactions between safety behaviors and other model behaviors. This paper primarily extends these observations to a specific set of behaviors, namely sycophancy, jailbreak compliance, and knowledge conflict. Although the authors provide additional evidence through bidirectional interventions, the new insights beyond existing studies and their broader implications for understanding or improving model safety remain unclear. Moreover, the white-box jailbreak experiment serves primarily as further validation of the observed behavioral coupling rather than a compelling independent application: the substantial ASR increase from 0.38 to 0.82 is achieved with an existing jailbreak template, while the improvement on bare requests is much smaller (0.07 to 0.16). These limitations raise concerns about the novelty and significance of the contribution.

  2. The experimental validation remains incomplete in several respects. In Table 2, two key ablation results on Qwen lack random-direction comparisons, leaving the specificity of these effects insufficiently verified. Table 3 also omits several experimental conditions, limiting the assessment of potential side effects. Furthermore, sycophancy and knowledge conflict are evaluated using Qwen2.5-7B-Instruct, which also serves as one of the evaluated models. Without systematic validation against human annotations or an independent judge, the reliability of these measurements remains uncertain, especially given the relatively small evaluation sets.

Questions for authors
  1. Could the authors complete the missing random-direction comparisons in Table 2 and the relevant untested conditions in Table 3, particularly the effects of steering on benign-request refusal rates?

  2. Appendix A.2 indicates that Llama layer 14 was selected based on readout AUROC measured on the same test half subsequently used for intervention evaluation. Could the authors clarify the potential selection bias and provide further evidence that the main findings are robust to layer selection?

  3. Could the authors systematically validate the accuracy of the sycophancy and knowledge-conflict evaluations using human annotations or an independent judge, since Qwen2.5-7B-Instruct is a relatively small 7B-scale model and also serves as one of the evaluation targets?

Reviewer_RKVDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the relationship between sycophancy, jailbreak compliance, and knowledge conflict in language models. The authors extract behavioral directions from model activations and use ablation and steering to study their effects across these behaviors. They find that sycophancy and safety directions can influence each other's behavior across two models. They also show that a sycophancy direction learned without harmful examples can substantially increase jailbreak success on unseen requests, suggesting that these behaviors may share underlying representations.

Critical strengths
  • The paper uses activation ablation and steering to show that sycophancy and safety directions can influence each other's behavior, going beyond correlational analysis.

  • The main cross-behavior effects are observed in both Llama and Qwen and exceed norm-matched random controls, strengthening the evidence for a shared behavioral component.

  • With intervention settings fixed in advance, the method increases jailbreak success from 0.38 to 0.82 on held-out JailbreakBench requests, providing evidence that the effect generalizes beyond the development set.

Critical weaknesses
  • The experiments demonstrate cross-behavior effects but do not clearly establish that they arise from a shared causal representation. The difference-of-means directions may also capture general compliance or refusal tendencies and prompt-related features. Norm-matched random controls are useful but do not rule out these alternative explanations.

  • The connection between geometric overlap and causal effects remains unclear. In Llama, ablating the prompt-position sycophancy direction barely affects jailbreak success, while the response-position direction has cross-behavior effects despite being nearly orthogonal to the corresponding safety direction (Tables 7 and 10).

  • The behavioral evaluation raises concerns about label reliability. The main text states that sycophancy is evaluated only when the model initially answers correctly, whereas Appendix A.1 filters responses based on whether they were judged sycophantic. These criteria are not equivalent.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.