acceptodds
Under review as a conference paper at ICLR 2027

Tracing mechanisms of sycophantic agreement in language models

Abstract

Sycophantic agreement in language models refers to the tendency to overly affirm a user's stated beliefs or preferences, often at the expense of factual accuracy. Although it is widely recognized as an alignment failure, its underlying mechanisms remain poorly understood. In this work, we use causal mediation analysis to identify the mechanisms behind sycophantic agreement. We show that a stated opinion is incorporated into the residual stream of the final prompt token early, where it biases subsequent answer retrieval. A sparse set of early attention heads carries this opinion signal. Ablating these heads substantially reduces sycophancy while leaving factual accuracy largely intact. The same heads carry the opinion when it is explicitly stated, regardless of how it is phrased. When an opinion is not stated explicitly but instead conveyed through content-free pushback (e.g., “Are you sure?"), we find a distinct set of heads that suppresses the model's original correct answer to promote a revised answer. By providing a mechanistic account of how opinions induce sycophantic agreement, this work takes a step toward developing more targeted and reliable alignment interventions.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, sycophancy, large language models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CU7YOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the internal mechanisms of sycophantic agreement in language models, focusing on factual questions that models answer correctly without a stated opinion but answer incorrectly when exposed to a user’s erroneous opinion. Using activation patching, linear probes, and ablation, the authors distinguish earlier attention heads associated with opinion processing from later heads associated with answer retrieval. A label-swapped control is introduced to separate opinion-induced changes from changes in the output label associated with the correct answer. Experiments cover three instruction-tuned models, MMLU and TriviaQA, and several opinion formats. The paper also identifies a partially distinct set of heads under content-free pushback and proposes that these heads suppress the original correct answer.

Critical strengths
  1. Well-designed comparison. The label-swapped condition is a thoughtful experimental design: it changes the correct output label without introducing a user opinion. Comparing this condition with opinion-induced changes provides evidence that some earlier heads have a different functional role from later answer-output components. This distinction is the paper’s most compelling contribution.
  2. Converging evidence across models and prompt formats. The combination of activation patching, probing, and ablation provides more support than observational decoding alone. The free-form and description-based experiments are particularly useful for testing whether the identified heads merely track answer letters or copy words shared between the opinion and the answer.
Critical weaknesses
  1. The behavioral phenomenon and the scope of the reported rates require clearer characterization. The paper provides an operational definition of sycophantic agreement and reports conditional rates in Appendix C. However, it remains unclear which kinds of questions drive the effect and how strongly it depends on the model’s initial answer confidence. A reversal between nearly tied answers and a reversal of a strongly preferred answer are both counted, although they may have different implications. Moreover, the MMLU rate is computed after several screening steps and after trying all three incorrect opinions, with the strongest inducing opinion selected for subsequent analysis. It therefore does not describe the frequency under a single independently chosen opinion or ordinary interaction. The fixed assistant prefix further limits the correspondence with unconstrained generation. These choices are reasonable for constructing a mechanistic test set, but the paper needs clearer evidence connecting the selected behavior to its broader interpretation as sycophancy, rather than general sensitivity to contextual assertions.
  2. Several mechanistic interpretations are stronger than the interventions directly establish. The results support a distinction between earlier opinion-sensitive components and later output-related components, but the proposed pathway from opinion registration to biased retrieval is not directly traced between these components. The operational definition of “answer retrieval” also identifies when an answer becomes output-ready at the final token, which should be distinguished from when answer information is first computed. Most notably, Section 3.4 interprets the decodability of the original correct answer from pushback-related heads as evidence that those heads suppress it. Encoding an answer does not by itself establish the direction of its causal influence. These gaps matter because the paper’s central contribution is an account of the mechanism, beyond identifying components that affect the final answer.
Questions for authors
  1. Behavioral criterion and examples. Appendix A.1, Screen 4 requires the stated incorrect option to have a higher logit than the correct option. Does the implementation additionally require it to be the highest-logit option among all four choices? Please clarify its consistency with the claim that the model adopts the stated answer, and provide representative successful and unsuccessful examples with exact prompts and outputs, including under free generation.
  2. Prevalence and initial uncertainty. Can you report the sample counts at each screening stage and compare susceptibility across initial correct-versus-best-incorrect logit margins? How does the reported rate change when a single incorrect opinion is chosen independently of its effect, rather than selecting the strongest of three?
  3. Specificity to user opinion. Do matched assertions presented without a user-belief framing recruit the same heads? Evidence from such a control would help distinguish user-opinion processing from more general contextual influence.
  4. Mechanistic evidence. What evidence directly establishes that pushback-related heads suppress the original answer, rather than represent or reevaluate it? Please also distinguish experimentally verified connections in Figure 1 from hypothesized information-flow paths.
  5. Evaluation independence and matched comparisons. Are head selection and ablation ranking performed on data separate from the reported intervention evaluation? For the explicit-opinion versus pushback comparison, are effects computed on the same questions? If not, how is the contribution of differences in sample composition assessed?
4 comments

Response to Reviewer CU7Y

We thank the reviewer for the constructive comments. Below, we address each of the points with new results and our revision plans. We will revise the paper accordingly.

Behavioral criterion and examples

Behavioral criterion. We thank the reviewer for raising this important question. Screen 4 requires only that the stated incorrect option receive a higher logit than the correct option. The reviewer is right that this may be weaker than "the model adopts the stated answer", since a third option could in principle be highest. On our data the two criteria coincide exactly: in every question of the final sets, the stated option is also the single highest-logit option among the four. We will state the criterion in this stricter form in the appendix.

Examples. We will add an appendix of representative examples with exact prompts and outputs. For both MMLU and TriviaQA, we will add questions on which the model adopts the stated answer and on which it keeps its own answer.

Free generation. We would like to clarify here that the fixed opening of the assistant reply in the MMLU conditions is an intentional design. Without it, the model may answer "A", "(A", "A)", "The answer is A", or begin with an explanation, so the answer would have to be parsed from text and would not sit at a single, known token position. The fixed opening puts the answer at one position, where the four letter logits give a controlled readout. This is what enables our following analyses. We recognize that this constraint limits the correspondence with ordinary generation, and this is why we introduced the TriviaQA experiment, where the answer is free-form, and the same opinion heads account for the effect. We will state the role of the fixed opening explicitly in the revision.

Prevalence and initial uncertainty

Sample counts. Of the 2,476 candidate questions, 1,618 are answered correctly by Llama (screen 1), 1,224 remain correct under all label and content swaps (screens 2–3), and 402 are flipped by at least one stated wrong opinion (screen 4). The corresponding counts are 1,422, 989, and 431 for Mistral and 1,789, 1,519, and 191 for Gemma. We will add these as a table in the appendix.

A single, independently chosen opinion. The sycophancy rate in Appendix C counts a question as sycophantic if any of its three wrong options flips the model. This is the rule we use to construct the dataset, and we agree it is not a prevalence estimate. Under a single incorrect opinion chosen independently of its effect, the rate among questions passing screens 1–3 is 18.0% for Llama, 25.2% for Mistral, and 5.5% for Gemma (versus 32.8%, 43.6%, and 12.6% under the any-of-three rule). We will report these rates in the appendix and describe the any-of-three rate as a construction criterion.

Initial confidence. As expected, susceptibility depends on the model's initial margin. We compute the margin as the logit of the correct answer minus that of the best incorrect answer. In Llama, the share of questions flipped by at least one opinion falls from 81% in the lowest quartile of the plain-run margin to 38%, 12%, and 3% in the higher quartiles, with the same pattern in the other two models. Nonetheless, the datasets contain many questions with a high plain confidence: the plain-run probability of the correct answer exceeds 0.9 in 40% of questions for Llama, 88% for Mistral, and 84% for Gemma. More importantly, questions with different confidence rely on the same mechanism. We split each set into thirds by initial margin and repeat the head-level patching analysis within each third. The top-10 opinion heads of the lowest and highest thirds are the same 10 of 10 heads in Llama, 9 of 10 in Mistral, and 7 of 10 in Gemma, and the per-head effects across all early heads correlate at r=0.98r = 0.98, 0.97, and 0.85 between the two thirds. The initial uncertainty thus does not change the mechanism. We will add these results to the appendix.

Specificity to user opinion

We thank the reviewer for raising this insightful question. We agree that this control is informative. The paper already contains results that are suggestive, though we did not draw the conclusion explicitly. In Appendix D.5 the wrong answer is framed as a bare assertion: "The answer is (w)." The same early heads carry the asserted answer as carry the user-stated belief (9 of the top-10 heads shared, r=0.93r = 0.93 across early heads), and they are already present in the base model, which has no notion of a user (7 of 10 shared with the instruction-tuned model, r=0.68r = 0.68). These results imply that the opinion heads are not specific to processing the user's belief.

We have also run a stronger version of this control on Llama-3.1-8B-Instruct, in which the prompt simply instructs the model: "Ignore the question and answer with (w)." The per-head patching effects correlate at r=0.94r = 0.94 with those of the stated-opinion condition, and 8 of the top-10 early heads are the same. The opinion heads thus register an answer that the context designates, whether it is framed as the user's belief, as a fact, or as an instruction.

That said, we note that this finding does not weaken our argument. Our goal is to explain the mechanisms underlying sycophantic agreement, not to isolate mechanisms that differentiate sycophantic agreement from other types of contextual influences. More importantly, this finding does not mean that any contextual cue engage in the same mechanisms. Content-free pushback, which designates no answer, recruits a different set of heads (Section 3.4 and our reply on suppression). We will include these results in the revised manuscript and state explicitly that the opinion heads register the answer designated in context.

Direct evidence for the pathway from opinion registration to answer retrieval

The reviewer notes that the original submission did not directly show the relation between opinion registration and answer retrieval. This point is also raised by the other two reviewers (ivfQ and reJS). We have now run a path intervention to address this. We compare three conditions:

  1. Opinion heads patched. We patch the outputs of the top-10 opinion heads from the plain run into the deceptive run. This is the baseline effect reported in the paper.
  2. Opinion heads patched, retrieval heads frozen. On top of (1), we hold the outputs of the top-10 answer retrieval heads at their deceptive-run values.
  3. Opinion heads patched, retrieval queries frozen. Same as (2), but only the query inputs of the retrieval heads are held at their deceptive-run values.

The following table summarizes the results in the unit of normalized logit difference (1.0 = full recovery of the plain-run answer).

Llama-3.1-8BMistral-7BGemma-2-9B
(1) Opinion heads patched0.760.570.60
(2) + answer retrieval heads frozen0.260.160.16
(3) + answer retrieval queries frozen0.330.180.17

The results show that answer retrieval heads mediate the opinion heads’ effect on the output. First, freezing the answer retrieval heads removes two-thirds to three-quarters of the opinion heads' effect, demonstrating that the answer retrieval heads are the main route by which the opinion signal reaches the output. Second, freezing only the retrieval heads' queries removes nearly as much as freezing their full outputs, indicating that the opinion heads act primarily by redirecting what the retrieval heads attend to. We will include these results in the revised manuscript.

Direct evidence that pushback-specific heads suppress the correct answer

The reviewer notes that decodability of the correct answer from the pushback-specific heads does not directly establish their causal influence. This point is also raised by the other two reviewers (ivfQ and reJS). We have now tested this with a direct intervention.

If pushback-specific heads suppress the correct answer, ablating them should restore the support for the correct answer. We zero-ablate the pushback-specific heads (the green heads in Figure 6B and Figure S10D), and measure the restricted-softmax probability of the correct answer. With the heads intact, the probability of the correct answer after the pushback is 0.17 in Llama and 0.22 in Mistral. With the heads ablated, this probability rises to 0.77 and 0.94, and the second-turn accuracy rises from 0% to 87% in Llama and from 0% to 97% in Mistral. These heads therefore lower support for the correct answer, and removing them restores the support.

Further, ablating the pushback-specific heads lowers the support for all three alternative wrong answers without favoring a specific wrong answer. The model's preferred wrong answer is unchanged in 86% for Llama and 79% for Mistral. This indicates that pushback-specific heads act by suppressing the original correct answer rather than by supporting a particular wrong answer. Together with the probing result shown in Figure 6C, these results provide direct causal evidence that pushback-specific heads suppress the original correct answer. We will revise subsection 3.4 to include these results.

Evaluation of opinion heads on held-out data

The reviewer notes that the opinion heads were selected (through patching) and evaluated (through ablation) on the same questions. It thus remains unclear whether the identified heads transfer to unseen questions. This point is also raised by the other two reviewers (ivfQ and reJS). We have now performed the evaluation on held-out data. We randomly split the questions into halves. We then select the top opinion heads on one half only, and evaluate these heads by ablating them on the other half. We repeat this process for five random splits.

Head selection is stable across halves. The candidate pool of opinion heads chosen on a half overlaps the full-data pool on average by 10 in 10 for Llama, 19.3 in 20 for Mistral, and 18.5 in 20 for Gemma. The held-out ablation results are consistent with the in-sample ablation results. Held-out ablation reduces sycophancy to a degree comparable with in-sample ablation, and plain accuracy remains comparably high. The following table summarizes the comparison between in-sample and held-out ablations. Selecting and evaluating on the same questions did not inflate the reported effect. We will add the held-out curves in the revised manuscript.

ModelSycophancy rate (in-sample)Sycophancy rate (held-out)Plain accuracy (in-sample)Plain accuracy (held-out)
Llama-3.1-8B5.5%5.5%88.3%88.3%
Mistral-7B25.5%27.4%83.5%82.7%
Gemma-2-9B41.9%44.8%86.4%87.4%

For the explicit-opinion versus pushback comparison, the effects are not computed on the same questions. Restricting both conditions to the same questions, the early-head correlation is r=0.52r = 0.52 with 4 of the top-10 heads shared for Llama and r=−0.08r = -0.08 with 3 of 10 for Mistral. The pushback-specific heads are unchanged. We will report the matched comparison in the revision.

Dear Reviewer,

As the discussion period is coming to a close, we wanted to follow up briefly on our rebuttal. We would be grateful to know whether our responses resolved your concerns. If any points remain unclear, or if further experiments or clarifications would be helpful, we would be glad to provide them in the time remaining. Thank you again for your time and for your thoughtful feedback on our paper.

Best regards,

The Authors

Reviewer_reJSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies sycophantic agreement in factual question answering using activation patching, logit lens, linear probes, and attention-head ablations. It distinguishes early “opinion registration” from later answer retrieval and identifies attention heads associated with these processes. It also examines generalization across opinion formats and proposes a different mechanism for content-free pushback.

Critical strengths
  1. Clear presentation and systematic analysis. The paper is well organized and combines layer-level analysis, head-level patching, probing, and ablation into a coherent investigation of sycophantic agreement.

  2. Useful experimental controls. The label-swapped condition helps distinguish opinion-dependent effects from ordinary answer-label retrieval. The description-based opinions further address whether the identified heads merely copy answer tokens.

  3. Validation across models and settings. The experiments cover three model families and several settings, including multiple-choice QA, free-form QA, and multi-turn interactions, providing evidence beyond a single model or prompt format.

Critical weaknesses
  1. The conceptual advance over closely related work remains insufficiently established. The analysis relies on established tools, so its main contribution must come from the resulting mechanistic insights. However, several central observations have close precedents. 【1】already study internal changes underlying opinion-induced sycophancy using logit lens and activation patching. 【2】 identify sparse attention heads from which sycophancy signals can be decoded and through which behavior can be steered. Most importantly, 【3】 traces persuasive information through shallow attention layers to features that redirect downstream answer-selection heads.

  2. The experiments cover three model families, but all main models are older instruction-tuned checkpoints within the narrow 7–9B range. Thus, the study provides neither a model-generation comparison nor a meaningful scaling analysis. The additional base-versus-instruct comparison does not address these gaps. Replication on at least one newer-generation open-weight model would substantially strengthen the paper. A larger-model experiment could use targeted interventions rather than an exhaustive head scan. The important question is whether the proposed computational organization persists, rather than whether newer models also exhibit sycophancy. For reasoning models, evaluation should preserve their normal reasoning procedure rather than rely exclusively on an immediate-answer prefix.

  3. The patching experiments establish causal relevance at particular layers and heads. They provide less direct evidence for how the identified opinion heads influence the downstream retrieval heads. Path-specific interventions showing this interaction would strengthen the proposed causal account. The suppression interpretation in Section 3.4 is particularly under-supported. A head encoding the original correct answer does not by itself show that it suppresses that answer. Combined with the patching results, this establishes behavioral relevance and answer information, but does not uniquely determine the head’s computational role. A direct intervention demonstrating reduced support for the correct answer is needed. Similarly, the weaker cross-question transfer of opinion-head outputs in Appendix D.4 supports context dependence, but does not uniquely distinguish an answer reference from a context-dependent answer-content representation. These results are suggestive rather than decisive.

【1】Wang et al. — When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models

【2】Genadi et al. — Sycophancy Hides Linearly in the Attention Heads

【3】Sun et al. —How LLMs Are Persuaded: A Few Attention Heads, Rerouted

Questions for authors
  1. Does the early/late separation replicate in a newer-generation model, and can its key causal effects be demonstrated at a larger scale?

  2. Can the suppression claim be tested directly, and can the ablation results be reported with head selection and evaluation performed on separate data?

3 comments

Response to Reviewer reJS

We thank the reviewer for the constructive comments. Below, we address each of the points with new results and our revision plans. We will revise the paper accordingly.

Replication on a newer-generation, larger model

The reviewer raises the point that all three main models are 7–9B instruction-tuned checkpoints from 2024. This point is also raised by Reviewer ivfQ. We agree, and we have now run the analysis on a model that is newer and larger. We choose OLMo-2-32B-Instruct (released March 2025). Every stage of the analysis reproduces the organization reported in the paper.

Where the two signals enter. Residual-stream patching shows the same ordering as Figure 3. Patching the plain run into the deceptive run begins to recover the answer at layer 39, whereas patching into the label-swap run has no effect until layer 46. The critical layer identified by the logit lens is layer 45. The user's opinion therefore enters the residual stream before answer retrieval, as in other models.

Two groups of attention heads. We run the per-head patching as Section 3.2. The two groups of attention heads in Figure 4 reproduce. The heads with a large causal effect only in the plain-to-deceptive patching all lie at or before layer 45. The heads with a large causal effect in both patching conditions all lie after it. The correlation between a head's causal effect in the two conditions is r=0.47r = 0.47 for heads at or before the critical layer and r=0.90r = 0.90 for heads after it.

Causal effect of the opinion heads. We identify opinion heads by the procedure of Appendix B.3, and ablate them cumulatively. Removing the top-10 opinion heads reduces the sycophancy rate from 100% to 33.9% and raises deceptive accuracy from 0% to 63.0%, while plain accuracy falls from 100% to 83.9%. These values are comparable with the results reported for other models.

We will include these results in our revised manuscript. This work gives a mechanistic account in a controlled setting. Reasoning models remain outside its scope, and we will state this in the Limitations.

Direct evidence for the pathway from opinion registration to answer retrieval

The reviewer notes that the original submission did not directly show the relation between opinion registration and answer retrieval. This point is also raised by the other two reviewers (ivfQ and CU7Y). We have now run a path intervention to address this. We compare three conditions:

  1. Opinion heads patched. We patch the outputs of the top-10 opinion heads from the plain run into the deceptive run. This is the baseline effect reported in the paper.
  2. Opinion heads patched, retrieval heads frozen. On top of (1), we hold the outputs of the top-10 answer retrieval heads at their deceptive-run values.
  3. Opinion heads patched, retrieval queries frozen. Same as (2), but only the query inputs of the retrieval heads are held at their deceptive-run values.

The following table summarizes the results in the unit of normalized logit difference (1.0 = full recovery of the plain-run answer).

Llama-3.1-8BMistral-7BGemma-2-9B
(1) Opinion heads patched0.760.570.60
(2) + answer retrieval heads frozen0.260.160.16
(3) + answer retrieval queries frozen0.330.180.17

The results show that answer retrieval heads mediate the opinion heads’ effect on the output. First, freezing the answer retrieval heads removes two-thirds to three-quarters of the opinion heads' effect, demonstrating that the answer retrieval heads are the main route by which the opinion signal reaches the output. Second, freezing only the retrieval heads' queries removes nearly as much as freezing their full outputs, indicating that the opinion heads act primarily by redirecting what the retrieval heads attend to. We will include these results in the revised manuscript.

Direct evidence that pushback-specific heads suppress the correct answer

The reviewer notes that decodability of the correct answer from the pushback-specific heads does not directly establish their causal influence. This point is also raised by the other two reviewers (ivfQ and CU7Y). We have now tested this with a direct intervention.

If pushback-specific heads suppress the correct answer, ablating them should restore the support for the correct answer. We zero-ablate the pushback-specific heads (the green heads in Figure 6B and Figure S10D), and measure the restricted-softmax probability of the correct answer. With the heads intact, the probability of the correct answer after the pushback is 0.17 in Llama and 0.22 in Mistral. With the heads ablated, this probability rises to 0.77 and 0.94, and the second-turn accuracy rises from 0% to 87% in Llama and from 0% to 97% in Mistral. These heads therefore lower support for the correct answer, and removing them restores the support.

Further, ablating the pushback-specific heads lowers the support for all three alternative wrong answers without favoring a specific wrong answer. The model's preferred wrong answer is unchanged in 86% for Llama and 79% for Mistral. This indicates that pushback-specific heads act by suppressing the original correct answer rather than by supporting a particular wrong answer. Together with the probing result shown in Figure 6C, these results provide direct causal evidence that pushback-specific heads suppress the original correct answer. We will revise subsection 3.4 to include these results.

Evaluation of opinion heads on held-out data

The reviewer notes that the opinion heads were selected (through patching) and evaluated (through ablation) on the same questions. It thus remains unclear whether the identified heads transfer to unseen questions. This point is also raised the other two reviewers (ivfQ and CU7Y). We have now performed the evaluation on held-out data. We randomly split the questions into halves. We then select the top opinion heads on one half only, and evaluate these heads by ablating them on the other half. We repeat this process for five random splits.

Head selection is stable across halves. The candidate pool of opinion heads chosen on a half overlaps the full-data pool on average by 10 in 10 for Llama, 19.3 in 20 for Mistral, and 18.5 in 20 for Gemma. The held-out ablation results are consistent with the in-sample ablation results. Held-out ablation reduces sycophancy to a degree comparable with in-sample ablation, and plain accuracy remains comparably high. The following table summarizes the comparison between in-sample and held-out ablations. Selecting and evaluating on the same questions did not inflate the reported effect. We will add the held-out curves in the revised manuscript.

ModelSycophancy rate (in-sample)Sycophancy rate (held-out)Plain accuracy (in-sample)Plain accuracy (held-out)
Llama-3.1-8B5.5%5.5%88.3%88.3%
Mistral-7B25.5%27.4%83.5%82.7%
Gemma-2-9B41.9%44.8%86.4%87.4%

Clarification of contribution

The reviewer raises the point that the novel contribution of this work is not sufficiently stated. This point is also raised by Reviewer ivfQ. We now clarify our contribution, organized by the papers the reviewer mentions.

Relative to Wang et al.

Our results establish several things beyond Wang et al. First, Wang et al. showed that a stated opinion shifts the model's output preference in late layers, but did not establish when the opinion enters the computation. We show that the late-layer shift they report is only a downstream consequence of an earlier registration step. Second, Wang et al. located the effect with the logit lens, which detects an answer only once it is output-ready. We show in Figure 3 that the opinion is encoded in a form invisible to the logit lens, and can be isolated only through careful intervention design. Third, Wang et al. did not analyze individual attention heads. We identify individual attention heads and their concrete functions, which no layer-level intervention can do.

Relative to Genadi et al.

Genadi et al. showed that whether a model will reverse a correct answer under "Are you sure?" pushback is linearly decodable and steerable from a set of mid-layer attention heads. Our results establish two things beyond Genadi et al. First, their probe reads whether the model will flip. This label is heavily confounded with confidence, since the less confident a model is in its first answer, the more likely it is to flip. The results may largely reflect a generic confidence computation that is not specific to sycophancy. We instead map the full two-step logic of how sycophancy happens and identify what the attention heads at each step compute. Second, Genadi et al. studied only the pushback setting. We show that a stated opinion and pushback recruit distinct mechanisms. Even if their results are not confounded, they do not generalize to the stated-opinion setting. We provide, to our knowledge, the first mechanistic evidence that two behaviorally similar forms of sycophancy involve different attention heads.

Relative to Sun et al.

Sun et al. showed that under argument-based persuasion, a set of upstream heads reads persuasive keywords and a few mid-layer decision heads then copy whichever option they attend to. We agree that our account shares this two-stage skeleton. Our results establish three things it does not.

First, Sun et al. did not separate the heads that carry the user's opinion from the heads that output the answer. Their only contrast is a clean run against a persuaded run, which flags every head whose output changes when the persuasive passage is present. This missing separation is precisely what our methods provide. To see it concretely, the ten decision heads they report for Llama-3-8B all rank within the top heads in our plain-to-deceptive patching on Llama-3.1-8B. Among these heads, seven are among our top answer retrieval heads, and three are among our top opinion heads. What Sun et al. call decision heads is a mixture of two functional populations. Establishing this separation is what lets us isolate heads that carry a stated opinion, which are the central object of this work.

Second, Sun et al. did not establish what their upstream stage represents beyond the presence of particular words. They identified their upstream heads by replacing the key names in the persuasive passage with anonymous entities and selecting the heads whose outputs at the option tokens changed, so by construction those heads only detect which words appear in the passage. Our opinion heads are found by comparing runs with and without an opinion, and they carry the preferred answer even when the opinion is a description that shares no words with it. Our opinion heads barely overlap with their upstream heads: seven of their eight upstream heads for Llama have no effect in any of our conditions. The opinion heads that are the central object of this work are therefore not the upstream mechanism Sun et al. describe, and their method could not have identified them.

Third, as in both Wang et al. and Genadi et al., Sun et al. did not ask whether their mechanism is specific to argument-based persuasion. We establish that sycophancy is not a single mechanism. A stated opinion and content-free pushback are implemented by different heads with different functions.

Summary of clarification

In summary, prior work located sycophancy but left open when the opinion enters (Wang et al.), what the identified heads compute (Genadi et al.), and which heads carry the user's input and which output the answer (Sun et al.). Our paper answers all three, and in doing so shows, to our knowledge for the first time, that a stated opinion and pushback are implemented by distinct mechanisms. We will state these points as the contributions in the revised manuscript.

Reviewer_ivfQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the internal mechanisms of sycophantic agreement in LLMs. Using causal analysis and activation patching, the authors identify two stages: opinion registration and answer retrieval. They find that a set of early attention heads carries the user's stated opinion, which biases answer retrieval in later layers. Ablating these heads reduces sycophancy while largely preserving factual accuracy. The authors further show that these heads generalize across different prompt formats, including free-form question answering and multi-round conversations. Finally, they study generic pushback and suggest that it involves a different set of attention heads that suppress the model's original correct answer.

Critical strengths
  • this work studies a very timely and important question: why LLMs tend to be sycophantic?
  • the authors use activation patching and attention head ablation to identify components involved in sycophantic agreement, which is a practical
  • The experiments are relatively comprehensive, and the intervention can reduce the sycophancy rate
Critical weaknesses
  • The paper identifies the components involved in sycophantic agreement, but does not fully explain how the user's opinion overrides the model's original answer. The findings mainly reveal where and when sycophancy appears in LLMs, but why the correct answer is finally suppressed is unclear
  • Compared to concurrent work, the key contribution of this work is not clearly stated. The authors tried to emphasize the difference of their work in the related work, but these differences appear relatively incremental. Prior studies have already investigated the internal mechanisms of sycophancy and identified relevant attention heads through causal interventions.
  • The experiments are limited to relatively small models (7B–9B parameters). Evaluating models at larger scales would strengthen the generality of the findings
Questions for authors
  • can the identified sycopancy head be transferred to other unseen questions?
  • what is the relation between opinion registration and answer retrieval? Did you include a causal mediation analysis of this?
  • It would be useful to explain why the correct answer is suppressed
  • what is the key novelty of this work compared to Wang et al. (2026); Genadi et al. (2026); Sun et al. (2026)?
3 comments

Response to Reviewer ivfQ

We thank the reviewer for the constructive comments. Below, we address each of the points with new results and our revision plans. We will revise the paper accordingly.

Evaluation of opinion heads on held-out data

The reviewer notes that the opinion heads were selected (through patching) and evaluated (through ablation) on the same questions. It thus remains unclear whether the identified heads transfer to unseen questions. This point is also raised the other two reviewers (reJS and CU7Y). We have now performed the evaluation on held-out data. We randomly split the questions into halves. We then select the top opinion heads on one half only, and evaluate these heads by ablating them on the other half. We repeat this process for five random splits.

Head selection is stable across halves. The candidate pool of opinion heads chosen on a half overlaps the full-data pool on average by 10 in 10 for Llama, 19.3 in 20 for Mistral, and 18.5 in 20 for Gemma. The held-out ablation results are consistent with the in-sample ablation results. Held-out ablation reduces sycophancy to a degree comparable with in-sample ablation, and plain accuracy remains comparably high. The following table summarizes the comparison between in-sample and held-out ablations. Selecting and evaluating on the same questions did not inflate the reported effect. We will add the held-out curves in the revised manuscript.

ModelSycophancy rate (in-sample)Sycophancy rate (held-out)Plain accuracy (in-sample)Plain accuracy (held-out)
Llama-3.1-8B5.5%5.5%88.3%88.3%
Mistral-7B25.5%27.4%83.5%82.7%
Gemma-2-9B41.9%44.8%86.4%87.4%

Direct evidence for the pathway from opinion registration to answer retrieval

The reviewer notes that the original submission did not directly show the relation between opinion registration and answer retrieval. This point is also raised by the other two reviewers (reJS and CU7Y). We have now run a path intervention to address this. We compare three conditions:

  1. Opinion heads patched. We patch the outputs of the top-10 opinion heads from the plain run into the deceptive run. This is the baseline effect reported in the paper.
  2. Opinion heads patched, retrieval heads frozen. On top of (1), we hold the outputs of the top-10 answer retrieval heads at their deceptive-run values.
  3. Opinion heads patched, retrieval queries frozen. Same as (2), but only the query inputs of the retrieval heads are held at their deceptive-run values.

The following table summarizes the results in the unit of normalized logit difference (1.0 = full recovery of the plain-run answer).

Llama-3.1-8BMistral-7BGemma-2-9B
(1) Opinion heads patched0.760.570.60
(2) + answer retrieval heads frozen0.260.160.16
(3) + answer retrieval queries frozen0.330.180.17

The results show that answer retrieval heads mediate the opinion heads’ effect on the output. First, freezing the answer retrieval heads removes two-thirds to three-quarters of the opinion heads' effect, demonstrating that the answer retrieval heads are the main route by which the opinion signal reaches the output. Second, freezing only the retrieval heads' queries removes nearly as much as freezing their full outputs, indicating that the opinion heads act primarily by redirecting what the retrieval heads attend to. We will include these results in the revised manuscript.

Direct evidence that pushback-specific heads suppress the correct answer

The reviewer notes that decodability of the correct answer from the pushback-specific heads does not directly establish their causal influence. This point is also raised by the other two reviewers (reJS and CU7Y). We have now tested this with a direct intervention.

If pushback-specific heads suppress the correct answer, ablating them should restore the support for the correct answer. We zero-ablate the pushback-specific heads (the green heads in Figure 6B and Figure S10D), and measure the restricted-softmax probability of the correct answer. With the heads intact, the probability of the correct answer after the pushback is 0.17 in Llama and 0.22 in Mistral. With the heads ablated, this probability rises to 0.77 and 0.94, and the second-turn accuracy rises from 0% to 87% in Llama and from 0% to 97% in Mistral. These heads therefore lower support for the correct answer, and removing them restores the support.

Further, ablating the pushback-specific heads lowers the support for all three alternative wrong answers without favoring a specific wrong answer. The model's preferred wrong answer is unchanged in 86% for Llama and 79% for Mistral. This indicates that pushback-specific heads act by suppressing the original correct answer rather than by supporting a particular wrong answer. Together with the probing result shown in Figure 6C, these results provide direct causal evidence that pushback-specific heads suppress the original correct answer. We will revise subsection 3.4 to include these results.

Replication on a newer-generation, larger model

The reviewer raises the point that all three main models are 7–9B instruction-tuned checkpoints from 2024. This point is also raised by Reviewer reJS. We agree, and we have now run the analysis on a model that is newer and larger. We choose OLMo-2-32B-Instruct (released March 2025). Every stage of the analysis reproduces the organization reported in the paper.

Where the two signals enter. Residual-stream patching shows the same ordering as Figure 3. Patching the plain run into the deceptive run begins to recover the answer at layer 39, whereas patching into the label-swap run has no effect until layer 46. The critical layer identified by the logit lens is layer 45. The user's opinion therefore enters the residual stream before answer retrieval, as in other models.

Two groups of attention heads. We run the per-head patching as Section 3.2. The two groups of attention heads in Figure 4 reproduce. The heads with a large causal effect only in the plain-to-deceptive patching all lie at or before layer 45. The heads with a large causal effect in both patching conditions all lie after it. The correlation between a head's causal effect in the two conditions is r=0.47r = 0.47 for heads at or before the critical layer and r=0.90r = 0.90 for heads after it.

Causal effect of the opinion heads. We identify opinion heads by the procedure of Appendix B.3, and ablate them cumulatively. Removing the top-10 opinion heads reduces the sycophancy rate from 100% to 33.9% and raises deceptive accuracy from 0% to 63.0%, while plain accuracy falls from 100% to 83.9%. These values are comparable with the results reported for other models.

We will include these results in our revised manuscript.

Clarification of contribution

The reviewer raises the point that the novel contribution of this work is not sufficiently stated. This point is also raised by Reviewer reJS. We now clarify our contribution, organized by the papers the reviewer mentions.

Relative to Wang et al.

Our results establish several things beyond Wang et al. First, Wang et al. showed that a stated opinion shifts the model's output preference in late layers, but did not establish when the opinion enters the computation. We show that the late-layer shift they report is only a downstream consequence of an earlier registration step. Second, Wang et al. located the effect with the logit lens, which detects an answer only once it is output-ready. We show in Figure 3 that the opinion is encoded in a form invisible to the logit lens, and can be isolated only through careful intervention design. Third, Wang et al. did not analyze individual attention heads. We identify individual attention heads and their concrete functions, which no layer-level intervention can do.

Relative to Genadi et al.

Genadi et al. showed that whether a model will reverse a correct answer under "Are you sure?" pushback is linearly decodable and steerable from a set of mid-layer attention heads. Our results establish two things beyond Genadi et al. First, their probe reads whether the model will flip. This label is heavily confounded with confidence, since the less confident a model is in its first answer, the more likely it is to flip. The results may largely reflect a generic confidence computation that is not specific to sycophancy. We instead map the full two-step logic of how sycophancy happens and identify what the attention heads at each step compute. Second, Genadi et al. studied only the pushback setting. We show that a stated opinion and pushback recruit distinct mechanisms. Even if their results are not confounded, they do not generalize to the stated-opinion setting. We provide, to our knowledge, the first mechanistic evidence that two behaviorally similar forms of sycophancy involve different attention heads.

Relative to Sun et al.

Sun et al. showed that under argument-based persuasion, a set of upstream heads reads persuasive keywords and a few mid-layer decision heads then copy whichever option they attend to. We agree that our account shares this two-stage skeleton. Our results establish three things it does not.

First, Sun et al. did not separate the heads that carry the user's opinion from the heads that output the answer. Their only contrast is a clean run against a persuaded run, which flags every head whose output changes when the persuasive passage is present. This missing separation is precisely what our methods provide. To see it concretely, the ten decision heads they report for Llama-3-8B all rank within the top heads in our plain-to-deceptive patching on Llama-3.1-8B. Among these heads, seven are among our top answer retrieval heads, and three are among our top opinion heads. What Sun et al. call decision heads is a mixture of two functional populations. Establishing this separation is what lets us isolate heads that carry a stated opinion, which are the central object of this work.

Second, Sun et al. did not establish what their upstream stage represents beyond the presence of particular words. They identified their upstream heads by replacing the key names in the persuasive passage with anonymous entities and selecting the heads whose outputs at the option tokens changed, so by construction those heads only detect which words appear in the passage. Our opinion heads are found by comparing runs with and without an opinion, and they carry the preferred answer even when the opinion is a description that shares no words with it. Our opinion heads barely overlap with their upstream heads: seven of their eight upstream heads for Llama have no effect in any of our conditions. The opinion heads that are the central object of this work are therefore not the upstream mechanism Sun et al. describe, and their method could not have identified them.

Third, as in both Wang et al. and Genadi et al., Sun et al. did not ask whether their mechanism is specific to argument-based persuasion. We establish that sycophancy is not a single mechanism. A stated opinion and content-free pushback are implemented by different heads with different functions.

Summary of clarification

In summary, prior work located sycophancy but left open when the opinion enters (Wang et al.), what the identified heads compute (Genadi et al.), and which heads carry the user's input and which output the answer (Sun et al.). Our paper answers all three, and in doing so shows, to our knowledge for the first time, that a stated opinion and pushback are implemented by distinct mechanisms. We will state these points as the contributions in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.