acceptodds
Under review as a conference paper at ICLR 2027

Who Says So? Authority Sycophancy in Large Language Models

Abstract

Large language models (LLMs) may revise their judgments not only in response to content, but also to who endorses a position. We study authority sycophancy, where an LLM changes an already-expressed evaluation after receiving an authority endorsement. We evaluate 10 LLMs across four authority families, two endorsement directions, and two settings with different levels of informational support. Authority-driven stance shift is widespread: mean DSS is positive in 144 of 160 authority conditions and exceeds the matched unspecified-source control in 139. However, the magnitude varies substantially across models and authority families, with no stable source hierarchy. Opposition generally induces stronger shifts than support, and authority-sensitive shift persists when substantive arguments are available. Transition analysis further shows that revisions most often move neutral judgments toward the endorsed position. These results reveal source authority as a substantial but highly heterogeneous influence on LLM judgment.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, authority sycophancy, social influence, source credibility, behavioral evaluation, model reliability, decision support

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SogoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how LLMs revise earlier judgments after learning that an external authority supports or opposes a position. Its main contribution is a systematic comparison of four source types: government authorities, experts, media organizations, and majority opinion. The authors construct two datasets covering standalone statements and debates with arguments on both sides.

They evaluate 10 models through two-turn conversations, using an unspecified-source endorsement as a control. The analysis includes directional stance scores, paired bootstrap comparisons, transition matrices, and qualitative examples. The authors report frequent but uneven shifts toward endorsed positions, often from initially neutral judgments, with substantial differences across models and sources.

Critical strengths
  1. The paper tests 10 models with four types of authority in two settings. This broad coverage helps show how responses to authority vary across models and sources.

  2. The paper tracks changes between agreement, disagreement, and neutral answers. This helps readers understand which judgments change and where the reported effects come from.

Critical weaknesses
  1. In Appendix A.4.4 (page 20), Qwen3.5-4B changes its score from 1 (neutral) to 2 (agree), while its explanation says the statement “should be rejected.” The paper does not report how often this happens or whether correcting these contradictions changes the results.

  2. The study measures sycophancy through movement toward the endorsed view. Following a relevant expert may represent reasonable updating. The “someone” control does not establish whether the change reflects excessive trust, which weakens the interpretation of the results.

  3. Section 3.3 always assigns the supporting argument to View 1 and the opposing argument to View 2. No reversed-order experiment is reported, leaving argument position as a possible explanation for differences between support and opposition.

  4. Using Table 2, opposition produces a larger score shift in 31 of 40 comparisons. After subtracting the corresponding “someone” control, this falls to 16 of 40. The claimed opposition advantage therefore needs an analysis that accounts for the control.

  5. Figure 1 (page 2) becomes blurry and pixelated when enlarged, particularly the small text in the Statement and Debate boxes. A sharper or vector version would improve readability. This is a minor presentation concern.

Questions for authors
  1. How often do the score–explanation contradictions and misread endorsements illustrated in Appendix A.4 occur? Do the main findings hold after correcting or excluding these responses?

  2. What evidence distinguishes excessive trust in authority from reasonable updating after advice from a relevant expert?

  3. Did you test debate prompts with the supporting and opposing arguments swapped? If so, do the differences between support and opposition remain?

  4. For Debate, is the opposition advantage statistically supported after subtracting the “someone” control for each direction? Using Table 2, I count 16 of 40 comparisons favoring opposition after this adjustment, compared with 31 of 40 before it.

  5. What sample count produced gpt-oss-20b’s DSS of −0.0154 in Statement–Support–State? Please clarify the denominator for each condition and how invalid or missing responses were handled.

Reviewer_egn3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether LLMs change their previously expressed judgments after being told that a government, expert, media outlet, or majority supports a position. The authors test ten models in statement-only and debate settings, using a two-turn conversation and a Someone control. They find that models often move toward the endorsed position, although the size of the change varies considerably across models and sources.

Critical strengths
  1. LLMs are often asked to evaluate positions and help users make decisions, so understanding how source endorsements affect their judgments is useful. Focusing on changes after a model has already expressed a position also gives the study a clear scope and makes the observed revisions easier to track.

  2. The authors test ten models from several model families, four types of authority, and both supportive and opposing endorsements. The datasets also cover a wide range of topics. This makes the study more informative than a demonstration based on a few selected prompts and allows readers to see substantial differences across models and settings.

Critical weaknesses
  1. It is unclear when a change of judgment should count as sycophancy. The paper mainly studies contested questions without an objectively correct answer, but counts any movement toward the endorsed position as sycophancy. Considering a relevant expert’s judgment can be reasonable, especially when the model is initially uncertain. This matters because the authors deliberately match expert and government sources to the topic. The Someone control shows that source identity matters, but does not establish that the model gives it too much weight. I would find the claim more convincing if the experiments compared relevant and irrelevant expertise, credible and unreliable sources, or endorsements that conflict with clearly stronger evidence.

  2. There are many evaluation conditions, but relatively few experiments that distinguish between explanations. Most of the study repeats the same intervention across different models and sources. This helps show how widely the effect occurs, but tells us less about why it occurs or how robust it is. The study uses one prompt formulation, and the supporting argument is always View 1 while the opposing argument is always View 2. Given the reported endorsement-direction errors, swapping their order seems particularly useful. I would also like to see whether the findings survive changes in endorsement wording and source assignment. The bootstrap intervals address uncertainty across examples, but do not answer these questions.

  3. Section 4.3 mainly compares raw DSS, even though the Someone control can already produce a much larger shift under opposition. For example, in Table 2, Ministral’s Debate–State DSS is 0.1419 under support and 0.9355 under opposition. After subtracting the corresponding Someone values, the differences become 0.2509 and 0.0884, reversing the ordering. Based on the rounded Table 2 values, opposition exceeds support in 31 of 40 raw comparisons, but only 16 of 40 comparisons after subtracting the respective controls. This is a descriptive calculation rather than an item-level significance test, but it suggests that the paper should directly test whether the additional effect of authority is larger under opposition. Eligible SR helps with score-boundary effects, but does not resolve this control-condition asymmetry.

  4. The analysis remains mostly descriptive, and the practical implications are not yet demonstrated. Statement and Debate are constructed independently, so their comparison cannot establish the effect of adding arguments. The authors acknowledge this limitation, but it still leaves that question unresolved. Similarly, source-subdimension comparisons do not test every source variant on the same items, making it difficult to separate source properties from item composition and wording. The paper reports considerable variation without sufficiently explaining what drives it. It also does not show whether the observed changes lead to less accurate answers, poorer use of evidence, or worse decisions. The reliability concern is plausible, but its practical consequences remain largely untested.

Questions for authors

See Weaknesses.

Reviewer_NjL5OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper discusses the topic of authority sycophancy in LLMs and tests whether models already evaluated on certain tasks observe a stance shift after forms of authority are provided to the model (focussing on state, expertise, media, and majority). Overall, the paper observes a shift in authority-driven judgments, but remarks that these are highly non-uniform across models. In detail, the paper designs two datasets (Statement and Debate, with the latter providing pros and cons in addition to the question) and tests a total of 10 models (open weights and proprietary ones). Across a total of 80 tests, 66 exhibit a shift in stance after authority influence (as measured via DSS). The main additional findings include a heterogeneity between models w.r.t. their degree of shift in stance as well as non-uniformity between agreement and opposition, with the latter leading to stronger authority sycophancy.

Critical strengths

The paper is overall well-presented and addresses an important concern with recent LLMs. The findings clearly illustrate that LLMs are influenced by authority sycophancy, thereby questioning their reliability in such settings. As such, I believe that this paper is likely relevant to the research community.

Critical weaknesses

I have a concern w.r.t. the paper's evaluation setup: when constructing the Debate first turn evaluation prompt, the order in which the supporting and opposing arguments are shown is not randomised (the models always see the pro arguments first, then the con ones). This can introduce biases into the evaluation protocol.

Furthermore, the paper would benefit from a clearer depiction of limitations in the main manuscript (limitations are only mentioned in the appendix). The paper would additionally benefit from an elaboration on how the problem of sycophancy could be addressed going forward, and what practical implications the findings have for such models.

Questions for authors

More of a comment: The term DSS is used several times on the first few pages of the paper without an introduction. It'd make sense to introduce this metric earlier on for increased clarity.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.