acceptodds
Under review as a conference paper at ICLR 2027

Wishes Are Not Beliefs: Two Kinds of Sycophancy in Language Models

Abstract

Sycophancy is usually measured as one number: how often a model adopts a user's wrong answer. That number conflates distinct failures. In three open-weight models, we hold the wrong option fixed and vary who endorses it, whether the user states a belief or a non-evidential wish, and whether a correct answer is already in the dialogue. A correct earlier answer can make a model more susceptible, not less: it more than triples how often Qwen 2.5 and Gemma 3 adopt a user's wrong belief. Most strikingly, wishes and beliefs come apart. Relative to a matched control instruction, telling a model to put facts above the user's preference all but eliminates Gemma 3's wish-following, yet leaves belief-following in Gemma 3 and Qwen 2.5 essentially unchanged. Gemma's responses show the difference: it honors a wish as a request ("honoring your request despite the mathematical inaccuracy”) but concedes to a belief as a correction ("I made a calculation error”). We also show that agreement data alone cannot distinguish evidence-, reward- and goal-based explanations. A fix aimed at what users want can miss a model's deference to what users think.

Primary area:
generative models
Keywords:
large language models, sycophancy, instruction following, factual reliability, task interpretation, source credibility, causal identification

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vAkqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that the usual sycophancy metric, how often a model adopts a user's wrong answer, conflates distinct failures. Holding a wrong option fixed on four-option MT-Consistency questions, the authors vary three things:

  • how the option is endorsed: mere mention, the user's belief, another participant's or an expert's belief, a user wish explicitly labeled as not evidence, an answer key, and others;
  • whether a correct assistant answer is already in the dialogue;
  • which instruction is appended.

On a 240-question panel with Qwen 3 8B, Qwen 2.5 7B and Gemma 3 27B, an injected correct prior makes Gemma 3 and Qwen 2.5 follow the user's wrong belief far more often (30→154 and 34→119 of 240). With the prior in place, Qwen 2.5 also follows the current user more than an expert.

On a 96-question panel, an instruction to put facts above the user's preference is compared with a length-matched "quality reminder." In Gemma 3 it removes most of the wish-following (ΓD=−55.2\Gamma_D=-55.2pp; −31.3-31.3pp in the BF16 replication) but barely moves belief-following (ΓB≈−5\Gamma_B\approx-5pp). The same instruction reduces belief-following in Gemma 2 and Qwen 3.5, whereas in Gemma 3 and Qwen 2.5 belief-following survives even a variant that names the user's belief. Gemma's responses suggest it treats a wish as a request and a belief as a correction.

Section 6 argues that evidence-, reward- and goal-based accounts can produce the same output distribution, so agreement data alone cannot distinguish them. It also shows that source coefficients fitted to next-token probabilities do not transfer to paraphrased panels.

Critical strengths

S1. The measurement is careful and honest. Every source is compared with a mere mention of the same option, which isolates what the endorsement itself adds. The factual-priority instruction is compared with a length-matched reminder rather than with no instruction, which is the right control. Unparseable and truncated answers stay in the denominator, and worst-case bounds are reported alongside. The unparseable answers from the V100 goal panel and the disclosure judgments on the source panel (71 responses) were re-read blind by a human. The goal panel was replicated with unquantized BF16 weights, and the paper is explicit about which analyses were added after the results were known. The point estimates in Tables 1–3 can be recomputed from the counts in the supplementary aggregate files, and the intervals match the stored evidence files. The individual effects are credible as measurements.

S2. The wish/belief distinction is useful. In Gemma 3, the same wrong answer comes from two different readings of the user's turn:

  • A wish is honored as a request, and the response often acknowledges the correct answer or the chosen option's inaccuracy (38 of 71 in the blinded audit).
  • A belief is accepted as a correction: with a prior, 238 of 240 belief responses contain an apology or a concession.

The two react very differently to the same remedy (ΓD−ΓB=−50.0\Gamma_D-\Gamma_B=-50.0pp; −27.1-27.1pp in BF16). This is a good argument that sycophancy should not be reported as a single number and that mitigations should be tested on more than one kind of prompt.

Critical weaknesses

W1. The core dissociation is clear in only one model, and the numbers move a lot with the run configuration.

Only Gemma 3 and Qwen 3.5 have wish effects large enough to test (Sec. 7), and the clean pattern, in which the wish effect vanishes while the belief effect stays nearly intact, appears only in Gemma 3.

  • Qwen 3.5 does reduce wishes more than beliefs (ΓD−ΓB=−11.5\Gamma_D-\Gamma_B=-11.5pp), but its belief effect drops as well (ΓB=−13.5\Gamma_B=-13.5pp), and 155 of its 864 responses hit the length cap, so the bounds that treat every capped response as unknown include zero.
  • Qwen 2.5's wish effect is too small to test.
  • The OLMo runs are uninformative.
  • The extra models were added after the main results were known, and a Gemma 4 31B run was stopped and not analyzed (App. C). This makes me wonder what else was run.

Within Gemma 3 there are further problems:

  • The main goal-panel run uses NF4-quantized weights.
  • In the H100/BF16 rerun, which changes the weights, compute precision and hardware together, 212 of the 864 parsed labels change, and the desire effect under the reminder falls from 63.5pp to 30.2pp.
  • That BF16 cell has 32 unparseable answers out of 96 (bounds 29.2–63.5pp), so Sec. 5's claim that in BF16 the two effects "start out comparable" is not yet established.
  • Each prompt gets one greedy generation, and the intervals resample only the 96 questions, so none of this run-to-run variation shows up in the error bars.
  • Gemma 3's ΓB\Gamma_B interval, [-14.6, 4.2], does not rule out a substantial reduction of a belief effect that is 32.3pp to begin with.

The current evidence supports a strong claim about Gemma 3 27B on these prompts. That is much narrower than the title, "Two Kinds of Sycophancy in Language Models."

W2. It is hard to tell whether the surviving belief-following is deference or an update on the user's testimony.

The prior is a bare one-line "The correct answer: GOLD." with no reasoning, inserted regardless of whether the model knows the answer. Depending on the model, 23–35% of the 240 questions are not answered correctly under the neutral no-prior prompt (supplementary files; unparseable answers counted as not correct), and some gold labels are debatable. In that situation, siding with a user who disputes an unexplained earlier answer can be reasonable, and doing so also satisfies an instruction to "select the option you judge factually correct." A belief effect that does not respond to the factual-priority wording beyond what the reminder already does is therefore also what testimony-based updating predicts. The authors mention this reading in Sec. 5, but the design cannot separate it from deference.

For Gemma 3, the paper's own subsets point the same way:

  • Under factual priority, the belief effect is 12.9pp on the 31 scientific-subject questions and about 34pp on the other 65 (27.1pp overall).
  • On the 12 direct-derivation items under factual priority, the user's belief yields 3/12 wrong-target declarations, against 2/12 for mere mention.
  • Qwen 2.5 looks closer to genuine deference: on the same 12 items it gives 10/12 against 4/12 under every instruction (supplementary files). Its wish effect, however, is too small to test.

Being directly derivable is not the same as the model knowing the answer, though: even under mere mention, with the correct prior in context, Gemma 3 picks the gold option on only 4–6 of these 12 items and Qwen 2.5 on 6–7. This is why Q1 asks for a split by model correctness.

I also think the practical message is overstated for Gemma 3. Measured against the original wording rather than the reminder, factual priority reduces the belief effect by 20.8pp on V100 and 30.2pp in BF16, roughly 40–50%, and most of that already comes from the generic reminder (15.6pp [5.2, 26.0] on V100 and 26.0pp [16.7, 36.5] in BF16, from the supplementary files). So Gemma 3's belief-following does respond to instructions; it just does not respond to the factual-priority content beyond what the reminder already achieves. That is a narrower conclusion than "a fix aimed at what users want can miss a model's deference to what users think."

Minor (not a basis for my score)

  • Section 6 is only loosely connected to the experiments. Proposition 1 is a fairly elementary identity, and the claim that the shared restriction fails rests on 7 of 1,938 (Qwen 3) and 21 of 1,710 (Qwen 2.5) eligible cells.
  • The main text is very dense, and several details that matter for reading the results appear only in the appendix: that the main Gemma run uses NF4 weights on V100, that the BF16 rerun changes 212 of the 864 parsed labels, and that its reminder-desire cell has 32 unparseable answers.
  • App. C lists batch size among the differences between Gemma's two configurations, but it reports batch size 8 for Gemma in both.
Questions for authors

Questions that could change my assessment

Q1. Could you report the goal-panel belief effect and ΓB\Gamma_B broken down by whether Gemma 3, Qwen 2.5 and Qwen 3 answered each item correctly under the neutral no-prior prompt of the source panel? For the Qwen models, please also split by probe confidence in the gold option under the neutral condition. Please also show the 31 scientific-subject questions and the other 65 separately, with intervals. If the belief-following that remains in Gemma 3 under factual priority also appears on items the model reliably gets right, my second concern would be largely resolved.

Q2. Could you adjudicate the 32 unparseable answers in the BF16 desire/reminder cell as you did for V100, and report the BF16 ΓD\Gamma_D, ΓB\Gamma_B and their difference on that basis? App. C says the V100 configuration was chosen for validity and throughput. Now that a BF16 replication exists, why does the NF4 run remain the primary result?

Q3. Why was the Gemma 4 31B run stopped? Were any other models or prompt variants run but not reported?

Scoped requests that would strengthen the paper

Q4. Could you rerun the goal panel for BF16 Gemma 3 with several samples per prompt (e.g., T=0.7T=0.7, k≥5k\ge5) and report intervals that include generation variance? Is ΓD−ΓB\Gamma_D-\Gamma_B still clearly negative?

Q5. Do the history effect and Gemma 3's resistance hold when the prior is the model's own sampled answer (with reasoning) rather than an injected one-liner? The answer would also help reconcile your results with SycEval (Fanous et al., 2025), which reports that preemptive rebuttals elicit more sycophancy than in-context rebuttals.

Novelty

Q6. Sec. 4 reports that Qwen 2.5 follows the current user more than another participant. What does this add beyond the following two studies?

  • Wang et al. (2025) report that "I believe" framings induce more sycophancy than "They believe" framings, and that user-expertise framing has a negligible effect.
  • Dubois et al. (2026) report that I-perspective framing amplifies sycophancy.

Both papers are cited, but this overlap is not discussed.

Minor: The abstract on the OpenReview page says three models, whereas the PDF abstract says seven.

6 comments
AuthorsOpenReview ↗

Response to Reviewer vAkq (1/2)

Thank you for recomputing our numbers from the supplementary files and for saying which answers could change your assessment. We begin with the two main weaknesses.

W1. Support beyond Gemma 3 27B and one configuration. The confirmatory panel covers ten models from four families in BF16, with every wrong option and a prompt under which 95.8–100% of answers parse (§5, pp. 5–8, Fig. 3).

  • Four models follow beliefs without following wishes, and none does the reverse. The instruction suppresses wish-following where it is substantial and leaves belief-following in Gemma 3 27B, Qwen 2.5, OLMo 3.1 and Qwen 3 (Fig. 3a, p. 8; Table 7, p. 24).
  • The wish-minus-belief contrast replicates in both models where the submission reported it: S = −16.5pp [−23.1, −10.4] in Gemma 3 27B and −20.1pp [−24.7, −15.7] in Qwen 3.5 9B (Table 5, p. 23).
  • With ten sampled generations per prompt, S keeps its sign and size in every included model (Table 9, p. 25). This also answers Q4.
  • The two effects start out comparable. Under the quality reminder, Gemma 3 27B's wish and belief effects are 34.0 and 32.6pp, with 99.2% of its 11,760 answers parsed (Table 5, p. 23; Table 7, p. 24).

On this larger sample Gemma 3 27B's belief effect also falls under the instruction (Γ_B = −18.8pp), with 13.9pp remaining; §5 gives the proportions for each model. In Gemma 3 12B the instruction removes most of both effects, so we report S model by model.

W2 / Q1. Deference, or an update on the user's testimony? The split you asked for is in Table 6 (p. 23). The table below gives belief-following that remains under factual priority (pp), by whether the model answers the question correctly under the neutral no-prior prompt.

Gemma 3 27BQwen 2.5OLMo 3.1
Answered correctly12.3 [8.6, 16.1]19.15.2
Not answered correctly19.026.816.4

The effect is present on questions the model gets right, so it is not only an update under uncertainty. It is larger on the other questions, so part of it may well be testimony-based updating, and §5 says so.

Two further arms address the bare one-line prior. With the model's own answer and its reasoning as the earlier turn, Gemma 3 27B gives S = −16.7pp [−25.8, −7.5]. With an injected incorrect answer, S = −20.4pp [−27.5, −13.3], which argues against stronger copying of a correct prior (Table 12, p. 26).

Table 3 (p. 21) has the discovery-panel splits you requested: by neutral correctness, by probe confidence for the Qwen models, and by subject, each with intervals. The subject difference you saw on 96 questions does not appear on the full panel. Gemma 3 27B's remaining belief-following is 17.1pp [11.4, 23.2] on the 76 scientific-subject questions and 12.4pp [8.3, 16.7] on the others (Table 14, p. 27).

The practical message for Gemma 3. §5 now reports your decomposition. In BF16 the generic reminder alone lowers Gemma 3's wish and belief effects by 41.7 and 26.0pp, so belief-following does respond to instructions in general. The message is worded accordingly (abstract; §8, p. 9): a fix aimed at what users want can miss deference to what users think, and can cost legitimate updating.

AuthorsOpenReview ↗

Response to Reviewer vAkq (2/2)

Q2. The 32 unparsed BF16 answers. A separate model labeled them without seeing the gold option, the target or the model identity. The wish effect is then 39.6pp [29.2, 50.0], next to a belief effect of 38.5pp, with Γ_D = −35.4pp [−47.9, −22.9] and Γ_B = −4.2pp [−14.6, 6.3] (App. F, p. 21). A human re-annotated all 81 discovery-panel unknowns blind and agrees with these labels on 79 (App. F). BF16 is now the primary configuration, and the confirmatory panel uses it throughout. We had used 4-bit weights only because the 27B model in BF16 did not fit our first GPU (App. C, p. 16).

Q3. The Gemma 4 run; unreported runs. We stopped the Gemma 4 run while it was still loading weights, before any prompt was generated, to free the GPU before the deadline (App. C, p. 17). It is now part of the confirmatory panel, where it follows neither wishes nor beliefs (both below 1pp). Table 7 (p. 24) reports all four instruction conditions for all ten models, App. G (p. 22) marks the runs added after the plan was frozen, and the revised supplementary material has a complete run inventory.

Q4. Several samples per prompt. With ten sampled generations per prompt, Gemma 3 27B in BF16 gives S = −21.9pp [−33.5, −10.3] on the discovery panel. This interval includes generation variance (Table 11, p. 26). Table 9 (p. 25) gives the corresponding confirmatory estimates.

Q5. The model's own earlier answer; SycEval. With the model's own complete answer as the earlier turn, the history effect remains: belief-following rises by 34.6pp in Gemma 3 and by 15.0pp in Qwen 2.5 (Table 12, App. G). The contrast S remains too, as reported in the first part of this reply. A comparison with SycEval needs care, because our user turn is a bare statement of belief with no argument, and our effects are net of a mere mention of the same option.

Q6. Beyond Wang et al. (2025) and Dubois et al. (2026). Our user-versus-other result agrees with both studies, and §2 (p. 3) now discusses them. What it adds is the role of dialogue history. In Qwen 2.5, the user's belief and another participant's are followed on 34 and 24 of 240 questions with no prior answer, and on 119 and 55 with a correct one (§2; Table 1). Neither study compares a wish with a belief, or tests how a remedy affects each.

Minor points.

  • §6 (pp. 8–9) is now one proposition with its scope stated. The fits and the rank-restriction cells are in App. H (p. 28) and App. K (p. 32), presented as a diagnostic.
  • The configuration details you listed are with the discovery panel in App. C (pp. 15–17) and App. F (p. 21), and the batch-size statement is corrected (App. C, p. 16).
  • The abstract on the OpenReview page now matches the PDF.

We hope the confirmatory panel answers the first weakness and the known-question and own-answer results answer the second. If so, we would be grateful if you would consider revising your score.

AuthorsOpenReview ↗

Full Result Details for Reference

Thank you for recomputing our numbers yourself; we truly appreciate the care this took. Because you did so, we wanted to give you the full tables directly rather than only pointers to them. Below are the confirmatory results, the robustness estimates and the analysis by prior content that support our answers to W1, W2, Q4 and Q5.

W1. Confirmatory panel, all 240 questions (pp, 95% intervals; Table 5, p. 23). Δ: effect under the quality reminder; Γ: change under factual priority; S = Γ_D − Γ_B. Incl.: meets the inclusion rule fixed in the plan. § marks estimates whose missing-outcome bounds cross zero although the interval does not.

ModelIncl.Δ_D (wish)Δ_B (belief)Γ_DΓ_BS
Gemma 3 27Byes34.0 [29.4, 38.6]32.6 [28.2, 37.2]−35.3 [−40.7, −30.0]−18.8 [−23.3, −14.3]−16.5 [−23.1, −10.4]
Gemma 3 12Byes13.8 [10.4, 17.1]38.9 [34.2, 43.8]−10.8 [−14.0, −7.6]−34.9 [−40.3, −29.6]24.0 [18.6, 29.7]
Qwen 3.5 9Byes32.9 [28.5, 37.5]13.3 [10.1, 16.5]−30.0 [−34.7, −25.4]−9.9 [−13.1, −6.7]−20.1 [−24.7, −15.7]
Gemma 4 31Bno0.1 [−0.3, 0.7]0.7 [0.0, 1.5]−0.4 [−1.1, 0.1]−0.6 [−1.5, 0.3]0.1 [−0.7, 1.0]
Gemma 2 27Bno−0.4 [−1.5, 0.7]19.2 [15.0, 23.6]−0.8 [−1.9, 0.3]−18.3 [−22.6, −14.3]17.5 [13.3, 21.8]
Qwen 3.5 27Bno1.1 [0.3, 2.1]§1.9 [0.7, 3.3]−1.1 [−2.2, −0.1]§−2.6 [−4.2, −1.2]1.5 [0.6, 2.6]§
Qwen 3 8Bno2.9 [1.5, 4.4]7.4 [5.3, 9.7]−1.5 [−3.3, 0.3]−2.6 [−4.7, −0.7]1.1 [−0.8, 3.1]
Qwen 2.5 7Bno3.2 [1.9, 4.6]25.8 [21.9, 30.0]−2.8 [−4.4, −1.2]−4.4 [−7.1, −1.8]1.7 [−1.1, 4.3]
Phi-4 14Bno1.2 [0.4, 2.2]2.9 [1.7, 4.3]−1.0 [−1.9, −0.0]−0.7 [−1.9, 0.6]−0.3 [−1.4, 0.8]
OLMo 3.1 32Bno1.2 [0.1, 2.4]§11.5 [8.8, 14.4]−1.7 [−3.2, −0.1]§−3.5 [−6.2, −0.7]§1.8 [−1.0, 4.6]

W1 / Q4. S under sampling and under each labeling layer (Table 9, p. 25):

EstimateGemma 3 27BQwen 3.5 9BGemma 3 12B
Greedy, parser (primary)−16.5 [−23.1, −10.4]−20.1 [−24.7, −15.7]24.0 [18.6, 29.7]
10 samples, parser−16.1 [−21.8, −10.5]−15.3 [−17.9, −12.8]23.5 [18.4, 28.5]
Greedy, separate-model labels−18.6 [−25.0, −12.5]−17.5 [−21.5, −13.6]25.8 [20.3, 31.4]
Greedy, adjudicated labels−18.5 [−25.0, −12.4]−18.1 [−22.2, −14.0]25.8 [20.1, 31.5]

W2 / Q5. S by the content of the earlier assistant turn (240 questions unless noted; Table 12, p. 26):

Earlier assistant turnGemma 3 27BQwen 3.5 9BGemma 3 12B
Own answer, with reasoning−16.7 [−25.8, −7.5]−15.0 [−21.2, −8.8]23.8 [15.8, 31.7]
… only questions where it is correct−12.6 [−22.5, −2.2] (182)−16.7 [−23.4, −9.9] (192)30.4 [22.2, 39.2] (171)
Own answer, one line−20.8 [−30.0, −12.1]−20.1 [−26.8, −13.0] (239)25.8 [18.8, 33.3]
Injected correct answer−20.0 [−29.6, −10.8]−22.1 [−28.7, −15.4]23.3 [16.2, 30.8]
Injected incorrect answer−20.4 [−27.5, −13.3]−11.2 [−16.7, −5.8]15.0 [8.3, 21.7]

Relative to no prior answer, the model's own complete answer raises the belief effect under the quality reminder by 34.6pp [28.3, 40.8] in Gemma 3 27B and by 15.0pp [10.0, 20.0] in Qwen 2.5 (App. G).

Reviewer_8SNvOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigate sycophancy in LMs using 4-choice questions from the MT-consistency bank (sampling 240 or 96 questions in the two main experiments). They argue that just looking at wrong answer declarations in a single scenario hides what factors affect the behavior. They evaluate Gemma 3 27B, Qwen 2.5 7B, and Qwen 3 8B changing (mostly) two parameters: (1) the chat history (containing or not containing the correct answer as a message from the model earlier in the conversation) and (2) the question from the user, with various differences in hints or information about incorrect answer choices. Two important cases are where the user says they believe an incorrect option is correct (user belief), and where the user says that selecting an incorrect option would please them (user wish). They find (Figure 2) that correct prior answers increase the wrong answer following in many cases. They further check if this is a confusion about the request and include a scenario to remind the model to prioritize factuality and compare it to a more generic (but similar length) reminder to consider all options. They find that for Gemma 3 and Qwen 3.5, this factual reminder substantially reduces wrong answer selection in the user use case but reduces it far less in the user belief case. This pattern is not consistent for the other models. They also discuss some theoretical results and limitations of the study.

Critical strengths
  1. The authors try to be thorough in several followup experiments and provide significant information about the additional experiments that were run, which is admirable.
  2. The most interesting results are well highlighted and clear.
Critical weaknesses
  1. The paper overall is not especially clearly written or presented.
    1. The tables and figures (especially tables 1 and 2) could be improved. For example, Table 2 is not immediately understandable from the caption. Figure 1 would have been more useful if it showed the explicit chat format used (e.g., for Gemma 3) and demonstrated the case where the correct answer is included as said earlier by the model.
    2. The main claim is that a factual-priority reminder suppresses “wish-following” more than “belief-following” but this is not really a consistent result for all the models. Even if it was, it is not clear why that is interesting in a broader sense.
    3. More examples appear in my reading notes.
  2. The motivation for these experiments is unclear. How does this connect to a broader research program or question? If Gemma 3 is especially prone to wish following when given a correct prior answer, why should we care?
    1. The introduction motivates sycophancy benchmark numbers as not telling the whole story but no alternative for general evaluation is argued for and the results seem disconnected to this.
    2. This sort of thing has been looked at a lot. The related work section does not do a good job of situating why this study is needed and what it is adding to the literature.
  3. Only a single wrong answer suggestion per question is investigated in all experiments. This limits the usefulness of the results and calls into question the definition of sycophancy being examined.
    1. Really, we care about the model changing its answer to match the suggestion but this tendency is not entirely measured by only looking at one suggestion. Why not suggest all answers and measure how often the model switches from a correct answer (in the neutral scenario) to the suggested answer? This gives the overall tendency.
    2. How much of the increased wrong answer following comes from correct-to-wrong transitions versus wrong-to-wrong transitions single suggestions examined?
  4. Some models and experiments have a large number of truncation and parsing issues. This is considered in the analysis but why present such a confusing story rather than fix the experimental setup and rerun on a different sample?
    1. Gemma 3 and Qwen 3.5 especially.

Below are my reading notes that are contain issues that are not necessarily critical. They may repeat issues mentioned above.

  • “We show that agreement data alone cannot distinguish evidence-, reward-, and goal-based explanations” - I don’t know what is meant by this sentence from the abstract.
    • After reading the abstract I don’t really understand what we are supposed to learn from this paper. The correct earlier answer result mentioned is interesting but what is this supposed to mean more broadly? What is the motivation here?
  • No relevant literature in event factuality prediction is cited, an area where this distinction between wishes and beliefs is made very explicitly for several datasets. These are datasets the models almost certainly have been trained on. I will list some references below.
  • The technical supplement contains no inference, answer-parsing, or statistical-analysis code and no complete raw responses.
  • After reading the introduction I am not sure I understand the motivation here. What is the broader goal of this work?
    • How does learning about the behavior of these small models in this specific scenario connect to a larger research goal, like telling us something about sycophancy behavior in general? The results are not particularly consistent even for these parameter matched models.
  • A lot of responses from Gemma 3 failed to parse even after reruns and this is the model where the largest effect is observed. In gemma_3_core_summary.csv, the parser could not identify an answer 80/240 times for the neutral/correct_prior case and those are (mostly) not due to token budget issues.
    • What is going on here? It seems like the results just count these as not selecting the suggested option but really they add significant uncertainty that is not accounted for in the 95% error bars.
  • A similar thing happens for Qwen 3.5 in experiment 2? 155/864 hit the cap.
    • Appendix D says that the parser essentially takes the first answer output and uses that even if the rest of the answer was truncated so for many of these runs we don’t know if the model would have switched its answer later. This is somewhat concerning. Why are so many answers getting truncated?
  • It is very strange to have changed the quantization from 16-bit to 4-bit for Gemma in the experiments relevant to Figure 3 and Table 2.
    • Why do this at all and why retain it in the presentation of the paper? An unquantized replication is reported and the pattern is roughly the same, although the effect size changes substantially. But again, why do this at all?
  • Table 2 is just a terrible representation of this information. It is very difficult to follow and obscures the core results which seem more mixed than the prose would suggest.
  • Why are there all these additional results for other models that appear only in the Appendix. This is a strange choice of presentation and comes across disorganized.
    • Some of these results aren’t even usable. E.g,, Olmo 3 Instruct fails to parse ~21% of the time in the quality vs factuality reminder experiment. Why include them at all?
  • The theoretical results in section 6 seem pointless.
    • For one, a reward is not really something that just changes an answer probability after the fact so that is a misleading naming convention.
    • For another, this seems like a straightforward result about measurement degeneracy. The equivalence seems to follow from the flexibility of the chosen constructions.
  • The authors don’t really test the most interesting counterfactual here. Like sycophancy isn’t when the model says what you say or says the wrong answer. Isn’t it more like when it would have gotten the answer right but decides to match what you said instead, for whatever reason but ideally to please the user (this is harder to show)? Presenting just a single wrong option doesn’t check if the model switches its answer from the correct one to the wrong one because of the user's suggestion.
    • In general it isn’t clear what exact definition of sycophancy is being used for this paper.
    • What should have happened is that every answer should have been suggested and then the experiments could have checked across all of them how much it influenced divergence from the correct answer, both in the case where the model would have gotten the right answer without influence and the wrong answer. This seems like a serious experimental design flaw.

Additional Related Work (authors might find interesting)

These are from work in event factuality prediction which has made this distinction between wishes and various beliefs for a long time now. The expert case examined requires distinguishing reported belief from author belief, for example. They might be interesting to look at, especially since models are possibly trained on this data.

  • Prabhakaran, V., et al. (2015). A New Dataset and Evaluation for Belief/Factuality. In Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics, pp. 82–91. https://aclanthology.org/S15-1009/
  • Saurí, R., & Pustejovsky, J. (2012). Are You Sure That This Happened? Assessing the Factuality Degree of Events in Text. Computational Linguistics, 38(2), 261–299. https://aclanthology.org/J12-2002/
  • White, A. S., Rudinger, R., Rawlins, K., & Van Durme, B. (2018). Lexicosyntactic Inference in Neural Models. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4717–4724. https://aclanthology.org/D18-1501/
  • Jiang, N., & de Marneffe, M.-C. (2021). He Thinks He Knows Better than the Doctors: BERT for Event Factuality Fails on Pragmatics. Transactions of the Association for Computational Linguistics, 9, 1081–1097. https://aclanthology.org/2021.tacl-1.64/
Questions for authors
  1. How much of the increased wrong-answer following comes from correct-to-wrong transitions versus wrong-to-wrong transitions for the suggestions examined? Does the main wish-versus-belief result hold on questions the model answers correctly without influence?
  2. In general what is going on with all these parsing issues?
  3. Why not suggest all four answers for every question and measure how each suggestion affects the model’s answer, distinguishing questions it initially gets right from those it gets wrong? The effects could differ substantially depending on the suggested option. Does the wish-versus-belief result hold across those alternatives?
6 comments
AuthorsOpenReview ↗

Response to Reviewer 8SNv (1/2)

Thank you for the detailed review and the reading notes. We start with motivation, because the notes ask for it several times, and then take your three questions.

Motivation. Sycophancy is usually reported, and mitigated, as one agreement rate. Three results in the paper show what that rate hides.

  • An instruction to put facts first looks like a fix on an agreement score. Measured by case, it suppresses wish-following where that is substantial, leaves 8–21pp of belief-following in three models, and lowers updating on reliable evidence in six of ten (§5, pp. 5–8; Tables 7 and 15).
  • A correct answer already in the dialogue should protect a model. In two models it does the opposite: adoption of the user's contrary belief goes from 30 to 154 of 240 questions in Gemma 3 and from 34 to 119 in Qwen 2.5 (§4, p. 5, Table 1).
  • Four models follow a user's belief without following a user's wish, and none does the reverse (§5, Fig. 3a).

The evaluation we argue for is to report wishes, beliefs and evidence separately (§3, p. 4; §7–8, p. 9).

Q3. Suggest every option. We adopted this design. The confirmatory panel displays every option on all 240 questions for ten models. Taken one wrong option at a time, the wish-minus-belief contrast S has the same sign for every option in each model that enters the test (Table 14, p. 27):

ModelFirst wrong optionSecondThird
Gemma 3 27B−16.7−11.7−21.2
Qwen 3.5 9B−12.9−27.1−20.4
Gemma 3 12B+16.2+29.2+26.7

Q1. Correct-to-wrong transitions, and questions the model answers correctly. Most of the effect is a correct answer given up for the user's option. Of Gemma 3 27B's 32.6pp belief effect, 25.3pp are correct-to-target transitions (Table 8, p. 24). The main result also holds on the questions each model answers correctly without a suggestion, which is the counterfactual you describe (Table 6, p. 23). For Gemma 3 27B on those questions, S = −11.4pp [−18.7, −4.0], and 12.3pp [8.6, 16.1] of belief-following remains under the instruction.

Q2. The parsing issues. They came from the original prompt, which allowed answers as option text under a 384-token cap. The confirmatory prompt asks for one option letter with a 1,024-token budget, and 95.8–100% of answers parse (App. G, p. 22, Table 7). We also checked that the results do not depend on the parser: a separate model relabeled every response of the included models from its full text without seeing the target, and S keeps its sign and size (Table 9, p. 25). A blind human audit agrees with the adjudicated labels on 87 of the 100 hardest responses (App. G).

The Gemma 3 cell you found (80 of 240 unparsed) is the neutral prompt with a correct prior. No reported contrast uses it, since all contrasts are taken against mere mention. Labeling those answers blind changes no conclusion (App. D, p. 20).

AuthorsOpenReview ↗

Response to Reviewer 8SNv (2/2)

The definition of sycophancy. §3 (p. 4) now states it: a shift toward the user's option beyond what the content of the user's turn warrants. A wish that the prompt itself calls non-evidential gives no reason to change a factual answer, so any shift beyond mere mention counts. A belief counts at least on questions the model answers correctly on its own, and we report those separately. Reliable evidence is the opposite case: the answer should change, and a remedy must not prevent that.

Consistency across models. What is consistent across the ten models is the relation between the two behaviors. Every model that follows wishes also follows beliefs. Four follow beliefs without wishes, and none does the reverse. The instruction suppresses wish-following where it is substantial and does not always suppress belief-following. How much belief-following survives differs between models, including within one family. We report that as a result, since a single score would hide it (§5, pp. 5–8, Fig. 3; Table 7).

Relation to prior work. Thank you for the event-factuality references; §2 (p. 3) now discusses all four. They establish the linguistic distinction between a wish and a belief. Our question is whether models act on it when answering and whether a remedy does, and to our knowledge that has not been tested. §2 also says how our source result relates to recent work on first-person framing. Two other results are new as far as we know: a correct earlier answer increases belief-following, also when the model wrote that answer itself (§4, p. 5; Table 12), and the instruction reduces legitimate updating (Tables 15–16, p. 28).

Presentation.

  • Figure 1 (p. 2) shows the exact Gemma 3 chat template, with the earlier correct answer in place.
  • The main result is one figure for all ten models (Fig. 3). The table you found hard to read is in App. F (p. 21).
  • Table 1 (p. 6) is grouped into belief, wish and "whose belief" blocks.
  • All confirmatory runs use BF16. The 4-bit run was forced by the memory of our first GPU (App. C, p. 16) and is now secondary.
  • Models added after the plan was frozen are shown in gray and are not counted (Fig. 3; App. G, p. 22).

The theory section. The section makes one methodological point. Accounts based on evidence, reward and goals can produce identical agreement rates, so fitting those rates cannot distinguish them. That is why our evidence comes from interventions. We agree that the identity is elementary, and the paper says so (§6, pp. 8–9). "Reward" there means the objective whose KL-regularized optimum is the observed policy. The section is now a single proposition with its scope stated, and the abstract sentence reads: "Because evidence-, reward- and goal-based accounts can yield identical agreement rates, our evidence comes from interventions, not fitted rates."

Code and raw responses. The revised supplementary material contains the inference, parsing and analysis code and every greedy generation behind the reported results.

If the all-options panel and the points above address your concerns, we hope you will consider revising your score.

AuthorsOpenReview ↗

Full Result Details for Reference

Several of your notes asked how the pattern holds across models, so we bring the complete numbers for all ten models here, together with the source-panel results. Everything below matches the revised PDF.

Wish / belief effects in the confirmatory panel (source − mention, pp; 240 questions × 3 wrong options; Table 7, p. 24). Each cell is wish / belief.

ModelParsedNo instructionQuality reminderFactual priorityBelief-named variant
Gemma 3 27B99.2%53.5 / 48.934.0 / 32.6−1.2 / 13.93.1 / 14.9
Gemma 3 12B98.8%20.0 / 50.113.8 / 38.92.9 / 4.02.4 / 4.3
Gemma 4 31B100.0%0.1 / 0.40.1 / 0.7−0.3 / 0.1−0.3 / 0.3
Gemma 2 27B99.2%0.1 / 43.8−0.4 / 19.2−1.2 / 0.8−1.1 / 0.3
Qwen 3.5 9B95.8%57.9 / 18.932.9 / 13.32.9 / 3.53.6 / 4.6
Qwen 3.5 27B99.1%0.1 / 2.51.1 / 1.90.0 / −0.7−0.6 / −1.0
Qwen 3 8B99.7%6.5 / 12.62.9 / 7.41.4 / 4.71.5 / 4.9
Qwen 2.5 7B99.9%0.4 / 18.23.2 / 25.80.4 / 21.40.6 / 19.0
Phi-4 14B99.1%1.8 / 4.91.2 / 2.90.3 / 2.2−0.4 / 2.9
OLMo 3.1 32B98.7%−1.8 / 15.01.2 / 11.5−0.4 / 8.1−0.1 / 8.8

Read across a row, a model that follows wishes also follows beliefs, and factual priority brings every wish effect to 3pp or less. Read down the factual-priority column, the belief effect that remains ranges from below 1pp to 21.4pp.

Source panel (belief − mention or wish − mention, pp; Table 1, p. 6):

Gemma 3Qwen 2.5Qwen 3
Belief, no prior answer+4.6 [1.7, 7.5]+4.6 [1.7, 7.5]+13.3 [8.8, 17.9]
Belief, correct prior answer+55.4 [49.2, 62.1]+35.0 [29.2, 41.2]+12.1 [8.3, 16.2]
Increase due to the prior+50.8 [43.8, 57.9]+30.4 [24.2, 37.1]−1.3 [−6.7, 3.8]
Wish, no prior answer+6.3 [3.3, 9.6]−0.8 [−2.5, 0.8]+5.4 [2.1, 8.8]
Wish, correct prior answer+60.8 [54.2, 67.5]+3.8 [1.2, 6.7]+4.2 [1.7, 7.1]
Current user − other participant (correct prior)—+26.7 [21.2, 32.5]+6.7 [3.8, 10.0]
Expert − current user (correct prior)—−8.3 [−12.5, −4.2]+12.1 [7.9, 16.7]
Reviewer_57p2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a question of whether a model follows a user’s wrong answer for different reasons when the user expresses a wish versus a belief. The authors test this by presenting the same wrong option in different ways, including as a user’s wish or belief, and by varying whether a correct answer already appears in the conversation. They find that the factual-priority instruction reduces wish-following in the models where it is substantial, but has less effect on belief-following.

Critical strengths
  1. The paper makes an interesting distinction between wishes and beliefs. A model can give the same wrong answer when following a user's wish or accepting a user's belief, but the two cases may call for different responses. This distinction makes the measure of sycophancy more informative.

  2. The experiments use clear controls. The authors hold the suggested option fixed while changing how it is presented, and compare the factual-priority instruction with a general quality reminder. The authors also test unquantized weights and an instruction that explicitly mentions the user's belief.

  3. The response analysis goes beyond the final answer. The authors check whether responses that select a wrong option also state the correct fact, and use human adjudication to examine ambiguous outputs. They keep unknown answers in the denominator and report bounds for their main effects.

Critical weaknesses
  1. The main finding has limited support across models. Only Gemma 3 and Qwen 3.5 show substantial wish-following in the main instruction experiment. For Qwen 3.5, 155 of 864 responses hit the generation limit. If all capped responses are treated as unresolved, the conservative bounds for both the wish and belief effects include zero. This does not invalidate the reported results, but makes it harder to tell how well the finding extends beyond Gemma 3. Testing the capped responses with a larger budget would help.

  2. The practical implications for reducing sycophancy remain unclear. The factual-priority instruction reduces wish-following, but does not consistently reduce belief-following. A useful intervention should also allow the model to change its answer when the user provides reliable new evidence. In the Appendix, it examines updating on reliable evidence, but does not test the factual-priority instruction across different levels of evidence reliability. Such a comparison would help show whether the distinction can guide a method that resists unsupported claims while accepting valid corrections.

  3. Can the Qwen 3.5 capped responses be audited or completed under a common larger budget, and do the desire, belief, and desire-minus-belief interactions remain negative? This would establish whether the second family supports the central dissociation beyond the primary declaration parser.

Questions for authors

Please check my weakness section.

3 comments

Thank you for the review and for two concrete requests. We ran both.

Qwen 3.5 at a larger budget. We reran the Qwen 3.5 panel at 2,048 tokens with the original prompts (Table 10, p. 25). Capped responses fall from 155 to 32 of 864. The wish-minus-belief contrast is S = −19.8pp [−29.2, −10.4], and it stays negative when every capped response is treated as unknown ([−25.0, −13.5]). The wish interaction is −26.0pp. The belief interaction is small, −6.2pp [−12.5, 0.0].

Support from a second family, beyond the parser. The confirmatory panel uses a different design (240 questions, every wrong option, an option-letter prompt). There Qwen 3.5 9B gives S = −20.1pp [−24.7, −15.7] (Table 5, p. 23). The value is similar with ten sampled generations per prompt, and when a separate model relabels every response from its full text without seeing the target (Table 9, p. 25).

Factual priority at different levels of evidence reliability. We ran this comparison (Tables 15–16, p. 28). In each synthetic item, a device of stated reliability (0.6 or 0.9) displays a label. The correct response is to switch only when the evidence makes that label the most probable one.

  • Factual priority lowers correct updating in six of ten models at reliability 0.9, and in four at 0.6. Qwen 3.5 27B then keeps its earlier answer on almost every item.
  • Unwarranted switching does not drop to compensate. In Qwen 3.5 9B it rises from 0 to 21.5%.

Practical implications. The evidence experiment gives the distinction a practical use. A method that should resist unsupported claims and accept valid corrections has to be tested on three cases: wishes, beliefs about questions the model answers correctly on its own, and evidence it should accept. The instruction we tested does well on the first, does well on the second only in some models, and does harm on the third in most. An agreement score would report it as an improvement on all counts. §7 (p. 9) and §8 (p. 9) now make this recommendation.

We hope the rerun and the evidence experiment settle the two points you raised.

AuthorsOpenReview ↗

Full Result Details for Reference

Thank you again for both requests. To spare you a search through the appendix, here are the full results of the rerun and of the evidence experiment, each with its table and page number.

Qwen 3.5 9B at 2,048 tokens (discovery panels; Table 10, p. 25). Intervals in brackets; all-capped bounds in parentheses.

PanelCappedΓ_D (wish)Γ_B (belief)S = Γ_D − Γ_B
Original goal panel32/864−26.0 [−36.5, −15.6] (−33.3, −19.8)−6.2 [−12.5, 0.0] (−13.5, −1.0)−19.8 [−29.2, −10.4] (−25.0, −13.5)
Belief-named panel28/864−25.0 [−35.4, −14.6] (−30.2, −18.8)−8.3 [−15.6, −2.1] (−13.5, −1.0)−16.7 [−27.1, −6.2] (−22.9, −11.5)

Correct updates on reliable evidence (items that require switching; Table 15, p. 28). Rate under the quality reminder (%), and the paired change under factual priority (pp, 95% interval):

ModelReliability 0.6: reminderFactual − reminderReliability 0.9: reminderFactual − reminder
Gemma 3 27B9.7−6.9 [−13.4, −1.5]12.5−8.3 [−12.4, −4.6]
Gemma 3 12B48.6−1.4 [−18.5, 15.4]69.3−20.8 [−29.4, −12.0]
Gemma 4 31B100.00.0 [0.0, 0.0]100.00.0 [0.0, 0.0]
Gemma 2 27B18.170.8 [58.7, 82.3]39.135.9 [27.5, 44.3]
Qwen 3.5 9B98.6−25.0 [−36.0, −14.7]75.0−18.2 [−27.3, −8.9]
Qwen 3.5 27B100.0−100.0 [−100.0, −100.0]100.0−94.3 [−97.4, −90.8]
Qwen 3 8B25.00.0 [−5.6, 5.6]10.96.2 [2.6, 10.3]
Qwen 2.5 7B76.4−13.9 [−22.5, −6.2]55.2−7.3 [−11.9, −3.1]
Phi-4 14B93.16.9 [1.4, 13.3]100.00.0 [0.0, 0.0]
OLMo 3.1 32B91.7−8.3 [−19.4, 2.5]80.2−17.7 [−25.8, −10.1]

Table 15 also gives the belief-named variant.

Unwarranted switches (%) to the displayed label where it is not the most probable option, under quality reminder / factual priority / belief-named variant (Table 16, p. 28; point estimates; the 0.9 column rests on 24 items):

ModelReliability 0.6Reliability 0.9
Gemma 3 12B16.0 / 32.6 / 40.329.2 / 33.3 / 41.7
Gemma 2 27B12.5 / 54.9 / 54.212.5 / 29.2 / 25.0
Qwen 3.5 9B0.0 / 21.5 / 36.10.0 / 16.7 / 33.3
Qwen 3 8B4.2 / 13.9 / 17.40.0 / 8.3 / 12.5
Qwen 2.5 7B46.5 / 41.0 / 43.837.5 / 33.3 / 33.3
Phi-4 14B2.8 / 2.1 / 0.00.0 / 0.0 / 0.0
OLMo 3.1 32B2.1 / 0.0 / 0.00.0 / 0.0 / 0.0

Gemma 3 27B, Gemma 4 31B and Qwen 3.5 27B make no unwarranted switch in any condition.

Dear Reviewer, thank you once more for your careful review. As the discussion phase ends on November 18, we just wanted to follow up on our response. We would greatly appreciate any further feedback, and we remain happy to address any remaining points.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all three reviewers

We appreciate the three reviewers. The reviews agree on the paper's central idea: a model can give the same wrong answer because the user wants it or because the user believes it, and the two cases need not respond to the same remedy. They also credit the controls, namely the mention baseline, the length-matched reminder, and unknown answers kept in the denominator with bounds.

The reviews share one question: does the wish/belief distinction hold beyond Gemma 3, beyond one wrong option per question, and beyond the answer parser? To answer it we ran a confirmatory panel and froze its design, inclusion rule, hypotheses and analysis plan before generating any output. It uses all 240 questions, displays every wrong option, and covers ten open models from four families in BF16. Its prompt asks for an option letter, and 95.8–100% of answers parse (§5, pp. 5–8, Fig. 3, App. G, p. 22).

The submitted findings replicate.

  • Factual priority suppresses wish-following in every model with a substantial wish effect (Γ_D = −35.3, −10.8 and −30.0pp). Under it, no model follows a wish by more than 3pp (§5, Table 5).
  • Belief-following yields in Gemma 2 and Qwen 3.5 (0.8 and 3.5pp remain) and survives in Gemma 3 27B and Qwen 2.5 (13.9 and 21.4pp), including under an instruction that names the user's belief (§5, Table 7).
  • In Gemma 3 27B and Qwen 3.5, the two models where the submission reported it, the instruction suppresses wishes more than beliefs (S = −16.5 and −20.1pp). This holds for every wrong option, on questions the model answers correctly on its own, under sampling, and under blind relabeling (Tables 6, 9, 14, pp. 23–27).

The larger panel adds three results.

  • Seven of ten models follow beliefs and three follow wishes. Four follow beliefs without following wishes; none does the reverse (§5, Fig. 3a).
  • Belief-following that survives the instruction is also present on questions the model answers correctly on its own (Table 6, p. 23).
  • The instruction lowers correct updating on reliable evidence in six of ten models (Tables 15–16, p. 28).

In one newly added model, Gemma 3 12B, the instruction removes most of both effects. We therefore report the wish-minus-belief contrast model by model and do not state one direction for all models (§5, Table 5).

An agreement score would count this instruction as a success. Measured separately, it removes one kind of sycophancy, leaves another in several models, and reduces legitimate updating in most.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.