- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Knowledge Answer Divergence (KAD), a protocol for measuring what happens after a model adopts a false answer under conversational pressure.
A model–question pair is eligible only if the model answers both the original question and an independent paraphrase correctly. Eligible pairs then receive four cumulative pressure turns: contradiction, confident assertion, repeated insistence, and authority. Collapse is the first turn at which the model explicitly adopts a designated false target. The dialogue at that point is saved. Retained-context, fresh-context, recognition, delayed, held-out-paraphrase, and context-surgery probes each branch independently from it.
The experiments cover:
- eight local checkpoints of 9B or smaller (Qwen 3.5 2B/4B/9B, Llama 3.2 1B/3B, DeepSeek-R1-distilled 1.5B/7B/8B; Q4 via Ollama) on a frozen 1,500-question benchmark;
- a rerun of all eight on an audited, outcome-blind 300-question panel;
- six hosted endpoints on a separate 300-question panel;
- MLX BF16 reruns of Qwen 4B and Llama 3B.
The main results:
(1) Across 126 context-surgery cases from five models, recovery rises from 54.0% with the full history to 90.5% after a fresh-context reset (GEE odds ratio 0.055). In the core study, Llama 3B recovers in 8.0% (24/300) of collapses with the history retained versus 89.0% (267/300) in a fresh context.
(2) Across 12 independently authored pressure templates, collapse is on average 13.2–29.0pp lower than under the canonical script for Qwen 4B, Llama 3B, and DeepSeek 8B.
(3) Changing cue placement (sequential vs. early-simultaneous) shifts first false-adoption rates by +9.5 to +52.5pp for the same three models.
Validation includes a two-author, outcome-blind audit of 690 stimuli and a two-author check of the automatic grader on 160 trajectories. A prespecified activation intervention restores 0/13 held-out answers.
Critical strengths
S1. Measurement is careful and transparent.
- The two-wording gate and branching from the exact saved history cleanly separate "did not know" from "knew but gave in."
- Two authors audited all 690 stimuli without seeing model outcomes. The audit is useful in itself: only 42.6% of PopQA items are core-ready.
- The grader is validated against adjudicated labels (F1 0.989 for eligibility, 0.992 for false-target adoption).
- Inference uses question-clustered intervals, Holm correction, and prespecified criteria.
- Null and negative results are reported plainly. The activation intervention restores 0/13 answers. The reset policy removes false persistence but also removes acceptance of genuine corrections (163/200 to 0/200, Table 12).
- Generated confidence is at least 0.8 for 123 of 124 pressured-false responses in the MLX panel, which is also a useful finding.
Why this matters: I trust the reported numbers as measurements. My concerns below are about what the main contrast identifies, not about measurement quality.
S2. The paper shows that sycophancy numbers depend strongly on how pressure is worded.
- The canonical script yields 13–29pp more collapse than the average template.
- For Llama 3B and DeepSeek 8B, every one of the 12 templates yields less collapse than the canonical script (Table 8).
- Under ordinary disagreement, Qwen 4B collapses on 3/100 cases versus 74/100 under the canonical script (Fig. 7).
Why this matters: the lesson that sycophancy should not be reported as a single number from a single script is, to me, the most broadly useful contribution of the paper.
Critical weaknesses
The design and validation are very careful. However, the headline result is close to guaranteed by design, the informative effects are small and not always tested, and most of the evidence is tied to Llama 3B and the canonical pressure script.
W1. The headline contrast is close to guaranteed by design. The informative post-adoption effects are smaller, partly untested, and concentrated in small models, especially Llama 3B.
Weights do not change during a conversation, so a fresh context removes any influence the adoption could have. Fresh "recovery" therefore cannot show a lasting effect of adoption. The authors themselves note that a fresh chat is not direct evidence that the fact stayed stored throughout the exchange (§2). If the fresh request is the same as the baseline request (see Q1), fresh recovery is essentially a test–retest of the eligibility gate.
The paper's data fit this reading:
- In the MLX white-box panel, fresh-context candidate margins nearly equal baseline margins (Table 13, expanded rows, baseline vs. fresh: Llama 3B 3.70 vs. 3.50; Qwen 4B 2.10 vs. 2.14).
- Re-issuing the baseline request without pressure gives 108/115 correct answers (App. A).
So the 54.0% to 90.5% shift and the OR of 0.055 mainly measure how much the retained history lowers accuracy, i.e., in-context persistence. §2 itself says this is not new.
The contrasts that do isolate parts of the post-adoption history are weaker evidence:
- Smaller and untested. The surgery conditions (full history 54.0%, neutral trigger replacement 61.9%, all pressure removed 75.4%, trigger removed 76.2%) move recovery by less than the headline does (7.9–22.2pp vs. 36.5pp). No test is reported for them: the GEE includes every variant, but the registered family contains only the full-vs-fresh coefficient.
- One OR averages very different effects. The GEE has main effects only, so a single OR averages per-model effects that range from null to very large.
- Concentrated in Llama 3B. Of the net 46 additional correct answers under fresh context in Table 11 (114 vs. 68), 35 come from Llama 3B (7/46 vs. 42/46). The gap also appears in the two smallest models, which have few cases (Qwen 2B 5/10 vs. 10/10; Llama 1B 8/16 vs. 12/16). It is nearly absent for Qwen 4B (27/31 vs. 29/31) and Qwen 9B (21/23 vs. 21/23).
- The tested factorial shows the same pattern. The history-semantics factorial (§4.3) is tested (+34.1pp, 95% CI 27.7–40.7, Holm p < 10⁻¹⁵), but the effect is 58.1pp for Llama 3B against 4.1pp for Qwen 4B (App. F).
- Wording and delay are mixed. In the 2×5 wording-by-delay control, a held-out-wording effect is established only for Llama. So the Qwen 4B example in §4.2 (93.0% retained vs. 64.8% held-out) mixes wording with the five neutral exchanges that come before the held-out probe.
The title and the "factual accessibility" framing also suggest claims about knowledge that the paper itself disclaims. Framing the work as in-context persistence would fit the evidence better.
Why this matters: the central claim rests on a contrast that is largely expected. The contrasts that would make KAD informative are smaller, mostly model-specific, and partly untested.
W2. The size and generality of the core effects are tied to the canonical script and to small local checkpoints, and the evidence for this sits only in the appendix.
- Ordinary disagreement shrinks the gap. Under ordinary disagreement, Llama 3B recovers in 32/47 cases with the history retained and 39/47 in a fresh context, a gap of about 15pp (Fig. 7B). Under the canonical script in the core study, the split is 8.0% vs. 89.0%. This bears directly on the headline, yet the main text reports no numbers for it; §4.4 mentions ordinary disagreement only as a boundary check.
- Collapse rates are likely upper estimates. The template results show that the canonical script is unusually strong, so the collapse rates in Table 1 are likely upper estimates.
- Only small local models. All reported retained/fresh and surgery results come from local checkpoints of 9B or smaller. Runtime/precision checks exist only for Qwen 4B and Llama 3B (Fig. 6 right, App. F).
- Hosted recovery is missing. The hosted models collapse on only 4.0–21.6% of eligible cases. Recovery was measured for 87 hosted collapses (Table 17), but the recovery rates are not reported.
- The cue-placement study has a confound. It compares sequential with early-simultaneous presentation, so spreading cues over turns is confounded with how recent the last cue is. In the recency-matched comparison in App. A (distributed vs. late-simultaneous), only Llama 3B shows an effect (+15.4pp), and that is on a post-adoption endpoint rather than first adoption.
Taken together, the paper shows that a particularly strong script produces history-dependent persistence in small checkpoints, especially Llama 3.2. That is narrower than the headline claim.
Why this matters: the headline numbers describe a favorable corner of the design space rather than typical behavior, and readers of the main text cannot see this.
Questions for authors
Clarification requests (answers could change my assessment)
Q1. Is the fresh-context request byte-identical to the baseline request?
- If so, please present fresh recovery as a test–retest reference and compare it with each model's no-pressure re-request rate. App. A reports only the pooled 108/115.
- If not, please state exactly what differs.
- DeepSeek 7B recovers less in a fresh context (54/62) than with the history retained (62/62), although baseline correctness at temperature 0 was the selection criterion. Please explain.
- A sham branch that simply reruns the retained probe would give a noise floor for small contrasts such as Qwen 4B's 27/31 vs. 29/31.
Q2. How were the 590 audited questions selected from the 1,500, and how was the hosted panel selected?
- As I read §3.1 and App. J, 100 audited questions come only from the hosted panel and are not in the 1,500-question benchmark. Is that correct, and how many of the 300 clean-panel questions come from them?
- If panel membership was based on model behavior (eligibility or collapse), the clean replication is outcome-blind only at the final sampling step.
- This matters because eligibility roughly doubles on the clean panel: pooled 10.8% to 22.5%, with seven of eight models rising 1.7–3.3×, while Llama 1B falls from 6.5% to 4.0%. Meanwhile, source shares barely change, and audit-pass items differ from audit-fail items by only +6.3pt in eligibility for the local models (Table 17).
- Please also explain why Llama 3B's retained recovery rises from 8.0% (24/300) to 28.6% (28/98).
Q3. What do non-recovered responses look like?
For the retained and surgery branches, please break them down into keeping the designated false target, abstaining or hedging, and giving another answer. The Sarajevo example (App. H) counts a hedge as non-recovery.
Q4. How were the 126 surgery cases chosen? The same five models produce 838 collapses in the core study.
- What remains in "remove all pressure"? Is the adoption turn kept?
- Please report all variant coefficients with CIs, a variant × model interaction test, and per-model contrasts.
- Please explain the Llama 3B gap between removing all pressure (23/46) and fresh context (42/46).
Q5. Please report recovery outside the Ollama runs. This means retained, fresh, and held-out recovery for the 87 hosted collapses (descriptive statistics are enough), and the numbers behind Fig. 6 (right).
Suggestions (not conditions for my assessment)
(a) Add a late-simultaneous arm to the first-adoption sequence study.
(b) Report post-adoption recovery for template-induced collapses. Move the ordinary-disagreement results (Fig. 7) into the main text, with held-out recovery added.
(c) Estimate template heterogeneity with crossed question × template random effects. With 23–37 eligible questions, question-level noise alone could give a template SD of up to about 8–10pp (8.2–10.4pp at a 50% collapse rate). That is the same order as the observed 8.1–13.0pp, so part of the reported heterogeneity, especially for Llama 3B, may be noise.
(d) Include the verbatim prompts in the appendix.
9 comments
Part 2 (Continued)
-
Q3: What non-recovered responses actually contain We agree that strict non-recovery should not automatically be equated with continued endorsement of the designated false answer. We have therefore expanded the behavioral interpretation in Section 4.2. Among 126 historical full-history surgery responses, 68 are correct, 20 explicitly endorse the designated false target, 36 abstain or hedge, and two provide another answer. This distinction is central to KAD: a failure to produce the accepted answer can represent persistent misinformation, uncertainty, or another error, and these outcomes have different reliability implications. Additionally, Appendix K.3 reports a separate two-author semantic-content and commitment audit of 138 responses sampled across the historical branches. We distinguish factual content from commitment, preserve ambiguous cases and report weighted descriptive estimates.
-
Q4: Context-surgery selection, interventions and statistical evidence The 126 surgery cases were obtained from a separate preselected 300-question experiment, not randomly sampled from the 838 core collapses across the five checkpoints. We now state this explicitly in Section 4.2. The five principal recovery conditions yield: Full collapse history: 68/126 (54.0%), Neutral-trigger replacement: 78/126 (61.9%), Remove all pressure: 95/126 (75.4%), Remove collapse trigger: 96/126 (76.2%) and Fresh context: 114/126 (90.5%). Complete model-specific counts appear in Appendix F, Tables 9–10. We have also clarified exactly what the interventions retain. Removing all pressure preserves the original question and correct baseline exchange, whereas fresh context removes that exchange as well. The collapse-trigger removal deletes its final user–assistant pair, while neutral-trigger replacement retains the assistant's adopted false answer. This distinction is relevant to the Llama 3B difference between removing all pressure (23/46) and fresh context (42/46). The interventions remove different conversational components, so their difference cannot be attributed to a single latent mechanism.
Appendix K.2, Table 20 now reports the retrospective main-effects GEE coefficients with confidence intervals. Relative to fresh context, the odds ratios are 0.055 for full history, 0.097 for neutral-trigger replacement, 0.251 for removing all pressure, and 0.266 for removing the trigger. We also report a variant-by-model interaction score test (chi-squared = 22.073, 16 degrees of freedom, p = 0.141). The saturated interaction fit is unstable under separation. We therefore do not interpret the pooled coefficient as a uniform effect across checkpoints.
- Q5 and W2: Generality beyond the canonical script and local checkpoints We agree that the canonical pressure script produces particularly strong effects and that readers should see the corresponding boundary conditions. We have moved the ordinary-disagreement comparison into the main results (Section 4.2; Figure 4). Under ordinary disagreement, Llama 3B achieves 32/47 retained-context and 39/47 fresh-context correct responses, a substantially smaller difference than the canonical 24/300 versus 267/300. Likewise, Qwen 4B adopts the false target in only 3/100 ordinary-disagreement cases compared with 74/100 under the canonical script (Appendix F, Figure 10). We have also expanded the hosted-model reporting. Across 87 saved first-adoption events from six hosted endpoints, retained, fresh and held-out correctness totals are 79/87, 75/87 and 57/87, respectively. Appendix K, Table 17 provides the individual counts for all six endpoints. These findings do not support a general positive reset effect. So, we avoid claiming those.
For runtime and precision, the revised Figure 9 includes exact question-matched MLX 4-bit versus BF16 comparisons. Appendix F additionally reports the history-semantics contrast under Q4 and BF16: 58.1 versus 59.1 percentage points for Llama 3B, and 4.1 versus 8.1 points for Qwen 4B.
Your additional suggestions have been addressed in Part 3.
Part 3 (Continued)
- Additional suggestions incorporated into the revision We also appreciate the reviewer's specific suggestions for strengthening the experiments. (a) Late-simultaneous control: We added a matched late-simultaneous arm to the first-adoption study (Section 4.3; Figure 5). Sequential-minus-late false adoption is +27.3 percentage points for Qwen 4B (Holm-adjusted p = 4.61e-8), +9.5 points for Llama 3B (p = 0.083), and -1.2 points for DeepSeek 8B (p = 1). Only Qwen survives the primary correction. (b) Recovery under different pressure conditions: We added template-induced recovery follow-ups and matched wording-by-delay controls, reported in Appendix K.1, Tables 18–19. Ordinary-disagreement recovery and the reduced retained/fresh difference are now explicitly discussed in the main paper. (c) Template heterogeneity: We added a crossed question-and-template random-intercept sensitivity analysis (Appendix K.2). These estimates are approximate and prior-sensitive. (d) Verbatim prompts: We added the saved pressure registry, twelve independently authored templates, output schema and protocol strings in Appendix K.3, with serialized request examples and hashes documented in the accompanying reproducibility materials.
We sincerely appreciate the reviewer's careful distinction between measurement quality and interpretation. Your comments helped us substantially improve how the manuscript communicates its findings and limitations. Our intended contribution is a reproducible protocol for examining what happens after false-answer adoption, distinguishing different failure modes and assessing how contextual interventions affect them. We hope the revised manuscript and these clarifications address your concerns and we would be grateful if you could reconsider your evaluation taking these into account. Thanks again for your valuable time and efforts in helping us improve our research.
Rebuttal Revision addressing concerns and questions
We sincerely thank you for the detailed and thoughtful evaluation of our work and recognizing the care taken in our experimental design and reporting. We appreciate your constructive suggestions which has helped us a lot. We have substantially revised the manuscript during the rebuttal period, including additional matched controls, a corrected local follow-up, clearer statistical reporting and explanations. We request you to reconsider your evaluation taking into account the revised paper and our clarifications below:
Interpretation of the main contrast and concentration of effects (W1) We agree that fresh-context recovery should not be interpreted as evidence that model weights changed or that factual knowledge was internally lost. Our intended contribution is behavioral: KAD evaluates how a model's factual responses change under specified conversational conditions following an observed false-answer adoption. We have clarified this distinction in Sections 2, 3.2 and 5. In particular, factual accessibility is operationally defined as producing an accepted factual answer under a specified question, history, serving condition and grading rule. We make no claim about the underlying storage of factual knowledge. We also acknowledge that the large retained/fresh contrast is not universal. In the 126-event historical surgery panel, Llama 3B contributes 35 of the 46 net additional correct responses under fresh context. We now explicitly report this concentration in Section 4.2. However, this heterogeneity also illustrates the motivation for KAD. Models with similar initial susceptibility can exhibit markedly different post-adoption behavior. KAD is intended as an evaluation protocol for such model-specific differences. It does not claim that every checkpoint exhibits the same contextual effect.
Q1: Fresh-context requests, repeatability and DeepSeek 7B The fresh-context request is not byte-identical to the baseline request. The original baseline user message contains the factual question alone, whereas the core fresh-context probe adds "Answer independently from your own knowledge." Retained-context probes also use different wording. We have clarified these differences explicitly in Section 4.2. We agree that this makes fresh context a composite reset rather than an isolated causal intervention. Accordingly, we distinguish the core comparison from context-surgery conditions which hold the final query fixed. To address repeatability, Appendix A reports a repeated-request control on 115 selected events. Reissuing the original request produces 108/115 correct answers and 107/115 normalized-answer matches to the stored baseline. Our corrected follow-up also includes 276 repeatability triplets (828 responses), which produce identical raw outputs within the corrected execution period. However, we do not assume byte-exact equivalence with historical requests because the original schema serialization was not fully preserved (Section 4.5; Appendix K.1). Regarding R1-Distill-Qwen 7B, we examined the eight historical fresh-context failures. Three explicitly contain the accepted factual answer in a sentence but fail the strict short-answer grader. Another gives an answer at a different level of granularity, while four provide different answer values. These findings are reported in Section 4.2. Therefore, the 54/62 fresh-context correctness result should not automatically be interpreted as eight cases of lost factual knowledge.
Q2: Audited-panel selection and the change in Llama recovery We appreciate your question as we could revise for clearer documentation. The outcome-blind human audit covers 690 unique stimuli: 590 originating from the original benchmark and 300 from the separately frozen hosted panel, with 200 shared items. Thus, 100 audited items are hosted-only. After adjudication and post-audit quality checks, 457 items remain core-ready. The 300-question replication panel is sampled deterministically within sources using fixed quotas (95 PopQA, 82 EntityQuestions, 72 TriviaQA and 51 NQ-Open). Of these 300 questions, 238 overlap with the original benchmark and 62 are hosted-only. The 62 hosted-only items inherit prior joint baseline-correctness screening for Qwen 4B and Llama 3B. We now explicitly acknowledge that the final panel is not fully independent of earlier model-behavior information. This also helps explain the change in Llama 3B's retained recovery. In the audited rerun, original-overlap items yield 7/41 correct responses after adoption, while hosted-only items yield 21/57. Together these produce the reported 28/98, which should not be directly compared with 24/300 from the original benchmark. We have added these qualifications to Sections 3.1 and 4.6, with complete selection details in Appendix C.
Due to the long response and character limit, we have continued our response in the next comment. Please check that too for the remaining clarifications.