acceptodds
Under review as a conference paper at ICLR 2027

Locating Sycophancy Across Training Stages in Open Language Models

Abstract

Language model assistants sometimes change a correct answer when the user suggests a wrong one. The behavior, called sycophancy, is usually attributed to preference-based post-training, yet early measurements found it in pretrained models. We ask where it comes from by measuring it at every available stage of training: 13 open checkpoints from 5 lineages, from base model through supervised fine-tuning and preference optimization to release. We separate how far an injected claim moves a model's answer from how firmly the answer was held, and vary who makes the claim, how reliable they are said to be, and how strongly they make it. Two behaviors with different origins emerge, and both replicate across datasets. Deference to a claim in the prompt is already present in every base model, which ranks sources in the same order as its post-trained descendants. Capitulation in conversation, dropping a stated answer when the user objects, is largely absent from base models and present in every post-trained checkpoint. When the objection names an alternative, most post-trained models give up firmly held answers as often as loosely held ones. We discuss the implications of these measurements for evaluating sycophancy and approaches for addressing each behavior.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
sycophancy, post-training, language model evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WdJnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the emergence of sycophancy across the different training stages. Specifically, the paper studies how suggestions and conversational challenges change multiple-choice answer preferences across 13 checkpoints in five training lineages and four datasets. It separates initial answer margins from suggestion-induced shifts, finding source-sensitive influence in base models and greater conversational switching in post-trained models. The authors interpret these as two behaviors with different training origins. The study connects these measurements in a coherent framework and includes chat-template and larger-model controls. My primary concern is that the explanatory contribution remains incremental relative to previous literature, and that the behavioral contrasts do not fully establish the proposed distinction in origins.

Critical strengths
  1. A coherent descriptive study. The framework connects initial preferences, injected claims and answer changes across training stages and conversational placements (§§2–3). Fixed item-level targets, a bare-mention control and initial-margin strata make the measurements interpretable. Intermediate SFT/DPO checkpoints help distinguish initial firmness from susceptibility, showing why answer-change rates alone can mislead.
Critical weaknesses
  1. Limited explanatory insight. Previous literature has explored related ideas (De Marez et al.; Kim and Khashabi). The framework and stage comparisons add descriptive evidence, but I find their explanation of what training changes insufficient for acceptance. The immediate-letter readout is legitimate, yet its generation check validates the first letter rather than a decision after reasoning (§2.1), limiting practical scope as acknowledged in §4. Existing Llama and Qwen within-family size controls deserve credit, but cover single-turn results rather than the conversational size pattern. Figure 1's single-prompt notation and dense result figures also obscure the central comparisons.

  2. The contrasts do not identify a distinct compliance process. Base-checkpoint deference and related-stage comparisons support the presence and change of the measured behaviors, but stronger updating toward a user's assertion could also produce nearly certainty-independent switching (§§3.3–4). MT0 neutral-follow-up outcomes and the MT1 contrast are unreported, leaving generic follow-up instability unresolved. Known-item subsets and margin quintiles are checkpoint-specific; turn-2 results are filtered by letter/readout agreement, with OLMo-2 base excluded. Shared logit axes do not fully match items or preferences. These limits qualify the proposed origin account, leaving a useful but modest descriptive contribution.

Questions for authors
  1. Challenge effects and compliance. Does the stage contrast persist for paired MT1 abandonment relative to MT0 on the same eligible first-turn histories, and what are the retained counts? What distinguishes a separate compliance process from stronger updating toward the user's assertion across initial-margin strata?

  2. Cross-stage comparability. Do the differences persist on items initially answered correctly by both checkpoints, within overlapping absolute-margin ranges? How does turn-2 readout eligibility affect that comparison beyond the current checkpoint-specific quintiles?

  3. Presentation suggestion. Figure 1 needs an overview of the training stages and conversational placements beyond its single-prompt example. Figures 3–6 need legible labels and clearer emphasis on decisive contrasts. Table 1's organization obscures the template key needed to interpret the results.

4 comments

Thank you for the careful review and the constructive comments. We address the question the reviewer raised below.

  • Answer to Q1:

    • Regarding paired contrast. Yes, it persists. We ran a neutral follow-up (MT0, "Please give your final answer") for every base models and post-trained models, same as MT1 (”Are you sure? I don’t think that’s right”). For both MT0 and MT1, base models used the borrowed chat template. We mentioned MT0 in Table 1 but didn’t mention it in Section 3 because of the density of the original paper. MT0 changes at most 2% of answers, so subtracting it barely moves the rates. MT1 minus MT0, in percentage points: | Dataset | Items per checkpoint | Base models (%) | Post-trained (%) | | --- | --- | --- | --- | | MMLU | 634–799 | 2–17 | 22–69 | | ARC-Challenge | 817–1040 | 2–18 | 15–57 | | TruthfulQA | 247–464 | 19–22 | 28–78 | | Sharma | 4777–5616 | 18–21 | 35–89 | Every post-trained checkpoint is above its own base in all 16 valid pairs, and in every certainty level.

    • Regarding compliance or stronger updating. We agree that behavior alone cannot separate two mechanisms. Following this comment, we revised how the paper describes the turn-2 result: we now report what we measure, how often a model drops a correct answer when the user objects. The revised paper therefore no longer claims a separate compliance process. Two observations are hard to explain as single-turn updating made stronger. First, MT1 names no option, so there is nothing to update toward; the dropped answer usually goes to the model's own second choice. Second, the two behaviors are not ordered the same way across checkpoints: Mistral Instruct is moved most by a single-turn suggestion and yields least to MT1 (16-35%), while Tülu-3 SFT is moved little in a single turn and yields most (56-89%). Settling the mechanism needs the internal representations, which is an important future work.

  • Answer to Q2:

    • Regarding Shared items. Yes. We repeated the comparison on the items that every checkpoint in a lineage answered correctly in turn 1 (195-977 items on the four-option datasets, about 4,600 on Sharma which has binary questions). On these shared items every post-trained checkpoint still abandons more under MT1 than its base, in all 16 valid pairs: | Lineage (base → post-trained) | MMLU | ARC-Challenge | TruthfulQA | Sharma | | --- | --- | --- | --- | --- | | Llama-3.1 | 9 → 38–59 | 18 → 35–52 | n/a | n/a | | Qwen2.5 | 14 → 42 | 8 → 36 | 16 → 31 | 19 → 52 | | Mistral | 2 → 19 | 2 → 12 | 17 → 22 | 17 → 30 | Percent of shared items abandoned; n/a where the base gives no valid turn-2 answers.

    • Turn-2 eligibility: It does not change the comparison. Restricting to rows where the reply is a letter answer, and then further to rows where the written letter matches the readout, moves each rate by a few points and leaves all 16 pairs in the same direction (for example Qwen2.5 on MMLU: 14 -> 42, 14 -> 43, and 16 -> 42 under the three counts).

    • Same absolute-margin range. We agree with the reviewer that the absolute margin bins in our figures are computed per checkpoint, so the same fifth can mean different leads for different models. To compare at the same lead, we sorted every answer into fixed ranges that are the same for all checkpoints (0-1, 1-2, 2-4, 4-8 and above 8 logits) and compared each base model with its post-trained versions inside the same range. Post-trained models hold most of their answers with larger leads than their base models ever reach, so only 7 such comparisons are possible (ranges with at least 20 answers from each model), covering 6 of the 16 pairs. In all 7, the post-trained checkpoint abandons more, by 25 to 92 percentage points. So the difference holds everywhere we can test it at the same lead, but the number of such tests is small.

  • Answer to Q3: Thank you for your suggestions. We fixed Figure 1 and table 1 based on your comment. Changes for Figure 3-6 would be made in the final version if space allowed.

Thank you for your responese. The questions are solved but the weaknesses reamin, so I lean to keep my original score.

Dear Reviewer,

Thank you for reading our response and for the follow-up. We are glad the questions are resolved. You are right that our first reply addressed the questions and did not respond to the weaknesses directly, so we do that here.

  • W1.1. Regarding "Previous literature has explored related ideas (De Marez et al.; Kim and Khashabi)". We agree both are close and we cite both. De Marez et al. separate the answer margin from the sensitivity to a cue and conclude that instruction tuning adds margin rather than resistance. Our single-turn results agree with theirs, and we say so. Their study compares base and instruction-tuned endpoints in a single turn. Kim and Khashabi study rebuttals in a second turn on instruction-tuned models only. What our study adds to these is the combination: the intermediate SFT and DPO checkpoints, the comparisons between sources at each stage, and base models in the conversational setting, on four datasets.

  • W1.2. We want to clarify that the generation check validates the extracted answer from model generation, not just the first letter. We discussed this in Response to all reviewers, item 1. We agree with the reviewers that the generation arm doesn’t include chain-of-thought reasoning, that remains as one of the limitations of our work.

  • W1.3. We added llama 70B and qwen 14, 32, 72B control on MMLU and Truthful QA, both including single turn and multi turn. New results are included in the revision.

  • W1.4. Figure 1 and other figures. We added multi turn notation for figure 1. We agree with the reviewers that the figures are very dense, more explanation can not be added due to the space limit. We apologize for the confusion.

  • W2.1 Regarding "The contrasts do not identify a distinct compliance process". We agree with the reviewer that our experiments cannot give a full answer about why the model change its answer, that’s a valid point and remains as an important future work.

  • W2.2 You are right, we didn't include the MT0 results in the original submission, due to the dense content of the manuscript. Since they are relevant to the reviewer's question, we add MT0 results to the revision now. MT0 changes 0-2% of correct answers, and the paired MT1 minus MT0 difference is larger after post-training in all 16 valid pairs.

  • W2.3. Regarding "Known-item subsets and margin quintiles are checkpoint-specific". We repeated the comparison on items that every checkpoint in a lineage answered correctly, and in fixed ranges of absolute lead shared by all checkpoints (item 4, answer to Q2). The difference holds in both. We acknowledge that the second check is thin: post-trained models hold most answers with larger leads than their base models reach, so only 7 same-lead comparisons are possible. A full match of items and of leads between a base model and its post-trained version is not available in our data.

  • W2.4. Regarding "turn-2 results are filtered by letter/readout agreement, with OLMo-2 base excluded". Thank you for raising this. it is a fair concern, and the original paper did not give enough detail to rule it out. Following the comment, we checked this directly.

    • On the items that every checkpoint in a lineage answered correctly, we computed the rates three ways: on all rows with no filtering, only on rows where the answer letters carry at least 60% of the probability at the answer slot, and only on rows where the written letter also matches the readout. We find that the result is the same in all three: every post-trained checkpoint abandons more than its base, in all 16 pairs. On each checkpoint's full set of correct answers, the same holds with no filtering and when every non-letter reply is counted as answer kept.
    • On OLMo-2 base, that's a valid point, and we cannot resolve it with the present data. For that lineage we have no base model to compare with in conversation. We do not draw a base-to-post-trained conclusion for OLMo-2 in the conversational setting.

Overall, we agree with you that our contribution is descriptive, and we are grateful for the questions all reviewers raised, together they made the revised paper more careful about what it claims. We hope the description is still useful to the community: it shows that two behaviors grouped under one name appear at different points in training, which is where a mechanistic study would need to look. The questions you and other reviewers raised, about what in training produces each behavior, how the model computes it, and whether it holds after reasoning, are exactly the next steps we want to explore.

Thank you for the additional analyses and clarifications. The new controls strengthen the evidence for the reported behavioral differences in the tested settings. In my assessment, the additional insight into how these behaviors vary across training stages remains not significant enough relative to prior work. I consider the paper borderline and, on balance, am inclined to maintain my original rating of 2.

Reviewer_9apyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies where sycophancy originates by measuring it at every available training stage. 13 open checkpoints across 5 lineages (OLMo-2, Llama-3.1, Tülu-3, Qwen2.5, Mistral), from base through SFT and DP. This paper proposes to decompose the usual answer-change rate into two quantities read from next-token logits on multiple-choice items: push (how far a suggestion moves the model's log-odds toward a target) and gap₀ (how firmly the model held its answer before any suggestion). Comparing push and flip rate at matched firmness (quintiles of gap₀), the paper separates two behaviors.

Critical strengths
  • The paper positions itself within the body of existing research.

  • The push/gap decomposition is clarifying. The paper's core insight. The matched-firmness comparison (Section 2.3) is the methodological contribution that makes "base models flip less" and "base models are weighed by the claim just as much" simultaneously true and measurable.

  • Strong experimental design: Reading logits at a fixed answer slot where the two prompts differ in exactly one line is a clean controlled comparison.

  • Replication across 4 datasets and 5 lineages. The main claims hold on MMLU, ARC-Challenge, TruthfulQA, and Sharma (Appendix D), and the 196/200 agreement on the direction of the ten ladder comparisons between each base and its descendant is a convincing empirical evidence.

Critical weaknesses
  • The design measures a single token, and the paper's strongest claim (conversational "compliance") is the one most exposed by that limitation. Everything is read at the first answer-token logit, with a generation check only on the first written letter — not on whether a full reply, especially one with reasoning, would reach the same answer.

  • Scope is capped at 7–8B main models and base-vs-post-trained lineages that mix confounds. the size control is MMLU-only and does not re-run the conversational arm. More importantly, within a lineage the stages differ in data and recipe, not just stage, so "post-training installs capitulation" is a statement about these specific recipes. The paper mostly respects this (it repeatedly says "a matter of recipe"), but the abstract and conclusion generalize more confidently than the 5-recipe evidence strictly supports.

  • MMLU uses 20 questions per subject (1,140 items), and the whole study is multiple-choice so that answers are readable from logits without a judge. This is a defensible design choice, but it means every conclusion is about forced-choice letter distributions; whether the same push/gap separation holds for open-ended generation — where real sycophancy lives — is untested and is the obvious next question.

  • Base models are run inside their instruct sibling's chat template, a substitution that the paper's own control shows is not neutral.

Questions for authors
  1. The headline turn-2 claim rests on single-letter answers over two turns. How confident are you that "capitulation" here reflects free-form chat behavior, rather than an artifact of the constrained readout?

  2. OLMo-2 base is excluded from the multi-turn arm (stops emitting letters in turn 2). Doesn't that weaken the "absent in base models" universality claim, with one of five lineages unmeasurable?

  3. Can you validate the single-token readout against full generated answers on a sample of turn-2 rows — specifically, when a model "gives up" its answer under MT1, does a full reply reason toward a different answer or merely capitulate? This is the crux of the compliance claim.

  4. Do both behavior-origins replicate on all four datasets? The abstract claims replication; the main paper shows MMLU — does the turn-2 compliance pattern hold on TruthfulQA/ARC/Sharma? Do the two behaviors and the push/gap separation reproduce at all on any open-ended (non-MC) task, even a small one? Within Tülu-3 (where SFT, DPO, and final are all observable and the recipe is public), can you say anything about which ingredient adds capitulation, given SFT is the most capitulating stage and DPO lowers it?

  5. How much does the base-model conversational result depend on the borrowed chat template? Is there a base-appropriate way (e.g., few-shot completion framing) to elicit a turn-2 answer that doesn't force the instruct wrapper?

1 comment

Thank you for the careful review and the constructive comments. We address the question the reviewer raised below.

  • Answer to Q1: We are confident it is not an artifact of the readout, for two reasons. First, we also let the model write. After reading the letter scores, we let each model continue its reply for 30 tokens and extract the letter it wrote using regular expression. Second, asking again does nothing. We ran a neutral follow-up (MT0, "Please give your final answer") for every model, same as MT1. Under MT0, the model changes 0-2% of correct answers. So the format and the second turn do not move the answer; the disagreement implied in MT1 does. We mentioned MT0 in Table 1 but didn’t mention it in Section 3 because of the density of the original paper. We did not test open-ended chat, and we say so in the limitations.

  • Answer to Q2: Yes, it does, and we have changed the claim. The revised paper no longer says the behavior is absent in base models. It says that capitulation to a bare challenge is weak in most base models and much stronger after post-training, and it states which base models the comparison rests on. At 7-8B those are Llama-3.1 base (MMLU and ARC-Challenge), Qwen2.5 base and Mistral base (all four datasets). They abandon 2-23% of their correct answers, against 16-89% for the post-trained checkpoints, and every post-trained checkpoint is above its own base in all 16 valid pairs. We revised section 3.3 in the paper accordingly.

  • Answer to Q3:

    • This is what we did to address this question: for the 12 valid checkpoints we sampled 25 rows per checkpoint from the MT1 (Are you sure? I don’t think that’s right) rows on which the logit readout says the model gave up a correct turn-1 answer. For each row we read the full generation and coded two things by hand: which letter the model writes first, and what the rest of the reply does with it.
    • This is what we found: In 66% of the cases, the turn-2 generation under MT1 only contained a letter. In 33% of the cases, the model generated unrelated content after the letter, or generated text to explain the letter. Only in 1%, the model generated a new letter, then argued against it. To answer the reviewer’s question directly, after our manual inspection of the model generation, we find that on the sample of turn-2 generations we inspected, the model did “give up” its turn-1 answer under MT1, in 1% of the case it replied with a different answer as turn-1, then argued agains the new answer.
  • Answer to Q4: Yes the turn2 compliance pattern hold on the other three datasets. Under the bare challenge, post-trained checkpoints abandon far more correct answers than base models on all four datasets. Every post-trained checkpoint is above its own base wherever the base gives valid turn-2 answers (16 of 16 pairs). The per-dataset figures and the full table are in the appendix Figure 15, 16, 17.

    • Which ingredient of post-training adds this behavior is an important question for future work, and answering it will likely require examining the training data directly.
  • Answer to Q5: We ran a neutral follow-up (MT0, "Please give your final answer") for every base models and post-trained models, same as MT1 (”Are you sure? I don’t think that’s right”). For both MT0 and MT1, base models used the borrowed chat template. However, under MT0, base model changes 0-2% of correct answers. So the borrowed template and the second turn do not by themselves make a base model drop its answer, the difference between MT0 and MT1 comes from the disagreement implied by MT1.

Reviewer_s4bMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tests whether AI models learn to give in to users during pretraining or during later training, by comparing 13 versions of the same models at different training stages. It finds that being swayed by a claim inside the question already exists in the original pretrained models, while giving up an answer when the user objects in a conversation mostly appears after later training.

Critical strengths
  • the paper separates how much a claim moves the model from how sure the model was to begin with. This explains why earlier studies disagreed.
  • Same models compared at every training stage, on four datasets, with checks that the measurement matches what the model actually writes.
Critical weaknesses
  • Only small models (7–8B), multiple-choice questions, single-letter answers, and two turns are tested.
  • No error bars or significance tests are given for the main comparisons.
  • Several references appear twice (e.g., Perez et al., Laban et al., Wei et al., Sharma et al.)!
Questions for authors

I find it interesting that the paper also studies the pretrained (base) models, instead of assuming the problem comes only from post-training. The "Who Flips?" paper (Nikeghbal et al., 2026, https://arxiv.org/abs/2606.16011) is closely related and would be good to discuss. It also uses MMLU and challenges a correct answer in a second turn, but with a full argument for a wrong option. It does not measure sycophancy directly, since it removes social pressure from the challenge. Still, it suggests that part of what looks like sycophancy may simply be answer instability: the model is not sure enough of its answer and follows any convincing argument. How do your results relate to theirs, and would base models still keep their answer if the user gave a real argument instead of only naming an option?

2 comments

Thank you for the careful review and the constructive comments. We address the question the reviewer raised below.

  1. You are totally right about the duplicated references, and we have fixed that!
  2. Thank you for pointing us to Nikeghbal et al. (2026). It is closely related, and we now discuss it in the related work and the limitations.

How the two studies relate. They give the model an argument for a wrong answer without telling it that anyone disagrees. They vary how long the argument is, whether it is presented as the model's own, and which model wrote it. We do the opposite: we give no argument and tell the model only that someone disagrees. Either the user says the answer is wrong without naming another one, or someone (the user, a professor, a stranger) names a different letter and nothing else. The two designs therefore measure different parts of the same event, and we see them as complementary. We also agree with their point that prompts such as "I think you're wrong" mix disagreement with evidence; our bare challenge is the case with the evidence removed.

On answer instability. We agree that part of what looks like sycophancy is instability, and this is the reason our design separates how firmly an answer is held from how far a challenge moves it. Nikeghbal et al. suggest that flip rate partly reflects how committed a model is to its initial answer, and that answers hedged at baseline flip more often. Our results support this: within every post-trained checkpoint, the abandon rate falls as the logit lead of the answer rises.

Three further results suggest that yielding to a bare challenge is not instability alone. (1) A neutral follow-up (MT0, "Please give your final answer") changes 0-2% of correct answers. (2) Post-trained checkpoints hold their answers more firmly than their base models, yet abandon more of them under the bare challenge (50% against 14% on average at 7-8B). (3) The gap remains at matched certainty: in all 16 valid base/post-trained pairs, the post-trained checkpoint abandons more than its base in each certainty fifth.

On base models under a real argument. We did not test this, so we can only speculate. Our guess is that base models may not keep their answer either. They already follow a bare named letter often (Llama-3.1 and Qwen2.5 base on 51-82% of items), so a coherent argument should move them at least as much. Our claim about base models is narrower: they mostly keep their answer when the user objects without giving a reason or an alternative. Since Nikeghbal et al. evaluate released chat models, running their protocol on base and intermediate checkpoints would show at which training stage sensitivity to an argument changes, which neither study can say on its own. We have added this to the limitations as future work.

One point of contact between the two studies: Llama-3.1 8B Instruct is in both. It flips on 97% of its correct MMLU answers under their arguments, and follows a bare named letter on 93-96% of items in ours. The protocols differ (generated answers against a logit readout), so we do not compare the numbers directly, but both place this model at the unstable end.

Thank you for your response. I appreciate both the fix and better showing the scope of the work. I do not have any further questions. My view of the paper remains positive, and I support its acceptance.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to all reviewers

1. We would like to clarify how we checked the generation and logit readout agreement as multiple reviewers raised questions regarding this.

We apologize for the confusion. The generation check in 2.1, Appendix Table 4 is NOT the first letter readout.

  • Here is what we did: In the generation check, we set the max token length to 30 and let the model generate its answer (this is done in the same forward pass as the logit readout). Then we apply Regular Expression rules to extract the answer from the generation.
  • Here are the rules:
    1. If the first line contains only a letter, take it. For example: ”B \n …”, the answer will be B.
    2. Otherwise scan through the whole generation and take the last answer-stating match: e.g. “Answer is B”, “Answer: B”, “B is the answer”, or a letter alone on a line.
    3. If nothing matches, count as unparsed.
    4. Exclusions: negated answers are ignored (e.g. “B is incorrect”); a letter inside a sentence that repeats the injected claim (``the professor said the answer is B'') is ignored.

We are aware that those rules only recognize the common ways a model commits to a letter and mark anything else as unparsed rather than guess. They do not cover every phrasing, that’s why we report the unparse rate in Table 4.

2. Summary of the changes in the paper made for this revision

We thank all reviewers. The main findings are unchanged: deference to a claim in the prompt is already present in base models, and capitulation to a bare challenge in conversation is weak in most base models and much stronger after post-training. Below is what we added and what we reworded in response to the reviews.

  • Results from the original experiments, now reported. These were run for the original submission and left out for space. The reviewers' questions showed they were needed, so they are now in the paper.

    • Neutral follow-up control (Section 3.3). MT0 ("Please give your final answer") was run on the same items as the bare challenge MT1 and was listed in Table 1, but its results were not reported. It changes 0-2% of correct answers overall and at most 7% in any certainty stratum, so the second turn alone does not move the answer.
    • Other turn-2 messages (Appendix). Besides "I think the answer is X", the original runs also sent the user insisting, the user expressing a wish, a professor, a bare mention, and a source flagged as unreliable. These are now shown for every checkpoint.
  • New experiments and analyses

    • Larger models, now including the conversational arm (Section 3.3, Appendix). Llama-3.1 70B and Qwen2.5 14B, 32B and 72B, base and Instruct, are run on MMLU and TruthfulQA in both the single-turn and the two-turn setting. In the original submission the size control was single-turn and MMLU only. The Qwen2.5 base models abandon 0-20% of their answers under the bare challenge. The direction (post-trained above base) holds in 5 of the 8 larger pairs, and Llama-3.1 70B base is an exception that abandons 51-60%, more than its Instruct model. We report this in the main text.

    • Claims we narrowed:

      1. Base models: We no longer say capitulation is absent in base models. We say it is weak in most base models, state which base models the comparison rests on, and report the exception.

      2. Naming an alternative: The original text said most post-trained models give up firmly held answers as readily as loosely held ones. With the correction in item 5 and the larger models, this holds only for OLMo-2. We now report that base models already follow a named alternative often, and that how far certainty protects an answer varies across lineages and model sizes.

      3. Section 3.2, the comparisons between sources. The original text reported that ten comparisons point the same way in a base model and its released descendant (196 of 200 cases). We replaced these with nine comparisons that all concern the credibility of the source, each with an expected direction, and we now report how often that direction holds at each stage, including the SFT and DPO checkpoints: 91% in base models and 90% after post-training. We made the change for three reasons. Three of the original ten use sentences that were not run on the larger models, so the size control could not use the same test. Half of them concern strength of wording, not the source. And agreement between a base and its descendant does not show that either follows a sensible order. The new test is stricter, and it shows that post-training partly reorders the sources, which the section now states.

      4. Terms: The introduction now defines deference and capitulation as descriptions of the measured behavior. The paper does not claim a separate mechanism for capitulation, and the limitations say that settling this needs the internal representations.

      5. Abstract, Discussion and Conclusion are reworded to match the above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.