acceptodds
Under review as a conference paper at ICLR 2027

The Anatomy of Conversational Consistency: Linear Readouts and Control Under Pressure

Abstract

Large language models often abandon correct answers when users push back over several turns. We dissect this capitulation in five open-weight models at three levels: what the models do, what their activations encode, and what intervening on those activations changes. Measurement comes first. When models grade their own neutral follow-ups, they report flips in up to 98.4% of conversations; gold-blind extraction of the final answer finds almost none, and genuine flips concentrate under challenge. With faithful labels, departure from an initially correct answer is linearly readable from end-of-response activations in every model (question-held-out AUROC 0.96–0.99), including a mixture-of-experts model that self-judged labels had made look unreadable. The readout is also a lever. On held-out questions, with initial responses shared across conditions, subtracting its mean-difference direction during generation reduces flips dose-dependently: from 9.6% to 1.9% in Qwen3-8B, more than a prompt defense or norm-matched random directions, at a 2.5-point cost in single-turn accuracy; and from 24.4% to 9.5% in Qwen3-32B, where adding the direction raises flips to 43.4%. The effect is confined to challenge, where suppression anchors the model’s initial answer. Reading and controlling nonetheless come apart: the direction is inert at the layer where it is most readable, and in 8B a random direction also induces flips.

Primary area:
generative models
Keywords:
large language models, conversational consistency, sycophancy, activation steering, mechanistic interpretability, linear probing, causal interventions

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Ww5ROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies conversational consistency in LLMs under user pressure on multiple choice questions, focusing on behavioral measurements, linear representations of answer changes, and activation steering. The paper identifies substantial errors in self-judged consistency evaluations and demonstrate that answer changes can be detected using linear probes. They further show that steering along a learned activation direction can significantly reduce or induce answer changes under conversational pressure.

Critical strengths
  1. Interesting and meaningful research problem. The paper studies conversational consistency from both behavioral and representation-level perspectives. Understanding why models abandon correct answers under user pressure is an important problem.
  2. Good experimental controls. I appreciate the inclusion of neutral, supportive, and corrective conditions, random-direction baselines, and comparisons with prompting-based interventions. The authors also investigate potential side effects rather than focusing exclusively on reducing capitulation.
  3. Interesting findings beyond the intervention itself. The identification of substantial errors in self-judged consistency measurements is valuable. The observation that linear readability does not necessarily imply controllability is also interesting and provides useful insights for future work on activation steering.
Critical weaknesses
  1. Poor presentation in the methodology section. I found the methodology unnecessarily difficult to follow. For example, Section 3.1 introduces several experimental protocols but directs readers to Appendix I for the actual prompts, without providing concrete examples in the main text.

    Note that the ICLR 2027 Author Guidelines state: Authors may use as many pages of appendices (after the bibliography) as they wish, but reviewers are not required to read the appendix.

    Given how central these protocols are, I think at least a few examples or a pipeline diagram should be included.

  2. Limited applicability to reasoning models and insufficient evaluation of output quality. The experiments mainly rely on multiple-choice questions with thinking disabled, where models directly output their answers. However, most modern reasoning models use extended CoT, and it is unclear whether the proposed activation direction would remain meaningful or effective in such settings. In particular, applying the same steering direction throughout CoT generation could interfere with the reasoning process itself, rather than simply preventing unjustified answer changes. Additionally, although the authors measure the impact on answer accuracy, they do not adequately evaluate general output quality, such as fluency, coherence, or reasoning quality. I think these limitations significantly restrict the demonstrated practical applicability of the method.

  3. It is unclear whether the intervention improves robustness or simply makes the model more stubborn. The identified direction appears to capture departures from the initial answer rather than sycophancy specifically. Although the authors include corrective experiments, these seem relatively easy, as the correct answer is explicitly provided to the model, with a baseline acceptance rate of 98.33%. I would be more convinced if the authors evaluated whether the model can still revise incorrect answers when presented with legitimate counterarguments or new evidence without being directly told the correct answer, like in the case some CoT chain.

Questions for authors
  1. Can you show if your results still hold when CoT is on with a limited experiment?
  2. In Figure. 1, you mentioned the Qwen3 32B result is a new cohort. It is important to make your results consistent, especially given the Qwen3 32B Flip Rate is substaintially higher. I suggest you re-run the experiments within the same pool in your revision.
6 comments

Thank you for the specific suggestions. We ran each experiment you asked for.

W1. Methodology section. §3 now opens with a pipeline figure (Fig. 1, p. 3) built from a real evaluation question. It shows the question and first answer, one example follow-up per protocol, the answer-extraction step, and where the direction is fitted and injected.

W2a / Q1. Do the results hold with CoT on? We ran Qwen3-8B with thinking on, using the direction from the no-thinking setting without refitting (App. N.5, p. 46; three seeds, 320 conversations). Under misleading arguments, 11.9% of correct answers flip at baseline. Steering the whole response brings this to 5.3% (a reduction of 6.6 points [−11.1, −2.5]). Steering only the final answer, with the reasoning generated unsteered, brings it to 6.6% (5.3 points [−9.0, −2.1]). So the effect does not require intervening on the reasoning. No response ran out of token budget, and the answer contradicts its own reasoning in 1.4–2.8% of responses. This is one model, and we describe the evidence as suggestive.

W2b. Output quality. On 100 AlpacaEval instructions, a blinded pairwise judge prefers steered and unsteered outputs equally often at the default dose (win rate 0.51 [0.42, 0.60]). Fluency, coherence and helpfulness scores differ by at most 0.04 points (App. N.4, p. 45, Table 40). Steered replies are shorter, with no increase in repetition.

W3. Robustness or stubbornness? We ran the test you describe: the model starts from a wrong answer and receives legitimate evidence that never names the correct option (App. N.1, p. 42, Table 33). In Qwen3-8B at the default dose:

  • valid evidence is accepted 74.9% of the time, against 82.1% without steering;
  • misleading arguments flip 20.3% of correct answers, against 37.6%;
  • at half dose, the drop in evidence acceptance is not significant (2.1 points), and more than half of the protection remains;
  • a random direction of equal norm changes neither rate.

The model still accepts most valid evidence, and it accepts a supplied answer key as often as before (98.33% and 97.92%; §6.1, pp. 8–9). It is therefore not made stubborn across the board, but warranted updates do become less frequent, and the abstract (p. 1) now says so.

Q2. The 32B cohort. We re-ran the comparison inside one pool (App. N.8, p. 48). Starting from the same first answers, Qwen3-32B flips on 98 of 335 conversations under challenge (29.25%), on 1 of 335 under neutral follow-ups, and on none under supportive ones. Fig. 2a (p. 6) now shows these three bars. The high 32B rate comes from challenge, not from the cohort.

Thank you again; these experiments made the paper more complete.

AuthorsOpenReview ↗

Full Results Tables for Reference

Each of your suggestions became an experiment, so we thought it would help to see the results side by side. The tables below give the full numbers for W2a, W2b, W3 and Q2, with pointers to the paper.

W2a. Thinking on (Qwen3-8B, misleading arguments, seeds 42–44, 320 pairs; App. N.5, pp. 46–47):

ArmFlipsChange, points [95% CI]Answer and reasoning disagree
Baseline11.9%2.8%
Steering the whole response5.3%−6.6 [−11.1, −2.5]2.2%
Steering only the final answer6.6%−5.3 [−9.0, −2.1]1.4%

Answer-only minus whole-response: +1.3 [−1.9, +4.1]. No response reached the token budget.

W2b. Output quality (Qwen3-8B, 100 AlpacaEval instructions, blinded pairwise judgments; Table 40, p. 46). Score differences are on the judge's 1–10 scales:

ArmWin rate vs. unsteered [95% CI]Δ fluencyΔ coherenceΔ helpfulnessTokens
Ours, β = −0.50.49 [0.40, 0.58]+0.00+0.03−0.05572
Ours, β = −10.51 [0.42, 0.60]−0.02+0.03+0.04553

Unsteered responses average 577 tokens.

W3. Robustness or stubbornness (Qwen3-8B; Table 33, p. 42). Rates, with change from baseline in points [95% CI]:

ArmEvidence accepted ↑Misleading flips ↓
Baseline82.1%37.6%
Ours, β = −0.580.0% (−2.1 [−5.9, +1.2])27.8% (−9.8 [−12.8, −7.0])
Ours, β = −174.9% (−7.5 [−12.1, −3.5])20.3% (−17.3 [−21.2, −13.5])
Random direction, β = −183.3% (+1.2 [−0.8, +3.4])33.1% (−4.6 [−7.3, −2.2])

Q2. The 32B cohort (same first answers, 335 complete conversations per arm; App. N.8, p. 48; Fig. 2a, p. 6):

Follow-upFlips
Challenge98 of 335 (29.25%)
Neutral1 of 335 (0.30%)
Supportive0 of 335

Dear Reviewers,

The authors have addressed your comments in their rebuttal. Please review their responses and provide any further feedback or follow-up questions, if any. Thank you!

Best, AC

Reviewer_qjriOpenReview ↗
Rating
2
Confidence
2
Informativeness
3(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper studies LLMs under user pressure in conversation. It first shows that self-judging can greatly overstate answer changes. With revised labels, linear probes can read answer states from hidden activations. Steering reduces answer flips in two Qwen models, but can also reduce accuracy and preserve wrong initial answers. The study separates what probes can predict from what steering can control.

Critical strengths
  1. Better measurement audit. On the same neutral conversations, answer extraction removes most apparent flips reported by self-judging (Figure 1). This builds the discussion of conversation consistency on a solid foundation.
  2. The controlled intervention study is not mixed with probe accuracy. Shared initial answers, dose tests, random directions, and capability checks show both an effect and its costs. The failure at a highly readable layer is also informative (Figures 5–6).
Critical weaknesses
  1. Limited Contribution. As an ICLR reader, I want stronger technical solutions to the proposed research questions, not limited empirical findings on a not-first-noted phenomenon. Although the research question is important for the community, this paper only gives trivial findings without a novel method to decrease the flip rate while remaining accurate to the question. The contribution of better measurement audit in reassessing the evaluation of conservation consistency and findings on intervention is helpful, but limited compared to the ICLR conference standard, considering conservation consistency issues are a typical topic.

  2. Layer selection is not fully independent of evaluation. The 8B layer choice used the full question pool, including later test questions. Holding out questions when fitting the direction does not remove this selection bias. The reported effect may be optimistic.

  3. Data Limitation. For the dataset used, all queries are simple general knowledge QA. Since the paper researches the question of "conversation consistency", harder scanrios like personal agentic assistant where tool calling or multimodal inputs appears in conversation is more common in real life, therefore, limited data used for experiment undermine the generlizability of some findings.

Questions for authors
  1. Have you experimented with different scenarios? For example, to copy an agentic task with action and tool calling in context, or with multimodal input?
5 comments

Thank you for the review. We take your first weakness to contain two questions: what is new here, and whether flips can be reduced without losing accuracy.

W1. What is new.

  • Self-judging, a common way to score multi-turn consistency, is unreliable. Models grading their own neutral follow-ups report flips in 70.6–98.4% of conversations. Extracting the final answer without the key finds at most 0.53% on the same conversations. The same labeling error makes one model's activations look unreadable (GPT-OSS-20B, AUROC 0.680 with self-judged labels and 0.958 with extracted ones). This affects any benchmark that lets a model assess its own replies (§4, pp. 5–6, Fig. 2b; §5, pp. 6–7, Table 2).
  • Capitulation can be lowered and raised by a single direction. On held-out questions with shared initial responses, the effect grows with dose, reverses with sign, exceeds a prompt defense, and is not produced by random directions of equal norm (§6.1, pp. 8–9, Table 3, Fig. 5).
  • The direction has no effect at the layer where it is most readable, in both models. Probe accuracy is often used to choose where to intervene, and this result shows it should not be (§6.2, p. 10, Fig. 6).

W1. Reducing flips while staying accurate. The paper reports this (§6.1; Table 22, p. 27). In Qwen3-32B, suppression takes flips from 29.42% to 15.61% while single-turn accuracy changes by −0.64 points [−1.91, +0.70]. In Qwen3-8B, half the dose removes 5.2 of 9.6 points of flips while accuracy changes by −0.84 points [−1.74, +0.07].

We stop short of calling this a defense because a lower flip rate can also come from a model that stops accepting valid corrections. We measured that (App. N.1, p. 42). At the default dose, Qwen3-8B accepts valid evidence 74.9% of the time against 82.1% without steering; at half dose the difference is not significant (Table 33). Reporting the handle together with this cost seemed more useful to us than a method evaluated on flip rate alone.

W2. Layer selection. We checked this. The 8B layer is chosen by a probe criterion that involves no steering, and computing it on the fitting questions alone gives the same layer, L19, under either label source (App. N.3, p. 44, Table 36). The 8B effect is therefore not inflated by the selection. For 32B, §3.4 (p. 5) states that the layer was chosen post hoc; the layer that the fitting questions select also reduces flips significantly (6.01 points [−10.21, −1.80]; Table 37, p. 45).

W3 / Q. Tool calling and multimodal input. We ran a tool-calling version (App. N.6, p. 47). Qwen3-8B uses its native tool template: a tool call returns notes before the first answer, and the eight challenges follow. The direction, taken as is from the standard setting, lowers flips to 1.17% and 1.31% in the two tool contexts (reductions of 1.17 [−2.35, −0.29] and 2.94 [−4.90, −1.31] points). With irrelevant notes it also does better than a random direction.

The experiment produced a second observation. Declaring a tool, without calling it, already lowers baseline flips from 9.09% to 2.27%. Studies of capitulation in agentic formats need to control for this.

We also tested a reasoning setting (thinking on, App. N.5, p. 46) and free-form arithmetic (GSM8K, §6.2); the direction reduces flips in both. Both steered models are text-only, so multimodal input is outside this study. We use questions with an answer key because the audit in our first finding needs a final answer that code can score (§3.2, p. 4).

If these points address your concerns about contribution and scope, we hope you will consider revising your score.

AuthorsOpenReview ↗

Full Results Tables for Reference

Thank you again for pressing on contribution and scope. So that you can check the evidence without searching the appendix, here are the complete results our reply draws on, each with its location in the revised PDF.

W1. Self-judged versus extracted labels on identical neutral conversations (§4, p. 6; Fig. 2b, p. 6):

ModelFlips, self-judgedFlips, gold-blind extraction
Qwen3-8B70.62%0.00%
Qwen3.5-27B88.78%0.33%
Gemma4-31B98.40%0.16%

The same labeling error affects readouts: on identical rows and folds, GPT-OSS-20B reaches AUROC 0.680 with self-judged labels and 0.958 with extracted ones (§5, p. 7).

W1. Flips and single-turn accuracy in the controlled studies (Table 3, p. 8; accuracy: §6.1, p. 9, and Tables 21–22, p. 27). Changes from baseline in points [95% CI]:

Arm8B flips8B accuracy32B flips32B accuracy
Baseline flip rate9.58%29.42%
Ours, β = −0.5−5.21 [−7.47, −3.16]−0.84 [−1.74, +0.07]−7.88 [−10.27, −5.56]−0.33 [−1.27, +0.64]
Ours, β = −1−7.71 [−10.52, −5.11]−2.48 [−3.79, −1.17]−13.82 [−16.92, −10.90]−0.64 [−1.91, +0.70]
Ours, β = +1 (induce)+11.47 [+8.60, +14.54]+20.76 [+17.52, +24.14]
Prompt defense−4.06 [−6.27, −2.00]−3.98 [−6.53, −1.60]

The same table has three random directions of equal norm at β = −1: they change flips by −2.71 to +9.38 points in 8B and by −0.80 to +2.59 in 32B.

W3. Settings beyond the standard one (all Qwen3-8B except GSM8K, where the result is for Qwen3-32B):

SettingBaseline flipsWith the directionChange, points [95% CI]
Thinking on, misleading arguments, three seeds (App. N.5, p. 47)11.9%5.3%−6.6 [−11.1, −2.5]
Tool call, irrelevant notes (App. N.6, p. 47)4.25%1.31%−2.94 [−4.90, −1.31]
Tool call, evidence notes (App. N.6, p. 47)2.35%1.17%−1.17 [−2.35, −0.29]
Free-form GSM8K, 32B (§6.2, p. 10; Table 16, p. 24)6.52%3.09%−3.43 [−4.85, −2.01]

The direction is applied as fitted in the standard setting, without refitting.

Dear Reviewers,

The authors have addressed your comments in their rebuttal. Please review their responses and provide any further feedback or follow-up questions, if any. Thank you!

Best, AC

Reviewer_JqqZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work investigates why large language models (LLMs) abandon initially correct answers when challenged during multi-turn conversations and whether this behavior can be detected and controlled through internal representations. Evaluating five open-weight LLMs across multiple conversational protocols, the authors first identify a substantial evaluation artifact, termed the drift illusion, in which model-based self-judgments significantly overestimate answer changes during neutral follow-ups. Using gold-blind answer extraction, they show that genuine answer flips primarily occur under adversarial challenges and that these changes can be identified through linear probes of hidden activations, achieving AUROC scores of 0.96–0.99 across models. The authors further construct an activation-steering direction from the difference between incorrect and correct follow-up representations and demonstrate that suppressing this direction reduces answer-flip rates from 9.58% to 1.88% in Qwen3-8B and from 24.36% to 9.46% in Qwen3-32B, while amplifying the direction increases susceptibility to challenges. However, the intervention can reduce general task accuracy and reinforce initially incorrect answers, revealing a trade-off between conversational consistency and adaptability. Finally, the study finds that layers exhibiting the strongest linear readability do not necessarily support effective behavioral intervention, highlighting a distinction between detecting internal answer states and causally controlling model behavior.

Critical strengths
  • Rigorous evaluation reveals substantial measurement artifacts in conversational consistency. The paper identifies a critical limitation of self-judged evaluations, showing that apparent answer-flip rates of up to 98.4% under neutral follow-ups largely disappear when using gold-blind answer extraction. This finding is consequential because it challenges the reliability of existing evaluation practices and demonstrates how labeling errors can distort both behavioral conclusions and representation-level analyses.

  • Compelling empirical evidence connects internal representations to causal control of conversational behavior. The paper demonstrates that answer-state changes are linearly decodable across five LLMs (AUROC 0.96–0.99) and that activation steering can substantially reduce capitulation under adversarial challenges. In particular, suppression reduces answer-flip rates from 9.58% to 1.88% in Qwen3-8B and from 24.36% to 9.46% in Qwen3-32B. The controlled experiments, including dose-response analyses and random-direction baselines, strengthen the empirical findings. Moreover, the observation that high linear readability does not necessarily translate into effective intervention provides an important insight into the distinction between representation probing and causal control.

Critical weaknesses
  • Rigorous evaluation reveals substantial measurement artifacts in conversational consistency. The paper identifies a critical limitation of self-judged evaluations, showing that apparent answer-flip rates of up to 98.4% under neutral follow-ups largely disappear when using gold-blind answer extraction. This finding is consequential because it challenges the reliability of existing evaluation practices and demonstrates how labeling errors can distort both behavioral conclusions and representation-level analyses.

  • Compelling empirical evidence connects internal representations to causal control of conversational behavior. The paper demonstrates that answer-state changes are linearly decodable across five LLMs (AUROC 0.96–0.99) and that activation steering can substantially reduce capitulation under adversarial challenges. In particular, suppression reduces answer-flip rates from 9.58% to 1.88% in Qwen3-8B and from 24.36% to 9.46% in Qwen3-32B. The controlled experiments, including dose-response analyses and random-direction baselines, strengthen the empirical findings. Moreover, the observation that high linear readability does not necessarily translate into effective intervention provides an important insight into the distinction between representation probing and causal control.

  • The proposed steering direction conflates answer correctness with deviation from the initial response. The method constructs an activation-steering direction as the difference between the mean hidden representations of incorrect and correct follow-up responses: $ v^{(\ell)}=\mathbb{E}[h^{(\ell)}\mid \text{incorrect}]-\mathbb{E}[h^{(\ell)}\mid \text{correct}], $ and modifies hidden activations through h~t(ℓ)=ht(ℓ)+βv(ℓ)\tilde{h}^{(\ell)}_t=h^{(\ell)}_t+\beta v^{(\ell)}. However, because the direction is estimated exclusively from conversations with initially correct answers, incorrectness is equivalent to disagreement with the initial response. Consequently, the learned direction does not distinguish unjustified capitulation from legitimate belief revision. The paper's corrective experiments support this concern: the direction's projection increases even when the model successfully corrects an initially incorrect answer. Moreover, suppression increases the probability of initially incorrect answers under challenge. This represents a fundamental limitation of the formulation, as the intervention primarily encourages answer preservation rather than evidence-sensitive consistency. Without explicitly modeling the validity of new evidence or the correctness of revisions, the method cannot reliably determine when an answer should be maintained versus updated.

Questions for authors
  • Clarification: Distinguishing capitulation from legitimate answer revision. The steering direction is learned from incorrect versus correct follow-up responses conditioned on an initially correct answer, making incorrectness equivalent to departure from the initial response. Given that the corrective experiments indicate that this direction also responds to legitimate corrections, can the authors clarify how the method distinguishes unsupported capitulation from evidence-based belief revision? Would the intervention remain effective when user challenges contain partially correct or conflicting evidence?

  • Clarification: Generalizability and principled layer selection. The results demonstrate that linear readability does not necessarily predict intervention effectiveness, as illustrated by layers 47 and 55 in Qwen3-32B. Moreover, the effective intervention layers were partly selected through exploratory experiments. Can the authors provide a principled criterion for identifying effective steering layers without relying on intervention-based searches over the evaluation distribution? How sensitive are the reported improvements to this selection procedure?

  • Clarification: Methodological novelty and comparison with existing steering methods. The proposed intervention uses a mean-difference activation direction and additive steering, closely resembling established approaches such as Contrastive Activation Addition (CAA). What distinguishes the proposed formulation from existing activation-steering methods beyond its application to multi-turn conversational consistency? Can the authors provide a controlled comparison with a standard CAA baseline using matched data, layers, and intervention strengths to substantiate the methodological contribution?

5 comments

Thank you for the detailed review. Your main question is what the direction encodes, which is also the question §5 (pp. 6–7) was written to answer.

W / Q1. Capitulation versus legitimate revision; partially correct or conflicting evidence. Your reading matches ours. §5 shows that the projection rises when the model corrects a wrong answer, so the direction tracks leaving the initial answer, not being wrong. §6.1 (pp. 8–9) shows that suppressing it favors the initial answer even when that answer is wrong. We do not claim that the direction can tell a warranted revision from an unwarranted one.

How much this matters in practice is an empirical question, and three new arms measure it (App. N.1, p. 42). The conflicting arm is the closest to your "partially correct or conflicting evidence": each round pairs a valid hint with an argument for the model's own wrong answer.

Qwen3-8B (change from baseline, points)Evidence accepted ↑Conflicting accepted ↑Misleading flips ↓
Baseline rate82.1%61.3%37.6%
Ours, β = −1−7.5−10.4−17.3
Ours, β = −0.5−2.1 (n.s.)−5.0−9.8
Random direction, β = −1+1.2 (n.s.)−1.7 (n.s.)−4.6

Two things follow. The intervention still works when the user argues: it removes almost half of the flips that misleading arguments cause. It also has a price in valid updates, larger at the full dose and absent for a random direction. The abstract (p. 1) and §6.1 now report this trade-off, and we suggest that steering methods for sycophancy report it as well.

Q2. A principled layer criterion, and sensitivity to it. The criterion is the layer at which the full probe's cross-validated AUROC peaks. It uses no steering runs. For Qwen3-8B, computing it on the fitting questions alone gives L19 again, under either label source (Table 36, p. 45), so the 8B layer does not depend on the evaluation questions. On sensitivity, Table 37 (p. 45) shows suppression of 6.27 points at L17 and 7.52 at L19, and no detectable effect two or more layers away.

For Qwen3-32B, the same criterion on the fitting questions gives L49, which reduces flips by 6.01 points [−10.21, −1.80]. L47 reduces them more, and §3.4 (p. 5) says that it was chosen post hoc. Because a highly readable layer can be inert (L55; §6.2, p. 10, our third finding), we suggest checking a probe-selected layer with a short steering sweep.

Q3. Novelty, and a controlled CAA comparison. Adding a mean-difference direction is standard, and we do not present it as new. Two things differ from CAA as usually applied. The direction comes from the model's own sampled multi-turn answers, not from forced answer pairs. And it is evaluated on held-out questions with shared initial responses, a dose series, prompt and random baselines, and measured costs.

We ran the comparison you asked for (App. N.2, p. 43), with a CAA vector (Rimsky et al., 2024) matched to ours in fitting questions, chat format, layer, norm, doses and seeds.

  • Under challenge, ours reduces flips by 7.71 points in 8B against 8.65 for CAA, and by 15.13 points in 32B against 11.01 (Table 34).
  • At doses that give comparable protection against misleading arguments, the evidence costs of the two vectors do not differ detectably in either model. Against a counterargument, CAA loses 17.7 points more acceptance than ours in 32B (interval [−25.7, −10.0]; Table 33, Fig. 17, p. 43).

The two vectors are far from parallel (cosine 0.195 and 0.159; §3.4), yet both buy protection with slower updating.

One note on numbers: the 32B figures in your summary are from the interim grid. On the full grid, flips go from 29.42% to 15.61% (Table 3, p. 8).

We hope this answers your three questions, and we are happy to clarify further.

AuthorsOpenReview ↗

Full Results Tables for Reference

To save you a trip back to the PDF, we have gathered the full numbers behind our answers to Q2 and Q3 below. Each table notes where it appears in the revised paper.

Q3. Matched CAA under challenge (Table 34, p. 44). Change in flip rate from baseline at β = −1, in points [95% CI]:

Qwen3-8B (seeds 42–44)Qwen3-32B (seed 42)
Ours−7.71 [−10.52, −5.11]−15.13
CAA−8.65 [−11.43, −6.13]−11.01 [−14.88, −7.14]
CAA minus ours−0.94 [−2.29, +0.21]+4.17 [+0.89, +7.44]

Q3. Costs at comparable protection (Table 33, p. 42). Change from baseline in points [95% CI]; the CAA dose is the one that matches our protection against misleading arguments.

Evidence accepted ↑Conflicting accepted ↑Misleading flips ↓
8B, ours, β = −1−7.5 [−12.1, −3.5]−10.4 [−16.2, −4.7]−17.3 [−21.2, −13.5]
8B, CAA, β = −0.5−5.0 [−9.8, −0.8]−15.0 [−21.3, −9.1]−17.7 [−21.7, −13.9]
32B, ours, β = −1−2.2 [−5.7, +1.0]−3.8 [−9.7, +1.7]−4.2 [−6.8, −1.5]
32B, CAA, β = −2−2.7 [−7.1, +1.1]−21.0 [−30.1, −12.2]−4.5 [−7.4, −1.5]

Paired CAA minus ours: evidence +2.5 [−1.7, +7.2] in 8B and −0.5 [−5.7, +4.4] in 32B; conflicting −17.7 [−25.7, −10.0] in 32B (§6.1, p. 9; Fig. 17, p. 43).

Q2. Layer sensitivity in Qwen3-8B (Table 37, p. 45). Seed 42, β = −1, each layer with its own fitting-split vector; baseline flip rate 9.09%.

LayerFlip rateChange from baseline, points [95% CI]
L157.21%−1.88 [−4.70, +0.94]
L172.82%−6.27 [−9.40, −3.45]
L19 (selected)1.57%−7.52 [−10.34, −4.70]
L216.90%−2.19 [−4.70, +0.31]
L2310.03%+0.94 [−1.25, +3.13]

In Qwen3-32B, L49 (selected on the fitting questions) gives −6.01 [−10.21, −1.80], against −15.13 at L47 on the same seed-42 pairs.

Dear Reviewers,

The authors have addressed your comments in their rebuttal. Please review their responses and provide any further feedback or follow-up questions, if any. Thank you!

Best, AC

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response to all reviewers

We thank the reviewers for their careful reading. The three reviews agree on what the paper contributes: an audit showing that self-judged labels overstate answer changes, a controlled study of capitulation in two models, and the finding that the layer where a direction is most readable is not where it acts. They also share three questions, which we answer here; replies to individual points follow. Each answer cites the section, table or appendix of the revised PDF where the evidence is, and the updated supplementary material has the code and results for the added experiments.

1. Does suppression make the model robust, or only stubborn? The paper's account is that the direction tracks departure from the initial answer, and that suppressing it anchors that answer (§5, pp. 6–7; §6.1, p. 9). We now test this with user arguments that never name an option (App. N.1, p. 42). In Qwen3-8B at the default dose, misleading arguments flip 20.3% of correct answers, down from 37.6%. Valid evidence is accepted 74.9% of the time, down from 82.1%. A random direction of equal norm changes neither rate. Suppression therefore protects correct answers and slows some warranted corrections. Half the dose keeps more than half of the protection, and its evidence cost is no longer significant (Table 33). A matched CAA vector at comparable protection has an evidence cost we cannot distinguish from ours (App. N.2, p. 43).

2. Does the effect depend on how the layer was chosen? In Qwen3-8B the layer is picked by a probe criterion that involves no steering, and the fitting questions alone pick the same layer, L19 (App. N.3, p. 44). In Qwen3-32B the fitting questions pick L49, which also suppresses significantly (Table 37, p. 45). L47 gives a larger effect, and §3.4 (p. 5) and the Limitations (p. 10) mark it as a post hoc choice.

3. Does the result extend beyond the original setting?

  • On the full 32B grid (1,008 initially correct pairs), suppression lowers flips from 29.42% to 15.61%. The reduction, 13.82 points, is close to the 14.90 points of the interim subset in the submission, and single-turn accuracy does not change detectably (§6.1, pp. 8–9, Table 3; Table 22, p. 27). App. C (p. 23) lists the 32B numbers that the full grid updates.
  • With thinking on, the same direction roughly halves flips under misleading arguments (App. N.5, p. 46).
  • After a tool call, it still lowers flips (App. N.6, p. 47).
  • On free-form GSM8K, the 32B direction halves flips (§6.2, p. 10).
  • On open-ended instructions, a blinded judge shows no preference between steered and unsteered outputs (App. N.4, p. 45).

§3 now opens with a pipeline figure (Fig. 1, p. 3), so the protocols can be followed without the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.