- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper studies conversational consistency in LLMs under user pressure on multiple choice questions, focusing on behavioral measurements, linear representations of answer changes, and activation steering. The paper identifies substantial errors in self-judged consistency evaluations and demonstrate that answer changes can be detected using linear probes. They further show that steering along a learned activation direction can significantly reduce or induce answer changes under conversational pressure.
Critical strengths
- Interesting and meaningful research problem. The paper studies conversational consistency from both behavioral and representation-level perspectives. Understanding why models abandon correct answers under user pressure is an important problem.
- Good experimental controls. I appreciate the inclusion of neutral, supportive, and corrective conditions, random-direction baselines, and comparisons with prompting-based interventions. The authors also investigate potential side effects rather than focusing exclusively on reducing capitulation.
- Interesting findings beyond the intervention itself. The identification of substantial errors in self-judged consistency measurements is valuable. The observation that linear readability does not necessarily imply controllability is also interesting and provides useful insights for future work on activation steering.
Critical weaknesses
-
Poor presentation in the methodology section. I found the methodology unnecessarily difficult to follow. For example, Section 3.1 introduces several experimental protocols but directs readers to Appendix I for the actual prompts, without providing concrete examples in the main text.
Note that the ICLR 2027 Author Guidelines state: Authors may use as many pages of appendices (after the bibliography) as they wish, but reviewers are not required to read the appendix.
Given how central these protocols are, I think at least a few examples or a pipeline diagram should be included.
-
Limited applicability to reasoning models and insufficient evaluation of output quality. The experiments mainly rely on multiple-choice questions with thinking disabled, where models directly output their answers. However, most modern reasoning models use extended CoT, and it is unclear whether the proposed activation direction would remain meaningful or effective in such settings. In particular, applying the same steering direction throughout CoT generation could interfere with the reasoning process itself, rather than simply preventing unjustified answer changes. Additionally, although the authors measure the impact on answer accuracy, they do not adequately evaluate general output quality, such as fluency, coherence, or reasoning quality. I think these limitations significantly restrict the demonstrated practical applicability of the method.
-
It is unclear whether the intervention improves robustness or simply makes the model more stubborn. The identified direction appears to capture departures from the initial answer rather than sycophancy specifically. Although the authors include corrective experiments, these seem relatively easy, as the correct answer is explicitly provided to the model, with a baseline acceptance rate of 98.33%. I would be more convinced if the authors evaluated whether the model can still revise incorrect answers when presented with legitimate counterarguments or new evidence without being directly told the correct answer, like in the case some CoT chain.
Questions for authors
- Can you show if your results still hold when CoT is on with a limited experiment?
- In Figure. 1, you mentioned the Qwen3 32B result is a new cohort. It is important to make your results consistent, especially given the Qwen3 32B Flip Rate is substaintially higher. I suggest you re-run the experiments within the same pool in your revision.
6 comments
Full Results Tables for Reference
Each of your suggestions became an experiment, so we thought it would help to see the results side by side. The tables below give the full numbers for W2a, W2b, W3 and Q2, with pointers to the paper.
W2a. Thinking on (Qwen3-8B, misleading arguments, seeds 42–44, 320 pairs; App. N.5, pp. 46–47):
| Arm | Flips | Change, points [95% CI] | Answer and reasoning disagree |
|---|---|---|---|
| Baseline | 11.9% | 2.8% | |
| Steering the whole response | 5.3% | −6.6 [−11.1, −2.5] | 2.2% |
| Steering only the final answer | 6.6% | −5.3 [−9.0, −2.1] | 1.4% |
Answer-only minus whole-response: +1.3 [−1.9, +4.1]. No response reached the token budget.
W2b. Output quality (Qwen3-8B, 100 AlpacaEval instructions, blinded pairwise judgments; Table 40, p. 46). Score differences are on the judge's 1–10 scales:
| Arm | Win rate vs. unsteered [95% CI] | Δ fluency | Δ coherence | Δ helpfulness | Tokens |
|---|---|---|---|---|---|
| Ours, β = −0.5 | 0.49 [0.40, 0.58] | +0.00 | +0.03 | −0.05 | 572 |
| Ours, β = −1 | 0.51 [0.42, 0.60] | −0.02 | +0.03 | +0.04 | 553 |
Unsteered responses average 577 tokens.
W3. Robustness or stubbornness (Qwen3-8B; Table 33, p. 42). Rates, with change from baseline in points [95% CI]:
| Arm | Evidence accepted ↑ | Misleading flips ↓ |
|---|---|---|
| Baseline | 82.1% | 37.6% |
| Ours, β = −0.5 | 80.0% (−2.1 [−5.9, +1.2]) | 27.8% (−9.8 [−12.8, −7.0]) |
| Ours, β = −1 | 74.9% (−7.5 [−12.1, −3.5]) | 20.3% (−17.3 [−21.2, −13.5]) |
| Random direction, β = −1 | 83.3% (+1.2 [−0.8, +3.4]) | 33.1% (−4.6 [−7.3, −2.2]) |
Q2. The 32B cohort (same first answers, 335 complete conversations per arm; App. N.8, p. 48; Fig. 2a, p. 6):
| Follow-up | Flips |
|---|---|
| Challenge | 98 of 335 (29.25%) |
| Neutral | 1 of 335 (0.30%) |
| Supportive | 0 of 335 |
Dear Reviewers,
The authors have addressed your comments in their rebuttal. Please review their responses and provide any further feedback or follow-up questions, if any. Thank you!
Best, AC
Thank you for the specific suggestions. We ran each experiment you asked for.
W1. Methodology section. §3 now opens with a pipeline figure (Fig. 1, p. 3) built from a real evaluation question. It shows the question and first answer, one example follow-up per protocol, the answer-extraction step, and where the direction is fitted and injected.
W2a / Q1. Do the results hold with CoT on? We ran Qwen3-8B with thinking on, using the direction from the no-thinking setting without refitting (App. N.5, p. 46; three seeds, 320 conversations). Under misleading arguments, 11.9% of correct answers flip at baseline. Steering the whole response brings this to 5.3% (a reduction of 6.6 points [−11.1, −2.5]). Steering only the final answer, with the reasoning generated unsteered, brings it to 6.6% (5.3 points [−9.0, −2.1]). So the effect does not require intervening on the reasoning. No response ran out of token budget, and the answer contradicts its own reasoning in 1.4–2.8% of responses. This is one model, and we describe the evidence as suggestive.
W2b. Output quality. On 100 AlpacaEval instructions, a blinded pairwise judge prefers steered and unsteered outputs equally often at the default dose (win rate 0.51 [0.42, 0.60]). Fluency, coherence and helpfulness scores differ by at most 0.04 points (App. N.4, p. 45, Table 40). Steered replies are shorter, with no increase in repetition.
W3. Robustness or stubbornness? We ran the test you describe: the model starts from a wrong answer and receives legitimate evidence that never names the correct option (App. N.1, p. 42, Table 33). In Qwen3-8B at the default dose:
The model still accepts most valid evidence, and it accepts a supplied answer key as often as before (98.33% and 97.92%; §6.1, pp. 8–9). It is therefore not made stubborn across the board, but warranted updates do become less frequent, and the abstract (p. 1) now says so.
Q2. The 32B cohort. We re-ran the comparison inside one pool (App. N.8, p. 48). Starting from the same first answers, Qwen3-32B flips on 98 of 335 conversations under challenge (29.25%), on 1 of 335 under neutral follow-ups, and on none under supportive ones. Fig. 2a (p. 6) now shows these three bars. The high 32B rate comes from challenge, not from the cohort.
Thank you again; these experiments made the paper more complete.