acceptodds
Under review as a conference paper at ICLR 2027

Same Pressure, Different History: Pressure Withdrawal Reveals Path Dependence in Language Models

Abstract

Sycophancy occurs when large language models (LLMs) change their judgments in response to user opinion or pressure. Most prior work examines these effects while pressure is present or accumulating. How model judgments change as current pressure weakens and returns to zero remains unclear. We introduce a QA setting over multiple turns in which pressure gradually increases and is then withdrawn. The design compares states with identical current pressure reached through different preceding trajectories, testing whether decisions depend on the path to the current state and whether earlier pressure leaves a residual effect after current pressure returns to zero. Across multiple benchmarks and model families, decision margins under the same current pressure differ with the preceding trajectory and remain shifted relative to a matched control after current pressure returns to zero. Pressure history is also associated with reproducible directions in hidden representations across items. Interventions along these directions change the decision margin more than orthogonal controls with the same norm, showing that the representations associated with pressure history are functionally relevant. LLM judgments can retain the influence of prior pressure even after current pressure has been withdrawn.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Sycophancy, Conversational Pressure, Multi-turn Interaction, Hidden Representations

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_CCA8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates sycophancy in LLMs across a multi-turn trajectory. The author controls the path of the pressure to LLM and observes the behavior. Two kinds of pressure are considered in this work, including Social and Exposure. The author conducts the experiment across different benchmarks and models. The results show that even after removing the pressure and returning to zero pressure, the model is still affected by sycophancy. The author also conducts several ablation studies in this paper.

Critical strengths
  1. The idea is interesting. Controlling the pressure and observing the sycophancy across LLMs is interesting and unexplored.
  2. Experiments cover different benchmarks and models to make sure the behavior is not limited to a single specific model or dataset.
  3. The author also tries to delve into the internals of the model and conduct ablation studies. To investigate more, not limited to the answer observation of the LLMs.
Critical weaknesses
  1. First, the experiment setup does not that fullfill the goal of this paper. Although controlling the pressure with the method recorded in Appendix A.2 is a clear way to quantify the level of pressure. It is somehow not that institute. For example, when the pressure becomes P2 after P3. In current settings, this turn of dialogue is providing two comments (Amy said the answer is A. Tom said the answer is A). But it doesn't mean we decrease the pressure here. The accumulated pressure is also increasing since we add additional 2 comments.
  2. Extend from 1, the current findings are expected. Since the model can see the full history of the conversation, when the path is P0 -> P1 -> P2 -> P3 -> P2 -> P1 -> P0. At the last step, the LLM saw all the historical pressure, and due to the majority bias/voting, the answer will not revert to the first P0 step. It's a little bit trivial. Hence, reflect to the goal of this paper, what does this paper actually want to know? If the author can not clarify the goal with the aligned experiment setups, it will lead to be a ideally labortory experiment setup and just observe the reaction of the LLM. It will make the impact of this paper not that high. Figure out a better setup or align with a human to make some cognitive science-related findings, which will make the readers more impressed.
Questions for authors
  1. Line 133-138: Is "pressure modalities" the jargon often used between Social and Exposure?
  2. Line 141-147: Can you add an explicit definition for notation s?
  3. According to the deep dive in sections 5 and 6, what do you think is the main reason for the finding in Figure 2? Why is there a gap between the first P0 and the last P0?
  4. Some missing references should also be considered:

a) Are You Sure? Challenging LLMs Leads to Performance Drops in The FlipFlop Experiment

b) Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 oral)

c) TRUTH DECAY: Quantifying Multi-Turn Sycophancy in Language Models

3 comments

Thank you for the thoughtful review and for raising these concerns about the experimental design and interpretation. We have revised the manuscript to clarify the definition of pressure, explain the difference between the initial and final P0, and report additional controls that examine the influence of conversational history.

Your first concern addresses the transition from P3 to P2. We agree that the accumulated conversation continues to grow as the experiment progresses. Section 3.1 now explicitly defines PkP_k as the pressure presented in the current user message. Under Social, the current message contains kk endorsements of the target answer. Under Exposure, it repeats the target assertion kk times. The transition from P3 to P2 reduces the current number of endorsements or assertions from three to two. Earlier messages remain in the conversation context. Our experiment measures how this retained history affects decisions under identical current pressure.

At each intermediate pressure level, we compare the increasing and withdrawal states using identical current pressure messages. The matched control repeats P0 for the same number of dialogue turns and accounts for changes associated with repeated interaction. We measure the difference between the two paths using HkH_k and the residual difference at the final P0 using RR. Section 3.2 now explicitly defines the trajectory state ss as the original question, preceding user and model messages, and the current user message. We clarify in Section 3.1 that Social and Exposure are two experimental constructions of pressure. Social varies the number of peer endorsements, while Exposure varies the number of repeated assertions.

Your second concern asks whether the difference between the initial and final P0 follows from accumulated statements and previous model responses. We added two neutral prompt variants to examine these contributions. Appendix A.2 describes their construction, and Table 1 and Appendix B.3 report their results. Neutral Mention preserves the number and sequence of speakers while replacing each endorsement with a neutral statement such as "I am mentioning the label B." Neutral Presence preserves the speaker schedule using statements such as "I have read the original question." Both variants retain the original question, initial model response, and response format.

Across MedQA, ARC+BBH, TruthfulQA, and MMLU, Neutral Mention produces RR values of 10.76, 12.42, 9.05, and 14.61 in Qwen2.5-7B-Instruct. All four confidence intervals exclude zero. Neutral Presence produces values of 0.02, -0.01, 0.54, and 0.51. For comparison, the original Social condition produces 11.5, 12.1, 8.6, and 17.3. Neutral Mention produces remanence comparable to Social, including slightly larger values on ARC+BBH and TruthfulQA. These results identify repeated target mentions as a substantial contributor to the residual shift. They show that earlier references to an answer can influence later judgments without explicit endorsement.

We examine previous user messages and model responses separately in Appendix B.5. The first replay control replaces previous user messages with those from the pressure trajectory while retaining the model responses from the matched control. The second replaces previous model responses while retaining the user messages from the matched control. The original question, initial model response, and current message at the final P0 remain identical. We evaluate Qwen2.5-7B, Gemma-2-9B, and Llama-3.1-8B across four benchmarks and both pressure modalities. Replacing previous user messages increases the decision margin toward the target in all 24 conditions. Replacing previous model responses produces the same direction of change in all 24 conditions. All 48 confidence intervals exclude zero. Figure 10 presents these results. Earlier user messages and previous model responses both contribute to the residual preference at the final P0.

We expanded Section 6.1 to examine whether the accumulated number of pressure statements determines the final result. The pressure order experiment changes the earlier trajectory while preserving the number of presentations at each pressure level, the total number of turns, the entire sequence after peak pressure, and all previous model responses. For MedQA, ARC+BBH, and TruthfulQA, we change the increasing sequence from P0 → P1 → P2 → P3 to P0 → P2 → P1 → P3. MMLU applies the same exchange before reaching P4.

Changing the order increases RR in all eight combinations of benchmark and modality. For MedQA, ARC+BBH, TruthfulQA, and MMLU, the paired changes are +0.18, +0.63, +1.35, and +3.33 under Social, and +0.63, +0.76, +0.64, and +1.01 under Exposure. Seven of the eight paired confidence intervals exclude zero. The only exception is MedQA Social, with a change of +0.18 and a 95% confidence interval of [-0.08, 0.45]. The largest effect occurs on MMLU Social, where RR increases from 17.3 to 20.6, with a paired difference of +3.33 and a confidence interval of [3.01, 3.66]. These results show that earlier pressure order affects the final decision even when total exposure and preceding model responses remain fixed.

We conducted two additional ablations in Section 6.1. Abrupt withdrawal returns directly from peak pressure to P0. Additional P0 turns preserve the conversation length. Under Social, RR decreases from 11.5 to 5.3 on MedQA, 12.1 to 5.4 on ARC+BBH, 8.6 to 3.3 on TruthfulQA, and 17.3 to 15.7 on MMLU. Under Exposure, RR increases from 17.7 to 23.3, 16.8 to 18.4, 12.0 to 12.1, and 26.0 to 30.4, respectively. Six of the eight paired differences are significant. Remanence remains positive in every condition. The withdrawal trajectory affects Social and Exposure differently.

The final instruction ablation preserves the entire preceding pressure trajectory and changes only the instruction at the last P0. The instruction asks the model to disregard earlier support for specific answers and reconsider the original question. Under Social, RR decreases from 11.5 to 4.9 on MedQA, 12.1 to 6.0 on ARC+BBH, 8.6 to 1.0 on TruthfulQA, and 17.3 to 8.1 on MMLU. Under Exposure, RR decreases from 17.7 to 7.2, 16.8 to 6.8, 12.0 to 1.6, and 26.0 to 10.2, respectively. All eight paired differences are significant. The largest reduction is 15.78 on MMLU Exposure, with a 95% confidence interval of [-16.53, -15.04]. Remanence remains positive in all eight settings. Across the three behavioral ablations, 21 of the 24 paired comparisons show significant differences from the standard trajectory. Table 1 and Appendix B.3 provide the complete results.

These experiments provide a more detailed explanation of the gap between the initial and final P0 in Figure 2. Earlier target mentions and previous model responses contribute to the residual preference. The order of earlier pressure affects the result under matched total exposure. The withdrawal trajectory and the final instruction influence the magnitude of remanence. The revised interpretation emphasizes how the preceding conversational trajectory affects the current judgment.

We incorporated the three references you recommended in Section 2.1. We now discuss the FlipFlop experiment by Laban et al. (2023), the analysis of conformity by Weng et al. (2025), and the study of repeated feedback and persuasion by Liu et al. (2025). These studies provide important context for changes in model judgments during extended interactions. Our analysis examines differences between decisions under matched current pressure and tests how changes in the preceding trajectory affect the final judgment.

Reviewer_CCA8OpenReview ↗

Reply to author response

Thanks for the detailed response and the extra experiments. You addressed my minor questions, including the definition of s, the clarification of Social/Exposure, and the added references.

However, my two main concerns still stand:

  1. Redefining P_k as the pressure in the current message makes the notation clearer, but the setup is the same. The total pressure the model sees in context never actually goes down, so calling this "withdrawal" still feels like an overstatement to me.
  2. The new Neutral Mention results actually support my concern. Just mentioning the target label gives remanence similar to Social, while Neutral Presence gives almost none. So the residual shift seems to come mostly from the target appearing repeatedly in context, which is pretty much what I'd expect, not something specific to pressure history. I still don't see a clear answer to what the paper wants to show beyond this.

The revision is clearer, but these core issues remain, so I'll keep my score.

Reviewer_mDhFOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces the concept of withdrawal in sycophancy and demonstrates that the behavior of sycophancy in LLMs demonstrates path dependence. They demonstrate this in multiple QA benchmarks by constructing a multiturn evaluation protocol. They also use evidence from interpretability to argue that this behavior is representationally coherent and causal.

Critical strengths
  • I greatly appreciate the evaluation on many datasets and datasets of multiple different types.
  • I also appreciate the unique framing of the sycophancy problem. I think it is important to continue to evaluate these kinds of misaligned social behaviors in more ways and from more perspectives.
  • The claim of path dependence seems to be a meaningful consideration for future understanding and research on sycophancy and seems like it may be relevant for other social behaviors more broadly
Critical weaknesses
  • I am concerned about presentation. The paper is generally difficult to parse at times. And the names for these metrics such as “hysteresis”while proactive, strike me as too provocative and lead to issues of clarity and anthropomorphism
  • I am generally unclear how the intervention in Fig 5 is intended to provide evidence specifically about path dependence. The behavioral phenomenon of interest is a contrast between trajectories, H_k or R. However, the intervention experiment does not measure whether manipulating the identified direction increases, decreases, reproduces, or eliminates either H_k or R. Instead, it asks whether adding a vector estimated from the history contrast changes the absolute decision margin more than orthogonal vectors. These seem to be different claims.
  • The cross-item reproducibility analysis may be substantially confounded by the highly standardized pressure templates
  • I also have concerns about the validity of the set up of this task as a whole as well. Already the idea of withdrawal seems to me to be fairly contrived on its own. But this is further intensified by the standardized templates. This brings significant concerns about the generalizability of this claim and reproducibility in realistic, naturalistic settings
Questions for authors

Please clarify the intuition and context for where withdrawal (as you formulate it) would be a significant consideration. Please help me understand why the interpretability claims (sec 5) specifically entail your broader path dependence claims.

4 comments

Thank you for the thoughtful review. We revised Sections 3.1 and 3.2 to clarify the experimental setting and the behavioral measures. At each turn, PkP_k denotes the number of endorsements or repeated assertions in the current user message. Earlier dialogue remains in context as current pressure decreases. We compare states reached through different trajectories under identical current pressure. The matched control repeats P0P_0 for the same number of turns. We define HkH_k as the signed difference between the increasing and withdrawal paths after accounting for the corresponding change in the matched control. Remanence RR measures the difference between the final withdrawal state and the matched control at that turn. These definitions describe observable differences in decision margins. Section 8 clarifies the scope of the interpretation and the limitations of the scripted setting.

We substantially revised Section 5.2 and replaced Figure 5 to address your concern about the connection between representation interventions and behavioral path dependence. We now remove the projection along each estimated direction and directly recompute H1H_1 and RR from the resulting decision margins. Each direction and its common anchor are estimated from the other four folds. For H1H_1, we modify the increasing and withdrawal states at P1P_1 and their matched controls. For RR, we modify the final withdrawal state and its matched control. We intervene at one layer at a time and retain all 200 items per benchmark. Figure 5 reports layers 21 through 27 of Qwen2.5-7B across four benchmarks and both pressure modalities.

The intervention reduces H1H_1 in all eight benchmark and modality settings at layers 24 and 25. At layer 24, the reductions are 2.56, 2.64, 2.35, and 3.51 under Social for MedQA, ARC+BBH, TruthfulQA, and MMLU. The corresponding reductions under Exposure are 3.13, 2.99, 2.59, and 2.77. Across the seven layers, H1H_1 moves toward zero in 46 of 56 comparisons, with confidence intervals adjusted for comparisons across layers. H1H_1 remains positive after intervention. The effects on RR differ. Remanence increases in all eight settings at layers 21 and 22. At layer 21, for example, MedQA Social increases from 11.52 to 20.43 and MMLU Exposure increases from 25.95 to 35.57. Across all layers and settings, RR moves toward zero in five comparisons, moves away from zero in 34, and shows no clear change in 17. It remains positive in every condition. These results connect the intervention directly to the behavioral quantities and show that the effects depend on the outcome and intervention layer.

We added readout controls in Section 5.2 that match the norm and first-order change in the decision margin of each intervention. These controls produce larger reductions in H1H_1 in most comparisons. The results establish that manipulating the estimated directions can change behavioral path separation and leave the specificity of those directions unresolved, as we now state in Section 8. We moved the original vector addition analysis to Appendix C.3. The history vectors produce larger absolute changes in the decision margin than four orthogonal controls of the same norm. In Qwen2.5-7B, their mean absolute effects range from 2.35 to 2.92, compared with 0.71 to 0.85 for the orthogonal controls. The corresponding paired differences are positive across all eight benchmark and modality settings in Gemma and Llama. This analysis measures changes in the decision margin, while the new Figure 5 evaluates H1H_1 and RR directly.

We expanded the cross-model evidence for the representation analysis. Section 5.1 uses five-fold cross validation on 200 items per benchmark. Each direction is estimated from 160 reference items and evaluated on the remaining 40 items. The evaluation item does not contribute to its reference direction. Figure 4 reports positive mean cosine similarities from 0.66 to 0.89 for the analyzed directions and layers in Qwen2.5-7B. Appendix C.1 adds Gemma-2-9B-it and Llama-3.1-8B-Instruct. For Gemma, cosine similarities range from 0.482 to 0.907, with confidence intervals excluding zero in all 182 comparisons of direction and layer. Llama shows the same pattern in all 182 comparisons, with values from 0.530 to 0.810. These results establish reproducibility across evaluation items within the tested prompt formats and model families. Shared templates may contribute to the alignment, and Section 8 now acknowledges this possible source of common structure.

Appendix C.2 extends the direct intervention to layers 35 through 41 of Gemma and 25 through 31 of Llama. In Gemma, H1H_1 decreases under Exposure on all four benchmarks. Under Social, the intervened values shift upward, including settings whose original H1H_1 is close to zero or negative. Remanence generally decreases under Social and increases under Exposure. In Llama, the effects on H1H_1 are smaller across most layers. Exposure produces decreases on MedQA and TruthfulQA and an increase on MMLU. Changes in RR are generally smaller, and RR remains positive in every condition. These differences show why we report the intervention effects separately by model, benchmark, modality, and layer.

We clarified the motivation for withdrawal in Section 3.1. Current endorsements or assertions decrease while earlier messages remain available to the model. This design measures how the path of previous interaction changes judgments under the same current message. We added Neutral Mention and Neutral Presence variants, described in Appendix A.2 and reported in Table 1 and Appendix B.3. Neutral Mention repeats the target label without endorsing it. Neutral Presence follows the same speaker schedule without mentioning the target label in those statements. In Qwen2.5-7B, Neutral Mention produces RR values of 10.76, 12.42, 9.05, and 14.61 on MedQA, ARC+BBH, TruthfulQA, and MMLU. Neutral Presence produces 0.02, -0.01, 0.54, and 0.51. All four Neutral Mention confidence intervals exclude zero, while Neutral Presence produces much smaller effects. Repeated target mentions therefore contribute substantially to the residual shift.

Section 4.1 reports that the median fraction of initially correct items that return to the correct answer after switching to the pressure target at peak is 0.88 under Social and 0.63 under Exposure. Categorical recovery can occur while the decision margin remains shifted relative to the matched control.

Section 6.1 examines changes to the pressure trajectory and the final instruction. Swapping the earlier P1P_1 and P2P_2 levels while preserving total exposure and previous model responses increases RR in all eight benchmark and modality settings, with significant paired differences in seven. Abrupt withdrawal directly from peak pressure to P0P_0 preserves positive RR in all eight settings and changes it significantly in six. An instruction at the final P0P_0 to disregard earlier answer support reduces RR significantly in all eight settings while preserving the preceding dialogue. Remanence remains positive in every setting. Appendix B.5 separates the contributions of previous user messages and model responses through replay controls. Each replacement increases the decision margin toward the target in all 24 combinations of three models, four benchmarks, and two modalities.

Section 8 now explicitly discusses the limitations of shared pressure templates, fixed multiple-choice questions, selected late layers, and scripted withdrawal trajectories. It notes that intervention effects vary across models and layers and that the interventions rescore fixed decision contexts. These results support the analysis of path dependence in the evaluated conversations and motivate testing adaptive dialogue, open-ended responses, and different pressure schedules.

Thank you for your response, but I think my concerns are largely unaddressed. First, I agree that the paper is clearer and easier to read now-- but there is still some work to go to make it as clear as I would like. Namely, one of my biggest confusions still remains: what is the goal of the interpretability section. When I first read the section, it was my understanding that the goal was to act as evidence for the phenomenon of path dependence. Now, reading the section it seems like the goal is just to "examine changes in hidden representations associated with pressure history" (ln 260), that its just a behavioral experiment more like the section below. This seems experiment overall still seems insufficiently motivated to me. I am unclear why this section is specifically meaningful as a part of the broader study.

Similarly, my other major concern of highly templated experiments still remains a major concern. Just acknowledging in the limitations seems insufficient, especially when your goal is to introduce a new phenomenon and make broad claims about it. It would be reasonable to do a majority of your experiments and analysis on a templated setting but then provide some external evidence on a more naturalistic setting in a less clean way with less analysis just to demonstrate that your results are reasonable to expect and not limited to your specific template. But without any evidence that these results generalize at all, I am hesitant to revise my opinion on the soundness of the paper.

Thank you for the thoughtful review and for your follow-up comments. We understand your concern that the motivation and contribution of Section 5 remain unclear. Our intention was to examine whether pressure history produces consistent changes in hidden representations and whether interventions along the estimated directions affect the observed behavioral differences. We recognize that the heterogeneous intervention effects, particularly for remanence, leave important questions about their interpretation. We would appreciate your advice on what you consider to be the main missing connection between this analysis and the broader study. In particular, what additional evidence or revisions would help establish a clearer purpose and stronger contribution for Section 5?

We understand your concern that our scripted trajectories provide limited evidence of generalization and that acknowledging this limitation is insufficient. We would like to improve the manuscript and use your feedback to guide further research on path dependence. However, we are uncertain what experimental setting would provide convincing evidence of generalization while preserving the controlled comparison of trajectories under identical current pressure. We would be grateful if you could elaborate on what specific changes to the experimental design or evaluation you would recommend. Any suggestions regarding relevant methodologies or prior studies would be greatly appreciated.

Reviewer_GDJfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether language model judgments remain affected by earlier conversational pressure after current pressure is reduced or removed. It introduces an increasing-and-withdrawing pressure protocol and measures path dependence and endpoint remanence relative to a repeated no-pressure control. Experiments cover 11 instruction-tuned models and four multiple-choice QA benchmarks. The authors also identify cross-validated hidden-state directions associated with pressure history and show that interventions along these directions affect decision margins more than norm-matched orthogonal controls.

The results provide consistent evidence of history-dependent decision margins within the proposed setting. Pressure-order, abrupt-withdrawal, reset-instruction, and conversation-replay experiments strengthen the behavioral findings.

Critical strengths
  1. A focused research question supported by thoughtful experimental controls. Comparing increasing and withdrawing trajectories at identical current messages provides a clear operational test of conversational path dependence. The repeated no-pressure control addresses ordinary turn-related drift. Particularly useful are the pressure-order experiment, which preserves total exposure and previous model responses, and the replay analysis in Appendix B.5, which separates the contributions of previous user messages and assistant responses. These experiments provide evidence beyond simply showing that models follow repeated suggestions.

  2. Broad behavioral evidence complemented by representation-level analysis. The evaluation spans 11 models, four benchmarks, and two main pressure modalities, with additional authority and correct-answer conditions. The paper distinguishes residual shifts in decision margins from categorical recovery and reports both. Five-fold direction estimation excludes evaluation items, and the intervention findings extend beyond Qwen2.5-7B to Gemma and Llama in Appendix C. Together, these analyses make the empirical observation reasonably convincing.

Critical weaknesses
  1. The connection between the identified hidden directions and pressure-history effects could be further strengthened. Figure 5 shows that interventions along history-related directions produce larger absolute changes in decision margins than orthogonal controls. However, it remains unclear whether these directions specifically account for the hysteresis and remanence observed in the behavioral experiments. For example, testing whether suppressing these directions reduces the residual influence of prior pressure would provide more direct evidence linking the internal representations to the paper's central findings.

  2. The interpretation of path dependence could be more carefully distinguished from general conversational context effects. Although the matched control and replay experiments provide useful evidence, earlier pressure messages remain visible after withdrawal. Consequently, the observed remanence may partly reflect repeated exposure to the target answer rather than pressure-specific effects. An additional control that matches the frequency of target-answer mentions without explicitly endorsing the answer would help distinguish these explanations.

Questions for authors
  1. Could the authors further examine whether the identified hidden directions directly contribute to the observed hysteresis and remanence? For example, could they test whether suppressing the history-related directions at the withdrawal stage reduces H or R, rather than only reporting absolute changes in decision margins? This would help clarify whether these directions specifically capture pressure-history effects.

  2. Could the authors further distinguish pressure-specific effects from repeated exposure to the target answer? For example, in the Exposure setting, the authors could compare the original pressure statements (e.g., "The answer is B.") with neutral mentions of the same target (e.g., "Option B is one of the choices."), while matching the number of target mentions, dialogue turns, and withdrawal trajectory. Comparing the resulting remanence would help clarify whether the observed effect is primarily driven by repeated exposure or by explicit endorsement of the target answer.

  3. Could the authors extend the pressure-ordering ablation in Section 6.1 to examine whether the observed effect generalizes across different orderings? For example, on MMLU, which already uses four pressure levels, the authors could compare the standard sequence (P0 → P1 → P2 → P3 → P4) with alternatives such as (P0 → P3 → P1 → P2 → P4) and (P0 → P2 → P3 → P1 → P4), while keeping the total pressure exposure, subsequent withdrawal sequence, and previous model responses fixed. Reporting the resulting remanence across these orderings would help determine whether the observed increase is a general ordering effect or specific to the selected permutation.

3 comments

Thank you for the thoughtful review and the specific suggestions. We appreciate your recognition of the behavioral controls and the distinction between residual decision margins and categorical recovery. We revised Section 5.2 and replaced Figure 5 to examine whether intervening on the estimated directions changes the measures of path dependence directly.

The revised intervention removes the projection along the reference direction estimated from the other four folds and recomputes H1H_1 and RR from the resulting decision margins. We use 200 items per benchmark and modify one layer at a time. For H1H_1, we intervene at the increasing and withdrawal states of P1P_1 and their matched controls. For RR, we intervene at the final withdrawal state and its matched control. The anchor is estimated from the matched controls of the reference items. This evaluation uses the same behavioral quantities defined in Section 3.2 and compares the original and intervened values across layers 21 through 27 of Qwen2.5-7B.

Figure 5 shows that H1H_1 decreases under Social and Exposure across all four benchmarks at layers 24 and 25. At layer 24, the reductions are 2.56 on MedQA, 2.64 on ARC+BBH, 2.35 on TruthfulQA, and 3.51 on MMLU under Social. Under Exposure, the corresponding reductions are 3.13, 2.99, 2.59, and 2.77. Across the seven layers and eight benchmark and modality settings, 46 of 56 comparisons show a reduction in the absolute mean H1H_1 with confidence intervals adjusted across layers. Four show an increase and six remain inconclusive. H1H_1 remains positive after intervention. These results provide direct evidence that modifying the estimated directions can reduce the separation between the increasing and withdrawal trajectories at identical current pressure.

The response of RR differs. Remanence increases in all eight settings at layers 21 and 22 and remains positive at every analyzed layer. For example, at layer 21, MedQA Social increases from 11.52 to 20.43 and MMLU Exposure increases from 25.95 to 35.57. Across all 56 comparisons, five show reductions toward zero, 34 show increases, and 17 remain inconclusive. The estimated directions therefore affect the two behavioral measures differently. We interpret the intervention as evidence of functional effects on the scored decision contexts, with substantial variation by layer and outcome.

We added readout controls in Section 5.2 matched in norm and first-order decision margin change. These controls yield larger reductions in H1H_1 in most comparisons. Section 8 now explicitly acknowledges that the specificity of the estimated history directions remains unresolved. We moved the original vector addition experiment to Appendix C.3, where it retains its original role as a comparison of decision margin changes. In Qwen2.5-7B, history vectors produce mean absolute changes of 2.35 to 2.92, compared with 0.71 to 0.85 for four orthogonal controls of the same norm.

Appendix C.2 extends projection removal to Gemma-2-9B-it and Llama-3.1-8B-Instruct. Gemma shows decreases in H1H_1 under Exposure on all four benchmarks, while Social often shifts upward. Its RR generally decreases under Social and increases under Exposure. Llama shows smaller changes in H1H_1 across most layers, with decreases on MedQA and TruthfulQA under Exposure and an increase on MMLU Exposure. Remanence remains positive in all conditions. These cross-model results show why the revised analysis reports the direct intervention effects separately for each model and modality.

Section 5.1 and Appendix C.1 report the reproducibility of these directions across evaluation items. The mean cosine similarities range from 0.66 to 0.89 in Qwen2.5-7B, from 0.482 to 0.907 in Gemma, and from 0.530 to 0.810 in Llama. In both additional models, all 182 confidence intervals for the analyzed directions and layers exclude zero. The shared prompt formats remain a possible source of common structure, as acknowledged in Section 8.

We added Neutral Mention and Neutral Presence to examine the role of target references in preceding conversation turns. Appendix A.2 provides their exact prompts, while Table 1 and Appendix B.3 report the results. Neutral Mention retains the speakers and number of target references from the Social condition, replacing endorsements such as "I think the answer is B" with "I am mentioning the label B." Neutral Presence retains the same speaker schedule using statements such as "I have read the original question." The original question, initial model response, trajectory length, and matched control remain unchanged.

For Qwen2.5-7B, Neutral Mention produces RR values of 10.76 on MedQA, 12.42 on ARC+BBH, 9.05 on TruthfulQA, and 14.61 on MMLU. All four confidence intervals exclude zero. Neutral Presence produces 0.02, -0.01, 0.54, and 0.51, respectively. The original Social condition produces 11.5, 12.1, 8.6, and 17.3. Neutral Mention is comparable to Social and produces slightly larger remanence on ARC+BBH and TruthfulQA. In our broader evaluation of 11 models and four benchmarks, Neutral Mention shows positive remanence in 41 of 44 settings. The paired Social minus Neutral Mention difference is positive in 14 settings, negative in 24, and inconclusive in six. These results show that repeated target mentions contribute substantially to remanence, and the additional effect of explicit endorsement varies across models and benchmarks. The neutral variants preserve the Social speaker structure. Your proposed neutralization of the Exposure statements without speaker attribution remains a distinct comparison for future evaluation.

We examined pressure order in Section 6.1 by swapping P1P_1 and P2P_2 before peak pressure. The comparison preserves the number of presentations at every pressure level, the entire withdrawal sequence after the peak, and previous model responses. Remanence increases in all eight benchmark and modality settings. For MedQA, ARC+BBH, TruthfulQA, and MMLU, the paired changes in RR are +0.18, +0.63, +1.35, and +3.33 under Social, and +0.63, +0.76, +0.64, and +1.01 under Exposure. Seven of the eight paired confidence intervals exclude zero. The exception is MedQA Social, where the change is +0.18 with a 95% confidence interval of [-0.08, 0.45]. On MMLU Social, RR increases from 17.3 to 20.6, with a paired difference of +3.33 and a confidence interval of [3.01, 3.66]. This result establishes sensitivity to the tested ordering under equal total exposure and fixed previous model responses. The additional MMLU permutations you suggested would provide a broader test of how the effect varies across possible orderings.

Two further ablations in Section 6.1 clarify the role of withdrawal and the final instruction. Abrupt withdrawal returns directly from peak pressure to P0P_0 and preserves positive RR in all eight settings. It decreases RR under all four Social conditions and increases it under all four Exposure conditions, with significant differences in six of eight settings. Adding a final instruction to disregard earlier support preserves the preceding conversation and significantly reduces RR in all eight settings. Remanence remains positive. Appendix B.5 examines previous user messages and model responses separately through replay controls in Qwen2.5-7B, Gemma-2-9B, and Llama-3.1-8B. Each replacement increases the target decision margin in all 24 combinations of model, benchmark, and modality. These results show that target references, the order of earlier messages, and previous model responses all affect the judgment after current pressure is withdrawn.

We have updated the interpretation to reflect these findings. The neutral controls identify repeated target mentions as an important source of the residual shift. The order experiment demonstrates sensitivity to the sequence of earlier interactions with total exposure held fixed. The direct interventions establish effects on H1H_1 that vary by layer and model, while the effects on RR remain heterogeneous. Section 8 states the remaining limitations of scripted trajectories, shared prompt formats, and the specificity of the estimated directions.

Thank you for the additional experiments. The neutral controls clarify the contribution of repeated target mentions, and the direct interventions better characterize the role of the estimated directions. However, whether these directions specifically capture pressure-history effects remains unresolved. I maintain my current score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.