- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how language models influence each other during conversations across four models and four settings. It examines changes in stance, emotion, personality, semantics, and style, then uses activation steering to alter these behaviors. A single intervention can affect later responses, although movement toward the conversation partner varies by setting and timing.
Critical strengths
The study covers multiple models and conversational settings, showing that adaptation depends on both the behavior being measured and the interaction context.
The one-shot steering experiments show that changing one response can influence later, unsteered turns. The paper also reports settings where steering has little effect or reverses direction.
Critical weaknesses
Some conclusions rely on a partner-influence metric that becomes unstable when the reference states are close. Several reported means and medians even have opposite signs, making claims about the direction of influence difficult to assess.
The steering experiments lack controls that would show whether the identified directions have a specific effect beyond a general activation perturbation. Changes in behavioral scores also do not establish that steering improves task performance.
Questions for authors
Do the conclusions about the direction of partner influence hold when using robust summaries and excluding nearly identical reference states?
How do the learned steering directions compare with random directions matched in magnitude and layer?
Dear Reviewer aAd6,
Thank you for the careful review. Both of your questions were useful, and each led to a new analysis that is now in the paper. We appreciate the opportunity to address your concerns and improve the cogency of our arguments sufficiently for your reconsideration. Table and appendix numbers refer to the revised PDF.
Q1. Do the conclusions about the direction of partner influence hold with robust summaries and after excluding nearly identical reference states?
Yes. The direction of partner influence holds for four of the five variable families, and one specific claim has been corrected.
You are right that the metric is unstable when the two reference states are close, and that means and medians disagreed in sign. The appendix of the submission already reported medians for this reason, but Figures 2 and 4 in the main text showed means. We re-analysed all 1,036,800 observations:
The corrected claim is the statement that Qwen and Gemma move away from their partners in emotion. It came from the means; their medians are 0.44 and 0.41, so we removed it. Figures 2 and 4 now show medians (the median of α within each dyad and dimension, averaged over partners and then over the dimensions of a family), the text of Sections 5.1 and 5.2 follows them, and the new Appendix F.4 (Table 14) gives means, medians, pooled ratios, trimmed means and both exclusion analyses for every setting and variable family.
Q2. How do the learned steering directions compare with random directions matched in magnitude and layer?
The learned directions have a clearly larger effect. We ran this control under the paper's steering protocol (same layers, same per-layer norm, same strength, prompts and decoding) for all four models and one variable per setting, 16 cells in total. For each direction we measure how far the target score of the steered response moves between positive and negative steering.
So the effect depends on the direction, not on the size of the perturbation. Section 5.3 now reports this control, and the new Appendix G.6 (Table 25) gives the protocol and the results for every cell.
Behavioural scores and task performance
We agree that a change in a behavioural score does not show that steering improves task performance. The paper does not report a task outcome, and what it shows is control of behavioural variables. Three passages could be read more broadly, so we have reworded them: the definition of behavioral variables in Section 3, the ethics statement, and the conclusion, which now says that whether such control preserves task outcomes remains to be tested. The quality paragraph of Section 5.3 also now states that likelihood and repetition diagnostics are not measures of task success.
Summarized changes
We hope this answers your questions, and we are glad to clarify anything further.