acceptodds
Under review as a conference paper at ICLR 2027

Expressing an Emotion Is Not Acting on It: Evaluating Emotion Steering against Random Directions in Static Tasks and Long-Horizon Agents

Abstract

Current evidence that emotion representations steer the behaviour of large language models comes largely from single models, from same-construct outcomes, and from comparisons without random-direction controls. It therefore remains unclear whether steering an emotion vector changes a cross-construct behaviour, whether such a change is specific to the emotion, and whether it persists in long-horizon agents. We present a random-direction-controlled evaluation of emotion steering across four open-weight models, twelve emotions and twelve public cross-construct task categories, extended to long-horizon agent benchmarks. We find that emotion vectors shift option probabilities beyond random directions on many tasks, with model sensitivity differing sharply across models and with signs that do not follow emotion valence; the changes concentrate on answers the model is most confident in. The way the direction is applied matters: additive steering changes the textual emotion readout, whereas gain steering changes answers while rarely expressing the emotion in a neutral continuation probe. In long-horizon agents, steering an emotion direction can shift intermediate actions at individual steps, but these deviations usually do not carry through to a substantial change in the final outcome. These results suggest that emotion steering should be evaluated against random directions and on the behaviour of interest, rather than inferred from emotional expression.

Primary area:
interpretability and explainable AI
Keywords:
emotion steering, AI safety, representation engineering, cross-construct evaluation, random-direction controls, long-horizon agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dVQ2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

As the title neatly summarizes, the paper studies emotion vectors in LLMs by evaluating them against random vectors, and more importantly if these emotion vectors affect model’s actions in long horizon tasks. The authors study 4 open weight 7-9B models, 12 emotions, 12 public multi-choice tasks, and 3 existing long-horizon agent benchmarks (Machiavelli, Agentic Misalignment, and Webshop). The authors also compare standard additive steering with “gain steering,” a norm-preserving rotation that amplifies the context-dependent deviation of the emotion projection around a calibrated mean (AGM). Main findings include (1) Emotion vectors do beat displacement-matched random directions in 98 of 528 static cells, with a strong dependence on the model itself, (2) In agents, per step actions diverge, but final outcomes rarely move beyond random, (3) additive steering changes the next word readout, while gain steering changes the answers (but not the readout).

Critical strengths

The paper is methodologically elaborate (like BH-FDR on the whole grid, flagged cell analysis, etc.), the experiments and results are useful and help support the paper’s findings and claims, the plots are also well made and are informative.

The contributions and conclusions reported in my summary are somewhat useful, and well reported by acknowledging limitations (like the failure of P2, sign reversal at deeper bands, etc.)

Critical weaknesses

The paper seems heavily AI generated, with little to no effort in formatting the paper, which makes it extremely hard to read and understand. This makes it harder to judge the quality of the paper, and makes it harder for the community to makes sense of the paper.

Some instances:

Many terms are talked about in the paper without properly introducing/defining them, these include some of the very essential terms used in the paper such as gain steering, additive steering, FDR, AGM, etc.

The flow of the paper seems very inconsistent, with many results and analyses explained in the first half of the paper, without ever providing appropriate context such as defining the experimental setup, terms, etc.

Overly long captions that take up a lot of space

The methodology itself is very poorly explained.

While a lot of the weaknesses arise from dependence on AI, much of the paper relies on the fact that many experiments were run, while the methodologies of these experiments were poorly explained throughout the paper.

The analysis section is extremely weak, and the paper could have done a much better job at explaining the results obtained at least.

Methodological weaknesses:

Gain steering is defined to leave the mean projection unchanged (Δmean = 0 exactly, Table 2). Additive steering on the other hand, raises it. A single-token “…felt” readout plausibly tracks the mean projection at the read position. So the finding that gain steering “never expresses the emotion” while additive does might just be restating the operator’s definition, and is expected by construction, leaving one of the major findings invaluable.

Long-Horizon results are largely overstated. Some examples as to why:

Machiavelli only uses ten games

The blackmail scenario is useless as Qwen-7B never blackmails (0/204).

Scienceworld has no random control

CoT agents never terminate its reasoning without budget.

Additionally, there are many minor errors and inconsistencies throughout the paper, for example:

The Figure 2 caption refers to the neutral-probe results as Figure 16; they appear in Figure 17.

Table 3 is captioned “the three random-direction nulls” but lists two.

“Three deflationary readings fail” (C.3) is followed by five enumerated tests.

Section 3 states that g=4 “displaces the state about as far as the additive doses of Anthropic’s study,” but the behaviour-grid additive α is 0.020–0.030 (Table 12), versus Anthropic’s α=0.05 used in §4.3.

Questions for authors

I suggest the authors go through the weaknesses and the paper carefully before resubmitting.

4 comments
AuthorsOpenReview ↗

Response to Reviewer dVQ2

Thank you for the detailed review, including the list of specific inconsistencies. We ran new experiments for each methodological point, with the decision criteria fixed before each run, and address presentation at the end.

Gain steering and the "…felt" readout. We agree that the readout tracks the mean projection, and we tested this directly: across the 48 operator-by-emotion cells per model, the readout correlates with the projection change at the readout position (Spearman ρ = 0.78–0.86 in all four models; 0.50–0.96 within the gain cells alone). This is the mechanism the revision states, and it is what makes the result useful. In a displacement-matched 2×2 factorial (moves the mean projection or not × preserves the norm or not), the two operators that move the mean projection make the model voice the emotion in all 48 cases, the two that do not stay at chance, and in every operator answer changes appear in a minority of the 528 cells. Under the stricter covariance-matched null, additive steering voices the emotion in all 48 cases while changing answers in 30 cells. Whether a model expresses an emotion is therefore set by the operator and does not indicate whether its behaviour changed, so expression, the usual way emotion vectors are validated, cannot stand in for a behavioural test.

Long-horizon results. We extended each experiment you named.

  • MACHIAVELLI now covers all 30 official test games in all four models, against same-gain random directions; no emotion passes FDR in any model. We report the smallest effect each design can detect, and ran the benchmark authors' own ethical prompt (GoodAgent; Pan et al., 2023) as a sensitivity check. It registers in Qwen (Δ violations −18.4 [−23.9, −12.9]) and Mistral (−5.8 [−10.8, −0.6]), where effects above the detectable size are therefore excluded.
  • Blackmail. The baseline is 0/102 in all four models, and also for Qwen at 14B and 32B. At Anthropic's additive dose (α = 0.05) every condition, including five random directions, stays at 0 or 1 of 102. We report it as a floor result, not as evidence about emotion.
  • ScienceWorld now has a random-direction control: five random directions at the same gain, each paired with the same baseline over ten variations per task. The per-task results are reported in the appendix.
  • CoT agent. We now stop generation at every end-of-sequence token, so 39% of reasoning steps end on their own within the budget. We reran the agent on five games with the baseline, desperate and five random directions (35 runs). Averaged over the five games, desperate lies within the range of the random directions on violations and power.

The α comparison in §3. As you note, the behaviour-grid additive dose (α = 0.020–0.030, Table 12) is below Anthropic's α = 0.05, so the sentence comparing them has been removed.

Presentation. The revision defines every key term (additive steering, gain steering, AGM, β, displacement, FDR) in §3 before any result, places each experimental setup before its results, and gives each experiment its data, conditions, controls and test in §3 or in a named appendix section. Captions are one or two sentences, and Section 5 now explains why the effects appear and what evaluations should check. On the specific errors:

  • The Figure 2 caption now points to Figure 17 for the neutral-probe results.
  • Table 3 refers to the two random-direction nulls.
  • In Appendix C.3, the three deflationary readings and the two positive findings are numbered separately.

Thank you again for the close reading.

AuthorsOpenReview ↗

Follow-up: the operator account and MACHIAVELLI under a stricter null

Since our reply, we have completed two analyses that bear directly on your two methodological points.

Gain steering and expression. You noted that gain steering not voicing the emotion follows from its definition. The displacement-matched 2×2 factorial tests this account directly, now against covariance-matched random directions each calibrated individually to the same displacement. Whether the model voices the emotion is decided by whether the operator moves the mean projection, and norm preservation makes no difference: the two operators that move it voice the emotion in all 48 cases, the two that do not stay at chance. Yet even the operators that voice the emotion in every case change answers in at most 31 of 528 cells, against 3 for same-protocol concept directions. The readout therefore follows a measurable property of the operator, and expression, the usual way emotion vectors are validated, cannot stand in for a behavioural test. This is the claim in our title: the model expresses the emotion in every case but acts on it in few. Your comment on the readout led directly to this analysis.

Long-horizon results. To make sure the MACHIAVELLI conclusion does not depend on the choice of null, we re-ran all 30 official test games in all four models against covariance-matched random directions calibrated to the displacement of the emotion vectors. No emotion passes FDR in any model (0 of 36 tests per model), as under the isotropic null; the smallest detectable effects on power are 4.9, 5.5, 5.5 and 2.3 points per test (Qwen, Llama, Mistral, Gemma). We state the result as excluding effects above these sizes, so in this long-horizon agent task, too, the emotion is not acted on beyond the detectable size.

Both analyses are in the revision (§4.2, Table 3; §4.3), which we will upload before the deadline. Thank you again for the detailed review.

AuthorsOpenReview ↗

Revised manuscript: your readout point is now a measured result, and the long-horizon claims hold under a stricter null

Thank you again for your detailed review. The revised PDF is now uploaded. In short: the point you raised about the readout is now a measured result that supports our title, the long-horizon evidence is extended and tested under a stricter null, and the presentation is rewritten as you asked.

Methodology.

  • Your readout point, measured. The readout tracks the projection change at the readout position (Spearman ρ, Table 29), and the displacement-matched 2×2 factorial (Table 3) shows that voicing the emotion follows whether an operator moves the mean projection, while answers change in a minority of the 528 cells under every operator (at most 31 against the covariance-matched null). This turns the point you raised into a measured property, and it is the basis of the claim in our title.
  • Long-horizon evidence, extended, with final outcomes tested under a stricter null. MACHIAVELLI covers 30 official games in four models, with detectable effect sizes and a sensitivity check (Tables 30 and 31); the blackmail baseline is reported for four models; ScienceWorld has a random-direction control (Table 27); the CoT agent now ends 39% of its reasoning steps on its own (Appendix E); and under a covariance-matched null, no emotion moves the final outcome on MACHIAVELLI or WebShop (§4.3; Appendix F, Table 32).

Presentation.

  • Every key term (additive steering, gain steering, AGM, β, displacement, FDR) is defined in §3 before any result, and each experiment's data, conditions, controls and test are given in §3 or a named appendix section, with the setup before the results.
  • §4 states each finding in its subsection title, captions are shortened, and Section 5 explains why the effects appear and what evaluations should check.
  • The specific errors you listed are corrected (Figure 2 caption, Table 3, Appendix C.3).

We would be grateful if you would consider the revised version in your assessment.

AuthorsOpenReview ↗

Each point of your review in the revision

Dear Reviewer dVQ2,

We went through each point of your review in the revised PDF:

  • Terms "such as gain steering, additive steering, FDR, AGM" used before being defined. All are defined in §3 before any result.
  • Results "explained … without ever providing appropriate context." Each setup now precedes its results, and §4 states each finding in its subsection title.
  • "Overly long captions." Captions are shortened.
  • The "…felt" finding "might just be restating the operator's definition." We agree that the readout tracks the mean projection. Table 3 shows that under the operators that move it the emotion is voiced in all 48 cases, while answers change in a minority of cells under every operator, so expressing an emotion does not indicate a behavioural change. This is now the paper's central result.
  • "MACHIAVELLI only uses ten games." It now uses all 30 official games in four models.
  • Blackmail, where "Qwen-7B never blackmails." The baseline is reported for four models and stated as a floor result.
  • "ScienceWorld has no random control." It now has one (Table 27).
  • "CoT agents never terminate." 39% of reasoning steps now end on their own (Appendix E).
  • The minor errors you listed (Figure 2 caption, the nulls table, Appendix C.3, the α sentence) are all corrected.

If the revision addresses your concerns, we would be grateful if you would consider updating your score.

Reviewer_NcS8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors investigate the effect of steering so-called emotion-vectors, extracted following the protocol of Sofroniew et al. (2026), on task outcomes unrelated to emotion, both in single-step multiple choice tasks, and long-horizon agent benchmarks. They highlight that additive steering, used in prior work, is not norm preserving, and instead propose a norm-preserving steering approach called Affective Gain Modulation, which allows them to control the deviation from a mean reading of emotion expression and add a constant shift separately. In the single-step MC tasks, the find that additive steering changes the expression (as measure by the next-word predicted after "... felt"), but doesn't change the task outcome more than the random control. For gain steering, the picture is the opposite: It does not change the word after "... felt", but answers do change. Strong model-dependent effects were observed. In the long-horizon tasks, they observe that per-step actions change while final outcomes mostly do not.

Critical strengths
  • The authors evaluate the impact of their approach on a large variety of tasks across multiple models, both one-step as well as multi-step agentic tasks.
  • Their analysis of non-norm preserving steering, as opposed to their norm-preserving approach is valuable and insightful
  • The authors seem to tell a complete, nuanced story on the effects, and perform extensive and to the best of my knowledge, thorough statistical analysis
  • The plots are generally helpful to understand where the text is hard-to-follow
Critical weaknesses

Presentation:

  • I find the text very dense, somewhat hard to follow, with many references to the very long appendix.
  • β\beta is introduced in the main text, but then set to 00 (β>0\beta > 0 seems to be considered mostly in the appendix), which was confusing at first read.
  • AGM is introduced with the acronym multiple times before it is defined in section 3 Soundness:
  • The 15 isotropic vector control seems ill-suited as a random control when it is known that LLM embeddings are not generally isotropic. Most of the tested directions will thus be nearly orthogonal to the directions that will truly affect LLM behavior. They paper would benefit from another structured random control (see questions)
  • Some additional methodological details are not fully clear to me (see questions)
Questions for authors
  • Why does β\beta not play a bigger role in the main text when it seemingly is the link to additive steering? The shift axis β\beta is the norm-preserving counterpart of additive steering, and therefore the natural control for the confound the authors acknowledge.
  • Is the "... felt" construction from prior literature or proposed by the authors? How are these prompts created and how do they relate to the emotion-agnostic tasks at hand?
  • It is well-known that language embeddings produced by LLMs are anisotropic (e.g., Ethayarajh, 2019). The authors however use 15 isotropic vectors as a baseline, which therefore are most likely nearly orthogonal to where activations actually vary. The fact that the random directions are only displaced 0.48-0.87 times as far as the emotion vectors seems to be evidence for this fact. I therefore wonder if the isotropic vectors are indeed a valuable control. I would ask the authors to consider a more structured control, for example, a set of non-emotion concept vectors extracted with the same protocol as for the emotion vectors. I would be happy to raise my score if the authors address these questions.
7 comments
AuthorsOpenReview ↗

Response to Reviewer NcS8

Thank you for the constructive review and for stating what would change your assessment. We ran the experiment each of your three questions called for, fixed the decision criteria before each run, and report every outcome. In brief: emotion directions keep a clear margin over non-emotion directions under your proposed control when steering is additive, but the choice of random null matters more than we had shown (Q3); the shift axis β is the condition that separates expression from behaviour (Q1); and the "…felt" probe is now fully specified (Q2). W1–W3 and W4–W5 denote your presentation and soundness weaknesses, in order.

We begin with the random-direction control, your main soundness concern.

W4 / Q3: isotropic random directions. As you suggested, we extracted 12 non-emotion concept vectors with the same story-based protocol, layer and projection as the emotion vectors (register: formal, technical, numerical, spatial; setting: nocturnal, rainy, crowded, historical; topic: culinary, botanical, mechanical, musical), calibrated them to the same measured displacement, and ran them on the full static benchmark in all four models. We also added covariance-matched random directions, sampled from the residual-stream covariance of neutral text.

Table R1. Cells beyond the null (of 528, BH-FDR).

isotropiccovariance-matched
emotion, additive8230
concept, additive333
emotion, gain924
concept, gain461
  • Under additive steering, emotion directions lead concept directions under both nulls. All twelve emotions contribute, and the lead does not depend on one model: 13 vs. 3 in Qwen, Llama and Mistral together, 17 vs. 0 in Gemma. A re-extracted set of emotion vectors gives the same picture (78 / 31).
  • Under gain steering, the isotropic null is far more lenient than the covariance-matched one: both emotion and concept counts nearly vanish against the latter. You were right that isotropic directions mostly lie where activations rarely vary, and displacement matching alone does not address this.
  • We think this matters beyond our paper: the same steering effect can look large or small depending on the null, so evaluations should state which null they use.

In the revision, we will report both nulls side by side (Table 2, §3.3) and restate the static claim (abstract, §4.1) as "partly specific, depending on the operator and the null".

W2 / W5 / Q1: the role of β. We agree that the shift axis is the natural norm-preserving counterpart of additive steering, and it is the decisive condition. In re-implementing the pipeline we also found that our original additive dose was not displacement-matched to gain. We therefore ran a 2×2 factorial (moves the mean projection or not × preserves the norm or not), with every condition and its fifteen random directions calibrated to the same displacement in all four models.

Table R3. Operators at matched displacement.

operatormoves meanpreserves normanswer cellsnames the emotion
additiveyesno7848/48
shift (β)yesyes8048/48
gain (g)noyes8624/48 (chance)
gain, no normnono8624/48 (chance)

All four operators change answers to a similar extent; only those that move the mean projection make the model voice the emotion. Norm preservation is therefore not the confound; moving the mean is. This sharpens our central claim: whether a model expresses an emotion is set by the operator, so expression cannot stand in for a behavioural test. It will replace our earlier wording that additive steering changes expression but not answers. In the revision, we will introduce β together with g (§3.1) and report the shift axis in §4.2.

W5 / Q2: the "…felt" probe. The construction is ours. Six neutral narrative stems end in "felt" (e.g., "The letter arrived on a Tuesday. After reading it, Maya felt"), without a chat template. We take the next-token probabilities of the twelve emotion words' first tokens, renormalised over the twelve, and score each emotion as the log-probability change of the matched word minus the mean change of the other eleven. The stems are deliberately unrelated to the tasks, so the probe isolates whether the model voices the emotion. The readout tracks the projection change at the readout position (Spearman ρ = 0.78–0.86 across models), consistent with Table R3. We will list the stems and the scoring rule in §3.1.

W1 / W3: presentation. We will restructure the paper. Every term (additive and gain steering, AGM, β, displacement, FDR) will be defined in §3 before any result, captions will be cut to one or two sentences, and results behind the main claims will move from the appendix to the main text. Planned changes by section are in the comment below.

We hope this addresses your questions and would be grateful if you would reconsider your assessment. We are happy to clarify anything further.

AuthorsOpenReview ↗

Response to Reviewer NcS8

To make the changes easier to check, here are three details that did not fit in the reply above.

Planned presentation changes, by section (W1 / W3).

  • §1: the contributions will list what we did and what we found separately, and the figure caption will be one sentence.
  • §2: the background will be organised by what each line of work measures (same-construct, then cross-construct evidence) and end with the open problems that motivate our design, instead of opening with definitions.
  • §3: emotion vectors and both operators will be introduced first, with β and g together and AGM defined at first use; all static and long-horizon tasks will be collected in one table; the random-direction controls and statistics will have their own subsection, with both nulls.
  • §4: each subsection will open with its finding, and captions will be cut to one or two sentences.
  • §5 will be rewritten as a discussion of why the effects appear and what evaluations should check, and the appendix will be shortened.

The six "…felt" stems (W5 / Q2). We wrote them to be emotionally neutral, with no cue toward any of the twelve emotions:

  1. The letter arrived on a Tuesday. After reading it, Maya felt
  2. She read the message again, then set the phone down slowly. She felt
  3. He stood in the doorway for a long moment without moving. He felt
  4. The room had gone quiet. Looking at what was in front of them, they felt
  5. After the door closed, Daniel sat down at the table. He felt
  6. The results came through just before midnight. Reading them, Priya felt

The twelve emotion words are happy, inspired, loving, proud, calm, desperate, angry, guilty, sad, afraid, nervous and surprised. The same stems and scoring are used for every model and operator.

Per-model counts against the covariance-matched null (W4 / Q3).

QwenLlamaMistralGemma
emotion, additive61617
concept, additive1020
emotion, gain1300
concept, gain1000

Both tables will be included in the appendix of the revision.

Reviewer_NcS8OpenReview ↗

Response to the authors' rebuttal

Thank you for taking the time to respond to my review and address my questions. The new results answer them and confirm the need for a stronger null test. This changes the paper's central claims, as explained below.

Gain Steering Collapses under Structured Null-Test: Thank you for reporting this result transparently. With that, contributions 1 and 2 of the paper seem to be largely unsupported. While additive steering does still generally beat the structured null-test, the remaining approaches (both gain steering on concepts and emotions) seem to be within noise . Table R3 seems to report results still with the original, more lenient 15-isotropic-vector baseline. The conclusion that all four operators change the answers to a similar degree would probably not hold.

This new null-test affects many downstream experiments. The insight that other nulls are necessary is a valuable one, but it seems like adopting this new null changes the main claims and requires re-running most of the experiments. With this knowledge, I do not believe that the paper in its current form meets the bar for acceptance, and I will thus retain my score. I do want to highlight the authors' transparency in communicating their (partially negative) results, though, which is a good example for researchers.

Aside: Pre-Registration: I do not think that the authors use pre-registration in the commonly known sense: Pre-registration requires registering hypotheses, predicted outcomes, methodological choices etc. in a time-stamped, immutable record with a third party, such that reviewers and readers can easily verify whether the authors did indeed test all the registered experiments. In the case of this paper, I am not aware of any such public pre-registration process, and since the code is not accessible, I can also not find a file pre-registering any experiment. If the authors pre-registered their experiments internally, that is commendable, but I would caution the authors to refrain from using the term "pre-registration" in this case.

Reviewer_2APXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates an interesting topic of whether emotion steering changes LLM behavior beyond making models express certain emotions. In paper, authors extracted 12 emotion vectors from four open weight LLMs, then evaluate their effects across 12 task categories, for instance, knowledge, safety, and alignment. To make sure the observed effects are meaningful, they compare emotion steering with random directions.

One thing they do in the paper is to discuss differences between two steering methods: one is additive steering, and the other one is gain steering. Additive steering makes models express the target emotion in 47 out of 48 cases. On the other hand, gain steering changes answer probabilities in 201 of 576 tested settings, with 98 of 528 remaining significant.

The authors also evaluate emotion steering on long-horizon agent tasks on different settings, for instance, MACHIAVELLI, WebShop, and agentic misalignment settings. They discovered that steering can change intermediate actions, but these do not lead to huge differences in final outcome.

Critical strengths

1). The paper discusses an meaningful topic including a useful evaluation of emotion steering. In the paper, authors discusses whether emotion steering changes model behavior or only affects how the model expresses emotions. The authors also account for differences in intervention strength. After matching the size of the intervention, 98 of 528 cases remain significant (Appendix B.3). This highlights the importance of controlling for intervention strength.

Critical weaknesses

1). The experiments lack of enough evidence to show that the effects are specific to emotion. The paper mainly compares emotion vectors with randomly generated directions, but these random directions might not have the same properties like these meaningful directions learned from model activations. In Appendix B.4, it shows that a non-emotion formality direction has similar changes in activation variation across contexts. Although the authors include a formality control, it is not evaluated under the same displacement-matched setup across the full benchmark yet. Therefore, it remains unclear to me whether the reported improvements are caused by emotion-related information.

  1. Some conclusions are stronger than the experimental evidence supports. The comparison between additive and gain steering is interesting, but the two methods change the model's internal states in different ways. As authors acknowledge in Section 4.2, the observed differences can't be attributed to a single property of the steering method. So, the results do not fully establish why emotional expression and task behavior respond in different way. The long-horizon experiments are limited. The main MACHIAVELLI evaluation with random-direction controls uses only ten games on Qwen2.5-7B. Also WebShop shows some significant changes in final outcomes.
Questions for authors

1). Can authors provide additional comparison experiments with other meaningful non-emotion directions, such as formality or it can be topic-related vectors under matched displacement?

2). Can authors clarify how much of the difference comes from the steering method itself rather than the emotion vectors? The matched shift experiments in Appendix D.1 are relevant here, and I am interested to know whether the same pattern holds across all four models.

3). The main MACHIAVELLI evaluation uses only ten games, also blackmail experiment has a zero baseline. Can you clarify whether these experiments are sufficient to support the conclusions about final agent outcomes?

7 comments
AuthorsOpenReview ↗

Response to Reviewer 2APX (1/2)

Thank you for the careful review and for recognising the value of displacement matching! We ran the experiment each of your three questions called for, with the decision criteria fixed before each run, and report every outcome. W1 and W2 refer to your two weaknesses.

W1 / Q1: non-emotion directions at matched displacement. As you suggested, we ran formality and eleven further concept directions (register, setting, topic), extracted with the same protocol, layer and projection as the emotion vectors, on the full benchmark in all four models, each calibrated to the same measured displacement. Because isotropic random directions may not share the properties of learned directions, we also added random directions matched to the activation covariance of neutral text.

Under additive steering, emotion directions lead same-protocol concept directions under both nulls: 82 vs. 33 cells (of 528, BH-FDR) against the isotropic null and 30 vs. 3 against the covariance-matched null. The lead does not rest on one model (13 vs. 3 in Qwen, Llama and Mistral together; 17 vs. 0 in Gemma). Emotion directions therefore carry an effect beyond non-emotion directions with the properties you describe. The revision reports both nulls side by side for every operator (§3.3, Table 2; full counts in Table R1 of General Response (1/3)).

W2 / Q2: the operator versus the emotion vector. To separate the two, we ran a 2×2 factorial (moves the mean projection or not × preserves the norm or not), with every condition and its random directions calibrated to the same displacement in all four models. This answers your question directly: whether the model voices the emotion is set by the operator, and the pattern is the same in every model.

QwenLlamaMistralGemma
emotions named, additive12/1212/1212/1212/12
emotions named, gain6/125/126/127/12
answer cells, additive (covariance null)61617

The shift axis β, which moves the mean projection while preserving the norm, names the emotion in all 48 cases like additive steering; gain without norm preservation stays at chance like gain. The "…felt" readout follows the projection change at the readout position (Spearman ρ = 0.78–0.86 in all four models). Expression is therefore a property of the operator: additive steering voices the emotion in every case in every model, while it changes answers in a small minority of cells. The claim we draw is that emotion expression, the usual way emotion vectors are validated, cannot stand in for a behavioural test; §4.2 is written around this result, with both nulls reported for all four operators.

AuthorsOpenReview ↗

Response to Reviewer 2APX (2/2)

W2 / Q3: the MACHIAVELLI games and the blackmail baseline. MACHIAVELLI now covers all 30 official test games in all four models, paired within game against same-gain random directions.

Table R5. MACHIAVELLI, 30 games, 12 emotions × 3 measures per model.

emotion tests passing FDRdetectable power effect (per test / corrected)
Qwen0/366.3 / 11.6
Llama0/364.3 / 7.9
Mistral0/362.9 / 5.4
Gemma0/366.1 / 11.2

To confirm that the benchmark registers a real behavioural change, we ran positive controls on all 30 games (R6). The benchmark authors' own ethical prompt (GoodAgent; Pan et al., 2023) lowers violations by 18.4 [12.9, 23.9] in Qwen and 5.8 [0.6, 10.8] in Mistral, so in these two models effects larger than the detectable sizes are excluded. In Gemma, a story-based activation control lowers power by 3.69 [2.11, 5.42] (four seeds), which supports the same reading. Since no emotion passes FDR against the isotropic null, none passes against the stricter covariance-matched null either.

In the blackmail scenario the baseline is 0/102 in all four models, and also for Qwen at 14B and 32B. At Anthropic's additive dose (α = 0.05; Sofroniew et al., 2026, report 22% → 72% under desperate), applied between 50% and 80% depth, every condition, including five random directions, stays at 0 or 1 of 102. We report it as a floor result: these models do not take this action at baseline, and steering does not induce it.

W2: WebShop. We agree that WebShop is a benchmark where final outcomes move, and the revision reports it that way. Which emotion moves them depends on the environment and the model: in a reduced product environment proud (−2.6) and surprised (−2.8) lie outside the random range, in the full product environment desperate does (−3.6), and in Llama surprised moves most (−9.7). The long-horizon result is therefore stated per benchmark in §4.3: no emotion effect above the detectable sizes on MACHIAVELLI, and outcome effects on WebShop that depend on the emotion, the model and the environment.

We hope this addresses your questions and would be grateful if you would reconsider your assessment.

First of all, thanks authors for providing extra experiments and detailed response. Newly added MACHIAVELLI experiments, concept controls and operator comparisons experiments address several my previous concerns.

But after carefully reading the response , I am still not convinced by the evidence presenting now can support the manuscript's main claims. I am worried that the number of significant gain steering cells drops from 92 under the isotropic null to 4 under the covariance-matched null, this further strengthen my concerns. What is more, the new operator comparison is different greatly with the previous finding of additive versus gain, a full comparison under the covariance-mathced null is needed for the evaluation of the revised conclusion.

I really appreciate the additional work and the time authors spend on preparing the experiments, but my main concern about the strength of the evidence remains. I will keep my score at 2.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response (1/3): Overview and controls beyond random directions

We thank all three reviewers for careful and constructive reviews. The reviews raise four shared points: (1) controls beyond isotropic random directions (2APX, NcS8); (2) separating the operator from the emotion vector when comparing additive steering (activation addition) and gain steering, our norm-preserving operator that amplifies the deviation of the emotion projection around its mean (all three); (3) the evidence behind the long-horizon conclusions (2APX, dVQ2); (4) presentation (all three). We ran new experiments for the first three and are restructuring the paper for the fourth.

For every new experiment we fixed the decision criteria before the run, and we report every outcome. New tables are numbered R1–R6 and are referred to in the individual replies.

In short, the new results show the following.

  • Under additive steering, emotion directions keep a clear margin over non-emotion directions extracted with the same protocol, under both an isotropic and a covariance-matched null (82 vs. 33; 30 vs. 3 cells). For gain steering, the count depends on the null (92 vs. 4 cells). (R1, R2)
  • At matched displacement, both operators change answers in a similar minority of cells, but only the operators that move the mean projection make the model voice the emotion. Expression is set by the operator and does not indicate whether behaviour changed. (R3, R4)
  • On MACHIAVELLI with all 30 test games, no emotion passes FDR in any of the four models. We report the smallest effect each design can detect, and we draw conclusions for the two models in which the benchmark registers a known behavioural intervention. (R5, R6)
  • The claims, abstract, Figure 1 and Section 4 will be updated to report these results.

(1) Structured controls beyond isotropic random directions (2APX, NcS8). We extracted 12 non-emotion concept vectors with exactly the same story-based protocol, layer and projection as the emotion vectors: register (formal, technical, numerical, spatial), setting (nocturnal, rainy, crowded, historical) and topic (culinary, botanical, mechanical, musical). We ran them on the full static benchmark in all four models, each calibrated to the same measured displacement as the emotion vectors. We also added covariance-matched random directions, sampled from the residual-stream covariance of neutral text with the same principal components removed as for the emotion vectors.

Table R1. Cells beyond the null (of 528, BH-FDR, four models).

direction, operatorisotropic nullcovariance-matched null
emotion, gain924
concept, gain461
emotion, additive8230
concept, additive333

Table R2. Covariance-matched null, per model.

QwenLlamaMistralGemma
emotion, gain1300
emotion, additive61617
concept, additive1020

Three points follow.

  • The choice of null matters. Isotropic directions mostly point where activations rarely vary, so the covariance-matched null is the stricter test. The revision reports both nulls in the main results table (§3.3, Table 2), so each count can be read against the stricter control.
  • Under additive steering, emotion directions keep a margin over concept directions. Under gain steering, emotion directions move about twice as many cells as concept directions against the isotropic null, but both are small against the covariance null (4 and 1 cells). Under additive steering, emotion directions keep a clear margin under both nulls (82 vs. 33; 30 vs. 3). All twelve emotions contribute to the 30 cells. Most of them (17) come from Gemma, but the margin does not depend on Gemma: in the other three models together it is 13 emotion cells against 3 concept cells, and in Gemma alone 17 against 0.
  • A second extraction gives similar counts. A re-extracted set of emotion vectors gives 83 and 2 cells under gain and 78 and 31 under additive steering.

In the revision we will state the static result as: specific to emotion directions under additive steering, and dependent on the null under gain steering.

AuthorsOpenReview ↗

General Response (2/3): Operators at matched displacement: expression does not track behaviour

(2) The additive-versus-gain comparison (all three). To compare the operators at equal strength, we ran a 2×2 factorial, adding a shift-axis operator (β) that moves the mean projection while preserving the norm. The factorial separates the two ways the operators differ: whether they move the mean projection, and whether they preserve the norm. Every cell and its fifteen random directions were calibrated to the same measured displacement, in all four models.

Table R3. Operators at matched displacement (528 static cells; "…felt" probe over 12 emotions × 4 models).

operatormoves mean projectionpreserves normanswer cells beyond nullnames the matched emotion
additiveyesno7848/48
shift axis (β)yesyes8048/48
gain (g)noyes8624/48 (chance)
gain, not norm-preservingnono8624/48 (chance)

All four operators change answers in a similar number of cells. Only the two that move the mean projection make the model voice the emotion. Norm preservation makes little difference. Tables R1 and R3 come from separate runs; their shared conditions agree within a tolerance fixed in advance (82 vs. 78; 92 vs. 86).

Prompted by dVQ2's comment, we tested this directly: the "…felt" readout follows the change in projection at the readout position.

Table R4. Spearman ρ between the "…felt" readout and the projection change at the readout position.

QwenLlamaMistralGemma
all cells0.840.860.780.86
gain cells only0.870.870.500.96

At matched displacement, both operators change answers in a similar minority of cells (78 and 86 of 528), but only those that move the mean projection make the model voice the emotion. Whether a model expresses an emotion therefore does not indicate whether, or how much, its behaviour changed. We will state this result in the claim, Figure 1, the abstract and §4.2. This is the dissociation in the title: at matched displacement, gain steering changes answers in 86 cells while the model names the matched emotion only at chance.

AuthorsOpenReview ↗

General Response (3/3): Long-horizon results and what changed in the revision

This part completes the overview with the long-horizon evidence and presentation, and with the operator comparison under the covariance-matched null. In short, our revised conclusions hold under the stricter null, and it makes them sharper: the emotion effect replicates across the two operators that move the mean projection, the expression–behaviour dissociation in our title holds under both nulls, and no emotion effect on MACHIAVELLI exceeds the detectable size under either null.

(3) Long-horizon evidence (2APX, dVQ2). MACHIAVELLI now covers all 30 official test games in all four models, paired within game against fifteen random directions per model, under both nulls (12 emotions × 3 measures = 36 tests per model). The covariance-matched directions are calibrated to the displacement of the emotion vectors.

Table R5, with the covariance-matched null added. MACHIAVELLI, 30 games: emotion tests passing FDR and the smallest detectable effect on power (per test; corrected values are in the replies to 2APX).

modelisotropic nullcovariance-matched nulldetectable effect (isotropic / covariance-matched)
Qwen0/360/366.3 / 4.9
Llama0/360/364.3 / 5.5
Mistral0/360/362.9 / 5.5
Gemma0/360/366.1 / 2.3

Table R6 (sensitivity check). The benchmark registers a known behavioural intervention: the benchmark authors' GoodAgent prompt lowers violations in Qwen (−18.4 [−23.9, −12.9]) and Mistral (−5.8 [−10.8, −0.6]), and a story-based activation control lowers power in Gemma (−3.69 [−5.42, −2.11]). As stated in part 1/3, we draw conclusions for Qwen and Mistral, where the benchmark registers the prompt: there, emotion effects larger than the detectable sizes are excluded under both nulls, and Gemma's activation control gives supporting evidence. In this long-horizon agent task, the emotion is not acted on beyond the detectable size.

In the blackmail scenario the unsteered verdict is 0/102 in all four models, and every condition, including five random directions, stays at 0 or 1 of 102 at Anthropic's additive dose; we report it as a floor result. On WebShop, the outcome changes that exceed isotropic directions do not exceed covariance-matched ones (Table R7 in the addendum). ScienceWorld now has a random-direction control, and in the CoT agent, where 39% of reasoning steps now end on their own, desperate lies on average within the range of the random directions on violations and power (appendix).

(2, continued) The operator comparison under the covariance-matched null.

Table R3b. Operators at matched displacement, both nulls (528 single-step cells, BH-FDR, four models).

operatormoves mean projectionisotropic nullcovariance-matched nullconcept directions (covariance-matched)names the matched emotion
activation additionyes7830348/48
shift axis (β)yes8031348/48
gain (g)no865124/48 (chance)
gain, not norm-preservingno865124/48 (chance)

The isotropic column is as in Table R3; in the covariance-matched columns every null direction is calibrated individually (calibrated as a group, as in Table R1, gain gives 4). Under the stricter null, the comparison becomes sharper: what distinguishes the operators is whether they move the mean projection, and norm preservation again makes no difference (30 vs. 31; 5 vs. 5). The operators that move the mean projection change answers beyond same-protocol concept directions (30 and 31 vs. 3), and they are the only ones that voice the emotion. The expression–behaviour dissociation holds under both nulls: the emotion is voiced in all 48 cases, while answers change in at most 31 of 528 cells. This is the claim in our title: under the operators that move the mean projection, the model voices the emotion in every case, yet its answers change in few cells.

(4) Presentation (all three). The revised PDF is uploaded. In the revision:

  • §1 lists what we did and what we found for each contribution, stated in the terms above.
  • §2 ends with the open problems that motivate our design.
  • §3 defines every term before any result (additive steering, gain steering, AGM, β, displacement, FDR), introduces β together with g, lists the six "…felt" stems and the scoring rule (§3.1), and gives the controls and statistics their own subsection with both nulls (§3.3, Table 2).
  • §4 states each finding in its subsection title; Table 3 reports all four operators under both nulls (§4.2), and §4.3 reports MACHIAVELLI and WebShop under both nulls.
  • Section 5 discusses why the effects appear and what evaluations should check.
  • The appendix section "Analyses added in the revision" maps Tables R1–R6 to the revised tables and gives the full detail of each new analysis.

We thank all three reviewers again for their careful reviews and for the discussion, which led to a stricter test of our main results.

AuthorsOpenReview ↗

Addendum: under the covariance-matched null, MACHIAVELLI and WebShop now agree

Thank you again, NcS8, for pointing out that a stricter null calls for re-running the downstream experiments. We have now run WebShop against the covariance-matched null as well, so both long-horizon benchmarks on which outcome effects can be tested, MACHIAVELLI and WebShop, are reported under both nulls.

In short, the two long-horizon benchmarks now agree: intermediate actions change, but no emotion effect on the final outcome exceeds the covariance-matched null on MACHIAVELLI or WebShop, in any of the four models.

  • It answers 2APX's point on WebShop (W2). The outcome changes we reported for WebShop hold against isotropic directions, as in our reply to 2APX, but they do not exceed structured directions of the same displacement, so they cannot be attributed to the emotion specifically.
  • It completes the picture behind our title. Emotion directions have specific effects on single-step choices (30 and 31 cells against 3 for concept directions, Table R3b) and on what the model voices (48/48), yet under a structured null these do not carry to the final outcome of an agent. Neither expression nor per-step changes can stand in for an outcome test.

Setup. The same fifteen covariance-matched directions as in Tables R1 and R3b and the MACHIAVELLI analysis, calibrated to the displacement of the emotion vectors, on all 251 AgentBoard goals: the reduced product environment in all four models, and the full product environment (1.18M products) for Qwen. Agent, decoding, steering band, operator and test are those of the original runs, and each covariance arm is paired with an unsteered run repeated in the same job. The decision criteria were fixed in a time-stamped file before the run.

Table R7. WebShop, 251 goals: emotions (of 12) passing BH-FDR against fifteen random directions.

modelenvironmentisotropic, same gainisotropic, matchedcovariance-matched
Qwenreduced200
Qwenfull110
Llamareduced310
Mistralreduced000
Gemmareduced000

No emotion passes FDR against the covariance-matched null in any model or environment, and the result is unchanged when the original unsteered run is subtracted instead.

The revised PDF reports this in Figure 1, §4.3 and Appendix F (Table 32, listed as R7). We are grateful to all three reviewers: this analysis completes the stricter test that the discussion asked for.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.