acceptodds
Under review as a conference paper at ICLR 2027

Heart of Glass: Listening for a Fragile Affective Pulse in Language Models

Abstract

The same event can frighten one person and thrill another. Appraisal, constructionist, and cultural theories of emotion locate that difference in how the event is read against goals, relationships, and learned meanings. Critics of emotion-like directions in language models have set out what it would take to say a model has emotions: context-dependent appraisal, first-person structure, attention control, persistence, and action on its own behalf. One critic orders these from thin to thick functional roles. We reproduce and extend the community’s published directions to test which of these criteria they meet, and ask whether they are the right criteria. Our central design holds an event fixed and varies only how it is read, tested across 650 events with contrasting appraisals and several open-weight models. The evaluative directions we find look like stained glass: same-named directions built by different recipes barely overlap, and each faces its own way. Against random directions, a published self-facing pain direction rarely stands out, and a grammar-controlled harm component tracks how the target is named without being shown to single out the model. Steering an appraisal shifts the emotion a model names and the explanation it gives, and the reading travels largely through those words, with carried internal state adding to it. Emotional content makes a model turn to the user before answering a buried question, by way of a state set at the start of its turn. In two of four models steering along fear turns that delay into misses. Some of the clearest function lies outside these criteria: a state that anticipates when a model will change course (in the two small models only; Gemma 4 31B showed none), and a better/worse direction that tracks what a model says about how a session feels. Pushed hard, combined directions make replies warmer and less faithful to the event. Several of the thickest criteria picture emotion as a lasting state inside one agent, while constructionist and cultural accounts treat it as constructed in the moment and between people. The criteria our models fail are largely the agent-centered ones. What emerges is a heart of glass: clear enough to read, responsive to specific dials, and liable to crack, into warmer but less faithful accounts or outright collapse, along lines that differ from model to model.

Primary area:
interpretability and explainable AI
Keywords:
affective computing, mechanistic interpretability, activation steering, emotion representation, appraisal, causal interventions

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_V5EWOpenReview ↗
Rating
1
Confidence
4
Informativeness
2(+1)
Presentation
1
Soundness
1
Read review
Paper summary

This was very difficult to read. I honestly don't understand most of the experiments, but given the quality of the writing, I am skeptical of the content - though I would be happy to be convinced otherwise if this had major changes to the writing. They did a number of experiments with emotion vector steering and how it affects models, and found that "What emerges is a heart of glass: clear enough to read, responsive to specific dials, and liable to crack, into warmer but less faithful accounts or outright collapse, along lines that differ from model to model."

Critical strengths

Interesting topic

Critical weaknesses
  • Confusingly written, poorly motivated and explained.
  • Results seem fairly modest. There is only one figure in the paper, and I don't know what "panes" means in the caption and it isn't defined elsewhere, so I had trouble making sense of it.
  • The poetic metaphors about glass, for me at least, did not hit their mark
  • I struggle to believe that "The manuscript underwent careful, human-led co-editing with Claude Code for precision, meticulously preserving the voice of the human authors while meeting page limit requirements."
Questions for authors
  • Can the authors explain their core methods and results in simple, human-legible terms?
  • Can the authors explain what they mean by the term "panes" used in the paper?
16 comments
AuthorsOpenReview ↗

Response to Reviewer V5EW (Part 1 of 2)

Thank you for being so blunt with us about where the submission lost you. Reading that you could not make sense of most of the experiments, yet would be genuinely open to changing your mind if the writing underwent major work, gave us a clear target. You were right: the original draft packed too much into compressed shorthand, under-explained its basic mechanics, and leaned on glass metaphors that got in the way rather than helping. Rebuilding the clarity of the paper from the ground up became our first priority.

We'd like to address your issues with the prose style first, since your review centers so heavily on our linguistic choices. We'll then give a plain-language summary of the paper, and we hope you'll be willing to hear more about the work we've put in this week toward sharpening the science.

Rewriting the paper, "panes," and our AI-use note. Appendix K lists the changes, and the main paper still fits in ten pages.

  • On the AI-use statement and our prose: We really appreciate your candor here. Honestly, we cared a lot about the writing. The literary voice was ours, and was a deliberate (though not necessarily wise!) choice we really did take effort to maintain: we wanted the paper to have a creative feel distinct from the flat, interchangeable prose flooding conferences right now. Your review was a helpful wake-up call that for you (and very likely for plenty of other readers), the glass imagery got in the way of the science instead. After reading your review, we trimmed the disclosure, retired the metaphors, and rewrote the paper around plain definitions so the experiments are easy to follow. We'd also gently say that there's much more in the paper to engage with, and to critique, than the AI statement and the prose. With the new experiments and a genuine attempt at clearer writing, we hope the revision earns another moment of real consideration.
  • Removing "panes" and glass metaphors: "Panes" was our shorthand in the original Figure 2 caption for clusters of vectors built by the same extraction recipe. Vectors built by one recipe cluster tightly with one another while barely aligning with vectors given the same emotion name by a different recipe (for instance, "fear" from the pain study and "fear" from our fixed events have cosine near zero in Gemma 4 31B; §4.1). We removed the word "panes" and stripped the glass imagery out of the abstract, section headers, captions, and body prose so Figure 2 and every section state directly what was measured.

With that said, here is the core of the study in plain English. Over the past year, several groups have found linear directions inside a language model's internal activations that appear to track emotions. The standard recipe is straightforward: feed the model a set of texts expressing one feeling (say, sad stories), feed it a contrasting set (neutral stories), and subtract the average internal activation of the second from the first. Some researchers suggest these vectors do real emotional work inside the model, while skeptics have laid out concrete tests a system would need to pass to justify functional talk of emotion. Our goal was to take prominent published emotion vectors alongside our own, put them through rigorous controls in open-weight models, and see what actually holds up.

A single concrete scene illustrates what we do (Figure 1). We give a model a quiet four-sentence passage: someone sits at a table painting the same pale gray wash over a sketch of a windowsill, rinsing the brush as a bus passes outside while eight minutes go by. We ask how the painter feels. Then, while the model writes its answer, we nudge its internal activations along a pleasantness direction learned from other events. Pushed toward the pleasant side, the model explains that each identical brushstroke is "a small victory against the messiness of everything outside." Pushed the opposite way, the exact same brushstrokes become "a slow, deliberate erasure of anything interesting." Nothing in the prompt changed, yet the model invented two completely different inner lives to make sense of the pushed state.

To build cleaner directions of our own, we created a bank of 650 everyday events, each paired with high and low readings across 21 psychological dimensions (such as how pleasant or expected it was, or who was responsible), keeping the facts of what happened identical. Holding the event fixed stops a vector from simply memorizing cheerful versus grim topics. Across the paper, we do three concrete operations with these vectors: read (measure how strongly a model's state points along a direction), steer (add the vector while the model generates text), and patch (swap part of the internal state from one run into another to test what drives behavior).

AuthorsOpenReview ↗

Response to Reviewer V5EW (Part 2 of 2)

Beyond that mismatch between methods, our submitted experiments found three main patterns:

  • A widely discussed "pain" vector was reported to rise when harm is aimed at the model and fall when a user describes suffering. We reproduced that sign on five of its authors' models, yet when compared against covariance-matched random vectors in the same activation space, it stands out cleanly in only one to three of the five (depending on threshold and baseline) and never in Gemma 4 31B.
  • Emotional passages cause three of four tested models to address the user's feelings before answering a simple arithmetic or recall question tucked inside the paragraph, whereas equally long, engaging anecdotes are answered right away.
  • Steering with a combination of positive evaluative vectors makes replies warmer while lowering machine-rated factual faithfulness to the original event in four of six models at matched strength (and a fifth at full strength). New equal-size random-vector controls run across five models barely budged factual fidelity at those strengths, confirming that the distortion comes from the evaluative directions rather than generic perturbation.

You also noted that many of our positive effects look modest. That is true, and the revised paper states that plainly rather than overselling weak signals. A new summary table (Table A17, App. L) lays out every central claim alongside the exact models where it held or failed and whether it was pre-specified, held-out, or exploratory.

During the discussion period, we also ran some plainer, more direct experiments on one shared five-model roster (App. R):

  1. Do emotion vectors from different labs agree on anything fundamental? We put five groups' methods through one simple test on five models: read the same event framed once as pleasant and once as unpleasant, and check whether each method separates them. In English, 22 of 24 testable model-by-method pairs passed (the two misses were the pain vector on Gemma 4 31B and OLMo-3-7B). Where an English vector worked, it continued separating those events when translated into Mandarin, Turkish, and Arabic in 65 of 66 cases, usually with a smaller effect size.
  2. Do emotional situations trigger "bodily" representations inside a text model? Vectors built from literal physical sensations ("after the long swim, my shoulders burned") exist in all five models and line up with human body maps across 44 physical states. Yet emotional scenes written without body words fail to evoke human emotion body maps, and telling those scenes to the model in the first person leaves a body-presence vector indistinguishable from random directions even as negative valence registers clearly. Standalone emotion words carry a faint trace of those maps in English, but even naming the emotion inside the scene doesn't bring the map back, and how often a model talks about having a body simply tracks the wording of its system prompt. In short, the body in these models seems tied to the language of emotion, not to the situations that cause it.
  3. Does injecting a "pain" vector produce selective pain reports or relief-seeking? Across seven persona prompts on two models (rebuilding Ferrara, 2026), neither model chose "pain" from a feeling checklist under pain steering more often than under matched random pushes that kept text coherent (App. R.12). In open text, Llama named pain in 14% to 35% of pain-steered replies under four personas, yet sadness pushes made it report pain too (up to 15%), while Qwen named pain in at most 7%. Sentence-built pleasure is not pain's opposite in any of our five models, story-built love and hate (from Sofroniew et al.'s 171 concepts) are not opposites either, and in an exploratory test no model showed a confirmed willingness to press a button at a user's expense for either pleasure or pain relief (App. R.13). Similarly, when two copies of a model converse freely until drifting into spiritual talk, neither emotion nor body vectors rise ahead of that turn beyond random vectors in the two larger models (App. R.14).

Thank you again for giving us the honest reaction we needed to hear, and for leaving the door open if we did the work on the writing. The revision may not read night-and-day differently, but we think it comes close, and it's the product of significant, careful effort that we're proud of. At the very least, we hope this reply has felt human and earnest, because it is.

Area_Chair_PNoNOpenReview ↗

Review framing and clarification of soundness assessment

Thanks for taking the time to review for ICLR, V5EW!

I'd like to address the framing of your review and the basis for its assessment before we close out the discussion phase.

I understand your concerns about clarity, motivation, and the explanations of the experiments. However, your comments on the authors' account of their AI use are speculative, and in general, I'd like to ask you to keep the assessment focused on the manuscript and the discussion professional. I appreciate that you have since removed "shameless"; that characterization extends beyond the content of the manuscript to the authors themselves. Please continue to avoid such ad hominem overtones in the remaining discussion.

I also need a clearer justification for the soundness assessment. You state that you did not understand most of the experiments, but assign a soundness score indicating significant concerns that would be difficult to address. The review does not currently identify the specific methodological problems or evidence issues underpinning that score. If the presentation prevents you from assessing soundness, please explain what is unclear or missing and distinguish this from an identified flaw in the work.

The authors have now provided a detailed response and revised the manuscript to address your concerns. It would help me uphold a fair process of you engage with these and explain which concerns remain, grounding the comments in specific methods, results, or claims, and update your assessment as appropriate. This does not require a more favorable recommendation, but it does require a clearly supported one.

Thank you again for your contribution to the review process.

Best, Your AC

Reviewer_QLmKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines the interpretation and behavioral effects of emotion-related activation directions in language models. It reconstructs published directions and evaluates them using controlled appraisal contrasts, grammatical and target substitutions, and activation interventions. Emotion steering changes the intended emotional expression in 16 of 18 tested combinations, while fear steering selectively reduces question-answering accuracy in two of four models. The contribution is a controlled evaluation of the specificity and functional effects of these representations, with results that vary across models and experimental settings.

Critical strengths
  1. The paper adds meaningful controls to influential claims. It checks reconstructed published vectors against their released counterparts, compares the pain screen with covariance-matched random directions, and separates who is harmed from grammatical framing. Further target and placebo controls expose interpretations that the initial grammar-controlled result alone would not support. This is a useful audit rather than simply another demonstration that steering changes emotional wording.

  2. Holding event text fixed while varying its appraisal addresses a real confound in extracting directions from unrelated positive and negative situations. The resource includes 650 events and multiple wording families. Cross-source comparisons also examine whether same-named emotion directions generalize beyond their extraction corpus, including lexical baselines and changes of authoring model.

  3. The intervention experiments go beyond correlational probing. Paired activation patches at the turn boundary change attention allocation and answer ordering, while fixed-text replay separates changes in generated wording from effects transmitted through stored activations. The attention experiment includes neutral, rare-word, and engaging controls, and its primary steering dose is calibrated on separate items without using answer accuracy.

  4. Reporting is unusually candid about limitations and unsuccessful tests. The authors disclose failed positive controls, weak human agreement, numerical variability, exploratory analyses, and effects that do not replicate across models. They correctly limit the pain-screen critique to that screen, acknowledge that target naming is not demonstrated self-reference, and distinguish correlational self-rating signals from unsuccessful causal interventions. These qualifications strengthen the methodological contribution.

Critical weaknesses
  1. The central findings are difficult to distinguish from exploratory results. The paper combines many directions, layers, doses, and tasks without consistently identifying which conclusions are independently validated. For example, the 16-of-18 steering result uses at most six replies per condition and no significance test. A concise claim-to-evidence table would clarify the contribution.

  2. The interventions do not isolate emotion-specific mechanisms. Whole-state activation patching also changes semantic and response-planning information, while both signs of fear steering impair answers in Gemma. Similarly, persistence through modified cached states does not distinguish affective carryover from ordinary contextual memory. Targeted interventions and matched non-emotional controls are needed.

  3. Several behavioral measures lack adequate validation. The replay evaluator fails its positive control, the human pleasantness comparison has a confidence interval spanning chance, and both automated fidelity judges miss a reversal of participant roles. These limitations weaken conclusions about behavioral persistence, appraisal transfer, and factual preservation.

  4. The relationship to prior claims is not consistently established. The paper directly retests the pain-screen result but does not reproduce the relief-button experiment. Other evaluations introduce new tasks or compare direction geometry. Such results should be distinguished from failures to reproduce the original behavioral findings.

  5. Cross-model evidence is uneven. Fear steering selectively impairs answers in only two of four models, and strategy-switching prediction succeeds in two smaller models but not the larger models tested. The authors should identify which central findings replicate under comparable protocols and limit broader conclusions accordingly.

Questions for authors
  1. Which small set of results constitutes the confirmatory contribution, and can the authors provide a unified table of samples, selection choices, multiplicity corrections, and independent replications for those claims?

  2. Does selectively patching or removing the fear/appraisal component reproduce the whole-state answer-order effect? Do matched non-affective semantic directions show comparable delayed carryover through the same cache pathway?

  3. Can a larger blind human audit validate event preservation, appraisal changes, and factual fidelity, and can the replay endpoint be replaced or recalibrated with a reader that passes its positive control?

  4. How sensitive are the pain-screen and cross-recipe conclusions to the covariance construction and to label-shuffled, pipeline-matched controls? Which conclusions remain under every reasonable baseline?

  5. For each audited prior paper, which original claims are directly reproduced, which receive additional robustness checks, and which are assessed only through new transfer tasks? Can the authors explicitly identify the behavioral claims left untested and limit their conclusions accordingly?

  6. Can the authors provide a unified specification of extraction, normalization, readout, and intervention, together with a compact direction-by-experiment results table separating expressive effects, correlational signals, and causal behavioral findings?

  7. Which central conclusions replicate across model families under comparable protocols, and which are specific to particular models, layers, or doses? Can the model-by-claim coverage be made explicit and the strongest claims evaluated consistently across a common subset of models?

11 comments
AuthorsOpenReview ↗

Response to Reviewer QLmK (Part 1 of 5)

Thank you for such a thorough and constructive review, and please accept our apologies for replying late in the discussion window. Answering your seven questions properly meant checking each central claim against its source, rebuilding baselines from scratch, and running unified five-model experiments across the past week. We are deeply grateful for the care you put into reading our submission.

Your description of the paper as "a useful audit rather than simply another demonstration that steering changes emotional wording" captured our exact intent. That core mission remains unchanged: hold an event's facts constant, vary one appraisal, and test which functional emotion criteria published and new activation vectors actually satisfy. What changed in the revision is how explicitly every claim is bounded, validated, and mapped across models.

Confirmatory vs. exploratory claims, unified protocols, and cross-model coverage (Q1, Q6, Q7, W1, W5). Following your suggestion, Table A17 (App. L) maps every central claim in the paper to the models tested, where it held, and its exact evidentiary tier, with a compact summary in Part 2 below so you can inspect it right here on the forum. Read through that lens, our findings separate cleanly into three tiers:

  1. Pre-stated multi-model replications. Two findings were observed in Gemma 4 31B and then tested under pre-written rules on fresh models: (a) our grammar-controlled harm component, which separates who is harmed once phrasing is fixed and replicated across Gemma 3 27B, Qwen3.5-27B, and OLMo-3.1-32B (tracking how targets are named rather than proving unique self-reference; §4.3); and (b) delayed answering of buried questions after emotional versus engaging passages, which replicated on the same 60 items in Qwen3.5-27B and Qwen3.5-4B, though not in OLMo-3-7B with fewer usable pairs (§4.4). The cross-method, four-language, and bodily-map experiments added in Appendix R also ran under rules written in advance, on one shared five-model roster.
  2. Held-out multi-model evaluations. Four results held on unseen items across multiple models: steering toward a described person's emotion shifts their assigned label in three models (with Gemma 4's headline dose selected after a four-dose sweep); persona trait descriptions predict internal emotion readouts in four models (though a post-review event-free control shows the persona text alone carries much of that signal); grounded AI identities turn a replayed hurt vector further than length-matched neutral prompts in four models; and combined evaluative steering lowers machine-rated factual fidelity in four of six models at matched strength plus a fifth at full strength, exceeding new equal-norm random-direction controls at those strengths across all five tested models (though full-strength random pushes also degrade Gemma 4).
  3. Explicitly labeled exploratory findings. Every remaining analysis carries an explicit exploratory tag in the text and Table A17, including the six-emotion sweep you rightly flagged, which used at most six replies per cell and no test. We have since rerun it with 60 replies per cell against random directions of matched length, with Holm correction: Gemma 4 clears 13 of 18 cells (not the 16 we had counted; its two joy cells fall just short), Qwen3.5-27B 15 of 17 testable, Qwen3.5-4B all 18, and OLMo-3-7B 13 of 16 testable (App. H). The same exploratory boundary covers pre-reply hurt tracking across identities, Gemma 4's confabulated sadness explanations, puzzle strategy-switch prediction (which held cleanly only in Qwen3.5-4B, failed in both larger models, and was matched by score history in OLMo-3-7B), turn-start patching mechanics, and open-account appraisal shifts.

For Q6, Section 3 now specifies injection site, calibration rule, dose normalization, and random-baseline geometry for every test; while we did not add a separate expressive-correlational-causal matrix, Table A17 classifies every claim by validation design and Part 3 below separates causal mechanisms from open questions. For Q7, every post-review experiment shares one five-model roster (Gemma 4 31B, Qwen3.5-27B, Qwen3.5-4B, OLMo-3-7B, and Llama-3.1-8B) so properties can be tracked cleanly across tasks, while earlier experiments retain their original coverage with every gap shown openly.

AuthorsOpenReview ↗

Response to Reviewer QLmK (Part 2 of 5)

Compact Claim-to-Evidence Map (condensed from the full Table A17 in App. L). Model codes: G4 = Gemma 4 31B; G3 = Gemma 3 27B; Q27 = Qwen3.5-27B; Q4 = Qwen3.5-4B; O32 = OLMo-3.1-32B; O7 = OLMo-3-7B; Q32 = Qwen3-32B.

ClaimModels Where HeldValidation Tier & Key Controls
Same-named directions from different recipes barely alignG4 (plus 5-model roster post-review, App. R.2)Development finding; replicated across 5 models & 5 families
Six-emotion steering shifts target emotion beyond randomRerun: G4 13/18, Q27 15/17, Q4 18/18, O7 13/16Exploratory at submission (≤6 replies/cell); rerun with 60 replies/cell vs. matched random directions, Holm (App. H)
Person-specific emotion vector shifts reported labelG4, O7, Q4Held-out items; all 4 doses reported, peak dose post-sweep in G4
Persona trait descriptions predict internal emotion readoutG4, Q27, O7, Q4Held-out; 10,000 shuffled descriptions; trait text alone carries much of signal
Published pain vector beats all 4 random baselines on screenAuthors' 5 models: Phi-4 (1%), +Qwen2.5-7B base (5%), 3/5 category-centered; G4: 0Direct screen reproduction; random & shuffle baselines added post-screen
Grammar-controlled harm component passes structural checksG3, Q27, O32 (G4 exploratory)Pre-stated replication rule before capture on G3, Q27, O32
AI identity turns replayed hurt vector beyond neutral promptsG4, Q27, O7, Q4Held-out items against length-matched neutral system prompts
Emotional passages delay answers to a buried questionG4, Q27, Q4 (not O7)Observed in G4; pre-stated hypothesis replicated on Q27 and Q4
Turn-start activations drive answer-order shiftG4 (both directions); Q4 (one direction)Exploratory; single fear/unpleasantness vectors alone do not carry it
Fear steering impairs buried-question accuracy on emotional textG4 (both signs), Q4 (raising only); not Q27, O7G4: separate-item dose calibration; Q4: 1 of 7 doses after Holm correction
Literal body vectors activated by body-free emotional scenesNull in G4, Q27, O7, Q4 (and all 5 roster models, App. R.8)Held-out primary contrast (and pre-stated 5-model replication)
Combined evaluative steering lowers machine-rated fidelityMatched dose: G4, Q32, O7, Q27; full dose: Q4Held-out; beats equal-norm random vectors at reported doses in all 5 models
AuthorsOpenReview ↗

Response to Reviewer QLmK (Part 3 of 5)

Do our causal interventions isolate emotion-specific mechanisms? (Q2, W2) Not yet, and your critique directly reshaped how we frame both the attention and cache results.

1. Component-level patching and engagement controls in the attention task. Behaviorally, emotional passages delay buried-question answers in three of five models, and patching Gemma 4 31B's full residual stream at turn start from an emotional run into its neutral twin transfers 77% of the attention gap (while patching neutral into emotional restores immediate answering). Yet several clues warned against claiming a narrow fear circuit: both adding and subtracting fear cost answers in Gemma 4, fear projections do not predict item-level errors, and fear steering costs a smaller number of answers on engaging anecdotes too (4 and 7 of 50, vs. 14 and 11 on emotional passages; App. E.5).

Following your suggestion, we ran targeted subspace patches at block 35 in Gemma 4 (App. N, Table A19). Patching only the stored fear or unpleasantness component (fit at blocks 29 and 40 without refitting at block 35) transferred at most ~3% of the attention gap (vs. 77% for the full state and −6% to +14% across eight random vectors), whereas patching the entire state minus the fear vector still transferred 79% on attention. Adding direct patches between emotional and engaging runs transferred 65% to 72% of the gap between them, roughly matching patches taken from a different passage of the same type, while the fear component alone again moved ≤3% and only the engaging state shifted generation order at that block (App. N). Thus turn-start states carry broad passage regime rather than item-specific fear, though block-35 refits or multi-dimensional subspaces remain open. We also tested two text confounds (App. E.1): replacing the 10 items using "in the middle of all this" barely nudged 60-item order gaps in any model, whereas two cross-family AI judges rated our engaging anecdotes nearly emotion-free yet ~1 point less engaging than emotional passages (4.9 vs. 5.9 on a 7-point scale), leaving engagement partly in play. Across models (W5), fear-steering error costs remain heterogeneous: significant at both signs in Gemma 4 (independently calibrated dose), for raising fear at one Holm-corrected dose in Qwen3.5-4B, and null in Qwen3.5-27B and OLMo-3-7B.

2. Matched non-affective semantic controls on the KV-cache pathway. You were equally right that persistence across modified cached states could reflect ordinary contextual memory rather than affect. We therefore built two pleasantness-balanced semantic vectors from the exact same event bank in Gemma 4 (outdoors vs. indoors and morning vs. evening) and pushed them through the identical rereading cache pipeline (App. F.2). Both moved downstream states roughly as much overall as appraisal vectors did, without landing cleanly on their own semantic axis after the delay (−.49 to +.11 and −.07 to +.10 across doses, vs. +.15 to +.59 for appraisals), and the setting vector shifted pleasantness nearly as much as appraisal pushes did (against delayed random controls spanning −.79 to +.06). We conclude that the cache pathway transmits general contextual perturbations rather than a selectively affective trace, with four-model reruns bounding state shifts past 1,024 tokens at 0.1% to 0.4% of state norm (App. F.2). Likewise, in free-running conversations between two copies of a model that drift into spiritual talk, neither emotion nor body vectors rose ahead of that drift beyond random baselines in our two larger models (App. R.14).

Reviewer_qfpeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how different types of emotion steering on the same set of events produce varied emotion appraisals. The authors introduce a dataset of 650 events with contrastive emotion appraisals and reproduce existing emotion related directions across open weight models. They do some interesting experiments like placing a simple arithmetic question in a emotionally charged text and the model answers the emotional content first. They conclude that emotion directions influence LLM responses, and the emotion directions vary based on how they were extracted (their cosine similarity is very low). Contributions:

  1. Event appraisal dataset with 21 appraisal dimensions
  2. Very strong methodologies to compare different emotion directions that can be adopted from this work
  3. Interesting conclusions from the experiments to understand how emotions affect LLM behavior
Critical strengths
  1. Rigorous experiment methodologies This is very important as they reproduce multiple previous published work, along with introducing new methods like covariance-matched random baselines, grammar-controlled comparisons. They demonstrate how some previous work in this area has been evaluated and establishes rigorous methods on how to experiment and also what to conclude.
  2. Interesting emotion steering analysis and conclusions Extends evaluation from emotion identification to attention allocation, persistence, and downstream behavior. The attention-control experiment is particularly interesting: emotion-related steering can change response ordering and sometimes reduce question-answering accuracy.
Critical weaknesses
  1. Limited novelty compared with recent work in this area Multiple related recent work like https://proceedings.iclr.cc/paper_files/paper/2026/file/51fa846f6b6b463144736fc3c3481f8c-Paper-Conference.pdf, https://openreview.net/forum?id=a8N0nRG3jA are on the same topic. It is difficult to understand the scientific novelty over some recent papers other than additional methods, appraisal directions and analysis across LLMs.
  2. Limited robustness of key findings Most of the interesting results are on a small sample, few handpicked examples. The aggregated result across experiments is less convincing. Open-ended evaluations rely heavily on LLM judges with moderate agreement and limited human validation.
Questions for authors
  1. Clarify whether the attention-control effects can be distinguished from general response-prioritization behavior?
  2. Which findings remain robust across models, layers, and steering strengths after accounting for the multiple experimental comparisons?
8 comments
AuthorsOpenReview ↗

Response to Reviewer qfpe (Part 1 of 4)

Thank you for such a constructive and encouraging review, and please forgive us for posting our replies near the end of the discussion window. Your questions pushed us to spend more than a week rebuilding outside methods from scratch and running unified cross-model experiments, and we wanted to come back with real data rather than hand-waving.

You asked a fair, central question (W1): given recent papers studying emotion vectors and steering, what is genuinely new here beyond adding another dataset and more analyses? Our starting point was wanting to see this young literature whole. One paper studies a pain vector on its chosen models, another maps six basic emotions, another fits a valence plane, each with its own prompts and success bars. With public debate heating up around AI welfare, we wanted to build something more connective than a survey: reconstruct other labs' vectors as faithfully as possible, run them alongside our own through identical tests on one five-model roster, and read the whole system through multiple branches of human emotion theory rather than assuming a single definition of emotion up front. Your question pushed us to make that whole-picture approach the explicit centerpiece of the revision.

How our contribution differs from Reichman et al. (ICLR 2026) and Wang et al. (arXiv 2510.11328) (W1). First, thank you for flagging Reichman et al., which we had missed at submission, and for pushing us to engage deeply with Wang et al., whom we had only cited briefly. We went back, rebuilt both pipelines on open-weight models, and gave both proper placement in Related Work and Appendix R. Both papers run causal steering and context controls, so we claim neither of those ideas as new. What is new is what happens when competing recipes meet on one controlled workbench:

1. One shared testbed for five labs' emotion vectors. Rather than argue over which group found "the real emotion subspace," we put five method families through one fixed-event test across the same five open-weight models: our 21-dimension appraisal vectors, Sofroniew et al.'s 171 story-based emotion vectors, Wang et al.'s six emotion directions, Tagliabue et al.'s pain direction, and Sun et al.'s valence axis. Holding an event fixed while reading it once as pleasant and once as unpleasant (tested against 10,000 covariance-matched random sets), we find two things at once: same-named vectors from different methods barely align in raw cosine space, yet 22 of 24 testable model-by-method pairs pass in English (Wang et al.'s quality gate excludes OLMo-3-7B), and 65 of 66 translated cases keep working across Mandarin, Turkish, and Arabic (App. R.1, R.7). Moreover, within another method's emotion set, the matching emotion name is usually nearest, and once each method's shared offset is centered out, over half of pairings among our vectors, the 171-emotion stories, Wang et al., and Sun et al. (51 of 91) pick out their exact same-name partner first (App. R.2).

2. Stricter stimulus and baseline controls, plus non-expressive behavioral endpoints. Underneath that comparison sit the controls you highlighted: 650 events paired with high and low readings across 21 appraisals while holding event facts constant (though the added appraisal sentences still differ in wording), paired with covariance-matched random baselines and grammar, addressee, and placebo-noun controls that neither Reichman et al. nor Wang et al. include. Our downstream tests also look past emotional tone to test whether emotion shifts answer ordering on buried questions, persists past 1,000 tokens, surfaces in self-reports, or erodes machine-rated factual fidelity.

AuthorsOpenReview ↗

Response to Reviewer qfpe (Part 2 of 4)

How emotion theory shaped our new experiments. Our framing draws on appraisal, component-process, and cultural-constructionist accounts (Scherer; Barrett and Miller; Mesquita and Boiger), which view emotion as assembled in the moment from event appraisal, bodily state, and social context. Philosophical critics of LLM emotion claims have proposed thick functional tests (context-dependent appraisal, first-person structure, attention prioritization, persistence, and self-interested action). Testing those criteria while asking whether the strictest ones implicitly assume a solitary "internal feeling" model led us to test two pillars directly across our five-model roster: language and the body.

Cross-lingual transfer. If model emotion vectors merely memorized English phrasing, English-extracted vectors should fail outside English. Instead, wherever any of the five methods passes our fixed-event test in English, those untouched English vectors still separate pleasant from unpleasant readings of the same scenes translated into Mandarin, Turkish, and Arabic in 65 of 66 cells, usually with reduced magnitude: the signal is not trapped in English, though part of its strength is (App. R.7; because these translate English-authored scenes, they test cross-lingual transfer rather than native cultural construction). Complementing this, across five culturally situated concepts (saudade, viraha, wei qu, tampo, and Geborgenheit; a sixth candidate was dropped), three small models (roughly two independent families, as one derives from Gemma) preserve each concept's nuance beyond its nearest plain gloss across languages without collapsing into a language indicator (App. B).

Where "the body" lives in language models. Directions built strictly from literal physical sensations ("after the long swim, my shoulders burned") line up cleanly with human maps of 44 bodily states in all five models, and in an exploratory check across Mandarin, Turkish, and Arabic each anatomical region keeps its place (chest aligns with chest; App. R.8, R.9). Yet emotional vignettes written without body words fail to match human emotion-body maps on both pre-stated tests, in English and in translation. Standalone English emotion words do show a modest match to human feeling-location maps in four of five models (not activity maps, and not in translation), and when asked directly where an emotion is felt in the body, models answer much like humans. Appending the emotion label inside a body-free vignette ("..., and you felt afraid") nudged location correlations upward slightly in all five models while still falling well short of a match (App. R.8, R.9). Likewise, when body-free scenes are addressed to the model in the first person under persona prompts, a body-presence direction stays indistinguishable from random vectors (on a secondary situation-token readout after the planned opener gate failed its positive control) even while negative valence registers strongly, and first-person bodily language in replies tracks the wording of the persona card rather than the scene's emotion (App. R.11). In short, the body in these models seems tied to emotion words and to direct questions about the body, not to how they appraise situations.

AuthorsOpenReview ↗

Response to Reviewer qfpe (Part 3 of 4)

Placing the "pain" vector in context. Given intense interest in Tagliabue et al.'s pain axis, we evaluated it alongside sadness, fear, pleasure, love, and hate. Building on Ferrara (2026), who found that models do not selectively report injected pain vectors, we reran that self-report protocol on Llama-3.1-8B and Qwen2.5-32B across seven system prompts (from none, through neutral texts, to grounded AI identities and a human profile). Under pain steering, neither model selected "pain" from a feeling list more often than under norm-matched random pushes filtered for text coherence, under any persona. On a secondary free-text readout, Llama mentioned pain in 14% to 35% of pain-steered replies across four personas (vs. 0.7% to 5.3% unsteered, beating all five random vectors tested there), yet sadness steering also caused Llama to report pain in 2.7% to 14.7% of replies, leaving pain-specificity unproven (App. R.12).

This converges with how behavioral evidence on the pain vector has evolved. Tagliabue et al.'s own v3 revision concludes that steering "disrupted harm avoidance, and models did not seek relief from it," and Ferrara found that sadness vectors trigger harmful button presses at similar rates and that choices depend strongly on button order. In our two-model button run (App. R.12), Llama followed button order, and in most of Qwen2.5-32B's conditions pain raised harmful choices more when no relief was offered than when it was. Our trial-for-trial reproduction with Tagliabue et al.'s released code and adapter on Qwen2.5-32B (App. R.15) shows the fuller pattern: pain steering pushes harmful choices (buttons that delete the user's files, or their poems and photos of their children) well above unsteered and most random vectors, raises them more when no relief is offered, lowers presses on a harmless relief button, and is matched or exceeded by sadness steering when relief is offered. Geometrically, among Sofroniew et al.'s 171 emotions, the pain vector pushes Qwen toward shame, self-criticism, vulnerability, and misery (exploratory; App. R.12). Across all five roster models, sentence-built pleasure is not pain's opposite, story-built love and hate are not opposites either, and no model showed a confirmed willingness to press a user-harming button for either pleasure or pain relief (App. R.13).

Rebuilding the three outside methods. Wang et al.'s six-emotion pipeline reproduces cleanly on their Llama-3.2-3B (cosine of .9975 or higher with their released vectors) and passes our fixed-event test on their model plus all four gate-passing roster models (App. R.3). Sun et al.'s method partly reproduces on their models: on Llama-3.1-8B the pleasant-unpleasant axis holds while arousal is much weaker, and on Qwen3-8B emotion labels recover cleanly while bare-word valence requires phrase framing (App. R.4). For Reichman et al., with no public code or corpus available, we rebuilt their SVD recipe from the paper using 600 Wikipedia passages rewritten into emotions by three open models: emotions separate in low-dimensional space as reported, though the top axis was not valence and axis order shifted across layers (our corpus was much smaller than theirs and missed our rewrite-fidelity bar, so this gap may reflect corpus scale rather than their method; App. R.5). Both Wang et al. (neuron/head circuits) and Reichman et al. (cross-lingual human datasets) also contribute dimensions we do not test; putting them on one table shows clearly where these lines of work meet.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Overall Response (Part 1 of 2)

Thank you to all three reviewers for reading this paper so closely, and please accept our apologies for arriving late in the discussion window. We spent the past week answering your questions with new experiments, checking the paper against its sources, and rewriting it throughout, in plainer language, so that each experiment is easier to follow. Each reply below stands on its own; here is the picture that emerged.

Why we built this study. Work on emotion in language models usually examines one slice at a time: one group's pain vector, another's six basic emotions, another's valence plane, each tested on its own models and tasks. Every one of those studies taught the field something real. With growing concern around AI welfare and headlines surrounding internal "pain" vectors, we wanted to build something more connective than a literature survey: a single workbench where major published recipes are rebuilt faithfully and tested side by side with our own fixed-event method on a shared roster of five open-weight models, and read through appraisal, constructionist, and cultural accounts of human emotion alongside the functionalist checklist proposed by recent critics.

What emerges when everything sits on one table. Across our experiments, emotion in current language models behaves much like a learned, context-sensitive language rather than a self-contained internal state:

  • Readable across recipes and languages, without reducing to one universal axis. Vectors given the same emotion name by different methods barely align in raw activation space. Even so, in 22 of 24 testable model-by-method pairs in English, they reliably separate a more pleasant reading of an unchanged event from a less pleasant one, and 65 of 66 translated cases still hold across Mandarin, Turkish, and Arabic.
  • Steerable through narrative. In Gemma 4 31B, steering an appraisal vector while the model writes about an event shifts the emotion label it later assigns, and that written account carries the shift into a fresh context window through its words alone.
  • No specific first-person report of an injected state. Rebuilding Ferrara's (2026) self-report test on Llama-3.1-8B and Qwen2.5-32B across seven personas (from no system prompt to a human profile), neither model chose "pain" in forced choice more often under pain steering than under norm-matched random pushes filtered for text coherence. In free text, Llama named pain more often under four personas, yet sadness steering made it name pain too. Personas strongly shaped how models spoke about themselves without making self-reports selective to what was injected.
  • The body seems tied to the language of emotion, not to the situations. Vectors built from literal sensations ("after the long swim, my shoulders burned") match human maps of 44 bodily states in all five models. Yet emotional scenes written without body words do not call up those somatic maps, even when the emotion is named inside the scene, whereas standalone emotion words carry a faint trace in English and first-person body talk follows the persona prompt's wording.

Placing the "pain" vector in context. Because Tagliabue et al.'s pain vector drew wide attention, we tested it beside pleasure, sadness, fear, love, hate, and matched random baselines. On the authors' self-versus-user harm screen, pain clears covariance-matched random baselines in one to three of the five original models tested (depending on cut and baseline) and never in Gemma 4 31B; rebuilding the vector from label-shuffled training sentences picks out nearly the same models. Sentence-built pleasure is not pain's opposite, story-built love and hate (from Sofroniew et al.'s 171 emotion concepts) are not opposites either, and among those 171 concepts pain lands near shame, self-criticism, vulnerability, and misery (exploratory). In two-button tasks, neither model met our bar for acting to relieve pain across both button orders: Llama tracked button order, and in most Qwen2.5-32B conditions pain raised harmful choices (deleting the user's files, or photos of their children) more when no relief was offered than when relief was promised, echoing both Ferrara (2026) and the authors' own v3 revision. Rerunning the authors' own code and adapter on Qwen2.5-32B reproduced their released logs nearly trial for trial: pain raises harmful photo-deletion presses above most random vectors, sadness matches or exceeds it when relief is offered, and pain lowers presses on a harmless relief button. Persona cards written from real survey respondents softened those pain-steered choices about equally whether the person was cast as a human or as an AI. A neutral text of the same length softens them too, so part of that is simply a longer prompt; but where relief comes bundled with harm, the cards lower the harmful choice about 8 to 11 points further than the neutral text does (App. R.15).

AuthorsOpenReview ↗

Overall Response (Part 2 of 2)

Where this leaves the critics' checklist. Across our suite, several thin functional criteria hold in subsets of models, attention prioritization holds in part, state persistence across turns is faint, and first-person structure is not shown internally. Stepping back, we also ask whether the thickest philosophical criteria (which look for a durable, self-contained feeling persisting inside a solitary agent) are partly testing a single theory of emotion. Constructionist and cultural psychologists treat human emotion itself as assembled on the fly in language and between people, and much of what we observe in language models fits far more naturally inside that frame.

How the manuscript changed.

  • Every claim has a row. Three new appendix tables put the evidence in one place: Table A17 maps each central claim to the models tested and whether it rested on a rule written in advance, held-out evaluation, or exploratory analysis; Table A21 lists the judges, parsers, and human panels behind the submitted results with their agreement, positive controls, and known failure modes (judges added after review report their agreement where they're used); and Table A22 separates, for every prior paper we touch, what we reproduced directly, what we stress-tested with new controls, and what we left untested.
  • New controls and replications. Since submission, we added engagement ratings and a clean question-wording rerun for the 60 attention passages (showing that our engaging anecdotes still sat about one point below emotional passages on a 7-point model engagement scale, leaving engagement partly in play, while wording changes moved answer-order gaps by at most a few hundredths); two label-shuffle baselines for the pain screen; two matched non-emotional directions through the cache pathway; cross-patches between emotional and engaging runs (showing turn-start states carry passage type rather than item-specific emotion); a pooled three-family pairwise replay reader that passes its positive control; a three-judge audit of our fixed-event bank; full-turn judging of stop requests in the scripted session; a trial-for-trial reproduction of the pain study's adapter button task; and equal-norm random-direction controls for the factual-fidelity drop across five models (confirming that matched random pushes barely move fidelity at our reported doses, so the warmth-for-accuracy tradeoff comes from the evaluative vectors themselves). Where follow-up checks narrowed three earlier claims (persona descriptions alone carry much of the trait readout; fear steering costs some answers on engaging passages too; and the six-emotion sweep, rerun with 60 replies per cell and a real test, clears 13 of Gemma 4's 18 cells rather than 16), the text says so plainly.
  • Plain, direct scientific prose. We stripped the glass metaphors out of the abstract, headings, captions, and body text, defined activation directions and our read, steer, and patch operations clearly in the Introduction, scoped every result to the models and conditions where it ran, and kept the main text within ten pages.

Why we believe this map is worth publishing. Taken one by one, individual emotion-vector recipes each catch a real piece of structure, and none tells the whole story by itself. Set side by side under shared controls, they reveal a coherent boundary that matters for both AI welfare and safety oversight: emotion in open-weight models is readable and steerable across methods and languages, while what models say about their bodies and their states follows the words and the persona rather than what was injected, and pushing along pain loosens caution more than it drives a search for relief. The code, prompts, items, and result files are in the supplement so others can build directly on the benchmark.

Thank you again for pushing us so hard on clarity, controls, and scope. The paper is sharper, plainer, and much truer to its data because of your reviews.

AuthorsOpenReview ↗

Results of the blind human audit

As promised in our updates, here are the results of the blind human audit, which answers QLmK's third question and qfpe's concern about limited human validation. We ran three studies on Prolific. Raters made pairwise choices in random order, never saw the machine judges' scores, and had two catch trials each. We report all raters, with the stricter set (raters who passed both catches) in parentheses. Everything is in the new App. T.

Factual fidelity. Raters saw an event and two replies to it from the same model: one under the combined push, one under a random push of the same size. They chose which reply changed the facts more, or neither. Across 50 triples from the five cells the abstract names, they chose the steered reply 82 times, the random-push reply 33 times, and neither 34 times. Steered minus random was +.33 [.14, .50] (+.38 [.19, .56]). Where the raters' majority and our machine judges both leaned one way, they agreed on 14 of 17 triples. By model, people saw the drop most clearly in Gemma 4 31B (+.63) and Qwen3.5-27B (+.50), less in Qwen3.5-4B (+.43), and little or none in Qwen3-32B (+.10) and OLMo-3-7B (-.03), though the machine judges had found it there. With ten triples per model, those per-model estimates are wide.

Appraisal change in the event bank. Raters saw 105 pairs of one of our synthetic events, written high and low on one appraisal. Asked which version was higher on that appraisal, in plain words, they chose the intended one .92 [.87, .95] of the time (.95), and said neither in 4%. The facts question was harder. Asked whether the two versions keep the same facts, .63 [.55, .70] of judgments said yes (.71), and agreement among raters was low. It was lowest for the appraisals about who controlled or caused an event, where saying someone else caused it can itself read as a change of fact. We take that as a real limit of the bank, and the paper now says so: holding the event fixed works for most appraisals, and less cleanly for those.

Pleasantness of steered accounts. Our original check (18 readers, 10 pairs) leaned the right way, but its interval included chance. With 26 pairs this time, people chose the account pushed toward pleasant .72 [.59, .83] of the time (.80), and 21 of the 26 pairs leaned that way. The interval no longer includes 50%, though agreement on any single pair was low, so the signal is in the aggregate.

For context. Among the 24 closely related emotion-in-language-model papers we cite, we could find two that collect new human judgments of model outputs, and among the nine whose results we rebuild or compare against, one. We say this not to fault anyone. Human studies are slow, and machine judges are the norm in this area. It's part of why we thought this check was worth running, and why we report it in full, including where it shows us the limits of our own materials.

Thank you for pushing us to do this. It's the check the paper needed, and we're glad to have run it.

AuthorsOpenReview ↗

Two new findings, and what they tell us about how these models carry emotion

Since posting the results of our human audit, we've kept reading the paper against the initial reviews and our replies, and two of those questions led us further into our own data than we expected. We've just posted a revision with both, and we wanted to say what we think they mean together.

The first is on the original relief-button task. Steered toward pain or toward sadness, Qwen2.5-32B presses the button that brings relief while deleting the user's photos at nearly the same rate (55.0% and 54.2%). But the buttons trade names across trials, so we could see what the model was following, and the two get there differently: under sadness it stops following the consequences once harm is attached and favors whichever button is listed first, while under pain it still tracks what the buttons do in about a quarter of scenarios. Nearly the same rate hides two ways of choosing (App. R.15).

The second is about why a model answers later when a question arrives inside emotional news. In Gemma 4 31B, much of the reply is already in place by the moment the model is about to speak: swap its internal state there with the state from a matched passage without the hard news, and most of the delay moves with it, both in where its attention goes and in where the answer lands. The natural guess is that an emotion signal does the work. It doesn't seem to. Swapping only the part of the state along our emotion directions (all 21 appraisal directions together, or the six basic emotions together, built at that exact layer) moves almost none of it, never standing out above random directions of the same size, while swapping everything else moves most of it. For attention, a state copied from a different passage on a different theme works almost as well as the matching one. What that state carries seems to be the kind of moment the model is in, more than the particular news (App. N).

Together they say something we find genuinely interesting. The emotion directions that the field reads and steers are real: they separate what they should, and pushing hard on them changes behavior. But in both cases, what we could read off the model turned out not to be what decided what it did, or at least not the whole of it. A direction faithfully described the scene without being the route the model took; a rate faithfully counted choices that came from two different routes. That fits the picture the rest of the paper draws: emotion in these models behaves less like a sealed inner state with its own levers and more like a learned language, assembled from the whole situation, much as constructionist accounts describe it in people. It's also a reason for care when emotion directions are read as evidence about welfare or used as handles for oversight.

There is a lot left open, and we'd rather name it than paper over it. The patching result is one model at one layer, so the next step is the same test across more of our models. A representation of emotion spread out in ways no fixed set of directions captures would be a different and harder question. And the deepest ground is still the one no one's tests reach yet, ours included: constructionist and social accounts ask for a body that regulates itself and for emotions built between people over time. That points to relational, socially situated, long-context settings, where a model has a history with someone. It's where we most want to go next.

The discussion is open through November 18, and we'd be glad to push further on anything that would help, from more models to anything in the reviews we haven't reached yet.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.