acceptodds
Withdrawn from ICLR 2027

Nine Emotion Anchors Recover a Valence Direction: No-Label Cross-Modal Transport and Scale-Aware Steering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Inside a modern language model sits an internal direction that tracks how positive or negative a sentence feels. We recover this valence axis (V-axis) from nine generic emotion-category anchors – no downstream sentiment labels – and follow it through four questions: can we recover it cheaply, is it broadly readable, does readable imply causally writable, and where does it fail? Recover: the recipe averages ∼50 emotion-anchored paragraphs per category into nine centroids and takes their top principal component (∼ 384× fewer labels than a supervised SST-2 probe: n=18 supervision events vs. 6,920). At matched budget the nine anchors – not the specific reader – carry the signal: difference-of-means, CAV, and LDA readers of the same anchors match or beat the principal-component recipe, all clearing a 500-direction random null on 8/9 models. Read: the axis is stable when explicit emotion words are stripped from the prompts (cosine 0.98 between original and word-removed axes), reads four disjoint sentiment corpora above null (SST-2, IMDB, Rotten-Tomatoes, Yelp), and appears in a self-supervised vision encoder that never saw language (DINOv2, valence AUC 0.790 vs. CLIP control 0.972). Write: readable does not mean freely writable. Removing the single axis leaves a retrained probe intact – valence spans a 4–36-dimensional redundant subspace – yet the model still causally uses it at inference time: directional ablation (Arditi et al., 2024) of the axis drops the model’s own sentiment output on 6/9 families while a norm-matched random-direction ablation moves it ≈0. Steerability follows a residual-stream-norm regularity that predicts held-out models (leave-one-model- out R2=0.88), and an explicitly-defined donor→receiver map transports the axis across models above a random-map null (3 model pairs, all 6 off-diagonal cells, 20–31×). Bound: the recipe is at-or-near chance on categorical concepts (AxBench Concept-500 (Wu et al., 2025), KS p=0.21) and on arousal, and on EEG valence is decodable but its geometry is not shared with machine encoders – a boundary case, not a fourth shared modality; a geometric “admissibility” criterion we tested does not predict held-out success and is reported only as a negative. Valence is thus the rare high-level attribute recoverable from category anchors alone that also transports without target labels across four independently-trained encoders and stays causally writable under a scale-aware fix – a reach that is real but, we show, bounded by residual-stream scale and cross-encoder geometry.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM, Emotion

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_AmmsOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a method for extracting a “valence axis” (V-axis) from foundation-model representations: category centroids are built from roughly 50 generated paragraphs for each of 9 emotion categories, and the first principal component of these centroids is taken as the axis.

Critical strengths

Whether simple, low-cost concept directions are shared across different encoders is a question of genuine research value.

Critical weaknesses
  1. The manuscript contains numerous unprofessional formatting artifacts. Nearly every table, figure, and paragraph carries an internal repository path (e.g., “Source: experiments/d32_steering_generalization/results/…”), the naming scheme is inconsistent (d8, d24, d39, d_joint_modality, E1–E9, E_M3), and most of the referenced experiments are never described in the text.
  2. The paper relies heavily on defensive phrasing (“honest null,” “honestly-bounded,” “the honest reading,” “lawful, not anecdotal”), and nearly every sentence is packed with numbers.
  3. Overall, the paper reads more like an accumulation of experiment logs than a curated argument, and it lacks clear, readable visualizations.
Questions for authors

I have concerns about the completeness of this work.

Reviewer_A6BQOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes a cheap way to find a valence direction (positive versus negative) inside a frozen model without sentiment labels. The authors write short stories for nine emotion categories, average the model’s hidden states for each category, and take the top principal component of the nine averages as the valence axis. They show that this axis reads sentiment almost as well as a supervised probe, can also be built in image, audio and EEG encoders, affects the model’s own sentiment predictions when removed, and can steer the model once the steering strength is scaled to the size of its hidden states. The supplementary material adds a separate study showing that different LLMs largely share one valence direction.

Critical strengths
  • The recipe is simple and cheap, and it is tested against many baselines, including random directions and other ways of reading the same emotion anchors.
  • The paper reports its negative results openly, including that PCA is not the best reader of the anchors and that the method fails on categorical concepts and arousal.
  • The steering analysis gives a useful finding. Steering fails on some models because their hidden states are much larger, and scaling the steering strength to this size fixes it.
Critical weaknesses
  • The EEG results are inconsistent. The main text reports AUC 0.840, while Figures 9 and 14 and Appendix M report 0.968 from a split the paper itself calls optimistic. Appendix H says Table 4 uses a supervised LDA for EEG, while Appendix M says it uses the anchor-guided axis. Figure 9 shows 0.51 for EEG where Table 4 shows 0.828, and Figure 13 shows 0.720 where its caption says 0.799.
  • The modality claims contradict each other. The abstract says one valence axis transports across four encoders, and also says EEG is not a shared modality.
  • The layer for each model is chosen on SST-2 dev, which is also the set used to report text results (Appendix B). A fixed middle layer still works for standard models, but the reasoning-distilled models depend on the selected layer.
  • The causal evidence is weaker than presented. A retrained probe almost fully recovers after the axis is removed, the drop in the model’s own sentiment predictions is small and weak or borderline on three of nine models, and cross-model steering is shown on only three models.
  • I think the novelty is limited. Prior work already found a linear sentiment direction in LLMs with causal ablations Tigges et al., 2024, which is not cited, and the paper itself cites recent work on a valence subspace in LLMs (Sun et al., 2026; Choi & Weber, 2026).
Questions for authors
  • Section 5 presents a cross-modal classifier that transfers from text to image, audio and EEG. Since a logistic head on a single score cannot change AUC, what does this classifier show beyond each modality’s own axis, given that every column of Table 4 has the same value regardless of the source modality?
  • The abstract and Section 2 state that the axis is built without target labels in every modality. How are the image and audio centroids built in Section 3, and how is valence defined for ESC-50, which has no emotion labels?
  • Section 3 reports several EEG results (0.840, 0.811 and 0.968) with different splits. Which EEG direction and which split are used for the text-to-EEG result of 0.828 in Section 5?
Reviewer_zwzsOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work identifies valence axes in language and other encoder models’ hidden representations. In contrast to work that uses labeled sentiment data or other supervision, the experiments instead use a set of AI-generated, emotion-evoking stories and unsupervised methods (e.g., the first principal component) of identifying a valence axis. Four sets of experiments are conducted. First, valence is broadly recoverable from the first principal component among other methods, and the researchers identify the valence axis aligning across sentiment datasets and in multiple modalities (i.e., vision, audio, and to a lesser extent, EEG). Through ablating the valence axis and retraining, they also find that valence remains recoverable after removing such primary axes.

Critical strengths
  1. Multiple Modalities. The experiments study a valence direction in encoder representations across multiple modalities, including EEG which is quite novel and interesting.
  2. Broad Experiment Conditions. The experiments cover a variety of models of different sizes, families, and modalities, supporting more generalizable results and that valence axes are largely independent of these factors.
Critical weaknesses
  1. Methods & Findings Clarity. Several parts of the paper are imprecise, underspecified, or unclear. For example, phrases like “the nine-anchor PC1 is a wash” (line 91), “matched-budget bake-off” (Table 1), and “interior optimum and monotone dose-response” (line 384) are unclear and make parts of the methodology and findings difficult to understand. Furthermore, methodological details are excluded or not referenced if present in Appendices (e.g., the “per-emotion lexicon” (282), the “geometric admissibility criterion” (lines 120, 130)).
  2. Unvalidated AI-Generated Data. The emotion narratives that serve as the stimuli across all text-based experiments are produced by generative AI (lines 491-493) with no apparent validation. The narratives should be validated with human judgments (i.e., reflect the intended emotion, are naturalistic, etc.) given that they underlie central claims, and the use of generative AI to produce the emotion-evoking stories should be reported in the methodology rather than the AI use statement alone; “we author” (line 156) in the Methods section implies they were written by the researchers which does not appear to be true.
Questions for authors
  1. Why do you choose the nine FACED emotion categories? The citations when introduced are to Russel—which does not specify a basic emotion set—and Ekman—which identifies a basic 6/7 emotion set—and are not clearly relevant. Instead, FACED appears to be based on the EEG emotion-evoking work cited elsewhere, and its application to language should be motivated.
  2. Can the authors please elaborate on the motivation for the nine-emotion anchor approach? One of the main motivating claims appears to be that a single sentiment direction “forces a sign and conflates valence (pleasant/unpleasant) with arousal (calm/intense)” (line 88) which includes no references to substantiate it and it does not appear to be supported by any conducted experiments.
  3. Given that the stimuli are a set of textual, AI-generated stories, how is the construction “identical” for the audio and visual modalities? What data is used for these encoders?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.