acceptodds
Under review as a conference paper at ICLR 2027

How Do Internal Affective Representations Relate to Self-Report in LLMs?

Abstract

Large language models (LLMs) are increasingly used in affect-sensitive settings such as psychological support, where sensitivity to human affect may need to be preserved while potentially misleading self-referential affective expression is selectively regulated. We investigate whether two such capabilities—predicting the affective response that a text is likely to evoke in a human reader (Reader Prediction) and producing an affective Self-Report for the same text—rely on shared internal representations and causal mechanisms. Using a matched-stimulus design across four model families, we compare the two tasks at the levels of behavioral correspondence, representation, and causal utilization, and examine differences between Base and Instruct models and across generation stages. Reader Prediction and Self-Report show strong behavioral coupling, and affect-relevant information is decodable from both tasks; however, evidence for shared representations and cross-task causal utilization is limited, indicating that behavioral correspondence and decodability do not imply a shared mechanism. Base–Instruct comparisons further reveal differences in representational organization and causal utilization, while layer- and generation-stage analyses provide limited evidence for robust endogenous control of Self-Report by decodable affect-relevant information. Overall, our results distinguish behavioral correspondence, representational sharing, and causal utilization as separate levels of evidence, suggesting that affect-related tasks may share structured information without utilizing it in the same way. This distinction provides a mechanistic basis for selective interventions that preserve human-affect prediction while regulating self-referential affective expression.

Primary area:
causal reasoning
Keywords:
Large Language Models, Hidden states, Mechanistic Interpretability, Causal Intervention, Post-training, Representation–Behavior Coupling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Vt8rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper compares models' predictions of a human reader's affect with their own prompted numerical affective reports. It studies four small model families using probing, representational alignment, and activation interventions. Reader and Self outputs are strongly correlated, and affect-related information is decodable in both tasks. However, the tested interventions do not establish reliable cross-task transfer or robust control of self-report. The state-induction gate fails for both valence and arousal, and the later layer/token-position analyses are presented as exploratory.

Critical strengths
  1. The distinction between output similarity, decodability, and causal use is worth studying. The experiments use matched stimuli with separate prompts, held-out probes, random-donor controls, and discovery/confirmation splits. These controls are useful because a correlation between Reader and Self outputs would otherwise be easy to overinterpret.
  2. The authors are open about unsuccessful interventions and variation across models. They report the failed gate, label subsequent analyses as exploratory, and do not equate numerical self-reports with subjective experience. The matched-plain Base/Instruct comparison also addresses an obvious prompt-format confound.
Critical weaknesses
  1. I am concerned that the output scoring could make intervention effects look artificially small. Appendix C.8.1 averages token log-probabilities and then applies softmax over JSON candidates. This is not the usual probability distribution conditioned on that candidate set. With equal candidate length m, it is equivalent to using temperature m on the summed sequence scores, which flattens the distribution. V1 effects are often around 10^-3 on a 1–9 scale (Tables 21–22), while the separate V3 analysis uses fixed .1 and .05 thresholds for sufficiency and specificity. These stages have different candidate spaces, so the numbers should not be directly equated; the concern is that the same scoring transformation is used throughout. Results under alternative scoring and calibration against natural stimulus effects are needed before interpreting failed thresholds as weak causal control.
  2. The negative results leave several mechanistic explanations open. Raw cross-task patching can fail because of coordinate differences, task offsets, or off-manifold interventions. Table 19 itself shows very poor direct cross-decoding and improvement after alignment. A failed intervention therefore establishes a limit of that intervention, not necessarily an intermediate mechanism between fully shared and fully independent processing. The discussion sometimes makes this stronger inference despite acknowledging the alternatives elsewhere. The generation-stage result is also measured along a fixed, teacher-forced neutral JSON response. It does not yet establish a mechanism that can support selective control in natural dialogue.
Questions for authors
  1. Do the main behavioral and intervention conclusions persist using summed sequence log-likelihoods, several softmax temperatures, and natural/free-generation readouts? Please report effect sizes relative to the clean Clinical–Neutral shifts as well as absolute thresholds.
  2. Can you show calibration with a positive-control intervention that reliably changes the measured affect score, and report all gate estimates and confidence intervals? The present Appendix Q contains no results table, making the gate and replication claims difficult to audit.
  3. Does cross-task causal transfer improve after training-only alignment or task-offset correction? Such results would help distinguish a coordinate mismatch from a lack of shared causal information.
  4. Please reconcile the direction of the H1 replication criterion: Appendix P.49 prints confidence bounds greater than zero for causal-minus-decoding depth, while the main text describes replication of Qwen's negative depth difference. Which signed statistic was actually tested?
6 comments

We thank the reviewer for the careful review and important comments. In particular, we re-examined the effect of the scoring procedure on estimated intervention effects, alternative explanations for the negative results, the lack of a positive control, and the replication criterion for H1. Below, we present the additional analyses and planned revisions.


W1. Shrinkage of effects due to scoring and fixed thresholds

The reviewer's point is mathematically correct.

Within each model and candidate space, all candidates have equal token length, and the softmax over mean log-likelihoods is equivalent to applying temperature T=mT = m to summed sequence log-likelihoods. The candidate length mm is 10, 12, 15, or 17 depending on the model family and candidate space, so there is the additional issue that the effective temperature differs across model families.

For Qwen Base with the 729-candidate space, the mean entropy of the distribution obtained from mean log-likelihoods was 6.590 nats, very close to the 6.592 nats of the uniform distribution. The original scoring therefore substantially flattens the candidate distribution.

In additional sensitivity analyses, the Reader–Self change correlations for Qwen Base (Valence/Arousal) were 0.809/0.866 in the main analysis, 0.880/0.911 under summed sequence log-likelihood scoring, and 0.441/0.442 under mode-based scoring. The sign of the correlations was preserved and none of the 95% confidence intervals included zero, but the effect sizes depended on the scoring method.

In addition, partial correlations controlling for Writer changes were substantially lower, so these results cannot demonstrate sharing of task-specific affective processing.

Intervention effects should likewise not be interpreted on the absolute scale alone. The median ratio of V1 intervention effects to the natural Clinical–Neutral changes was 0.039–0.329. In contrast, the point estimate of Sufficiency for the V3 Gate was 1.8% (Valence) and 0.2% (Arousal) of the natural displacement. However, because V1 uses a 729-candidate space and V3 an 81-candidate space, their absolute values cannot be compared directly.

Furthermore, the V3 Sufficiency threshold of 0.1 was larger than the natural displacement of the target model, 0.072 (Valence) and 0.085 (Arousal), and was thus not properly calibrated.

To examine this issue, we evaluated 96 threshold settings under the original scoring, and no setting yielded GO. We also re-evaluated the Gate for Qwen2.5-1.5B-Instruct using 12 alternative scoring methods, and all were NO_GO for both Valence and Arousal.

However, in the alternative-scoring analyses, the intervention directions were fixed to those estimated under the original scoring; re-estimating the direction for each score has not been tested. In addition, all of these are post-hoc sensitivity analyses and do not replace the confirmatory results of the main analysis.

In the revised version, we will clearly describe the flattening caused by the scoring procedure, the effective temperature, comparisons against natural displacement, and the distinction between threshold sensitivity and scoring sensitivity. We retain the Gate's NO_GO as a decision under the specified measurement and intervention conditions, but do not interpret it as implying that causal control is impossible in general.


W2. Alternative explanations for negative results and teacher forcing

We agree with the reviewer. A failed intervention primarily indicates the limits of that intervention and measurement procedure, and does not directly prove the absence of a shared mechanism.

Because E4 injects Clinical–Neutral differences, mean offsets between tasks cancel by construction. The cosine similarity between the Reader-derived injection vectors and Self's own differences was 0.571–0.957 for matched stimuli and 0.069–0.143 for random stimuli.

On the other hand, the possibility of off-manifold interventions remains. In a simple diagnostic using Mahalanobis distance, the proportion of intervened states exceeding the 95th percentile of real Self states was 34% for Qwen Base and 67% for Gemma Instruct. For the random control, it was even higher at 69–92%, which introduces the additional limitation that the degree of off-manifold deviation differs between the conditions being compared. This diagnostic also depends on shrinkage estimation of the covariance.

More importantly, even the within-task Self→Self positive control produced effects only on the order of 10−310^{-3}. This suggests that the E4 measurement procedure may have lacked the sensitivity to detect causal transfer at the tested intervention site.

Accordingly, we withdraw the interpretation that the results "support an intermediate mechanism between fully shared and fully independent processing." What we can currently show is only that the predictions of a simple shared model were not supported under the tested intervention conditions.

We also withdraw the statement, based solely on the V2 alignment results, that the differences "cannot be explained by linear coordinate mismatch."

Regarding teacher forcing, our generation-stage analyses were performed on a fixed neutral JSON sequence. Up to pre_V for Valence, the prefix is shared across all candidates, but Arousal and later generation stages depend on the preceding tokens.

In the revised version, we will position these results as exploratory findings conditional on the specific response format and generation positions, and will not claim to have demonstrated selective causal control in natural dialogue.


Q1. Do the main conclusions persist under alternative scoring?

At present, we cannot claim that all conclusions persist under every scoring method.

For behavioral coupling, we re-analyzed 4 of the 8 models with alternative scoring. Under summed sequence log-likelihood scoring, positive Reader–Self change correlations were observed for each of Valence, Arousal, and Dominance.

However, partial correlations after controlling for Writer shrank to 0.09–0.47, and in some conditions the confidence intervals included zero. In addition, the mode is discrete and carries little information, and in some conditions the correlation could not be computed.

For Qwen Base, the magnitude of the correlation coefficients differed substantially between the original scoring and the mode. Invariance of effect sizes across scoring methods has therefore not been demonstrated.

For the V3 Gate, all 12 alternative scores evaluated on Qwen2.5-1.5B-Instruct yielded NO_GO. However, the direction vectors were fixed to those estimated under the original scoring.

Effects relative to the natural stimulus changes were 0.039–0.329 for V1 and 1.8% (Valence) and 0.2% (Arousal) for V3 Gate Sufficiency. These are results under the original scoring; relative intervention effects under alternative scoring have not been verified.

Measurement under free generation, formal scoring comparisons for the remaining 4 models, and re-estimation of direction vectors based on alternative scores have also not been completed.

In the revised version, we will distinguish verified results from unverified scope, retain the main-analysis decisions based on absolute thresholds, and add the post-hoc sensitivity analyses.


Q2. Positive control and Gate estimates/confidence intervals

As the reviewer points out, Appendix Q in the submitted version was missing the necessary quantitative results. In the revised version, we will restore all estimates, 95% confidence intervals, thresholds, and decisions.

Results for Qwen2.5-1.5B-Instruct, L14, prompt_end, and 58 test stimuli are as follows:

MetricValence (95% CI)Arousal (95% CI)
Sufficiency γ\gamma0.00129 [0.00043, 0.00212]0.00017 [−0.00005, 0.00039]
Specificity SS−0.00147 [−0.00248, −0.00044]−0.00011 [−0.00042, 0.00022]
Endogenous AR−0.0075 [−0.0366, 0.0216]0.0082 [0.0013, 0.0162]
Topic TVD0.0036 [0.0031, 0.0043]0.0039 [0.0033, 0.0045]

The prespecified decision criteria were: the lower bound of the 95% CI for Sufficiency > 0.1, Specificity > 0.05, Endogenous AR > 0.05, and the upper bound of the 95% CI for Topic TVD < 0.15.

Under these criteria, both Valence and Arousal were NO_GO. In particular, the lower bound of the confidence interval for Specificity was negative on both axes, so the decision would not change even if the threshold were lowered to 0.

On the other hand, the submitted version did not include a sufficient positive control to verify measurement sensitivity. Calibration via full replacement with Clinical activations or stronger interventions has not been performed.

Therefore, the current NO_GO means that the criteria were not met at the specified site, intervention method, and scoring procedure. It does not guarantee that the measurement procedure can detect every valid causal intervention.

In the revised version, we will restore Appendix Q and state this limitation explicitly.


Q3. Does cross-task causal transfer improve with alignment or offset correction?

At present, we have not demonstrated that alignment or offset correction improves cross-task causal transfer.

Because E4 injects difference vectors, mean offsets between tasks cancel. In addition, vector directions after Procrustes alignment computed from the existing representations did not differ substantially from those before alignment.

However, these results do not test causal transfer after alignment itself.

As the reviewer suggests, an experiment that learns the alignment on the Discovery data, performs absolute-state replacement including offset correction, and evaluates within-task replacement as a positive control has not been conducted.

Therefore, we cannot currently conclude to what extent coordinate-system mismatch contributed to the failure of causal transfer. In the revised version, we will revise the text so as not to draw strong mechanistic conclusions from the failure of direct transfer.


Q4. Sign of the H1 replication criterion

We thank the reviewer for pointing out this important inconsistency.

In the actual analysis, we defined the difference between the causal peak depth and the decoding peak depth as Δ=dC∗−dD∗\Delta = d_C^* - d_D^*, multiplied it by a sign ss chosen to match the direction observed in Qwen, and used as the criterion that the lower bound of the 95% confidence interval of sΔs\Delta exceeds zero.

For Valence, s=−1s = -1 for both peak and COM; for Arousal, s=−1s = -1 for peak and s=+1s = +1 for COM.

However, Appendix P.49 retained the criterion before sign alignment. This is a descriptive inconsistency.

More importantly, the sign-aligned criterion was introduced after the replication results for the other model families had been computed, and was not fixed in advance.

The actual results are as follows:

  • Under the sign-aligned criterion, Llama and OLMo passed for Valence, while Gemma failed. For Arousal, all three families failed.
  • Applying the criterion as written in Appendix P.49, all three families failed.
  • The Valence passes for Llama and OLMo arise from causal peaks located at layer 0, which is qualitatively different from the peak at relative depths of approximately 0.74–0.78 observed in Qwen.

Therefore, we do not claim that the depth dissociation observed in Qwen was replicated in the other model families.

In the revised version, we will correct the test statistic and the treatment of the sign to match the implementation, and report results under both criteria. We will also state when the criterion was introduced, and revise the statements about replication to partial, exploratory findings.

Reviewer_ncUVOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors investigate whether LLMs share the same internal representations and causal mechanisms when predicting human emotional responses (Reader Prediction) and describing their own apparent emotional responses (Self-Report). They also examine the behavioral correspondence between these two tasks and test whether their relationship varies between Base and Instruct models to understand post-training-associated differences.

They find that Reader Prediction and Self-Report covary under controlled stimulus manipulations, and affect-relevant information is decodable from both tasks. However, evidence for shared representations and cross-task causal utilization is limited. The Base–Instruct comparisons further reveal differences in representational organization and causal utilization, while the generation-stage analyses suggest that decodability does not necessarily imply robust endogenous causal control.

Critical strengths
  1. The experiments are comprehensive in scope. The authors propose a multi-level experimental design and test various matched stimuli and model families.
  2. The motivation for this topic is compelling, as it addresses an important underlying question about how LLMs represent and express emotions compared to humans. However, the experiments do not directly provide insights into this broader question.
  3. The authors acknowledge inconclusive results and appropriately discuss the limitations of their findings.
Critical weaknesses
  1. The paper has no figures or tables presenting detailed result statistics in the main text. Almost all tables referenced in the Results section are located in the appendix, making the findings difficult to follow and comprehend. Many terms used throughout the paper are also confusing or insufficiently explained. For example, what does "Causally Interchangeable Code" mean? What does the statement “the Base–Instruct difference is more consistent with Information Remains Decodable + Its Organization Differs than with a simple account in which Information Is Erased” mean? Furthermore, shorthand terms such as “Erasure,” “Uniform Relocation,” “Uniform Reader–Self Separation,” and “Single Recoverable Bottleneck” are not clearly introduced in the Methods section. Although some of the terms might be clarified in the appendix, it's still hard to follow given that there are over 100 pages of supplemental materials to navigate.

  2. The datasets and task types evaluated in this paper are limited. The limited diversity of datasets and emotional situations in this paper raises concerns about the generalizability of its findings.

  3. The authors describe their research questions as “progressively more mechanistic,” but the motivation for the Base vs. Instruct experiment remains unclear to me. Why do the authors introduce the hypotheses of Erasure, Relocation, Reader–Self Separation, and Recoverable Bottleneck? The paper does not sufficiently explain why instruction tuning would be expected to remove affect-relevant information or how ruling out this hypothesis directly advances the central research question concerning shared mechanisms between Reader Prediction and Self-Report. Furthermore, what exactly do the findings reveal about how the two mechanisms differ? The paper does not provide a complete characterization of these differences.

Questions for authors
  1. There has been extensive prior work on affective representations and emotion representations in LLMs. What are the key novel insights provided by this paper, particularly regarding the relationship between Reader Prediction and Self-Report? How is this task different/interesting, and how do these findings advance our understanding beyond existing studies? Connection between those would make the scope and space of the research more clear.

  2. The word choice of Self-Report, although I understand it is used as an operational term here, still carries strong anthropomorphism of models in this setting. There might be better word choices to explain the idea without the potential inferring that model subjectively understands emotion. To clarify, if it's just to elicit the model output, how is it different to a baseline prediction? How is it different from ask the model to role-play and then output an prediction? Or, how is "Reader Prediction" potentially role playing?

4 comments

We thank the reviewer for the careful review and comments. The concerns regarding the organization of the paper, the motivation for the Base–Instruct comparison, the explanation of novelty, and the term "Self-Report" are important for accurately conveying the aims and results of this study. We respond to each point below.


W1. Lack of figures/tables and unclear terminology

As the reviewer points out, in the submitted version the main results were concentrated in the appendix, and several concepts were not sufficiently defined. In particular, "Single Recoverable Bottleneck" was not given a clear operational definition.

In the revised version, we will organize the definitions of the key terms and the corresponding tests in the Methods section:

  • Causally Interchangeable Code: Injecting a difference representation obtained from one task into another task produces an output change comparable to that produced by within-task injection.
  • Erasure: Decodability of affect-relevant information is lost after post-training.
  • Uniform Relocation: The peak of information or causal effects shifts uniformly toward deeper layers.
  • Reader–Self Separation: Representational sharing between Reader and Self decreases.
  • Recoverable Bottleneck: Transplanting Base model representations restores the altered rating distribution.

We will also add to the main text the four-stage experimental framework figure, the main scatter plots, layer-wise profiles, and a summary table of results. We will also reorganize the hypothesis labels so that they can be distinguished across analyses.

The sentence the reviewer quoted will be revised to: "Affect-relevant information remains decodable in Instruct models, whereas differences from Base models were observed in its representational structure, cross-task sharing, and causal utilization." However, a decrease in decodability was observed for Qwen Valence, and we do not claim that the information is retained to the same degree in all models.


W2. Limited datasets and tasks

We agree with the reviewer. This study is limited to two datasets (EmoBank and AIPsy-Affect), a numerical rating task, and 1–1.5B-parameter models from four model families.

Therefore, generalization to different affective situations, natural dialogue, other task formats, and larger models has not been established.

In the revised version, we will state these constraints clearly in the Limitations section and will not treat incomplete scale-extension analyses as empirical results.


W3. Motivation and results of the Base–Instruct comparison

We acknowledge that the motivation was insufficiently explained.

The starting observation was that, in some Instruct models, Self ratings concentrated on the fully neutral value (5, 5, 5). On EmoBank, this occurred for 41.4% of stimuli in Qwen Instruct and 76.9% in OLMo Instruct, compared with approximately 0–1.5% in the Base models and 0% in Llama Instruct and Gemma Instruct. This phenomenon is thus not common to all Instruct models.

This output-level change alone cannot distinguish whether affect-relevant information was lost, or whether the information remains but its representation, readout, or utilization has changed.

We therefore formulated the hypotheses of Erasure, Uniform Relocation, Reader–Self Separation, and Recoverable Bottleneck, and tested them using internal representations and causal interventions.

As a result, simple Erasure was not supported overall, although a decrease in decodability was observed for Qwen Valence. Uniform deeper relocation, a uniform decrease in Reader–Self sharing, and recovery through transplanting Base representations were also not supported. On the other hand, a change in the causal profile was observed for the task-dependent component of Valence (q=0.044q = 0.044).

Therefore, the current results do not fully characterize the differences between the two mechanisms. Rather, they show that post-training-associated changes cannot be explained as simple information loss or uniform relocation.

In the revised version, we will restructure the motivation in the following order: the starting observation → competing explanations → the four hypotheses and their decision criteria → their relation to the central question of shared Reader–Self mechanisms.


Q1. Novelty of this study

The novelty of this study lies in eliciting, for the same stimuli, the prediction of a human reader's affect and the model's affective rating about itself as independent tasks that differ only in the perspective of the instruction, and comparing their relationship at three levels: behavior, internal representation, and causal intervention.

In addition, through the Base–Instruct comparison, we examine post-training-associated changes in this relationship under conditions that minimize differences in prompt format.

Prior work has examined the relationship between affect-relevant internal representations and self-referential outputs, whereas this study directly targets the correspondence with predicting others' affect and the cross-task causal interchangeability.

However, our additional analyses revealed that the behavioral coupling, our main positive result, contains a large task-nonspecific component. Therefore, the contribution of this study is not to show that the two tasks share the same affective mechanism.

Rather, the contribution lies in showing, through controlled comparisons, that behavioral covariation, decodability of affective information, and causal utilization are distinct forms of evidence that cannot be equated with one another.

In the revised version, we will clarify the differences from related work and this more limited contribution in the Introduction and Related Work.


Q2. The term "Self-Report" and its difference from role-play

We agree with the concern about anthropomorphizing language. In this study, Self-Report does not imply the existence of subjective experience; it refers to an operational task that asks the model for a self-referential affective rating.

The Reader, Self, and Writer tasks use the same stimuli and response format, and only the perspective of the instruction is changed. The Self condition does not explicitly instruct a persona or role-play; it directly asks the model to rate itself.

However, in the sense that the Self condition also assigns a particular perspective through the instruction, it is a conditioned prediction, and we cannot claim that it is fundamentally distinguishable from ordinary prediction or role-play. What we examined is the extent to which outputs and internal processing are shared across differences in the instructed perspective.

In the revised version, we will replace "Self-Report" where appropriate with "self-referential affect rating," a term that more clearly describes the output format. We will also make explicit the actual differences between the instructions in each experiment.

Because we did not run a separate control condition that instructs role-play, we do not claim to have experimentally tested this difference.

We would like to thank you again for your careful review and for your important comments on the organization, exposition, and positioning of the paper.

In our response of November 14, we committed to revisions regarding the definitions of key terms, the addition of figures and tables, the motivation for the Base–Instruct comparison, the explanation of novelty, and the term "Self-Report." We have since incorporated these into the revised manuscript. Below, we report how each point has been addressed.


W1. Figures/tables and clarification of key terms

We added to the main text a table summarizing the main results, a scatter plot of Reader–Self changes, and a figure showing layer-wise decodability and causal intervention-effect profiles.

We also added explicit operational definitions of the following key concepts to the Methods section:

  • Causally Interchangeable Code: Injecting a difference representation obtained from one task into the other produces an output change comparable to that produced by within-task injection.
  • Erasure: Decodability of affect-relevant information is lost after post-training.
  • Uniform Relocation: The peak of information or causal effects shifts uniformly toward deeper layers.
  • Reader–Self Separation: Representational sharing between the two tasks decreases.
  • Recoverable Bottleneck: Transplanting Base-derived representations restores the altered rating distribution.

The operational definitions can now be found in the main text, and the detailed evaluation procedure for each hypothesis is described in the appendix.

For the four-stage experimental framework, the main text now explains the purpose of each stage and how the stages relate to one another. However, deviating from the figure placement planned in our November 14 response, we placed the overview figure and additional detailed results in the appendix to meet the page limit for the main text. The main text includes explicit references to the corresponding sections.

For the V2 results, we state explicitly that affect-relevant information can remain decodable in Instruct models, but that the degree to which it does so varies across models and affective axes. In particular, decodability decreased for Qwen Valence, and we do not claim that the information is retained to the same degree in all models.


W2. Limitations of datasets, task format, and model scale

This study covers two datasets (EmoBank and AIPsy-Affect), a numerical affective rating task, and 1–1.5B-scale models from four model families.

We state these constraints in the Limitations section, noting that generalization to natural dialogue, different affective situations, other task formats, and larger models has not been established.

We have not conducted full scale-extension experiments, and we do not treat incomplete auxiliary analyses as empirical results.


W3. Motivation and results of the Base–Instruct comparison

We added to the Introduction the observation that served as the starting point for the comparison.

On EmoBank, the proportion of Self ratings concentrated on the fully neutral value (5, 5, 5) was 41.4% for Qwen Instruct and 76.9% for OLMo Instruct, compared with approximately 0–1.5% for the Base models and 0% for Llama and Gemma Instruct.

This phenomenon is therefore not common to all Instruct models.

This output-level change alone cannot distinguish whether affect-relevant information itself was lost, or whether its representation, readout, or utilization changed. We therefore formulated the competing hypotheses of Erasure, Uniform Relocation, Reader–Self Separation, and Recoverable Bottleneck, and examined them through analyses of internal representations and causal interventions.

The results did not support simple explanations such as uniform information loss or uniform deeper relocation. However, we have not fully characterized the mechanisms that give rise to Base–Instruct differences, nor have we independently identified the causal effect of post-training itself.

These qualifications are reflected in the Results and Discussion sections.


Q1. Novelty of this study and differences from prior work

The distinguishing feature of this study is that, for the same stimuli, we independently elicit Reader Prediction and Self-Report by changing only the perspective of the instruction, and compare their relationship at three levels: behavior, internal representation, and causal intervention.

Whereas prior work has examined the decoding of affect-relevant representations and self-referential outputs, this study directly examines the correspondence between predicting others' affect and self-referential rating, as well as cross-task causal interchangeability.

However, our additional control analyses revealed that the strong Reader–Self correlation contains a large task-nonspecific component.

Accordingly, we do not present the contribution of this study as "proof of a shared affective mechanism." Rather, we position it as showing that behavioral covariation, decodability of affect-relevant information, and causal utilization cannot be equated with one another and must each be evaluated independently.

We have stated this scope of novelty and the relationship to prior work in the Introduction and in the Related Work section of the appendix.


Q2. The term "Self-Report" and its difference from role-play

As you point out, the term "Self-Report" may give the misleading impression that the model has subjective affective experience.

In the revised manuscript, we explicitly define Self-Report as "self-referential affect rating," an operational name for the output format produced in response to an instruction requesting a self-referential affective rating.

This does not imply that the model has subjective emotions, consciousness, or privileged introspective access.

The Reader, Self, and Writer tasks use the same stimuli and response format, and only the instructed perspective is changed. The Self condition does not explicitly assign any particular persona.

However, the Self condition is also a conditioned prediction in which a particular perspective is assigned through the instruction. Because we did not run an independent role-play control, we do not claim to have demonstrated that it is fundamentally different from ordinary prediction or role-play.

This point is also stated in the Methods section.


As described above, we have incorporated the explanatory and organizational revisions promised in our November 14 response. At the same time, the limited experimental scope and the insufficiency of mechanistic evidence remain, and we do not consider improvements to the writing to have strengthened the scientific support for our claims.

We are grateful once again for your important comments, which helped us present the research question and the scope of our contribution more accurately.

Reviewer_56jhOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates whether LLMs use shared internal representations and causal mechanisms when predicting human readers' affective responses (Reader Prediction) and reporting their own affective states (Self-Report). The authors conduct behavioral, representational, and causal analyses across four model families, including Base and Instruct variants. They find strong behavioral coupling (Pearson correlations of 0.56–0.93), but limited evidence for directly shared representations or cross-task causal transfer. The results further suggest that post-training-associated differences involve representational changes, while affective decodability does not guarantee robust causal control over Self-Report.

Critical strengths
  1. Interesting research question. The relationship between affect prediction and affective Self-Report is relevant to mechanistic interpretability and LLM safety. The paper investigates whether two behaviorally related tasks use the same internal mechanisms.

  2. Comprehensive experimental framework. The study combines behavioral correlation, linear probing, representational similarity analysis, Procrustes alignment, activation patching, and subspace interventions. These analyses examine different levels of mechanistic evidence.

  3. Multiple models and controls. The experiments cover four model families with Base and Instruct variants. The authors also consider prompt-format confounding, semantic perturbations, and random intervention controls.

  4. Careful treatment of negative results. The paper distinguishes decodability from causal utilization. It also uses held-out evaluation, multiple-testing correction, and discovery/confirmation splits. The authors explicitly acknowledge that the V3 State Induction Gate fails.

Critical weaknesses

I'm not very sure about W2. Please correct me if I'm wrong. Happy to discuss more.

  1. There is not a single figure / diagram in this paper, which makes understanding a bit difficult. I think the submission can hugely benefit from adding informative figures, such as for the 4 stage framework.

  2. The output probability estimation procedure raises some concern. The paper calculates the mean token log-likelihood of each candidate and applies a softmax. This is not the model's actual conditional probability distribution over complete JSON responses. In Table 11, many statistically significant affective shifts are only 0.001-0.03 on a nominal 1-9 scale. This raises concerns about whether the measurement captures meaningful behavioral differences. The authors should compare their results against summed sequence log-likelihoods, alternative temperatures, and actual generated responses. This is particularly important because V3 uses fixed thresholds on these derived scores to determine whether causal interventions succeed. But I may be wrong, and please correct me if I'm missing anything here.

  3. The behavioral coupling analysis does not adequately isolate affect-specific correspondence. The main behavioral finding is a strong correlation between Reader and Self changes across 192 Clinical-Neutral pairs. However, these pairs are organized into eight emotion categories. Both tasks may produce similar changes simply because they recognize the same emotion categories or stimulus semantics. The matched-pair design does not eliminate this explanation. The authors should report within-emotion correlations and partial correlations controlling for emotion category and domain. This concern is reinforced by the weak human-affect correspondence and largely unsuccessful strict dose-response tests in several models. Strong Reader–Self coupling alone may reflect common stimulus classification rather than a meaningful shared affective computation.

  4. The cross-task causal transfer experiment does not adequately account for representational misalignment. V1 reports strongly negative direct cross-decoding performance despite high representational similarity. For example, Qwen Base Valence has Reader-to-Self R^2 of -52.885 but RSA of 0.900. Procrustes alignment substantially improves cross-decoding. These findings suggest that task-dependent offsets or transformations may explain the direct transfer failure. However, E4 directly injects unaligned Reader-derived activation differences into Self representations. This intervention may fail even if both tasks use equivalent affective information in different coordinate systems. The authors should evaluate cross-task activation transfer after learning an alignment on the Discovery split. They should also distinguish task-specific mean offsets from genuine representational transformations.

  5. The V3 State Induction Gate may reject valid causal mechanisms because of its intervention design. The gate tests interventions at a fixed relative depth of 0.5. However, subsequent exploratory analyses locate the strongest causal displacement around relative depths 0.74-0.78. The gate and subsequent causal maps also use different procedures for estimating affective directions. RQ1 regresses hidden representations on Reader scores, while RQ2 regresses Reader scores on hidden representations. These procedures do not generally identify identical directions. Consequently, failure of the initial gate does not establish that the representations analyzed later lack robust causal influence. The authors should evaluate a consistent intervention method at independently selected sites before drawing broader conclusions from the gate failure.

  6. The V3 causal evidence is incomplete and difficult to verify. "Appendix Q V3 Results" contains no substantive results. The subsequent discussion states that the gate failed but does not provide a complete quantitative table of sufficiency slopes, specificity effects, attenuation ratios, and confidence intervals. This prevents independent assessment of the central negative finding. Furthermore, the generation-stage experiments use a fixed teacher-forced JSON response with Valence and Arousal both set to 5. The observed temporal effects may depend on this artificial response trajectory or the position of the JSON fields. They do not establish that the same causal structure exists during natural generation. The authors should provide the missing quantitative results and evaluate alternative candidate trajectories or free-generation settings.

  7. The connection to practical psychological guardrails remains unvalidated. The paper motivates its analysis through selective regulation of self-referential emotional expression. However, Self-Report is measured through numerical ratings rather than natural conversational responses. The authors do not demonstrate an intervention that suppresses self-referential affective expression while preserving Reader Prediction performance. The experiments also use only 1-1.5B parameter models. The findings may not generalize to other scales. These limitations leave a gap between the mechanistic observations and the proposed application to deployed conversational assistants.

Questions for authors

see weakness

5 comments

We thank the reviewer for the careful review and constructive comments. We consider the concerns regarding the validity of the scoring procedure, the interpretation of behavioral coupling, cross-task causal transfer, and the evaluation design of the State Induction Gate to be important. Below, we respond to each point and describe the additional analyses and planned revisions.

W1. Lack of figures

As the reviewer points out, the submitted version lacked figures to support understanding of the results.

In the revised version, we will move the four-stage experimental framework figure from the appendix into the main text, and add a scatter plot of Reader–Self changes, layer-wise decodability and causal-effect profiles, and a figure showing the relationship between generation stage and layer. These additions will clarify how the behavioral, representational, and causal analyses relate to one another.

W2. Validity of the scoring procedure

The reviewer's concern is mathematically correct. We verified that all candidates have equal token length within each model and candidate space. Therefore, our procedure of applying a softmax to mean token log-likelihoods is exactly equivalent to applying a softmax with temperature T=mT = m to summed sequence log-likelihoods.

The token length mm differs across model families and candidate spaces, and so does the effective temperature. For Qwen Base with the 729-candidate space, the mean entropy of the distribution obtained from mean log-likelihoods was 6.590 nats, very close to the 6.592 nats of the uniform distribution. We acknowledge that this flattening affects measurement sensitivity.

To assess this issue, we conducted post-hoc sensitivity analyses. For Qwen Base, the Reader–Self change correlations for Valence/Arousal were 0.809/0.866 in the main analysis, 0.880/0.911 under summed sequence log-likelihood scoring, and 0.441/0.442 under mode-based scoring. All correlations were positive, and none of the 95% confidence intervals included zero. However, the magnitude of the correlation depends on the scoring method.

In addition, controlling for Writer changes substantially reduced the correlations. This pattern was consistent across scoring methods, and the observed coupling therefore cannot be interpreted as sharing of Reader–Self-specific affective processing.

At present, we have re-analyzed behavioral coupling under alternative scoring for 4 of the 8 models. Under summed sequence log-likelihood scoring, positive correlations were observed for Valence/Arousal/Dominance in each model, but results after controlling for Writer vary across models and axes. We therefore do not claim that robustness has been established across all models.

For the V3 Gate, we also compared 12 scoring methods on Qwen2.5-1.5B-Instruct. Under every method, both Valence and Arousal were NO_GO, and the pass/fail outcome of each criterion did not change. In a separate analysis of 96 threshold settings, no setting yielded GO. However, these analyses are post hoc, and we have not re-estimated direction vectors for each alternative score or evaluated free generation.

In the revised version, we will explicitly describe the mathematical properties of the scoring procedure and the resulting flattening, and add the verified alternative-scoring results. We will also report effect sizes not only on the absolute scale but also relative to the natural Clinical–Neutral changes. We retain the main-analysis results, but make their interpretation conditional on the measurement procedure.

W3. Confounding of behavioral coupling with emotion category

Following the reviewer's comment, we conducted post-hoc analyses controlling for emotion category and domain.

Across Valence/Arousal for all 8 models, partial correlations controlling for emotion category ranged from 0.543 to 0.927, and partial correlations controlling for the 48 emotion-category × domain cells ranged from 0.483 to 0.940; in all cases, the lower bound of the 95% confidence interval exceeded zero. Thus, the observed coupling cannot be fully explained by these categorical variables alone.

On the other hand, controlling for Writer changes reduced the Reader–Self partial correlations to −0.153 to 0.432. Moreover, Reader and Self ratings were highly correlated even on Neutral stimuli alone, and randomly re-pairing Clinical and Neutral stimuli barely changed the correlation.

Additional analyses adjusting for word-count differences produced only small changes in the correlations; the main reduction came from controlling for Writer. However, we cannot fully rule out confounds arising from stimulus length or semantic content.

Taken together, we interpret the originally strong Reader–Self coupling as containing a large task-nonspecific response component to the common stimuli. That the correlation persists after controlling for emotion category does not directly support the existence of shared affective processing.

In the revised version, we will position this result as descriptive covariation and add the Writer-controlled and task-nonspecificity analyses.


W4. Representational alignment and cross-task causal transfer

Following the reviewer's comment, we re-examined the failure of direct transfer.

For Qwen Base Valence, the direct Reader-to-Self decoding R2R^2 was −52.9, which improved to approximately −4.7 after task-specific standardization. Furthermore, Table 19 in the submitted version selected layers post hoc based on the alignment improvement; when we re-evaluated at layers selected based on Reader decoding performance, the corresponding value changed from −9.6 to −2.8. Thus, the large negative R2R^2 values reflect offset/scale mismatches and limited generalization of the probes.

On the other hand, because E4 injects Clinical–Neutral difference vectors, mean offsets between tasks cancel by construction. Across the 8 models, the cosine similarity between the injected Reader-derived difference and Self's own difference was 0.571–0.957 for matched stimuli and 0.069–0.143 for random stimuli.

However, a more important limitation is that even the within-task positive control, Self→Self injection, produced effects only on the order of 10−310^{-3}. Therefore, the current intervention sites and measurement procedure are insufficient to determine cross-task causal interchangeability.

Accordingly, we do not treat the negative result of E4 as evidence for the absence of a shared mechanism, but position it as an inconclusive result. In the revised version, we will withdraw the strong interpretation that the results support an intermediate mechanism. Aligned absolute-state replacement and its validation with a positive control have not been performed, and we will state them explicitly as future work.


W5. Intervention site and direction estimation for the Gate

As the reviewer points out, the site used for the Gate differs from the sites where stronger effects were observed in subsequent exploratory analyses.

The Gate intervention site was L14 (relative depth 0.519) at prompt_end, which was fixed as the code default prior to the subsequent exploratory maps. However, it was not an externally pre-registered site.

In addition, RQ1 uses an encoding direction (predicting hidden representations from Reader ratings), whereas RQ2 uses a decoding direction (predicting Reader ratings from hidden representations), and we have not verified that the two coincide.

In all-layer interventions using a subset of the independent Discovery data, the maximum effect at prompt_end was 0.0055. Larger effects were instead observed at pre_V/pre_A, immediately before the rating tokens, at relative depths of approximately 0.74–0.78. However, the latter results were obtained on the data used for exploration and are subject to limitations: a sample size of 15 and no random control.

Therefore, the failure of the Gate does not allow us to conclude that no robust causal effect exists at other intervention sites or under other direction-estimation methods.

In the revised version, we will restrict the conclusion of the Gate to the statement that "the criteria were not met for the specified site, direction-estimation method, and intervention procedure."


W6. Missing Appendix Q and teacher forcing

We apologize for the missing quantitative results table. All estimates have been preserved, and we will restore them in Appendix Q of the revised version.

For Qwen2.5-1.5B-Instruct, L14, prompt_end, and 58 test stimuli, the main results are as follows (Valence / Arousal; 95% confidence intervals in brackets):

  • Sufficiency: 0.00129 [0.00043, 0.00212] / 0.00017 [−0.00005, 0.00039]
  • Specificity: −0.00147 [−0.00248, −0.00044] / −0.00011 [−0.00042, 0.00022]
  • Endogenous attenuation ratio: −0.0075 [−0.0366, 0.0216] / 0.0082 [0.0013, 0.0162]
  • Topic TVD: 0.0036 [0.0031, 0.0043] / 0.0039 [0.0033, 0.0045]

Under the prespecified criteria, both axes were NO_GO. The revised version will include the threshold, confidence interval, and pass/fail outcome for each metric, as well as the related attenuation results.

We also agree with the comment on teacher forcing. Up to pre_V, the primary evaluation position for Valence, the prefix is shared across all candidates, but Arousal and later generation stages depend on fixed preceding tokens.

In addition, the "stage-position invariance" used in the submitted version verified the consistency of token positions, not the temporal robustness of the generation process.

In the revised version, we will restrict these results to exploratory findings under a fixed JSON response format. We will explicitly state that reproducibility under free generation or different candidate sequences has not been verified, and we will also correct the erroneous table references.


W7. Application to practical guardrails

We agree with the reviewer. This study does not demonstrate an intervention that selectively suppresses self-referential expression while preserving Reader Prediction.

Moreover, the evaluation is limited to a numerical rating format, two datasets, and models with 1–1.5B parameters. Generalization to natural dialogue and larger models has not been verified.

In the revised version, we will withdraw the claim that the study "provides a mechanistic basis" for selective psychological guardrails, and instead position our results as exploratory findings that motivate future evaluation.


The main contribution of this study is that it separately evaluates behavioral coupling, decodability of internal information, and causal utilization. We will revise the Abstract, Discussion, and Conclusion so as not to make claims about these relationships that go beyond the evidence obtained.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.