acceptodds
Under review as a conference paper at ICLR 2027

Behavior and Self-Report Are Differentially Controllable: A Within-Model Causal Test in Ten Instruction-Tuned LLMs

Abstract

Instruction-tuned language models describe their own dispositions fluently, and safety and alignment evaluation increasingly treats such self-reports as evidence about how a model will act. Yet a growing body of work finds that what LLMs say about themselves and what they do come apart: across models, across contexts, and between explicit and implicit measures. That evidence is correlational, and much of it compares models with one another, so it cannot say why—whether a single model has distinct internal causes for saying and doing, or the two are merely measured differently. We test this causally, within a single model. For each trait we build two steering vectors: a persona vector, contrasting high- and low-trait persona prompts, and an item vector, contrasting held-out first-person trait statements. Using Contrastive Activation Addition, we add each to the residual stream and read both forced-choice behavior and Likert self-report. The trait is never named, so a behavioral shift cannot be instruction-following. Across a preregistered panel of ten models (14–70B; dense and mixture-of-experts; full-attention and hybrid attention–Mamba), steering controls behavior: the dose–response slope is positive for every model (averaged over factors) and every factor (pooled over models), and the pooled slope exceeds a matched random-vector null (composite +2.28, null margin +1.47). Behavior and self-report are also differentially controllable: the persona vector moves behavior more than self-report and the item vector the reverse, a crossover (difference-in-differences +0.69) positive in nine of ten models, consistent in direction but model-dependent in size. Both results survive a preregistered sensitivity battery (dose exclusion, disattenuation, Bayesian hierarchical pooling), and the crossover strengthens with depth in the residual stream. Each vector's effect also holds when its channel is measured a different way (judge-scored free generation; a non-Likert self-report), the pattern expected if the vectors move underlying representations rather than a single readout. We establish differential controllability and read the two directions as most likely separate representations. The consequence for evaluation is direct: because steering can pull self-report and behavior apart, a model can pass a verbal evaluation while behaving otherwise. LLM self-report and behavior are not interchangeable evidence about the same state.

Primary area:
interpretability and explainable AI
Keywords:
self-report, behavior, activation steering, contrastive activation addition, representation engineering, interpretability, model evaluation, LLM psychometrics, introspection, double dissociation, preregistration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ZV3ROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses the topic of whether LLM behavior and self report can be separately manipulated. They use 5 personality traits and 10 models (the design was preregistered). They steer with item and persona vectors and measure behavior (avg log-prob of 2 responses) and self report. They find that persona and item vectors affect behavior and self report respectively, and conclude that this supports the idea of separate representations for behavior and self report.

This is an important topic and this study adopts an interesting approach. A strength of the approach is the wide model coverage and pre-registration. However, I find it difficult to know how to interpret the findings without more details about the methodology which i found to be very unclear (see below)

Critical strengths

S1: This is an important topic and the authors use an interesting design S2: a large number of models is tested S3: the study is preregistered

Critical weaknesses

W1: The methodology is insufficiently clear. Could the authors clarify how exactly the vectors are constructed? This is especially an issue for construction of the item vector: the model is given opposing statements. Is the model's generated response included? Related to this: I do not understand why a vector of the item differences would reflect a representation of a model trait -- rather than semantic differences between the sentences, and also whether a vector would also capture if the model disagrees with either sentence. W2: There is also an identification problem: it is shown that different interventions affect behavior and self report differently but there are several differences between the construction of the item and persona vector --- one is in the first person (item) vs second person (persona). But they differ in whether they are in the system prompt or the main context, and in the content of the messages. Hence it is difficult to know exactly which manipulation is responsible for producing the differences. In particular, I don't think necessarily follows that separate representations must involved.

Questions for authors

Q1: Can the authors clarify exactly how the steering vectors are constructed? Q2: Can the authors clarify their position on why the constructed vectors reflect the factor they focus on rather than other factors? In particular, why couldn't the differences in steering results be due to one being in the system prompt and the other being in context -- or content variations between the messages. If I understand correctly, the authors focus on the first-person vs second-person nature of the item vs persona vectors, but aren't there other differences?

Reviewer_Jcu1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigated whether LLM self-reports and behavior can be manipulated via internal interventions within the LLM. The work used Contrastive Activation Addition applied on the LLM activaitons. These steering vectors were derived per trait: persona vectors of high/low persona traits; item vectors from held-out trait statements. The experiments are performed on ten instruction-tuned models as well as five questionnaire-derived traits. The effects were evaluated using: forced-choice tasks on the behavior of the model as well as on its self-reports using Likert-scales. Additionally, the generated responses were also scored by an LLM judge, and a non-Likert self-report. The results showed that persona-derived vectors affected the behavior of a model more than its self-reports, whereas the reverse was found for item vectors derived from statements. The main finding of this paper is a within-model causal investigation of differential controllability. It also provides evidence for separate representations that steer the self-description (via self-reprot-based self-evaluation) and the corresponding behavior. Thus, in practice the paper highlights that verbal self-evaluation should not be confounded with the corresponding behavior, as they are driven by different representatons, that are separately controllable.

Critical strengths
  • The within-model intervention provided some useful information on the causal effect of manipulating internal activations on both behavior and self-report, that addresses the recent interest in understanding why self-reports are not predictive of behavior in LLMs.
  • The results are made more substantial due to a panel of ten models and a matched random-vector control for the behavioral effect. The persona and item vectors were contrasted in a crossed design and pitted against both behavioral and self-report readouts.
  • High Transparency: Preregistration, test set evaluation, sensitivity analysis, and thorough reporting of deviations and unsuccessful approaches.
Critical weaknesses
  1. Controllability does not necessarily imply construct validity. While the paper shows that questionnaire responses can be steered in a certain way, this does not necessarily imply that the factors derived from the questionnaire in one way measure meaningful dispositions that would be expected to also affect behaviors. What I mean here, is that for the same steering direction to move both self-report and behavior, the steering method would need to affect some shared disposition that then is used by the model to decode the self-report and behavior form a shared place. Since the factors are derived from questionnaire responses, and the study found they barely predicted behavior, a weak connection between the two channels might not be that surprising. Section 5.1 argues that steering the self-report measures suggests these measures capture something real, like a disposition. However, a controllable response pattern does not necessarily imply valid measurement of a disposition.

  2. The self-report null seems to be missing. The analysis of impact on behavior used matched random vectors for control, which is good, because injecting just random direciton will affect the model redoputs. However, for self-report and the crossover, no such control seems to have been used. As a consequence, we do not know whether the observed questionnaire shifts exceed those that comparable untargeted perturbations would have produced. Please let me know in case I missed it somehwere.

  3. It's possible that the instrument used in the study wasn't a good fit, which could explain the results. The paper suggests that even with a better questionnaire, there would still be a gap. But the experiments didn't really show this, and it's hard to make that claim without a survey that's been validated by outside sources (e.g., a pscyhometric method that has been shown to link to behavior in humans; I understand psychometric method have their own challenges and assumptions). The behavioral tests in the experiments mostly looked at very specific parts of the factors that the questionnaire was trying to measure. For example, the Deference test was only looking at the peripheral warmth facet of that factor, and it actually gave the opposite result of what was expected. The paper registers this caveat in advance, but its two accounts seem to conflict: Section 3 selects that facet as the one carried by the factor’s highest-loading items, while Sections 4.2 and 5.2 attribute the reversal to the facet being peripheral. The persona vectors in general failed the factor-selectivity check. While none of these results would individually dispute the use of an instrument, together they leave open the question whether the two channels even measure comparable enough constructs.

  4. The construction of each vector may favor the readout it subsequently moves. The item vectors are built from statementsr endered from the self-report template, whereas the persona vectors are built from persona-conditioned neutral prompts. Hence, there is construction context difference that could be transferred to and favor performance in the type of task for which the vectors were derived. The paper does not report the positions at which each vector is extracted (at least I could not find it), so the reader cannot assess whether an answer-slot versus generation-slot confound contributes. Although the cross-format self-report test is provided, converting the Likert-scaled ratings to forced-choice self-descriptions does not solve this problem. Thus, a test of equal extraction position and task context for the two vector types would be particularly useful to assess this central issue.

  5. The way a judge's behavior is recorded is clear, but it hasn't been proven to be accurate. Even though there's a detailed description of the judge and the rules it follows in Appendix H, there's no confirmation from humans that the scores it generates actually measure what they're supposed to. This is important because the experiment that uses the judge’s scores is the main evidence that the steering effect is not specific to the forced-choice format. There are still some missing scores and chain-of-thought leakage from one of the steered models, as mentioned in Appendix D, and the checks that were done to address these problems only partially solved them.

  6. The abstract is hard to follow. Many key terms and methods are introduced before they are explained - e.g. disattenuation and Bayesian hierarchical pooling. Furthermore, the headline figures and the null margin are expressed in non-interpretable scales. This is later introduce in Section 4.1 where the composite is introduce.

  7. Another issue is that the mixed scales seem to be dominated by the Verbosity factor. Section 4.1 itself rests on per-factor results rather than the composite. Thus the abstract should present an interpretable result, and make a clear distinction between the controllability that is demonstrated and the representational separation that is proposed.

  8. Minor issues, that might be due to different backgrounds: I don't think the DiD term for the analysis conducted is actually used correctly. I think what is actually being tested is more an interaction between the steering method constructions and the impact on the different readouts (self-report or behavior). As far as I know DiD is about causal inference, that relies on parallel trend hypothesis. Essentially in DiD we have two groups form the same population, that we assume would behave similarly over time, then an intervention is applied to one of them, while the other stays as control. DiD would then measure the impact of the intervention, while accounting for the potential parallel changes in the control group (hence the difference in the differences accrued over time between the groups). I think the authors try to adopt this terminology to a setup without temoral evolutiuon, but I feel that simple interaction is more appropriate here as there is no temporal evolution that happens in parallel. I am willign to be persuaded, but I would like to limit unnecessairly complex terms.

Questions for authors
  1. Could you also evaluate matched random directions for self-report? Can you also specify a corresponding null hypothesis for the crossover? This should be done using the same selection and analysis approach. Do the targeted effects exceed these controls?
  2. Can you specify the exact activation positions for both vector sources in the two conditions? A comparison to the same extraction positions but with reversed task contexts would then be very interesting and would show whether the crossover also persists when the extraction positions are controlled.
  3. Can you provide evidence that the measures in the questionnaire banks (intended to be the same as in the behavioral banks) actually measure the same intended construct? Would the crossover effect stay the same with the use of broader, better-matched behavioral measures? Especially for Responsiveness and Deference?
  4. Can you get human raters to score a blinded, representative sample of your generated responses for the behaviorals and report effects on that subset as well? Please report whether the behavioral effects persist on the validated subset?
Reviewer_MEmQOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

It has been known that there is a discrepancy between self-reports and actual behavior. To investigate the cause, this study constructs the following two steering vectors and shows that self-reports and actual behavior can be controlled independently:

  • The difference in internal activations between high-trait and low-trait persona prompts
  • The difference in internal activations between high-trait and low-trait self-descriptive statements
Critical strengths
  1. The evaluation covers a diverse set of models that differ in scale and architecture.
  2. The analysis is based on careful statistical testing.
Critical weaknesses
  1. Although the authors seem to acknowledge this issue themselves (L463–465), it is unclear what the finding that steering vectors can dissociate self-reports from behavior tells us about the cause of the self-report–behavior discrepancy documented in prior work.
Questions for authors
  1. Related to Weakness 1, what is the central question this study aims to answer and why is it important?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.