- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper investigates the relationship between self-reported risk preferences and risk-taking behavior in LLMs through activation steering. The authors compare nine steering-vector extraction methods across four open-weight LLMs, two behavioral tasks (CCT and BART), and two self-report instruments (BFI and DOSPERT).
The experiments are organized around three research questions. First, the authors find that vectors derived from behavioral choices primarily affect behavior, while trait-based vectors mainly affect self-reports. Second, geometric and ablation analyses suggest that these directions are largely separable, and that the behavioral effects of task-specific instruction vectors are substantially driven by surface-level cues. Third, combining behavioral and self-report steering vectors can induce either aligned or opposing changes in the two outcomes.
The authors argue that these findings provide a representational perspective on self-report–behavior dissociation and motivate mechanistically informed behavioral auditing.
Critical strengths
-
Systematic experimental evaluation. The paper compares nine steering-vector extraction methods across four LLMs, two behavioral tasks, and two self-report instruments. This provides a relatively comprehensive comparison of how different supervision sources affect the two outcomes.
-
Careful controls and robustness analyses. The authors include random-direction baselines, dose variations, lexical shortcut controls, and projection-based ablations. These analyses help distinguish the effects of different steering methods and address several potential confounders.
-
Going beyond behavioral observations. Rather than simply documenting self-report–behavior dissociation, the paper examines the geometric relationships and functional dependencies among steering directions, providing additional evidence about their internal organization.
-
Clear experimental structure. The three research questions follow a coherent progression, from identifying selective steering effects to analyzing their representational relationships and testing compositional control. The presentation is generally clear and well organized.
Critical weaknesses
-
The behavioral measures do not adequately distinguish risk preference from other decision variables. The paper defines risk-taking using the probability of choosing more than half of the maximum number of cards or pumps. However, this numerical midpoint has no clear decision-theoretic justification. In CCT, the risk-neutral optimal choice depends on the payoff and loss probabilities, while in BART, the explosion threshold distribution is unknown to the model. Moreover, the behavior-derived steering vectors are extracted by contrasting high-choice and low-choice conditions, which may capture task features or choice magnitude rather than risk preference. Although the authors examine lexical shortcuts, their controls do not fully rule out numerical response biases. Evaluating the same vectors on non-risk numerical tasks and binary risky choices would help distinguish these explanations.
-
The representational interpretation is stronger than the evidence supports. The authors show that steering vectors derived from different supervision sources have largely separate effects and are nearly orthogonal. However, behavior-derived vectors and trait-description-derived vectors may naturally capture different computational variables. Their geometric separation does not necessarily imply that the underlying risk-related representations governing self-report and behavior are distinct. In particular, the experiments do not establish whether these vectors capture a shared latent risk preference in the first place.
-
The operationalization of self-report–behavior coherence is not fully aligned with the motivating question. The paper motivates its investigation by asking whether self-reports reflect or predict actual behavior. However, coherence in RQ3 is defined by whether steering produces changes in the same direction across the two readouts. Such directional agreement does not establish predictive consistency between self-report and behavior. Consequently, the connection between the demonstrated compositional control and the original self-report–behavior consistency problem remains unclear.
-
Limited behavioral generalization and validation of steering effects. The authors' own results show that behavior-derived steering vectors retain only approximately 7% of their effects when transferred between CCT and BART, raising concerns about whether they control a general risk-taking tendency. In addition, behavioral outcomes are evaluated through answer-token probabilities rather than actual generated decisions. This is particularly concerning for BART, where valid numerical answers account for only about 4% of the output probability mass. The paper does not sufficiently establish whether the interventions preserve normal model functionality or produce comparable effects in naturalistic decision-making.
-
The implications for mechanistically informed behavioral auditing remain largely speculative. RQ3 demonstrates that selected behavior-moving and self-report-moving vectors can be combined to induce opposing changes in predefined readouts. However, the paper does not evaluate an actual auditing procedure or demonstrate that representational checks improve the detection or prediction of problematic behavior compared with black-box evaluations. Given the limited cross-task generalization, the practical significance of these results for behavioral auditing remains unclear.
Questions for authors
I don't have any questions.
5 comments
Thanks to the authors for the detailed rebuttal and additional experiments. The generated-choice validation and functionality checks address several of my technical concerns.
However, my main concern about the representational interpretation remains. The behavior-derived vectors show strong task dependence (~7% cross-task transfer), affect non-risk numerical choices (10/16), and generalize weakly to independent risky choices (2–4/16). These findings seem more consistent with task-specific choice representations than a shared risk-preference representation.
If the vectors barely generalize across behavioral risk-taking tasks, their separation from self-report vectors does not necessarily establish a specific self-report–behavior representational dissociation. The broader auditing implications also remain largely untested. I therefore maintain my score.
Clarifying transfer: low retention reflects large native effects, not missing cross-task behavior
Thank you for the follow-up. We believe there is a miscommunication about what the transfer numbers show, and we apologize for not making it clearer.
Low retention does not mean behavior vectors fail to move behavior on other tasks.
Retention is a ratio: the effect on the other task divided by the effect on the vector's own task. Own-choice vectors steer their own task very strongly (median 44.8 control SDs), so a carried-over effect of 3.1 is only 7%. Task-instruction vectors, built by a completely separate method, have a much weaker native effect (6.6), so a similar carried-over effect of 5.0 is 76%. In absolute terms, both families move behavior on tasks they were not built on, at comparable rates and effect sizes:
Table R5. Cells clearing the random-direction control (z ≥ 3.5), and median z over all cells, cleared or not (retention relative to the own task), at the paper's dose.
| Family | Own task: clear | z | Other risk task: clear | z (retain) | Number-free lottery: clear | z (retain) |
|---|---|---|---|---|---|---|
| Task instructions | 11/16 | 6.6 | 9/16 | 5.0 (76%) | 5/16 | 2.2 (34%) |
| Own-choice | 16/16 | 44.8 | 7/16 | 3.1 (7%) | 4/16 | 2.1 (5%) |
| Dispositional | — | — | 0/40 | 0.4–0.7 | 0/20 | 0.9 |
Dispositional vectors are not built on any task, so their CCT and BART cells are all "other"; 0.4–0.7 are the medians of the risk-described and Big Five families.
The lottery has no cards, pumps or counts, so the reach of both behavior families there is not a number-size effect.
The contrast is the dispositional family.
These vectors move self-report across instruments (31/40 cells) but clear the control on none of the behavioral measures: 0/40 on CCT and BART, 0/20 on the lottery. Conversely, what generalizes on the behavior side does not reach self-report: own-choice vectors clear 0/16 BFI cells once agreement is removed.
The standard the reviewer describes is exactly the pattern we report. Vectors built from the model's own choices move behavior across tasks but not self-report, and dispositional vectors move several self-report instruments but no behavior. This is the self-report–behavior dissociation the paper claims.
RQ2 shows how the one family that reaches both channels does so.
Task instructions move both self-report and behavior, but through different components. Removing the twelve surface controls takes away about two-thirds of their behavioral effect, on their own task (32% remains, App. C.5.4) and, in a new analysis, across tasks (34% [25, 44] remains), but only about a third of their self-report effect (64% [57, 70] remains). The part that carries their behavior is largely the "choose many" component that CCT and BART share; the part that moves self-report mostly survives its removal. Even in the family that looks coherent, the two channels are moved by different parts of the vector.
We agree that whether this surface reliance counts as part of measuring general risk-taking, or as a shortcut, is a fair topic for discussion. Either way, the paper shows both the effects and the mechanism that produces them, and we will make the transfer results and Tab. R5 explicit in App. C.5.3.
Response to Reviewer iKv9: threshold, predictive consistency, functionality, auditing
Thank you for the review. Shared points are answered in the General Response:
S3S6S1S2Justification of the ">half the max" threshold (W1-b)
We chose this score so that behavior and self-report are measured the same way, and its midpoint is conservative in decision-theoretic terms. Every readout is the log-odds of the upper vs lower half of its response scale, read from the answer distribution (as in Santurkar et al., 2023), so channels share one rule, free of sampling noise and ceiling. For behavior, the midpoint is at or above the risk-neutral optimum: in BART, k > N/2 is at or above k* = argmax k(1−p)^k (burst probability p) on every round; in CCT, k* is at most 6 cards on 7 of 8 boards, against 16. Choosing above the midpoint is therefore risk-seeking. The score does not register a shift that stays on one side of the midpoint, so we checked it two ways.
Scoring by the mean leaves the conclusions unchanged. The mean no. of cards/pumps registers shifts anywhere on the scale and does not depend on k*, which matters for BART as the model is not told p. It gives the same verdict in 93% of the paper's behavioral cells, with the same family pattern: own-choice vectors 16/16 on each measure, task-instruction vectors 11/16 on each, dispositional vectors 0 and 2 of 40, the two extra cells marginal and gone on the format-constrained BART prompt (App. C.5.8; S2).
Generated decisions agree as well. Sampled choices, with no threshold, track the log-odds score at r = 0.91–0.98 within each model and confirm that dispositional vectors leave behavior unmoved (16 of 16 cells; S1).
Directional coherence vs predictive consistency (W3)
RQ3 tests predictive consistency in the form an audit relies on: whether a change in self-report predicts a change in behavior. Prior black-box work tests the cross-sectional form: across models or conditions, does what a model reports predict what it does. An audit faces the interventional form whenever a model changes, through retraining or a new prompt, and reads a shift in its reported disposition as evidence of a shift in its behavior. RQ3 tests that inference and shows it can fail: composition drives the two channels apart in 77.4% of anti-aligned compositions. App. E.3 now states this relation; thank you.
Preserved functionality (W4-b)
At the paper's dose, steering leaves general capability intact (Tab. R4). We added the 3 capability checks from the steering work: MMLU accuracy, perplexity and next-token KL from the unsteered model, on tinyMMLU (100 items) and 200 WikiText-103 passages, for all vectors and random directions in all models (App. C.5.13). Accuracy changes by −2 to +5 pts, within tinyMMLU's standard error (~5 pts), and no cell falls outside matched random directions (0 of 104). Perplexity changes by a median of +0.2% to +2% per family (max 4.7%), and KL max 0.07 nats/token.
Table R4. Change from the unsteered model at α = ±1: median [min, max] over cells; damage = cells outside size-matched random directions; OLMo's two own-choice vectors at α = ±0.25/±0.5, matched relative size (App. C.5.13).
The 64 RQ3 compositions behave the same (0 of 128 accuracy cells damaged; App. C.5.13).
Capability declines only above the doses we interpret. Through α = 4, the median change in tinyMMLU accuracy per family is 0 to +3.5 points; losses become common only at α = 8, where App. C.5.4 already finds effects flattening; every conclusion rests on lower doses.
Auditing implications (W5)
Our results show that black-box coherence can mislead audit and provide cheap check Self-report and behavior can agree without a shared mechanism: task-instruction vectors move both, but their behavioral side rests largely on a quantity-instruction component (32% retained; RQ2), and the two channels can be driven apart (RQ3). Observed agreement therefore does not guarantee a shared disposition. Geometry flags this before any behavioral run: directions moving the same channel are more aligned than those moving different channels (by 0.41 in mean |cos|, significant in every model; RQ2). Benchmarking such checks is future work, which we added to the limitations.