acceptodds
Under review as a conference paper at ICLR 2027

A Steerable Internal Direction Links Distinct Mental-Accounting Biases in Large Language Models

Abstract

Large language models (LLMs) increasingly stand in for human participants in economic and behavioral research, and reproduce many human decision-making biases in their answers. But matching an answer does not reveal whether a model carries the underlying concept or only imitates familiar patterns, which determines how far such synthetic respondents can be trusted. We probe this using mental accounting, the tendency to treat money as non-fungible, valued differently by its source or use, which behavioral economists have cataloged as several separate biases: segregating gains, integrating losses, the sunk-cost effect, and transaction utility. Rather than score answers alone, we read and edit the models' internal activations, the numerical state a model builds while processing a prompt. In LLaMA-3-70B, the three categorical biases share a common internal direction: reading it from the model and then adding or subtracting it shifts economic choices as predicted, while a matched random direction does not, showing the representation is causal, not merely correlated with behavior. A related transaction-utility direction widens the venue-dependent willingness-to-pay gap. A controllable de-biasing direction recurs across independently built large models (Qwen2.5-72B, Gemma-2-27B) but is largely absent in three smaller ones, which still show the biases yet offer little steerable control. Controllability is more evident at larger scale in the models tested. The results reveal a mechanism for why high-capacity LLMs replicate economic behavior, and a standard for auditing them as synthetic respondents: match not only their choices, but whether the structure behind those choices is stable and controllable.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, activation steering, mental accounting, behavioral biases, large language models, synthetic respondents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3EWCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether distinct mental-accounting biases in LLMs share an internally manipulable component. It constructs contrastive activation directions from mental-accounting and fungibility persona prompts, and studies behavioral dose responses, cross-dimension transfer, geometric alignment, and several control interventions. Across six models, the paper finds strong alignment between segregating-gains and integrating-losses directions; sunk cost is connected primarily through functional transfer rather than vector collinearity. The most extensive intervention evidence is obtained on LLaMA-3-70B. These results support a shared functional steering component, while leaving open whether the biases arise from a single underlying mechanism.

Critical strengths
  1. The paper moves beyond reproducing behavioral biases to testing whether they can be jointly manipulated through internal interventions. Cross-dimension transfer provides meaningful evidence for shared functional structure.

  2. The controls are substantive: norm-matched random directions, adjustment for a generic instruction-following direction, and persona-related analyses address several straightforward alternative explanations. The distinction between collinearity and functional transfer is also valuable.

Critical weaknesses
  1. There is an unresolved mismatch between the range used for the central transfer statistics and the range supported by capability-preservation tests. Appendix B.3 describes a sweep through |α|=0.40 and reports 13 significant arms out of 15 assessable arms. Appendix C.5 states that main analyses are capped at |α|≤0.30 because stronger negative interventions substantially degrade general-knowledge performance. It is unclear whether the 13/15 result survives the restricted range. Since the capability test applies edits at every decoding step, I do not assume that its degradation directly transfers to every main experiment; the intervention protocol and statistical range need to be reconciled.

  2. The strongest transfer evidence is concentrated in one model and a small prompt panel. Repeated generations support effects conditional on these prompts, but broader scenario-level generalization is less well established. The structurally different prompt tests are useful, although their support is uneven across dimensions.

Questions for authors
  1. Please report the complete cross-dimension transfer matrix, effect sizes, and multiplicity-corrected significance using only |α|≤0.30. Does the 13/15 conclusion remain valid? Please also reconcile the intervention positions and application frequency used in B.3 and C.5. A robust result under the capability-preserving configuration would materially improve my assessment.
  2. Which transfer effects remain consistent when reported separately by prompt, or when uncertainty is computed with prompts as clusters? This would clarify whether the shared component generalizes beyond the particular sampled prompt panel.
Reviewer_BW3xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Please concisely summarize the paper, contributions, and validation as you understand them to be. The paper investigates whether several mental-accounting behaviors in LLMs are associated with reusable and causally manipulable internal structure. It estimates contrastive activation directions from mental-accounting versus fungibility personas, probes response-related information across layers, and applies signed activation interventions. In LLaMA-3-70B, the reported interventions alter segregating-gains, integrating-losses and sunk-cost responses, with substantial cross-task transfer. SG and IL directions are highly aligned, while SC is geometrically distinct and controlled at a different layer band. A separate transaction-utility direction increases the premium-versus-ordinary WTP gap under amplification; categorical directions mainly transfer to WTP level. Six-model comparisons, random-vector and instruction-specificity controls, persona ablation, capability checks, and held-out prompt tests examine specificity and generalization. Replication beyond LLaMA-70B is partial, and held-out generalization is clearest for SG.

Critical strengths
  1. The paper moves beyond behavioral resemblance to testable internal interventions. Signed dose-response curves, cross-dimension transfer, matched-norm random controls, and persona-related ablations provide useful constraints on the interpretation. The LLaMA-70B results, if robust within the stated primary intervention range, could establish meaningful functional links among behaviors usually evaluated separately.
  2. The experimental coverage is substantial and includes informative limitations. Three families at two scales, geometric analyses, prompt-only comparisons, held-out families and capability checks give the investigation breadth. The acknowledgment that prompting can match behavioral control, and the reporting of weak generalization and TU replication, help delimit the contribution.
Critical weaknesses
  1. The evidence supports partial shared control more clearly than a unified mental-accounting mechanism. The contrastive personas jointly vary accounting, emotional framing and objective versus subjective evaluation. Removing one generic instruction direction and ablating under explicit personas does not uniquely identify the mechanism of unprompted bias. SC is not collinear with SG/IL, categorical-to-TU transfer changes WTP level rather than its defining venue gap, and held-out transfer is established most clearly for SG. The SC task also lacks the matched lost-cash/no-prior-cost contrast needed to isolate an effect of past accounting. These points limit the stronger representational and synthetic-respondent claims.
  2. Several inconsistencies affect the reliability of the central evidence. Appendix G and C.5 specify |alpha| <= 0.30, but B.3 and the 7/8 scorecard use an extended sweep; at -0.40, C.5 reports severe capability degradation. The explanation of near-perfect ROC-AUC by near-constant labels (p. 4, lines 203–205) confuses AUC with accuracy, and Figure S1 displays layer-ranked AUC/negative MSE despite its PCA-variance caption. Table S8 labels some arms ceiling/floor despite substantial room to move and counts Gemma-27B’s reversed IL amplification effect among steerable arms. Figure 3 shows a Gemma-2B TU panel whereas S8 marks it n/a. The TU sample-count product in F.1 is also incorrect. Together with per-generation inference on a small prompt panel, these issues prevent a consistent reconstruction of the key results and model comparisons.
Questions for authors
  1. Is Figure 2 actually ROC-AUC? Please also reconcile Figure S1 with its caption and specify the development-data rule for choosing layer bands.
  2. Please correct the headroom and sign criteria in Table S8 and distinguish predicted-direction success, reversed effects, nulls, and untested arms. How does the cross-model comparison change? Was Gemma-2B TU evaluated, as Figure 3 suggests, or not, as S8 states?
  3. Please reconcile the TU run-count factors and plotted sample sizes, and apply-once versus every-step settings. Do the transfer conclusions survive scenario-aware inference and the same statistical test for real and shuffled directions? Targeted validation rather than broad future-work requests:
  4. Can the authors verify semantic rather than answer-position transfer using label/person swaps, and isolate the SC effect using a matched lost-cash/no-prior-cost condition?
Reviewer_DpyBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explores whether mental accounting, the family of behavioral economics biases whereby people treat money as non fungible, tagging it by source and purpose is represented in LLM activations as a stable, causal, and steerable direction. The authors contributions are a shared steering, a cross-bias transfer, and the dissociation between decodability and controllability, Experimental study follows six models for steering, logistic regression with BH correction, random vector and capability preservation controls, cross-family replication, and prompt diversity generalization on Jaccard disjoint test families.

Critical strengths
  • The methodology distinguishes a probe can read it from an intervention can move it is the correct distinction for mechanistic interpretability, and the authors operationalize it across a 3×2 capacity×family grid. The result that small models carry readable bias signals yet resist steering (while large models steer) is consequential.

  • The cross-bias transfer matrix triangulated with geometry and head level decomposition is unusually thorough evidence for a shared component. The authors triangulate behavioral transfer 13/15 arms, vector geometry, and head-level decomposition. The clarity about SC being geometrically distinct yet functionally coupled joining the shared component through the transfer itself rather than through vector alignment is a more nuanced contribution.

Critical weaknesses
  • The evidence for a shared component across biases is estimated only in LLaMA-3-70B. The cross family are therefore much weaker, and the TU effect replicates in no other model. The paper overstates what two de-biasing arms in each of two models can support. More importantly, the shared axis notion is a within LLaMA-3-70B, whether the sharing structure itself generalizes is untested. This matters for the mechanistic aspects. Is the shared component a property of mental accounting, or a property of LLaMA-3-70B's particular pretraining mixture? This should be explained and justified.

  • Transaction utility is the dimension where the behavioral economics notion is sharpest, however, it is the least portable. It steers only in LLaMA-3-70B, and even there the de-biasing arm is marginal within the primary range, the gap effect is given only under amplification, and one of six shuffled vector control conditions leaks i.e. shuffle 44, ment arm, ρ\rho = +0.93, surviving BH. The abstract and conclusion mentioning "the analogous transaction-utility intervention modulates WTP" is an overclaim relative to the evidence. The mechanistic approach would be stronger if either demoted TU to a secondary, model specific finding, or provided a positive account of why TU alone fails to replicate.

  • The scale variable is confounded with training data, tokenizer, and instruction tuning across model releases, however, it weakens any causal case about parameter count.

Questions for authors
  • The transfer matrix is LLaMA-3-70B only. The authors should show even partially, whether any cross dimension transfer is assessable in Qwen2.5-72B or Gemma-2-27B at their de-biasable dimensions i.e. SG/fung and SC/fung?

  • Given the marginal primary range de-biasing, the amplification only gap effect, and the one leaking shuffled control, what is your posterior that TU is part of the same shared axis?

  • For the small models and for Qwen/Gemma, how much of the null is attributable to baseline saturation versus genuine absence of controllable structure?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.