acceptodds
Under review as a conference paper at ICLR 2027

A Formal Theory of Persona Selection as Bayesian Updating on the Persona Simplex

Abstract

The Persona Selection Model (PSM) proposes that pretraining teaches language models a distribution over personas, which further training and in-context evidence then update. Despite its popularity, the PSM leaves personas and persona selection formally undefined. In this work, we provide a formal theory of personas and persona selection. Drawing on existing work on Bayesian studies of in-context learning, we define a persona as a probability distribution over behavioral traits, and the model's belief as a probability distribution over personas, which we represent as a point on the probability simplex. With these definitions, we operationalize persona selection as the updating of this belief in response to evidence in-context. To test this theory, we finetune seven models across two families on single-persona documents generated from personas with known trait distributions, making the theory's predictions exactly computable. We then measure each model's belief from its expressed behavior and from its residual-stream activations as evidence accumulates in context, and compare both with these predictions. We find that the theoretical predictions largely hold: (1) finetuned models learn the prior over personas implied by their training corpus; (2) the model's belief over personas is encoded linearly on a simplex in the residual stream; (3) as evidence accumulates, these internal belief updates are approximately Bayesian, while expressed behavior updates are sub-Bayesian, discounting evidence more as context grows; and (4) updates in both behavior and the residual stream become more Bayesian as models grow, suggesting that our theory of persona selection becomes more accurate as models become more capable. More broadly, our theory turns persona selection into a testable hypothesis, and takes a step toward a more mathematical account of language model behavior.

Primary area:
interpretability and explainable AI
Keywords:
interpretability, representation geometry, personas

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RbqQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work formalizes personality selection as a finite mixture model: personality is a joint probability distribution over behavioral features, the model’s beliefs over different personalities correspond to mixture weights on the simplex, and contextual evidence updates these weights via Bayesian or discounted sub-Bayesian rules. The authors construct training data using three synthetic personalities and four categories of behavioral features, fine-tune seven models spanning two model families, and test theoretical predictions through output sampling, residual-flow decoding, and activation intervention. Experiments show that models can learn the co-occurrence structure of personality-related features; behavioral updates are well described by a two-parameter discount model; activation decoding aligns more closely with Bayesian predictions than behavioral inversion results; and interventions along personality directions can coherently alter multiple features. The main contribution is a controlled experimental framework for quantitatively testing hypotheses of personality selection, together with insights into potential discrepancies between internal representations and behavioral expression.

Critical strengths
  1. Translate the personality-selection hypothesis into concrete, falsifiable quantitative predictions. Knowledge of personality distributions and training mixture ratios allows the authors to predict how evidence affects unobserved features, while examining both marginal frequencies and cross-feature covariances. This moves the work beyond post-hoc explanations of "personality shifts" from a small number of outputs and constitutes the paper’s most important methodological value.

  2. Combine behavioral measurements, internal representations, and causal interventions with substantial out-of-sample validation. Experiments span two model families and seven model scales. In particular, raw-count analysis in Appendix B, cross-validation across conditions, and tests on new conditions with frozen parameters demonstrate that the predictive advantage of the discount model does not merely arise from in-sample fitting. Activation interventions further show that the discovered directions can control associated behaviors. Collectively these results support the research value of this framework, even though the mechanistic explanation remains to be strengthened.

Critical weaknesses
  1. Measurements of beliefs have not sufficiently ruled out emission model misspecification, which affects the interpretation of "sub-Bayesian updates." Equation (3) inverts personality weights from output frequencies using the emission matrix AA specified by the synthetic training data. This implicitly requires that the feature expression probabilities learned by the model match the training specification and remain stable across different contexts. If expression probabilities themselves change, the inferred weight shifts may conflate belief updates with changes in output expression. This concern has direct experimental grounding: Section 5.1 and Figure 3 show that the empirical cross-feature covariance under the null context is approximately 65% of the theoretical prediction and later drops to around 40%. This supports that the model learns personality-related structure, yet also demonstrates that the fixed mixture model exhibits notable bias for the joint behavioral distribution. Appendix B mentions the importance of projection residuals but does not clearly report their quantitative results. It therefore remains difficult to disentangle how much of the behavioral discounting reflects genuine under-updating of beliefs versus emission model misspecification. This is a critical issue affecting the core interpretation.

  2. Activation results support that posterior-related information can be read out and intervened on, but are insufficient to fully identify the "linear probabilistic belief" mechanism. Section 4.5 constructs personality vertices using conditions with high theoretical posteriors and builds a coordinate system over evaluation data; constrained decoding restricts results to the simplex, and uniform coordinates under the null context follow by construction. The authors’ unconstrained decoding, grouped cross-validation, and independent prompt-distribution checks mitigate this concern, yet a distinction remains between "Bayesian posteriors can be predicted from activations" and "the model actually uses these probabilities as mixture weights for behavior generation." Furthermore, similar probe performance for probability and logit objectives in Appendix B indicates that the coordinate interpretation is not uniquely determined. The linear mixing relation r=Awr=Aw in the main text also needs reconciliation with the sigmoidal behavioral readout in Appendix C: if the two readouts correspond to different variables, their mapping should be explicitly defined before assessing the gap between internal representations and behavior. Current evidence more strongly supports low-dimensional, intervenable personality-related representations, while claims about their probabilistic semantics and Bayesian mechanisms are overstated.

Questions for authors
  • Have you measured the model’s actual conditional trait distributions to verify that they match the designed emission matrix AA? Please report the projection residuals from Equation (3) and how they vary with context length. If AA is calibrated using independent data, do the sub-Bayesian discount and the internal–behavioral gap persist?
  • Does the quantitative advantage of the activation-based readout over the behavioral readout in matching the Bayesian posterior persist when vertex construction and layer selection are performed on separate calibration data and the decoder is then frozen? Could the independent-frame analysis in the appendix report this comparison directly, rather than only the correlation between the two decoders?
  • How should the linear mixture relation r=Awr=Aw in the main text be reconciled with the sigmoidal behavioral readout in Appendix C? Are the activation-decoded coordinates and the behaviorally inferred weights intended to represent the same latent variable? If not, what mapping relates them, and can that mapping predict behavior on held-out data?
Reviewer_ihbsOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper gives a formal interpretation of persona selection as Bayesian inference over latent personas. The authors construct synthetic personas with known trait distributions and study how seven fine-tuned LLMs learn and update these distributions. The main finding is that persona-related information can be decoded from model activations and appears to follow Bayesian predictions more closely than observable behavior. The paper also provides evidence from activation steering and model scaling.

Critical strengths
  • A useful formalization of persona selection. The main contribution is making an otherwise rather vague hypothesis testable. The formulation is simple, but it leads to concrete predictions about how different behavioral traits should change together.

  • Interesting evidence beyond behavioral correlations. The combination of Bayesian predictions, activation decoding, and steering is compelling. In particular, the gap between internal representations and expressed behavior is an interesting finding.

Critical weaknesses

The simplex representation is constructed using conditions selected according to Bayesian posterior predictions. Although the authors provide cross-validation and steering results, I am not entirely convinced that the decoded coordinates represent genuine probabilistic beliefs rather than contextual features correlated with the posterior.

Questions for authors
  1. Would the internal-versus-behavioral Bayesian agreement remain similar if the entire simplex construction and layer selection were performed on independent conditions?

  2. The evaluation includes mixed-persona contexts, while the Bayesian reference assumes a shared latent persona. How much do the conclusions depend on this mismatch?

Reviewer_MKvKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to provide a theory for persona model selection. A Bayesian-based method is used to interpret the model output. However, there are significant limitations in terms of the applicability of the theory. Experiments are conducted on only three personas (there could be more than 100 personas), which is a less challenging setting.

Critical strengths
  1. A formal principle for persona model selection could be useful in real scenarios.
  2. Bayesian-based selection provides good explanations.
Critical weaknesses
  1. The paper only discusses a fine-tuning-based personalization method, which is a major drawback. There are many other personalization methods, such as RAG, Steerling vector, pruned subnetwork, prompts, etc. The paper has not discussed them at all.
  2. The theory is not applicable to training-free personalization methods where the parameters are not changing.
  3. The paper only discussed 3 personas, which is an easy setting. There are more than 100 personas. Even in existing benchmarks, the number is much higher.
  4. Limited related work. Only 20 references, and most are from before 2025. There are more than 30 published works about personalization in top ML references.
  5. Writing: The author uses all the space on the 9th page for the conclusion and limitations, which makes the paper superficial. The fact is that the workload and technological contribution are limited.
Questions for authors

NA

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.