acceptodds
Under review as a conference paper at ICLR 2027

The Trait Structure of Language Model Personas

Abstract

Psychology suggests that traits are the basic units of personality, and that how a person converses and behaves depends on these traits. In this paper we study whether the same can be extended to language models and whether a model's personas are built from a set of simpler personality traits that act as primitives. We work with a set of 60 personality traits, and study personas that include everyday roles and characters. For each persona we find the internal activations the model uses when it takes on that persona and describe it as a mixture of traits. We then steer these mixtures inside the model and use a separate judge to read the answers without knowing how they were produced and find that each persona can be described by a small mixture of traits that hold about half of its internal direction. Adding the mixture to a model makes many personas recognisable in its answers, and removing it from a model playing a persona makes that persona fade. When we move the model from one persona to another, the model takes on the kind of character the new persona is. For sycophancy, adding a trait recipe increases endorsement of false claims, while removing it reduces giving in under pressure. Fitting ten traits to the refusal direction and subtracting them reduces refusal and increases harmful help on HarmBench and AdvBench datasets. Together, these results show that trait recipes capture a causally active part of personas and can shape safety relevant behaviour.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, AI safety

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_XhpYOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether persona directions in LLMs can be represented as combinations of a small number of interpretable trait directions. The authors construct 60 traits, 37 personas, and 108 scenarios, extract activation directions from three models, and identify “persona recipes” containing a median of approximately six traits. They then add, subtract, or swap these recipes to evaluate persona recognition, sycophancy, and refusal behavior.

Overall, I think the paper is well presented. The figures are visually appealing, and I expect that a poster based on this work would attract considerable attention at the conference.

However, my main concern is that the contribution is unclear. To my knowledge, prior work has already investigated a wide range of personas using similar approaches, including Personality Alignment of Large Language Models. It was not immediately apparent to me how this work goes beyond existing research.

Critical strengths

I appreciate the quality of the figures, the logical organization of the paper, and the detailed experiments across multiple models.

Critical weaknesses

My main concern is that the contribution appears incremental. CAA has already studied behavioral control through the addition and subtraction of activation directions; Persona Vectors has extracted and controlled trait directions; and The Assistant Axis has investigated the overall structure of persona space. I have also encountered compositional approaches in Personality Alignment of Large Language Models.

The authors need to explain more clearly what their method adds to this body of work. I understand that traits offer one way to represent personas, but alternative representations are also possible. The paper does not clearly establish what specific advantages the trait-based representation provides or how well those advantages are supported by the results. As a reader, this makes it difficult to assess the contribution.

Questions for authors

The trait dictionary contains correlations and linear dependencies. Using different synonyms, scenarios, or bases could therefore produce different recipes. Have you tested how sensitive the recipes and their behavioral effects are to changes in prompt wording?

3 comments
AuthorsOpenReview ↗

Response to reviewer XhpY

Thank you for your comments on the presentation and for asking us to clarify the contribution and its sensitivity.

Contribution and relation to prior work.

A trait decomposition makes several questions accessible. Which traits do two personas share? Which associations recur across models? How much of a persona direction can a trait description explain? Does changing a selected component change the resulting behaviour? Our paper brings these questions together in a common framework.

The relationship to the papers you identify is as follows. Thee CAA paper extracts the behavioural directions and use them for steering. The persona vectors paper monitors and controls the traits, and its SAE analysis examines those features associated with the respective trait directions. The Assistant Axis paper studies the geometry of the roles and traits and identifies directions that stabilise the Assistant persona. The Personality Alignment paper learns interventions that match individual preferences. Our paper studies wheteher these individual roles and named personas can be described through combinations of shared traits, and how those combinations then contribute to their behaviour.

Persona Vectors explicitly asks whether there exists a natural “persona basis”. We address this question by testing named traits as candidate components of persona representations. We extract the persona and trait directions separately, select combinations against a random dictionary reference, and measure both the part they explain and the remainder. Steering then tests whether the proposed description has behavioural consequences. This connects a readable account of a persona to measurable structure and intervention effects.

What the trait representation adds.

The representation provides four concrete forms of information.

  1. A common description of different personas. Each persona is represented by a small set of signed traits, with a median of six. This makes shared components and model differences directly comparable. Across the three models, 34 of 37 personas share at least one selected trait.

  2. A measure of how much trait language explains. The selected trait span captures, on average, 0.62 of a persona direction's length. These measurements establish where the trait description is informative and where substantial structure remains unexplained.

  3. Tests of the proposed components. Steering each trait up versus down changes its own blind rating more than the ratings of other traits, by an average of 4.0 points on a scale from 1 to 7. All 60 traits move in the intended direction. Across all 64 subsets of six selected traits for eight personas, an additive model explains a mean of 0.69 of the variance in recognition.

  4. A connection between representation and behaviour. In an analysis across the 37 personas, greater recipe coverage is associated with greater recognition when the recipe is added (Spearman correlation 0.39, 95% CI 0.06 to 0.66, p=0.016p = 0.016).

Stering with the whole persona direction remains stronger, as reported in the appendix. The decomposition adds access to the components, their shared structure and the limits of the trait explanation.

Anothe dictionary control using random words further tests this interpretation. In the evaluation, recognition falls to 8%, close to their random control at 7%. This supports the behavioural relevance of overlap with the trait span for these recipes.

Sensitivity to scenarios, wording and dictionary choice

We reran recipe selection under each change on Gemma 4 12B. For scenarios and wording, overlap is the number of shared signed traits divided by the number in either recipe. Dictionary overlap measures recovery of the original signed traits. For subsampling, the denominator includes only original traits still available in the smaller dictionary. The corresponding chance levels range from 0.06 to 0.11.

ChangeOverlapBehavioural evaluation
Two disjoint halves of the 108 situations0.91
Each instruction wording fitted separately0.52
50 subsamples containing 40 of the 60 traits0.83
Dictionary expanded to 240 words0.38
Every trait replaced by a synonym0.43Addition 40% versus 58% for the original. Removal leaves 33% versus 34%

The recipes are most stable across situation splits. The synonym experiment also tests whether changing the vocabulary changes the behavioural effect. Its recognition rates are similar to those of the original dictionary under the same fixed dose protocol and judge panel.

We hope these responses address your questions and clarify our contribution. If our replies leave anything unclear or raise further concerns, we would welcome your feedback and suggestions for additional experiments that would help address them. We are also working to obtain human annotations for a random subset of the model responses and hope to share the results before the rebuttal period ends

Thank you to the authors for their response. I have carefully read the full response, sentence by sentence, but my concerns remain.

I am still unsure in which scenarios I would choose this method over existing, well-established alternatives. The improvements are marginal, and the contribution of the work itself is difficult to distinguish from that of prior work.

AuthorsOpenReview ↗

Response to reviewer XhpY

We thank the reviewer for reading our response closely and for the follow up. The two points that were raised which are when one would use this approach and how it differs from prior work.

When one would use it. Our paper studies what a persona direction is made of, and the steering experiments serve as the causal test of that description. For steering strength alone the full persona direction remains the better choice, as the paper reports. The trait decomposition then answers questions that a single direction leaves open, and we see four such scenarios.

  1. Auditing a behaviour direction : A direction found for sycophancy or refusal can be read as a mixture of named traits. In the paper the sycophant recipes of all three models share Gullible, less Creative and less Analytical, and subtracting a mixture of traits fitted to the refusal direction raises answers without refusal from about 2.6% to 66.7%.
  2. Comparing models : Recipes fitted independently on each model are expressed in the same vocabulary, and 34 of the 37 personas share at least one selected trait across the three models.
  3. Knowing what trait steering can reach : The recipe covers 0.62 of a role direction and 0.39 of a named persona direction, and in our new analysis this coverage predicts how recognisable a persona becomes when its recipe is added.
  4. Acting on one named component : In the new experiments each trait direction moves its own rating by 4.0 points more than the ratings of other traits on a scale from 1 to 7, and subsets of a recipe combine approximately additively, with additive models explaining 0.69 of the variance.

**How it differs from prior work : ** We have listed the exact differences from the closest works in our reply under the Area Chair's comment. We would be grateful if the reviewer could name the specific papers in which these results already appear, so that we can compare against them directly.

Reviewer_ZPTKOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the extraction of personality traits in LLMs by the motivation of psychological studies. A set of 60 personality traits are used to add or fade personality. Evaluation is based on GPT 5.5 instead of human label.

Critical strengths

There is no much insight in this paper, TBH. I appreciate the author's efforts to construct a set of 60 personality traits. But this is an ML conference, and there is little to no technological contribution.

Critical weaknesses
  1. The technological contribution is very marginal.
  2. The vector extraction is not novel. Steering vectors have been widely used for personalization, and the paper does not discuss them at all.
  3. Table 2 is out of bounds, and the paper should be desk-rejected.
  4. Figures 1 and 2 appear to be AI-generated to me.
  5. Using GPT 5.5 as the judge is not a conclusive evaluation.
  6. The paper has not been compared with baselines published in ICLR 2026 and ICML 2026.
Questions for authors

My suggestion is to expand the paper to 9 pages. It is unlikely that a conference will accept a paper with 8.5 pages of content (AI USE STATEMENT, etc does not count the pape limit).

1 comment
AuthorsOpenReview ↗

Response to reviewer ZPTK

Thank you for the review. We address each point below.

Contribution and novelty of extraction (points 1 and 2). Our contribution is a compositional account of persona representations. It addresses how individual personas can be explained through shared trait components and whether those components causally shape their behaviour. This provides a common vocabulary for comparing personas, measuring how much their trait descriptions explain, and testing the contributions of the selected components.

We use established extraction procedures to obtain the directions being studied. The novelty lies in explaining their composition and testing that explanation through intervention. Extracting a direction for a concept leaves open which other concepts contribute to that direction and how their contributions affect behaviour. Our framework makes these relationships explicit and experimentally testable. The general response explains how this contribution relates to CAA, Persona Vectors, the Assistant Axis and Personality Alignment.

We will expand the discussion of steering for personalization, including Cao et al. (2024), Steerable Chatbots (2025) and SteerX (2025). Personalization studies how model behaviour can be aligned with a user's preferences or profile. Our study examines the composition of existing persona representations. We will make this distinction and the connections between these research questions clearer in the revision.

Table 2 (point 3) Thank you for catching this. Table 2 sits slightly above the top margin of page 6. The revision will fix its placement.

Figures 1 and 2 (point 4) We would like to bring to the attention of the reviewer that the figures were drawn by us and we used Apple Freeform on an iPad. We did not use any image generation model

GPT 5.5 as the only judge (point 5) We have addressed this concern by evaluating the main experiments with three additional judges from different vendors, GLM 5.3 Flash, DeepSeek V4.1 Flash and Qwen 3.8 Flash. For addition among distinct candidates, recipe recognition is 58/61/55 % for all three judges, compared with 8%, 7% and 8% for their respective random recipe controls. For removal, source recognition falls to 34% to 35% after subtracting the recipe, compared with 50% to 54% after subtracting a random recipe. The recipe advantage therefore persists across all three judges. The response to Reviewer XeaX reports their agreement and the detailed statistical comparisons.

Baselines from ICLR 2026 and ICML 2026 (point 6). We would appreciate references to the specific papers so that we can address the proposed comparisons directly. Our evaluation examines both steering effectiveness and the explanatory value of the trait decomposition.

For steering effectiveness, the appendix already compares each recipe with the full persona direction. The full direction provides stronger control, as reported in the paper. The trait representation additionally explains which named components contribute to that control and how persona representations relate through shared traits.

For the decomposition itself, we have added a random word dictionary baseline. This tests whether a dictionary of arbitrary words can explain and steer the same persona directions as effectively as the trait dictionary. These comparisons directly assess the two parts of our contribution. We will discuss the suggested methods against these same objectives once the references are specified.

Page length. The 9 page limit for ICLR is an upper bound, hence there is no requirement to use all nine pages. The revision will use the remaining space for the new experiments, expanded comparisons and a clearer explanation of the contribution.

We hope these responses address your questions and clarify our contribution. If our replies leave anything unclear or raise further concerns, we would welcome your feedback and suggestions for additional experiments that would help address them. We are also working to obtain human annotations for a random subset of the model responses and hope to share the results before the rebuttal period ends

Reviewer_XeaXOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether language-model personas can be decomposed into sparse combinations of more primitive personality-trait directions. The authors construct a dictionary of 60 traits and extract activation directions for 37 personas across three models. They fit each persona direction using a small set of selected traits, termed a “trait recipe,” finding a median of roughly six traits that captures about half of the persona direction.

The paper then tests whether these decompositions are behaviorally meaningful through activation steering. Adding trait recipes increases recognition of the corresponding persona, subtracting them from prompted personas reduces recognition, and exchanging recipe/core components shifts behavior toward target personas. The authors further apply the framework to safety-relevant behaviors, showing that a sycophancy recipe modulates false-claim endorsement and capitulation, while a ten-trait reconstruction of the refusal direction can substantially reduce refusal and increase harmful assistance. Overall, the work provides evidence that sparse trait mixtures capture a causally active, though incomplete, component of persona representations.

Critical strengths
  1. The paper provides a compelling compositional perspective on persona representations. Rather than treating each persona direction as an indivisible feature, it identifies sparse mixtures of reusable trait directions and demonstrates substantial consistency across three models. This is a useful conceptual and empirical step toward understanding the internal structure of persona representations.

  2. The causal intervention experiments are broad and convincing. The paper goes beyond correlational decomposition by testing addition, removal, and exchange of trait recipes, with matched random controls. The extension to sycophancy and refusal is particularly consequential, showing that the proposed decomposition captures components that influence safety-relevant behavior rather than only persona recognition.

Critical weaknesses

1.The interpretation of individual traits as meaningful primitives is not yet fully established. The trait dictionary is correlated and linearly dependent, so the decomposition is not unique, and the paper itself notes that individual coefficients cannot necessarily be interpreted causally. The interventions establish that the joint recipe is behaviorally active, but not that the selected traits constitute a uniquely meaningful or mechanistically privileged decomposition. This weakens the strongest interpretation of traits as underlying “primitives.”

2.Several evaluation choices make the reported effect sizes somewhat difficult to interpret. Persona recognition relies entirely on a single LLM judge and fixed candidate lists, without human or alternative-judge validation. In addition, the creation results report the best of three steering doses rather than a dose fixed prior to evaluation. These choices do not invalidate the main qualitative results, especially given the strong random controls, but stronger independent evaluation and pre-specified intervention strengths would improve confidence in the quantitative conclusions.

Questions for authors

1.How stable are the selected trait recipes to changes in the trait dictionary? For example, if the dictionary is expanded, subsampled, or replaced with semantically related adjectives, do the same underlying trait concepts remain selected and does the resulting recipe retain similar causal effectiveness? This would materially affect how strongly I interpret the selected traits as meaningful primitives rather than one useful basis for the persona subspace.

2.Have the authors evaluated persona recognition with an independent judge model or human annotators? Given that GPT-5.5 is used for every reported recognition rate, evidence that the main creation/removal effects persist under a substantially different evaluator would increase my confidence in the behavioral validation.

3.For the creation experiments, how much do conclusions change if the steering dose is selected entirely on calibration data and fixed before evaluating the 24 held-out scenes, rather than reporting the best of three doses? I would be interested in whether the headline recipe-versus-random gap remains similar under this stricter protocol.

1 comment
AuthorsOpenReview ↗

Response to reviewer XeaX

We thank the reviewer for the careful review and suggestions. We ran the three analyses that the reviewer asked for, and we report each of them below.

Q1. Stability to the dictionary. To test this we changed the dictionary in three ways and reran the full selection procedure each time. We then measured the overlap with the original recipe, which is the share of its still available traits that come back with the same sign.

Dictionary changeOverlapChance
50 random subsamples of 40 of the 60 traits0.830.11
Synonym dictionary, one synonym per trait0.430.09
The 60 traits plus 180 adjectives from a published personality inventory0.380.06

The recipes are stable to the prompts as well. Here the overlap is the number of signed traits that two recipes share, divided by the number in either. It is 0.91 when the recipes are fitted on two disjoint halves of the 108 situations, and 0.52 against a chance level of 0.07 when they are fitted separately on our two instruction wordings.

Under synonyms the recipe keeps much of its causal effect. Compared with the original recipe in the paper, the synonym recipe is 18 points lower for adding (40% against 58%) and leaves 7 points less for removing (33% against 40%). Before judging we set a test which asked that the synonym recipe's advantage over its random control fall at most 10 points below that of the original recipe judged in the same run. The test passes for removing, and for adding with similar candidates (p < 0.016). The table gives recognition in percent with distinct / similar candidates, where each random word is represented through the part of its direction that lies outside the trait directions.

Recipe built fromAddingRemoving (lower means more removed)
Original 60 traits, as reported in the paper58 / 2640 / 26
Synonyms40 / 2033 / 20
Random words8 / 372 / 57
Random recipe control8 / 452 / 42
No intervention6 / 3 (unsteered)97 / 86 (prompted)

Primitives or one useful basis. We think the evidence supports a position between the two. Each trait direction carries its named meaning in isolation, is reused across personas and models and combines roughly additively, and these three properties are what we mean by a primitive. We agree that the 60 traits are one basis among several possible ones. The synonym recipes recover much of the behavioural effect through partly different words, which is what one would expect if the primitive is the trait concept and the word is a label for it. Recipes built from random words steer at 8% against 58% for traits in the paper, level with a 7% control. The causally active part of a persona direction thus lies in the subspace that the trait concepts span, and directions outside it add nothing.

Q2. Independent judge. We judged everything again with three models from three vendors, none of which is the paper's judge. The pairwise agreement among the three is 78 to 82% (Cohen's kappa 0.62 to 0.72). We did not collect human annotations.

Recognition with distinct candidates (%)GLMDeepSeekQwen
Adding, recipe / random recipe / unsteered58 / 8 / 661 / 7 / 655 / 8 / 7
Removing, prompted / minus random / minus recipe97 / 53 / 3497 / 54 / 3497 / 50 / 35

Q3. Dose fixed before evaluation. We first took the paper's own judge with a single dose calibrated on separate scenes, and the gap between the recipe and a random recipe remains.

ModelBest of three (paper)Fixed doseRandom recipeGap, 95% CI
Gemma 4 12B58511140 [29, 52]
Qwen3 32B5147642 [30, 53]
Gemma 4 26B52521438 [26, 52]

All the new experiments then use one dose set in advance, with no choice among doses and no persona dropped.

W1. Uniqueness of the decomposition. We agree that the decomposition is not unique and that the paper did not suggest otherwise, and Q1 above sets out our position. What was missing is supplied by the single trait experiments, which show that each trait direction carries its named meaning and contributes roughly additively. This is the claim we attach to the word primitives.

We hope these responses address your questions and clarify our contribution. If our replies leave anything unclear or raise further concerns, we would welcome your feedback and suggestions for additional experiments that would help address them. We are also working to obtain human annotations for a random subset of the model responses and hope to share the results before the rebuttal period ends

Reviewer_e4bUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether language-model persona representations can be described using a shared vocabulary of personality traits. Across three models, the authors extract activation directions for 60 traits and 37 personas, select compact trait mixtures using Shapley-based attribution and random-dictionary thresholds, and report that the selected spans capture approximately 47–56% of persona-direction length on average. Adding or subtracting these mixtures changes persona recognition, while combined interventions on trait mixtures and dictionary residuals shift recognition between personas; additional experiments examine sycophancy and refusal. The evidence supports the narrower conclusion that these fitted trait mixtures provide useful directions for controlling broad persona resemblance, while leaving the interpretation of individual traits and the remaining residual substantially unresolved.

Critical strengths
  1. The paper asks a useful question beyond whether a whole-persona steering vector exists.
  2. The experimental design contains several thoughtful controls.
Critical weaknesses
  1. Important inconsistencies: (1) Section 3.4 describes the main exchange as adding 10% of the target core. However, the results in Table 4 differ from the corresponding 10% rows in Table 12:
ModelMain Table 4: target / source recognitionTable 12, 10% target core: target / source recognition
Qwen3 32B71% / 4%66% / 11%
Gemma 4 26B-A4B69% / 1%56% / 25%
  1. Joint steering establishes behavioral influence, but does not yet identify individual traits as causal constituents:

The main behavioral interventions manipulate the entire fitted recipe. These experiments establish that the combined direction affects recognition. They do not establish that each selected adjective has the intended behavioral meaning, that its Shapley share measures causal importance, or that the traits compose as independently meaningful components.

Questions for authors

See above

1 comment
AuthorsOpenReview ↗

Response to reviewer e4bU

We thank the reviewer for the careful review and questions, and we address each of them below.

W1. Tables 4 and 12 disagree. The two tables use different removal operators and our text did not explain this difference correctly. Table 4 removes the source persona through plain subtraction. The 10% rows of Table 12 use the whitened operator, which sets the source coordinate to its group mean. Section 3.4 and Appendix E describe these as the same intervention, so we will correct both descriptions and specify the operator used in each table in the final draft. The values below give target / source recognition on 40 pairs.

ModelTable 4 (plain subtraction)Table 12, 10% row (whitened)
Gemma 4 12B59 / 1572 / 17
Qwen3 32B71 / 466 / 11
Gemma 4 26B69 / 156 / 25

W2. Joint steering does not identify individual traits as causal constituents. The review raises three questions about the individual traits and we ran a new experiment for each of them. These experiments use Gemma 4 12B and 24 scenes, with all 37 personas included in the component tests. Each persona gets one steering dose, which is fixed on calibration scenes before evaluation. Three judges then score the answers, GLM 5.3 Flash, DeepSeek V4.1 Flash and Qwen 3.8 Flash, all different from the paper's judge. The main tests were specified before judging and their p values are Holm corrected.

The first question is whether each selected adjective has its intended behavioural meaning. We steered each of the 60 trait directions alone, both up and down, and the judges rated the answers blind for the steered trait and for three other traits. We then compared the change between steering up and down for the intended trait with the corresponding changes for the other traits. On a scale from 1 to 7 the intended trait's rating changes by 4.0 points more on average (95% CI 3.5 to 4.5), and all 60 traits move in the intended direction.

The second is whether the Shapley share measures causal importance. For every persona and every trait in its recipe we omitted that trait while keeping the other coefficients fixed, and then examined whether the trait's Shapley share was related to the loss of steering effectiveness. The mean within persona rank correlation is 0.39 for addition (CI 0.24 to 0.54) and 0.29 for removal (CI 0.12 to 0.45). The shares are thus informative about the measured contributions of individual traits to the recipe's effect.

The third is whether the traits combine approximately additively. For 8 personas we steered with all 64 subsets of the top six traits. Additive models with one term per trait explain a mean of 0.69 of the variance in recognition across these personas, against approximately 0.10 under permutation (p = 0.0011 after Holm correction). The mean absolute pairwise interaction is 6 percentage points.

The ablation experiments also provide evidence that the selected components contribute to steering. Omitting a trait while holding the other coefficients fixed reduces effectiveness by an average of 5.7 percentage points for addition (95% CI 2.8 to 8.6) and 7.7 points for removal (95% CI 4.2 to 11.6). For removal this means that recognition of the source persona increases. Replacing the omitted trait with a random trait of equal weight also reduces effectiveness relative to the intact recipe, by 8.1 and 6.4 points respectively, so the selected traits outperform these random substitutes on average. The planned comparison did not establish that omitting a trait causes a greater loss than replacing it with a random trait.

Together with the experiments on individual trait meanings and composition, these results support our interpretation of the selected traits as meaningful components that contribute on average and combine approximately additively.

We hope these responses address your questions and clarify our contribution. If our replies leave anything unclear or raise further concerns, we would welcome your feedback and suggestions for additional experiments that would help address them. We are also working to obtain human annotations for a random subset of the model responses and hope to share the results before the rebuttal period ends

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to the Reviewers

We thank the reviewers for the careful review, questions and suggestions. The reviews altogether raise questions about what the contribution of the papers beyond the existing steering methods and explainability of the selected traits. We aim to clarify that contribution here and through individual replies we address the other questions raised through new experiments and their corresponding results.

Our paper asks " how the traits that are used to describe a persona are related to the representation of the persona inside the model ". To address this we fit extract the persona directions separately using a shared vocabulary of trait directions. This gives each persona a description whose components can then be compared, and allows us to measure and test them through interventions.

The cited papers approach related questions at different levels. Thee CAA paper extracts the behavioural directions and use them for steering. The persona vectors paper monitors and controls the traits, and its SAE analysis examines those features associated with the respective trait directions. The Assistant Axis paper studies the geometry of the roles and traits and identifies directions that stabilise the Assistant persona. The Personality Alignment paper learns interventions that match individual preferences. Our paper studies wheteher these individual roles and named personas can be described through combinations of shared traits, and how those combinations then contribute to their behaviour.

Persona Vectors also asks how the space of personas should be characterised.

“Another natural question is whether we could use our methods to characterize the space of all personas. How high-dimensional is it, and does there exist a natural ‘persona basis’? Do correlations between persona vectors predict co-expression of the corresponding traits? Are some personality traits less accessible using linear methods? We expect that future work will enrich our mechanistic understanding of model personas even further.”

Our paper then develops a compositional and mechanistic account of the persona representations. We extract the persona and trait directions independently, then fit each individual roles and named personas through a shared vocabulary of traits. This allows us to give the proposed persona basis an explicit and testable form and turns the proposed relationship between traits and personas into a quantitative hypothesis about their representations. The resulting descriptions thus allow recipes fitted independently on different models to be compared in common semantic terms. They make it possible to examine whether models associate a persona with similar traits and where their descriptions then differ.

We thus use the word “primitives” in this compositional sense where the relevant properties are identifiable meaning, reuse across persona descriptions and approximately additive composition. This representation accommodates both the correlated traits and alternative recipes. Its explanatory scope is determined by the coverage of the fitted mixture and the behavioural effects of intervening on its components. This gives the term an operational meaning that can be evaluated experimentally.

This connection between decomposition and intervention is central to our contribution. It allows us to evaluate the traits as behavioural constituents of personas and to examine how the trait mixtures influence safety related behaviours such as sycophancy and refusal. The new experiments test the individual components, their composition and their sensitivity to the dictionary and evaluation choices. We describe these experiments in the replies to each reviewer.

We hope these responses address your questions and clarify our contribution. If our replies leave anything unclear or raise further concerns, we would welcome your feedback and suggestions for additional experiments that would help address them. We are also working to obtain human annotations for a random subset of the model responses and hope to share the results before the rebuttal period ends

AuthorsOpenReview ↗

Final remark from authors

We thank the reviewers and the Area Chair for their comments, suggestions and time throughout the discussion. As the discussion closes we would like to state our contribution once more in plain terms.

We do not claim to have found a new steering method. Instead, we used already existing methods to find a more important result, which is that a persona direction has structure at a finer level than the persona itself. This structure forms a primitive mechanistic basis, where we show that personas are made from combinations of traits that generalise across characters and models.

We test this basis using random dictionaries, synonym dictinoaries and very strict random controls. We also show that this finding applies to steering, through vector arithmetic on personas, and to AI safety, by extending it to refusal and sycophancy across three models from two families. The persona results also hold with different judges.

We have compared our work against papers published at the top conferences in the past few years and stated the exact differences in our replies

We would like to note that while novelty lies in developing new tools and methodologies, it also equally lies in using existing ones to gain a deeper understanding of a topic. Much of interpretability research has advanced through new findings obtained with established methods, and we hope the paper will be assessed in that light.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.