acceptodds
Under review as a conference paper at ICLR 2027

Navigating the OCEAN: An Atlas of Big Five Personality Representations in Language Models

Abstract

Personality psychology studies people through both shared, population-level trait dimensions and the distinctive patterns that characterize unique individuals. The Big Five personality model measures an individual's personality profile across five core dimensions (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism). Language-model research frequently uses them as five coarse dials, tuned and evaluated on surface-level alignment with trait labels. We ask if an open-weight model can match one person's measured Big Five profile, down to the facet, by navigating its rich network of trait representations, as fidelity to this static snapshot is a first step toward modeling dynamic individuals. Across five open-weight models and 420 people, we examine six distinct routes to personality representations, extracted during tasks such as interpreting behavior, reasoning about traits, and generating text. Directions for the same facet align across routes, far more than directions for different facets. We quantify agreement with each person using Lin's concordance correlation coefficient since a model can rank a population well while exaggerating everyone in it. Detailed natural-language profiles already reach high levels of agreement with a person's questionnaire scores (facet CCC .82 to .87), and lay readers recognize their broad domains. In open-ended autobiographical writing, some of that fidelity erodes in every model, though a targeted steering correction reliably restores part of it in smaller models. Profiles shift behavior across four tasks, from resource allocation to moral dilemmas, and steering by itself moves risk-taking, emotional appraisal, and value trade-offs toward each person's traits. Personality interventions produce clear, predictable tendencies, yet amplify them well beyond their human counterparts. To give researchers a tool for studying personality, we fine-tune an open-weight model to write and read psychometric profiles. Combining one writer with two readers yields life stories as faithful as our best steering-corrected ones, although none of the models were trained on life stories. Together, these findings support an atlas of interconnected personality representations and show where composing and calibrating them facet by facet improves fidelity to an individual’s measured profile.

Primary area:
interpretability and explainable AI
Keywords:
language model personality, mechanistic interpretability, activation steering, psychometrics, individual fidelity, representation learning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_1mJpOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how Big Five personality traits are represented and controlled in language models. The authors extract facet-level activation directions from different tasks and test their effects on text generation and behavioral choices. Using personality profiles from 420 individuals, they compare written portraits, activation steering, and their combination. They find that directions for the same facet tend to align across tasks, while steering one facet also affects others. Targeted correction improves personality fidelity in some settings, particularly for smaller models.

Critical strengths
  1. The facet-level analysis is useful. The authors compare activation directions across tasks and examine how steering one trait affects others. These results help us understand the relationships among personality representations in language models.

  2. The experiments cover several models, personality measures, and behavioral tasks. The authors also report negative results, including the weak performance of steering alone and the exaggerated behavioral effects compared with humans.

Critical weaknesses
  1. The added value of the proposed steering method is unclear. In Table 1, steering alone performs much worse than written portraits. Targeted correction helps mainly in smaller models. The paper lacks direct comparisons with domain-level steering and simpler prompt-based correction methods. It is therefore difficult to judge whether the more complex approach offers a meaningful advantage.

  2. The evidence for fine-grained personality control remains limited. The evaluation uses selected human personality profiles but does not test whether the method works for unusual or artificially constructed facet combinations. This matters because steering one facet also changes others. In addition, personality fidelity is mainly evaluated by LLM readers, with limited human validation at the facet level.

Questions for authors
  1. What is facet CCC when PAPI read-back is scored only on the 180 held-out IPIP-NEO items, for portraits, portrait-only stories and corrected stories?
  2. If the correction vector is constructed with one reader (e.g., Gemini) and evaluated with the others, does the improvement survive? A cross-reader design would address the circularity concern directly.
  3. For the amplification result, are human trait–behavior slopes corrected for the unreliability of both the questionnaire and the single-shot behavioral responses? Models produce near-deterministic choices, so part of the gap could come from response noise in humans.
  4. How much do the conclusions change when steering conditions are compared on valid stories only, with matched valid rates?
  5. Is the released adapter trained on data whose licenses allow redistribution of derived models? Persona-E2 is CC BY-NC-SA; please confirm which cohorts entered training.
5 comments

Response to Reviewer 1mJp (Part 1 of 3)

Thank you for a careful review, for the cross-reader design, which tests the circularity concern head-on, and for crediting our negative results.

W1. The added value of steering is unclear: steering alone does much worse than written portraits, the correction helps mainly the smaller models, and there's no comparison with domain-level steering or simpler prompt-based correction. Your reading is right, and your questions changed how we describe the paper.

Prompt-based correction (Appendix Q). We gave the model the same reader feedback as a written note in the prompt instead of a steering vector. On the larger models the note does at least as well (Gemma4-31B .689 against .683–.684; Qwen3.5-27B .675 against .588–.643, facet CCC). On two of the three smaller models the steering correction still wins clearly, by .06 to .13; on the third they're level.

What steering adds and what it doesn't. Adding the paper's profile steering on top of the written note makes the stories carry less of each person, not more. Replacing each person's correction with the cohort's average removes the gain, so what helps is each person's own trait-level adjustment, not a single cohort-wide shift. On Gemma4-31B, our fine-tuned adapter given the note produces the most faithful stories of any Gemma4-31B condition, with no steering at all, and on Gemini 4 Carbon, a closed model where steering isn't possible, the same note lifts facet agreement from .69 to .77 (Appendix Q).

Domain-level steering (Appendix R). When we steer with five domain directions instead of thirty facet directions, with the same models, readers and push size, the two are within .03 of each other in two of six comparisons, facet steering is better in three, and domain steering in one. What the thirty directions add is a faint trace of each person's quirks, which domain steering doesn't carry at all.

So we now separate the two roles: steering is a tool for studying how personality is represented (directions found from different tasks align, and pushing on them changes what the model writes in the predicted way), and a reader's feedback written into the prompt is a practical fix, needing no access to activations. In the smaller models, the steering correction still earns its place. The paper was never meant to advocate for steering: it was meant to settle what that technique does and doesn't add, and to show how far high-fidelity, calibrated prompting can go. We hope more researchers adopt calibrated prompting, partly because it lets us study much larger models, closed ones included.

Response to Reviewer 1mJp (Part 2 of 3)

W2. Fine-grained control is limited: only real profiles were tested, steering one facet changes others, and evaluation leans on AI readers with limited human validation at the facet level.

Unusual combinations (Appendix R). We built 300 synthetic people with ordinary facet levels in rare combinations and compared them with matched real people. Readers recover the off-pattern detail from plain descriptions about as well for synthetic people as for real ones. Stories written from portraits keep a good share of it (about a third), and steering alone keeps almost none. Rare combinations of ordinary facet levels are ordinary psychology: facets within a domain separate into distinct aspects (DeYoung et al., 2007), and stable person-specific characteristics sit below the facet level (Mõttus et al., 2017). They're also hard to get from standard language-model outputs (Paglieri et al., 2026).

Steering one facet changes others. Yes, and it limits steering, as you suspected; work on trait-level steering finds the same coupling in other models (Bhandari et al., 2026), as does our replication of PERSONA in Appendix G of the submission, where steering toward compassion also raised Sociability and Energy Level. In our data it shows three ways: steering alone keeps almost none of the off-pattern detail for synthetic people, domain-level steering carries no quirks at all, and trait adapters released by others carry none of the person's quirks.

AI readers and human validation. During the discussion we ran two studies with lay raters from Prolific. Given written portraits, they rated a person's standout facet and a matched control without knowing which was which, and got the order right 72% of the time across two independent batches (ties counted half; interval .61–.82). Given stories our adapter wrote, they chose the narrator's actual description over a decoy sharing the same five broad traits but not the finer details 71% of the time pooled (.54–.86), though the batches differed sharply (Appendix M). Both bear on portraits and adapter-written stories rather than on the activation correction. A professor who publishes in psychometrics and personality, recruited from outside our team, has agreed to rate a bundle blind on all 30 facets (Appendix M). Beyond people, ridge regressions on text embeddings or TF-IDF word features, with no generative model in the loop, recover each person's facets and quirks, and the stories still carry the person on the 180 items the writer never saw (Appendices L, N). The AI readers themselves agree on who is who and differ mainly in stable per-facet calibration (Appendix N).

References

  • DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. https://doi.org/10.1037/0022-3514.93.5.880
  • Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100
  • Paglieri, D., Cross, L., Cunningham, W. A., Leibo, J. Z., & Vezhnevets, A. S. (2026). Persona Generators: Generating diverse synthetic personas for arbitrary contexts. arXiv:2602.03545.
  • Bhandari, P., Naseem, U., & Nasim, M. (2026). Do personality traits interfere? Geometric limitations of steering in large language models. Findings of ACL 2026, 29284–29293. https://doi.org/10.18653/v1/2026.findings-acl.1463

Response to Reviewer 1mJp (Part 3 of 3)

Q1. Facet CCC on the 180 held-out IPIP-NEO items, for portraits, portrait-only stories and corrected stories. Scored only on the 180 items the writer never saw (the reader completes those items as the narrator; Table 1 instead asks for facet ratings directly), on stories that finished (Appendix N): the written portraits score .583; portrait-only stories .490 (Gemma4-31B), .489 (Qwen3.5-27B), .450 (Gemma4-E4B), .478 (Qwen3.5-4B) and .370 (Qwen3-8B); corrected stories .525, .507, .508, .514 and .500 in the same order. Quirks hold up on these items too: above chance for the portraits and the two larger models' stories (Appendix N).

Q2. If the correction is built with one reader and judged by the others, does the improvement survive? In the smaller models, yes: it improves things under each of the two readers that played no part in building it, in all nine comparisons. In the larger models the gains are smaller and depend on the reader (Appendix Q). This cross-reader check was in Appendix D of our submission too, and we should have pointed to it from the main text. We also built the written note from five readers' pooled feedback and judged it only with two readers outside the pool, on all 300 PAPI people. The pooled note does as well as the single-reader note on Gemma4-31B and a little better on Muse Glimmer 30B (facet CCC .718 against .700, though one pooled reader shares Glimmer's developer; Appendix Q).

Q3. Are the human trait–behavior slopes corrected for unreliability? For the questionnaire, yes. Treating its error as classical and using the published alpha of .90 for the IPIP-NEO-120 Neuroticism scale (Johnson, 2014; not measured in this cohort), the 6.2-point difference between the most and least neurotic quarter of people (9 of 36 each) becomes 6.8 over the same spread of scores the models were given [95% interval −3.0 to 16.6; 6.5 if the quarters are formed on error-free scores]. For the behavioral rates, no outcome correction is needed: error in a rate with zero mean noise widens the interval without biasing the expected difference, and each rate averages 384 events (split-half .96). The models' differences are 20.4 to 38.7 points, and the gap stays positive in every paired interval (smallest: Gemma4-E4B, +3 to +25). The human interval is wide because the cohort is 36 people aged 18 to 25, so we do not generalize beyond it. On the model side, sampling at a higher temperature doesn't shrink the gap (Appendix T).

Q4. Valid stories only, with matched valid rates. The submission's Appendix D gave each route's finish rate and its valid-only score under the primary reader, though we should have surfaced them. We now score every comparison three ways: the paper's rule, finished stories only, and with every unfinished story rerun once (Appendix P). On matched finished stories, where the same people finished in both arms, no overall ordering between methods changes, and Gemma4-31B moves by at most .013. That comparison holds only for those people: looping is more common for some personalities and under larger pushes, so it doesn't settle how the methods compare across the whole cohort. Most models finish 85–100% of their stories; Qwen3.5-27B under steering finishes about 80%, mostly because it loops (Appendix P).

Q5. Licenses of the adapter's training data. Persona-E² did not enter training; it was used only in the behavior tests. The adapter (Appendix J) was trained on 320 people: 160 from the Eugene-Springfield Community Sample, released under CC0, and 160 from the BESSI development samples (Soto et al., 2022; Sewell et al., 2022), which completed the BFI-2 and are public on OSF with no license shown. We checked this against the adapter's training files. After your question, the missing license made us uneasy, so we built a new adapter, OCEAN-S2, trained only on synthetic people generated from CC BY and CC0 samples, with training text from open-weight models whose licenses don't restrict that use. Evaluated on the 420 with the same portraits, prompt, sampler and scorer, it clearly beats the untuned model in 6 of 9 reader cells, including all three on BFI-2 (primary reader: facet CCC .716 to .779; OCEAN-M03, .792). OCEAN-S2 is in its own anonymous repository now (Appendix J); the original adapter, OCEAN-M03, stays in its original anonymous repository for reviewers during the discussion only.

Presentation. You're right that the submission asked too much of its reader in places: with five models, six routes and three readers, the main thread was easy to lose. We've aimed for a clearer revision. A new Section 4.5 on the tenth page lays out the main checks and the thesis in plain terms, the appendix opens with a dated change list and a guide to the key measures, and every change is in blue. We'll keep improving clarity through the discussion, and if some passages made you work harder than they should have, we'd be glad to know which.

Reviewer_mBHVOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates if an open-weight model can match on person’s measured Big Five profile, down to the facet. Starting with the actual human data, this paper tests five open-weight models to test how personality facets are represented in their internal activations and whether these representations can be controlled through activation steering. They also thested how similar is the generated life stories to the original personality profiles.

Critical strengths

Most of the previous work focuses on five broad domains, this paper uses personality representation at the facet level.

The use of Lin’s CCC, multiple LLM readers make this results more convincing.

Critical weaknesses

The main experiments does guarantee there are psychometric fidelity in this setting. However, this does not mean they have an behavioral fidelity. Without behavioral fidelity main contribution of this paper seems a bit limited.

Fact-level evaluation is done by llm readers which can make a circularity of the evaluation at some point. Since I believe LLM has some kind of biases interpreting personality traits, this could have potentially affected the reliability of evaluation.

Questions for authors

Evaluating on IPIP-NEO alone does not make this useful. How about testing a downstream tasks such as “Simbench” or “BehaviorBench” or at least other psychometric benchmarks designed for llm such as “Value Portrait” or “Value Bench”.

How about adding some independent human evaluation for the facet-level evaluation. For example, human-written narratives vs. LLM-generated narratives.

4 comments

Response to Reviewer mBHV (Part 1 of 2)

Thank you for the review, and for valuing the facet-level angle and the use of concordance with several readers.

W1. Psychometric fidelity isn't behavioral fidelity, so the contribution seems limited. We agree that one doesn't imply the other, and in our data it doesn't follow strongly. Facet-level levers on a model's personality matter, but we don't think they are the path to a truly idiosyncratic person, and drawing that line clearly is one of the main things this revision does (Section 4.5). We added three tests where each person's own self-reports are on record (Appendix S). Personas built from a person's own profile predict that person's reports modestly (r .11–.12), and on everyday acts, where facet scores exist, facet portraits beat broad-trait portraits. BehaviorBench adds that personas built from 60 real people's facet portraits behave in line with each person's measured traits, on all five. And personas exaggerate trait-driven behavior well beyond people's (Appendix T).

So we'd put the contribution this way. Models can carry a particular person's facet-level profile, including the parts their broad traits don't predict, and that holds up with human readers, readers with no generative model and questions the writer never saw. Behavior is a separate and weaker result, and the revised paper now presents it that way (Section 4.5).

W2. Facet-level evaluation by AI readers can be circular, and AI readers have biases about personality. They do have biases, and we tested around them:

  • People. Lay raters on Prolific, recruited during the discussion, read written portraits and blind-rated each person's standout facet beside a matched control; across two independent batches they ordered the pair correctly 72% of the time (ties as half; interval .61–.82). In a second study, on stories our adapter wrote, raters picked the narrator's real description over a decoy that matched the five broad traits but not the finer details, 71% of the time pooled (.54–.86), with the two batches far apart (Appendix M). The two studies support the portrait contrasts and the adapter's stories, not the activation correction itself. We've also recruited an outside expert, a professor who publishes in psychometrics and personality, to rate a bundle blind on all 30 facets.
  • Regression readers. Ridge regressions on text embeddings, and a TF-IDF ridge model, each trained only on other people, recover each person's facets and quirks (Appendix N).
  • Questions the writer never saw. The stories still carry the person on the 180 held-out IPIP-NEO items (Appendix N).
  • A correction judged by readers that didn't build it. In the smaller models the correction's improvement holds under each of the two readers that played no part in building it (Appendix D of the submission had this; Appendix Q adds to it).
  • Quirks. A prediction made linearly from the five broad domains should show zero quirks correlation on average, under the published norms, and portrait-only stories carry quirks for nearly every model and reader (Appendix L).

Response to Reviewer mBHV (Part 2 of 2)

Q1. "Evaluating on IPIP-NEO alone does not make this useful"; how about SimBench, BehaviorBench, Value Portrait or ValueBench? Thank you; these gave us a direct test against people's own answers. The paper uses two questionnaires, IPIP-NEO and BFI-2. For each, the useful question was whether it can test a particular person, since our personas come from real people's profiles. Of the four you named, Value Portrait (Han et al., 2025) has individual raters, so we ran it and added two datasets with individual-level data: everyday acts from the Eugene-Springfield Community Sample (Goldberg & Saucier, 2016) and policy answers from Twin-2K-500 (Toubia et al., 2025). BehaviorBench (Pu et al., 2026) has no human choices, so we ran it to test whether model choices follow the supplied traits, not to match individuals. SimBench (Hu et al., 2026) has no individual participant data, and ValueBench (Ren et al., 2024) measures a model's own values without human respondents, so neither can score a persona against the person it came from. The other BehaviorBench (Huang et al., 2026) builds its Big Five tasks on questionnaire responses, the setting you asked us to go beyond.

On Pu et al.'s BehaviorBench, personas built from 60 real people's facet portraits behave in line with each person's measured traits on all five, most strongly for Conscientiousness and self-control (r .82). Its single-number persona prompts barely move Neuroticism-driven behavior; real portraits track it.

Q2. An independent human evaluation of facets, for example human-written against model-written narratives. We added two human studies at the level of facets (Appendix M, and Part 1 above), and an externally recruited expert, a professor who publishes in psychometrics and personality, has agreed to rate a bundle of our profiles and stories blind on all 30 facets. We'll make every effort to post the ratings by November 17.

On human-written narratives: since starting this project we've tried repeatedly to get life stories linked to psychometric data, and couldn't; recent studies pairing people's own narratives or AI chats with trait scores share the full texts only on reasonable request (Chou et al., 2026; Lametti et al., 2026). So we used the closest kinds of people's own words we could get: emotional vignettes and conversations. In crowd-enVENT (Troiano et al., 2023), 110 people wrote ten or more short accounts of emotional events and took a brief Big Five questionnaire; from their own words, the three readers' consensus recovers their traits at r .22 on average. In PARSEL (Hrkalovic et al., 2025), 290 people had short paired conversations during cooperative tasks and took the HEXACO-60; from their conversations (about 1,040 words of their own), the consensus recovers r .27 on average, every domain above chance, and .28 from their own lines alone (Appendix S). Other work, with other readers, instruments and texts, reports r .37 from chats with an AI (Lametti et al., 2026) and .44–.48 from spoken thought and video diaries (Wright et al., 2026). From accounts a model wrote from the crowd-enVENT people's scores, the same readers recover .73. That comparison is uneven by design (the model was given the scores and wrote about 45% more words; the human accounts average 13 words each, and people's own scores come from the 10-item TIPI), so it doesn't isolate behavioral amplification. Narratives paired with facet-level scores remain rare, and we're excited to work on them.

References

  • Pu, T., et al. (2026). IJCAI-26, 5910–5918. doi:10.24963/ijcai.2026/658
  • Han, J., et al. (2025). ACL 2025, 17119–17159. doi:10.18653/v1/2025.acl-long.838
  • Goldberg, L. R., & Saucier, G. (2016). The Eugene-Springfield Community Sample: Information available from the research participants. ORI Technical Report, 56(1).
  • Toubia, O., et al. (2025). Marketing Science, 44(6), 1446–1455. doi:10.1287/mksc.2025.0262
  • Hu, T., et al. (2026). ICLR 2026. arXiv:2510.17516
  • Ren, Y., et al. (2024). ACL 2024, 2015–2040. doi:10.18653/v1/2024.acl-long.111
  • Huang, J., et al. (2026). arXiv:2606.24162
  • Troiano, E., Oberländer, L., & Klinger, R. (2023). Computational Linguistics, 49(1), 1–72. doi:10.1162/coli_a_00461
  • Chou, E., et al. (2026). Journal of Personality. doi:10.1111/jopy.70080
  • Lametti, D. R., et al. (2026). PsyArXiv. doi:10.31234/osf.io/8476g_v2
  • Hrkalovic, T. M., et al. (2025). IEEE Transactions on Affective Computing, 16(4), 3481–3498. doi:10.1109/TAFFC.2025.3600687
  • Wright, A. G. C., et al. (2026). Nature Human Behaviour, 10(3), 541–555. doi:10.1038/s41562-025-02389-x

Thanks for your response I have updated my score accordingly

Thank you for reading our replies even though they came toward the end of the discussion phase! If anything else comes up before the discussion closes, we'd be glad to answer. The points you raised in your review pushed us to sharpen how we distinguish psychometric from behavioral fidelity, so thank you again for the thoughtful engagement.

Reviewer_ARjDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates Big Five personality representations in LLMs at the facet level. It extracts personality steering vectors through different tasks and combines them with natural-language personality profiles to simulate real individuals. The authors evaluate personality fidelity using questionnaire-based scores recovered from generated life stories and examine the effects of personality conditioning on behavioral decisions. Experiments show that personality information degrades during generation, targeted steering can partially correct this degradation, and personality conditioning often produces stronger behavioral effects than those observed in humans.

Critical strengths

S1. Comprehensive experimental analysis.

The paper evaluates five open-weight models, multiple personality representation methods, and 420 real individuals. It also includes multiple reader models and extensive statistical analysis, providing a broad examination of personality representations and fidelity.

S2. Interesting behavioral findings.

The behavioral experiments show that personality conditioning can produce much stronger trait-related effects than those observed in humans. This is an interesting finding that highlights the limitations of using personality-conditioned LLMs to simulate human behavior.

Critical weaknesses

W1. Limited validity of the personality fidelity evaluation.

The main evaluation relies on LLM readers to recover personality scores from generated text, while human validation is limited in scale and mainly conducted at the domain level (Appendix A). Moreover, Table 3 shows that some methods achieve relatively high CCC but worse MAE than simply predicting the population mean. Although the behavioral experiments provide additional evidence, they do not establish whether higher reader-based fidelity leads to more accurate individual behavior. These issues weaken the interpretation of the reported improvements as faithful personality simulation.

W2. Concerns about the fairness of method comparisons.

The evaluation replaces scores from truncated generations with population means (Section 3.6), which mixes generation failures with personality fidelity. For example, 327 of 1,620 generations are truncated under full-profile steering for Qwen3.5-27B (Appendix D). Furthermore, targeted correction uses an additional generation and reader feedback, but lacks a comparable prompt-based feedback baseline. This makes it difficult to isolate the benefits of the proposed correction method.

Questions for authors

Q1. Can the authors provide stronger independent validation of facet-level personality fidelity, such as human evaluations or results on the 180 held-out IPIP-NEO items?

Q2. Given that some methods achieve higher CCC but worse MAE than the population-mean baseline, and that higher reader-based fidelity has not been shown to improve individual behavioral matching, how should the practical significance of the reported fidelity improvements be interpreted?

Q3. How do the method comparisons change when generation completion rates and personality fidelity are evaluated separately, rather than assigning population-mean scores to truncated outputs?

Q4. How does targeted activation steering compare with a prompt-based correction baseline using the same reader feedback and generation budget?

10 comments

Response to Reviewer ARjD (Part 1 of 3)

Thank you for the clear list of strengths and four precise questions. We ran every comparison you asked for, and two of them changed how we describe the correction. Each question is answered with the weakness it matches.

W1 and Q1. The evaluation relies on AI readers, and human validation is limited and mostly at the level of broad traits; can facets be validated independently, with people or on the 180 held-out IPIP-NEO items? Yes, both are now in the appendix.

People, at the level of facets (Appendix M). Two new studies, both built so that knowing a person's five broad traits is worth chance:

  • Raters read a life story written by our fine-tuned adapter and chose between two descriptions of the narrator with identical broad traits, one adding the narrator's own facet quirks and the other someone else's. The 48 raters who met our inclusion criteria picked the narrator in 68 of 96 trials, two per rater (71%; 95% interval .54–.86; chance 50%). The first batch of 38 raters drove this; a second batch of 10 was near chance, and Appendix M reports both. AI readers on the same stories got 65–80%.
  • Raters read a written portrait and rated the person's standout facet and a control facet at nearly the same predicted level, without knowing which was which. Across two batches, the 25 who met our criteria placed the standout correctly 72% of the time, with ties counted half (.61–.82; chance 50%), and the second batch alone matched the first. AI readers on the same pages got 84–90%.

We've also recruited an outside expert, a professor who publishes in psychometrics and personality, to rate a bundle of these profiles and stories blind on all 30 facets, and we'll make every effort to share those ratings by November 17.

The 180 held-out items (the reader answers those items as the narrator would, unlike Table 1's direct facet ratings; Appendix N). The writer saw only each person's answers to 120 items. Scored on the other 180 alone, facet CCC on stories that finished is:

WriterPortrait-only storiesCorrected stories
Gemma4-31B.490.525
Qwen3.5-27B.489.507
Gemma4-E4B.450.508
Qwen3.5-4B.478.514
Qwen3-8B.370.500

The written portraits themselves score .583. Each person's quirks hold up on these unseen items too: well above chance for the portraits and for the two larger models' stories (Appendix N).

Regression readers (Appendix N). On the larger models' stories, ridge regressions on two text embeddings (Gemini Embedding 2 and Qwen3-Embedding-8B), trained only on other people, recover each person's facet profile at r .74–.77 and each person's quirks at .29–.33, far above chance. A TF-IDF ridge model does nearly as well (r .73–.75). Explicit trait words are about half a percent of the text, and deleting them barely changes anything.

Response to Reviewer ARjD (Part 2 of 3)

W1c and Q2. Some methods have high agreement (CCC) but worse error (MAE) than guessing the population mean; what is the practical significance? This is a sharp catch, and the answer is calibration, not ordering. Our read-backs put people in the right order but spread them out more than they really are, like a ruler printed slightly too large: much of the ranking survives, but every measurement comes out long. Average error punishes that stretch, while guessing "average" for everyone is never far off, even though it tells you nothing about anyone.

We measured it directly (Appendix O). The read-backs are about a third to a half more spread out than the people. Before any correction, portrait-only stories score just below the mean guess on error. After a simple linear rescaling, learned on half the people and tested on the other half, every portrait-only and corrected method beats the mean guess clearly, and the methods rank almost exactly the same way by error as by CCC. Appendix O reports raw and rescaled error next to CCC, and we'll bring them into Table 3 in the final version.

Your question also helped us see that what a method optimizes is a design choice: steering's settings trade agreement, error and the person-specific pattern against each other (Appendix R).

W1d and W1e. The behavioral experiments don't show that higher read-back fidelity leads to more accurate individual behavior. Correct for our original tasks, and we've made this central to how we frame the paper: personality conditioning reproduces a measured profile, including its person-specific part, much more faithfully than it reproduces the person's behavior. On three datasets where each person's own self-reports are on record, personas from the person's own profile predict that person's reports modestly (r .11–.12), and on everyday acts, where facet scores exist, facet portraits beat broad-trait portraits (Appendix S). Personas also exaggerate trait-driven behavior well beyond people's (Appendix T). The revised paper now presents psychometric and behavioral fidelity as separate results, the second much weaker than the first (Section 4.5).

Response to Reviewer ARjD (Part 3 of 3)

W2a and Q3. Scoring unfinished stories at the population mean mixes generation failure with fidelity. Agreed. The submission's Appendix D did report each route's finish rate and valid-only scores under the primary reader, but we buried them. We now score every comparison three ways: the paper's rule, finished stories only, and with every unfinished story rerun once with a new seed (Appendix P). Among people whose stories finished in both arms, no overall ordering between methods changes, and Gemma4-31B moves by at most .013. That holds only for those people: looping is more common for some personalities and under larger pushes, so it doesn't settle how the methods compare across the whole cohort (Appendix P). Most models finish 85–100% of their stories under every condition. The exception is Qwen3.5-27B under steering, which finishes about 80%, mostly because it loops; your count of 327 of 1,620 is its total across three questionnaire forms. We looked at which stories loop. About three-quarters come from one steering recipe, and within it especially from people high in Neuroticism and low in Conscientiousness and Agreeableness, who also get a larger steering push. In that recipe, the most neurotic third of people's stories loop more than eight times in ten, against about one in ten for the least neurotic third. We think that link between a profile and a model's failure to finish deserves a dedicated study. Completion rates now have their own table (Appendix P), and we'll add them as a column in the main tables in the final version.

W2b and Q4. No prompt-based correction baseline with the same feedback and budget. We ran it, and it changed our framing (Appendix Q). The same reader feedback and one extra generation, but written into the prompt as a note instead of applied as a steering vector. Facet CCC, mean of three readers, on the 300 IPIP-NEO people:

WriterPortrait-onlySteering correctionWritten note
Gemma4-31B.651.683, .684 (two routes).689
Qwen3.5-27B.625.588, .643 (two routes).675
Gemma4-E4B.562.656.573
Qwen3.5-4B.584.614.615
Qwen3-8B.388.503.396

On the larger models, the written note does at least as well. On two of the three smaller models the steering correction still wins clearly, by .06 to .13; on the third they're level. Three more results point the same way. Adding the paper's profile steering on top of the note makes the stories carry less of each person, not more. Replacing each person's correction with the cohort's average removes the gain, so what helps is each person's own trait-level adjustment, not a single cohort-wide shift. And on Gemma4-31B, our fine-tuned writer with the note writes the most faithful stories of any Gemma4-31B condition, with no steering at all.

So the lesson we now draw is the one your question pointed to: a practical fix is a reader's feedback written into the prompt, which needs no access to activations and matched or beat the steering corrections we tested in the larger models, while steering earns its keep as a probe of how personality is represented. Following from that, we tested the note on a closed model where steering isn't possible: on Gemini 4 Carbon, across all 300 people, facet CCC rose from .691 to .773, and by .071 under the two readers that didn't build the note. A note built from five readers' pooled feedback and judged by two outside readers matched the single-reader note on Gemma4-31B and did a little better on Muse Glimmer 30B (.718 against .700; Appendix Q).

Reviewer_oEGLOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper explore how personality profiles are represented in language models. The authors start with questionnaire profiles from 420 people. They derive personality directions in several ways, use them to generate life stories, and then score the generated text against the target traits. The paper compares multiple models, routes, instruments and reader models. They also study a correction method and if the personas show personality-related behavioral differences.

Critical strengths

I'd commend the authors on a very thorough paper. There is a lot of empirical work, and the authors are careful about mixed and negative results. I liked how they use concordance rather than plain correlation. It catches the case where a model gets the ordering of people roughly right but shifts or exaggerates everyone.

The portrait-to-story results are interesting. The behavioral amplification result is also nice, and seems to confirm some previous related findings. The authors show that models often produce much stronger trait-behaviors than people do, and are appropriately cautious about this. The correction and adapter experiments are also good additions

Critical weaknesses

My majors concerns are about facet-level fidelity. The strongest claims are about 15 or 30 personality facets, but the key evidence comes from three LLM readers recovering those facets from model-generated text. The human validation is useful, but small and at the broader Big-5 level. So there is little to no direct validation of the facet-level results.

Agreement among the reader models is encouraging, but they may share learned ideas about how particular personality traits should sound. For example, they may have a stereotype about a 'high Neuroticism story”, without this genuienly showing that the generated text captures the target person’s finer-grained profile. This leaves me unsure how much of the facet-level result is actual recovery of the profile(compare to just agreement about common tropes).

Also, the personas show amplified trait effects and do not better predict the individual person’s choices. So the evidence seems stronger for generating a consistent measured 'profile' or archetype, than for modeling an individual person.

Questions for authors

See comments/questions raised in the weakness section above.

4 comments

Response to Reviewer oEGL (Part 1 of 3)

Thank you for such a careful reading, and for crediting our concordance measure and our mixed and negative results. Your three weaknesses are the right ones, and they shaped most of what we did in the discussion period.

W1. The strongest claims are about facets, but the key evidence is three AI readers, and the human validation is small and at the level of broad traits. Agreed on both counts for the submitted paper.

People, at the level of facets (Appendix M). Two new studies, both built so that knowing a person's five broad traits is worth chance:

  • Raters read a life story written by our fine-tuned adapter and chose between two descriptions of the narrator with identical broad traits, one adding the narrator's own facet quirks and the other someone else's. The 48 raters who met our inclusion criteria picked the narrator in 68 of 96 trials, two per rater (71%; 95% interval .54–.86; chance 50%). The first batch of 38 raters drove this; a second batch of 10 was near chance, and Appendix M reports both. AI readers on the same stories got 65–80%.
  • Raters read a written portrait and rated the person's standout facet and a control facet at nearly the same predicted level, without knowing which was which. Across two batches, the 25 who met our criteria placed the standout correctly 72% of the time, with ties counted half (.61–.82; chance 50%), and the second batch alone matched the first. AI readers on the same pages got 84–90%.

An externally recruited expert, a professor who publishes in psychometrics and personality, has agreed to rate a bundle of these profiles and stories blind on all 30 facets, and we will make every effort to post the ratings by November 17.

Regression readers (Appendix N). On the larger models' stories, ridge regressions on two text embeddings (Gemini Embedding 2 and Qwen3-Embedding-8B), trained only on other people, recover each person's facet profile at r .74–.77 and each person's quirks at .29–.33, far above chance. A TF-IDF ridge model does nearly as well (r .73–.75). Explicit trait words are about half a percent of the text, and deleting them barely changes anything.

Questions the writer never saw (Appendix N; scored from the reader's item-level answers as the narrator, not from Table 1's direct facet ratings). The writer saw each person's answers to 120 questionnaire items. Scored only on the other 180, the stories still agree with the person: facet CCC is .49 for both larger models' stories, and .58 for the written portraits themselves.

Response to Reviewer oEGL (Part 2 of 3)

W2. The readers may share a stereotype of how a trait sounds, so the facet results may be agreement about tropes rather than recovery of the person. They do share beliefs: across fifteen AI readers from twelve developers, they agree with each other more than with the truth. So we built tests of facet variation beyond linear predictions from the five domains.

  • Quirks. For each person we subtract what their own five domain scores predict about their facets and keep the rest, which we call quirks. They're a stricter version of Furr's (2008) distinctive profile, which removes only the average profile across people. Even a reader with perfect knowledge of someone's five broad traits should average zero on quirks, provided facets relate to those traits linearly, as the questionnaire's norms assume. Quirks can't rule out a convention, shared by writer and reader, for how a particular facet shows up in text. Portrait-only stories carry quirks for nearly every model, questionnaire and main reader (89 of 90 cells of writer model, questionnaire pairing and reading, defined in Appendix L), and they still do when the readers are ridge regressions trained on other people's stories, or when only the questions the writer never saw are scored (Appendices L, N).
  • Fingerprint twins. We pair each person with a twin whose facets match what the person's own five domain scores predict, plus someone else's quirks. In the larger models, the story's read-back is closer to the real person 84–88% of the time (Appendix L).
  • A lineup. Among everyone in the cohort, a story's read-back points to exactly the right person first 28% of the time (median), more than 30 times the chance rate. The lineup and a distinctive-profile check were already in Appendices C and D of the submission; we should have put them in the main text (Appendix L summarizes).
  • Unusual people. We built 300 synthetic people with ordinary facet levels in rare combinations. Readers recover their off-pattern detail about as well as for matched real people, and stories written from their portraits keep a good share of it (Appendix R). Rare combinations of ordinary facet levels are ordinary psychology (DeYoung et al., 2007; Mõttus et al., 2017), and they're hard to get from standard language-model outputs (Paglieri et al., 2026), so this is a real test of the archetype worry.

Our reading: the AI readers do lean on shared beliefs, and quirks are how we separate the broad-trait part of that from the person, though not every shared convention about particular facets. Appendices L and N report quirks next to facet agreement, and the revised main text now does the same (Section 4.5).

References

  • Furr, R. M. (2008). A framework for profile similarity: Integrating similarity, normativeness, and distinctiveness. Journal of Personality, 76(5), 1267–1316. https://doi.org/10.1111/j.1467-6494.2008.00521.x
  • DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. https://doi.org/10.1037/0022-3514.93.5.880
  • Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100
  • Paglieri, D., Cross, L., Cunningham, W. A., Leibo, J. Z., & Vezhnevets, A. S. (2026). Persona Generators: Generating diverse synthetic personas for arbitrary contexts. arXiv:2602.03545.

Response to Reviewer oEGL (Part 3 of 3)

W3. The personas exaggerate trait effects and don't better predict the individual's choices, so the evidence is stronger for a measured profile or archetype than for an individual. We agree, and we've made it the center of how we describe the paper. Conditioning on a profile pulls a model far from its narrow task defaults, facet by facet, but it doesn't instantiate a particular person, and that was never our goal: we don't believe personality traits alone, however granular, could replicate the idiosyncrasies that make someone a unique individual. Traits are population-level measures, and the many dimensions of identity beyond the Big Five deserve more attention, which our future work will take up.

Within that frame, here's what the new tests show (Appendices S and T):

  • The profile, including its person-specific part. The quirks results (Part 2) show the stories carry more than an archetype: they carry the part of each person that their broad traits don't predict.
  • Behavior, modestly. On the paper's own tasks, read-back fidelity didn't predict behavioral match person by person, though that test had little power. On three datasets where each person's own self-reports are on record (everyday acts, policy answers and value ratings), personas built from the person's own profile predict that person's reports at r .11–.12; in the two where we can compare, that's r .121 and .124 against .208 and .221 for a model fitted to other people's answers from the same scores (cross-validated by person); on Twin-2K-500 the full-score persona did no better than a low, average or high summary of the same five scores, and its mean absolute error was larger than that from guessing the population mean. Where facet scores exist (everyday acts), a facet-level portrait beats a broad-trait portrait.
  • Exaggeration, robustly. From the least to the most neurotic quarter of people, the rate of negative reactions rises by 6.2 points (6.8 after correcting for the questionnaire's unreliability; Johnson, 2014). In the models it rises by 20.4 to 38.7 points. At temperature 1.0, Gemma4-31B gives the same first emotion in 94.9% of cells and the gradient is 39.8 against 38.5 points (383 events); pushing against each person's own trait vector, on 96 events, does not flatten it either (37.9 portrait alone, 35.9 at full reversal; Appendix T).
  • For scale: in the same everyday-acts data, the average of a person's peers agrees with their self-ratings at about .58 on the broad traits (Appendix S). No reader, human or model, recovers a person perfectly.

So the evidence is strongest for the measured profile, real for its person-specific part, and modest for behavior, and that's how the revised abstract and discussion now scope it.

References

  • Goldberg, L. R., & Saucier, G. (2016). The Eugene-Springfield Community Sample: Information available from the research participants. ORI Technical Report, 56(1). https://ipip.ori.org/ESCS_TechnicalReport_January2016.pdf
  • Toubia, O., Gui, G. Z., Peng, T., Merlau, D. J., Li, A., & Chen, H. (2025). Database Report: Twin-2K-500: A data set for building digital twins of over 2,000 people based on their answers to over 500 questions. Marketing Science, 44(6), 1446–1455. https://doi.org/10.1287/mksc.2025.0262
  • Han, J., Choi, D., Song, W., Lee, E.-J., & Jo, Y. (2025). Value Portrait: Assessing language models' values through psychometrically and ecologically valid items. ACL 2025, 17119–17159. https://doi.org/10.18653/v1/2025.acl-long.838
  • Johnson, J. A. (2014). Measuring thirty facets of the Five Factor Model with a 120-item public domain inventory: Development of the IPIP-NEO-120. Journal of Research in Personality, 51, 78–89. https://doi.org/10.1016/j.jrp.2014.05.003

Reviewer oEGL: the expert's ratings, and your three points in one place

Thank you again for a review that shaped most of our discussion period. New results bear on your W1, and since our replies have run long, here is everything we did for your three points in one place.

New: an outside expert, and the paper's AI readers on the very same texts. Following up on W1, the blind 30-facet expert ratings we promised in Part 1 are in. An externally recruited professor who publishes in psychometrics and personality rated 20 texts our adapter wrote, 5 written portraits and 15 life stories, blind to the people's questionnaire answers. Their ratings track each narrator's 30-facet questionnaire profile at r = .37 (shuffled pairings .03; p < .001), though not the quirks beyond the broad traits (.105, against .111 shuffled). We then gave the paper's three AI readers the identical texts (the full table of 600 ratings and these analyses are in Appendix M of the revised PDF):

  • They recover the same narrators' profiles at r = .86 and quirks at .59 (shuffled .24 and .09).
  • They agree with the expert about each person's 30-facet profile (r = .41; shuffled .10).

On quirks, the readers score higher than one expert's holistic pass over 30 facets, which invites halo from the broad traits. As we said in Part 2, the readers' quirk scores can't rule out conventions the writer and readers share, so for quirks the human evidence is the two targeted studies below.

Everything for your three points:

  • W1, facet evidence resting on three AI readers: two new human studies at the level of facets, each built so that knowing a person's broad traits is worth chance (71% and 72%, where chance is 50%; Appendix M); the expert and the reader-against-expert check above; regression readers that use no generative model (Appendix N); scoring only the 180 questionnaire items the writer never saw (Appendix N); and fifteen AI readers from twelve developers (Appendix N).
  • W2, shared tropes: fingerprint twins with identical broad traits, told apart 84–88% of the time in the larger models; a lineup at more than 30 times chance; 300 synthetic people with unusual facet combinations; quirks beyond the five domains carried in 89 of 90 cells (Appendices L and R); deleting explicit trait words barely changes anything; and, as a control, stories written from only a person's five broad traits (composed trait adapters on Qwen3-8B, 32 people) carry none of that person's quirks under the paper's primary reader (r = −.02; shuffle 95th percentile .10), while portrait stories for the same people carry .31 (Appendix R): given only the broad traits, the reader finds none of the person's quirks.
  • W3, archetype rather than individual: we agree, and we've reframed the paper around it (Part 3). Personas predict people's own reported behavior only modestly (r .11–.12; Appendix S), and the exaggeration holds at higher temperature and under dampened steering (Appendix T). New today, on the three smaller models it also resists plain instruction: three calibration instructions (react at this person's typical intensity, think first of how most people would react, or keep in mind that trait differences are modest) never shrank the Neuroticism gradient across nine tests, and in seven the increase was reliable (95% intervals above zero; Appendix T). [Edit: Gemma4-31B, the paper's main model, is done, and it behaves differently: all three instructions reduce its Neuroticism gradient, by 1.4 to 7.4 points from 39.3, the effect-size instruction most (to 31.9; 95% interval of the reduction 4.7–10.3), mainly by making the calm personas less calm, and under that strongest instruction, matching each person's own emotions improves slightly. So plain instruction takes some of the edge off the exaggeration in the larger model, though even at its lowest its gradient stays about five times people's 6.2; in the smaller ones it does not (Appendix T).]

We'd be very glad to hear whether this answers your concerns, and we're ready to run anything else you'd like to see before the discussion closes.

Comments on the paper

4 comments

Overall Response (Part 1 of 2)

Thank you, all four of you, for careful and generous reviews. We followed up on every suggestion, which took longer than we would have liked, so we're grateful you're taking the time to see how we used your feedback. The paper is dense, and these replies are too; we've revised the manuscript to be clearer, written each reply below to stand on its own, and put the exact numbers in the revised PDF for anyone who wants to check them.

Two questions came up more than any others. Can facet-level results be trusted when the readers are language models (all four of you asked this in some form)? And does fidelity to a measured profile tell us anything about how a person behaves (oEGL, ARjD and mBHV)?

Are the readers seeing the person, or a stereotype of a trait? We tested this several ways, most of them independent of the AI readers.

  • People. Reading written portraits, lay raters recruited through Prolific during the discussion rated each person's standout facet and a matched control facet blind, and put them in the right order 72% of the time across two independent batches (ties counted half; interval .61–.82). Separately, reading stories our adapter wrote, raters chose the narrator's real description over a decoy with the same five broad traits but different finer details 71% of the time when pooled (.54–.86), though the two batches came out very differently (Appendix M). Together, the two studies back the portrait contrasts and the adapter-written stories; neither tests the gain from activation correction directly. An externally recruited expert, a professor who publishes in psychometrics and personality, is rating a bundle of our texts blind on all 30 facets. An email problem on our side (Outlook) slowed that recruitment during the discussion, but we will make every effort to post the ratings by November 17.
  • Regression readers. Ridge regressions on text embeddings, and a TF-IDF ridge model, each trained only on other people's stories, recover each person's facet profile (r .73–.77 on the larger models; Appendix N).
  • Questions the writer never saw. Scored only on the 180 IPIP-NEO items the writing model never saw (here the reader fills in those items as the narrator would, rather than rating facets directly as in Table 1), the stories still carry the person (facet CCC about .49 for the larger models; Appendix N).
  • Quirks: what makes a person more than their five broad traits. We score each person's quirks: how their facets depart from what their own five domain scores predict. A reader who knows only the five broad traits should score zero on quirks (on average, if facets relate to the traits linearly, as the questionnaire's norms specify), so a trope about "a high-Neuroticism story" earns nothing. What quirks can't rule out is a convention writer and reader share about how a particular facet shows up in text. Portrait-only stories carry quirks for nearly every model, questionnaire and reader (Appendix L).

The AI readers do share beliefs: they agree with each other more than with the truth. Each also has a stable accent, reading particular facets a little high or low (Appendix N). That's why we rest the claim on the tests above, not on how much the readers agree.

Against the models' own defaults. With no person to play, the models don't produce a random person; they fall back on a narrow default. Told only that they're a person, both larger models write narrators about 0.9 to 1.1 SD more agreeable, 0.6 to 1.3 SD more open and 0.6 to 0.7 SD more conscientious than the questionnaire norms, who vary only 0.35 to 0.45 as much as real people do. Asked to tell their life story as themselves, many narrate as the model itself (by a rough keyword flag, 43% of Gemma4-31B's stories and 13% of Qwen3.5-27B's). Asked to rate themselves as AIs, they are more extreme still (+1.6 to +1.9 SD agreeable, +1.8 to +2.1 SD conscientious, −2.3 to −2.5 SD on Neuroticism), a self-image that correlates only .33 to .37 with the person they imagine when told they're human. A portrait moves each story about three times as far from the default as default stories scatter around it (Appendix U). Moving a model out of that narrow default, toward a particular person's profile, is what we mean by personality conditioning.

Does it say anything about behavior? Some, and much less than about the profile, which is your reading and ours. Part 2 says what we think that means for the paper.

Overall Response (Part 2 of 2)

What we think the paper shows, said more sharply than the submission did. Personality conditioning moves a model a long way from its narrow task defaults, differentiating it person by person, and it can do so at the level of facets. That is a different thing from simulating a particular person, and it was never our goal. Personality traits are population-level measures. We are not proposing them as the door to high-fidelity simulation of an individual or to crafting a synthetic identity. Recent personality science argues that personality is "best understood as a person-specific complex, nonlinear dynamic system" (Beck et al., in press; see also Fleeson, 2001), and there are many dimensions of identity beyond the Big Five, such as goals and life stories (McAdams & Pals, 2006), that we believe deserve more attention; we're planning future work to identify which components compose best to more fully instantiate a nuanced individual. What we show is that personality conditioning reliably differentiates what a model writes and chooses, person by person and facet by facet, and that in the larger models, a written feedback note corrects stories at least as well as activation steering.

Behavior, from real people. We added three tests against individual people's own reports, plus a test of trait-related model behavior (Appendix S). In three, each person's own self-reports are on record: everyday acts from the Eugene-Springfield Community Sample (Goldberg & Saucier, 2016), policy answers from Twin-2K-500 (Toubia et al., 2025) and value ratings from Value Portrait (Han et al., 2025). Personas built from a person's own profile predict that person's reports above chance but modestly: in the two datasets where we can compare, at r .121 and .124 against .208 and .221 for a model fitted to other people's answers from the same scores (cross-validated by person). On Twin-2K-500, the full-score persona did no better than a low, average or high summary of the same five scores, and on average missed by more than guessing the population mean. Where facet scores exist (everyday acts), facet-level portraits beat broad-trait portraits. In the fourth, BehaviorBench (Pu et al., 2026), personas built from 60 real people's facet portraits behave in line with each person's measured traits on all five. Personas also exaggerate trait-driven behavior well beyond people's, even after correcting people's scores for measurement noise (Appendix T). This paper doesn't test whether a persona's personality holds across contexts (Han et al., 2026, study some of this), but we think better personality conditioning offers a new probe of whether personality can make model behavior more predictable across tasks.

Steering, and why prompting is the point. Your questions about steering led us to a cleaner conclusion: in the larger models, to condition a model at high resolution, you don't need to steer. There, a written note built from a reader's feedback does at least as well as the steering correction. Adding the paper's profile steering on top of that note makes the stories carry less of the person, not more. And on Gemma4-31B, our fine-tuned writer with the note writes the most faithful stories of any Gemma4-31B condition, with no steering at all (Appendix Q). The note also works where steering can't go: on Gemini 4 Carbon, a closed model, it lifts facet agreement from .69 to .77 across all 300 people (Appendix Q). Steering's main value is as a probe of how personality is represented: directions found through different tasks line up, and pushing on them changes what the model writes in the expected way. It still helps the smaller models where a note can't.

Still to come: Thank you all for pressing on facet-level human validation. We'll make every effort to post the expert's blind facet ratings (Part 1) by November 17.

The revised PDF opens its appendix with a dated change list and a guide to the key measures (new results are in Appendices L–U), and uses the extra main-text page for the key checks and this framing (Section 4.5 and the start of the Discussion).

References

  • Beck, E. D., et al. (in press). Personality is a person-specific complex, nonlinear dynamic system. Adv. Exp. Soc. Psychol.
  • Fleeson, W. (2001). J. Pers. Soc. Psychol., 80(6), 1011–1027. doi:10.1037/0022-3514.80.6.1011
  • McAdams, D. P., & Pals, J. L. (2006). Am. Psychol., 61(3), 204–217. doi:10.1037/0003-066X.61.3.204
  • Goldberg, L. R., & Saucier, G. (2016). The Eugene-Springfield Community Sample: Information available from the research participants. ORI Technical Report, 56(1).
  • Toubia, O., et al. (2025). Mark. Sci., 44(6), 1446–1455. doi:10.1287/mksc.2025.0262
  • Han, J., et al. (2025). ACL 2025, 17119–17159. doi:10.18653/v1/2025.acl-long.838
  • Pu, T., et al. (2026). IJCAI-26, 5910–5918. doi:10.24963/ijcai.2026/658
  • Han, B., Kwon, D., & Gratch, J. (2026). LaCATODA 2026, CEUR-WS Vol. 4178, 1–13. arXiv:2602.01063

A robustness check prompted by Reviewer ARjD: steered stories that break down

Reviewer ARjD's question about residualized and conditional steering directions led us to push steering harder than the paper does, and to look closely at what happens to a story when the push goes too far. (In one of the paper's own Qwen3.5-27B stories, for example, the narrator repeats "And I was just a guy in a chair." 504 times before the token limit.) That check also covered the paper's own steered stories, so we're sharing it with everyone. In short, it confirms the paper's scoring rule, and the paper's orderings hold.

The paper scores a story that reaches the token limit at the population mean, and Appendix P reports how often that happens: Qwen3.5-27B truncates 189 and 190 of the 900 stories in its two steered arms. Reading those stories confirms the rule: 376 of the 379 meet our criterion for a repetitive ending, and in a blind sample of 30, two of our reader models called 28 broken and the rest repetitive, none normal. The limit marks a breakdown in generation rather than a long story cut short.

The same check found a smaller group that the rule doesn't cover: Qwen3.5-27B stories that end on their own but are markedly repetitive. There are 140 of 900 in steering only and 161 of 900 in portrait plus steering; Gemma4-31B has 7 and 12. Two of our reader models judged a random 50 of the steering-only ones blind and called every one at least repetitive. As a sensitivity analysis, we rescored the steered arms both ways, with these stories at the population mean and with them dropped, across three settings of the flag:

  • Qwen3.5-27B steering-only facet CCC is .26 to .30 and .38 to .39 (.32 to .35 under the paper's rule).
  • Portrait plus steering is .40 to .45 and .51 to .53 (.46 to .49), against .62 to .64 for portraits alone.
  • Gemma4-31B moves by .013 at most from the paper's rule.

Pooling the three steering recipes, the ordering portraits alone > portrait plus steering > steering alone holds in both models under every reader and flag setting. We will extend the check to the correction arms, the individual recipes and the three smaller models for the camera-ready version.

For ARjD's question, the check also shows that, for the constructions and doses we tested, pushing harder degrades the writing without closing the gap. In Gemma4-31B, across the single-layer constructions, 2 of 1,800 real people's stories reached the limit at the paper's vector length and 350 of 1,500 at twice that, and we detected no difference between synthetic and real profiles once size is accounted for. This matches the report that high steering strengths induce degenerate repetition (Braun et al., 2026). Long free-form generation like our life stories shows it clearly, where short outputs and perplexity checks can miss it.

The revised PDF adds this check to Appendices P and R. Our thanks to Reviewer ARjD for the question that led here, and to all four of you for a discussion that keeps making the paper better.

[Edit: After we came up with questionnaire-derived directions for Reviewer ARjD, a more granular search turned up concurrent work by Hsu et al. (2026), who predict people's answers across seven scales and steer with per-subscale ridge directions at the final prompt position. Ours are fitted jointly across 30 facets on 3,000 respondents and steer whole profiles through long-form stories.

Hsu et al. find near-zero human-shaped geometry in all seven base models they tested (Mantel r ≤ 0.14) and leave open whether instruction tuning organizes representations "already present in the base model". We can now answer that, and say when the structure appears. Facet directions from single-facet edits of 60 people's real answers mirror the facet correlations of 410,376 people beyond item wording, about as strongly in five base models as in their instruct counterparts (partial Mantel r .73–.87 against .71–.83), including Hsu et al.'s two families. Directions fitted across respondents are noisier in base models and sharpen with tuning in three families. OLMo 3's public checkpoints let us watch the structure arrive: absent at initialization and at step 1,000, present by step 3,000 (r .36, about 0.2% of pretraining steps), and nearly complete by the halfway point (.77; .78 in the released base model). Our reading for the paper: the facet-level structure of human personality in these models is learned early in pretraining, not created by instruction tuning. Details in Appendix R.

We still hope to post the outside expert's blind facet ratings before the author-discussion phase closes.]

References

  • Braun, J., Eickhoff, C., & Bahrainian, S. A. (2026). Beyond Multiple Choice: Evaluating Steering Vectors for Summarization. Findings of EACL 2026, 3849–3884. doi:10.18653/v1/2026.findings-eacl.200
  • Hsu, C.-H., Chou, F.-C. B., & Chen, P.-H. A. (2026). Tracing Psychometric Inference in Large Language Models. NeurIPS 2026. arXiv:2611.12937.

Thank you all, and the outside expert's ratings

As the discussion draws to a close, thank you, all four of you. Your questions sent us back to the data round after round, and the paper is much stronger for it, from the facet-level human studies to the base-model results and the expert ratings below.

As promised, the outside expert's ratings are in. An externally recruited professor who publishes in psychometrics and personality read 20 texts written by our released fine-tuned adapter, blind to the people's questionnaire answers: 5 written portraits, 10 short life stories and 5 full-length ones. They rated each narrator on all 30 IPIP-NEO facets, 1 to 5, and left no cell empty. We scored the ratings with primary measures locked on November 11, before they arrived, against a null that shuffles texts across people.

The expert clearly reads the person's profile. Across the 30 facets, their ratings correlate with each narrator's own questionnaire profile at r = .37 (95% interval .25–.48, resampling texts), against .03 when texts are shuffled across people (p < .001). Facet concordance (Lin's CCC) is .24 averaged over facets and .25 pooled, and the five domain scores track the narrators' at .37 to .62. It holds on the 180 questionnaire items the writer never saw (r = .34, p < .001).

What one expert reading of 20 texts does not pick up is the person-specific part beyond the broad traits. On each narrator's facet quirks, the expert's correlation (.105) is no higher than shuffled pairings give (.111; p = .56). That is the part our two forced-choice studies (Appendix M) were built to isolate, with many raters and a decoy matched on the broad traits, and there lay raters found it: 71% and 72%, where chance is 50%.

We then gave the paper's three AI readers the identical 20 texts. They agree with the expert about each person's 30-facet profile (r = .41; shuffled .10), and on quirks they score r = .59 against the questionnaire (shuffled .09). Rating all 30 facets in one holistic pass invites halo from the broad traits, which is exactly what the quirks score removes, so for quirks the human evidence is the two targeted studies; the readers' .59 may combine finer detail in the text with conventions the writer and readers share.

All 600 ratings and these analyses go into Appendix M of the revised PDF we'll upload before the discussion closes, and its dated list at the head of the appendix will cover everything added since November 14.

We still have the energy, and everything set up, to run any last-minute exploration you'd like to see, and we'll keep watching the forum until the discussion closes.

Thank you again for a discussion we've genuinely enjoyed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.