- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how Big Five personality traits are represented and controlled in language models. The authors extract facet-level activation directions from different tasks and test their effects on text generation and behavioral choices. Using personality profiles from 420 individuals, they compare written portraits, activation steering, and their combination. They find that directions for the same facet tend to align across tasks, while steering one facet also affects others. Targeted correction improves personality fidelity in some settings, particularly for smaller models.
Critical strengths
-
The facet-level analysis is useful. The authors compare activation directions across tasks and examine how steering one trait affects others. These results help us understand the relationships among personality representations in language models.
-
The experiments cover several models, personality measures, and behavioral tasks. The authors also report negative results, including the weak performance of steering alone and the exaggerated behavioral effects compared with humans.
Critical weaknesses
-
The added value of the proposed steering method is unclear. In Table 1, steering alone performs much worse than written portraits. Targeted correction helps mainly in smaller models. The paper lacks direct comparisons with domain-level steering and simpler prompt-based correction methods. It is therefore difficult to judge whether the more complex approach offers a meaningful advantage.
-
The evidence for fine-grained personality control remains limited. The evaluation uses selected human personality profiles but does not test whether the method works for unusual or artificially constructed facet combinations. This matters because steering one facet also changes others. In addition, personality fidelity is mainly evaluated by LLM readers, with limited human validation at the facet level.
Questions for authors
- What is facet CCC when PAPI read-back is scored only on the 180 held-out IPIP-NEO items, for portraits, portrait-only stories and corrected stories?
- If the correction vector is constructed with one reader (e.g., Gemini) and evaluated with the others, does the improvement survive? A cross-reader design would address the circularity concern directly.
- For the amplification result, are human trait–behavior slopes corrected for the unreliability of both the questionnaire and the single-shot behavioral responses? Models produce near-deterministic choices, so part of the gap could come from response noise in humans.
- How much do the conclusions change when steering conditions are compared on valid stories only, with matched valid rates?
- Is the released adapter trained on data whose licenses allow redistribution of derived models? Persona-E2 is CC BY-NC-SA; please confirm which cohorts entered training.
5 comments
Response to Reviewer 1mJp (Part 2 of 3)
W2. Fine-grained control is limited: only real profiles were tested, steering one facet changes others, and evaluation leans on AI readers with limited human validation at the facet level.
Unusual combinations (Appendix R). We built 300 synthetic people with ordinary facet levels in rare combinations and compared them with matched real people. Readers recover the off-pattern detail from plain descriptions about as well for synthetic people as for real ones. Stories written from portraits keep a good share of it (about a third), and steering alone keeps almost none. Rare combinations of ordinary facet levels are ordinary psychology: facets within a domain separate into distinct aspects (DeYoung et al., 2007), and stable person-specific characteristics sit below the facet level (Mõttus et al., 2017). They're also hard to get from standard language-model outputs (Paglieri et al., 2026).
Steering one facet changes others. Yes, and it limits steering, as you suspected; work on trait-level steering finds the same coupling in other models (Bhandari et al., 2026), as does our replication of PERSONA in Appendix G of the submission, where steering toward compassion also raised Sociability and Energy Level. In our data it shows three ways: steering alone keeps almost none of the off-pattern detail for synthetic people, domain-level steering carries no quirks at all, and trait adapters released by others carry none of the person's quirks.
AI readers and human validation. During the discussion we ran two studies with lay raters from Prolific. Given written portraits, they rated a person's standout facet and a matched control without knowing which was which, and got the order right 72% of the time across two independent batches (ties counted half; interval .61–.82). Given stories our adapter wrote, they chose the narrator's actual description over a decoy sharing the same five broad traits but not the finer details 71% of the time pooled (.54–.86), though the batches differed sharply (Appendix M). Both bear on portraits and adapter-written stories rather than on the activation correction. A professor who publishes in psychometrics and personality, recruited from outside our team, has agreed to rate a bundle blind on all 30 facets (Appendix M). Beyond people, ridge regressions on text embeddings or TF-IDF word features, with no generative model in the loop, recover each person's facets and quirks, and the stories still carry the person on the 180 items the writer never saw (Appendices L, N). The AI readers themselves agree on who is who and differ mainly in stable per-facet calibration (Appendix N).
References
- DeYoung, C. G., Quilty, L. C., & Peterson, J. B. (2007). Between facets and domains: 10 aspects of the Big Five. Journal of Personality and Social Psychology, 93(5), 880–896. https://doi.org/10.1037/0022-3514.93.5.880
- Mõttus, R., Kandler, C., Bleidorn, W., Riemann, R., & McCrae, R. R. (2017). Personality traits below facets: The consensual validity, longitudinal stability, heritability, and utility of personality nuances. Journal of Personality and Social Psychology, 112(3), 474–490. https://doi.org/10.1037/pspp0000100
- Paglieri, D., Cross, L., Cunningham, W. A., Leibo, J. Z., & Vezhnevets, A. S. (2026). Persona Generators: Generating diverse synthetic personas for arbitrary contexts. arXiv:2602.03545.
- Bhandari, P., Naseem, U., & Nasim, M. (2026). Do personality traits interfere? Geometric limitations of steering in large language models. Findings of ACL 2026, 29284–29293. https://doi.org/10.18653/v1/2026.findings-acl.1463
Response to Reviewer 1mJp (Part 3 of 3)
Q1. Facet CCC on the 180 held-out IPIP-NEO items, for portraits, portrait-only stories and corrected stories. Scored only on the 180 items the writer never saw (the reader completes those items as the narrator; Table 1 instead asks for facet ratings directly), on stories that finished (Appendix N): the written portraits score .583; portrait-only stories .490 (Gemma4-31B), .489 (Qwen3.5-27B), .450 (Gemma4-E4B), .478 (Qwen3.5-4B) and .370 (Qwen3-8B); corrected stories .525, .507, .508, .514 and .500 in the same order. Quirks hold up on these items too: above chance for the portraits and the two larger models' stories (Appendix N).
Q2. If the correction is built with one reader and judged by the others, does the improvement survive? In the smaller models, yes: it improves things under each of the two readers that played no part in building it, in all nine comparisons. In the larger models the gains are smaller and depend on the reader (Appendix Q). This cross-reader check was in Appendix D of our submission too, and we should have pointed to it from the main text. We also built the written note from five readers' pooled feedback and judged it only with two readers outside the pool, on all 300 PAPI people. The pooled note does as well as the single-reader note on Gemma4-31B and a little better on Muse Glimmer 30B (facet CCC .718 against .700, though one pooled reader shares Glimmer's developer; Appendix Q).
Q3. Are the human trait–behavior slopes corrected for unreliability? For the questionnaire, yes. Treating its error as classical and using the published alpha of .90 for the IPIP-NEO-120 Neuroticism scale (Johnson, 2014; not measured in this cohort), the 6.2-point difference between the most and least neurotic quarter of people (9 of 36 each) becomes 6.8 over the same spread of scores the models were given [95% interval −3.0 to 16.6; 6.5 if the quarters are formed on error-free scores]. For the behavioral rates, no outcome correction is needed: error in a rate with zero mean noise widens the interval without biasing the expected difference, and each rate averages 384 events (split-half .96). The models' differences are 20.4 to 38.7 points, and the gap stays positive in every paired interval (smallest: Gemma4-E4B, +3 to +25). The human interval is wide because the cohort is 36 people aged 18 to 25, so we do not generalize beyond it. On the model side, sampling at a higher temperature doesn't shrink the gap (Appendix T).
Q4. Valid stories only, with matched valid rates. The submission's Appendix D gave each route's finish rate and its valid-only score under the primary reader, though we should have surfaced them. We now score every comparison three ways: the paper's rule, finished stories only, and with every unfinished story rerun once (Appendix P). On matched finished stories, where the same people finished in both arms, no overall ordering between methods changes, and Gemma4-31B moves by at most .013. That comparison holds only for those people: looping is more common for some personalities and under larger pushes, so it doesn't settle how the methods compare across the whole cohort. Most models finish 85–100% of their stories; Qwen3.5-27B under steering finishes about 80%, mostly because it loops (Appendix P).
Q5. Licenses of the adapter's training data. Persona-E² did not enter training; it was used only in the behavior tests. The adapter (Appendix J) was trained on 320 people: 160 from the Eugene-Springfield Community Sample, released under CC0, and 160 from the BESSI development samples (Soto et al., 2022; Sewell et al., 2022), which completed the BFI-2 and are public on OSF with no license shown. We checked this against the adapter's training files. After your question, the missing license made us uneasy, so we built a new adapter, OCEAN-S2, trained only on synthetic people generated from CC BY and CC0 samples, with training text from open-weight models whose licenses don't restrict that use. Evaluated on the 420 with the same portraits, prompt, sampler and scorer, it clearly beats the untuned model in 6 of 9 reader cells, including all three on BFI-2 (primary reader: facet CCC .716 to .779; OCEAN-M03, .792). OCEAN-S2 is in its own anonymous repository now (Appendix J); the original adapter, OCEAN-M03, stays in its original anonymous repository for reviewers during the discussion only.
Presentation. You're right that the submission asked too much of its reader in places: with five models, six routes and three readers, the main thread was easy to lose. We've aimed for a clearer revision. A new Section 4.5 on the tenth page lays out the main checks and the thesis in plain terms, the appendix opens with a dated change list and a guide to the key measures, and every change is in blue. We'll keep improving clarity through the discussion, and if some passages made you work harder than they should have, we'd be glad to know which.
Response to Reviewer 1mJp (Part 1 of 3)
Thank you for a careful review, for the cross-reader design, which tests the circularity concern head-on, and for crediting our negative results.
W1. The added value of steering is unclear: steering alone does much worse than written portraits, the correction helps mainly the smaller models, and there's no comparison with domain-level steering or simpler prompt-based correction. Your reading is right, and your questions changed how we describe the paper.
Prompt-based correction (Appendix Q). We gave the model the same reader feedback as a written note in the prompt instead of a steering vector. On the larger models the note does at least as well (Gemma4-31B .689 against .683–.684; Qwen3.5-27B .675 against .588–.643, facet CCC). On two of the three smaller models the steering correction still wins clearly, by .06 to .13; on the third they're level.
What steering adds and what it doesn't. Adding the paper's profile steering on top of the written note makes the stories carry less of each person, not more. Replacing each person's correction with the cohort's average removes the gain, so what helps is each person's own trait-level adjustment, not a single cohort-wide shift. On Gemma4-31B, our fine-tuned adapter given the note produces the most faithful stories of any Gemma4-31B condition, with no steering at all, and on Gemini 4 Carbon, a closed model where steering isn't possible, the same note lifts facet agreement from .69 to .77 (Appendix Q).
Domain-level steering (Appendix R). When we steer with five domain directions instead of thirty facet directions, with the same models, readers and push size, the two are within .03 of each other in two of six comparisons, facet steering is better in three, and domain steering in one. What the thirty directions add is a faint trace of each person's quirks, which domain steering doesn't carry at all.
So we now separate the two roles: steering is a tool for studying how personality is represented (directions found from different tasks align, and pushing on them changes what the model writes in the predicted way), and a reader's feedback written into the prompt is a practical fix, needing no access to activations. In the smaller models, the steering correction still earns its place. The paper was never meant to advocate for steering: it was meant to settle what that technique does and doesn't add, and to show how far high-fidelity, calibrated prompting can go. We hope more researchers adopt calibrated prompting, partly because it lets us study much larger models, closed ones included.