acceptodds
Under review as a conference paper at ICLR 2027

Many in One: Grounding Individuality in a Language Model

Abstract

Large language models can produce many apparent personas, but this does not establish that their differences are grounded in the individuals they represent. Names, biographies, or demographics may retrieve associations learned during pretraining, and such associations need not track measured differences between actual people. We ask: can a model learn person-specific variation from observed behaviour alone, and can we read that variation back? Because natural speech entangles content with manner, we study one isolable dimension of grounded individuality, register: how speakers formulate given content. We introduce addresses, one learned speaker embedding per person, stored as a row of the model's input-embedding matrix and fitted by paraphrase inversion. We fit addresses for speakers within one Qwen2.5-7B model, on 6.7 million training words from a new voice-verified corpus of public-figure podcast speech. We evaluate generated personas on human-calibrated register axes that separate grounding, diversity, and coverage. Addresses recover person-specific register, including in episodes and shows held out from training. A speaker's own turns ground personas best, and on held-out episodes a name, prompted or written in the address's place during fine-tuning, grounds them better than an address alone. Addresses yield more distinct styles and more collective variety of ideas than a prompted name, and a name written in their place during fine-tuning yields more of both. Because addresses are parameters, we read, steer and extend them. These results show that a single language model can represent grounded person-specific variation that we can measure and interpret.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
persona modelling, speaker embeddings, register, individual differences, grounded personas, paraphrase inversion, continual learning, parameter-efficient conditioning, diversity, homogenisation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gFuKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an individual-grounded evaluation framework and demonstrates that a single language model can encode measurable, interpretable speaker-specific linguistic variation, while revealing a fundamental distinction between persona diversity and grounding.

Critical strengths
  1. A meaningful conceptual contribution: separating grounding from diversity.

  2. Convincing evidence for measurable speaker-specific variation, supported by extensive controls.

Critical weaknesses
  1. whether the grounding metric really isolates individual speaking style. Even though all speakers receive the same prompts, their responses can still differ in content and conversational context.

  2. the fine-tuned name-label baseline performs better than addresses on several key metrics. This makes it less clear which part of the proposed method is essential.

Questions for authors
  1. To what extent do the grounding correlations reflect individual-specific register rather than differences in semantic content or conversational context? Would the main conclusions hold under stricter content controls?

  2. How do the authors explain that removing content words improves grounding, while removing the contrastive objective has little effect? What is the key advantage of the proposed training objective over simpler alternatives?

Reviewer_KQcuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper conducted several case studies on persons embedded in LLMs. It aims to learn a vector for each individual. Some insights about diversity and interpretability are provided. Experiments are conducted on Qwen2.5-7B.

Critical strengths
  1. Some new terms are introduced, such as register and addresses.
  2. The results are interpretable.
Critical weaknesses
  1. The motivation: "This does not establish that their differences are grounded in the individuals they represent. Names, biographies, or demographics may retrieve associations learned during pretraining, and such associations need not track measured differences between actual people" has not been verified empirically.
  2. The terms register and addresses need clear and formal definitions. It is really confusing.
  3. The paper is basically case studies. There is not much depth in terms of ML contribution.
  4. The paper only discusses the fine-tuning strategy. A lot of new work published in 2026 is missing: https://iclr.cc/virtual/2026/papers.html?search=personality, https://icml.cc/virtual/2026/papers.html?search=personality
  5. The experiment is only conducted on Qwen2.5-7B, which is not conclusive at all.
  6. The evaluation is based on human-calibrated register axes, which may not be accurate; e.g., being close to an axis does not mean being similar to a persona.
Questions for authors

Sincere suggestion: This is more like a case study paper than an ML research paper. I believe it may be more suitable for the psychology research community than the ML community.

Reviewer_9x34OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this work the authors explore the existence of individual differences in language models and propose a method to learn speaker-specific embeddings (addresses) using paraphrase inversion. Authors evaluate register grounding, diversity, and coverage using 640 podcast speakers and a Qwen2.5-7B backbone. Experiments show that the model can recover speaker-specific register on unseen speech, and can be interpreted to steer the model and add new speakers without changing any existing parameters. The paper also compares address-based personalisation to name prompting, few-shot demonstrations, and other parameterizations.

Critical strengths
  • Separation of grounding, diversity and coverage is a useful framework to evaluate personalised generation.
  • The empirical analysis is strengthened by held-out episodes, unseen shows, address-swapping experiments, and multiple training seeds.
  • It is instructive to compare with name prompting, few-shot demonstrations, and other parameter-efficient representations.
  • The ability to add speakers and freeze the existing parameters is practically useful.
  • The decoding and steering experiments indicate that the learned representations capture interpretable register characteristics.
Critical weaknesses

Major

1. Limited validation of individuality

  • Grounding relies mainly on ten predefined linguistic features, which may not capture richer stylistic differences.
  • No human perceptual study establishes whether generated text resembles the target speaker.
  • The correlation-based grounding metric does not directly measure absolute similarity to an individual's speech.

2. Limited advantage over alternative conditioning methods

  • The trained name-label baseline achieves stronger held-out grounding than addresses (0.2260 vs. 0.1166, Table 9).
  • Few-shot demonstrations also outperform addresses on grounding.
  • Withholding content words improves grounding at two of three seeds (Table 6), questioning the necessity of the proposed inversion formulation.
  • Removing the contrastive loss has limited and inconsistent effects on grounding.

Minor

  • The single-vector representation closely resembles existing control-token and soft-prompt approaches. The main novelty lies more in evaluation framing and training objective than in representation form.
  • The paraphrase inversion setup partially mitigates content–manner entanglement, yet plain-likelihood rows (empty prompts) are included during training; potential residual entanglement cannot be fully excluded.
  • The InfoNCE-style address contrast may incentivize maximal separability across speakers, potentially inflating diversity/coverage independent of true person-level differences.
  • The proposed method is evaluated using only Qwen2.5-7B, should do on atleast one more model. This limits the evidence of generalization across architectures.
  • Some comparisons change across the three training seeds (Appendix B.15).
  • Transcript attribution precision is 0.780, potentially introducing noise into speaker profiles.
  • Appendix B.17 compares alternative parameterizations primarily through likelihood and swap gaps, rather than the complete register-grounding evaluation.
Questions for authors
  1. How sensitive are grounding and diversity to the InfoNCE contrast term weight and the number of negatives? Could you report diversity/coverage changes in an ablation that removes the contrast but holds all other factors constant, especially on held-out episodes?

  2. Have you compared addresses to multi-token soft prompts or prefix tuning of length >1, or to per-user LoRA adapters of matched parameter budgets? If not, could you provide a small-scale comparison if possible to better situate addresses among common personalization containers?

  3. What happens if you add more Biber-inspired features or a learned style encoder as a complementary grounding target - do relative conclusions across conditions remain stable?

  4. Could you expand on the content - manner disentanglement achieved by the inversion setup? Beyond deleted habits and fixed prompts, did you test interventions (e.g., swapping reduced content across speakers) to verify that address-conditioned outputs preserve content while modifying only manner?

  5. For linear steering, do decoded directions for other habits (e.g., hedging, intensifiers) also generalize across speakers, or is involvement special? A small figure with effect sizes across all decoded habits would be useful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.