acceptodds
Under review as a conference paper at ICLR 2027

Do Language Models Have a Model of You? Discovering and Enhancing Latent Representations of User Personas

11 views

Abstract

Humans carry working models of the people we know, and these models are useful because they persist across contexts and support predictions about future behavior. Language-model personalization is usually evaluated very differently: almost entirely by whether the model produces an adapted response. This creates a central blind spot. A system can achieve strong personalized output quality while maintaining a user state that is weakly recoverable, poorly calibrated, or unstable across different histories of the same person. We therefore distinguish behavioral personalization, which concerns whether outputs look adapted to a user, from representational personalization, which concerns whether the system maintains a recoverable and reusable user-state estimate that explains behavior across interactions. We formalize representational personalization through persona identifiability: a useful user state should be predictive of held-out behavior, recoverable from past interactions, and stable across partial histories of the same user. To study this distinction, we introduce \benchmarkname, a controlled diagnostic protocol built from HumanLM user profiles and interaction records under a strict public/private split, and \methodname, an EM-inspired latent refinement procedure that couples behavior prediction with persona inference over explicit user-state estimates. Across controlled recovery, calibration, history-stability, user-shuffling, long-history, and real-user personalization evaluations, we find that output quality and user-model quality are related but not equivalent: models with similar or strong downstream personalization can differ substantially in the recoverability and stability of their inferred user states. Moreover, explicitly optimizing for identifiable user states improves recovery, calibration, cross-history consistency, and held-out user behavior prediction while remaining competitive or stronger on downstream personalization tasks. These results suggest that personalization should be evaluated not only by what a model says, but also by the quality of the user model that supports its behavior.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
personalization, user modeling, persona representations, language models, latent user states, representation learning, user simulation, behavioral prediction, interpretability, human-centered NLP

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HSJQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper asks whether personalized output quality adequately reflects the quality of user representations. It proposes evaluating explicit user states through recoverability, stability, and predictivity. The authors introduce HUMANLM-PROBE and PERSONA-LR, which updates user profiles through behavioral scoring and alternating fine-tuning. Experiments report improvements in persona recovery and several personalization tasks.

Critical strengths
  1. The paper distinguishes personalized output quality from user representation quality and introduces HUMANLM-PROBE as a concrete framework for evaluating the recoverability, stability, and predictivity of user states.
  2. PERSONA-LR uses feedback from behavior prediction to update explicit user profiles, connecting persona inference with behavior modeling. It improves persona recovery and performance on several downstream personalization tasks.
Critical weaknesses
  1. The evidence for the central claim is not sufficiently direct. Figure 3 compares generation and recovery scores across different datasets, making it difficult to rule out differences in task capabilities. The shuffling experiments in Tables 5 and 7 show that user information is useful, but do not demonstrate reuse of the same state across contexts. Table 6 also lacks baselines under the same conditions. It therefore remains unclear whether the method builds more stable user models beyond improving persona recovery scores.

  2. The contribution of repeated refinement remains unclear. Section 5.4 mentions an ablation using a single refinement step without cache updates, but provides no numerical results. It is also unclear whether the SFT-only baseline uses a matched training budget. In Table 3, this baseline has a PANDORA parse rate of 0%, which confounds the comparison with differences in output formatting ability. Comparisons with matched budgets and output constraints are needed to establish whether iterative refinement accounts for the gains.

Questions for authors
  1. If a user state is inferred once and then kept fixed, does it still outperform SFT-only in other held-out contexts?
  2. Can the authors provide results for the single-step, no-refresh ablation with matched training budgets and output constraints?
  3. How are the history views in Table 6 sampled? After controlling for overlap between views, is the full method still more stable than SFT-only?
2 comments
AuthorsOpenReview ↗

Response to questions on Persona Identifiability

Thank you for your feedback. We conducted additional experiments to address the concerns about predictive validity, independent-history consistency, and missing baselines.

1. Frozen-persona transfer — We agree that the original cross-benchmark association was insufficient to establish predictive utility.

To address this, we infer personas from observed histories, freeze them, and evaluate held-out behavior using the same frozen predictor (LM1). No evaluation targets are used to update personas. CE gain measures the reduction in held-out cross-entropy from conditioning on the correct user's inferred persona.

Persona sourceRecovery MAE ↓Held-out CE gain ↑ [95% CI]
Untrained Qwen3-8B0.380+0.060 [0.034, 0.086]
SFT-only0.363+0.061 [0.035, 0.086]
One-shot0.368+0.063 [0.037, 0.090]
PERSONA-LR final0.307+0.091 [0.052, 0.121]

Paired user-clustered bootstrap CE improvements (95% CI):

ComparisonPERSONA-LR improvement
vs. SFT-only+0.0303 [0.0109, 0.0459]
vs. One-shot+0.0281 [0.0055, 0.0483]
vs. Untrained+0.0312 [0.0105, 0.0526]

All three paired intervals exclude zero. Unlike Figure 3's cross-benchmark association, this experiment fixes the predictor and evaluation contexts. Thus, we now have direct evidence that PERSONA-LR's inferred personas provide greater held-out predictive utility than the principal baselines.

Independent predictor check: With an unadapted HumanLM predictor, PERSONA-LR obtains CE gain +1.103 [0.540, 1.676], compared with untrained +1.008 [0.493, 1.554], SFT +1.014 [0.482, 1.551], and one-shot +1.064 [0.536, 1.630]. The trend is consistent, although statistical superiority is not established. HumanLM's larger CE gains result from overconfident predictions and the 10-12 probability floor, which inflates cross-entropy differences. Since changing this floor also changes model rankings, we report these results for completeness but do not draw conclusions from them.

Disjoint-history consistency — We agree that shared-history agreement alone does not establish user-specific consistency, but we have conducted a new experiment evidence beyond that confound.

We independently infer personas from two non-overlapping history subsets for each user. Separation measures same-user minus different-user cosine similarity.

Items/viewPERSONA-LR ↑SFT ↑Difference [95% CI]
10.0640.054+0.010 [-0.009, 0.056]
20.0930.085+0.008 [-0.008, 0.022]
30.1320.121+0.011 [-0.014, 0.032]
50.2330.208+0.025 [0.009, 0.049]

Recovery MAE also favors PERSONA-LR across all observation budgets: 0.410/0.375/0.357/0.337, compared with SFT's 0.435/0.402/0.383/0.362.

At five items/view, the separation improvement is statistically significant. At smaller budgets, the differences remain inconclusive. Unlike Table 6, the two representations share no observed history items, and SFT follows the identical protocol.

We therefore disagree that the observed consistency can be explained solely by overlapping evidence. The new results support improved user discrimination from independent histories, although they do not establish uniformly higher absolute within-user similarity or formal identifiability.

2. One-shot refinement and compute — We agree that the original manuscript did not report the one-shot baseline clearly, and that compute differences matter for causal attribution.

We therefore added a one-shot baseline under the same evaluation protocol. PERSONA-LR improves recovery MAE and held-out CE gain.

These results establish that the full method outperforms the evaluated one-shot baseline. However, because training budgets differ, we do not claim that this isolates the effect of iterative refinement alone.The comparison establishes a performance advantage under the evaluated training budgets, but does not isolate the effect of repeated refinement from additional training compute.

PANDORA evaluation. We agree that the SFT-only model's 0% parse rate makes its reported correlation unreliable. The evaluation requires five numerical Big Five trait predictions; missing values are imputed to 50. Because SFT-only failed to produce complete valid outputs, its predictions became constant, making Spearman correlation undefined rather than 0.000. We will correct this to N/A.

We evaluate 200 test records (seed 17), using up to 10 comments per record. Spearman correlation is computed separately for each of the five traits and then averaged. We will clarify the parsing procedure and distinguish output-format compliance from trait-recovery accuracy. Consequently, we do not treat SFT-only's formatting failure as evidence of inferior persona recovery.

I appreciate the authors' efforts in addressing my concerns through the additional results. After considering the rebuttal, my final rating is 2.

Reviewer_wLgHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper distinguishes personalized output quality from recoverable, reusable user-state estimates. HUMANLM-PROBE measures recovery, consistency across partial histories and predictive utility using controlled latent traits. PERSONA-LR couples state inference with behavior prediction: it proposes explicit states, selects them by fit to observed training behavior, and uses them in alternating supervised updates. The full framework improves controlled recovery and several downstream outcomes over supervised tuning alone. User-matching and partial-history controls assess specificity and consistency, while external evaluations cover prediction, generation and trait inference. The empirical findings support evaluating state quality alongside task performance, with direct trait-recovery conclusions limited to the controlled benchmark schema.

Critical strengths
  1. The introduction clearly distinguishes response adaptation from explicit user-state estimation. Organizing the evaluation around recovery, consistency and utility makes the diagnostic argument easy to follow.

  2. Behavior-based candidate selection gives the inference model a concrete training target without requiring persona labels. Explicit states also make the learned user estimate inspectable.

  3. Withholding hidden traits and final targets makes direct recovery comparisons interpretable within the controlled schema. Using equal observable evidence across methods strengthens this protocol.

  4. Preference-prediction and generation results support practical utility beyond controlled recovery. The SFT-only comparisons evaluate the full framework against ordinary tuning.

  5. Shuffled histories reduce controlled recovery, and shuffled states remove the accuracy gain in the controlled behavior-prediction test. Partial-history comparisons separately measure consistency as evidence grows.

Critical weaknesses
  1. Please point to the settings and results for the one-shot/no-refresh ablation invoked in Sec. 5.4. The SFT-only control does not isolate the effect of repeated refinement.
Questions for authors
    1. For Table 3's PANDORA correlations, which users and traits enter the calculation, how are invalid outputs handled, and how are correlations averaged?
  1. For Table 4, how do the continuous trait targets define the probabilities or soft targets used in Brier and ECE? Please specify the binning and aggregation needed to interpret the calibration claim.

4 comments
AuthorsOpenReview ↗

Response to questions on Persona Identifiability

We sincerely thank the reviewer for the thoughtful and constructive feedback, and particularly for recognizing the importance of evaluating persona recoverability, consistency, and predictive utility separately. The questions raised helped us identify opportunities to strengthen both our experiments and methodological reporting. We address each point below.

1. Quantitative one-shot refinement ablation — We agree that a direct comparison is necessary to better establish the contribution of the full procedure.

Following the reviewer's suggestion, we conducted an additional frozen-persona evaluation comparing PERSONA-LR with one-shot refinement and SFT-only.

ModelsRecovery MAE ↓Held-out CE gain ↑
SFT-only0.363+0.061
One-shot (50 cycles)0.368+0.063
PERSONA-LR0.307+0.091

PERSONA-LR improves held-out CE gain over one-shot by +0.0281 (95% CI [0.0055, 0.0483]) and SFT-only by +0.0303 ([0.0109, 0.0459]). Both paired intervals exclude zero.

These results provide quantitative evidence that the full procedure outperforms the evaluated alternatives in recovery accuracy and downstream predictive utility. We agree that differences in training budgets prevent a compute-matched conclusion, and will make this limitation explicit in the revision.

2. PANDORA correlation and invalid-output handling (Table 3) — We agree that the evaluation protocol and treatment of invalid predictions require clarification.

Thank you for highlighting this issue. The evaluation uses 200 sampled records from the PANDORA test split, with up to ten comments per record. We evaluate five Big Five traits (O, C, E, A, N), computing Spearman correlation separately for each trait across records and reporting their unweighted mean.

A shared parser extracts scores on a 0–100 scale. Missing predictions are imputed as 50, and the parse rate measures the percentage of records for which all five traits are successfully extracted. Thus, the reported correlations include imputed values.

Importantly, The SFT-only model produced no parseable trait scores within the 64-token generation budget. After imputing missing values to 50, all predictions were constant, making Spearman correlation undefined.We have corrected this entry to N/A in the revised manuscript.

We also agree that differences in parsing reliability complicate comparisons of trait recovery. Given the small correlations, bootstrap uncertainty, and differences in parsing reliability, we do not interpret the PANDORA rankings as evidence of statistically established superiority. We appreciate the reviewer identifying this reporting issue.

3. Brier score and ECE for continuous traits (Table 4) — We agree that the original metric descriptions could be misleading without explaining the target transformation.

The controlled benchmark contains 16 continuous trait targets in [0, 1]. Our current implementation thresholds each target at 0.5 to obtain a binary label, while retaining the model's continuous prediction as the scoring input.

Brier is calculated as the mean squared difference between the predicted value and the thresholded label.

ECE uses ten equal-width prediction bins. Within each bin, we calculate the absolute difference between the mean predicted value and the observed fraction of positive thresholded labels, weighted by bin occupancy. ECE is computed separately across the 16 traits in each task and averaged across tasks.

We agree with the reviewer that these metrics should not be presented as direct measures of continuous-trait calibration. We will therefore relabel them threshold Brier and threshold ECE, include their exact definitions, and emphasize Spearman, MAE, and RMSE as the primary measures of continuous-trait recovery.

This clarification improves the interpretability of Table 4 without changing the underlying experimental measurements.

Thank you for the additional experiments and clarifications. There are some issues remain unresolved:

  1. For my W1, please explain how one-shot/no-refresh changes the loop at step 4 and cache refresh at step 14 of Alg. 1.

  2. Since Response 1 acknowledges different training budgets, please revise the claim in Sec. 5.5, so that it does not attribute the gain to repeated refinement alone.

  3. For my Q2, Response 3 explains how the metrics are calculated. However, the paper still says "for probabilistic calibration" in the Sec. 4 and "Calibration also improves" in Sec. 5.4.

I retain my weak acceptance.

AuthorsOpenReview ↗

Clarifications for questions

Thank you for your follow-up and for retaining your positive assessment. We have revised the manuscript to address your remaining concerns.

1. One-shot/no-refresh ablation. In the one-shot/no-refresh setting, step 4 of Algorithm 1 continues for 50 training cycles. Each cycle generates K=8K=8 candidate personas, which LM1 evaluates against observed behavior to select the lowest-loss candidate for LM2 supervision. However, step 14 is disabled: the cached persona remains fixed at its initial value rather than being replaced by the selected candidate. Thus, candidate selection and model updates continue, but selected personas are not propagated across refinement rounds. This removes iterative state refinement while retaining behavior-grounded candidate selection.

2. Attribution to repeated refinement (§5.5). We have revised the claim to acknowledge the unmatched training budgets. The manuscript now states:

"The one-shot/no-refresh comparison shows that the full PERSONA-LR procedure achieves stronger held-out predictive utility than the evaluated one-shot baseline. Because training budgets differ, this comparison does not isolate the effect of repeated refinement from additional training compute."

3. Brier and ECE interpretation (§§4 and 5.4). We have revised both sections. Section 4 now identifies these metrics as threshold Brier and threshold ECE, computed after binarizing continuous traits at 0.5. Section 5.4 now explicitly states that the results concern thresholded binary trait labels rather than calibration of the original continuous traits.

We appreciate these suggestions, which have helped us make the evaluation and interpretation more precise. We have now updated the PDF, thank you for your advice!

Thank you for the clarifications and revisions. I will raise my Soundness score from 2 to 3 while retaining my overall rating of 3, weak acceptance. I have no further questions.

Reviewer_514AOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
1
Read review
Paper summary

This paper studies whether strong behavioral personalization implies that a language model maintains a recoverable and consistent representation of its user. The authors introduce HumanLM-Probe, a controlled evaluation protocol for persona identifiability, and PERSONA-LR, an EM-inspired training procedure that alternates between user-state inference and behavior prediction. Candidate persona states are selected according to their ability to explain observed user behavior and then used as supervision for the inference model.

Critical strengths

The paper addresses a limitation of output-based personalization evaluation. Personalized response quality alone does not determine whether the inferred user state is accurate, user-specific, or consistent across interactions. Evaluating these properties separately is a useful research direction, particularly for systems that maintain persistent user profiles.

Critical weaknesses
  1. The central claim is that PERSONA-LR learns identifiable and stable user representations. However, its core mechanism selects personas based on their likelihood of explaining observed behavior. A persona that predicts observed behavior well is not necessarily an accurate or stable representation of the user, since different personas may explain the same behavior. Moreover, the behavior model is updated using the selected personas, potentially reinforcing its own preferences.

  2. Algorithm 1 conditions persona inference on behavior-model feedback (fif_i), which depends on the observed target behavior (xix_i). However, this target is unavailable when predicting future user behavior. The paper initially defines inference as (q_\phi(z\mid e)), while the training algorithm uses (qϕ(z∣e,z(t),f(t)))(q_\phi(z\mid e,z^{(t)},f^{(t)})), and Appendix M again describes inference conditioned only on (e). It is unclear which formulation is actually used at inference time and how the model handles the absence of target-dependent feedback. This ambiguity affects the understanding of the proposed learning procedure.

  3. PERSONA-LR combines additional supervision, candidate selection, and repeated model updates, using approximately 1,224 GPU-hours. However, the experiments do not establish whether its advantage comes specifically from behavior-grounded selection rather than additional iterative training. The SFT-only comparison is useful, but its computational comparability is not sufficiently established, and the one-shot/no-refresh ablation mentioned in Section 5.4 lacks clearly reported quantitative results. Without these controls, the independent contribution of the proposed mechanism remains uncertain, particularly given existing behavior-guided persona refinement methods.

  4. HumanLM-Probe evaluates recovery of predefined synthetic traits, and authors acknowledge that observable histories may contain informative semantic cues. Thus, successful recovery does not necessarily demonstrate inference of persistent behavioral characteristics beyond those cues. More importantly, the stability experiments do not directly support the authors' interpretation. In Table 10, PERSONA-LR has lower same-user similarity than Base Qwen3-8B, despite a larger same-user/different-user gap. This supports better user discrimination, but not necessarily stronger stability. Most real-user experiments evaluate downstream performance rather than these representation properties directly.

  5. Several results cannot be reconciled from the reported evaluation procedures. Tables 4 and 9 give different improvements over Base Qwen3-8B: the Spearman difference is 0.020 in Table 4 but 0.0042 in Table 9, while the corresponding MAE improvements are 0.068 and 0.0456. Furthermore, averaging the four equally sized trait-family MAEs in Table 12 gives approximately 0.327 for PERSONA-LR, rather than the overall MAE of 0.304 in Table 4. These differences require an explicit explanation of evaluation subsets or aggregation procedures, as they directly concern the paper's main recovery results.

  6. The paper contains many presentation issues. In particular, the inference and update procedures are described inconsistently across Section 3, Algorithm 1, and Appendix M. The one-shot/no-refresh ablation mentioned in Section 5.4 lacks clearly reported quantitative results. The caption of Figure 2 does not accurately describe its right-hand plot. The plot shows the relationship between mean cosine similarity and Spearman correlation, whereas the caption discusses downstream utility and user-state recoverability. The latter relationship is actually illustrated in Figure 3. There are also incorrect appendix references (e.g., "App. A.3" and "Sections A.2–A.4") and minor grammatical errors (e.g., line 651).

Questions for authors

Please refer to the weaknesses

2 comments
AuthorsOpenReview ↗

Response to questions on Persona Identifiability

Thank you for your feedback. We clarify your methodological concerns below and report two additional experiments testing predictive transfer and persona consistency.

1. Persona identifiability. We agree that behavioral likelihood alone does not guarantee an identifiable user representation. We therefore evaluate recovery of ground-truth traits, stability across histories, and prediction of unseen behavior. HumanLM-Probe provides withheld ground-truth traits, while shuffled-history controls test dependence on user-specific evidence.

Experiment A: Frozen-Persona Transfer. We test predictive reuse by inferring personas from observed user histories, freezing them, and evaluating their ability to predict unseen user responses. All methods use the same frozen behavior predictor (LM1), and only the inferred persona changes. CE gain measures the reduction in held-out cross-entropy relative to LM1 without a persona.

Persona modelRecovery MAE ↓Held-out CE gain ↑ [95% CI]
Untrained Qwen3-8B0.380+0.060 [0.034, 0.086]
SFT-only0.363+0.061 [0.035, 0.086]
One-shot0.368+0.063 [0.037, 0.090]
PERSONA-LR0.307+0.091 [0.052, 0.121]

Paired user-clustered bootstrap CE improvements (95% CI):

  • vs. SFT-only: +0.0303 [0.0109, 0.0459]
  • vs. one-shot: +0.0281 [0.0055, 0.0483]
  • vs. untrained: +0.0312 [0.0105, 0.0526]

All three intervals exclude zero. Unlike Figure 3's cross-benchmark association, this experiment tests held-out predictive utility under a common predictor, without adapting personas or LM1 to evaluation targets.

2. Training versus inference. Algorithm 1 uses observed behavioral targets during training to score candidate personas and construct supervision. At inference, held-out behavior and feedback are unavailable; the trained inference model predicts personas using observable user histories alone. Thus, behavior-grounded refinement is a training procedure, not an inference-time requirement.

3. Refinement and compute. We agree that compute-matched comparisons are needed to isolate the effect of iterative refinement. However, our new experiments provide quantitative evidence of improvement over one-shot refinement. PERSONA-LR achieves lower recovery MAE (0.307 vs. 0.368) and higher held-out CE gain (+0.091 vs. +0.063), with a paired improvement of +0.0281 [0.0055, 0.0483]. Since training budgets differ, these results establish a performance advantage over the evaluated one-shot baseline but do not isolate the effect of iterative refinement from additional training compute.

4. Generalization. We agree that synthetic traits and semantically informative histories do not establish unrestricted real-world persona identifiability. HumanLM-Probe evaluates recovery of controlled latent attributes, complemented by real-user downstream evaluations. We further address the reviewer's stability concern through Experiment B, which tests whether inferred personas remain user-specific across independent histories.

Experiment B: Disjoint-History Consistency. We agree that the original stability evaluation did not fully address overlapping histories or include an SFT comparison. Our new experiment addresses both by independently inferring personas from non-overlapping histories and measuring same-user minus different-user mean-centered cosine similarity.

Items/viewPERSONA-LRSFTDifference [95% CI]
10.0640.054+0.010 [-0.009, 0.056]
20.0930.085+0.008 [-0.008, 0.022]
30.1320.121+0.011 [-0.014, 0.032]
50.2330.208+0.025 [0.009, 0.049]

PERSONA-LR achieves greater separation at every budget, although only the five-item difference has a confidence interval excluding zero. Recovery MAE also favors PERSONA-LR at all four budgets: 0.410/0.375/0.357/0.337 versus SFT's 0.435/0.402/0.383/0.362.

Regarding Table 10, we agree that absolute within-user similarity is lower than Base Qwen3-8B, but emphasize that user-specific separation and absolute similarity measure different properties. We claim improved discrimination, not uniformly higher within-user similarity.

5. Numerical discrepancies. We agree that the original Table 4 contained a transcription error. The correct PERSONA-LR recovery MAE is 0.327 ± 0.004, rather than 0.304 ± 0.003. This correction reduces the reported improvement but does not change our main finding: PERSONA-LR achieves lower MAE than Base Qwen3-8B (0.372) and SFT-only (0.352 ± 0.007). Yi-1.5-9B achieves a lower MAE (0.318), and we have corrected the corresponding ranking and highlighting. The corrected PERSONA-LR and SFT-only MAEs also agree with the averages of their four trait-family results. We have updated the recovery results consistently across the revised manuscript.

Thank you for the response and additional experiments. I appreciate the clarifications, which address some of my concerns.

However, I am still not convinced that improved prediction demonstrates stable and identifiable user representations. The contribution of iterative refinement also remains unclear without compute-matched comparisons. Moreover, the substantial correction to the main results raises concerns about the reliability of the evaluation.

Overall, I do not think it resolves the main weaknesses of the paper. I raise the Presentation score from 1 to 2, but maintain my original final rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.