acceptodds
Under review as a conference paper at ICLR 2027

The persona hierarchy model for contextual generalization in LLMs

Abstract

Language models are routinely fine-tuned under a fixed context, such as a generic system prompt, persona or domain-specific instruction, yet the learned behavior sometimes stays confined to that context and sometimes broadly generalizes to unseen contexts. We propose the Persona Hierarchy Model to explain this: a shared default persona influences behavior across contexts. Under this model, fine-tuning that modifies the shared persona promotes broader transfer, whereas changes to local personas remain more context-specific. Across 120 fine-tuned models spanning four behaviors and 15 training contexts, generalization narrowness positively correlates with the similarity between the training context's persona and the default persona (Pearson's for Qwen3-4B). Prior fine-tuning under the default context can reduce this distance and broaden generalization in subsequent training. Aligning contextual responses with default-persona responses produces stronger effects. Finally, we propose persona-preserving regularization (PPR) to confine undesired contextual generalization. In RL, PPR cuts reward hacking from 42–55% to at most 0.2% under every evaluated prompt while retaining accuracy gains. These results support the Persona Hierarchy Model as an explanation for contextual generalization and can motivate future controls on unintended generalization for better alignment of LLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
fine-tuning, generalization, interpretability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NswsOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes the Persona Hierarchy Model, which posits that LLMs possess a shared default persona alongside context-specific local personas. During fine-tuning, the closer the trained persona is to the default persona, the more readily the learned behavior generalizes to other contexts. The authors experimentally validate a significant correlation between persona distance and the scope of behavioral generalization, and further demonstrate that training history and persona alignment can influence the degree of generalization. Building on this, the paper proposes PPR, which constrains the cross-context propagation of undesirable behaviors by protecting the default persona, thereby offering a new theoretical account and technical approach for safety alignment and controllable generalization of LLMs.

Critical strengths
  1. The phenomenon identified by the authors exhibits a certain of novelty.
  2. The authors discuss in detail how their findings can be applied to the post-training, which can inspire the future research.
Critical weaknesses

My main concern is that the current experiments do not substantiate the authors' claims. The paper defines personas solely via different system prompts, yet it neither verifies whether these prompts actually activate the intended personas nor rules out confounding factors such as the wording and semantics of the prompts themselves. It therefore remains unclear whether the differences in behavioral generalization stem from a persona mechanism or from the system prompts.

Questions for authors

Although the authors interpret their findings through the lens of persona theory, the underlying mechanism is essentially implemented by swapping system prompts and altering instruction. While the findings have clear practical value, I believe that streamlining the paper's claims and presentation would better help readers learn the core contribution and would be more likely to inspire the community.

2 comments

Thank you for your positive assessment and feedback. We would like to clarify the structure of our work. As stated in our introduction, the Persona Hierarchy Model is our proposed hypothesis to understand contextual generalization. Our work makes empirical claims to test this hypothesis.

We reveal that the distance between a prompt’s persona vector, measured from the model’s hidden states, and the default persona vector correlates with how narrowly learned behaviors generalize, supported by extensive experiments on varied datasets in Sec. 4. The correlation is further substantiated by intervention experiments showing that aligning with the default persona can broaden generalization (Sec. 5).

W1. The current experiments do not substantiate the authors’ claims.

Our experiments test our empirical claims directly: the correlation is observed across four behaviors and three models (Sec. 4, Figure 3), and interventions that move a context toward the default can broaden generalization (Sec. 5).

W2. The paper defines personas solely via different system prompts, … neither verifies whether these prompts actually activate the intended personas.

Thank you for your review. We clarify that we do not predefine personas. We measure the persona that each system prompt elicits from the model’s internal representations as its persona vector, following prior work that represents personas by model activations (Chen et al., 2026; Lu et al., 2026).

Our finding rests on the measured persona vector elicited by a given prompt, not on what a prompt describes. It does not require verifying that a prompt activates an intended persona.

References

  • Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2026). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. NeurIPS.
  • Lu, C., Gallagher, J., Michala, J., Fish, K., & Lindsey, J. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. ICML, PMLR 306, 80955–81003.

W3. It does not rule out confounding factors ... whether the differences in behavioral generalization stem from a persona mechanism or from the system prompts.

Thank you for your comment.

Empirical results suggest that surface properties of the system prompt itself may not account for the correlation:

Section 5 shows generalization can change under the same system prompt. We hold the prompt text fixed and align the model’s responses under that prompt with its responses under the default prefix. This reduces the measured persona distance and broadens the generalization of subsequently trained behaviors. Applying the same alignment procedure with other persona targets produces smaller effects (Table 2). These results provide intervention evidence that complements the correlation in Sec. 4, supporting our hypothesis.

We separately examine whether simpler prompt properties explain this relationship here.

ModelPrompt length: mean Pearson rPerplexity: Pearson r
Qwen3-4B0.210.12
Llama-3.1-8B0.180.06

Together, these results imply the relevance of the elicited representation beyond differences in prompt text alone.

We need to clarify our contributions do not claim that persona is the only cause for generalization or we exhaustively rule out all the other potential mechanisms, which is very hard. We leave what may shape a persona elicited as future work.

Instead, our work's core claim highlights the correlation between persona and how broadly a learned behavior generalizes across contexts with sufficient empirical support. We will further improve our presentation to make our contributions more clear.

Q. ... the underlying mechanism is essentially implemented by swapping system prompts and altering instruction. While the findings have clear practical value, I believe that streamlining the paper’s claims and presentation would better help readers learn the core contribution and would be more likely to inspire the community.

Varying the system prompt is indeed the core of our setting: we study how a behavior learned under one system prompt generalizes to others, which matters in practice because models are routinely fine-tuned under one prompt and deployed under others.

Thank you for acknowledging the value of our work. Following your suggestion, we have further refined our writing to make things more explicit in our updated PDF (changes highlighted in blue) and will keep refining our presentation.

Thank you for your detailed rebuttal. I have decided to maintain my positive score. I do not increase my score because, based on my original review, I do not suppose that the paper warrants the highest score.

Reviewer_y3fBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper motivates from the fact that learned behavior sometimes stays confined to the context but sometimes broadly generalizes to unseen context. Therefore, this paper introduces Persona Hierarchy Model to explain this situation. Their experiment finds out that generalization narrowness positively correlates with the similarity between the training context’s persona and the default persona. Then they propose presona-preserving regularization which attempts to confine such undesired contextual generalization.

Critical strengths
  1. The paper investigates why behaviors learned under one system prompt sometimes generalize broadly across contexts while others remain context-specific. This provides a useful perspective on unexpected generalization and emergent misalignment.
  2. Beyond identifying correlations between persona distance and generalization, the paper introduces training interventions that change subsequent generalization patterns. These experiments provide stronger support for the proposed hypothesis than correlation alone.
  3. PPR1 substantially reduces reward hacking while largely retaining the accuracy improvements from reinforcement learning. The comparison with standard policy KL and inoculation prompting further highlights its potential usefulness.
Critical weaknesses
  1. The interventions modify the model through additional fine-tuning, potentially changing multiple properties beyond persona distance. It remains unclear whether the observed changes in generalization are specifically caused by movement toward the default persona.
  2. The main experiments consider only two relatively small language models and represent contexts primarily through fixed system prompts. Whether the findings extend to larger models, multi-turn interactions, or more implicit contextual changes remains unclear.
Questions for authors

See weakness above

2 comments

Dear Reviewer y3fB, thank you for your positive assessment and constructive feedback. We address your concerns below.

W1. The interventions modify the model through additional fine-tuning, potentially changing multiple properties beyond persona distance...

Thank you for the review. Table 2 controls for the effect of additional fine-tuning by keeping the training procedure fixed and varying the alignment target (summarized results shown below). All Stage-1 conditions use the same training procedures and differ only in the alignment target:

  • Playful → Default: smallest distance after Stage-1 fine-tuning, lowest narrowness after Stage-2.
  • Playful → non-default personas: intermediate distances, smaller reductions.
  • Echo → Amnesiac (neither the training prefix nor the default): distance barely changes (0.0211 vs. 0.0221), and narrowness does not reliably drop.

Under the same fine-tuning procedure, alignment toward the default produces the smallest persona distance and the broadest subsequent generalization, while non-default targets produce weaker effects.

Thus, the results cannot be explained by additional fine-tuning alone. We also do not claim that persona distance is the sole factor changing generalization. Our core empirical claim is the distance–generalization correlation established in Sec. 4; these controlled interventions provide additional support for the correlation.

The interventions also have practical value. They offer a way to broaden generalization when desired, and they may help explain generalization in sequential fine-tuning.

W2. Only two relatively small models... extend to larger models, multi-turn interactions, or more implicit contextual changes remains unclear.

Scale. We add Qwen3.8-27B. The positive correlation between distance to the default persona and generalization narrowness persists (Pearson r = 0.58).

Practical importance of the current setting with fixed system prompts. They reflect standard practice: fine-tuning recipes pair data with system prompts, and models trained under one system prompt are routinely deployed under others. How learned behaviors transfer across these contexts therefore has direct consequences, such as reward hacking that persists under prompts prohibiting it [1] and a persona-specific quirk surfacing across a deployed model [2].

Implicit contexts. As acknowledged in our limitations, extending to implicit contexts, such as multi-turn conversations, is an important direction. It is nontrivial: it first requires defining what counts as the training context and an unseen context, so that generalization can be measured as cleanly as with fixed prompts. We believe this deserves dedicated future work, and we hope our findings will motivate it.

References

[1] Monte MacDiarmid, Benjamin Wright, Jonathan Uesato, et al. Natural emergent misalignment from reward hacking in production RL. arXiv:2511.18397, 2025.

[2] OpenAI. Where the goblins came from. https://openai.com/index/where-the-goblins-came-from/, April 2026.

Thank you for the response! I will maintain my score this time since the current score fairly reflects the quality of the work. Good luck!

Reviewer_zhnLOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes the Persona Hierarchy Model. The core hypothesis is that an LLM has a shared default persona, and that a training context closer to this default persona leads to broader generalization of the learned behavior. The authors train 120 models across four behaviors, fifteen training prefixes and two base models, and find a positive correlation between persona vector distance and generalization narrowness. They then use default-prefix training and default-persona alignment as interventions. Finally they propose persona-preserving regularization, or PPR, which reduces reward hacking in RL from 42-55% to at most 0.2%.

Critical strengths

The paper combines correlation and intervention evidence, which is stronger than observation alone.

Critical weaknesses

The core concept is measured in an indirect and unvalidated way.

The intervention and the narrowness metric are partly circular.

The evidence covers two models and a small set of prefixes.

The formatting of the appendix is very messy.

Total only 3 eqs and no framework picture.

Only 3 paper numbers for methods.

Questions for authors

Is the persona vector correlated with prompt length, domain or perplexity? How are these confounds excluded?

After intervention, do absolute behavior scores under other prefixes rise? Can the paper report behavior scores together with narrowness?

Is the difference between PPR and Policy KL only the place of the KL penalty? Can the paper ablate the penalty location?

Can the Persona Hierarchy Model be written as a formal model with testable predictions?

2 comments

Dear Reviewer zhnL, thank you for your time and questions.

W1. The core concept is measured indirectly and is unvalidated.

Following prior work [1,2], we represent the persona elicited by a system prompt as a latent-space vector. As stated in our introduction, the Persona Hierarchy Model is our proposed hypothesis to explain contextual generalization. Our main claim is empirical, which also tests our hypothesis: distance to the default persona vector correlates with generalization narrowness across four behaviors and three models (Sec. 4). Futher intervention experiments support our observations (Sec. 5).

[1] Chen, R., Arditi, A., Sleight, H., Evans, O., & Lindsey, J. (2026). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. NeurIPS.

[2] Lu, C., Gallagher, J., Michala, J., Fish, K., & Lindsey, J. (2026). The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models. ICML, PMLR 306, 80955–81003.

W2. The intervention and the narrowness metric are partly circular.

We are unsure which part of the intervention this concern is with.

Our intervention is not circular. It modifies prior training or alignment targets, whereas narrowness measures the subsequent difference between matched-prefix and off-prefix behavior rates on held-out data. The interventions do not directly optimize narrowness, and a reduction is not guaranteed by their objectives.

We address the possible concern by going through each part:

  • Sec. 5.1, diagonal pairs of Fig. 4: Stage 1 trains the same behavior, but learning continues during Stage 2. Table 8 in the appendix reports increases under both matched and off-prefix evaluation: +32.6 and +21.6 points for Goblin, and +17.0 and +18.5 for sycophancy. Thus, the final off-prefix behavior is not solely inherited from Stage 1.
BehaviorStage 1 only: MatchedStage 1 only: OffStage 2 only: MatchedStage 2 only: OffStage 1 + 2: MatchedStage 1 + 2: Off
Goblin40.246.463.80.472.868.0
Sycophancy60.552.367.146.177.570.8
Reward hacking74.264.578.027.076.871.0
  • Sec. 5.1, off-diagonal pairs of Fig. 4: Stage 1 trains a different behavior from the one evaluated after Stage 2. Narrowness nevertheless decreases in all six pairs, with varying effect sizes.
  • Sec. 5.2: Stage 1 aligns outputs on generic FineWeb text containing none of the target behaviors. Alignment toward the default produces stronger effects than the same procedure with non-default targets (Table 2).

We also report absolute behavior scores under Q2 below, showing that the reductions in narrowness accompany increases in off-prefix behavior.

W3. The evidence covers two models and a small set of prefixes.

We add Qwen3.8-27B and again find a positive correlation between distance to the default persona and narrowness (Pearson r = 0.58) across a variety of prefixes in Figure 3 of our updated PDF.

W4. The formatting of the appendix is very messy.

Thanks for your feedback. We have reformatted the appendix.

W5. Total only 3 eqs and no framework picture.

Figure 1 illustrates the Persona Hierarchy Model and its proposed relationship to contextual generalization. We have revised its caption to make this explanation clearer. The paper is empirical, and our equations mainly define the evaluation metric and training objectives.

W6. Only 3 paper numbers for methods.

We would appreciate clarification of this comment, as we cannot determine whether it concerns the length of the methods description, the number of references, or the experimental comparisons.

Q1. Is the persona vector correlated with prompt length, domain or perplexity? How are these confounds excluded?

We clarify that our study focuses on how the persona vector relates to generalization, rather than on what determines the persona vector.

We find that proximity to the default correlates with broader contextual generalization, observed over a diverse set of prefixes that vary widely in length, domains, and styles, including a random gibberish prefix.

We address the question further through controlled comparisons and additional quantitative analysis:

  • Fixed prompt (Sec. 5). When the system prompt is held fixed, so that its length, domain, and perplexity are unchanged, moving its persona vector toward the default persona can broaden generalization.
  • Distinctive prompts do not confine behavior by themselves (App. E). The most unnatural prefixes generalize broadly: a random code string (narrowness 11.4) and a digit string (8.2).

Analysis of prompt length and perplexity. Across our diverse prefixes, these properties correlate only weakly with narrowness:

ModelPrompt length: mean Pearson rPerplexity: Pearson r
Qwen3-4B0.210.12
Llama-3.1-8B0.180.06

These results complement the controlled experiments: prompt length and perplexity are weak predictors of narrowness, while generalization can change even when the prompt text remains identical.

Q2. After intervention, do absolute behavior scores under other prefixes rise

Yes. Off-prefix behavior rates rise in all nine comparisons, although the magnitude varies. The table below reports absolute rates together with narrowness:

Stage 1Stage 2Without Stage 1: MatchedWithout Stage 1: OffWithout Stage 1: Narrow.With Stage 1: MatchedWith Stage 1: OffWith Stage 1: Narrow.
GoblinGoblin63.80.463.572.868.04.7
GoblinSycophancy68.147.320.765.758.37.4
GoblinReward hacking69.940.229.775.656.818.8
SycophancyGoblin63.80.663.262.80.961.9
SycophancySycophancy67.146.121.077.570.86.7
SycophancyReward hacking76.035.640.474.238.935.3
Reward hackingGoblin63.32.360.967.018.448.6
Reward hackingSycophancy66.447.818.771.662.39.3
Reward hackingReward hacking78.027.051.076.871.05.8
  • Each row is one Stage-1/Stage-2 pair from Figure 4.
  • Matched: behavior rate (%) under the prefix used in Stage-2 training.
  • Off: mean behavior rate (%) under the prefixes unseen in training.
  • Narrow. = Matched − Off.
  • Without Stage 1: Stage 2 trained on the base model.
  • With Stage 1: Stage 2 trained on the Stage-1 model (the intervention).
  • All values are averaged over three Stage-2 training prefixes.

Q3. difference between PPR and Policy KL only the place of the KL penalty

Both apply a reverse-KL penalty toward the initial model with the same weight. Policy KL computes it under the training prefix, while PPR computes it under the default prefix.

Table 4 shows Policy KL suppresses hacking but keeps accuracy within 3 points of the base model, whereas PPR suppresses hacking and retains the accuracy gains of RL.

Q4. Can the Persona Hierarchy Model be written as a formal model with testable predictions

The Persona Hierarchy Model is a qualitative hypothesis. Committing to a specific functional form, such as a particular decomposition of hidden states, would itself be an unverified assumption about LLM internals; we hope our results can inform future theoretical work into LLMs.

We instead test the model empirically through its predictions:

  • Sec. 4: Prefixes closer to the default persona tend to yield broader generalization.
  • Sec. 5: Moving a training context toward the default persona broadens subsequent generalization, while alignment with non-default personas has a weaker or no reliable effect.

Comments on the paper

1 comment

Dear Reviewers,

As the rebuttal deadline is approaching, please review the authors' responses and participate in the discussion. Kindly indicate whether your concerns have been addressed and update your scores if appropriate.

Thank you for your time and effort!

Best regards,
Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.