acceptodds
Under review as a conference paper at ICLR 2027

Is Persona Generalization Predictable?

Abstract

Fine-tuning a language model on narrow data can induce emergent misalignment, with unsafe advice, excessive sycophancy, or other undesirable traits appearing in seemingly unrelated domains. The breadth of these changes is difficult to anticipate from training data alone. Yet we uncover a simple quantitative structure underlying this cross-domain persona generalization: across model families and fine-tuning configurations, in-domain and out-of-domain persona elicitation rates often follow a shared linear relationship. This relationship allows the frequency of persona elicitation in unseen domains to be predicted from behavior in the training domain, even for open-ended generation. Common regularization methods, such as low-rank fine-tuning, fail to limit leakage into other domains without sacrificing in-domain elicitation. However, we show that the strength of this generalization can be controlled by the depth of the updated layers. The slope of the relationship generally decreases as updates move to a later layer, limiting out-of-domain generalization while maintaining the trained behavior. In-domain persona elicitation thus provides a measurable predictor of cross-domain behavior, while update depth provides a direct lever for controlling how broadly fine-tuning-induced traits generalize.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.