The Severance Problem: LLMs are Unaware of the Person Beyond the Prompt
Abstract
Personal AI assistants have attracted significant interest for their potential to enhance everyday life by automating routine tasks, supporting consequential decisions, and assisting with everyday personal matters. Yet despite rapid technical advances, these assistants continue to exhibit undesirable behaviors, such as sycophancy, overconfidence, and hallucination. We argue that these failures stem from a fundamental limitation: language models lack an explicit representation of the person beyond the context they are given, which we term the **Severance Problem**. Even with rich personal context and strong commonsense reasoning from the backbone model, current AI assistants fail to represent what remains unknown about the user. We propose a simple solution: incorporating structured ignorance into the language model context via the **Severance Schema**, which explicitly outlines dimensions along which the model lacks knowledge about the user, including physicality, temporality, consequences, continuity, multiplicity, and interiority. Empirically, we test the schema on both a frontier proprietary model and open-source models. Across all five, the Severance Schema consistently reduces sycophancy and harmful advice. We further observe that assistants with memory tend to hallucinate more, while adding the schema mitigates this effect. Notably, models with the schema ask clarifying questions when information about the user is missing, rather than confidently extrapolating from incomplete user information. Code and data: https://anonymous.4open.science/r/severance-7B6C/
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.