Persona Learning from Feedback for Personalized AI Assistants

Personalizing AI assistants requires adapting their behavior to users’ prefer- ences and task contexts. Existing work elicits assistant personas through persona prompting or finetuning on persona-conditioned generations. However, persona descriptions alone do not identify what to change or preserve in a particular re- sponse, or which behaviors suit the current request. We introduce Persona Learn- ing from Feedback (PLF), a framework that learns reusable, preference-grounded assistant personas as independent LoRA adapters on a shared frozen backbone. During training, a frozen self-teacher conditions on persona specifications and response-specific feedback to provide token-level supervision on each adapter’s own generated prefixes. A feedback gate focuses corrective updates on responses judged to need improvement. At deployment, a supervised router uses the request or dialogue to select a single adapter or a fixed composition, without user-specific finetuning. We evaluate PLF on single-turn personalization with model-generated critiques and multi-turn tutoring with simulated students’ reactions. Across both settings, PLF improves model-judged preference fit over fixed-persona policies and persona prompting with the same router, demonstrating how feedback-trained behaviors can be reused for adaptive assistant personalization.