Bayesian User-State Inference for Online LLM Personalization

Personalizing large language models (LLMs) online requires inferring user preferences from sparse histories and refining these estimates as new interactions arrive. Existing approaches typically select or summarize histories without explicitly modeling uncertainty about preferences. Yet observed user behavior reflects both context and personal preferences, making limited evidence ambiguous and complicating how much to revise estimates. To address this ambiguity and refine preference estimates as evidence accumulates, we propose Bayesian User-State Inference (BayUSI), a framework that treats deviations from input-conditioned population predictions as personal evidence and combines them with shared preference patterns. A learned mixture prior captures recurring preferences across users without annotated groups, while Bayesian updates incorporate each user’s history into a posterior over their preferences. As interactions accumulate, posterior uncertainty governs how strongly new evidence updates preference estimates, providing query-specific conditioning for a frozen LLM. Training through the LLM’s prediction loss aligns preference inference with personalized generation without per-user fine-tuning. Experiments on four LaMP tasks show consistent improvements under sparse-history and evolving-history protocols, demonstrating the value of uncertainty-aware preference inference for online LLM personalization.