Teaching LLMs to Remember: Parametric Memory via Personalized Adapter Weights

Long-term personalization requires AI assistants to recall personal facts across sessions. Existing long-term memory systems mostly rely on token memory: facts are written as text and retrieved by searching among external embeddings. Although this design is practical and easy to maintain, retrieval is decoupled from answer utility, and the memory store grows with user history. Motivated by these limitations and the way LLMs compress knowledge into parameters, we explore whether personal memory can instead be represented at the parameter level. In this paper, we introduce **ParaMem**, a parametric memory framework for personal LLM assistants, where a user’s history is compressed into a compact per-user LoRA adapter and recalled by generating explicit memory contexts from the learned weights. This formulation shifts memory retrieval from external search to learned, personalized generation, while introducing two distinctive challenges: parametric recall must cover sufficient encoded history for a given query, and the generated memory should actually help answer the query rather than merely be related to it. To address recall coverage, we propose *ParamRecall*, an inference-time strategy that combines input-side query probing with output-side memory sampling, providing a parametric counterpart to top-K retrieval in token-memory systems. To study answer helpfulness, we further propose *memory-aware preference learning* and show that the quality of the negative construction is critical: different-subject hard negatives, which contrast memories about different people sharing the same fact type, provide a meaningful signal for distinguishing helpful memories from plausible but insufficient ones. Finally, we show that parametric and token memory are complementary: grounding the adapter's recalled memories in a stored fact bank corrects attribution errors and yields the best overall system. Experiments on real-world benchmarks show that ParaMem outperforms strong token-memory baselines across most question types, suggesting that parameter space is a promising design direction for long-term memory in AI assistants.