Parameters as Agentic Memory: Internalizing Long-Horizon Memories for Efficient LLM Agents

LLM agents need long-term memory, but most current systems keep memory outside the model and repeatedly append retrieved text to the prompt, which burdens both long-context reasoning and serving efficiency. We propose Parameters as Memory (PAM), a high-level methodology that treats durable agent knowledge as memory that can be internalized into parameter-efficient adapters. Inspired by neuroscience views of replay and consolidation, we instantiate PAM with Reflective Memory Optimization (REMO), a data rewriting and training pipeline that treats reflection over dialogue history as memory-oriented optimization. REMO reconstructs overlapping dialogue windows, rewrites them through intrinsic, cross-turn, network-structuring, and interaction-oriented views, and then trains LoRA adapters with supervised fine-tuning so the model better internalizes the rewritten memories. Across LoCoMo, LongMemEval, and MemoryAgentBench, PAM improves over the base model and non-rewriting parameter-memory baselines; it is especially effective for temporal reasoning, test-time learning, recommendation, summarization, and conflict resolution, while explicit retrieval remains stronger for verbatim lookup.