BRANCHMEM: Branching Reasoning with Persistent Latent Memory for Language Model Agents

Agent memory enables LLM-based agents to improve by accumulating and reusing past experience. Latent memory provides a more compact representation of past experience than raw trajectories, but existing approaches often either absorb experience into shared trainable components or require additional optimization for each explicit latent memory. We introduce BRANCHMEM, a framework that uses a lightweight compressor to transform successful experiences into persistent latent memories. Each successful experience is encoded once into a memory that can later be retrieved and reused for subsequent tasks, without updating the underlying reasoner or optimizing each memory separately. We further find that different latent memories often alter generation only at a few critical points, yet these differences can redirect the entire reasoning process. Based on this observation, BRANCHMEM introduces Action Branching, which allows different latent memories to guide alternative reasoning paths at critical points while preserving the reasoning already performed, enabling more effective use of past experience for solving new problems. We evaluate BRANCHMEM across mathematical reasoning, code generation, and information-seeking tasks, demonstrating its ability to leverage past experience for new problems and showing how Action Branching further improves reasoning by exploiting alternative paths induced by latent memories.