EvoMemory: Learning Future Utility for Bounded Memory Replacement

Large language models (LLMs) increasingly rely on long-term memory to support interactions that extend beyond a fixed context window. Yet memory capacity is inherently limited: as new experiences continually arrive, a system must decide not only what to store, but also what to retain, replace, or discard. Existing memory systems often rely on local signals such as recency, similarity, or standalone importance, which do not directly capture how a memory update will affect future task performance. We introduce **EvoMemory**, a framework that learns bounded memory replacement based on future utility. We formulate replacement as a sequential decision problem in which each incoming memory induces alternative successor memory states. During training, EvoMemory evaluates these counterfactual states on future examples to construct utility supervision and learns a contextual value model to score their downstream utility. At deployment time, the learned policy updates memory using only the current bank and incoming memory, without access to future examples. We evaluate EvoMemory on two complementary benchmarks, JRE-L and WikiLarge, against heuristic and random memory-management strategies. On JRE-L, EvoMemory achieves a held-out utility of 0.4719, compared with 0.4599 for five-seed random replacement and 0.4590 for the strongest deterministic heuristic. On WikiLarge, EvoMemory achieves a held-out utility of 0.8152, compared with 0.7887 for retaining the initial memory bank and 0.7726 for five-seed random replacement. Across both benchmarks, paired tests on held-out examples confirm significant improvements over all evaluated baselines after multiple-comparison correction. These results support future-utility learning as an effective principle for bounded memory replacement, where useful updates depend on their consequences for the resulting memory state.