No Single Memory Mechanism Wins: A Systematic Evaluation of Memory Routing Policies for LLM Agents

Long-horizon agents accumulate experience across sessions, and a growing number of memory mechanisms have been proposed to store it and read it back, e.g., vector indexes, knowledge graphs, summary trees, and strategy banks. These mechanisms are developed and evaluated separately, each on the regime it targets, which presumes that the type of the upcoming request is known. In deployment, however, an agent faces a mixed stream of requests and has to decide, per request, which form of memory to rely on. This decision has received little attention, and its value has never been measured against an oracle. In this work, we present the first systematic evaluation of memory routing for agents. We merge three benchmarks into a single pool of 2,614 requests and execute every request under all 12 memory substrates, which yields a complete matrix of 31,368 cells. Based on this matrix, we first show that query semantics, on which conventional routers operate, are insufficient to determine which memory is needed. We then quantify how much the choice is worth, i.e., selecting the right substrate for each request attains 64.74 against 36.82 for the best fixed one, a relative improvement of 75.85%. We finally evaluate 14 routers in five tiers under two generalization protocols. The best of them recovers only 32.8% of this headroom, and generalization to unseen memory units remains the binding constraint.