MCPool: Learning Joint Memory Allocation for Heterogeneous Reuse in MCP Agents

Agents using the Model Context Protocol (MCP) can reduce tool-invocation latency by retaining initialized execution contexts and reusing eligible tool results. These two forms of reuse differ in memory footprint, lifetime, and validity, yet compete for the same memory budget. Their benefits also overlap: a result hit avoids both context acquisition and tool execution. We formulate this heterogeneous dual-resource management task as a partially observable sequential decision problem, where future requests are unknown and retention decisions yield delayed latency benefits. To address this, we present MCPool, an RL-based controller for jointly managing context and result retention under a shared memory budget. Specifically, a recurrent policy summarizes class-level runtime statistics and selects a memory split and class-specific admission thresholds. A deterministic manager translates these controls into entity-level admission and eviction decisions, and enforces the shared budget limit independently of the learned policy. This separation enables robust scalability to shifting cache populations. Our extensive experiments across a wide range of tasks and baselines demonstrate that MCPool can achieve over a 1.34x 3.10x speedup while satisfying the memory budget with zero observed violations.