The Trap of Trajectory: Towards Understanding and Mitigating Spurious Correlations in Agentic Memory

Agentic memory lets LLMs persist information beyond a single context window and reuse it in later decisions, but it also opens a new vulnerability: spurious correlations, where retrieved memory carries mis-correlated evidence and propagates erroneous reasoning into downstream decisions. Despite the widespread use of agentic memory, this risk remains largely underexplored. We address it from two aspects. First, we benchmark several canonical types of spurious patterns identified through causal structure recovered from reasoning trajectories. Diagnosing agentic memory systems on this benchmark reveals that memory improves reasoning on clean inputs but amplifies reliance on spurious patterns when present. Subsequently, we propose CAMEL, a plug-and-play calibration that operates on diverse memory architectures at write and retrieval time. CAMEL consistently reduces reliance on spurious patterns across all three types while preserving or improving performance on clean inputs, and stays robust under adaptive attacks targeting the calibration. Overall, CAMEL offers a principled and lightweight solution toward more reliable agentic memory deployment.