Decision Instability Beyond Retrieval Metrics in Consolidated Agent Memory
Abstract
Language agents increasingly consolidate stored experiences to operate under bounded memory, yet consolidated memory is commonly evaluated by whether relevant information remains retrievable. We show that retrieval preservation does not imply preservation of the decisions supported by memory. Using a controlled intervention that holds target queries and outcome labels fixed while storing the same experiences in either original or consolidated form, we trace the effects of consolidation through the representation → retrieval → decision pathway. Aggregate retrieval success remains unchanged, with Hit@3 of 87.06% in both conditions, while downstream accuracy also remains nearly unchanged. Despite this aggregate stability, consolidation substantially reorganizes retrieved evidence and produces decision changes beyond ordinary repeat variability, with cross-condition disagreement exceeding same-memory disagreement by 5.20 percentage points. This instability concentrates on a reproducible subset of experiences. Vulnerable instances recur 9.43× and 10.84× above independence expectation under consumer-model and retriever substitution, respectively, even though the retrievers exhibit weak correspondence in their consolidation-induced changes. These results show that consolidation can preserve conventional retrieval and task-level performance while changing the evidence presented to the consumer and the decisions associated with particular experiences. Evaluating consolidated memory therefore requires considering not only whether relevant information remains retrievable, but whether the decision-relevant structure of stored experience is preserved.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.