When Does Agent Memory Matter? Source Access Conditions Its Marginal Contribution

Agent memory is designed to preserve information beyond an agent’s immediate context, but when does that memory actually matter? We identify source accessibility (whether stored information remains obtainable from its original sources) as one condition shaping memory’s marginal contribution. Our full-text coding of a purposive sample of 19 agent-memory evaluations finds that none reports memory’s contribution across multiple source-access regimes. We introduce Memory Opportunity Audit (MOA), which freezes acquired memory and execution state and crosses downstream memory use with source access. In our cleanest document study, memory is constructed before the question and source loss is assigned independently of memory content. Factual notes contribute +3.33pp under intact access and +6.33pp under denial, yielding a +3.00pp interaction [+0.33, +5.74]. Adjudicated human scoring and a disjoint second cohort provide further evidence of access-conditioned contribution. The effect is heterogeneous rather than universal across readers, scorers, modalities, and retention regimes, with no consistent benefit for post-observation video memory. These results show that memory contribution need not be source-access invariant: evaluating memory in only one access regime can mischaracterize when that memory matters. Claims spanning access regimes should therefore report contribution within each regime.