MemLadder: Controlled Auditing of Agent Memory

Agent memory is commonly evaluated by comparing endpoint success with and without retrieved experience, but that comparison is not an attribution test for the retrieved text. It jointly changes the prompt channel, control text, action policy, exposure duration, and realized continuation. We introduce MemLadder, a controlled audit that separates behavioral rerouting, single-decision downstream value, and remaining-episode exposure-policy return. Its five mutually exclusive conditions range from bare context to retrieved source-grounded memory, including structural, length- and format-matched generic, and mismatched-real controls. From the same restored ALFWorld state, MemLadder executes each induced action, removes exposure text, and estimates return with five repeated memory-free executions under the frozen no-memory serving configuration. In a contemporaneous, human-verified audit, retrieved memory versus an actionable domain-general control produced actor-state-averaged substantive rerouting rates of 25.6% and 28.7%. On a fresh episode-disjoint 180-state panel, however, the one-step value contrast was −0.67 percentage points for both Qwen actors (episode 95% CI [−1.78, +0.45]). Under the binary K = 5 instrument, 94–98% of fresh-panel states were at an empirical text-arm floor or ceiling, so this small average contrast is a low-resolution estimate rather than evidence that memory is generally valueless. An exhaustive environment-only audit of all 5,231 admissible state–action choices finds a controller-relative outcome boundary in 50/180 states (27.8%), yet retrieved memory and the actionable control fall on opposite sides in only 2/180 and 5/180 states for 35B and 9B, respectively; mean preservation is 90.2% on controller-solvable states. Thus both weak-continuation failure and strong-continuation recovery can compress observed one-step separation. Behavioral rerouting, one-step downstream value, and remaining-episode exposure-policy return therefore require separate controls and should be reported as distinct estimands.