What a Memory Can and Cannot Carry: Representation Shapes What an Experience Layer Can Hold
Abstract
Language-model agents are increasingly given persistent memory, and the usual question is how much it helps. We report an experiment in which one archive of stored observations helps, does nothing, or hurts according to what is derived from it and how that is written down. A deployed observing agent at a solar telescope accumulates nightly experience over 62 days; we ask it a quantitative question its input cannot answer — how many minutes until an obscured target is observable again — and compare memories built from that one archive: the prose its consolidation cycle generates, a stratified statistical snapshot rebuilt nightly, and the snapshot plus a note on the configuration's own past error. Against truth from future frames the prose form is indistinguishable from having no memory, while the statistical form cuts mean absolute error from 22–25 to about 16 minutes and bias from +16 to +5; the note on past error reaches 15–16 minutes and +2 to +3. Both reproduce across two runs against a pre-fixed criterion. What that rung supplies is a level, not a conditional prediction: it does not raise the slope of truth on prediction, and an arithmetic version of the correction cannot be told apart from it. Replaying the 680 recorded prompts against five models from four laboratories sharpens this: the last rung beats no memory resolvably on all five, whereas the prose form raises the bias above the memory-free level on four of five, failing a pre-specified criterion. The one model the prose does not harm is the one that wrote it, which invites an author effect; a writer-by-reader matrix does not support it — across this three-writer panel the ablest reader writes the prose least useful to every reader. A second agent supplies a zero point predicted in advance: where the memory-free configuration is already near-unbiased, every rung was run and the ladder is flat, so the effect tracks the miscalibration it corrects. No configuration reaches a one-line statistical rule, whose bias-corrected form is the strongest predictor here. Four measurement settings that each invalidated an earlier run are reported with the released harness.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.