All Memory Is Local: Measuring the Horizon of Persistent Test-Time Training

Test-time training (TTT) lets a language model update a small set of fast weights as it reads. The drop-in methods considered here reset those weights at document boundaries, so nothing carries across documents. When the reset is removed, the model recalls symbols from earlier documents well after they have left the context, which looks like persistent memory. We ask how much of this signal is memory. Delayed recall has at least three other sources: information the model already had (pre-training contamination, in-context copying, synthetic vocabularies shared across sessions), properties of the evaluation (scoring order, length-normalized accuracy), and adaptation to the stream as a whole. We build a protocol, Streaming-TTT, that separates them: per-session obfuscated vocabularies, first-mention targets, fixed-lag re-scoring, and a control of sessions that were never streamed. On 180 code repositories and 200 prose sessions with Qwen3 models from 0.6B to 4B, persistence raises first-mention recall to 7.6 times the frozen model's immediately after a session, but the estimated session-specific component falls by about four fifths within five sessions; by lag 15 what remains cannot be distinguished from the gain on never-streamed sessions. A linear-interference model predicts that this decay depends on subsequent write mass rather than on the learning rate, and the measured curves collapse accordingly. Resetting at session boundaries keeps 60% of the immediate benefit and avoids the general-capability cost that unrestricted persistence incurs. In this setting, delayed recall alone did not establish persistent memory; the relevant quantity is what remains after adaptation is subtracted.