Liquid Resonance Memory: Ultra-Long-Context Recall and Write-Time Reasoning Under a Fixed GPU Memory Budget

Long-context benchmarks have grown to tens of millions of tokens. At that scale the key-value cache of an attention model occupies terabytes, while a recurrent state of fixed size has no way to hold on to a sentence seen 10^8 steps ago. This paper studies self-written memory (SWM): a language model reads a stream in chunks of 240 tokens, judges for each chunk whether it holds a fact worth keeping, writes such facts down as short notes, and later answers questions from a small memory drawn from those notes. Whether to write is decided by the model's own probability of answering "Nothing" to a fixed writing instruction; there are no extraction rules, and the criterion is the same for every task and every length. We run SWM with a 247M-parameter attention-free model made of liquid recurrent layers and spiking delta-memory layers, pretrained on roughly one billion tokens. On streams of 1M, 10M, 50M and 100M tokens in which BABILong facts and needle tasks are buried in Project Gutenberg text, SWM with a fine-tuned reader checkpoint scores 61.4%, 55.0%, 52.1% and 47.1%. With the writer checkpoint itself as reader the scores are 42.1%, 42.9%, 41.4% and 42.9%, compared with 24.3%, 25.7%, 17.9% and 5.7% when the same checkpoint answers from BM25-retrieved raw chunks. Because the task formats appear in our generated fine-tuning data, these numbers measure in-distribution scaling rather than zero-shot ability. The writer flags 98.2-100% of the fact-bearing chunks and 0.20-0.46% of the filler chunks; GPU memory during writing and reading is independent of stream length, and only the note store grows. Ablations, an oracle-memory bound and an error analysis trace most of the remaining errors to multi-hop reasoning in the reader and to facts that were written but not selected into the memory.