- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether a frozen transformer can use its own early-layer residual states as a memory of a context. For every position of a context, it stores the residual activations output by layer 1 (the second decoder layer); to read them back, it places a period token at each stored position of a new prompt, overwrites the layer-1 activations there with the stored ones and recomputes all higher layers, without any training. The authors use this state memory in four ways: reading it in place of the source, combining memories written separately, revising a memory without the original text, and writing memory from experience, as the states of a note the model writes after reading tool-use episodes. Across long-context, reading-comprehension and synthetic tasks on Qwen and Granite models, memory performs close to having the text in context in most comparisons, and on the synthetic policy tasks it outperforms having the episodes in context.
Critical strengths
-
Careful experimental design. The comparisons are well controlled. The four-way swap test is a nice design: since every pairing of two memories has a different correct answer, a reader that ignored either memory could not pass it, and indeed the single-memory controls pass none of these tests (Tables 11 and 16). In Section 6, the 224-state episode windows and the reordering experiment in Figure 5 convincingly rule out that the note memory merely keeps a favorable stretch of the log. The conditions and analyses were also fixed on development data in advance (Appendix B.4). As a result, my concerns are less about whether the comparisons hold than about what state memory is for.
-
An interesting angle. Treating a frozen model's own layer-1 states as a memory that can be read, combined, revised and even written from experience, without any training, is a fresh angle. Storing residual or activation states in place of the KV cache has been explored before (e.g., HCache; XQuant, arXiv:2508.10395; KV-Direct, arXiv:2603.19664), concurrent work (CoMem) reads intermediate-layer residuals under new queries, and combining or editing precomputed KV caches has also been studied (Prompt Cache, CacheBlend, Li, 2026). To my knowledge, however, revising the residual states of a frozen model or writing them from experience has not been studied before, and this is what I found most interesting in the paper.
Critical weaknesses
-
Unclear advantage over keeping the text. My main concern is that the paper does not show what state memory offers over simply putting the text in the prompt. In Sections 3–5, memory is at best on par with full history, with differences within about one point (e.g., 75.5 vs. 75.0 on MRCR, 41 vs. 41 four-way tests on DROP, 935 vs. 935 on SQuAD). Meanwhile, writing a memory requires a pass over the text, revising it requires regenerating the text, and reading it still recomputes all but two layers, yet storage is compared only with the KV cache (Fig. 2b) and read-time compute is not reported.
Section 6 is the one place where memory outperforms the text in context (170 vs. 130 policy decisions, Table 4). However, no condition gives the reader the note as text, so the gain may come from having the model write the note rather than from the note's states. In the tool-call task, Appendix A.6 likewise suggests that the note's content decides the outcome: the note memory succeeds on 212/214 requests when the note names the echoed field correctly, but on only 1/42 when it does not.
Taken together, for none of the four uses does the paper show that the corresponding text in context would do worse, and beyond these four uses I find it hard to see what state memory would be useful for. The introduction motivates memory in general, but neither the abstract nor the main text explains why one would prefer state memory to the text; they largely list comparisons instead.
-
Incomplete release of raw results. The reproducibility statement says that the supplement contains the raw outputs of every experiment, but I could not find observation rows for handbook reuse (Tables 5 and 7–10), handbook composition (Table 11), ten-cycle revision (Table 12) or LoCoMo (Table 14). As a result, none of the results on Qwen3.5-27B and Qwen3.5-9B can currently be checked against released data.
Questions for authors
- What can state memory do that the text in the prompt cannot? How do its storage and read-time compute (or latency) compare with keeping the text and re-reading it, and is there a setting in which the states are available but the text is not, or in which reading the states is cheaper or more accurate than reading the text?
- In Section 6, how does a reader given the note as text compare with the note memory on the same 256 requests? In other words, does the gain over the episodes come from the states or from the note itself?
- Where in the supplement are the raw outputs for Tables 5, 7–12 and 14?
3 comments
I appreciate the authors’ additional experiments and clarifications. I still have concerns about the benefits over text, but would like to hear other reviewers’ views on the significance of these findings before deciding whether to raise my score.
We appreciate f1xg weighing the new measurements. For the reviewers' discussion of significance, we would ask them to weigh what these findings show a frozen model can do with its own early states and no training. Read in place of a 136,577-token context on Qwen3.8-27B, the states matched the context itself (75.5 against 75.0). Memories written separately could be combined, which the key/value caches from the same passes could not do (47 against 13 of 96 questions on Granite 4.2 8B). A memory revised ten times without rereading its source answered the same 935 of 1,056 SQuAD questions as the corrected context. The model also wrote such a memory from its own experience, and it carried a tool convention that no same-size slice of the original episodes carried (213 against at most 5 of 256 requests on Granite). Each trained model sets the deepest layer the states can come from, layer 2 on Qwen3.5-9B and layer 10 on Qwen3.5-27B, although the two share one layout.
Response to Reviewer f1xg
We are grateful to f1xg for the careful map of prior work, from HCache, XQuant and KV-Direct to CoMem, Prompt Cache, CacheBlend and Li (2026), and for locating what is new in revising a frozen model's residual states and writing them from experience. The review asks for one comparison, a setting at equal storage or read-time compute where the states do better than the text, and Section 3.4 measures one at equal read-time compute, with the same evenly spaced positions kept.
The review reads parity in Sections 3 to 5 as memory being at best on par with full history. Full history is the frozen model reading the context itself, the upper reference for any stored form of that context, and parity with it is the pass condition of each fidelity test. MRCR at 75.5 against 75.0 over histories of up to 136,577 tokens, LongBench, LoCoMo and the handbooks within one point, and 41 of 52 four-way DROP tests against 41 for full history each show that the stored states lost nothing the answers needed.
That comparison is Section 3.4 and Table 22. A reader given K evenly spaced positions of a handbook, as layer-1 states or as those K tokens, runs a prompt of the same length either way, and from half of the positions upward the Qwen3.5-9B states answered between 4.8 and 10.9 points more, with Qwen3.5-4B following the same pattern and LongBench MultiFieldQA-en adding 4.9 F1 at half of each document. The general response lists every interval. Part of the extra content is each position's neighbourhood, since a linear probe reads the preceding token from 44.7% of states and from 12.0% of embeddings (Appendix G.2).
On cost, one position on Qwen3.8-27B is 10 KiB at bf16 precision and near 2.7 KiB in the four-bit code, beside 64 KiB of key/value cache, and reading it recomputes 62 of the 64 layers, the prefill any reader of that context runs.
Several of the methods the review cites move and join precomputed key/value caches, so Section 4 now joins the caches that our own passage and question writers produced on 24 Granite DROP pairs. Those joined caches got 13 of 96 questions right where the states from the very same writer passes got 47 (Appendix G.5). Each joined cache keeps upper layers computed without its partner, and the state reader rebuilds them with the partner in view.
On question 2, we ran the note as text on all 64 Granite environment sets. In the same 224 positions it executed 213 of 256 requests, the count of its states, with the same outcome on 254, and decided 172 policy records against 171 (Table 2, Appendix G.4). So the policy gain over the fourteen episodes in context, 170 records against 130 and 438 composed workflows against 202, belongs to the note the frozen model chose to write, as the review infers from Appendix A.6, and the states carry that note in full. The revised abstract now reports the tie beside the 170 against 130 result. Section 6 claims exactly this, that a frozen model writes its own memory from experience and a reader holding only that memory's states acts on it.
On question 3, the supplement now holds the observation rows of handbook reuse (Tables 5 and 7 to 9), generation (Table 10), composition (Table 11), ten-cycle revision (Table 12) and LoCoMo (Table 14) on Granite 4.2 8B, Qwen3.5-27B and Qwen3.5-9B, with scripts that recompute every value of these tables on CPU, together with the rows of every new measurement in the revised PDF.
The review's main concern was what state memory offers over the text. The states answer more at equal read compute with evenly spaced positions, compose where joined caches fail, and carry the model's own note in full, and we ask f1xg to weigh these measurements in the final rating.