- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Building on δ-mem, this paper studies how delta-rule associative memory states can be transferred from a writer model to readers with different backbones. The authors use a lightweight read head for each reader and introduce writer-side state canonicalization to support transfer across writers. Experiments show that the transferred states can improve question-answering performance, even when the source text is absent from the reader’s prompt.
Critical strengths
- The paper is well-written and technically sound.
- The motivation is clear.
Critical weaknesses
-
What's the stratege to set the relative-depth mapping in Eq. (6) across different backbones? A small ablation with alternative layer mappings would help justify this choice.
-
Paper only talk the condition that once-writter write and different-readers read. The paper handles multi-session histories, but it is unclear how independently written memory states would be retrieved, combined (or Merged), or updated as new conversations arrive.
-
How does the reader handle conflicts between memory and the current context? Testing outdated memories against explicit corrections in the prompt would help assess the risk of factual errors or hallucinations.
-
Beyond the alignment results in Appendix D, could the authors report how episode states vary in the coordinates defined by $\boldsymbol{M}_\ell$, and how stable these coordinates are across calibration samples?
-
Paper treates the KV-states as associative memory states to preserve, which is similar to LoRA adapation [1] (or HyperNet adaptation) which is also considered as memory adapted for each tasks. Could author explain some advantages for KV-states writting-and-reading?
[1] Mind Lab. MinT: Managed Infrastructure for Training and Serving Millions of LLMs. arXiv:2605.13779, 2026. https://arxiv.org/abs/2605.13779
Questions for authors
See above
Reviewer response
Thanks for your response!
W1: Novelty. This is the main problem. The rebuttal never says what is conceptually new or surprising. The title and introduction are ambitious, but the content is neither new nor surprising.
W2: Shared training. Not addressed. If writers trained on the same data agree only after per-writer SVD, that is exactly what my explanation predicts: the same structure up to a change of basis. Table 4c varies the calibration text, not the training data.
Q1: LoRA. Answered in letter, not in spirit. LoRA never sees the random facts, so its 3–5% was fixed by the design. The question is to know how much of the benchmark gain comes from QA fine-tuning, and the rebuttal explicitly declines to measure that.