- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TransMem, an inference-time parametric memory module that reuses hidden states a frozen LLM has already computed during prefill. The context is split into segments (four by default) and the final hidden state of each segment serves as a sparse memory feature. During answer generation only, a small transformer block with a learned projection and a gating network maps these features to a memory shift that residually corrects hidden states at the last four layers. Training uses evidence-conditioned self-distillation: a teacher sharing the same frozen backbone sees only the gold evidence and the question, while the student sees the full context, and the module is optimized with a forward KL at the output distribution, updating only the memory modules.
Critical strengths
- The method is genuinely lightweight with context-independent overhead. With only four segment-final hidden states as memory, modules inserted into the last four layers and activated only during generation, compute and latency stay nearly constant as context grows (Figure 3), in contrast to the roughly linear cost of δ-Mem. Gains are consistent across three backbones and three benchmarks, so the improvement is not tied to one architecture.
- The ECSD objective is well motivated and the transfer results support the central claim. Distilling an evidence-only teacher into a full-context student provides token-level evidence-conditioned supervision, and Table 4 (SFT 48.82, GRPO 52.42, OPD 52.80, ECSD 54.11) isolates the contribution of the objective. Training only on HotpotQA and evaluating on LoCoMo and MemoryAgentBench supports the claim that the module learns a general memory capability rather than task knowledge.
Critical weaknesses
- Baseline comparability and statistical rigor are insufficient. The training corpora and setups of the parametric baselines in Tables 1 and 2 (-Mem, MLP Memory, Context2LoRA, MemGen) are not specified; if they follow their original configurations, the comparison confounds training-data effects with architectural ones. No multi-seed variance is reported, yet differences of 0.2 to 1 F1 in Tables 5 and 7 are used to support conclusions. On MemoryAgentBench (Table 2), TransMem trails δ-Mem on Accurate Retrieval (41.60 vs. 44.40) and MemGen on Long-Range Understanding (36.19 vs. 44.30); the aggregate lead rests on Test-Time Learning and Selective Forgetting, so precise retrieval ability is not established.
- The memory mechanism itself lacks direct evidence. Four memory states must stand for contexts up to 256K tokens, yet the paper offers no analysis of their capacity. Table 13 shows performance is nearly invariant to the number of segments B (three of five subtask scores are identical from B = 4 to 32), which is more consistent with the memory states providing a coarse global signal than with retrievable long-range evidence. Combined with Table 4, where plain SFT already recovers +8.0 of the +13.3 total gain and ECSD adds about +5.3, a substantial part of the improvement may come from generic output correction learned by training the module rather than from recovering long-range information. Without probing or perturbation experiments on H_M (e.g., masking or shuffling the segments containing gold evidence), the core claim is under-supported.
Questions for authors
- Were the parametric baselines in Tables 1 and 2 retrained on identical data, or are they original checkpoints? Please specify the training corpus and budget per baseline so that training-data effects can be ruled out.
- Please explain the insensitivity to B in Table 13 and provide analysis (e.g., masking or shuffling the segments that contain gold evidence, probing H_M) showing that the four memory states carry retrievable long-range information.
- Are the main results single runs? Please report mean and standard deviation over at least three seeds, especially for the differences below 1 F1 in Tables 5 and 7.
We thank all reviewers for their careful reading and constructive feedback. We have revised the manuscript to address the points raised in the reviews.
We reply to each reviewer below.