- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AdaMem combines semantic chunking, surprise-based memory allocation, and selective replay for long-document QA. It compresses context into soft memory tokens and retrieves original chunks when the compressed memory is insufficient. Experiments on extended versions of HotpotQA, 2WikiMQA, and TriviaQA show promising improvements over the evaluated baselines.
Critical strengths
- The framework is well motivated and brings together semantic chunking, adaptive compression, and selective replay in a coherent way.
- Selective replay offers a practical way to recover evidence lost during compression.
- The reported gains are consistent across three datasets, and the component ablations support the usefulness of the main modules.
Critical weaknesses
- Some important hyperparameters are not clearly specified. For example, the local window size () affects boundary detection, but its value and selection procedure are unclear. A sensitivity analysis would help assess robustness.
- The ablations report accuracy but not computational cost. Since efficiency is a central motivation, the paper should also report end-to-end latency and memory usage, including the costs of chunking, compression, and replay.
- The fairness of the main comparison is unclear. AdaMem is fine-tuned on HotpotQA, but it is unclear whether the baselines receive comparable task-specific training. The authors should clarify the training protocol for each baseline and provide matched comparisons where possible.
- The paper describes the three boundary signals as complementary, but does not test them individually or in combination. Removing the entire chunking module does not show whether all three signals are needed.
- Equation (5) produces a continuous budget , while memory-slot construction requires an integer. The paper should explain how is discretized and how this affects training and the total memory budget.
Questions for authors
Recent context compression methods, such as SAC [1] and EPL [2], have reported improvements over ICAE. How does AdaMem differ from these methods in terms of its approach to context compression?
References
[1] Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors. ICLR 2026.
[2] Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models. EMNLP 2024.
5 comments
Response to Reviewer n3ZM – Part II
W3: The fairness of the main comparison is unclear. AdaMem is fine-tuned on HotpotQA, but it is unclear whether the baselines receive comparable task-specific training. The authors should clarify the training protocol for each baseline and provide matched comparisons where possible.
R3: We apologize for the unclear description of the baseline training protocols. For the industry models, we use their released checkpoints, as their extensive pretraining and post-training pipelines establish capabilities that additional task-specific fine-tuning may not consistently improve. This preserves their original configurations as strong, readily available baselines.
For the three academic baselines, we adopt the following protocols. For ICAE, we follow its instruction fine-tuning formulation, using HotpotQA contexts, questions, and answers to train the compression encoder through the answer-generation loss. We update the encoder’s LoRA parameters and learnable memory-token embeddings while keeping the decoder parameters frozen. For LLMLingua-2, we fix the token-selection compressor and fine-tune the downstream reader on compressed HotpotQA contexts with the corresponding questions and answers. Both use the same HotpotQA training examples as AdaMem. For LCLM, we initially use the released checkpoint without additional fine-tuning, as its reported training mixture already includes HotpotQA alongside extensive other data. We have clarified the checkpoints, training data, and trainable components for each baseline in the appendix.
Although LCLM’s original training already includes HotpotQA, we additionally examine whether further task-specific fine-tuning helps address potential forgetting during its broader training process to ensure a fair comparison. We separately fine-tune the released checkpoint on the same HotpotQA training examples used for AdaMem. As shown in the table below, additional fine-tuning improves LCLM at most lengths but does not eliminate its substantial performance decline as contexts grow longer. AdaMem consistently achieves higher EM under matched task-specific training data and maintains a clear advantage at longer contexts, further supporting its effectiveness in compressing and using long-context information.
| Context length | LCLM | LCLM-Fine tuning | AdaMem |
|---|---|---|---|
| 8K | 78.13 | 77.34 | 78.91 |
| 16K | 75.00 | 76.56 | 80.47 |
| 32K | 70.31 | 73.44 | 75.78 |
| 64K | 65.63 | 67.97 | 76.56 |
| 128K | 60.94 | 62.50 | 73.44 |
| 256K | 55.47 | 59.38 | 73.44 |
| 512K | 48.44 | 53.13 | 71.88 |
| 1M | 30.47 | 37.50 | 69.53 |
W4: The paper describes the three boundary signals as complementary, but does not test them individually or in combination. Removing the entire chunking module does not show whether all three signals are needed.
R4: Thank you for this suggestion. We evaluated each boundary cue individually and all pairwise combinations on HotpotQA at 16K: P (token-level predictability shift), L (state-level latent transition), and S (semantic-level representation divergence).
| Boundary Cues | HotpotQA EM (%) |
|---|---|
| P + L + S (AdaMem) | 80.47 |
| P | 60.94 |
| L | 72.66 |
| S | 62.50 |
| P + L | 76.56 |
| P + S | 63.28 |
| L + S | 74.22 |
The full combination outperforms every individual cue and pairwise combination. Removing P, L, or S reduces EM by 6.25, 17.19, and 3.91 percentage points, respectively. Although latent transitions provide the strongest individual signal, adding predictability shifts and semantic divergence further improves performance. These cues capture changes in token predictability, adjacent latent states, and window-level semantics, providing distinct evidence for boundary decisions. The results therefore support their complementary contributions to downstream, beyond the benefit of introducing a chunking module alone.
Response to Reviewer n3ZM – Part III
W5: Equation (5) produces a continuous budget , while memory-slot construction requires an integer. The paper should explain how is discretized and how this affects training and the total memory budget.
R5: Thank you for pointing out this missing detail. We round the continuous budget in Eq. (5) to the nearest integer and clip it to [8, 32]. This integer count determines the number of consecutive embedding segments and the corresponding memory slots constructed in Eq. (6). During training, the forward pass uses the integer allocation, while a straight-through estimator supplies an identity surrogate gradient for the rounding operation during backpropagation. This provides an approximation to the gradient through rounding. At inference, the same discretization determines the number of memory slots retained for each chunk.
Regarding the total memory budget, each chunk is compressed into 8–32 memory slots, substantially fewer than its original tokens. Rounding changes each continuous allocation by at most half a slot, so its effect on the accumulated memory length is limited and does not materially alter the overall degree of compression. For chunks, the total difference from the continuous allocation is bounded by slots, while all per-chunk allocations remain within the prescribed limits.
Q1: Recent context compression methods, such as SAC [1] and EPL [2], have reported improvements over ICAE. How does AdaMem differ from these methods in terms of its approach to context compression?
R6: Thank you for highlighting these relevant works. We have explicitly cited SAC and EPL and expanded the related-work discussion to clarify how AdaMem builds on their insights and addresses additional challenges in long-context compression.
Adaptive allocation. SAC strengthens compressed representations through input-derived semantic anchors and bidirectional attention, while EPL improves positional coverage and consistency. These are valuable improvements to compression quality. However, long contexts often mix repetitive passages with concentrated new evidence, so equal-length regions need not warrant equal memory capacity. SAC’s reported setup determines anchor counts from length and a prescribed compression ratio, while EPL optimizes positions for a supplied token budget. Neither mechanism explicitly adjusts capacity according to historical novelty. AdaMem addresses this mismatch by assigning variable budgets to coherent semantic chunks based on associative-memory prediction residuals.
Memory construction. SAC demonstrates the benefit of grounding compression carriers in the input. AdaMem extends this principle to variable-budget memory construction: the allocated budget determines how each chunk’s embeddings are partitioned and pooled to initialize appended slots. This gives additional slots distinct local input anchors while preserving the chunk as a coherent unit. Unlike SAC’s evaluated KV-based compressor with bidirectional attention, AdaMem retains slot output states under standard causal attention. EPL’s position-layout optimization is complementary to these choices.
Evidence recovery. Better compressed representations reduce information loss, but cannot guarantee preservation of every detail required by a later question. The described SAC and EPL QA pipelines answer from compressed representations without uncertainty-triggered access to the source. AdaMem preserves the correspondence between chunk memories and original text, enabling selective replay when the memory-based answer is uncertain. This provides a recovery path for details omitted during compression.
Empirical comparison. Table 21 of SAC reports HotpotQA scores of 26.03 for SAC and 23.72 for EPL in its 24K evaluation with 15× compression. Separately, we fine-tuned both methods on HotpotQA data and evaluated them under our protocol. At 16K with 5× compression, SAC and EPL achieve 35.16% and 30.47% EM, respectively, compared with 80.47% for AdaMem. This within-protocol comparison supports the effectiveness of AdaMem’s complete memory framework for long-context QA.
Response to Reviewer n3ZM – Part I
Thank you for recognizing our framework’s motivation, methodological design, and empirical results. We respond to your comments point by point below.
R1: Thank you for pointing out this missing detail. We use a local window of w=32 tokens on each side of a candidate boundary to compute token-level predictability shifts and semantic-level divergence. This setting is fixed across all datasets and context lengths and is now explicitly stated in the implementation details.
To examine the suitability of this default and its sensitivity, we evaluated w∈{16,32,64,128,256} on all three datasets at a context length of 64K, keeping all other settings unchanged. The results are reported below as EM (%).
The default w=32 achieves the highest EM on all three datasets. This pattern is consistent with a balance between sufficient context for estimating semantic transitions and sufficient locality to preserve boundary cues. Smaller windows may provide less reliable contextual estimates, while larger windows may blur nearby transitions. The consistent preference for w=32 across datasets supports using a shared default without dataset-specific adjustment. Performance remains within a relatively bounded range across the tested window sizes, suggesting some robustness to this choice, although w=32 provides the best overall results.
R2: Thank you for highlighting the need to report computational costs alongside accuracy. We added a breakdown of per-query inference latency and peak allocated GPU memory on HotpotQA at 16K. The measurements cover the complete pipeline, including boundary detection, budget allocation, memory compression, coarse reasoning, and selective replay.
AdaMem’s measured end-to-end inference latency is 2.567 seconds per query, with a peak allocated GPU memory of 15.02 GB, including boundary detection, budget allocation, memory compression, coarse reasoning, and selective replay. Compared with Qwen2.5-7B-Instruct-1M, which is among the fastest baselines in Figure 1, AdaMem achieves similar latency (2.567s vs. 2.582s) while using less peak GPU memory (15.02 GB vs. 16.21 GB) at 16K. This shows that AdaMem maintains competitive inference speed while reducing peak memory usage by approximately 7.3%, even with the full memory-processing pipeline included. Memory construction accounts for most of the measured latency. Additional FLOPs breakdowns at 512K and 1M, together with compression ratios and replay trigger rates, are provided in our response R2 to Reviewer UKe2.