acceptodds
Under review as a conference paper at ICLR 2027

AdaMem: Adaptive Memory Organization for Scalable Long-Context Reasoning

Abstract

Large language models (LLMs) face fundamental scalability challenges when reasoning over extremely long contexts, where processing the full sequence incurs prohibitive computational and memory costs. Existing context compression methods alleviate this burden by transforming long contexts into compact memory representations, but they typically rely on fixed segmentation and uniform compression strategies, leading to semantic fragmentation and inefficient information preservation. To address these limitations, we propose AdaMem, an adaptive memory organization framework that enables scalable long-context reasoning by dynamically organizing, allocating, and preserving contextual information. AdaMem first identifies semantically coherent units through boundary-aware chunking, avoiding information fragmentation caused by rigid partitioning. It then estimates the information novelty of each chunk and adaptively allocates memory budgets, allowing critical evidence to retain richer representations while compressing redundant content aggressively. Finally, AdaMem accumulates chunk-level memories and introduces selective replay to recover original contexts only when compressed memories are insufficient. Extensive experiments on long-context question answering benchmarks up to 1M tokens demonstrate that AdaMem consistently outperforms state-of-the-art long-context models, achieving superior reasoning accuracy with substantially improved efficiency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-Context Reasoning, Context Compression, Adaptive Memory Organization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_n3ZMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AdaMem combines semantic chunking, surprise-based memory allocation, and selective replay for long-document QA. It compresses context into soft memory tokens and retrieves original chunks when the compressed memory is insufficient. Experiments on extended versions of HotpotQA, 2WikiMQA, and TriviaQA show promising improvements over the evaluated baselines.

Critical strengths
  1. The framework is well motivated and brings together semantic chunking, adaptive compression, and selective replay in a coherent way.
  2. Selective replay offers a practical way to recover evidence lost during compression.
  3. The reported gains are consistent across three datasets, and the component ablations support the usefulness of the main modules.
Critical weaknesses
  1. Some important hyperparameters are not clearly specified. For example, the local window size (ww) affects boundary detection, but its value and selection procedure are unclear. A sensitivity analysis would help assess robustness.
  2. The ablations report accuracy but not computational cost. Since efficiency is a central motivation, the paper should also report end-to-end latency and memory usage, including the costs of chunking, compression, and replay.
  3. The fairness of the main comparison is unclear. AdaMem is fine-tuned on HotpotQA, but it is unclear whether the baselines receive comparable task-specific training. The authors should clarify the training protocol for each baseline and provide matched comparisons where possible.
  4. The paper describes the three boundary signals as complementary, but does not test them individually or in combination. Removing the entire chunking module does not show whether all three signals are needed.
  5. Equation (5) produces a continuous budget BiB_i, while memory-slot construction requires an integer. The paper should explain how BiB_i is discretized and how this affects training and the total memory budget.
Questions for authors

Recent context compression methods, such as SAC [1] and EPL [2], have reported improvements over ICAE. How does AdaMem differ from these methods in terms of its approach to context compression?

References

[1] Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors. ICLR 2026.

[2] Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models. EMNLP 2024.

5 comments
AuthorsOpenReview ↗

Response to Reviewer n3ZM – Part I

Thank you for recognizing our framework’s motivation, methodological design, and empirical results. We respond to your comments point by point below.

W1: Some important hyperparameters are not clearly specified. For example, the local window size (ww) affects boundary detection, but its value and selection procedure are unclear. A sensitivity analysis would help assess robustness.

R1: Thank you for pointing out this missing detail. We use a local window of w=32w=32 tokens on each side of a candidate boundary to compute token-level predictability shifts and semantic-level divergence. This setting is fixed across all datasets and context lengths and is now explicitly stated in the implementation details.

To examine the suitability of this default and its sensitivity, we evaluated w∈{16,32,64,128,256}w\in\{16,32,64,128,256\} on all three datasets at a context length of 64K, keeping all other settings unchanged. The results are reported below as EM (%).

Local window size wwHotpotQA2WikiMQATriviaQAAverage
1672.6667.9770.3170.31
32 (default)76.5668.7573.4472.92
6473.4464.8469.5369.27
12871.8864.0668.7568.23
25671.8862.5069.5367.97

The default w=32w=32 achieves the highest EM on all three datasets. This pattern is consistent with a balance between sufficient context for estimating semantic transitions and sufficient locality to preserve boundary cues. Smaller windows may provide less reliable contextual estimates, while larger windows may blur nearby transitions. The consistent preference for w=32w=32 across datasets supports using a shared default without dataset-specific adjustment. Performance remains within a relatively bounded range across the tested window sizes, suggesting some robustness to this choice, although w=32w=32 provides the best overall results.

W2: The ablations report accuracy but not computational cost. Since efficiency is a central motivation, the paper should also report end-to-end latency and memory usage, including the costs of chunking, compression, and replay.

R2: Thank you for highlighting the need to report computational costs alongside accuracy. We added a breakdown of per-query inference latency and peak allocated GPU memory on HotpotQA at 16K. The measurements cover the complete pipeline, including boundary detection, budget allocation, memory compression, coarse reasoning, and selective replay.

Model(stage)Mean Latency per Query (s)Peak Allocated GPU Memory (GB)
AdaMem(Boundary Detection)1.08014.37
AdaMem(Surprise Scoring & Budgeting)0.58314.86
AdaMem(Memory Compression)0.75414.53
AdaMem(Coarse Reasoning)0.07014.96
AdaMem(Selective Replay)0.08015.02
AdaMem2.56715.02
Qwen2.5-7B-Inctruct-1M2.58216.21

AdaMem’s measured end-to-end inference latency is 2.567 seconds per query, with a peak allocated GPU memory of 15.02 GB, including boundary detection, budget allocation, memory compression, coarse reasoning, and selective replay. Compared with Qwen2.5-7B-Instruct-1M, which is among the fastest baselines in Figure 1, AdaMem achieves similar latency (2.567s vs. 2.582s) while using less peak GPU memory (15.02 GB vs. 16.21 GB) at 16K. This shows that AdaMem maintains competitive inference speed while reducing peak memory usage by approximately 7.3%, even with the full memory-processing pipeline included. Memory construction accounts for most of the measured latency. Additional FLOPs breakdowns at 512K and 1M, together with compression ratios and replay trigger rates, are provided in our response R2 to Reviewer UKe2.

AuthorsOpenReview ↗

Response to Reviewer n3ZM – Part II

W3: The fairness of the main comparison is unclear. AdaMem is fine-tuned on HotpotQA, but it is unclear whether the baselines receive comparable task-specific training. The authors should clarify the training protocol for each baseline and provide matched comparisons where possible.

R3: We apologize for the unclear description of the baseline training protocols. For the industry models, we use their released checkpoints, as their extensive pretraining and post-training pipelines establish capabilities that additional task-specific fine-tuning may not consistently improve. This preserves their original configurations as strong, readily available baselines.

For the three academic baselines, we adopt the following protocols. For ICAE, we follow its instruction fine-tuning formulation, using HotpotQA contexts, questions, and answers to train the compression encoder through the answer-generation loss. We update the encoder’s LoRA parameters and learnable memory-token embeddings while keeping the decoder parameters frozen. For LLMLingua-2, we fix the token-selection compressor and fine-tune the downstream reader on compressed HotpotQA contexts with the corresponding questions and answers. Both use the same HotpotQA training examples as AdaMem. For LCLM, we initially use the released checkpoint without additional fine-tuning, as its reported training mixture already includes HotpotQA alongside extensive other data. We have clarified the checkpoints, training data, and trainable components for each baseline in the appendix.

Although LCLM’s original training already includes HotpotQA, we additionally examine whether further task-specific fine-tuning helps address potential forgetting during its broader training process to ensure a fair comparison. We separately fine-tune the released checkpoint on the same HotpotQA training examples used for AdaMem. As shown in the table below, additional fine-tuning improves LCLM at most lengths but does not eliminate its substantial performance decline as contexts grow longer. AdaMem consistently achieves higher EM under matched task-specific training data and maintains a clear advantage at longer contexts, further supporting its effectiveness in compressing and using long-context information.

Context lengthLCLMLCLM-Fine tuningAdaMem
8K78.1377.3478.91
16K75.0076.5680.47
32K70.3173.4475.78
64K65.6367.9776.56
128K60.9462.5073.44
256K55.4759.3873.44
512K48.4453.1371.88
1M30.4737.5069.53

W4: The paper describes the three boundary signals as complementary, but does not test them individually or in combination. Removing the entire chunking module does not show whether all three signals are needed.

R4: Thank you for this suggestion. We evaluated each boundary cue individually and all pairwise combinations on HotpotQA at 16K: P (token-level predictability shift), L (state-level latent transition), and S (semantic-level representation divergence).

Boundary CuesHotpotQA EM (%)
P + L + S (AdaMem)80.47
P60.94
L72.66
S62.50
P + L76.56
P + S63.28
L + S74.22

The full combination outperforms every individual cue and pairwise combination. Removing P, L, or S reduces EM by 6.25, 17.19, and 3.91 percentage points, respectively. Although latent transitions provide the strongest individual signal, adding predictability shifts and semantic divergence further improves performance. These cues capture changes in token predictability, adjacent latent states, and window-level semantics, providing distinct evidence for boundary decisions. The results therefore support their complementary contributions to downstream, beyond the benefit of introducing a chunking module alone.

AuthorsOpenReview ↗

Response to Reviewer n3ZM – Part III

W5: Equation (5) produces a continuous budget BiB_i, while memory-slot construction requires an integer. The paper should explain how BiB_i is discretized and how this affects training and the total memory budget.

R5: Thank you for pointing out this missing detail. We round the continuous budget BiB_i in Eq. (5) to the nearest integer and clip it to [8, 32]. This integer count determines the number of consecutive embedding segments and the corresponding memory slots constructed in Eq. (6). During training, the forward pass uses the integer allocation, while a straight-through estimator supplies an identity surrogate gradient for the rounding operation during backpropagation. This provides an approximation to the gradient through rounding. At inference, the same discretization determines the number of memory slots retained for each chunk.

Regarding the total memory budget, each chunk is compressed into 8–32 memory slots, substantially fewer than its original tokens. Rounding changes each continuous allocation by at most half a slot, so its effect on the accumulated memory length is limited and does not materially alter the overall degree of compression. For KK chunks, the total difference from the continuous allocation is bounded by K/2K/2 slots, while all per-chunk allocations remain within the prescribed limits.

Q1: Recent context compression methods, such as SAC [1] and EPL [2], have reported improvements over ICAE. How does AdaMem differ from these methods in terms of its approach to context compression?

R6: Thank you for highlighting these relevant works. We have explicitly cited SAC and EPL and expanded the related-work discussion to clarify how AdaMem builds on their insights and addresses additional challenges in long-context compression.

Adaptive allocation. SAC strengthens compressed representations through input-derived semantic anchors and bidirectional attention, while EPL improves positional coverage and consistency. These are valuable improvements to compression quality. However, long contexts often mix repetitive passages with concentrated new evidence, so equal-length regions need not warrant equal memory capacity. SAC’s reported setup determines anchor counts from length and a prescribed compression ratio, while EPL optimizes positions for a supplied token budget. Neither mechanism explicitly adjusts capacity according to historical novelty. AdaMem addresses this mismatch by assigning variable budgets to coherent semantic chunks based on associative-memory prediction residuals.

Memory construction. SAC demonstrates the benefit of grounding compression carriers in the input. AdaMem extends this principle to variable-budget memory construction: the allocated budget determines how each chunk’s embeddings are partitioned and pooled to initialize appended slots. This gives additional slots distinct local input anchors while preserving the chunk as a coherent unit. Unlike SAC’s evaluated KV-based compressor with bidirectional attention, AdaMem retains slot output states under standard causal attention. EPL’s position-layout optimization is complementary to these choices.

Evidence recovery. Better compressed representations reduce information loss, but cannot guarantee preservation of every detail required by a later question. The described SAC and EPL QA pipelines answer from compressed representations without uncertainty-triggered access to the source. AdaMem preserves the correspondence between chunk memories and original text, enabling selective replay when the memory-based answer is uncertain. This provides a recovery path for details omitted during compression.

Empirical comparison. Table 21 of SAC reports HotpotQA scores of 26.03 for SAC and 23.72 for EPL in its 24K evaluation with 15× compression. Separately, we fine-tuned both methods on HotpotQA data and evaluated them under our protocol. At 16K with 5× compression, SAC and EPL achieve 35.16% and 30.47% EM, respectively, compared with 80.47% for AdaMem. This within-protocol comparison supports the effectiveness of AdaMem’s complete memory framework for long-context QA.

Reviewer_MZ9FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AdaMem, a memory-organization pipeline for long-context question answering built on Qwen2.5-7B-Instruct. The input context is first segmented by a learned boundary detector that fuses three cues — token-level predictability shift, hidden-state-level discontinuity, and window-level semantic divergence — and is trained against labels produced by a teacher LLM. Each resulting chunk is then assigned a memory budget from a "surprise" score measured against an auxiliary associative memory, and is compressed into a latent slot by pooling its contextualized input embeddings under that budget. The per-chunk memories are concatenated into an accumulated sequence, from which the model answers on its own; an entropy-based gate then decides whether to retrieve the top-R raw chunks and re-run the model with them in context. The authors fine-tune only on HotpotQA and evaluate on RULER-style variants of HotpotQA, 2WikiMQA and TriviaQA, reporting large gains over the compared baselines.

Critical strengths
  1. The paper separates long-context memory QA into three decisions that can be measured in isolation — where to cut, how much to keep, and when to fall back to the raw text — and ablates each of them on all three datasets. Most work in this area reports only end-to-end numbers, so the ablation table is what makes the design legible enough to reuse or extend.
  2. Three datasets, several context lengths, and component ablations run on all three, with reported gaps at matched input length that are large.
Critical weaknesses
  1. The experiments do not identify compression as the source of the gains. a. At 32K, AdaMem-7B reaches 75.78% at 3.49s while Qwen2.5-7B-Instruct-1M reaches 45.31% at 3.48s. With latency identical to two decimal places, a 30-point gap cannot be attributed to compression: if compression bought efficiency, latency should drop. A natural explanation is the 10K-example HotpotQA LoRA fine-tuning described in Appendix C, which no baseline receives. b. Qwen2.5-7B-Instruct LoRA-tuned on the same data under the same setting; (b) the same chunker and retriever returning the top-k raw chunks with no compression at all. Control (b) is especially important: replay (Eq. 9) already puts raw text back into the context, so roughly half of the framework is a retriever; and the ablation shows that AdaMem without replay does not beat all baselines, i.e. the memory path alone is not sufficient. c. The ablation removes chunking, budgeting and replay but never compression, and the budget-aware initialization of Eq. (6) — arguably the most transferable idea in the paper — has no ablation at all. The claim that compression quality matters is therefore untested. d. All three datasets share the same RULER-style construction: gold evidence is preserved verbatim, distractors come from the same source, and Appendix B states that the transition between adjacent passages serves as the positive boundary label. The Figure 5 case consists of three mutually unrelated Wikipedia articles. The boundary detector may therefore be learning document boundaries, which is much easier than segmenting a single coherent 100K-token document. With 128 test examples per dataset, several of the reported gaps of a few points are also within sampling noise.
  2. Each component has a close precedent that the paper does not discuss: divergence-based boundary detection between adjacent representations is precisely the boundary mechanism of DLCM and ConceptMoE; surprise-driven memory writes are the core mechanism of Titans (Behrouz et al., 2025) — the paper cites a later Behrouz et al. paper but never positions against Titans itself; information-density-based adaptive compression budgets, and semantic grouping with hierarchical memory, are likewise pre-existing components. The claimed contribution is "an adaptive memory-organization framework," and that claim lives entirely in the composition and the interaction between components. If the composition is the contribution, the paper needs to state which interaction is new and show that it matters.
Questions for authors

Q1 Qwen2.5-7B-Instruct LoRA-tuned on the same data under the same setting; Could you report the two controls in critical weakness: (a) Qwen2.5-7B-Instruct LoRA-tuned on the same 10K HotpotQA examples with identical settings, and (b) the same chunker and retriever returning the top-k raw chunks with no compression ? These two numbers determine whether the gains come from compression or from fine-tuning plus retrieval. Q2 What fraction of queries trigger replay (p_rep < τ) at each context length and on each dataset? This determines how much of the end-to-end result is attributable to the memory path rather than the retriever. Q3 Which R does Table 1 use? Appendix C specifies top-3 and τ = 0.5, yet Section 4.5 reports that on HotpotQA at 64K, R = 6 is optimal with EM 76.56 — identical to the 64K cell of Table 1,while the TriviaQA 64K cell in Table 1 (73.44) is well below the R = 4 peak in Section 4.5 (78.44). Were the parameter studies run on a different evaluation set, or does Table 1 report the per-dataset best R? Q4 Section 4.2 gives a TriviaQA mean of 75.68% while Table 1 gives 75.20%. Which is correct? The y-axis label in Figure 1 reads "accuracy time", which I assume is a typo. Q5 Do you have results on naturally coherent long documents rather than concatenations of unrelated passages? That is the setting where semantic chunking should matter most, and it is currently untested.

6 comments
AuthorsOpenReview ↗

Response to Reviewer MZ9F – Part I

Thank you for your insightful comments and for taking the time to review our paper. We respond to your suggestions point by point as follows.

W1-(b) & Q1-(a): Qwen2.5-7B-Instruct LoRA-tuned on the same 10K HotpotQA examples with identical settings.

R1-1: Both AdaMem and this control use Qwen2.5-7B-Instruct as the backbone. We perform LoRA fine-tuning using the same 10K HotpotQA training examples and identical training settings, and evaluate on the same test examples. The LoRA-only control does not incorporate AdaMem’s memory organization or replay mechanisms.

EM (%)8K16K32K64K128K256K512K1MAverage
LoRA-only35.1637.5035.9439.0642.9744.5338.2831.2538.09
AdaMem78.9180.4775.7876.5673.4473.4471.8869.5375.00

AdaMem achieves higher EM at every context length, with an average advantage of 36.91 percentage points. With the backbone, training examples, and training settings held constant, these results indicate that AdaMem’s performance advantage cannot be explained solely by task-specific fine-tuning. Its mechanisms for organizing and using long-context information provide additional benefits.

W1-(b) & Q1-(b): The same chunker and retriever returning the top-k raw chunks with no compression? These two numbers determine whether the gains come from compression or from fine-tuning plus retrieval.

R1-2: This control retains AdaMem’s chunker, retrieval ranking procedure, and answer model, but provides no compressed memory to the answer model. Instead, it directly supplies the retrieved top-RR raw chunks. We evaluate R=2R=2 and R=4R=4 to examine whether the selected raw evidence alone can match the complete model while retaining its chunking and retrieval capabilities. Results are reported as EM (%).

Context Lengthw/o compression, R=2R=2w/o compression, R=4R=4AdaMem
8K59.3856.2578.91
16K54.6964.0680.47
32K57.0370.3175.78
64K59.3866.4176.56
128K62.5065.6373.44
256K61.7266.4173.44
512K58.5962.5071.88
1M54.6960.1669.53
Average58.5063.9775.00

With chunking and retrieval retained, this control remains competitive even without compressed memory in the answer input: R=4R=4 achieves an average EM of 63.97%, indicating that locating and supplying relevant source text is an important capability of the system. Nevertheless, the complete AdaMem model outperforms both configurations at every context length, with average advantages of 16.50 and 11.03 percentage points, respectively. Increasing the number of retrieved chunks improves performance at most context lengths, but does not close the gap with the complete model.

These results support the complementarity of compressed memory and retrieved evidence. When only the top-RR raw chunks are supplied, the contextual information available for answering is limited to those selected passages. AdaMem retains compressed representations from all chunks and supplements them with original details when needed. Its advantage therefore lies in combining broader context coverage with fine-grained evidence, rather than relying solely on a small set of retrieved raw chunks.

Q2: What fraction of queries trigger replay (p_rep < τ) at each context length and on each dataset? This determines how much of the end-to-end result is attributable to the memory path rather than the retriever

R1-3: The table below reports the proportion of HotpotQA examples that trigger raw-context replay. Across context lengths, only 23.44%–37.50% of queries trigger replay, with an average trigger rate of 32.13%. Most queries directly use the answer produced through the compressed-memory path. This indicates that compressed memory serves as AdaMem’s default basis for answering, while raw-context replay provides on-demand supplementation. Together with the control experiments above, these results show that the complete system effectively integrates memory organization and evidence retrieval, achieving higher accuracy without supplying raw context for every query.

Context Length8K16K32K64K128K256K512K1M
Replay Trigger Rate23.44%27.34%31.25%33.59%35.16%32.81%37.50%35.94%
AuthorsOpenReview ↗

Response to Reviewer MZ9F – Part II

W1-(a): The experiments do not identify compression as the source of the gains. At 32K, AdaMem-7B reaches 75.78% at 3.49s while Qwen2.5-7B-Instruct-1M reaches 45.31% at 3.48s. With latency identical to two decimal places, a 30-point gap cannot be attributed to compression: if compression bought efficiency, latency should drop. A natural explanation is the 10K-example HotpotQA LoRA fine-tuning described in Appendix C, which no baseline receives.

R1-4: Taken together, the controls above show that neither fine-tuning on the same data nor using retrieved raw text alone matches the performance of the complete AdaMem system, supporting the complementary value of compressed-memory organization and on-demand replay. Regarding the latency comparison in W1-(a), Qwen2.5-7B-Instruct-1M has undergone dedicated long-context pretraining and post-training, making it a strong long-context baseline. At 32K, AdaMem achieves higher EM (75.78% vs. 45.31%) at a similar reported latency (3.49s vs. 3.48s), demonstrating a better accuracy–latency trade-off. In addition, ICAE and LLMLingua-2 also received adaptation on the same 10K HotpotQA examples in the original experiments. LCLM’s original training mixture already includes HotpotQA, and AdaMem retains its advantage even after LCLM receives additional fine-tuning on our training set, as detailed in our response to Reviewer UKe2, W1 & Q1-(b). These comparisons support the contribution of AdaMem’s memory organization and selective replay beyond task-specific fine-tuning alone.

W1-(c): The ablation removes chunking, budgeting and replay but never compression, and the budget-aware initialization of Eq. (6)—arguably the most transferable idea in the paper—has no ablation at all. The claim that compression quality matters is therefore untested.

R1-5: We added an ablation on HotpotQA at 16K, replacing Eq. (6)’s segment-wise pooling with input-independent learnable slots. Only the slot initialization changes, with the same number of learnable slot parameters and all other settings unchanged. This replacement reduces EM from 80.47% to 71.09%, a 9.38-percentage-point drop, supporting the effectiveness of our budget-aware initialization in constructing compressed memories.

W1-(d): All three datasets share the same RULER-style construction: gold evidence is preserved verbatim, distractors come from the same source, and Appendix B states that the transition between adjacent passages serves as the positive boundary label. The Figure 5 case consists of three mutually unrelated Wikipedia articles. The boundary detector may therefore be learning document boundaries, which is much easier than segmenting a single coherent 100K-token document. With 128 test examples per dataset, several of the reported gaps of a few points are also within sampling noise. Q5: Do you have results on naturally coherent long documents rather than concatenations of unrelated passages? That is the setting where semantic chunking should matter most, and it is currently untested.

W1-(d) & Q5: Do you have results on naturally coherent long documents rather than concatenations of unrelated passages? That is the setting where semantic chunking should matter most, and it is currently untested.

R1-6: To address this concern, we added book-based QA experiments using PG-19. PG-19 contains Project Gutenberg books published before 1919, providing long, original book texts [1]. We selected contiguous book excerpts because they preserve the original narrative order and contextual connections, with characters, events, and themes developing throughout the text. This setting is well suited to evaluating the ability to organize and use long-range information without obvious transitions between separate documents.

Specifically, we extracted contiguous passages of varying lengths from PG-19 books, without concatenating unrelated passages or inserting additional distractor documents. We used the DeepSeek API to generate questions and answers based on these excerpts, followed by manual verification of their correctness. We evaluated 128 questions at each context length using EM. We also evaluated Qwen2.5-7B-Instruct-1M on the same examples as a baseline that answers questions directly from the long context.

Context Length8K16K64K256K
AdaMem EM (%)80.4778.1372.6670.31
Qwen2.5-7B-Instruct-1M EM (%)58.5957.0353.1337.50

AdaMem performs better at every evaluated context length. These results indicate that its effectiveness extends beyond concatenated contexts with explicit document boundaries to naturally coherent book-length passages, where it also demonstrates strong information organization and question-answering capabilities.

[1] Jack W. Rae et al. “Compressive Transformers for Long-Range Sequence Modelling.” arXiv:1911.05507, 2019.

AuthorsOpenReview ↗

Response to Reviewer MZ9F – Part III

W2: Each component has a close precedent that the paper does not discuss: divergence-based boundary detection between adjacent representations is precisely the boundary mechanism of DLCM and ConceptMoE; surprise-driven memory writes are the core mechanism of Titans (Behrouz et al., 2025)—the paper cites a later Behrouz et al. paper but never positions against Titans itself; information-density-based adaptive compression budgets, and semantic grouping with hierarchical memory, are likewise pre-existing components. The claimed contribution is “an adaptive memory-organization framework,” and that claim lives entirely in the composition and the interaction between components. If the composition is the contribution, the paper needs to state which interaction is new and show that it matters.

R2: AdaMem’s contribution lies in coordinating semantic organization, compression capacity, and evidence recovery within a long-context memory framework. We have added explicit citations and discussions of DLCM, ConceptMoE, and Titans, and clarified the limitations our design addresses.

Boundary-based allocation does not explicitly measure historical redundancy. DLCM and ConceptMoE adapt compression through variable-length segmentation, assigning finer representations to difficult content. However, deciding where content changes does not establish how much new information it adds to existing memory. Repeated material can contain clear semantic transitions, while a coherent passage can introduce many previously unseen facts. Allocating capacity through segmentation alone therefore leaves this distinction implicit. AdaMem explicitly estimates each chunk’s novelty against historical associations and uses it to determine the memory budget. This provides a direct basis for compressing predictable content more aggressively while reserving additional capacity for underrepresented information, without requiring additional semantic boundaries.

Surprise-driven writing does not resolve future evidence recovery. Surprise is an established modeling principle[2,3]. The key distinction is how it is computed and which decision it controls. Titans uses associative-memory gradients, momentum, and forgetting to update neural memory at test time. Its shared long-term state must accommodate incoming information through updates that can modify earlier associations. This creates a retention trade-off: details weakened during updating may later become necessary for a query, and querying the updated state cannot guarantee their recovery. AdaMem separates the evolving novelty estimator from independently retained chunk memories and preserves access to their original text. Consequently, updating the estimator does not rewrite stored chunk representations, and uncertainty-triggered replay can recover source details absent from compressed memory.

The new interaction and its value. AdaMem connects historical novelty to both compression resolution and evidence access. The allocated budget determines the within-chunk initialization granularity in Eq. (6), and the resulting memory supports both reasoning and source-chunk retrieval. Thus, capacity allocation shapes an input-grounded representation that remains linked to recoverable evidence. This addresses two complementary needs: preserving more information before answering and recovering missing details when the question exposes a compression loss. The component ablations in Table 2 further support the effectiveness of the complete design.

[2] Itti, Laurent, and Pierre Baldi. “Bayesian Surprise Attracts Human Attention.” Advances in Neural Information Processing Systems 18 (2005).

[3] Houthooft, Rein, et al. “VIME: Variational Information Maximizing Exploration.” Advances in Neural Information Processing Systems 29 (2016).

Reviewer_UKe2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AdaMem, a context-compression framework for long-context QA built on Qwen2.5-7B-Instruct with LoRA. It has three components: (1) boundary-aware semantic chunking, where a lightweight prediction head combines token-level predictability shifts, latent-state transitions, and semantic divergence to detect semantic boundaries instead of fixed-length segmentation; (2) surprise-aware adaptive memory budgeting, where an associative memory estimates each chunk's novelty relative to previously seen content and allocates a per-chunk budget of memory tokens; (3) accumulated memory reasoning with selective replay, where chunk-level compressed memories are concatenated (rather than recurrently overwritten) and raw chunks are retrieved only when an uncertainty gate flags memory insufficiency. The system is fine-tuned on 10K RULER-style HotpotQA samples and evaluated on RULER-style HotpotQA, 2WikiMQA, and TriviaQA at 8K–1M tokens against compression baselines (ICAE, LCLM, LLMLingua-2) and long-context models (Qwen2.5-1M, QwenLong-L1, GLM-4-Z1, Kimi-Linear-48B), reporting large accuracy and latency advantages.

Critical strengths
  1. Internally consistent ablations with mechanism-level evidence. I recomputed all nine dataset×length deltas in Table 2 and they check out, including the reported 14.41-point average degradation from removing selective replay. Figure 3 goes beyond end-to-end scores to show why adaptive budgeting works: gold chunks receive a 2.6× larger mean budget (26 vs. 10 summary vectors) with overlapping, graded distributions — evidence that the allocation emerges from the surprise signal rather than a hard rule. This is the kind of diagnostic that supports the design claims.

  2. Each design choice is individually well-motivated against a named failure mode. Semantic chunking targets fragmentation from fixed windows; history-conditional budgeting targets uniform allocation; append-only accumulation targets recurrent-state forgetting. Each failure mode maps to one module and one ablation, making the paper easy to evaluate component-wise even where the empirical comparison is confounded (see weaknesses).

Critical weaknesses

1.Unfair empirical comparison; generalization is asserted but not demonstrated. AdaMem is fine-tuned on 10K RULER-style HotpotQA samples and its headline dataset is in-domain HotpotQA, while all baselines (including Kimi-Linear-48B) are zero-shot on this synthetic protocol. A 7B fine-tuned model beating a 48B zero-shot model by ~29 EM points at 1M is more parsimoniously explained by task-format fine-tuning than by the proposed memory organization; no fine-tuned baseline is provided. The claimed "zero-shot" transfer to 2WikiMQA/TriviaQA uses the identical construction with same-corpus distractors — near-transfer, not task generalization — and no public or reasoning-heavy benchmark (e.g., BABILong, LongBench v2, RULER's full task suite) is evaluated. This makes the central claim ("consistently outperforms state-of-the-art long-context models") uninterpretable as evidence for the method itself.

2.Efficiency claims lack the data needed to evaluate them. The overall compression ratio (total memory tokens vs. context tokens) is never reported, so AdaMem and the compression baselines cannot be compared at matched budgets. Figure 1 shows AdaMem's per-query latency (3.49 s) is essentially identical to its own zero-shot backbone, Qwen2.5-7B-Instruct-1M (3.48 s), at 32K — the "3.97x" claim holds only against a 32B model, and the "2.28x higher accuracy" ratio silently uses a different baseline's average score. No end-to-end cost accounting is given for the full pipeline: boundary detection and surprise estimation each require a forward pass over the raw context, and selective replay adds a second pass over retrieved raw chunks, yet the replay trigger rate is never reported even though the w/o-SR ablation (average -14.41 points) implies replay fires frequently and the headline results depend substantially on access to uncompressed context. The amortized cost may therefore approach or exceed full attention.

3.Statistically underpowered evaluation containing at least one impossible reported value. Every accuracy point rests on 128 test samples (1/128 = 0.78% granularity) with no confidence intervals, repeated runs, or seeds, while many adjacent table entries differ by fewer than 2 points. Worse, one entry (GLM-4-Z1, 2WikiMQA, 256K: 34.62) is not on the 0.78125-multiple grid that any 128-sample EM score must lie on (nearest: 34.38, 35.16) — it is not achievable as reported and indicates a transcription or computation error in Table 1. Additional unreconciled inconsistencies: the text reports a 75.68% TriviaQA average while Table 1 averages to 75.20%; Appendix C states top-3 retrieval while Sec. 4.5 reports an R=6 optimum that outperforms the main-table configuration.

4.The two headline "adaptive" contributions have direct, uncited predecessors. DAST (Chen et al., arXiv:2502.11493, Feb 2025) dynamically allocates soft-token budgets to information-rich chunks via perplexity and attention signals — the same contribution as "surprise-aware adaptive memory budgeting," with its own ablation showing dynamic over uniform allocation. AdmTree (Li et al., NeurIPS 2025, arXiv:2512.04550) dynamically segments input into variable-length segments by information density — the same goal as "boundary-aware adaptive semantic chunking." Both share core authors with LCLM, which this paper does cite and benchmark against, making the omission hard to attribute to unawareness. Consequently, the Introduction's claims that existing methods rely on "fixed-size sliding windows" and "fixed compression ratio" were outdated at submission, and the "Perspective"/"Algorithm" contributions reduce to a combination whose component-level deltas (learned boundary head vs. density-based segmentation; associative-memory surprise vs. perplexity+attention) are neither discussed nor evaluated in isolation.

5.Baseline coverage is incomplete and anomalous, and the scenarios the architecture targets are never tested. ICAE has no results at 512K/1M with no explanation; LLMLingua-2 scores below ICAE everywhere and is non-monotonic (27.34 at 64K > 22.66 at 128K), suggesting a misconfigured baseline; and several cited, directly relevant methods (MemAgent, AutoCompressor, COMET, RMT) are absent from experiments. Meanwhile, the paper's core mechanisms are explicitly designed for growing contexts — history-conditional surprise (Eq. 3), append-only memory accumulation (Sec. 3.5), memory-conditioned streaming pre-training (Eq. 11) — and MemAgent (a multi-conversation memory agent) is cited as related work, yet no experiment evaluates multi-turn dialogue, incrementally-arriving contexts, or agentic/iterative multi-hop retrieval: every evaluation presents a static context once and asks a single question.

Questions for authors
  1. Were any baselines fine-tuned on your 10K training set? Please provide (a) AdaMem results on a task format not used in training, and (b) at least one compression baseline (e.g., LCLM) fine-tuned on the same data.
  2. What is the overall compression ratio (total memory tokens / context tokens) at each context length, and what budget did each compression baseline use? Please provide an end-to-end prefill + replay latency/FLOPs breakdown at 512K and 1M, and the replay trigger rate per context length.
  3. Please reconcile: the 75.68% text claim vs. 75.20% table average (TriviaQA); the 34.62 entry (GLM-4-Z1, 2WikiMQA, 256K) that is not achievable with 128 samples; the mixed-baseline ratios in Figure 1; and which retrieval top-R the Table 1 results use relative to the R=6 optimum.
  4. Your mechanisms (history-conditional surprise, append-only accumulation, memory-conditioned streaming pre-training) are designed for growing contexts, and MemAgent is cited as related work — yet all evaluations use a single query over a static context. Please evaluate on multi-turn dialogue (e.g., MSC-style), streaming/incrementally-arriving contexts, or an agentic iterative-retrieval task, or narrow the implied streaming/multi-turn applicability claims accordingly.
  5. Please cite and compare against DAST (arXiv:2502.11493) and AdmTree (arXiv:2512.04550), including a controlled comparison isolating the allocation signal (your associative-memory surprise vs. DAST's perplexity+attention, under identical chunking) and the segmentation mechanism (your learned boundary head vs. AdmTree's density-based segmentation, under identical budgeting).
4 comments
AuthorsOpenReview ↗

Response to Reviewer UKe2 – Part I

Thank you so much for your positive and encouraging feedback. Below, we address your comments and suggestions in details:

W1 & Q1-(b): Were any baselines fine-tuned on your 10K training set? Please provide at least one compression baseline (e.g., LCLM) fine-tuned on the same data.

R1-1: Both ICAE and LLMLingua-2 were adapted using the same 10K HotpotQA training examples as AdaMem in the original experiments, and LCLM’s reported original training mixture already includes HotpotQA. These comparisons therefore already account for access to task-specific training data, indicating that AdaMem’s advantage cannot be attributed solely to receiving such adaptation.

Specifically, ICAE updates the compression encoder’s LoRA parameters and learnable memory-token embeddings through the answer-generation loss, while keeping the decoder frozen. For LLMLingua-2, we keep the token-selection compressor fixed and fine-tune the downstream reader on compressed contexts and their corresponding questions and answers. LCLM was initially evaluated using its released checkpoint, whose reported training mixture already includes HotpotQA. Industry models were evaluated using their released checkpoints as representative off-the-shelf systems, preserving the configurations established through their extensive pretraining and post-training. These comparisons complement the academic baselines adapted on the same task data. We have explicitly documented the training data and trainable components for each baseline in the revised appendix.

Although LCLM has already been fine-tuned on data containing HotpotQA, we additionally fine-tuned its released checkpoint on exactly the same training examples as AdaMem to better control for differences in task-format adaptation and potential forgetting effects. At each context length, we evaluated it on the same HotpotQA test examples used for AdaMem. The results below report EM (%).

Context LengthLCLMLCLM, Fine-tunedAdaMem
8K78.1377.3478.91
16K75.0076.5680.47
32K70.3173.4475.78
64K65.6367.9776.56
128K60.9462.5073.44
256K55.4759.3873.44
512K48.4453.1371.88
1M30.4737.5069.53

Additional fine-tuning provides limited improvements to LCLM, whose performance still declines substantially at longer context lengths. AdaMem retains its performance advantage when both methods receive adaptation on the same training data, further supporting its effectiveness in compressing and using long-context information.

W1 & Q1-(a): Please provide AdaMem results on a task format not used in training.

R1-2: To evaluate transfer beyond the training task format, we added a zero-shot evaluation on QMSum, a public query-focused meeting summarization benchmark also used in SAC’s long-context evaluation [1]. This task differs from the synthetic long-context QA used during training. We directly use the HotpotQA-trained AdaMem checkpoint, without any QMSum or summarization fine-tuning.

For reference, we include the QMSum results reported in Table 20 of the SAC paper for Semantic-Anchor Compression (SAC) [1] and Enhanced Position Layout (EPL) [2]. Both reference methods were fine-tuned on QMSum, and their scores are taken from the paper.

MethodQMSum (ROUGE-1 F1)
AdaMem (zero-shot)16.79
SAC14.95
EPL14.82

AdaMem achieves 16.79 ROUGE-1 F1 without task-specific adaptation, providing further evidence of its ability to organize and use long-context information in an unseen task format. The published scores offer additional context, rather than a comparison under identical experimental settings.

[1] Autoencoding-Free Context Compression for LLMs via Contextual Semantic Anchors. ICLR 2026.

[2] Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models. EMNLP 2024.

AuthorsOpenReview ↗

Response to Reviewer UKe2 – Part II

W2 & Q2: What is the overall compression ratio (total memory tokens / context tokens) at each context length, and what budget did each compression baseline use? Please provide an end-to-end prefill + replay latency/FLOPs breakdown at 512K and 1M, and the replay trigger rate per context length.

R2: We have added the budget settings for several representative compression baselines, AdaMem’s overall compression ratios and replay trigger rates at each context length, and FLOPs comparisons at 512K and 1M.

Compression budgets and replay trigger rates. We consistently define the token retention ratio as the number of compressed tokens divided by the number of original context tokens. The compression baselines use fixed compression budgets, as follows.

Compression BaselineFixed Compression FactorToken Retention Ratio
ICAE4×25.00%
LCLM16×6.25%
LLMLingua-24×25.00%

AdaMem adaptively allocates memory budgets based on semantic chunking and the informational novelty of each chunk. The following table reports the overall compression ratio at each context length on HotpotQA, together with the proportion of evaluation examples that trigger raw-context replay.

Context Length8K16K32K64K128K256K512K1M
Token Retention Ratio15.76%14.57%14.63%14.24%14.21%10.68%5.91%5.46%
Replay Trigger Rate23.44%27.34%31.25%33.59%35.16%32.81%37.50%35.94%

At 512K and 1M, AdaMem compresses the context to 5.91% and 5.46% of its original length, corresponding to approximately 16.92× and 18.32× compression, respectively. Both retention ratios are lower than those of the fixed-budget baselines listed above. Across context lengths, only 23.44%–37.50% of examples trigger raw-context replay. Most examples directly use the answer produced through the compressed-memory path, combining compressed memory with on-demand supplementation of original evidence.

Computational costs at 512K and 1M.

We computed the total FLOPs of each method at these two long-context lengths and further decomposed AdaMem’s computational costs into five stages: boundary detection, surprise scoring and budget allocation, memory compression, coarse reasoning, and selective replay. This breakdown accounts for the computational costs of memory construction and answer generation. Results are shown below.

Method (Component)512K FLOPs ↓1M FLOPs ↓
AdaMem (Boundary Detection)1.10e172.99e17
AdaMem (Surprise Scoring & Budgeting)6.86e161.37e17
AdaMem (Memory Compression)7.21e161.40e17
AdaMem (Coarse Reasoning)5.90e155.90e15
AdaMem (Selective Replay)6.00e156.00e15
AdaMem (Total)2.63e175.88e17
DAST-8×3.41e171.02e18
AdmTree-8×2.25e191.25e20

Under the reported accounting scope, AdaMem has the lowest computational cost among the compared methods at both context lengths. Compared with DAST-8×, its FLOPs are approximately 23.0% and 42.4% lower at 512K and 1M, respectively, demonstrating its ability to effectively control computational costs as context length increases.

W3 & Q3: Please reconcile: the 75.68% text claim vs. 75.20% table average (TriviaQA); the 34.62 entry (GLM-4-Z1, 2WikiMQA, 256K) that is not achievable with 128 samples; the mixed-baseline ratios in Figure 1; and which retrieval top-R the Table 1 results use relative to the R=6 optimum.

R3: We apologize for the reporting errors and ambiguities in the presentation. The TriviaQA average is 75.20%, rather than 75.68%, and the GLM-4-Z1 result on 2WikiMQA at 256K is 34.38%. In Figure 1, the accuracy and efficiency ratios refer to different baselines, as indicated by their respective annotations. The R=6R=6 optimum discussed in Section 4.5 refers specifically to the parameter sensitivity experiment and does not indicate the configuration used for the main results. Table 1 consistently uses R=3R=3. Following the evaluation protocols used in prior context-compression studies [3,4], we report exact match (EM) scores averaged over three repeated runs. We have also checked the manuscript throughout and corrected related reporting errors and ambiguous descriptions. These corrections do not affect the experimental results or conclusions.

[3] End-to-End Context Compression at Scale. arXiv 2026.

[4] LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression. Findings of ACL 2024.

AuthorsOpenReview ↗

Response to Reviewer UKe2 – Part III

W4 & Q5: Please cite and compare against DAST (arXiv:2502.11493) and AdmTree (arXiv:2512.04550), including a controlled comparison isolating the allocation signal (your associative-memory surprise vs. DAST's perplexity+attention, under identical chunking) and the segmentation mechanism (your learned boundary head vs. AdmTree's density-based segmentation, under identical budgeting).

R4: Thank you for highlighting these important predecessors. DAST and AdmTree establish the benefits of adaptive compression. We have explicitly cited both methods, and expanded Related Work to clarify the challenges AdaMem further addresses.

Allocation signal. DAST combines chunk-local perplexity and global attention to prioritize informative content. However, linguistic predictability and contextual importance do not necessarily indicate how much additional memory a chunk requires. A passage containing familiar language can introduce new facts that deserve preservation, while difficult expressions may repeat previously represented information. This creates a potential mismatch between these importance signals and the capacity needed to retain new information. AdaMem addresses this mismatch by using prediction residuals against historical associative memory as an allocation signal, directing additional capacity toward content poorly represented by preceding associations.

Segmentation mechanism. AdmTree adapts compression granularity to information density, improving the balance of compression loads. Its density-dependent budgets determine equal-length subdivisions within each initial segment, however, so the resulting cuts need not follow semantic transitions. Such cuts can split related evidence or combine unrelated content within a compression unit. AdaMem directly predicts semantic boundaries and assigns capacity separately, allowing a coherent, information-rich passage to receive more slots without introducing additional cuts solely to increase capacity.

Full-model comparison. We evaluated DAST using its released code and reproduced AdmTree because we could not locate a public implementation. The table below reports EM (%) under a shared evaluation protocol.

Method8K16K64K
DAST-8×27.3426.5627.34
AdmTree-8×31.2525.7823.44
AdaMem78.9180.4776.56

AdaMem achieves higher EM at all three lengths and maintains strong performance as context length increases, supporting the effectiveness of its complete memory framework in this evaluation.

Controlled module comparisons. We additionally conducted two module-replacement experiments within AdaMem on HotpotQA at 16K. First, we retained AdaMem’s chunking and replaced its allocation signal with DAST’s perplexity-and-attention signal. Second, we retained AdaMem’s budgeting mechanism and replaced its boundary predictor with AdmTree’s density-based segmentation, using an initial segment length of 1,024 tokens. Total memory budgets were matched, with all other training and evaluation settings held constant.

Controlled ComparisonConfigurationEM (%)
Allocation signalAdaMem80.47
Allocation signalw/ DAST allocation signal37.50
Segmentation mechanismAdaMem80.47
Segmentation mechanismw/ AdmTree segmentation25.00

Replacing either component substantially reduces EM despite using the same total memory budget. These results show that performance depends on both how memory capacity is distributed and how the input is divided for compression. Within AdaMem, history-conditioned allocation and learned semantic boundaries provide more effective choices than the tested alternatives, supporting the contribution of these specific designs beyond adaptive compression alone. We thank the reviewer again for highlighting these relevant works and have added citations to DAST and AdmTree, in the Related Work section of the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer UKe2 – Part Ⅳ

W5 & Q4: Please clarify ICAE’s missing results at 512K/1M and LLMLingua-2’s non-monotonic performance, and strengthen coverage of relevant memory-based baselines. Since AdaMem targets growing contexts but is evaluated only on static-context QA, please evaluate multi-turn dialogue, streaming contexts, or iterative retrieval, or narrow the corresponding applicability claims.

R5: Thank you for highlighting baseline coverage and the need to evaluate beyond static document QA. We clarify the baseline results and report additional comparisons and multi-session dialogue experiments below.

Baseline results. ICAE encounters out-of-memory errors at 512K and 1M because its peak memory requirements exceed the available GPU capacity under our evaluation configuration. We have explicitly marked these entries as OOM and clarified the reason.

LLMLingua-2’s accuracy depends on the evidence retained after token selection, rather than input length alone. Its question-agnostic compression objective preserves generally informative content, which may not coincide with the specific evidence required by a downstream question. Changes in surrounding context, evidence position, and the number of retained tokens can therefore affect accuracy without producing a strictly monotonic relationship with input length. On HotpotQA, LLMLingua-2 increases from 24.22% at 32K to 27.34% at 64K, then decreases to 22.66% at 128K. It also outperforms ICAE at the latter two lengths, where ICAE achieves 25.78% and 20.31%, respectively.

Additional memory baseline. We evaluated AutoCompressor under the same protocol and compared it with AdaMem:

Method8K16K32K64K
AutoCompressor27.3420.3115.639.38
AdaMem78.9180.4775.7876.56

The evaluated AutoCompressor configuration processes 2K-token segments and supports approximately 80K context tokens before truncation; we therefore report results through 64K. AdaMem consistently achieves higher EM and maintains strong performance as context length increases, whereas AutoCompressor’s accuracy declines substantially. This comparison further supports the effectiveness of AdaMem’s adaptive memory organization and selective evidence recovery for long-context QA.

Multi-session dialogue. We additionally evaluated AdaMem on LongMemEval-S, extending the assessment from document-based QA to memory and reasoning over multi-session dialogue histories.

MethodLongMemEval-S Token F1 (%)
AdaMem20.19
DAST-16×12.54
AdmTree10.66

AdaMem outperforms DAST-16× and AdmTree by 7.65 and 9.53 percentage points, respectively. This extends its empirical benefits beyond document QA to conversational memory, supporting its ability to preserve and use information distributed across multiple sessions. The results provide additional evidence that AdaMem’s memory organization remains effective when relevant information is dispersed throughout dialogue histories.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

Dear Program Chairs, Area Chairs, and Reviewers,

We sincerely thank the reviewers for their careful reading and constructive feedback. We appreciate your recognition of AdaMem’s well-motivated design, the ablation evidence supporting its components, and its consistent performance gains.

  • A well-motivated framework in which each module contributes. AdaMem segments text at semantic boundaries, allocates memory according to how much new information each chunk introduces, and revisits relevant source passages when the compressed memory is insufficient to answer a question. (Reviewers UKe2, MZ9F, and n3ZM)
  • Ablation studies and budget analysis support the design. Ablations across datasets demonstrate the contribution of each module. The budget analysis also shows that AdaMem tends to allocate more memory to chunks containing the evidence needed to answer a question. (Reviewers UKe2, MZ9F, and n3ZM)
  • Consistent performance gains. AdaMem outperforms the evaluated baselines across dataset formats and context lengths. (Reviewers MZ9F and n3ZM)

During the rebuttal, we conducted additional experiments and revised the manuscript to address the reviewers’ questions about baseline comparisons, inference costs, task coverage, and the sources of AdaMem’s performance gains.

  • Additional baseline comparisons. When both models were fine-tuned on the same data, AdaMem outperformed LCLM at every context length tested. Our point-by-point responses also include comparisons with AutoCompressor, SAC, and EPL. (Reviewers UKe2 and n3ZM, revisions in Section 4.1, Table 3, and Appendix C.1)
  • End-to-end inference costs. On HotpotQA at 16K tokens, AdaMem achieved latency comparable to Qwen2.5-7B-Instruct-1M with lower peak GPU memory usage. We also report token retention rates and the computational cost of each stage. (Reviewers UKe2 and n3ZM, revisions in Section 4.3, Tables 14–16, and Appendix C.6)
  • Evaluation across additional tasks and settings. We added zero-shot meeting summarization on QMSum, cross-session question answering on LongMemEval-S, and long-document question answering on PG-19. AdaMem exceeded the published reference scores on QMSum and outperformed the evaluated baselines on the other two tasks. (Reviewers UKe2 and MZ9F, revisions in Section 4.2, Tables 10–12, and Appendices B and C.5)
  • Validation of adaptive compression. With the total memory budget held fixed, replacing either AdaMem’s chunking strategy or its budget allocation signal reduced accuracy. Further ablations support initializing memory slots from source-text representations and combining multiple boundary cues. (Reviewers UKe2, MZ9F, and n3ZM, revisions in Section 4.4, Tables 6 and 8, and Appendices C.2 and C.3)
  • Sources of performance gains. At every context length tested, AdaMem outperformed both the Qwen backbone fine-tuned on the same data and a model that answered using only retrieved source passages. These results show how compressed memory and on-demand replay complement each other. (Reviewer MZ9F, revisions in Section 4.4, Tables 4–5, and Appendix C.2)
  • Replay parameters and trigger rates. All main experiments used the same replay threshold. When replay was triggered, AdaMem retrieved the R most relevant source chunks. (Reviewers UKe2 and MZ9F, revisions in Table 14 and Appendices C.2 and C.6)

We again thank the Program Chairs, Area Chairs, and Reviewers for their time and thoughtful feedback. The additional experiments and ablation studies clarify where AdaMem’s performance gains come from, extend the evaluation to a broader range of tasks, and provide a fuller account of end-to-end inference latency, computational cost, and peak GPU memory usage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.