acceptodds
Under review as a conference paper at ICLR 2027

Continual Context Compression: Training Models to Learn Continually at Test Time

Abstract

We formalize test-time continual learning (TTCL), in which models continually acquire, retain, and use information from an open-ended deployment stream. We focus on observation-driven TTCL, where future queries are unknown during learning. The de facto solution, continual in-context learning (ICL), retains deployment history in context, causing memory and inference costs to grow with experience. We instead argue that models should be explicitly trained before deployment to learn continually at test time. Our training algorithm, Continual Context Compression (C3), combines long-context data, context distillation, and bounded memory to learn a state-update rule. The resulting algorithm performs test-time compression: it consolidates evicted context into persistent state in a single forward pass, without deployment-time supervision generation or gradient descent. Across repeated updates, it outperforms gradient-based sleep-time methods with orders of magnitude less deployment compute. Scaling to Qwen3-8B, C3 outperforms matched-budget memory baselines on future-use tasks, with strong acquisition and retention and limited interference with existing capabilities. Moreover, on held-out label-learning tasks, its state can even outperform continual ICL within the model's native context range. These results support TTCL as a first-class objective of model development; models should be trained not only for what they know, but for how effectively they learn afterward.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, lifelong learning, in-context learning, test-time training, KV cache eviction, memory systems, compression

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vzPUOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper formalizes test-time continual learning in the observation-driven regime and proposes C3, which trains a bounded-state consolidation operator offline by context distillation from a frozen full-context teacher, so evicted blocks are folded into fixed-size KV-row state in one forward pass at deployment. It reports that C3 beats gradient-based sleep-time consolidation at far lower deployment compute and beats matched-byte memory baselines, with the learned state occasionally exceeding full-history ICL.

Critical strengths
  1. Clear formulation of test-time continual learning as constraints plus capabilities, with evaluations mapped to each requirement.

  2. Deployment compute is measured for the main comparisons, including supervision generation, and the oracle-GD control is informative.

  3. Diagnostics separate acquisition, retention and interference rather than only ranking methods.

Critical weaknesses
  1. The central accuracy claim is weak: C3 is trained to reproduce full-history ICL, but it does not exceed that teacher on any task except TREC, where the margin is 0.01 to 0.03 and no intervals or seeds are reported.

  2. The stated distinctions from the closest prior work are the consolidation rule (versus CCM) and the continual-learning objective (versus Still), yet neither method is evaluated.

  3. The organizing claim, that learning the update rule offline beats optimizing state during deployment, is tested only at 1.7B on 8K to 16K streams, and the one replay-based gradient variant is given storage outside the stated budget.

Questions for authors
  1. Can you evaluate CCM and Still under the same streaming protocol and byte budget, since your related work distinguishes C3 from CCM by the consolidation rule and from Still by the continual-learning objective, yet neither appears in the experiments?

  2. Can you add intervals and multiple seeds to Figures 3, 4, 11 and 13, and does the TREC advantage over full-history ICL survive?

  3. Which evaluation tasks appear in the decay mixture, and can you report format-held-out banking77 and NIAH variants?

  4. Can GD be given a retention mechanism that respects the byte budget, and can the asymmetric YaRN factor at 131K in EventQA be justified?

4 comments

Thank you for recognizing the formulation, compute accounting, and acquisition and retention diagnostics. Your questions prompted us to clarify how we compare with the full-history teacher and exactly which gradient-based comparisons fit the memory budget.


W1. Comparison with the full-history teacher. We agree that the revised C3 falls below full-history ICL on TREC, and we withdraw the claim that it surpasses the teacher. At fixed memory, C3 recovers most of the teacher’s retrieval gain on NIAH multikey. At 32K, accuracy is .948 for C3, .985 for the full-history teacher, and .062 for the live window alone.

W2 / Q1. Direct comparisons with CCM and Still. We tested CCM-style averaging within C3 and found that retaining whole rows outperformed averaging the old and new states. The published CCM setting writes 2–8 compression tokens per step, while matching our state directly would require about 2,048. Still compacts an individual context and keeps each compacted chunk in its multi-cycle version, making it not bounded state sized for the continual version of this method.

W3 / Q4. Scale and an in-budget retention baseline. We evaluated continual GD performance at 1.7B on 8K–16K streams. At 8B we measured cost only. Knowledge Seeding takes 1,014 seconds to consolidate one 32K stream on an H100, against 4.4 seconds for C3 (Fig. 8). At 1.7B, the oracle row-state experiment separates acquisition from retention. GD answers .998 of the newest block’s questions correctly just after fitting them with gold supervision. One write later, GD accuracy on those questions is at most .02, compared with .435 for C3 (Fig. 2).

Knowledge Seeding without replay carries its updated adapter from block to block and fits the deployment budget. C3 leads the in-budget variant on all three tasks at 8K, including .766 versus .393 on TriviaQA (Fig. 13, App. E). We also tested a version that retains all prior quiz pairs and excluded that buffer from its budget, giving GD extra storage, which only improved performance slightly.

Q2. Intervals and training seeds. The ablations use a single training seed. Figures 4 and 12 report two trained state sizes and 14 tasks, most at eight context lengths. The advantage of C3 over full-history ICL on TREC did not carry over to the revised merge rule, we update the statement.

Q3. Exposure to evaluation tasks. The revised C3 trains on questions generated about general long documents. The revised banking77 and NIAH results come from a model trained without either evaluation task. The submitted model did receive task data in its decay stage, including banking77 demonstrations, but never the held out test questions.

Q4. YaRN at 131K. The submitted EventQA comparison used a factor of 2 for C3 and 4 for the baselines. We now use 4 for all methods. Matching the factors changes C3’s accuracy by .014. The number of retained rows is fixed, but the rows keep their original rotary positions, so their distances to a query on the live window grow with the stream.


Overall, thank you for your feedback and interest in our paper! We hope to have addressed all of you questions, and would appreciate your reconsideration of the score. Please let us know if you have further comments or questions.

Thanks for the rebuttal and the new results. While the revised C3 looks better than the submitted version, some concerns remain. In the 128K NIAH multikey panel, both merge rules sit at the no-memory level for evidence more than 32K tokens back, so retention beyond the native window is not yet demonstrated. The revised rule is also compared only against the teacher and the previous rule, not against the memory baselines, and the ablations remain a single training seed. For these reasons, I keep my negative rating.

We want to thank the Reviewer for their fast reply and for looking closely at the new results. We have updated the PDF so that every figure uses the revised C3, including the comparisons with all memory baselines, which is uploaded in the following anonymized link for convenience: https://anonymous.4open.science/r/ICLR-C3-5573/ctx8b_r2048_grid.pdf

In Figure 18 we show the performance versus evidence age at 32k and 64k context length, where C3 demonstrates retention beyond the native 32k context window.

We agree that at 128K, NIAH retention beyond 32K is weak. For needles more than 32K tokens back, C3 averages .05 (at most .13) against 0 for no memory. However, in this setting the same 2048 rows hold all 63 evicted blocks (126k tokens), where there is much more competition for admission than within a 32k token stream. On TriviaQA at 128K, C3 stays above no memory at every distance (.32 against .21 for evidence more than 32K back.

We could not train additional seeds within the discussion period. The ablation conclusions we draw rest on differences of .10 or more (Appendix D), well above the .03 we treat as inconclusive.

We hope these clarifications address the Reviewer’s remaining concerns, and we thank them again for their time and careful reading.

Thanks for the reply, updated PDF and clarifications. These are nice but I think that the real value of the context compression would lie in the case of longer context (e.g., at or beyond 128K), for which the current results seem inconclusive. Therefore, I thank the authors for the effort but keep my evaluation as is.

Reviewer_EKjaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Test-Time Continual Learning (TTCL), where models need to learn and remember new information during deployment. The authors focus on a setting where future questions are unknown when the model receives new information. They propose Continual Context Compression (C3), which learns to compress old context into a fixed-size memory. C3 is trained using context distillation and updates the memory with a single forward pass, without gradient updates at test time. The experiments show that C3 performs better than several memory baselines and gradient-based methods, while using much less compute during deployment. The authors also report good knowledge retention and limited interference with the model's existing abilities.

Critical strengths
  1. The paper studies an important problem and provides a clear setting for test-time continual learning.
  2. The proposed C3 method is simple and efficient, as it can update memory without gradient descent at test time.
  3. The experiments cover different tasks and include useful comparisons and ablation studies to support the method.
Critical weaknesses
  1. Novelty of the proposed paradigm and comparison with related work. The proposed Test-Time Continual Learning paradigm is interesting and meaningful. However, it shares several similarities with existing work on continual memory and test-time learning. For example, MEMORYLLM: Towards Self-Updatable Large Language Models (ICML 2024) explores continual knowledge updates with a fixed-size memory, while Compressed Context Memory for Online Language Model Interaction (ICLR 2024) investigates recurrent KV memory compression. It would be helpful if the authors could further clarify the key differences between C3 and these approaches, and include more direct comparisons with these relevant baselines to better highlight the paper's contributions.

  2. Potential limitations of Competitive Merge for long-term continual learning. The Competitive Merge mechanism ensures that the gate value of each existing Memory Row never decreases. As the number of updates increases, some gate values may gradually approach their upper bound, making it increasingly difficult for new information to replace existing memories. This could potentially lead to memory saturation and limit the model's ability to continuously acquire new knowledge. The Admission Bottleneck observed in the paper also seems related to this concern. Since long-term continual learning is a central motivation of the work, it would be valuable to further investigate this behavior and evaluate the method over substantially longer update sequences.

  3. Limited explicit integration of information across time. Competitive Merge selects between the existing Memory Row and a new candidate rather than explicitly combining their contents. While this design may help reduce interference, it raises the question of how effectively the model can accumulate and integrate information across different time steps. For example, some tasks require combining evidence from multiple historical blocks to derive new knowledge or patterns. Although such integration may occur implicitly, the current experiments do not fully clarify this capability. It would strengthen the paper to include additional evaluations requiring information integration across multiple blocks and discuss how the proposed mechanism supports this process.

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for highlighting both the simplicity of C3 and the difficulty of admitting new information into a full state. Your observation and Reviewer Kjvv’s prompted us to replace the merge rule.


W1. Relationship to MemoryLLM and CCM. All three methods maintain a fixed-size memory through forward passes. MemoryLLM drops old tokens at random, and CCM’s averaging variant combines the old and new compressed states. C3 uses the current context to decide which individual rows to retain. We tested this distinction directly within C3. Retaining whole rows outperformed CCM-style averaging, a learned gate, and a learned rewrite. Our comparison with MemoryLLM and CCM is at the level of the update rule. We will clarify this in Appendix A.

W2. Saturation over repeated updates. The old rule favored rows admitted early, for it carried each stored row’s score into the next competition. The new probes recompute every score from the current context. At 32K, TriviaQA recall is nearly identical for recent and older evidence (.71 for ages 1–7 and .72 for ages 8–15, Fig. 17), in contrast to the submitted rule whose recall for recently evicted evidence fell to .30–.47 as we show in Comparison by evidence age.

W3. Integration across blocks. A query attends to rows from multiple blocks, and later backbone layers can combine what earlier layers read. On OOLONG-synth, where answering requires aggregating labels across the stream, C3 scores .57–.66 at 8K–32K, compared with .38–.45 for the live window alone. On banking77, demonstrations spread across the stream teach a label mapping that C3 applies to new utterances (.80 versus .49 at 32K).


Overall, thank you for your thoughtful response and detailed feedback. We hope that the revision of the merge rule and the manuscript address your concerns, and would appreciate your reconsideration of the score.

Thank you for the response. I appreciate the authors' clarifications, but I still have two concerns. First, the results only cover up to 15 memory updates, which is not enough to address long-term memory saturation. Second, it remains unclear whether cross-block information is integrated during memory updates or only during reading. Therefore, I will keep my original score.

Reviewer_KjvvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The idea is: freeze Qwen3, and every time a 2048-token block falls out of the window, a trained “writer” squashes it into a fixed bank of about 2k KV rows. The write step pools the block’s tokens, the merge step keeps whichever row has more “mass” (old or new, winner takes all), and the read step is an extra gated attention head over the rows. It’s trained so that frozen model + memory behaves like frozen model + full history. They call the whole thing “test-time continual learning” (TTCL). They claim it beats gradient-based sleep-time methods with way less compute, beats KeyDiff, text summaries, SHINE etc. at the same memory budget, and on TREC even beats just keeping everything in context.

Critical strengths
  • Clean idea and solid engineering.
  • Logical story
  • The appendix is honestly great. The memory-substrate comparisons and the GD diagnostics are probably the most interesting part of the paper.
Critical weaknesses
  1. The merge rule can only fill up, never free up. Mass is max(old, new), so it only goes up, and over time new stuff has a harder and harder time getting in. You can already see it in Fig 14. Blocks evicted recently land around 0.3–0.45 while full history gets about 0.8, and the oldest blocks do best. That's kind of the opposite of what you want from a system pitched as learning forever. Streams top out around 80 updates, and the one decay ablation used 8 blocks. I'd really like to see 500+ blocks.

  2. The GD baseline looks undertuned. It has a high fixed learning rate, resets the optimizer every block, and uses no replay or regularization. With just 4 training pairs per block it already tanks in-window accuracy from 1.0 to about 0.55. That smells like an optimization problem, not proof that "gradient methods are bad." And the big claim rests on one NIAH setup at 1.7B.

  3. Missing related work. They say CCM is the closest method and then don't compare to it. There's also no RMT, AutoCompressors, Gist, ICAE, Activation Beacon, Infini-attention, Titans, MemoryLLM, or TTT layers. So "first demonstration" is a stretch.

  4. The framing is a bit much. The weights are frozen, so this is really learned recurrent compression of long context with a continual-learning label on top. The TTCL formalism is mostly definitions.

  5. Some comparisons look uneven:

    • On EventQA at 131K the baselines get YaRN ×4 and C3 gets ×2. And why does C3 need YaRN at all when its context is bounded?
    • C3 adds about 1–1.3B trainable params, while KeyDiff and the text summaries add zero.
    • The offline SFT control is a small LoRA trained on the 1.7B data mix, which doesn't match the 8B one.
    • SHINE does worse than no memory in places, so something's probably off with how it was run.
  6. "Beats ICL" is shaky. It only shows up on TREC, at low accuracy (around 0.3–0.48), by a few points, with no error bars. Full-history ICL also looks like it's degrading at that length. I'd want to see recent-only demos, retrieved demos, or a many-shot curve before buying it.

  7. Stats: every ablation is one seed, and the main plots have no confidence intervals.

Questions for authors
  1. Is banking77, or its label set, anywhere in the training data?
  2. What happens to memory over 500+ blocks?
  3. Does the GD result survive a learning-rate sweep plus replay or regularization?
  4. How does C3 do against CCM or Activation Beacon at the same budget?
  5. Why do C3 and the baselines get different YaRN factors on EventQA?
  6. When does the ~2800 GPU-hours of offline training per state size pay for itself?
2 comments
AuthorsOpenReview ↗

Response 1/2

Thank you for the careful review and for highlighting the memory and gradient-descent diagnostics. Your observation about the merge rule led us to change how C3 admits new information once its memory is full.


W1 / Q2. Memory saturation and the 500-block horizon. The submitted rule let stored rows accumulate attention mass across writes. The revised rule scores every row afresh, so any stored row can be displaced. At 32K, the original rule recalled only .30–.47 of recently evicted TriviaQA evidence. With the revised rule, recall is .71 for ages 1–7 and .72 for ages 8–15 (Fig. 17). Comparison of both merge rules.

Our longest test covers 255 writes (512K tokens). At 128K, NIAH multikey recall favors recent evidence. The advantage over the live window shrinks at longer horizons. On evicted TriviaQA evidence, C3 scores .349 versus .214 for the live window at 128K, and .224 versus .200 at 512K. On NIAH multikey the gain at 512K is .003.

W2 / Q3. Gradient-descent tuning and replay. The row-state oracle uses a fixed learning rate within each task. With gold supervision, GD answers .998 of the newest block’s questions correctly immediately after the write, but at most .02 one write later. C3 retains .435 on the same questions (Fig. 2).

We also evaluated Knowledge Seeding (Behrouz et al., 2026), which updates a per-stream LoRA at a learning rate 300–1,000 times smaller than in the row-state experiment. C3 leads both Knowledge Seeding variants on all three tasks at 1.7B and 8K tokens (Fig. 13, App. E), which we hope answers the Reviewer's concern.

TaskKnowledge Seeding + replayKnowledge Seeding without replayC3No memory
TriviaQA.369.393.766.333
banking77.393.390.648.337
TREC-fine.163.157.188.140

The replay variant retains all prior quiz pairs outside the adapter budget. The version without replay carries only the adapter.

W3 / Q4. Related work and matched-budget comparisons. We agree that the “first demonstration” wording was too broad and have removed it. CCM’s averaging variant, MemoryLLM, and RMT also update bounded memory through forward passes. The methods differ in what happens to stored information when a new block arrives.

MethodHow the method updates existing memory
CCM (averaging variant)Averages the stored and new compressed states.
MemoryLLMDrops old memory tokens at random to make room for new ones.
RMTRewrites a fixed set of memory tokens after each segment.
C3Scores stored and new rows together and keeps the highest-scoring rows.

Within C3, keeping whole rows outperformed CCM-style averaging, a learned gate, and a learned rewrite of stored rows. This compares merge rules within the same architecture. For the end-to-end comparison you ask about, CCM was designed for 2–8 compression tokens per write, while matching our state directly would require about 2,048. Activation Beacon accumulates compressed representations as the stream grows. We did not benchmark either full system at matched bytes.

Appendix A will also distinguish methods that accumulate summaries (AutoCompressors), compress a single prompt or span (Gist and ICAE), maintain a fixed recurrent matrix (Infini-attention), or update fast weights (Titans and TTT layers).

W4. What learns when the backbone is frozen? In our definition of test-time continual learning, an observation changes persistent state before the model knows how that observation will be used. In C3 the state consists of KV rows. On banking77, for example, demonstrations that have already been evicted still help the model classify a new utterance. The interpretation is consistent with fast-weight programming (Schlag et al., 2021) and with work treating KV prefixes as adaptable parameters in prefix tuning (Li & Liang, 2021), its connection to adapters (He et al., 2022), and Cartridges (Eyuboglu et al., 2025). C3 learns offline how to write those parameters in a forward pass.

AuthorsOpenReview ↗

Response 2/2

W5 / Q5. Fairness of the comparisons.

  • YaRN. The submitted EventQA comparison at 131K used a factor of 2 for C3 and 4 for the baselines. We reran C3 at 4 so that every method uses the same factor. Matching YaRN changes C3’s accuracy by .014 (.522 and .536). C3 keeps a fixed number of rows, but each keeps its original rotary position. The distance from a stored row to the current query can therefore exceed the backbone’s native context range.
  • Trainable parameters. The revision reduces the number of trained parameters by a factor of 57, to 18.3M shared parameters. The per-session budget measures the state carried from one block to the next. KeyDiff requires no training, while text compaction generates a new summary at each write.
  • SFT control. The revised C3 trains on general long documents. The offline SFT control still trains on the task mixture, so it receives more direct task exposure than C3.
  • SHINE. We used the authors’ released weights and code. Our 6K–63K contexts exceed the sub-1,150-token contexts SHINE was designed for. Its per-stream LoRA also affects the entire subsequent computation. With an unrelated essay in memory, GSM8K accuracy falls from .95 to .22 for SHINE and from .91 to .90 for C3 (Fig. 19).

W6. Comparison with full-history ICL. The revised C3 falls below full-history ICL on TREC, and we withdraw the claim that it exceeds ICL. C3 still recovers most of the benefit of earlier demonstrations at a fixed budget. On banking77 at 32K, C3 reaches .803 accuracy, compared with .875 for full-history ICL.

W7. Statistical uncertainty. The ablations use a single training seed. Figures 4 and 12 cover two trained state sizes and 14 tasks, most at eight context lengths.

Q1. Exposure to banking77. The submitted model’s final training stage included banking77 demonstrations, with the test questions held out. The revised C3 uses general long documents and receives no banking77 demonstrations or label set as training supervision.

Q6. When does training pay for itself? Training the revised 8B C3 took about 65 GPU-hours, compared with about 1,100 for the submitted model. For scale, Knowledge Seeding takes .3 GPU-hours to consolidate one 32K stream at 8B (Fig. 8). In GPU-hours, training the revised C3 once costs about as much as consolidating 230 such streams with Knowledge Seeding. Compute that amortizes accross session and user.


Overall, we hope the revised manuscript and new results address your concerns, and would appreciate your reconsideration of the score in light of this revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their thoughtful feedback. Your comments have prompted us to improve the merge rule and to measure how well C3 recalls evidence of different ages.

  1. Admitting new evidence. At every write, a learned probe now scores the stored rows together with the newly evicted rows and keeps the highest-scoring ones. A stored row can therefore be displaced even if it scored highly at an earlier write. At 32K, TriviaQA recall is nearly identical for recent and older evidence (.71 for evidence evicted 1–7 writes ago and .72 for 8–15 writes ago). We show the comparison between the old and the new admission rule in the following Figure Recall by evidence age.
  2. Performance of the revised model. Across 8K–64K tokens, the revised C3 improves on 10 out of the 12 tasks, gaining 14.3 percentage points on average over the submitted model. as we show in the Figure Comparison across all 12 tasks. It continues to lead the matched-budget baselines in our main comparisons, which we have updated in the submitted pdf.

We address each reviewer's specific concerns in the corresponding response thread.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.