acceptodds
Under review as a conference paper at ICLR 2027

Towards continual in-context learning via training cartridges at test time

Abstract

Language model agents can learn from experience through in-context learning (ICL), but ICL is limited by finite context window and declining performance as context grows. We formulate **continual in-context learning** as preserving ICL's performance and adaptability as experience grows beyond these limits. Our controlled experiments identify two requirements for iterative key-value (KV) cache compaction: each memory must preserve knowledge as new context arrives, and memories from successive compactions must work together. To meet these requirements, we introduce Accordion, which repeatedly consolidates an agent's oldest experiences into trained KV memory modules while keeping the base model frozen. The agent revisits these experiences to generate self-study examples, then distills them into a compact cache initialized with improved Attention Matching. Each new module is trained alongside frozen earlier modules and varying amounts of subsequent context, preparing it to remain useful as experience accumulates. Across QuALITY, Blind Spectrum Monitoring from Continual Learning Bench, and Harvey LAB, Accordion achieves relative improvements of approximately 13%, 60%, and 18%, respectively, over the strongest evaluated baselines using the same base model. It supports repeated compaction at - compression and handles tasks over corpora of 1M-12M tokens through compact memory and tool use.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV compaction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BiumOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper formulates continual in-context learning as preserving ICL performance as an agent's experience outgrows its window, and identifies two requirements for repeated KV compaction: each module must survive appended context, and modules from successive compactions must compose. The proposed method meets both by distilling self-study from the oldest experiences into a new KV cartridge, initialized from ridge-regularized Attention Matching and trained behind the frozen stack. It reports roughly 13%, 60% and 18% relative gains over the strongest same-model baselines on three benchmarks at 19 to 23× compression.

Critical strengths
  1. The problem is well posed, and the failure analysis, separating survival of appended context from composition across modules, is a contribution independent of the proposed method.

  2. Evaluation is broad and largely budget-matched, covering recall, reasoning over a stream and agentic tool use against text-based, weight-based and KV baselines.

  3. Gains are consistent across these regimes at high compression, and the claims are hedged where the evidence is thin.

Critical weaknesses
  1. The distinguishing mechanism, training each module behind the frozen stack, is ablated only on the recall benchmark and worth a few points there, while the large composability effect comes from the synthetic task and the stream and agentic settings ablate only cadence, capacity and representation.

  2. On the most realistic benchmark, retrieval appears only as an oracle that still beats the method by 11 points, and the document store is never compacted, so the 12M-token figure reflects what tools reach rather than what the memory covers.

  3. Statistically thin evaluation: no intervals are reported for the stream, database or agentic benchmarks, and the database accuracy gap of 77.5% versus 71.7% over 20 questions, from two method runs against three baseline runs, is about one question.

Questions for authors
  1. Can you ablate the consolidation mechanism, stack-aware training and the initialization, on the stream and agentic benchmarks rather than only on the synthetic and recall tasks?

  2. Can you add a non-oracle retrieval baseline on the agentic benchmark, using the same tools the agent already has, since the oracle still outperforms the memory by 11 points?

  3. How many runs underlie the stream and agentic results, and can you report intervals for the 0.358 mean IoU and the 111 graded criteria, which come from a single judge pass?

  4. Can you report the database task with the original ordering and the recurrent-state-preserving configuration, so that setting matches the configuration and compression ratio used elsewhere?

5 comments
AuthorsOpenReview ↗

Reply by Authors (1/2)

Thank you for the close reading. We appreciate that you saw value in the failure analysis independent of the method, and that you found the evaluation broad and largely budget-matched. Following your suggestions, we ablated the method components on BSM and Database (W1, Q1), ran a non-oracle retrieval baseline on Harvey LAB (W2, Q2), and now report three-seed intervals on BSM, Database and Harvey LAB (W3, Q3). We also reran Database with the original ordering and GDN-state caching (Q4).

Method ablations on BSM and Database (W1, Q1)

Blind Spectrum Monitoring. We ablate each method component one at a time, fixing everything else. Training on top of the frozen stack adds about 0.06 to the BSM score on every seed. Ridge regularization matters within AM initialization, since removing it costs 0.062. All arms in a seed share the same self-study data, so the differences are paired.

Table 1. Component ablations on BSM, Qwen3.5-9B at cadence 15, paired per-seed differences, mean ± s.d. over 3 seeds. Differences are from Accordion trained on the same self-study data (0.327 ± 0.002).

MethodΔ vs Accordion
Accordion (sequential, ridge AM initialization)–
Independent compaction−0.058 ± 0.002
AM initialization with no ridge regularization−0.062 ± 0.018

Database. We replay existing Accordion trajectories and change one component at a time, so every arm uses the same trajectory and self-study data. We ablate independent vs. stack-aware training. Independent compaction lowers the score on every seed, by 0.062 on average, with most of that drop coming after the database change at Q21 (learning edits to old knowledge). We will add both ablation tables to the paper.

Table 2. Component ablations on Database, 3 seeds, paired per-seed differences from the control replay (0.347 ± 0.116).

MethodΔ vs Accordion (mean ± s.d.)
Accordion (control replay)–
Independent compaction−0.062 ± 0.015

Harvey LAB. Harvey LAB is an agentic task, so its results vary from run to run. Each Harvey LAB ablation arm would need to average results from multiple seeds to show a component's true effect. That cost does not fit in the rebuttal window. The BSM and Database ablations above already isolate the components on shared self-study data. We will add the Harvey component ablations to the revision.

Non-oracle retrieval baselines on Harvey LAB (W2, Q2)

The tools-only baseline in the paper is already a non-oracle retrieval baseline. Its agent searches and reads the same firm document store with the same tools as Accordion. In the paper, Accordion passes 87 of 111 criteria and this baseline passes 73.

As another non-oracle retrieval baseline, we ran retrieval-augmented generation (RAG). Before each task, BM25 uses the task prompt to preload 560K characters of passages from the document base. RAG keeps raw passages in context and compresses nothing, so it sits outside the compression methods we compare against (KVzip, SnapKV, Attention Matching, C²KV and CAS). RAG passes 88 of 111 criteria and Accordion passes 87, which is within run-to-run noise (Table 3). RAG needs far more memory in context to reach that score. We will add this result to the Harvey LAB appendix.

Table 3. Harvey LAB, Qwen3.8-27B, paper setting, single runs. Criteria passed out of 111, higher is better. All arms have the same document tools.

MethodMemoryCriteria passed (of 111)
RAG (BM25)Retrieved passages88
AccordionKV cartridges87
AuthorsOpenReview ↗

Reply by Authors (2/2)

Results hold over independent seeds on BSM, Database and Harvey LAB (W3, Q3)

Blind Spectrum Monitoring. The results hold across independent seeds. We reran every cadence on Qwen3.5-9B and Qwen3-32B with three seeds, and each seed regenerates its own trajectory and self-study data. The seed means stay within 0.013 of the submitted numbers (Table 4).

Table 4. BSM IoU, submission vs. mean ± s.d. over 3 independent seeds.

CadenceQwen3.5-9B, submissionQwen3.5-9B, 3 seedsQwen3-32B, submissionQwen3-32B, 3 seeds
Every 15 scans0.3580.345 ± 0.0170.3450.345 ± 0.049
Every 30 scans0.3200.324 ± 0.0120.2760.270 ± 0.037
Every 45 scans0.2550.263 ± 0.0110.2460.246 ± 0.046

Database. Over 40 questions and three seeds, Accordion has the highest mean benchmark score and is within noise of the top baselines (Table 5). In the updated results, on accuracy, Accordion leads KVzip by about 1.2 questions, AM only by about 3, ICL by about 6 and ACE by about 11.

Table 5. Database, 40 questions, 3 seeds, original ordering and GDN-state caching.

MethodRepeats per Day 1Mean benchmark score (before shift / after shift)Mean accuracy (before shift / after shift)STD across seeds
Accordion 20x10.348 (0.371 / 0.326)0.533 (0.533 / 0.533)0.069
KVzip 20x20.341 (0.364 / 0.318)0.504 (0.533 / 0.475)0.009
AM only 20x20.317 (0.326 / 0.308)0.458 (0.467 / 0.450)0.050
ICL20.309 (0.328 / 0.291)0.375 (0.425 / 0.325)0.034
ACE 20x20.142 (0.203 / 0.081)0.271 (0.342 / 0.200)0.036

Harvey LAB. Following the same setup as the paper, we run Accordion with 3 independent seeds and the tools-only ICL baseline with 7 independent seeds. On average, Accordion (86.7 of 111 criteria) outperforms the seven tools-only ICL runs (77.0) by about 10 criteria, which is smaller than the 14-criterion gap in the single-seed result of the submission. Judge noise is small next to run noise. Two independent Sonnet judges agree on 95.5% of 333 criterion verdicts and change run totals by at most 2 criteria. The revised paper will report these multi-seed means and intervals on all three tasks.

Table 6. Harvey LAB, Qwen3.8-27B. Criteria passed out of 111, higher is better.

MethodIndependent runsMean ± s.d.
ICL (tools only)777.0 ± 5.1
Accordion386.7 ± 5.5

Database with the original ordering and GDN-state caching (Q4)

As requested, we rerun Database with the original ordering, GDN-state caching and 20x compression, over all 40 questions and three independent seeds for every method. Each seed samples a fresh Day 1, and all methods continue independently from that same Day 1, which removes noise common to every method. Training-free methods get one extra repeat per seed. Results are in Table 5. Accordion has the highest mean benchmark score at 0.348 and is within noise of ICL and the best compaction baselines. We will replace Database results in the revision with three-seed results.

Interestingly, we find that Accordion is able to handle ground truth distribution shifts even with a frozen stack without retraining or modification. Training each new cartridge in context with older ones allows information to update cleanly. We present Database results with the purpose of matching ICL’s ability to handle problem solving under uncertainty, so Accordion being within noise of ICL and top training-free methods achieves this (under the new setting, this agrees with our submitted result of matching ICL). We strengthen this result by adding that Accordion matches top methods on ground truth shift.

We hope these ablations and three-seed intervals address your reservations about the frozen-stack mechanism and the statistical strength of the evaluation, and we would be grateful if you would consider raising your score.

Thanks for the extensive rebuttal. I appreciate the new results. However, the standard deviations (std) in some of these results seem high (e.g., Accordion 20x and KVzip 20x are close with high std in the Table 5 Database) and only three seeds have been used for these experiments. Therefore, I think my concern about statistically thin evaluation remains so I keep my rating.

Reviewer_muBLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how frozen language models can retain and reuse past experience across extended horizons. ACCORDION stores sequential experiences into a series of trained key-value (KV) cartridges, optimizing each new module alongside frozen predecessors. The framework integrates ridge-regularized Attention Matching (AM) initialization, compressed positional encoding, and multi-regime self-distillation. The method is evaluated across synthetic recall, long-document QA, time-series signal monitoring, database exploration, and legal reasoning, demonstrating gains over the evaluated KV-compaction and memory-augmented baselines.

Critical strengths
  1. Principled Analysis of AM Initialization: The paper provides a diagnosis of optimization instability in baseline Attention Matching, framing it as an ill-conditioned value-fitting problem. Demonstrating that ridge regularization stabilizes the conditioning and improves downstream accuracy offers a grounded insight beyond prior work.

  2. Architectural and Task Coverage: The evaluation spans multiple model families and attention mechanisms, including grouped-query attention, sliding-window attention, and hybrid linear attention. Covering both static retrieval tasks and dynamic, multi-turn tool/agent benchmarks provides evidence regarding the applicability of the approach.

Critical weaknesses

Incomplete Comparison with Closely Related Sequential Modules: While CAS established mixed-visibility training for coexisting modules, ACCORDION extends this to an incremental regime with frozen historical cartridges and varying raw context. However, the empirical benefit of ACCORDION’s specific formulation over a CAS-style training scheme adapted to the same sequential protocol is not directly quantified. Including a budget-matched comparison against this immediate neighbor would better delineate the precise methodological contribution.

Questions for authors
  1. Can the authors conduct the adaptation of CAS-style mixed-visibility training to this sequential setting? A controlled comparison under matched memory capacity and training budgets would clarify which components of ACCORDION are essential when prior cartridges are kept frozen.

  2. The ablation against recursive text summarization on the same self-study data is appreciated. For the cadence experiment (compaction every 15, 30, and 45 scans), changing the interval simultaneously varies the total study opportunities and the final cartridge capacity. Could you provide a controlled comparison that holds final capacity and compute budget fixed to isolate the effect of cadence?

3 comments
AuthorsOpenReview ↗

Reply by Authors (1/2)

Thank you for engaging so closely with the method. We appreciate that you saw the ridge-regularized AM analysis as a grounded insight beyond prior work, and that you found the range of attention mechanisms and tasks broad enough to support the approach. Following your questions, we adapted CAS to the sequential setting and compared it with Accordion on Cities, QuALITY and BSM, giving CAS the same memory and at least as much compute (W1, Q1). We also reran the BSM cadence study at matched compression and compute (Q2).

Accordion matches or beats a budget-matched sequential CAS (W1, Q1)

Cartridges at Scale (CAS) is designed as an offline approach. It trains one cartridge per document group, and each cartridge is trained in composition with random other cartridges. This requires all documents at once, so when new documents arrive during day phases, every cartridge must be retrained jointly each night. We adapt CAS to the sequential setting. On Cities, BSM, and QuALITY, we benchmark CAS with same memory capacity as Accordion and equal or greater training compute.

Cities. CAS differs from Accordion in (1) the training setup (previous cartridges stacked vs. mixed in as distractors), (2) the use of replay data, and (3) the initialization (the first tokens of the day vs. our ridge, re-rotated Attention Matching). Table 1 lists the settings, and Table 2 reports every variant we ran. Each variant is matched in compute and memory (the same training tokens and 512-token cartridge per night). With replay, CAS loads previously trained cartridges (“warm start”) and keeps all of them trainable, so it adds flexibility over Accordion under matched budget. CAS is served without the live window, its best setting, since it collapses with one.

Table 1. CAS vs. Accordion variants.

AccordionCAS
Training setupTrain current cartridge only. Past cartridges frozen and stacked.Past cartridges mixed in as distractors. Frozen without replay, retrained with replay.
ReplayNone (default), 10%, or full matching CAS.None or full: retrain all cartridges every night on data from all nights (closest to original).
InitializationRidge, re-rotated Attention Matching (ours).First tokens of the day (its own), or our ridge, re-rotated Attention Matching.

Accordion by default does not use replay and reaches 63.1% final accuracy. CAS with no replay (past cartridges frozen and mixed in as distractors) reaches 11.3% with its own initialization and 22.5% with our AM initialization. CAS by default assumes access to all data, so a sequential CAS needs full replay each night. Retraining each past cartridge each night, CAS reaches 62.5%, and 72.5% with our AM initialization. With full replay and past cartridges frozen, Accordion reaches 90.0%, and 82.5% with only 10% replay (Table 2). Accordion therefore exceeds CAS with and without replay, and our AM initialization also improves CAS by 10 points. These results demonstrate that Accordion enables strong knowledge retention without the requirement of retraining old cartridges or replaying old data (which would incur even more significant costs with more compactions).

Table 2. Accordion vs. CAS on Cities.

Method200300400500600700800
Accordion, AM init, full replay23.7%35.6%46.3%58.8%70.0%81.3%90.0%
Accordion, AM init, 10% replay23.7%35.0%46.9%58.8%67.5%79.4%82.5%
Accordion, AM init, no replay (default)23.7%31.2%40.6%47.5%51.2%55.0%63.1%
CAS, AM init, full replay11.3%23.1%33.8%45.0%53.8%61.3%72.5%
CAS, own init, full replay11.3%23.1%31.2%41.2%51.9%55.0%62.5%
CAS, AM init, no replay11.3%11.3%13.8%17.5%17.5%21.3%22.5%
CAS, own init, no replay11.3%11.3%11.9%9.4%9.4%9.4%11.3%
AuthorsOpenReview ↗

Reply by Authors (2/2)

QuALITY. Joint training is one of our oracles at 66% accuracy (all 11 cartridges initialized at once and tuned jointly in a preserved order). Accordion runs in the harder sequential setup, where the corpus grows over time, and reaches 60.7%. We let CAS use the offline, joint training setting, where all documents are available at once. CAS differs from the oracle because each cartridge is trained with mixed distractors (a random subset of the other cartridges, visible on 25% of rows). CAS reaches 54.7% with all documents available at once and 2.5x the compute. Accordion therefore leads CAS by 6.0 points while seeing the corpus incrementally.

Blind Spectrum Monitoring. We run a sequential CAS that retrains its cartridges each night with the same data and optimizer steps as Accordion. The gap between Accordion and CAS grows with the number of compactions, from about a tie after 1 night to +0.13 IoU after 5 nights (Table 3).

Table 3. BSM IoU, Qwen3.5-9B.

MethodCadence 15 (5 nights)Cadence 30 (2 nights)Cadence 45 (1 night)
CAS (matched compute)0.1970.2810.247
Accordion0.3270.3120.253

CAS retrains every earlier cartridge each night, so at fixed epochs each night costs more than the last, and at a fixed budget the same steps are spread over more cartridges each night. Over 13 compactions on Database and 18 on Harvey LAB, the CAS cost becomes prohibitively expensive. Accordion avoids this cost by keeping past cartridges frozen and training only the new one. We will add all of our new CAS results and full discussion to the paper.

Repeated compaction helps at matched compression and compute (Q2)

We reran the three cadences with compression and training compute held fixed, at about 2,600 optimizer steps each, over three seeds. Compacting more than once beats compacting once on every seed. Cadence 15 and cadence 30 score 0.305 and 0.338, against 0.245 for a single compaction (Table 4). Repeated compaction therefore beats a single compaction even at equal capacity and compute. Cadence 15 is best at the submission's capacity, where it ends with the most total capacity. Cadence 30 is best at matched compression and compute. We will revise Section 6.2, the Figure 5 caption and Appendix B.3.1 to report both settings.

Table 4. Qwen3.5-9B BSM score at matched 20x compression and compute, mean ± s.d. over 3 seeds. Slot counts differ from Table 3, so compare cadences within this table.

Cadence153045
BSM score0.305 ± 0.0180.338 ± 0.0040.245 ± 0.002

The comparison with CAS was the main weakness you raised. Our matched-compute ablations delineate the contribution of Accordion over CAS across tasks. We hope this comparison and the controlled cadence study address your concerns, and we would be grateful if you would consider raising your score.

Thank you for your response and the new experiment results. After careful consideration, I decide to maintain my rating.

Reviewer_qriKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates in-context learning in large language models under limited context windows. To address the limitations of finite context windows and performance degradation as context length increases, the authors propose ACCORDION, a continual in-context learning framework based on iterative KV cache compaction. The method enables agents to revisit historical experiences and generate self-study examples, which are then distilled into a compact KV cache initialized with an improved Attention Matching method. During the training of each new memory module, it is used alongside previously frozen memory modules and varying lengths of subsequent context, allowing the new module to remain effective as experience accumulates. The authors conduct experiments on three benchmarks: QuALITY, Blind Spectrum Monitoring from Continual Learning Bench, and Harvey LAB. The results show that ACCORDION achieves relative performance improvements of approximately 13%, 60%, and 18%, respectively, over the strongest evaluated baselines using the same base model.

Critical strengths
  1. Clear research motivation:This paper proposes ACCORDION to address the limitations of context windows and the forgetting of historical knowledge in large language models. The research motivation is clear.

  2. Relatively novel methodological design:This paper integrates multiple techniques into iterative memory construction with a frozen base model and introduces joint training with historical memory modules and dynamic contexts. The methodological design is relatively novel.

  3. Diverse experimental scenarios:This paper conducts experiments on precise knowledge recall, dynamic environment reasoning, database querying, and legal tool-use tasks, covering multiple model architectures. The experimental scenarios are relatively diverse.

Critical weaknesses
  1. Insufficient baseline comparisons:This paper lacks empirical comparisons under unified settings with methods directly related to the composability of multiple KV memory modules, such as Cartridges at Scale, C²KV, and online KV cache compaction approaches. The baseline comparisons remain insufficient.

  2. Insufficiently rigorous ablation experiments:In Blind Spectrum Monitoring, each module uses a fixed number of memory slots and self-study examples. Consequently, different settings simultaneously change the total memory capacity, the number of self-study rounds, and the training compute budget, making the ablation experiments insufficiently rigorous.

  3. Insufficient reliability of experimental results:In the Database SQLite experiment, the baseline results are obtained from three trajectories, whereas ACCORDION is evaluated using two continuations of the same historical trajectory. The lack of sufficient independent repeated experiments and statistical analysis raises concerns about the reliability of the experimental results.

Questions for authors
  1. RQ1: Could the authors further discuss the differences between Cartridges at Scale, C²KV, and the proposed method in terms of composability training, and provide comparisons under the same tasks and computational budgets where feasible?

  2. RQ2: Why does the Harvey LAB experiment use execution trajectories from the Tools-only baseline to construct ACCORDION's memory?

3 comments
AuthorsOpenReview ↗

Reply by Authors (1/3)

Thank you for the careful review. We are glad you found the research motivation clear, and that you saw the joint training with historical memory modules as a novel part of the design. Following your points, we added five baselines (CAS, C²KV, KVzip, SnapKV and IntentKV), each compared with Accordion at matched or more generous budgets (W1). We ran the cadence experiment with compression and compute held fixed (W2), and evaluated the Database benchmark over three independent seeds for every method (W3). Below we analyze how CAS, C²KV and Accordion differ in composability training (Q1) and explain how we built the Harvey LAB memory (Q2).

Accordion leads composable and online KV baselines across tasks (W1)

Following your suggestion, we add the requested baselines. (1) Cartridges at Scale (CAS) trains cartridges jointly, with other cartridges as distractors. (2) C²KV trains a general compressor once and compresses each document in one forward pass. We adapt the released codebase to train a compressor for Qwen3.5-9B, Qwen3.8-27B and Llama3.1-8B, and ensure checkpoints match the original paper's held-out eval scores. (3) Online compaction baselines. We choose KVzip and SnapKV, which are popular baselines in the KV cache compaction literature, including Attention Matching and C²KV. We also run IntentKV, a recent method designed for agentic settings. Its code is not yet released, so we implemented its main mechanism (Phase 1, which their ablation puts within 0.6 points of the full method on Qwen3-8B).

Cities. Every baseline other than CAS stays below 30% final accuracy, while Accordion reaches 63.1%. We run KVzip and SnapKV at 4x and 12.7x compression, since full 20x compression loses most of their accuracy, and C²KV at 16x (each at its paper's reported maximum).

CAS is designed for the offline case, where all data is known at once and all cartridges are trained together into a static, amortized memory. Regardless, we adapt it to the online sequential setting to provide more comparison. CAS differs from Accordion in: (1) the training setup (previous cartridges stacked vs. mixed in as distractors), (2) the use of replay data, and (3) the initialization (the first tokens of the day vs. our ridge, re-rotated Attention Matching). Table 1 lists the settings, and Table 2 reports every variant we ran. Each variant is matched in compute and memory (the same training tokens and 512-token cartridge per night). With replay, CAS loads previously trained cartridges (“warm start”) and keeps all of them trainable, so it adds flexibility over Accordion under matched budget. CAS is served without the live window, its best setting, since it collapses with one.

Table 1. CAS vs. Accordion variants.

AccordionCAS
Training setupTrain current cartridge only. Past cartridges frozen and stacked.Past cartridges mixed in as distractors. Frozen without replay, retrained with replay.
ReplayNone (default), 10%, or full matching CAS.None or full: retrain all cartridges every night on data from all nights (closest to original).
InitializationRidge, re-rotated Attention Matching (ours).First tokens of the day (its own), or our ridge, re-rotated Attention Matching.

Accordion by default does not use replay and reaches 63.1% final accuracy. CAS with no replay (past cartridges frozen and mixed in as distractors) reaches 11.3% with its own initialization and 22.5% with our AM initialization. CAS by default assumes access to all data, so a sequential CAS needs full replay each night. Retraining each past cartridge each night, CAS reaches 62.5%, and 72.5% with our AM initialization. With full replay and past cartridges frozen, Accordion reaches 90.0%, and 82.5% with only 10% replay (Table 2). Accordion therefore exceeds CAS with and without replay, and our AM initialization also improves CAS by 10 points. These results demonstrate that Accordion enables strong knowledge retention without the requirement of retraining old cartridges or replaying old data (which would incur even more significant costs with more compactions).

Table 2. Accordion vs. CAS on Cities.

Method200300400500600700800
Accordion, AM init, full replay23.7%35.6%46.3%58.8%70.0%81.3%90.0%
Accordion, AM init, 10% replay23.7%35.0%46.9%58.8%67.5%79.4%82.5%
Accordion, AM init, no replay (default)23.7%31.2%40.6%47.5%51.2%55.0%63.1%
CAS, AM init, full replay11.3%23.1%33.8%45.0%53.8%61.3%72.5%
CAS, own init, full replay11.3%23.1%31.2%41.2%51.9%55.0%62.5%
CAS, AM init, no replay11.3%11.3%13.8%17.5%17.5%21.3%22.5%
CAS, own init, no replay11.3%11.3%11.9%9.4%9.4%9.4%11.3%
AuthorsOpenReview ↗

Reply by Authors (2/3)

QuALITY. Accordion reaches 60.7% accuracy over the corpus and outperforms CAS, KVzip, SnapKV, Attention Matching and C²KV (Table 3). We give CAS access to all documents up front (offline setting), and it reaches 54.7%. Our oracle, which trains all documents jointly at once, reaches 66%.

Blind Spectrum Monitoring. Accordion (0.345) leads C²KV and all online KV cache compaction baselines. KVzip, SnapKV and Attention Matching stay near 0.23 (Table 3). Accordion also outperforms a CAS baseline that retrains its cartridges each night with the same data and optimizer steps.

Database. Accordion has the highest mean benchmark score (0.348) and is on par with ICL (0.309) and the top KV compaction baselines, KVzip (0.341), IntentKV (0.326), Attention Matching (0.317) and SnapKV (0.309). On Database, our main purpose is to match ICL and training-free methods on handling uncertainty and flexibility with ground truth shift, and Accordion is indeed tied among the top methods (under the new setting, this agrees with our submitted result of matching ICL). Under C²KV, memories become unstable once four to five compressed caches are concatenated, so its score is low (0.056). CAS retraining over 13 compactions is prohibitively expensive, and does not make sense to run on the Database task.

Table 3. Baselines across tasks. BSM is Qwen3.5-9B IoU at cadence 15, Database is Qwen3.8-27B mean benchmark score, and ± is s.d. over 3 seeds.

MethodCities, % (20x unless noted)BSM (20x)Database (20x)QuALITY, % (20x)
C²KV10.0 (16x)0.062 ± 0.0050.056 ± 0.05222.2
KVzip23.7 (4x)0.216 ± 0.0030.341 ± 0.00946.3
SnapKV11.3 (12.7x)0.218 ± 0.0030.309 ± 0.01549.4
IntentKV–0.220 ± 0.0030.326 ± 0.018–
Attention Matching (no training)15.60.232 ± 0.0080.317 ± 0.05052.3
CAS62.5 (own init), 72.5 (AM init), full replay0.244 ± 0.040–54.7
Accordion63.1 (no replay), 90.0 (full replay)0.345 ± 0.0170.348 ± 0.06960.7 ± 0.4

Repeated compaction helps at matched compression and compute (W2)

We reran the three cadences with compression and training compute held fixed, at about 2,600 optimizer steps each, over three seeds. Compacting more than once beats compacting once on every seed. Cadence 15 and cadence 30 score 0.305 and 0.338, against 0.245 for a single compaction (Table 4). Repeated compaction therefore beats a single compaction even at equal capacity and compute. Cadence 15 is best at the submission's capacity, where it ends with the most total capacity. Cadence 30 is best at matched compression and compute. We will revise Section 6.2, the Figure 5 caption and Appendix B.3.1 to report both settings.

Table 4. Qwen3.5-9B BSM score at matched 20x compression and compute, mean ± s.d. over 3 seeds. Slot counts differ from Table 3, so compare cadences within this table.

Cadence153045
BSM score0.305 ± 0.0180.338 ± 0.0040.245 ± 0.002
AuthorsOpenReview ↗

Reply by Authors (3/3)

Three independent seeds on Database (W3)

We run Database on all 40 questions with three independent seeds for every method, including Accordion. Each seed samples a fresh Day 1, and all methods continue independently from that same Day 1. Day 1 precedes any compaction, so sharing it removes noise common to every method and isolates the method difference. Training-free methods get one extra repeat per Day 1. Accordion has the highest mean benchmark score at 0.348 and is within noise of ICL and the best compaction baselines (Table 5). We will replace Database results in the paper with three-seed results.

Table 5. Database, Qwen3.8-27B, 40 questions, 3 seeds.

MethodRepeats per Day 1Mean benchmark score (before shift / after shift)STD across seeds
Accordion 20x10.348 (0.371 / 0.326)0.069
Attention Matching only 20x20.317 (0.326 / 0.308)0.050
ICL20.309 (0.328 / 0.291)0.034
ACE 20x20.142 (0.203 / 0.081)0.036

Since the submission, we also have solidified our Database setup by running 40 questions instead of 20 and setting compression rate to 20x and even compaction schedule. Interestingly, we find that Accordion is able to handle the ground truth distribution shift in the second 20 questions even with a frozen stack without retraining or modification. Training each new cartridge in context with older ones allows information to update cleanly. We present Database results with the purpose of matching ICL’s ability to handle problem solving under uncertainty, so Accordion being within noise of the top methods achieves this (under the new setting, this agrees with our submitted result of matching ICL). We strengthen this result by adding that Accordion matches top methods on ground truth shift.

Only Accordion composes memory online while adapting to the corpus (Q1)

Table 6. How CAS, C²KV and Accordion compose memory.

MethodOnline?Adapts to corpus?Sees existing memory when compressing?
CASNoYesYes
C²KVYesNoNo
AccordionYesYesYes

CAS trains one cartridge per document group, and each cartridge is trained in composition with random other cartridges. This requires all documents at once, so new documents force a joint retrain of every cartridge. Under full retraining with replay, at equal or greater compute, CAS is behind Accordion on Cities, BSM and QuALITY.

C²KV trains a compressor once on generic multi-document QA. Each document is then compressed online by one forward pass, and the caches are concatenated. The compressor cannot specialize to the corpus or task, and it never sees existing memory when compressing a new document. A single C²KV cache scores 49.5% on the first QuALITY group, close to KVzip (51.2%) and SnapKV (52.3%), but because the compressing mechanism is independent of previous memories, C²KV’s memories do not compose well, and accuracy falls to 22.2% after 11 groups. C²KV accuracy falls the same way on Cities, BSM and Database, and it underperforms Accordion.

Accordion trains each new cartridge behind the frozen stack, without retraining previous cartridges. It adapts to the task or corpus through its self-study data and the existing memory in the stack. Under Accordion, accuracy rises with each compaction on Cities and BSM. We will add Table 6 and this discussion to the related work section.

Shared trajectories isolate how each method compacts experience (Q2)

In Harvey LAB, we built Accordion KV caches from the tools-only execution trajectories. Fixing the experience each method sees attributes any gap between methods to how each one compacts experience. When we instead build the caches from the agent's own on-policy trajectory, Accordion passes 92 of 111 criteria, and building from the tools-only trajectories passes 87 in the paper (Table 7).

Table 7. Harvey LAB, Qwen3.8-27B. Criteria passed by Accordion.

TasksCriteriaPaper (tools-only trajectories)Agent's own on-policy trajectories
1-6262123
7-12362829
13-18493840
Total1118792

We hope the matched-compute baselines across tasks, the discussion of composability, and the BSM and Database reruns address your concerns about baseline comparisons and reliability, and we would be grateful if you would consider raising your score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.