acceptodds
Under review as a conference paper at ICLR 2027

Persistent Context Graphs for Efficient Memory Compaction in LLM Agents

Abstract

As LLM capabilities advance, agents are tackling increasingly complex tasks over longer horizons. Their growing interaction histories make memory compaction essential for staying within context windows and reducing prefill cost. Existing methods summarize the history or compress its KV cache, often adding model computation to preserve information for future requests. A new user request can change which history matters, but reassessing that history with the model requires re-encoding it if the KV cache has expired. Past attention provides signals of historical importance and dependencies between messages, while relevance to the current task must be assessed using the new user request. We introduce ReCAP, a memory compaction method that stores attention-derived importance scores and dependency links in a lightweight, persistent context graph. For each new request, ReCAP combines stored importance with relevance cues from the request and follows dependency links to select messages and their supporting context, without additional model calls for selection. Compared with Codex's default summarization-based compaction, ReCAP reduces estimated latency for compaction and cold restoration by approximately 95% on both Qwen3-Coder and gpt-oss. It also roughly halves the historical context per call on SWE-Together at comparable task quality and improves accuracy on the code tasks of Lost-in-Conversation over full history by 19.8 and 41.2 points.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents, Memory Compaction, Context Management

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uPXNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RECAP for managing the history of multi-turn coding agents. It retains the full conversation archive while maintaining a lightweight graph of message blocks. The graph records attention-derived importance scores and links between blocks; when a new user request arrives, the method combines these signals with identifier overlap to select a smaller working context without another model call.The problem this paper addresses is real. It proposes a memory compression approach based on blocks and a block index. In essence, it stores the memory as blocks elsewhere, retains a summary, and uses heuristic retrieval to bring relevant blocks back. However, I see several problems. It does not actually reduce total storage requirements; it may simply move the memory from RAM to disk, while still requiring frequent transfers between them. The memory selection process relies heavily on heuristics and empirically tuned parameters. The paper offers no substantive theory explaining the relationship between attention and importance, or why its various thresholds and weights were chosen. The experimental evaluation is incomplete: it does not compare against some of the methods discussed in the related work, and the results are not sufficiently convincing. Overall, I would recommend rejection.

Critical strengths

The paper addresses a real problem: long agent histories increase context costs, while future user requests may need details that were previously omitted. The method can retrieve such details from the archive. Its Lost-in-Conversation results are substantial, and the reported context-selection cost is low. The evaluation includes full history, recency, random selection, compression methods, KV eviction, and a Codex-style summarization baseline.

Critical weaknesses

My main concern is that the evidence does not yet isolate why the proposed selection rule works or establish its full operational cost. Attention is used both as an importance signal and as a proxy for dependency, but the paper does not establish that an attention link reliably identifies context needed for a later task. The choices of thresholds and weights receive limited justification or sensitivity analysis. Moreover, the “no attention graph” ablation removes importance scores, dependency links, and stale-block handling together, so it cannot attribute the observed gain to any one component.

The efficiency claim also needs a clearer accounting. RECAP reduces the working context, but it retains the complete text archive; the reported graph-memory figure excludes that archive. The paper does not measure archive storage or retrieval I/O, nor does it provide an end-to-end measurement covering the overhead of collecting attention statistics. Its cold-restoration latency result combines separately measured compression time with synthetic cold-request TTFT, rather than measuring the complete workflow.

Finally, the quality–context trade-off is mixed. On SWE-Together with Qwen3-Coder, the Codex-style summary attains a slightly higher reported score than RECAP (28.7 versus 28.5) while using substantially fewer context tokens (5,239 versus 15,344). The Lost-in-Conversation gains are compelling, but matched-budget comparisons and more targeted ablations would make the overall conclusion more convincing.

Questions for authors

How sensitive are the results to the attention threshold and the importance/query weights? What are the separate contributions of node importance, dependency expansion, stale-block handling, and protected records? What are the end-to-end costs of attention collection and archive retrieval? Can the authors report quality across matched context budgets, particularly against the summarization baseline?

8 comments
AuthorsOpenReview ↗

Response to Reviewer uPXN (Part 1)

We thank the reviewer for the detailed assessment and for recognizing the importance of efficient agent memory, the strong results, and the low cost of RECAP. We address the mechanism, system costs, and quality comparisons with additional experiments.

[W1&Q2] The evidence does not isolate why the selection rule works: separate contributions of node importance, dependency expansion, stale-block handling and protected records, and whether an attention link identifies context needed later.

The earlier no-graph ablation removed multiple components together. We now separate their contributions and test two properties of attention links. The component ablations measure task quality. Next-turn attention tests whether the ranking predicts later reuse. Parent removal tests whether linked supporting blocks matter after selection.

  1. We remove protected records, attention-derived importance, stale handling, and dependency expansion one at a time, keeping all remaining mechanisms unchanged (Table 1).
  • Protected records retain all historical user messages and the latest edit or write record for each modified file.
  • Node importance accumulates the attention received by a historical block through an exponential moving average.
  • Stale handling downweights earlier blocks that substantially overlap with later, more strongly attended content and excludes them from dependency expansion.
  • Dependency expansion follows stored attention links from initially selected blocks to earlier supporting blocks.
  • We compare the next-turn attention received by score-selected blocks with that received by random additions. We also remove linked parent blocks or the same number of random selected blocks and refill the freed budget (Table 2). The first test checks whether the score predicts later use. The second tests the added value of linked supporting context.

The isolated ablations retain all components except the one removed. We report SWE-Together reward with Qwen3-Coder-30B-A3B and LiC accuracy with gpt-oss-20b, both scaled by 100. Each Δ is the score with the component minus the score without it. The linked-parent removal test also includes a setting without mandatory protected records.

The results are as follows:

[Table 1] Isolated contributions of RECAP components on SWE-Together and LiC

ComponentBenchmarkWithWithoutΔ
Protected recordsSWE-Together28.525.2+3.3
Attention-derived importanceSWE-Together28.523.6+4.9
Stale handlingSWE-Together28.526.2+2.3
Dependency expansionSWE-Together28.526.4+2.1
Protected recordsLiC67.663.5+4.1
Attention-derived importanceLiC67.658.7+8.9
Dependency expansionLiC67.665.3+2.3

Stale handling is not triggered by the LiC histories in these runs, so we report this ablation only on SWE-Together.

[Table 2] Removing linked parents versus random selected blocks on LiC without mandatory protection

Nothing removedRandom blocks removedParents removed
Accuracy63.548.245.1

The results show complementary contributions from persistent attention and dependency links.

  • Importance is the largest isolated contributor in these tests. Removing it lowers LiC accuracy by 8.9 points and SWE-Together reward by 4.9 points (Table 1). This supports preserving historical attention across turns.
  • Stale handling adds 2.3 points of SWE-Together reward by lowering the priority of likely superseded content (Table 1).
  • Dependency expansion adds 2.1 points on SWE-Together and 2.3 points on LiC when the other components are retained (Table 1). It therefore contributes beyond attention-based ranking alone.
  • The next-turn analysis tests whether the importance signal predicts later use. Score-selected blocks are at the 49th to 51st percentile of next-turn attention, compared with the 38th to 39th percentile for random additions. The parent removal test examines the links more directly. On LiC without mandatory protection, removing graph-linked parents lowers accuracy to 45.1, compared with 48.2 after removing the same number of random selected blocks. The unmodified score in this setting is 63.5 (Table 2).
  • A SWE-Together trace illustrates the mechanism. In the sd-scripts task involving Lumina FP8 support, an earlier git diff lumina_util.py output ranked 40th among 91 history blocks. Score packing retained only 19, so it missed the diff. An earlier user question about FP8_OPTIMIZATION_TARGET_KEYS in Lumina2 had assigned 28% of its attention to the diff. Because that user question was protected, dependency expansion followed the stored edge and recovered the diff. At turn 6, the agent then made the requested SM 8.9+ correction in one successful edit without rereading the file. This shows how a link can restore a low-ranked supporting block that scalar ranking misses.
AuthorsOpenReview ↗

Response to Reviewer uPXN (Part 2)

[W1&Q1] Sensitivity to the attention threshold and to the importance and query weights.

We vary the query weight λ, the importance update rate α, and the dependency expansion allowance δ around their shared defaults. These parameters control how the method balances current relevance, historical attention, and supporting context.

  1. We change λ from 0.5 to 0.25 and 1.0 on LiC with gpt-oss and SWE-Together with Qwen3-Coder.
  2. We change the importance EMA rate α from 0.6 to 1.0. With α = 1.0, only the latest observed attention contributes to the updated score.
  3. We increase the one-hop expansion allowance δ from 0.1 to 0.2, alongside the separate no-expansion ablation in Table 1. This tests the influence of the expansion budget. The edge threshold θ determines which links are formed and is not varied in this experiment.

The results are as follows:

[Table 3] Parameter sensitivity with shared defaults λ = 0.5, α = 0.6, δ = 0.1

VariantBenchmarkDefaultVariantΔ
λ = 0.25LiC67.667.0+0.6
λ = 0.25SWE-Together28.529.4-0.9
λ = 1.0LiC67.666.8+0.8
λ = 1.0SWE-Together28.527.7+0.8
α = 1.0LiC67.664.4+3.2
α = 1.0SWE-Together28.529.9-1.4
δ = 0.2SWE-Together28.530.3-1.8

We define Δ as the default score minus the variant score. No tested alternative improves both benchmarks consistently. For example, λ = 0.25 and α = 1.0 improve SWE-Together reward but reduce LiC accuracy. Increasing δ to 0.2 raises SWE-Together reward from 28.5 to 30.3. The results show that shared defaults work across tasks, while a larger expansion budget can help on SWE-Together. These tests do not establish sensitivity to θ, which would require a direct threshold sweep.

[W2&Q3] Archive storage, retrieval I/O, and the end-to-end cost of attention collection.

The efficiency question concerns the complete system, not just cheap selection. We therefore measure archive storage and retrieval, the cost of obtaining attention, and request-path latency over real sessions. These tests show where the savings come from and which operations add overhead.

  1. We measure the size of the text archive, the time to read a recorded request, and the time to assemble selected context on SWE-Together (Table 4).
  2. We measure additional native attention collection cost and the latency of a separate 0.6B proxy pass at different context lengths (Table 5).
  3. We replay 30 real SWE-Together sessions with 280 user boundaries, measuring first-token latency, compaction, and post-response graph work (Table 6).

The results are as follows:

[Table 4] SWE-Together archive storage and retrieval cost

Cost
Text archive per session, median549 KB (1,265 KB)
KV cache for the same history, median12.2 GiB (28.0 GiB)
Read one recorded request0.80 ms
Assemble the selected context0.13 ms

[Table 5] Cost of collecting attention per request (H100, medians of 10 runs)

Context tokens1K4K16K32K65K
Prefill (Qwen3-Coder-30B)0.068 s0.182 s0.810 s2.078 s6.215 s
+ Attention collection+0.012 s+0.034 s+0.199 s+0.463 s+1.441 s
Extra GPU memory0.02 GiB0.29 GiB1.31 GiB2.58 GiB5.19 GiB
0.6B proxy pass instead—0.059 s0.315 s0.861 s2.656 s

[Table 6] Complete workflow on 30 SWE-Together sessions. Medians over 280 user boundaries

Full history, coldFull history, prefix cacheCodex-style summaryReCAP
First-token latency9.92 s0.95 s10.78 s0.93 s
Compaction time (included above)007.57 s0.016 s
Prompt tokens88,38488,38413,16719,920
Graph update after the response———1.20 s
AuthorsOpenReview ↗

Response to Reviewer uPXN (Part 3)

These measurements separate persistent storage, signal collection, and the benefit at a cache miss.

  • The text archive occupies a median 549 KB per SWE-Together session, compared with 12.2 GiB of KV cache for the same history. This archive is the transcript the agent harness already keeps, rather than an additional KV state. Reading a recorded request takes 0.80 ms, and assembling the selected context takes 0.13 ms (Table 4).
  • Native attention collection adds 18% to 25% to a same-length Qwen3-Coder prefill. A separate 0.6B proxy pass takes 0.059 to 2.656 s for contexts from 4K to 65K tokens (Table 5). These measurements distinguish the cost of obtaining attention from CPU-based context selection.
  • The session replay measures first-token latency of 0.93 s for RECAP, 9.92 s for cold full history, and 10.78 s for Codex summarization. Warm full history takes 0.95 s with a prefix cache (Table 6). RECAP selection takes 0.016 s, while 1.20 s of post-response graph work is reported separately. These direct measurements replace the paper's earlier synthetic cold estimate.

[W3&Q4] The quality–context trade-off is mixed; quality across matched context budgets, particularly against the summarization baseline.

We add two comparisons to test whether the quality advantage persists under tighter context budgets. The first compares RECAP with a budget-bounded summarizer at the same nominal retention fractions on LiC. The second tightens RECAP's budget on SWE-Together until its realized context is shorter than the Codex-style summary.

  1. On LiC, both methods use β values of 0.05, 0.10, and 0.20. This holds the nominal budget setting fixed while testing task quality across a range of retention levels (Table 7).
  2. On SWE-Together, we test a tighter RECAP configuration that retains 4,809 tokens per call, below the 5,239 used by Codex-style summarization (Table 8). This directly addresses the reviewer's concern about the higher context use of RECAP at its original default setting.

The results are as follows:

[Table 7] LiC accuracy for RECAP and budget-bounded summarization at shared nominal retention fractions

BackboneβReCAPSummaryΔ
gpt-oss0.0570.461.2+9.2
gpt-oss0.1067.657.2+10.4
gpt-oss0.2055.246.6+8.6
Qwen3-Coder0.0586.484.8+1.6
Qwen3-Coder0.1085.481.8+3.6
Qwen3-Coder0.2081.280.6+0.6

[Table 8] SWE-Together reward, realized context, and first-token latency with tighter RECAP retention

RewardContext tokensFirst-token latency
ReCAP31.94,8090.44 s
Codex-style summary28.75,2397.78 s

Together, the comparisons show that RECAP retains a quality advantage across nominal LiC budgets and under a tighter SWE-Together context cap.

  • RECAP outperforms budget-bounded summarization at all three nominal retention fractions on both LiC backbones. Its gains are 9.2, 10.4, and 8.6 accuracy points with gpt-oss, and 1.6, 3.6, and 0.6 points with Qwen3-Coder (Table 7).
  • On SWE-Together, the tighter RECAP setting reaches 31.9 reward with 4,809 context tokens, compared with 28.7 and 5,239 for Codex-style summarization. It also lowers measured first-token latency from 7.78 s to 0.44 s (Table 8). RECAP therefore performs better while using fewer tokens in this controlled setting.
  • In the paper's original default-budget comparison, summarization scored 28.7 against RECAP's 28.5 and used fewer context tokens. That comparison did not match realized context length. The new tighter-budget result addresses this directly. RECAP also avoids a summarization model call and can restore history that was omitted at an earlier turn.
Reviewer_o1NfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReCAP, a training-free memory compaction method that stores attention-derived importance scores and dependencies in a persistent graph. At each new user request, it combines these signals with identifier overlap to select historical context, including previously omitted messages. Experiments on two coding benchmarks show substantial improvements on Lost-in-Conversation and comparable or better SWE-Together performance with less context than full-history conditioning.

Critical strengths

The method addresses a practical problem in agent harnesses: preserving useful memory across cache eviction while allowing later requests to recover earlier context. Protecting user instructions and recent edits, preserving tool-call/result pairs, and supporting prefix caching make the design relevant to deployed agents.

The results across two model families are encouraging. The no-attention ablation provides evidence that stored attention adds value beyond protected records and lexical retrieval. The small-model proxy results also suggest that the approach could extend to agents without direct attention access.

Critical weaknesses

The contribution of dependency structure remains unclear. The main ablation removes importance scores, dependency expansion, and stale marks together, so it does not establish whether graph traversal improves over persistent attention scores alone. The no-attention variant already performs strongly on LiC, particularly with gpt-oss (63.4 versus 67.6). Separating these components would clarify how much the gains come from the graph versus the surrounding selection and protection rules.

The efficiency evidence is narrower than the headline reduction suggests. The approximately 95% improvement is an estimate for compaction plus cold restoration, using synthetic inputs and a prepared graph. CPU graph-update costs are reported, but the incremental inference cost of collecting attention is not clearly quantified, and proxy replay introduces additional computation. These omissions limit the conclusions about total serving efficiency.

Questions for authors

Q1. How does performance change when dependency expansion and stale marking are removed separately, retaining attention scores, protected records, and comparable context budgets? Results on SWE-Together would be especially informative.

Q2. What are the measured latency and memory overheads of attention collection during inference, and the total cost when using the proxy model? How much of this cost can reliably overlap with other work?

5 comments
AuthorsOpenReview ↗

Response to Reviewer o1Nf (Part 1)

We thank the reviewer for the insightful assessment and for recognizing the practical motivation for RECAP and its encouraging results across two model families. We address the component contributions and the full cost of attention collection below.

[W1&Q1] The contribution of dependency structure is unclear; performance when dependency expansion and stale marking are removed separately, especially on SWE-Together.

The original no-graph ablation removed importance, dependency expansion, and stale handling together. We now isolate their contributions and examine why the earlier no-attention variant remained competitive on LiC with gpt-oss. The two new analyses answer these questions in different ways.

  1. We remove each component from an otherwise unchanged system. This measures its effect on SWE-Together reward with Qwen3-Coder-30B-A3B and LiC accuracy with gpt-oss-20b (Table 1). The protected set includes historical user messages and the latest edit or write record for each modified file. Importance scores accumulate received attention through an exponential moving average.
  2. We replay recorded LiC histories with attention-derived importance disabled. We do this once with mandatory protection and once without it (Table 2). Because the history is fixed, this test isolates how much importance changes context selection before differences in agent execution can accumulate.

The results are as follows:

[Table 1] Isolated component ablations on SWE-Together and LiC. SWE-Together uses reward and LiC uses accuracy, both scaled by 100. Δ = With − Without.

ComponentBenchmarkWithWithoutΔ
Protected recordsSWE-Together28.525.2+3.3
Attention-derived importanceSWE-Together28.523.6+4.9
Stale handlingSWE-Together28.526.2+2.3
Dependency expansionSWE-Together28.526.4+2.1
Protected recordsLiC67.663.5+4.1
Attention-derived importanceLiC67.658.7+8.9
Dependency expansionLiC67.665.3+2.3

[Table 2] Changes in LiC context selection when importance is disabled, with recorded histories held fixed

Full systemScore-only
Requests whose selection changes2.4%18%
Share of the selected text that changes15%71%

The two analyses show why the individual components matter and why protection changes how much ranking is needed.

  • Removing importance lowers SWE-Together reward by 4.9 points and LiC accuracy by 8.9 points. Removing dependency expansion lowers them by 2.1 and 2.3 points. Stale handling adds 2.3 points on SWE-Together, where superseded history can distract the agent (Table 1). The components therefore make separate contributions beyond the earlier combined ablation.
  • Protected records also matter. Retaining user instructions and the latest edits contributes 3.3 points on SWE-Together and 4.1 points on LiC (Table 1). These rules keep essential blocks without making them compete with ordinary history for a high ranking.
  • The offline replay explains the role of protection on short LiC histories. With protected records, disabling importance changes the selected set in only 2.4% of requests. Without protection, the fraction rises to 18%, and the reported share of changed selected text rises from 15% to 71% (Table 2). The earlier no-attention variant removed several graph components together, so its 63.4 score is not the same experiment as the isolated importance ablation. The replay measures selection on fixed histories, while the online ablation allows later behavior to diverge. A few changes at important turns can therefore have larger effects on task accuracy than their frequency alone suggests.
AuthorsOpenReview ↗

Response to Reviewer o1Nf (Part 2)

[W2&Q2] Latency and memory overhead of attention collection, total cost with the proxy model, and how much of it overlaps with other work.

The paper's approximately 95% latency estimate used synthetic cold prompts and a prepared graph. We now report a replay of real session boundaries and measure native attention collection and proxy replay separately. This directly addresses the cost of obtaining the attention signal as well as the benefit of compact restoration.

  1. We measure the additional latency and GPU memory of native attention collection during Qwen3-Coder prefill over different context lengths. We separately time a 0.6B proxy pass, which provides an alternative when the serving model does not expose attention (Table 3).
  2. We replay 30 SWE-Together sessions with 280 user boundaries. We measure first-token latency, prompt size, and compaction time for cold full history, cached full history, Codex summarization, and RECAP. We report the graph work after each response separately (Table 4).

The results are as follows:

[Table 3] Native attention collection and proxy replay costs on one H100. Medians over 10 runs

Context tokens1K4K16K32K65K
Prefill (Qwen3-Coder-30B)0.068 s0.182 s0.810 s2.078 s6.215 s
+ Attention collection+0.012 s+0.034 s+0.199 s+0.463 s+1.441 s
Extra GPU memory0.02 GiB0.29 GiB1.31 GiB2.58 GiB5.19 GiB
0.6B proxy pass instead—0.059 s0.315 s0.861 s2.656 s

For gpt-oss-20b, native collection adds 0.105 / 0.298 / 0.490 s at 1K / 4K / 8K tokens.

[Table 4] Complete workflow on 30 SWE-Together sessions. Medians over 280 user boundaries

Full history, coldFull history, prefix cacheCodex-style summaryReCAP
First-token latency9.92 s0.95 s10.78 s0.93 s
Compaction time (included above)007.57 s0.016 s
Prompt tokens88,38488,38413,16719,920
Graph update after the response———1.20 s

These measurements separate the cost of obtaining attention from the latency saved at a cold boundary.

  • Native collection adds 18% to 25% to a Qwen3-Coder prefill of the same length. It requires 0.02 to 5.19 GiB of additional GPU memory over the tested lengths (Table 3). These costs are reported rather than assumed to be free.
  • A separate 0.6B proxy pass takes 0.059 to 2.656 s for contexts from 4K to 65K tokens. It avoids requiring attention access from the serving model, but the proxy itself still consumes computation and memory on its execution device (Table 3).
  • In the session replay, query-time selection takes 0.016 s. Graph work after the response takes 1.20 s and is measured separately from first-token latency (Table 4). This post-response work can overlap later activity, but the amount of overlap depends on the serving schedule.
  • On the measured boundaries, RECAP reaches the first token in 0.93 s. Cold full history takes 9.92 s and Codex summarization takes 10.78 s (Table 4). A warm full-history prefix takes 0.95 s. This is why RECAP complements prefix caching and is most valuable after cache expiration.

We thank the reviewer again for the thoughtful feedback and would be happy to address any further questions.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer o1Nf,

Thank you again for your careful evaluation of our work. We have provided detailed responses and additional experiments addressing the concerns raised in your reviews. As the discussion period is approaching its end, we would greatly appreciate it if you could review our rebuttal and let us know whether it addresses your concerns or if any further clarification would be helpful.

Thank you for your time and consideration!

Reviewer_M84COpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReCAP maintains a persistent archive and uses attention-derived importance and graph edges to reconstruct a smaller active context for coding agents. Selection combines historical usage with current-query relevance, protects designated blocks, and expands to supporting neighbors. The idea is to make context reduction reversible while retaining information that isolated relevance scores might omit.

Critical strengths
  • Combining past usage with current-query relevance is a sensible response to changing information needs across agent turns. Keeping the archive separate from the active context also avoids making an irreversible summary the sole record of earlier work.

  • The no-attention-graph ablation retains the archive, protected blocks, and lexical selection, yet drops performance from 85.4 to 76.6 at similar token reduction. This is meaningful evidence that the attention-derived machinery contributes beyond archival storage alone.

Critical weaknesses
  • The evidence does not yet isolate the graph as the source of that benefit. The no-graph ablation removes importance, edges, and staleness-related machinery together. The attention analyses show useful relevance signals, but do not establish that dependency expansion recovers supporting context that a scalar importance selector would miss. This distinction matters: the paper's distinctive methodological claim is graph-based reconstruction, not simply attention-informed block ranking.

  • The efficiency claim needs to be evaluated at the level of the proposed architecture. The reported large latency reduction uses prepared graphs and a cold synthetic setup; it does not establish the end-to-end cost once attention collection is included. Background CPU updates and prefix-cache experiments address parts of this issue, but attention collection or proxy replay can change the practical tradeoff. The question is whether graph construction pays for itself over a realistic agent session, not whether selection from an already-built graph is cheap.

Questions for authors
  1. Can you compare an importance-only selector against the same selector plus graph expansion, preserving the archive, protected blocks, staleness treatment, and actual context-token budget? An analysis of cases rescued specifically by low-scoring supporting neighbors would directly validate the graph mechanism.

  2. What is the end-to-end latency over complete sessions with attention collection enabled, including graph maintenance and any proxy replay? Please separate cold and cache-reusing regimes so that the benefit of smaller context can be weighed against the cost of maintaining its selection signal.

6 comments
AuthorsOpenReview ↗

Response to Reviewer M84C (Part 1)

We thank the reviewer for the thoughtful assessment and for recognizing the motivation behind RECAP's design. We address the concerns with targeted ablations, a concrete recovery case, and measurements of the full workflow.

[W1&Q1] The evidence does not yet isolate the graph as the source of the benefit; comparison with an importance-only selector.

The original no-graph ablation removed importance, dependency expansion, and stale handling together. It could not isolate the contribution of dependency edges. We now test expansion both with the full protection rules and with mandatory protection disabled. This separates the value of recovering supporting context from the value of keeping essential records.

  1. We compare the same importance-based selector with and without one-hop dependency expansion. We disable mandatory protection but retain the archive, importance scores, stale handling, and nominal budget. We also report realized context use to separate the quality gain from the additional tokens introduced by expansion.
  2. We remove each component from the full system in turn (Table 2). All other mechanisms remain in place. This shows whether dependency expansion still helps when user instructions and recent edits are protected.
  • Protected records retain all historical user messages and the latest edit or write record for each modified file.
  • Attention-derived importance accumulates the attention received by each block through an exponential moving average.
  • Stale handling downweights earlier blocks that substantially overlap with later, more strongly attended content. It also excludes stale blocks from dependency expansion.
  • Dependency expansion follows stored attention links from initially selected blocks to their earlier supporting blocks, subject to the expansion budget.
  • We test whether linked parents are useful rather than simply extra text. We remove graph-linked parents or the same number of randomly chosen selected blocks. In both cases, we refill the freed budget with the next highest scoring blocks (Table 3).

We report SWE-Together reward with Qwen3-Coder-30B-A3B and LiC accuracy with gpt-oss-20b. Both scores are scaled by 100. Each Δ is the score with the component minus the score without it.

[Table 1] Dependency expansion without mandatory protection. Other mechanisms and the nominal budget are retained. SWE-Together context tokens are 12,540 with expansion and 11,980 without.

BenchmarkWith expansionWithoutΔ
SWE-Together25.223.0+2.2
LiC63.560.6+2.9

[Table 2] Isolated component ablations with the remaining mechanisms retained

ComponentBenchmarkWithWithoutΔ
Protected recordsSWE-Together28.525.2+3.3
Protected recordsLiC67.663.5+4.1
Attention-derived importanceSWE-Together28.523.6+4.9
Attention-derived importanceLiC67.658.7+8.9
Stale handlingSWE-Together28.526.2+2.3
Dependency expansionSWE-Together28.526.4+2.1
Dependency expansionLiC67.665.3+2.3

Stale handling is not triggered by the LiC histories in these runs, so we report this ablation only on SWE-Together.

[Table 3] Removing graph-linked parents versus random selected blocks on LiC

Nothing removedRandom blocks removedParents removed
Accuracy67.648.245.1
AuthorsOpenReview ↗

Response to Reviewer M84C (Part 2)

These experiments separate the contributions of ranking and dependency recovery.

  • Persistent importance is the largest isolated contributor in these ablations. Removing it lowers SWE-Together reward by 4.9 points and LiC accuracy by 8.9 points (Table 2). This shows that saved attention remains useful across turns.
  • Dependency expansion improves SWE-Together reward from 23.0 to 25.2 and LiC accuracy from 60.6 to 63.5 when mandatory protection is disabled (Table 1). On SWE-Together, it also adds 560 context tokens, from 11,980 to 12,540, a 4.7% increase. Thus, this comparison measures the gain from dependency-guided additions under the same nominal budget. With protection enabled, the gains remain 2.1 and 2.3 points (Table 2). The parent-removal test further checks whether the linked blocks are useful, rather than treating any additional context as equally valuable.
  • The removal test asks whether the linked parents are especially useful. Removing them reduces LiC accuracy from 67.6 to 45.1, compared with 48.2 after removing the same number of random selected blocks and refilling the budget (Table 3). The difference is 3.1 points. Random removal also hits a parent in about 73% of draws, so the two groups are not fully disjoint.

A SWE-Together trace illustrates how the graph recovers a block missed by ranking. In the sd-scripts task on Lumina FP8 support at turn 6, an earlier git diff lumina_util.py output ranked 40th among 91 historical blocks. Score-based packing selected only the first 19, so the diff was excluded. An earlier user question about whether FP8_OPTIMIZATION_TARGET_KEYS exists in Lumina2 had assigned 28% of its attention to this diff. The edge from the diff to the question was stored in the graph. Because the user question was protected, one-hop expansion followed the edge and restored the diff. The agent then corrected the SM 8.9+ check in one successful edit without rereading the file.

Together, these results establish two complementary roles. Persistent importance ranks historical evidence, while dependency edges recover lower-ranked supporting blocks when they are needed. We will make this distinction explicit in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer M84C (Part 3)

[W2&Q2] The efficiency claim should be evaluated end to end, including attention collection, in both cold and cache-reusing regimes.

The paper's latency figure combined separately measured compaction and synthetic cold restoration with a prepared graph. We now measure actual user boundaries and the overhead of obtaining attention. These experiments distinguish request-path latency from the cost of maintaining the selection signal.

  1. We replay 30 SWE-Together sessions with 280 user boundaries on one H100. We measure first-token latency for cold full history, cached full history, Codex-style summarization, and RECAP. We also report compaction and post-response graph work separately (Table 4).
  2. We measure the extra time and GPU memory needed to collect attention during Qwen3-Coder prefill from 1K to 65K tokens. We time a separate 0.6B proxy pass over the same input lengths where supported (Table 5). This isolates the cost of producing attention from the cost of querying the graph.

The results are as follows:

[Table 4] Complete workflow on 30 SWE-Together sessions. Medians over 280 user boundaries

Full history, coldFull history, prefix cacheCodex-style summaryReCAP
First-token latency9.92 s0.95 s10.78 s0.93 s
Compaction time (included above)007.57 s0.016 s
Prompt tokens88,38488,38413,16719,920
Graph update after the response———1.20 s

The 1.20 s post-response graph work is reported separately from first-token latency. It can overlap later activity, but the amount of overlap depends on the serving schedule.

[Table 5] Cost of collecting attention per request (H100, medians of 10 runs)

Context tokens1K4K16K32K65K
Prefill (Qwen3-Coder-30B)0.068 s0.182 s0.810 s2.078 s6.215 s
+ Attention collection+0.012 s+0.034 s+0.199 s+0.463 s+1.441 s
Extra GPU memory0.02 GiB0.29 GiB1.31 GiB2.58 GiB5.19 GiB
0.6B proxy pass instead—0.059 s0.315 s0.861 s2.656 s

The results distinguish cold restoration, cache reuse, and attention overhead.

  • At cold boundaries, RECAP reaches the first token in 0.93 s, compared with 9.92 s for full-history restoration and 10.78 s for Codex-style summarization (Table 4). This is a 90.6% reduction relative to cold full history. RECAP spends 0.016 s on request-time selection, while the summarizer spends 7.57 s on compaction. These values come from replayed session boundaries rather than synthetic prompt lengths.
  • A warm full-history prefix is already fast at 0.95 s (Table 4). RECAP complements prefix caching rather than replacing it. As described in Section 5.5, a warm turn can reuse its cached prefix, while graph-guided selection is most useful after cache expiration. RECAP does not need to keep the full historical KV state resident during an idle gap.
  • Native attention collection adds 18% to 25% to a same-length Qwen3-Coder prefill and uses 0.02 to 5.19 GiB of extra GPU memory over the tested lengths (Table 5). A separate 0.6B proxy pass takes 0.059 to 2.656 s from 4K to 65K tokens. The replay reports 1.20 s of graph work after the response. This work may overlap later activity, but we report it explicitly instead of treating signal maintenance as free.

We will update the latency analysis with these session replay measurements.

We thank the reviewer again for the thoughtful feedback and would be happy to address any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal and Discussion Summary

We thank the Area Chair and reviewers for their careful assessment and constructive feedback. The reviewers recognized RECAP's practical motivation, strong results, and efficient context selection. Their main concerns focused on the contribution of dependency structure, end-to-end efficiency, and the quality–context trade-off.

During rebuttal, we conducted substantial additional experiments to address these concerns.

(1) Component ablations isolate the contributions of attention-derived importance (+4.9 SWE / +8.9 LiC) and dependency expansion (+2.1 SWE / +2.3 LiC). Additional parent-removal experiments and a concrete recovery case demonstrate how dependency links recover useful context missed by importance-only ranking.

(2) End-to-end efficiency measurements on 30 real SWE-Together sessions (280 user boundaries) show 0.93 s first-token latency for RECAP, compared with 9.92 s for full-history restoration and 10.78 s for Codex summarization (86.3% latency reduction). We also measured attention collection, proxy replay, and graph maintenance overhead.

(3) Controlled-budget comparisons show that RECAP outperforms summarization across all tested LiC budgets with both backbones. On SWE-Together, RECAP achieves 31.9 reward with 4,809 tokens, compared with 28.7 reward with 5,239 tokens for summarization.

We additionally evaluated two related-work baselines, archive storage and retrieval costs, and parameter sensitivity.

Together, these results strengthen the evidence for RECAP's graph-based context compaction, superior quality–efficiency trade-off, and practical effectiveness in long-horizon agents. We hope the Area Chair will consider this additional evidence in the final assessment.

We thank the Area Chair and reviewers again for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.