acceptodds
Desk rejected at ICLR 2027

Hidden, Not Erased: Attention-Side Access Control for Compacted Agent Memory

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Compacting an agent's trajectory is usually treated as deletion: once older interactions are summarized, their exact contents stop influencing the model, even though they still exist in an archived tool result. This conflates two separable decisions, storage and visibility. We make visibility the object of control. Triggered Sparse Attention keeps raw pre-compaction spans archived and ineligible by default. A trigger decides whether they may be attended to at all; the model's own sparse indexer then chooses which hidden tokens to read, independently for each layer and decoding position. In the model-native form, opened evidence is never written back into the prompt. On Qwen3-4B with a native sparse indexer, in 1,000 fresh synthetic agent traces whose queried fact survives only in the archive, triggered opening answers 342 while a closed archive and random opening at a higher rate answer none, and with sixteen archived events it answers 123 against 14 for random opening at exactly its opening rate. Reusing DeepSeek-V3.2's pretrained sparse indexer without retraining hidden-token identity, archive access recovers 21 of 26 hidden details where summary-only memory recovers none. In two pre-registered replications on 512 fresh paired WebShop tasks each, with byte-identical prompts across arms, selective recall beats random access at similar archive volume (157 against 134, p=0.015; 163 against 132, p<0.001) and raises mean reward over indiscriminate access at 2.7 times less archive volume.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language agents, long-term memory, context compaction, sparse attention, memory retrieval, LLM agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ze82OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

LLM architecture-related & context management paper. This paper studies post-compaction memory access for long-horizon language agents. Its central observation is that trajectory compaction conflates storage with visibility: old raw interactions may still exist in an archive even though they are no longer available to the model's attention.

The paper introduces Triggered Sparse Attention (TSA), which separates these two decisions:

  • A trigger determines whether archived tokens are eligible to participate in attention.
  • A sparse-attention indexer then performs fine-grained token selection, separately for each layer and query. In the model-native setting, retrieved evidence does not need to be reinserted into the prompt.

The paper evaluates the idea along several complementary axes:

  • native sparse-attention probes on synthetic traces, where query-conditioned opening is compared with closed and random archive access, including a rate-matched random control;
  • two pre-registered WebShop waves, where selective access is compared with random and indiscriminate archive exposure at similar or lower archive volume;
  • a frozen DeepSeek-V3.2 sparse indexer, showing that archived evidence can be recovered without retraining token-identity selection.

Overall, the main contribution is a useful reframing of compacted agent memory as an attention-side eligibility and control problem, rather than only a storage, summarization, or prompt-retrieval problem. The empirical validation includes matched controls, pre-registration, paired statistical testing, and separate mechanism-level and agent-level studies.

Critical strengths
  1. A clean and potentially important reframing of compacted agent memory.
  • The paper separates memory storage from memory visibility and formulates post-compaction recall as an attention-eligibility problem (§1, §3).
  • This leads to a conceptually clean two-stage design. A trigger decides whether archived memory may participate at all, and the model's own sparse indexer decides which individual tokens to read (§4, Eqs. 2–3).
  • This matters because it provides a memory interface that differs from both summary-only systems and prompt-based retrieval, and it may generalize beyond the particular implementation studied here (§2; §7).
  1. Strong mechanism-oriented controls and unusually careful empirical methodology.
  • The paper does more than compare against a closed-memory baseline. It also includes random access, rate-matched random access, always-open controls, fixed-total attention budgets, matched-content controls, paired task evaluation, and pre-registered WebShop replications (§5; App. C).
  • The native sparse probe shows a large gap between Triggered-SA and random opening, even at matched opening rates. This supports the claim that where archive access occurs matters, not merely how much memory is exposed (§6, Table 1).
  • Matched-content controls show that the effect is carried by the archived content rather than by extra attention mass (§6, l.372–375).
  • The two WebShop waves reproduce the ordering of selective over random access under matched tasks and controlled prompts (§6, Fig. 3; App. Table 7).
Critical weaknesses
  1. The full proposed system is not demonstrated end-to-end in a live agent.
  • The model-native experiments use Qwen3-4B with native sparse reading, but on synthetic traces and with a training-free score-margin trigger.
  • The large WebShop agent study instead evaluates a learned access decision by restoring opened raw observations into the model context. It does not use the proposed native attention-side reading path (l.320; App. Table 6).
  • The paper therefore provides strong evidence for each component separately. It does not directly validate the complete combination of a learned access gate, native sparse archive reading, and long-horizon agent execution at scale.
  • This is consequential because the paper's main architectural contribution is precisely the attention-side interface, and the current agent-scale results do not instantiate it.
  1. The comparison to strong learned retrieval or prompt-memory systems is not yet sufficient to establish a broader systems advantage.
  • The random, closed, and always-open controls are excellent for isolating the causal role of selective archive access. However, they are weaker competitors than a well-tuned learned retriever that selectively reinserts archived evidence under a comparable token, latency, and storage budget.
  • The paper does report a matched-evidence comparison, in which attention-side admission yields a larger action-likelihood margin than prompt insertion (§6, l.374–375). This is narrower than an end-to-end comparison against a strong retrieval-based memory policy.
  • This limits how strongly the results can be read as showing superiority over alternative long-term memory interfaces. At present they show that the proposed interface is viable and promising.
Questions for authors

Main questions:

Q1. What is the expected performance of the full proposed configuration, a learned NULL/OPEN gate combined with native attention-side sparse archive reading in a live agent? App. Table 6 indicates that the WebShop study restores opened observations to the context, whereas the native Qwen probe uses sparse top-k archive reading. Is there any existing experiment, even at smaller scale, that combines the learned gate from the agent setting with the native sparse reading path? Evidence that this full composition preserves the reported agent-level gains would materially strengthen my assessment.

Q2. How should the WebShop results be interpreted relative to the Visible-Only baseline? In the pooled WebShop evaluation, Triggered-SA improves mean reward substantially over random and always-open access, but only slightly over Visible-Only (0.171 vs. 0.168). The nonzero-reward count increases from 300 to 320 (App. Table 7). Which of these is the intended main claim?

  • that Triggered-SA improves absolute agent capability over well-compacted memory; or
  • that it mainly avoids the degradation caused by indiscriminate access while selectively recovering useful archived information? Clarifying this would affect how I assess the empirical significance.

Q3. Do you have a comparison against a learned prompt-retrieval baseline under approximately matched archive-selection and token budgets? The current random and always-open controls isolate selective access well. I would like to know whether the advantage remains against a learned retriever that selects one or more archived records and reinserts them into the prompt. This would materially affect whether I see the contribution mainly as a new, viable memory interface or as a practically superior alternative to prompt-based retrieval.

Minor questions:

  • Title layout: The title breaks at an odd point on p.1. "Control" sits alone on the second line, followed by a blank line before "for Compacted Agent Memory" (l.001–004). Please reflow it, e.g., with a manual line break after "Hidden, Not Erased:".
  • Figure size in the experiments: Figs. 2–4 (pp.7–8, l.337–417) are taller than their content needs and contain a lot of white space. Figs. 3 and 4 alone take most of p.8, and the legend in Fig. 4(e) overlaps the curves. Making these figures shorter and more compact, e.g., by reducing panel height or merging related panels into a single row, would free space. That space could bring key appendix material into the main text, such as App. Table 6, which summarizes what each experiment executes, or a fuller discussion of the WebShop and DeepSeek-V3.2 results.
6 comments
AuthorsOpenReview ↗

Response to Reviewer Ze82 (Part 1/5)

We thank Reviewer Ze82 for the supportive assessment, for highlighting our separation of memory storage from memory visibility, and for the layout suggestions. We ran seven experimental and kernel-level systems suites on H200 and incorporated all suggested presentation updates into the revised manuscript (9 main pages, Limitations starting on p. 10, no layout overflows). Notation used below: WCT denotes the working-context token budget that triggers compaction; expanded_l2 denotes raw archived (L2) text re-injected into the active prompt (0.0 on all native arms, vs. 0.87 for non-compacting flat snippet retrieval); R1–R4 denote the four prompt-injection formats of the text-retrieval baselines (R1 trailing user message, R2 inside the tool observation, R3 synthetic tool turn, R4 in-place history splice); and Suite A / B / C denote our 64-state archive-detail, stale-override, and visible-sufficient post-compaction suites.

Summary of Revisions

  1. W1 & Q1: Full live configuration (learned NULL/OPEN gate + native sparse reading, no prompt restoration) (PDF: §5 Table 1 p. 5, §6.2–§6.3 pp. 7–9, Limitations p. 10, App. D.6 & D.8 Tables 13–14, 17, Fig. 5a pp. 20–21, 23, App. E Tables 23–24 pp. 26–27): (1) Live 256-task WebShop (Qwen3-4B + native DSA, homogeneous expanded_l2 = 0.0): at WCT = 1600 (49 compacted), Visible-Only completes 1/49 = 2.0% (0.0122 reward) while Learned-router DSA (13.7% compacted trigger rate, 5.8× < Always-open 79.6%) completes 11/49 = 22.4% (+20.4 pp, 11W/1L, exact McNemar p=0.00635p = 0.00635; reward 0.0687, Wilcoxon p=0.0269p = 0.0269, −2.22 steps), raising 256-task reward to 0.3628 vs. 0.3585 Matched-random (+0.0042, 4W/0L); at WCT = 2800 (46 compacted), compacted R>0 rises 4/46 (8.7%) → 9/46 (19.6%, reward 0.0289 → 0.0697). (2) Live 128-cell loop: 8/32 vs. 0/32 Visible-Only (p=0.0078p = 0.0078; 8/8 config_merge). (3) 192-state Pareto & latency: 74.48% (143/192) (−53.1% Suite C false opens), running 133.2× faster in the eager harness and 1.39×–2.90× (GPU) / 1.41×–2.81× (PCIe5) faster than hook-free FA2 R4 re-prefill at 12K–32K (Table 24).
  2. Q2: Interpreting WebShop vs. Visible-Only (Claim (1) capability gain vs. Claim (2) avoiding recall harm) (PDF: Abstract p. 1, §1 p. 2, §6.2 & Fig. 3 pp. 7–8, §7 p. 9, App. D.3–D.4 & D.6 Tables 8–11, 14 pp. 18–19, 21): Claim (2) holds on zero-shot Qwen3-30B-A3B WebShop (n = 1,024 paired tasks): where summaries preserve coarse product cues, Triggered-SA avoids unselective-recall degradation (+0.027 over matched-volume Random, which reaches 266 R>0 vs. 300 for Visible-Only, p=0.0045p = 0.0045; +0.017 over Always-Open at 2.7× lower volume) while preserving closed-memory reliability (0.171 vs. 0.168 reward; 320 vs. 300 R>0, 91W/71L, p=0.135p = 0.135). Claim (1) (Triggered-SA > Visible-Only) emerges with statistical significance when compaction drops needed details: live 256-task WebShop at WCT = 1600 (1/49 → 11/49, p=0.00635p = 0.00635), DeepSeek-V3.2 (0/26 → 21/26), synthetic probes (Table 2), and 192-state decisions (68/192 → 143/192).
  3. W2 & Q3: Learned and adaptive prompt-retrieval baselines (512-tok matched budget) (PDF: §6.1 & §6.3 pp. 6–9, App. D.5, D.7, D.8 Tables 12, 15–17, Fig. 5a–c pp. 20, 22–23): (1) 4×4 retriever × placement matrix (Suite A, N = 64): crossing BM25 (26.6% recall@512), MiniLM+BM25 (28.1%), 4B Qwen3-Embedding-4B (37.5%), and Oracle (50.0%) with R1–R4 gives 1.56% (R1–R3) and 10.94% (R4) for Qwen3-Embedding-4B vs. 23.44% (15/64; 15/16 FP32 / 16/16 BF16 on config_merge) for native Triggered-SA (p=0.0078p = 0.0078 vs. R4). (2) Matched-gate dense & FLARE (N = 128): 79/128 (61.7%) vs. 1–2/128 (p≤1.22×10−4p \le 1.22\times 10^{-4}); Qwen3-Embedding-4B R2 degrades Suite C (100% → 95.31%, 36.46% on 192 states). (3) 128-trace copy & 0.65K → 15.8K scaling: turn-latching (88–90/128 = 68.8%–70.3%) beats Qwen3-Embedding-4B top-1 (72/128 = 56.3%); Global top-k and Equal-slice TSA reach 81.3% (13/16) at 10.6K vs. 12.5% R2.
AuthorsOpenReview ↗

Response to Reviewer Ze82 (Part 2/5)

Summary of Revisions (continued)

  1. Presentation updates (Page 1 title break, Figs. 2–4 whitespace & Fig. 4e legend, App. Table 6 [now Main Table 1 / App. Table 7] in main text; PDF: Title p. 1, §5 Table 1 p. 5, Figs. 2–4 pp. 7–8, App. D.1 Table 7 p. 17): Reflowed p. 1 title across two balanced lines after Hidden, Not Erased:; compacted Figures 2–4 and separated the Figure 4(e) legend from the curves; promoted the experimental interface summary into Main §5 Table 1 (cross-referenced to App. D.1 Table 7) within 9 main pages.

W1 & Q1: Full Configuration (Learned NULL/OPEN Gate + Native Sparse Archive Reading) in a Live Agent

Q1: "What is the expected performance of the full proposed configuration, a learned NULL/OPEN gate combined with native attention-side sparse archive reading in a live agent? App. Table 6 [now Main Table 1 / App. Table 7] indicates that the WebShop study restores opened observations to the context, whereas the native Qwen probe uses sparse top-k archive reading. Is there any existing experiment, even at smaller scale, that combines the learned gate from the agent setting with the native sparse reading path?"

Main §5 Table 1 (p. 5) maps each experimental interface across backbones. To evaluate the fixed 8-feature learned gate composed directly with model-native DSA sparse reading, we ran four H200 evaluations on converted Qwen3-4B + native DSA + LoRA (top-k = 1024):

  1. Live 256-task WebShop with attention-side reading (§6.2 p. 8 & Appendix D.6 Table 14, p. 21): We ran the first 256 unselected official WebShop tasks across two working-context token budgets (WCT = 2800 and tighter WCT = 1600), comparing Visible-Only, Learned-router DSA, Native score-margin DSA, Always-open DSA, and Matched-random DSA (all homogeneously with expanded_l2 = 0.0 across all 256 tasks). As a non-compacting reference, flat prompt snippet retrieval re-injects raw L2 text into the active prompt (expanded_l2 = 0.87, 0 compactions; 0.4199 mean reward, 233/256 = 91.0% R>0 overall):
    • At tighter WCT = 1600 (49/256 tasks compacted, 8.47 compactions/task in Visible-Only): Visible-Only fails on 48/49 compacted tasks (1/49 = 2.0% R>0, 0.0122 reward, 15.35 steps). Learned-router DSA (triggering on only 13.7% of compacted steps, 5.8× sparser than Always-open 79.6%) lifts compacted R>0 11× to 11/49 (22.4%, +20.4 pp, 11W/1L, exact two-sided McNemar p=0.00635p = 0.00635) and compacted reward 5.6× to 0.0687 (paired Wilcoxon p=0.0269p = 0.0269, permutation p=0.0278p = 0.0278, 13.12 steps), matching Native score-margin DSA (11/49, 0.0687 at 53.1% trigger) and lifting 256-task overall reward to 0.3628 (200/256 = 78.1%) vs. 0.3608 (198/256) for Visible-Only and 0.3585 (199/256) for Matched-random DSA (+0.0042, 95% CI [+0.0004, +0.0099], 4W/0L).
    • At WCT = 2800 (46/256 tasks compacted, 5.30 compactions/task): Learned-router DSA (15.8% compacted trigger rate) lifts compacted R > 0 from 4/46 (8.7%) to 9/46 (19.6%, 7W/2L) and compacted reward from 0.0289 to 0.0697 (Wilcoxon p=0.0664p = 0.0664; 0.0749 for Score-margin DSA, steps 15.43 → 14.09) while keeping 203/256 (79.3%, 0.3626) overall performance.
  2. 128-cell live synthetic multi-step agent loop (32 tasks × 4 policies, 964 generations; §6.2 pp. 7–8 & Appendix D.6 Table 13, p. 20): Coupling the learned 8-feature gate (τg=0.005\tau_g = 0.005) with native DSA across four multi-step tool families (config_merge, relational_inventory, runbook_dependency, weighted_route; 8 tasks each, ITT scoring) solves 8/32 (25.0%, 8/8 on config_merge, 0 format errors) for both Learned gate + native DSA and Always-open native DSA vs. 0/32 for Visible-Only (p=0.0078p = 0.0078, Holm p=0.0234p = 0.0234) and 5/32 (15.6%) for semantic prompt retrieval (p=0.25p = 0.25).
AuthorsOpenReview ↗

Response to Reviewer Ze82 (Part 3/5)

  1. 192-state balanced 3-suite Gate Pareto (64 archive-detail + 64 stale-override + 64 visible-sufficient; §6.3 pp. 8–9 & Appendix D.8 Table 17, Fig. 5a, p. 23): Testing false-open cost on Suite C (64 visible-sufficient states paired with multi-episode archived distractors), the fixed 8-feature gate at τg=0.005\tau_g = 0.005 reaches 74.48% (143/192: 23.44% A [15/16 FP32, 16/16 BF16 on config_merge], 100.0% B, 100.0% C) at 82.29% open rate. It opens on 46.88% of Suite C states, cutting unnecessary openings by 53.1% with zero accuracy loss and beating matched-rate Random (67.56%, +6.92 pp), 0-parameter DSA tail-margin (69.27% at 79.69% open, +5.21 pp), and Qwen3-Embedding-4B R2 (36.46%). At τg=0.01\tau_g = 0.01, it cuts Suite C false openings to 0.0% (50.0% overall open rate, +4.95 pp over Random).
  2. BF16 Eq. 2 invariance, DynamicCache fork parity, and hook-free FlashAttention-2 + PCIe Gen5 latency (Tables 17, 23–24, pp. 23, 26–27): In BF16 on H200, closed-gate execution matches a no-archive run on 32/32 (100.0%) greedy top-1 tokens (max |Δ| = 0.0 bitwise equality across 39 FP32 DynamicCache fork forwards in Table 23), running 133.2× faster than R4 re-prefill in the 12K eager diagnostic harness (695.8 ms Open / 308.8 ms Closed vs. 92.7 s). In a hook-free merged-BF16 FlashAttention-2 benchmark (WCT = 2,048, 64-tok obs. prefill + 32-tok decode, Table 24), turn-latched Open takes 816.9 ms at 12K (1.39×) and 812.9 ms at 32K (2.90×) vs. 1,135.6 ms and 2,358.3 ms for R4 FA2 re-prefill, while PCIe Gen5 selective top-k = 1024 host staging (9 KiB/tok HBM index) takes 803.6 ms (1.41×) and 839.2 ms (2.81×) (12.8 s vs. 37.9 s over a 16-turn 32K episode, 2.97×).

Scope (§6.2 and Limitations, p. 10): The composed learned gate + native DSA path operates without prompt restoration, cuts compacted-task WebShop openings 5.8× below Always-open and Suite C false openings by 53.1%, and significantly recovers compacted details (11/49 vs. 1/49 on 256-task WebShop at WCT = 1600, p=0.00635p = 0.00635; 8/8 live config_merge; 64/64 Suite B). Scaling live multi-hop chains to larger native backbones is a natural next step.


Q2: Interpreting WebShop Relative to Visible-Only (0.171 vs. 0.168, 320 vs. 300)

Q2: "How should the WebShop results be interpreted relative to the Visible-Only baseline? ... Which of these is the intended main claim: (1) that Triggered-SA improves absolute agent capability over well-compacted memory; or (2) that it mainly avoids the degradation caused by indiscriminate access while selectively recovering useful archived information?"

Our main claim on the 1,024-task Qwen3-30B-A3B WebShop study is (2): where summaries keep coarse cues, selective access avoids unselective-recall degradation while preserving closed-memory reliability. Claim (1) holds with clear statistical significance when compaction tightens or drops decisive identifiers, as shown in our live 256-task WebShop budget sweep (WCT = 2800 vs. WCT = 1600; Abstract p. 1, §1 p. 2, §6.2 pp. 7–8, Appendix D Tables 8–11 & 14):

  • Claim (2) when summaries retain coarse cues (Qwen3-30B-A3B, n = 1,024, Tables 8–11): Visible-Only reaches 0.168 mean reward (300/1,024 R>0). Triggered-SA (0.171, 320 R>0) preserves closed-memory reliability (+0.003, 95% CI [−0.010, +0.016], 91W/71L, exact McNemar p=0.135p = 0.135) while eliminating unselective-recall loss: Random drops to 266 R>0 vs. 300 for Visible-Only (p=0.0045p = 0.0045), whereas Triggered-SA gains +0.027 over Random (p=3.00×10−5p = 3.00\times 10^{-5}) and +0.017 over Always-Open (291 R>0, p=0.0215p = 0.0215) at 2.7× fewer expansions/step.
  • Claim (1) when compaction drops decisive details or tightens working memory (Tables 2, 13–14, 17, 19): Under tighter WebShop working memory on 256 tasks (WCT = 1600, Table 14), Visible-Only completes only 1/49 (2.0%) compacted tasks whereas native Triggered-SA completes 11/49 (22.4%, p=0.00635p = 0.00635, reward 0.0122 → 0.0687, Wilcoxon p=0.0269p = 0.0269; and 4/46 → 9/46 at WCT = 2800). Consistent gains hold on DeepSeek-V3.2 (0/26 → 21/26, p=9.54×10−7p = 9.54\times 10^{-7}), synthetic probes (Table 2), live 4B config_merge (0/8 → 8/8, p=0.0078p = 0.0078), and 192-state decisions (68/192 → 143/192).
Reviewer_nhb8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

When an agent harness compacts its trajectory, older raw interactions are summarized and archived. The model can then no longer attend to their exact contents. The paper argues that compaction conflates storage with visibility, and proposes Triggered Sparse Attention (Triggered-SA):

  • Archive. Raw pre-compaction spans, with their K/V states and sparse-index keys, are kept in an archive that is ineligible for attention by default.
  • Gate. A NULL/OPEN gate decides whether the archive may enter the candidate set of a DeepSeek-style sparse attention (DSA) indexer. It can be a training-free score-margin rule or a head trained on same-history counterfactual pairs.
  • Indexer. The indexer then chooses which hidden tokens to read, separately for each layer and decoding position. Opened evidence is never written back into the prompt.

The paper defines two attention "contracts": a fixed-total top-k, and a visible-preserving hidden sidecar.

Evidence comes from three separate setups:

  1. Native probe. Qwen3-4B, converted to DSA by distillation, on synthetic 20-event tool logs where a code survives only in the archive:
  2. One archived event: Triggered-SA copies the code in 342 of 1,000 traces. A closed archive and random per-position opening (54% of positions) copy it in 0. Full access copies it in 658.
  3. Sixteen archived events: 123 of 300, against 14 for random opening at the trigger's exact rate.
  4. WebShop. Qwen3-30B-A3B, where opened records are restored as text to the prompt. Two pre-registered waves of 512 tasks each:
  5. Against random opening, nonzero-reward tasks rise from 134 to 157 and from 132 to 163.
  6. Mean reward is +0.027 over random and +0.017 over opening every record, at 2.7× fewer expansions.
  7. DeepSeek-V3.2 with its frozen native indexer.
  8. A 26-query detail suite: 21 recovered vs 0 for visible-only.
  9. A 4-task transfer suite.
  10. An 897-parameter gate head.
Critical strengths

S1. Careful, well-controlled experimental methodology.

  • The WebShop arms receive byte-identical prompts and take identical actions until the first archive opening, verified by audit (App. D).
  • Endpoint contrasts were registered before outputs existed.
  • Paired outcomes use exact McNemar tests, and reward differences use paired bootstrap.
  • The random controls are probability- and rate-matched. With sixteen archived events, random opening at the trigger's exact rate selects 917 hidden tokens against the trigger's 918, so the 123 vs 14 gap cannot be explained by volume.
  • Content dyads with fixed placement and K/V amount show the effect comes from the archived content rather than extra attention mass.
  • A closed-path equivalence contract guarantees that NULL reproduces visible-only computation exactly.

This level of control is above the norm.

S2. A clean conceptual framing, with two genuinely informative findings. Treating compaction as a visibility decision rather than deletion, and splitting "whether to open" (gate) from "what to read" (the indexer, per layer and per query), is a useful lens for agent-memory design. Two results are informative beyond this paper:

  • Undirected access is harmful. Random opening falls below a closed archive in the agent study (266 vs 300, p = 0.0045).
  • Where to open matters far more than how much. In the native probe, random opening at a higher rate (0.54 vs 0.33) recovers nothing. Even at the trigger's exact rate it recovers 14 against 123. Inside opened positions the indexer retains far more answer tokens than a blind choice (95% and 74% under triggered and random opening, against 60%), although part of the triggered figure reflects where the trigger opens (W1).
Critical weaknesses

W1. The core claims are not supported end to end. The full proposed mechanism (gate + native reading) is never evaluated in a live agent, and the decisive comparisons are null or unfavorable.

  • The full mechanism never runs in a live agent. Per Table 6, attention-side reading appears in a live agent only in the 4-task DeepSeek WebShop suite. Table 6 describes that suite as using "native top-k membership (no separate gate)", which appears to make the archive always eligible, i.e. always-open K/V rather than Triggered-SA. For the showcased trajectory, the model's original full-context top-k path reproduces the same result. That suite is favorable against a closed archive (3 wins, 1 tie, 0 losses; mean reward 0.443 vs 0), but with 4 tasks and no gate it cannot establish the proposed mechanism. Contribution 3's "at similar expansion rates in two controlled agent studies" refers to the WebShop text-restoration waves. Listed next to the "model-native two-stage mechanism" of contribution 2, it reads as support for that mechanism, but it is not. The contribution list also omits the null comparison against Visible-Only.
  • The agent study does not test the paper's mechanism. In WebShop, an opened record's raw observations are restored to the prompt (§6, App. C, Table 6). That is prompt insertion, which the paper argues against ("retrieval solves location; access control solves placement"). The gate there is a logistic model on eight structural features. The authors concede that "combining the learned gate with native reading in a live agent is the natural next step." So the title's attention-side access control is tested only on synthetic copy traces (Qwen) and on tiny DeepSeek suites. Stripped of the attention mechanism, the WebShop study tests a learned decision about when to retrieve archived text. That is the adaptive-retrieval problem addressed by FLARE and Self-RAG (both cited), Adaptive-RAG and MemGPT-style self-directed recall, yet none of these is compared.
  • In the agent, Triggered-SA does not beat simply keeping the archive closed. §3 names Triggered-SA versus Visible-Only (together with undirected-exposure controls) as "the primary evaluation target." Yet the headline WebShop result is the contrast with random opening, and the comparison with Visible-Only appears only in appendix Tables 7–9.
  • Against a closed archive: nonzero reward is 320 vs 300, with p = 0.37 and 0.27 in the two waves.
  • Mean reward is +0.003
−0.010,+0.016−0.010, +0.016

(Table 8).

  • On R ≥ 0.5 the closed archive is better (211 vs 206).
  • Full matches are 14–16 of 1,024 in every arm.

The abstract and contributions report only the comparisons against random and always-open access. - In the native probe, plain full access is about twice as good. Making the whole trace eligible answers 658 vs 342 (one event) and 197 vs 123 (sixteen events). With K/V retained, this is the "always-open K/V memory" interface of Table 3. The paper labels it an "information reference, not a memory policy" and never compares against it as a policy. The motivating claim, that opening the archive pollutes the working context, is demonstrated only in the text-restoration agent setting, never with attention-side access. In DeepSeek too, "the model's original full-context top-k path reproduces the same trajectory" (§6, App. D). Fig. 2b points the same way: lowering the trigger threshold from −3.65 to −4.5 raises the opened fraction from 0.33 to 0.53 and accuracy from 71 to 129 of 250, and the curve heads toward full access. Opening more helps monotonically, and no pollution appears. - The random baseline is weak. In the native probe it flips an independent coin per (layer, query) position. The authors note that such scattered openings cannot sustain the multi-step access exact copying requires, and that 233 of 300 random answers copy a visible code instead. Beating this mainly shows that the indexer's own scores are informative, which full access already exploits. Fig. 2c (95% of answer tokens retained at triggered positions, against 74% under random opening) is partly selection by construction. The trigger opens exactly where archived index scores approach the visible maximum, so answer tokens are more likely to be in the top-k at those positions. The gap between random opening and a blind choice (74% vs 60%) reflects the pretrained indexer's own skill, not the gate. Stronger controls would be: per-trace or per-decision random opening; opening the top-scored positions without a threshold; and standard retrieval (RAG insertion, MemGPT-style recall, or kNN/KV-retrieval memory). - Several supporting results rest on tiny samples: - the DeepSeek transfer suite has 4 tasks (3 wins, 1 tie); - the DeepSeek detail suite has 26 queries; - the DeepSeek gate is calibrated on 32 pairs; - the ScienceWorld gate AUROC comes from 25 pairs (50 rows); - the content controls use 8 dyads; - the 1.96-nat attention-versus-prompt margin is measured at a single matched WebShop decision.

W2. Novelty relative to KV-retrieval long-context methods is not established, and the practical cost is high.

  • Missing related work. Storing past K/V outside the active context and letting sparse or top-k attention retrieve tokens or blocks from it is the core of a substantial line of work that is not discussed: Landmark Attention (2023), InfLLM (2024), Quest (2024), RetrievalAttention (2024), and KV-offloading systems such as ShadowKV and MagicPIG. The paper cites Memorizing Transformers and MemLong as "the closest line." Beyond these, the new element is a binary eligibility gate over the archive, and the paper shows no comparison against any of these methods.
  • The pieces are not new. The training-free trigger is a threshold on the indexer's own score margin (open if the best archived score is within 3.65 of the best visible). The learned gate is a small classifier on pooled index features or structural features. Neither is a methodological advance on its own.
  • The cost is high. For Qwen3-4B in bf16, the archive needs 153 KiB per token: 4.78 GiB for 32K archived tokens (Table 13). Admitting a 32K archive takes 105.8 ms from pinned CPU memory (12.2 ms if GPU-resident). Storage and admission costs for DeepSeek-V3.2 are not reported. The paper does not compare this against text archives with re-encoding or prompt retrieval, the obvious cheaper alternatives. The approach also needs model-side K/V and index keys, so it cannot be used with API-served models.
  • Absolute effects are small. WebShop mean reward stays near 0.15–0.17 in every arm, and full success (R = 1) is about 1.5%. The headline gains are measured on "nonzero reward" (R > 0), a lenient criterion.
Questions for authors
  1. Can you evaluate the full proposed system, with the learned or trained gate andattention-side native reading, in a live agent (WebShop, ALFWorld or SWE-style tasks), and compare it against (a) a closed archive, (b) always-open K/V with the same indexer, and (c) prompt-insertion retrieval?
  2. In WebShop, Triggered-SA does not significantly beat Visible-Only (mean reward +0.003
−0.010,+0.016−0.010, +0.016

; R > 0 p = 0.37 and 0.27). Since §3 names this the primary target, why was random opening registered as the primary contrast? What does the method offer a practitioner over keeping the archive closed?
3. In the native probe, full access answers 658 vs 342 and 197 vs 123. Is there any setting where attention-side always-open K/V access is worse than Triggered-SA? If not, what justifies hiding the archive, other than the cost of the indexer computing over eligible tokens?
4. Please compare against KV-retrieval long-context methods (Landmark Attention, InfLLM, Quest, RetrievalAttention) and a kNN K/V memory such as Memorizing Transformers. For the text-restoration WebShop study, please compare against adaptive-retrieval baselines that decide when to retrieve (FLARE, Self-RAG, MemGPT-style self-directed recall). Please also add stronger random controls: per-decision random opening, and top-scored positions without a threshold.
5. What are the archive storage and admission costs for DeepSeek-V3.2, and how do they compare with storing text and re-encoding on demand?
6. Can you report Figure 4 (repository multihop, ALFWorld, WebShop) as a table with n, intervals and paired tests?
7. Can you enlarge the DeepSeek transfer suite (currently 4 tasks), the gate-calibration sets (25 and 32 pairs) and the content dyads (8), and measure the 1.96-nat attention-versus-prompt margin over many decisions rather than one?
8. Where are the pre-registrations recorded, and can reviewers check their timestamps?
9. What is the intended scope of the archive? App. A states that archived K/V keeps its original rotary positions, and that in the native probes the span "stays in place in the sequence." Is the archive therefore still bounded by the model's context window and trained position range? Can an archive be reused across sessions or after a model update, which long-term agent memory would require, or is the method limited to within-trajectory compaction? (This determines whether the method applies to general agent memory or only to single-episode compaction.)

6 comments
AuthorsOpenReview ↗

Response to Reviewer nhb8 (Part 1/6)

We thank Reviewer nhb8 for the thorough review (W1–W2, Q1–Q9). We ran seven H200 experimental and kernel-level systems suites and updated the manuscript (9 main pages, Limitations on p. 10, Main Table 1, App. D–E Tables 7, 11–17, 22–24, Fig. 5). Below is a summary of revisions followed by Q1–Q9 in order. Notation: WCT = working-context token budget triggering compaction; expanded_l2 = raw archived (L2) text re-injected into the active prompt (0.0 on all native arms vs. 0.87 for non-compacting flat snippet retrieval); R1–R4 = prompt-injection formats (R1 trailing user msg., R2 within tool obs., R3 synthetic tool turn, R4 in-place splice); Suites A/B/C = 64-state archive-detail, stale-override, and visible-sufficient panels.

Summary of Revisions

  1. Q1 (W1): Full live agent (gate + native reading) vs. (a) Closed, (b) Always-Open K/V, (c) Prompt Retrieval (PDF: §5 Tab. 1 p. 5, §6.2–6.3 pp. 7–9, App. D.6–D.8 Tab. 13–17, Fig. 5a pp. 20–23, App. E Tab. 23–24 pp. 26–27): (1) Live 256-task WebShop (Qwen3-4B + native DSA, expanded_l2 = 0.0 across all layered arms): at WCT = 1600 (49 compacted), (a) Visible-Only gets 1/49 = 2.0% (0.0122 reward) while Learned-router DSA (13.7% compacted trigger rate, 5.8× < (b) Always-open 79.6%) gets 11/49 = 22.4% (+20.4 pp, 11W/1L, exact McNemar p=0.00635p = 0.00635; reward 0.0687, Wilcoxon p=0.0269p = 0.0269, −2.22 steps), raising 256-task reward to 0.3628 vs. 0.3615 Always-open, 0.3608 Visible-Only, and 0.3585 Matched-random (+0.0042, 4W/0L); at WCT = 2800 (46 compacted), compacted R>0 rises 4/46 (8.7%) → 9/46 (19.6%, reward 0.0289 → 0.0697). (2) Live 128-cell loop: 8/32 vs. 0/32 Visible-Only (p=0.0078p = 0.0078; 5/32 semantic). (3) 192-state panel: 74.48% (143/192) vs. 35.42% Closed and 36.46% Qwen3-Embedding-4B R2.
  2. Q2 (W1): WebShop vs. Visible-Only (+0.003) and primary contrast (PDF: Abs. p. 1, §1 p. 2, §6.2 & Fig. 3 pp. 7–8, App. D.3–D.4 & D.6 Tab. 8–11, 14 pp. 18–19, 21): Triggered-SA vs. Visible-Only is reported upfront (320 vs. 300 R>0, 91W/71L, pooled p=0.135p = 0.135; per-wave p=0.37/0.27p = 0.37 / 0.27; R≥0.5: 206 vs. 211; R=1: 15 vs. 14). On zero-shot 30B WebShop, where summaries keep coarse cues, gating prevents unselective-recall harm (Random 266 vs. Visible-Only 300, p=0.0045p = 0.0045; +0.027 vs. Random, +0.017 vs. Always-Open at 2.7× less volume). Under tighter WCT = 1600 (Table 14) or missing details (Tables 13, 17, 19), access significantly recovers dropped identifiers (1/49 → 11/49, p=0.00635p = 0.00635 on 256-task WebShop).
  3. Q3 (W1): Always-Open vs. Triggered-SA and why hide the archive (PDF: §6.1 & §6.3 pp. 6–9, Lim. p. 10, App. D.5–D.8 Tab. 12, 14–15, 17, Fig. 5a–b pp. 20–23, App. E Tab. 21–24 pp. 25–27): (1) 128-trace copy continuity resolved: causal turn-latched prefill gating (τ=−2.5\tau = -2.5) lifts Triggered-SA from 52/128 (40.6%) to 88–90/128 (68.8%–70.3%, p=5.63×10−9p = 5.63\times 10^{-9}), matching Always-open (90/128). (2) Stale overrides (Suite B, N = 64): pre-compaction RoPE (+21.33 nats, 0% leak) protects Always-open (64/64). (3) Why hide: on 256-task WebShop (WCT = 1600), Learned-router cuts compacted openings 5.8× (13.7% vs. 79.6% Always-open) with higher compacted reward (0.0687 vs. 0.0585); on 192 states (Suite C), τg=0.005\tau_g = 0.005 cuts false openings by 53.1% (100% at τg=0.01\tau_g = 0.01) with Eq. 2 invariance (32/32 BF16, 0.0 FP32 diff).
  4. Q4 (W1/W2): KV-retrieval literature, adaptive RAG, and stronger controls (PDF: §2 pp. 2–3, §6.1 & §6.3 pp. 6–9, App. D.5, D.7, D.8 Tab. 12, 15–17, Fig. 5a–c pp. 20, 22–23): Expanded §2 with Landmark, InfLLM, Quest, RetrievalAttention, ShadowKV, and MagicPIG; tested FLARE (τ∈{0.50,0.75,0.90}\tau \in \{0.50,0.75,0.90\}), matched-gate MiniLM, and 4×4 BM25 / MiniLM / Qwen3-Embedding-4B / Oracle × R1–R4 (Tables 15–16); added persistent whole-trace Random (80/128, beaten by turn-latched Triggered-SA 88/128, p=0.0215p = 0.0215, Table 12), matched Random (+6.92 pp), 0-param DSA tail-margin (+5.21 pp, Table 17), and 0.65K → 15.8K scaling (Fig. 5c).
AuthorsOpenReview ↗

Response to Reviewer nhb8 (Part 2/6)

Summary of Revisions (continued)

  1. Q5 (W2): DeepSeek-V3.2 storage, hook-free FA2 + PCIe5 staging, and re-encoding (PDF: §6.3–6.4 pp. 8–9, App. E.1 & E.3 Tab. 22, 24 pp. 25–27): DeepSeek-V3.2 uses 76.49 / 46.94 KiB/tok (BF16/FP8; 2.39 / 1.47 GiB at 32K), staging in 46.4 / 28.5 ms on 1×PCIe5 (55.3 GB/s) or 5.8–9.3 / 3.6–5.7 ms across 8×PCIe5 vs. 2.42 PFLOPs (~970 ms) re-prefill. In hook-free merged-BF16 FlashAttention-2 on H200 (Tab. 24), Qwen3-4B turn-latched Open takes 816.9 / 812.9 ms at 12K/32K (1.39× / 2.90× < R4 FA2 re-prefill 1,135.6 / 2,358.3 ms), and PCIe5 selective top-k host staging takes 803.6 / 839.2 ms (1.41× / 2.81× faster).
  2. Q6: Full Figure 4 breakdown table (PDF: Fig. 4 p. 8, App. D.4 Tab. 11 p. 19): Added Table 11 with exact n, W/L (91/71 WebShop), 95% CIs, exact McNemar pp, tool calls, and compactions (Repo n=9n=9 p=0.250p=0.250, n=10n=10 ITT p=0.125p=0.125; ALFWorld n=134n=134 p=0.727p=0.727, compactions 15.37 → 16.46), marking Repo and ALFWorld exploratory.
  3. Q7 (W1): Multi-decision margin (N = 64) and sample sizes (PDF: §6.2–6.4 pp. 8–9, App. D.5–D.10 pp. 19–24): Scaled action margin from 1 to 64 decisions (+9.30 nats vs. Visible-Only, −0.22 nats vs. oracle text; +21.33 nats stale margin on 64 Suite B states); expanded content dyads 8× to 64 dyads (128/128 vs. 0/128, +49.90 nats, p=1.08×10−19p = 1.08\times 10^{-19}); expanded live WebShop 4× to n = 256 (Tab. 14); scoped DeepSeek-V3.2 (Tab. 19) and ScienceWorld (Tab. 18) as targeted checks.
  4. Q8: Provenance and timestamps of task-list manifests (PDF: Abs. p. 1, §5 p. 6, App. C–D pp. 16–20): Clarified that "registered before outputs existed" meant task lists and primary contrasts were locked via SHA-256 manifests in the anonymous artifact before running jobs (not an external registry; no notarized timestamps), and described them plainly as fixed held-out splits and pre-specified contrasts.
  5. Q9: RoPE window and session scope (PDF: §4 p. 4, §7 p. 9, Lim. p. 10, App. A & E pp. 13, 25–27): Stated upfront: TSA targets within-trajectory, single-checkpoint compaction bounded by the trained RoPE window (32K–128K); cross-session or cross-checkpoint memory stores compact raw text (0.125 MiB / 32K) and re-encodes on demand.

Q1 (and W1): Full System (Learned Gate + Native Reading) vs. (a) Closed, (b) Always-Open K/V, and (c) Prompt Retrieval

Q1: "Can you evaluate the full proposed system, with the learned or trained gate and attention-side native reading, in a live agent (WebShop, ALFWorld or SWE-style tasks), and compare it against (a) a closed archive, (b) always-open K/V with the same indexer, and (c) prompt-insertion retrieval?"

Main §5 Table 1 (p. 5) maps each interface across backbones. To test the fixed 8-feature gate composed directly with native DSA reading (expanded_l2 = 0.0, 0 B prompt restoration across all native arms), we ran three live and post-compaction suites on Qwen3-4B + native DSA + LoRA (top-k = 1024), plus BF16 equivalence and hook-free FlashAttention-2 benchmarks:

  1. Live unselected official WebShop (256 tasks) (§6.2 p. 8 & App. D.6 Table 14, p. 21): We ran the first 256 official WebShop tasks at WCT = 2800 and tighter WCT = 1600 with homogeneous expanded_l2 = 0.0 across all layered arms, whereas non-compacting reference (c) flat snippet retrieval re-injects raw L2 text (expanded_l2 = 0.87, 0 compactions, 0.4199 overall reward, 233/256 = 91.0% R>0):
    • WCT = 1600 (49/256 compacted, 8.47 compactions/task in Visible-Only): (a) Visible-Only completes 1/49 compacted tasks (2.0% R>0, 0.0122 reward, 15.35 steps). Learned-router DSA (13.7% compacted trigger rate, 5.8× sparser than (b) Always-open DSA at 79.6%) and Native score-margin DSA complete 11/49 (22.4%, +20.4 pp, 11W/1L, exact McNemar p=0.00635p = 0.00635) with 0.0687 compacted reward (paired Wilcoxon p=0.0269p = 0.0269, 13.12 steps), above (b) Always-open DSA (10/49 = 20.4%, 0.0585 reward) and lifting 256-task reward to 0.3628 (200/256) vs. 0.3608 Visible-Only and 0.3585 Matched-random (+0.0042, 4W/0L).
    • WCT = 2800 (46/256 compacted, 5.30 compactions/task): Learned-router DSA (15.8% compacted trigger rate) lifts compacted R > 0 from 4/46 (8.7%) to 9/46 (19.6%, 7W/2L) and compacted reward from 0.0289 to 0.0697 (Wilcoxon p=0.0664p = 0.0664; 0.0749 for Score-margin DSA, steps 15.43 → 14.09) while preserving 203/256 (79.3%, 0.3626) overall.
AuthorsOpenReview ↗

Response to Reviewer nhb8 (Part 3/6)

  1. 128-cell live multi-step synthetic agent loop (32 tasks × 4 policies, 964 generations; Appendix D.6 Table 13, p. 20): Across four tool-use families (config_merge, relational_inventory, runbook_dependency, weighted_route; 8 tasks each, ITT scoring) with our fixed 8-feature gate (τg=0.005\tau_g = 0.005), Learned gate + native DSA and (b) Always-open native DSA both solve 8/32 (25.0%, 8/8 on config_merge) vs. 0/32 (0.0%) for (a) Visible-Only (exact McNemar p=0.0078p = 0.0078, Holm 0.0234) and 5/32 (15.6%) for (c) semantic prompt retrieval (p=0.25p = 0.25; 5/8 on config_merge, per-family counts in Table 13).
  2. 192-state balanced 3-suite panel and 4×4 retriever × placement matrix (Appendix D.7–D.8 Tables 15–17, Fig. 5a, pp. 22–23): Across 64 Archive-Detail (Suite A) + 64 Stale-Override (Suite B) + 64 Visible-Sufficient (Suite C) states, Learned 8-feat. gate + native DSA (τg=0.005\tau_g = 0.005) reaches 74.48% (143/192) at 82.29% open rate (46.88% open on Suite C). It outperforms (a) Visible-Only (35.42%, 68/192), (c) 4B Qwen3-Embedding-4B across all four prompt placements (1.56% R1–R3 and 10.94% R4 vs. 23.44% native on Suite A, p=0.0078p = 0.0078 vs. R4; 36.46% on 192 states under R2, Tables 16–17), matched-gate all-MiniLM-L6-v2 (2/128 = 1.6%), and FLARE (1/128 = 0.8%, Table 15). Cached-prefix open execution runs 133.2× faster than R4 re-prefill in the 12K eager harness (695.8 ms vs. 92.7 s; max |Δ| = 0.0 in FP32 fork, Table 23) and 1.39×–2.90× (GPU) / 1.41×–2.81× (PCIe5) faster than hook-free FlashAttention-2 R4 re-prefill across 12K–32K archives (Table 24).

Q2 (and W1): WebShop Visible-Only Reliability (+0.003), Primary Contrast Choice, and Practical Value

Q2: "In WebShop, Triggered-SA does not significantly beat Visible-Only (mean reward +0.003 [-0.010, +0.016]; R > 0 p = 0.37 and 0.27). Since §3 names this the primary target, why was random opening registered as the primary contrast? What does the method offer a practitioner over keeping the archive closed?"

  1. Why matched-rate Random was pre-specified as the primary contrast and upfront Visible-Only reporting: Because Visible-Only exposes 0 archived tokens while any recall policy exposes >0>0 tokens, Triggered-SA vs. Random (0.70 vs. 0.63 expansions/step) isolates selective conditioning at matched archive volume. We updated the Abstract, Contributions (§1), §3, and §6.2 (Tables 8–11) so Triggered-SA vs. Visible-Only (320 vs. 300 R>0, 91W/71L, pooled p=0.135p = 0.135, per-wave p=0.37/0.27p = 0.37 / 0.27; R≥0.5: 206 vs. 211; R=1: 15 vs. 14) appears upfront in the Abstract and main text.
  2. Practical value over permanently closing the archive:
    • Protection when summaries suffice (Tables 8–10): On zero-shot 30B WebShop where summaries retain coarse category and price cues, unselective recall degrades reward (Random 266 vs. Visible-Only 300 R>0, p=0.0045p = 0.0045). Triggered-SA preserves closed-memory reliability (320 vs. 300 R>0, pooled p=0.135p = 0.135, ΔR=+0.003\Delta R = +0.003) while gaining +0.027 over Random (p=3.00×10−5p = 3.00\times 10^{-5}) and +0.017 over Always-Open (p=0.0215p = 0.0215) at 2.7× fewer expansions.
    • Statistically significant detail recovery under tighter compaction (Tables 13–14, 17, 19): On live 256-task WebShop at WCT = 1600 (Table 14), Visible-Only completes only 1/49 (2.0%) compacted tasks whereas Triggered-SA completes 11/49 (22.4%, 11W/1L, exact McNemar p=0.00635p = 0.00635, reward 0.0122 → 0.0687, Wilcoxon p=0.0269p = 0.0269; and 4/46 → 9/46 at WCT = 2800). Consistent recovery holds on DeepSeek-V3.2 (0/26 → 21/26, p=9.54×10−7p = 9.54\times 10^{-7}, Table 19), live config_merge (0/8 → 8/8, Table 13), and 192-state decisions (68/192 → 143/192, Table 17).
Reviewer_Brb1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that compressing an agent's history should not mean deleting it. Instead, the original tokens are kept in a hidden archive that attention cannot see by default. A learned gate decides when the archive may be opened, and the model's sparse-attention indexer then picks which hidden tokens to read. The method is tested on synthetic recall tasks with Qwen3-4B, on WebShop agent tasks with Qwen3-30B-A3B, and in a small study with DeepSeek-V3.2's native indexer. The authors report that the gate beats opening the archive at random, and that the native indexer can recover most hidden details without retraining.

Critical strengths
  • Clear problem framing with careful controls. Separating storage from visibility after compaction is a useful idea, and the paper backs it with pre-registered comparisons, paired McNemar tests, byte-identical prompts across WebShop arms, and a rate-matched random control in the sixteen-event probe.

  • Closed-path safety by design. The visible-preserving contract (Eq. 4) makes a closed gate exactly reproduce the visible-only computation, so the method cannot hurt when it declines to open. This is a practical deployment property.

Critical weaknesses
  • The agent study does not test the proposed mechanism. In WebShop, an opened record's raw observations are restored to the agent's context (Section 6; Table 6), which is prompt insertion, the very thing the paper argues against. The gate is a logistic model on 8 structural features, not the native-index access head. So the main real-environment result tests a retrieval-timing heuristic, not attention-side access. The Limitations section admits the learned gate and native reading are never combined in a live agent.

  • Gains over the simplest baseline are not shown. In WebShop, Triggered-SA does not significantly beat a permanently closed archive (320 vs 300, p = 0.37 and 0.27; pooled delta R = +0.003, CI [-0.010, +0.016], Table 8). The wins are only against random and Always-Open, which the paper itself shows are harmful. In the native probe, full access (open rate 1.00) answers 658 vs 342 and 197 vs 123 (Table 1), so opening everything beats selective opening there. The "selection over volume" claim thus rests on weak comparators.

  • Missing retrieval baselines and small transfer evidence. No experiment compares against text retrieval (e.g., re-inserting the top-scoring archived record) or cached-representation retrieval like Memorizing Transformers, which the paper calls the closest line of work (Section 2). The argument that attention admission beats prompt insertion relies on one matched WebShop decision (1.96 nats, Section 6). The DeepSeek results use 26 queries, 4 tasks, and 32 pairs, with no separate gate in the trajectory suite (Table 6), so they say little about the full method.

Questions for authors
  1. Can you run the WebShop study with the actual attention-side mechanism (native-index gate plus sparse reading, no prompt restoration), and report it against Visible-Only and a retrieval-and-reinsert baseline?
  2. Given that Triggered-SA does not beat the closed archive in WebShop, what evidence supports a practical benefit of opening the archive at all in that setting?
  3. In the native probe, how does a simple retrieval baseline (insert the highest-scoring archived event as text) compare with 342/1,000 and with full access?
  4. The archive costs about 4.8 GiB per 32K tokens for a 4B model (Table 13). How does this scale for DeepSeek-V3.2 and long agent runs, and is storing K/V more practical than storing raw text and re-encoding on demand?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Brb1 (Part 1/5)

We thank Reviewer Brb1 for the direct, soundness-focused review (W1–W3 and Q1–Q4). To address these points, we ran seven experimental and kernel-level systems suites on H200, including 256-task official WebShop evaluated directly with native attention-side reading (expanded_l2 = 0.0), and clarified the scope of each claim in the revised manuscript (9 main pages, Limitations starting on p. 10, Main Table 1, Appendix D–E Tables 7, 11–17, 22–24, and Figure 5). Notation used below: WCT denotes the working-context token budget that triggers compaction; expanded_l2 denotes raw archived (L2) text re-injected into the active prompt (0.0 on all native arms, vs. 0.87 for non-compacting flat snippet retrieval); R1–R4 denote the four prompt-injection formats of the text-retrieval baselines (R1 trailing user message, R2 inside the tool observation, R3 synthetic tool turn, R4 in-place history splice); and Suite A / B / C denote our 64-state archive-detail, stale-override, and visible-sufficient post-compaction suites.

Summary of Revisions

  1. W1 & Q1: WebShop with native attention-side mechanism (native-index gate + sparse reading, no prompt restoration, vs. Visible-Only & retrieval-and-reinsert) (PDF: §5 Table 1 p. 5, §6.2–§6.3 pp. 7–9, Limitations p. 10, App. D.6–D.8 Tables 13–17, Fig. 5a pp. 20–23, App. E Tables 23–24 pp. 26–27): (1) Live 256-task WebShop on Qwen3-4B + native DSA (homogeneous expanded_l2 = 0.0, 0 B prompt restoration), testing both Native score-margin DSA (native-index gate) and Learned-router DSA: at tight WCT = 1600 (49 compacted), Visible-Only completes 1/49 = 2.0% (0.0122 reward) while both native Triggered-SA arms complete 11/49 = 22.4% (+20.4 pp, 11W/1L, exact McNemar p=0.00635p = 0.00635; reward 0.0687, Wilcoxon p=0.0269p = 0.0269, −2.22 steps), with Learned-router opening on only 13.7% of compacted steps (5.8× < Always-open 79.6%) and beating Matched-random overall (0.3628 vs. 0.3585, 4W/0L); at WCT = 2800 (46 compacted), compacted R>0 rises 4/46 (8.7%) → 9/46 (19.6%, reward 0.0289 → 0.0697 / 0.0749). (2) Live 128-cell loop (Table 13) & 192-state panel (Table 17): 8/32 vs. 0/32 Visible-Only (p=0.0078p = 0.0078); 74.48% (143/192) vs. 35.42% Closed and 36.46% Qwen3-Embedding-4B R2.
  2. W2 & Q2: Benefit over Visible-Only on WebShop and Full access vs. Triggered-SA (PDF: Abstract p. 1, §1 p. 2, §6.1–§6.3 pp. 6–9, Limitations p. 10, App. D.3, D.5, D.6, D.8 Tables 8–10, 12, 14, 17, Fig. 5a–b pp. 18–21, 23): (1) WebShop: on 1,024 paired 30B tasks where summaries retain coarse cues, selective gating avoids unselective-recall loss (Random 266 vs. Visible-Only 300 R>0, p=0.0045p = 0.0045) while preserving closed-memory reliability (320 vs. 300, 91W/71L, p=0.135p = 0.135); under tighter WCT = 1600 on 256 tasks (Table 14), native Triggered-SA significantly beats Visible-Only (11/49 vs. 1/49, p=0.00635p = 0.00635). (2) Full access gap resolved (Table 12, Fig. 5b): causal turn-latching (τ=−2.5\tau = -2.5) lifts Triggered-SA from 52/128 (40.6%) to 88–90/128 (68.8%–70.3%, p=5.63×10−9p = 5.63\times 10^{-9}), matching Always-open (90/128) and beating persistent Random (80/128, p=0.0215p = 0.0215).
  3. W3 & Q3: Top-1 text retrieval on native probe, 4×4 retrieval matrix, and N = 64 margins (PDF: §2 pp. 2–3, §6.1 & §6.3 pp. 6–9, §7 p. 9, App. D.5, D.7–D.10 Tables 12, 15–19, Fig. 5a–c pp. 20, 22–24): (1) Native probe top-1 retrieval (128-trace copy, Table 12): turn-latched Triggered-SA (88–90/128 = 68.8%–70.3%) beats Qwen3-Embedding-4B top-1 reinsertion (72/128 = 56.3%), while query-lexical top-1 reaches 93/128 (72.7%) on a shorter 618-tok prompt. (2) Tool trajectories (Tables 15–16) & N = 64 margins: Qwen3-Embedding-4B gets 1.56% (R1–R3) and 10.94% (R4) vs. 23.44% native (p=0.0078p = 0.0078 vs. R4; +9.30 nats margin; 64 dyads 128/128, p=1.08×10−19p = 1.08\times 10^{-19}); across 0.65K–15.8K hidden tokens, Global top-k and Equal-slice TSA reach 81.3% (13/16) at 10.6K vs. 12.5% R2.
AuthorsOpenReview ↗

Response to Reviewer Brb1 (Part 2/5)

Summary of Revisions (continued)

  1. Q4: DeepSeek-V3.2 archive scaling & hook-free FA2 + PCIe5 vs. raw text re-encoding (PDF: §6.3–§7 pp. 8–9, Limitations p. 10, App. D.8 & E Tables 17, 21–24 pp. 23, 25–27): DeepSeek-V3.2 MLA + FP8 DSA uses 76.49 / 46.94 KiB/tok (BF16/FP8; 1.47–2.39 GiB at 32K), staging in 46.4 / 28.5 ms (1×PCIe5) or 5.8–9.3 / 3.6–5.7 ms (8×PCIe5) vs. 2.42 PFLOPs (~970 ms) re-prefill (Table 22). On H200 in hook-free merged-BF16 FlashAttention-2 (Table 24), Qwen3-4B turn-latched Open takes 816.9 / 812.9 ms at 12K/32K (1.39× / 2.90× < R4 FA2 re-prefill 1,135.6 / 2,358.3 ms), and PCIe5 selective top-k host staging takes 803.6 / 839.2 ms (1.41× / 2.81× faster; 133.2× in the eager harness, Table 17).

W1 & Q1: WebShop with the Attention-Side Mechanism (Native-Index / Learned Gate + Sparse Reading, No Prompt Restoration)

Q1: "Can you run the WebShop study with the actual attention-side mechanism (native-index gate plus sparse reading, no prompt restoration), and report it against Visible-Only and a retrieval-and-reinsert baseline?"

We evaluated the first 256 unselected official WebShop tasks on converted Qwen3-4B + native DSA + LoRA (top-k = 1024) with attention-side memory transport (homogeneous expanded_l2 = 0.0 across all layered arms), testing both Native score-margin DSA (native-index gate) and Learned-router DSA alongside a 128-cell live tool loop and 192-state decision panel (Appendix D.6 Table 14, p. 21):

Policy (Qwen3-4B + Native DSA, n = 256)WCT=2800 All Rˉ\bar{R} (R>0)WCT=2800 Comp. (n=46; Trig.%)WCT=1600 All Rˉ\bar{R} (R>0)WCT=1600 Comp. (n=49; Trig.%)
Visible-Only (expanded_l2 = 0.0)0.3628 (203/256 = 79.3%)0.0289 (4/46 = 8.7%; 0.0%)0.3608 (198/256 = 77.3%)0.0122 (1/49 = 2.0%; 0.0%)
Native score-margin DSA (expL2 = 0.0)0.3594 (202/256 = 78.9%)0.0749 (9/46 = 19.6%; 58.3%)0.3628 (200/256 = 78.1%)0.0687 (11/49 = 22.4%; 53.1%)
Learned-router DSA (expL2 = 0.0)0.3626 (203/256 = 79.3%)0.0697 (9/46 = 19.6%; 15.8%)0.3628 (200/256 = 78.1%)0.0687 (11/49 = 22.4%; 13.7%)
Always-open DSA (expL2 = 0.0)0.3608 (201/256 = 78.5%)0.0749 (8/46 = 17.4%; 87.0%)0.3615 (200/256 = 78.1%)0.0585 (10/49 = 20.4%; 79.6%)
Matched-random DSA (expL2 = 0.0)0.3575 (199/256 = 77.7%)0.0532 (6/46 = 13.0%; 36.3%)0.3585 (199/256 = 77.7%)0.0595 (11/49 = 22.4%; 34.7%)
Flat snippet retrieval (expL2 = 0.87, no comp.)0.4199 (233/256 = 91.0%)0.2665 (36/46 = 78.3%; n/a)0.4199 (233/256 = 91.0%)0.2544 (39/49 = 79.6%; n/a)
  • WCT = 1600 (49 compacted, 8.47 compactions/task in Visible-Only): Both Native score-margin DSA and Learned-router DSA lift compacted R>0 11× from 1/49 (2.0%) to 11/49 (22.4%, 11W/1L, exact McNemar p=0.00635p = 0.00635) and compacted reward 5.6× from 0.0122 to 0.0687 (paired Wilcoxon p=0.0269p = 0.0269, −2.22 steps), with Learned-router triggering 5.8× less than Always-open (13.7% vs. 79.6%) and beating Matched-random overall (+0.0042, 4W/0L). At WCT = 2800, compacted R>0 rises from 4/46 (8.7%) to 9/46 (19.6%, reward 0.0289 → 0.0697 / 0.0749).
  • 128-cell live loop (Table 13) & 192-state panel (Table 17): Learned gate + native DSA solves 8/32 vs. 0/32 Visible-Only (p=0.0078p = 0.0078; 5/32 semantic) and 74.48% (143/192) vs. 35.42% Closed and 36.46% Qwen3-Embedding-4B R2 (max |Δ| = 0.0 FP32 fork parity, Table 23).
AuthorsOpenReview ↗

Response to Reviewer Brb1 (Part 3/5)

W2 & Q2: Practical Benefit Over Visible-Only on WebShop and Full Access vs. Triggered-SA

Q2: "Given that Triggered-SA does not beat the closed archive in WebShop, what evidence supports a practical benefit of opening the archive at all in that setting?"

  1. 1,024-task WebShop reliability plus statistically significant WCT = 1600 recovery on 256 tasks (Abstract, §1, §6.2, Tables 8–10 & 14): On the 1,024-task Qwen3-30B-A3B study, where summaries retain coarse product cues, selective gating avoids the degradation caused by unselective recall (Random 266 vs. Visible-Only 300 R>0, p=0.0045p = 0.0045; +0.027 over Random, p=3.00×10−5p = 3.00\times 10^{-5}; +0.017 over Always-Open, p=0.0215p = 0.0215, at 2.7× less volume) while preserving closed-memory reliability (Triggered-SA vs. Visible-Only: 320 vs. 300 R>0, 91W/71L, pooled p=0.135p = 0.135; ΔR=+0.003\Delta R = +0.003, [−0.010, +0.016]). When WebShop working memory is tightened on 256 tasks (WCT = 1600, Table 14), Visible-Only completes only 1/49 (2.0%) compacted tasks whereas Triggered-SA completes 11/49 (22.4%, +20.4 pp, 11W/1L, exact McNemar p=0.00635p = 0.00635, reward 0.0122 → 0.0687, Wilcoxon p=0.0269p = 0.0269, −2.22 steps; and 4/46 → 9/46 at WCT = 2800), matching the recovery pattern on DeepSeek-V3.2 (0/26 → 21/26, p=9.54×10−7p = 9.54\times 10^{-7}, Table 19), live config_merge (0/8 → 8/8, Table 13), and 192-state decisions (68/192 → 143/192, Table 17).
  2. Resolving the Full access gap on 16-event copying (Table 12, Fig. 5b, pp. 20, 23): On the 128-trace copy suite (fresh 16-event traces, top-k = 1024 fixed), independent per-(ℓ,i)(\ell, i) gating (52/128 = 40.6%) fragmented multi-token code copying. Replacing per-token switching with a causal turn-latched prefill rule (τ=−2.5\tau = -2.5 upper-half prefill margin evaluated once at the final query token) lifts Triggered-SA to 88/128 (68.8%, upper-mean, +28.1 pp, p=5.63×10−9p = 5.63\times 10^{-9}) and 90/128 (70.3%, upper-max), matching Always-open (90/128 = 70.3%) and beating persistent whole-trace Random (80/128 = 62.5%, +6.25 pp, p=0.0215p = 0.0215).
  3. Role of the gate on native DSA (Tables 14 & 17, Fig. 5a): On Suite B (64 stale-override decisions), native DSA preserves pre-compaction RoPE positions (+21.33 nats, 0% stale leak; unrotated RoPE also reaches 100%), so Triggered-SA and Always-open both score 64/64. Across 256-task WebShop (WCT = 1600), Learned-router DSA achieves 5.8× lower compacted trigger rate than Always-open (13.7% vs. 79.6%; 3.9× lower than Score-margin 53.1%) while reaching slightly higher compacted reward (0.0687 vs. 0.0585) and beating Matched-random overall (0.3628 vs. 0.3585). On the 192-state panel, fixed τg=0.005\tau_g = 0.005 cuts Suite C false openings by 53.1% (100% at τg=0.01\tau_g = 0.01) with zero accuracy loss, beating matched Random (+6.92 pp) and 0-parameter DSA tail-margin (+5.21 pp).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Response to All Reviewers

We thank Reviewers Ze82, nhb8, and Brb1 for constructive reviews. Our 27-page revised manuscript (9-page main body, Limitations on p. 10; Table 1 p. 5, Table 7 p. 17) adds seven H200 experimental and kernel suites plus full-cohort statistical checks. Below, WCT is the compaction budget, expanded_l2 is the archived text share re-injected into the prompt (0.0 native vs. 0.87 non-compacting snippet retrieval), R1–R4 are prompt-retrieval formats (user, observation, tool, and in-place splice), and Suites A/B/C are our 64-state detail, stale-override, and visible-sufficient panels.

Headline Native Attention-Side & Kernel Results

Benchmark / SettingVisible-OnlyPrompt Retrieval / R4Triggered-SA (Native)Selectivity / Gain
Live WebShop (WCT=1600, 49 compacted)1/49 (2.0%, 0.0122)n/a (non-compacting 0.87)11/49 (22.4%, 0.0687)13.7% open (5.8× < Always-Open; p=0.00635p=0.00635)
Suite A 4×4 retrieval (N=64)0/64 (0.0%)7/64 (10.94%, R4)15/64 (23.44%)p=0.0078p=0.0078 vs. R4 (+9.30 nats)
Balanced 3-suite Pareto (N=192)68/192 (35.42%)70/192 (36.46%, R2)143/192 (74.48%)−53.1% Suite C opens (+6.92 pp vs. Random)
16-event copy (N=128, τ=−2.5\tau=-2.5)0/128 (0.0%)72/128 (56.25%, Qwen3-Emb)88–90/128 (68.75%–70.31%)Matches Always-Open (90/128); p=0.0215p=0.0215
Hook-free FA2 (12K / 32K, Tab. 24)845.3 / 845.3 ms1,135.6 / 2,358.3 ms (R4)816.9 / 812.9 ms (HBM)1.39× / 2.90× (PCIe5: 1.41× / 2.81×)

Summary of Revisions

  1. Live composition (256-task WebShop & tool loops) (Ze82, nhb8, Brb1 W1/Q1; §6.2–6.3 pp. 7–9, Tab. 13–14 pp. 20–21): On Qwen3-4B + native DSA (expanded_l2 = 0.0), Learned-router DSA gains +20.4 pp at WCT = 1600 (11W/1L, p=0.00635p=0.00635; Wilcoxon p=0.0269p=0.0269, −2.22 steps), lifting 256-task reward to 0.3628 vs. 0.3585 Random (+0.0042, 4W/0L). At WCT = 2800 (46 compacted), R>0 rises from 4/46 (8.7%) to 9/46 (19.6%, reward 0.0289 → 0.0697, 15.8% open). Live 128-cell tool loop: 8/32 vs. 0/32 Visible-Only (p=0.0078p=0.0078; 8/8 config_merge).
  2. 4B retrieval & scaling (R1–R4, 0.65K → 15.8K) (Ze82 W2/Q3, nhb8 Q4/Q7, Brb1 W3/Q3; §6.3 pp. 8–9, Tab. 15–16, Fig. 5c pp. 22–23): Across BM25 (26.6% recall@512), MiniLM+BM25 (28.1%), Qwen3-Embedding-4B (37.5%), and Oracle (50.0%) × R1–R4 on Suite A, Qwen3-Embedding-4B gets 1.56% (R1–R3) and FLARE gets 0.0% vs. R4/TSA above (16/16 BF16 config_merge, −0.22 nats vs. Oracle). From 0.65K → 15.8K hidden tokens (16.0K max), Global and Equal-slice TSA hit 81.25% at 10.6K (13/16) vs. 0.0%–12.5% R2 (0% Closed), keeping 12.5% Global vs. 6.25% R2 at 15.8K.
  3. Pareto, turn-latching & RoPE (nhb8 W1/Q3/Q4, Brb1 W2/W3/Q2/Q3; §6.1–7 pp. 6–9, Tab. 12, 15, 17, Fig. 5a–b pp. 20–23): On 192 states (64 A+B+C), Qwen3-Embedding-4B R2 drops Suite C from 100% to 95.31%; our learned 8-feature gate (τg=0.005\tau_g=0.005) reaches 74.48% at 82.29% open (0% loss), beating Random (67.56%) and 0-parameter margin (+5.21 pp), with 0.0% Suite C opens at τg=0.01\tau_g=0.01. On 128-trace copy, turn-latching (τ=−2.5\tau=-2.5) lifts TSA from 40.63% (p=5.63×10−9p=5.63\times 10^{-9}) past Random (80/128) and Qwen3-Embedding-4B top-1. Suite B: 64/64 (+21.33 nats, 0% leak); 64 dyads: 128/128 (p=1.08×10−19p=1.08\times 10^{-19}).
  4. DeepSeek-V3.2 staging, hook-free FA2 & statistics (Ze82 Q2, nhb8 W1–2/Q2/Q5–6/Q8–9, Brb1 W2/Q2/Q4; §5–6.4 pp. 5–9, Tab. 11, 21–24 pp. 19, 25–27): DeepSeek-V3.2 MLA + FP8 DSA uses 76.49 / 46.94 KiB/tok (BF16/FP8; 1.47–2.39 GiB at 32K), staging in 46.4 / 28.5 ms (1×PCIe5, 55.3 GB/s) vs. 2.42 PFLOPs (~970 ms) re-prefill. In 12K eager, Open fork is 133.2× faster than R4 (695.8 ms vs. 92.7 s; 32/32 BF16 Eq. 2 match, 0.0 FP32 diff). In hook-free merged-BF16 FA2 (Tab. 24), PCIe5 top-k staging (9 KiB/tok HBM index) takes 803.6 / 839.2 ms (12.8 s vs. 37.9 s over a 16-turn 32K episode, 2.97×).

Takeaways

  • Mechanism: R1–R3 shift visible RoPE (61–63/64 redundant search calls) and R4 forces re-prefill; native DSA keeps prompts intact within the trained RoPE window (§4, §7, Limitations p. 10).
  • Selectivity: On 1,024 30B WebShop tasks, TSA beats Random (+0.027, p=3.00×10−5p=3.00\times 10^{-5}) and Always-Open (+0.017, p=0.0215p=0.0215, 2.7× less volume) while keeping Visible-Only reliability (320 vs. 300 R>0, ΔR=+0.003\Delta R=+0.003, p=0.135p=0.135). Table 11 adds Repo (p=0.250p=0.250; ITT p=0.125p=0.125), ALFWorld (p=0.727p=0.727), and fixed split SHA-256 manifests (§5, p. 6).

Full point-by-point analyses and tables follow in our individual replies.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.