- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper argues that a coding agent's own writes make its earlier reads stale, and text-based context management cannot see this. StateTape keeps a symbol-level code graph updated at every write, marks which symbols each write changed, and flags every held record that touches them. A small prompted LLM, the manager, then refreshes, drops or keeps flagged records, and pulls in code within two call hops of the changed symbols. TraceBench labels 403 pre-edit points from Claude Code runs on SWE-EVO and reports 73.4% of held tokens unneeded and 58.5% of needed tokens missing. On six agents in Claude Code and Codex CLI and three benchmarks, StateTape has the best resolve rate in all 18 cells at 0.8% to 4.7% manager overhead.
Critical strengths
- The paper formulates the problem that a coding agent can be fooled by context its own writes have revised, and demonstrates this phenomenon with Figure 1.
- The paper evaluates coding agents in realistic setup, Claude Code and Codex CLI, with LLMs of different size (Haiku 4.5, Sonnet 5, Opus 4.8; GPT-5.6 luna, terra, sol), on three edit-heavy benchmarks.
Critical weaknesses
- Current coding agents do not let the context rot as the paper assumes. Claude Code, Codex CLI and Cursor keep edit history through git, and analyze code through file-system and code tools such as grep, LSP find-references and code-graph MCP. They also compact the context when the window is full. In the experiments compaction is disabled (proof of Theorem 2(b)), so Control is the harness with its own context management turned off. I consider this an unfair comparison. No baseline gives the agent the same code graph as a tool either.
- The paper lacks novelty. Code graph as a solution to context localization has been studied for years, producing papers such as RepoGraph, LocAgent and CodexGraph. Context rot in LLMs is also a known problem with a long line of research. So the paper identifies a known problem and uses code graph as the main solution. I also do not see AI or ML rigor in the paper. Nothing is trained, and the manager is an off-the-shelf small LLM with a prompt. Most figures present logged numbers. The theoretical analysis proves properties of the code graph, which is not proposed by the paper but used out of the box, and does not connect to resolve rate.
Questions for authors
- When applying coding agents as baseline, did you enable harness native features, such as compaction, git history, and code tools like grep and LSP find-references?
- What is the tape's advantage over git history management? In my view tape is a stripped-down version of it. git diff already tells the agent which files and lines a write changed, and the agent can read the previous version at any time.
4 comments
Response to Reviewer WmFk (2/4)
Resolve rates (SWE-EVO / SlopCode / SWE-bench Pro); the best cell per agent and suite is in bold.
| Main agent | Method, resolve rate (%) | SWE-EVO | SlopCode | SWE-Pro | Calls to the added StateTape-graph server per trajectory | Share of graph calls for localization | Stale share of held tokens |
|---|---|---|---|---|---|---|---|
| Haiku 4.5 | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 25.0 | 4.6 | 26.3 | — | — | 74.6% |
| Control + compaction forced on (every 40 turns) | 25.0 | 5.1 | 26.5 | — | — | 72.9% | |
| Control + StateTape's graph as MCP tool + forced compaction | 27.1 | 6.1 | 26.9 | 4.7 | 88% | 70.3% | |
| StateTape (harness compaction off) | 29.2 | 8.1 | 27.8 | — | — | 17.2% | |
| Sonnet 5 | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 41.7 | 9.2 | 63.5 | — | — | 71.8% |
| Control + compaction forced on (every 40 turns) | 41.7 | 9.7 | 63.9 | — | — | 69.5% | |
| Control + StateTape's graph as MCP tool + forced compaction | 43.8 | 10.7 | 65.0 | 6.1 | 85% | 66.4% | |
| StateTape (harness compaction off) | 47.9 | 12.8 | 68.4 | — | — | 14.8% | |
| Opus 4.8 | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 58.3 | 9.7 | 69.2 | — | — | 70.2% |
| Control + compaction forced on (every 40 turns) | 58.3 | 10.2 | 69.5 | — | — | 67.8% | |
| Control + StateTape's graph as MCP tool + forced compaction | 60.4 | 10.7 | 70.2 | 6.9 | 82% | 64.5% | |
| StateTape (harness compaction off) | 62.5 | 12.8 | 71.4 | — | — | 13.9% | |
| GPT-5.6 luna | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 56.3 | 4.1 | 64.3 | — | — | 73.1% |
| Control + compaction forced on (every 40 turns) | 56.3 | 4.6 | 64.7 | — | — | 70.6% | |
| Control + StateTape's graph as MCP tool + forced compaction | 58.3 | 5.1 | 65.8 | 5.2 | 87% | 67.7% | |
| StateTape (harness compaction off) | 60.4 | 6.6 | 67.7 | — | — | 16.1% | |
| GPT-5.6 terra | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 52.1 | 6.6 | 65.8 | — | — | 72.4% |
| Control + compaction forced on (every 40 turns) | 52.1 | 6.6 | 66.2 | — | — | 70.1% | |
| Control + StateTape's graph as MCP tool + forced compaction | 54.2 | 7.1 | 67.0 | 5.8 | 85% | 67.0% | |
| StateTape (harness compaction off) | 58.3 | 8.2 | 68.4 | — | — | 15.3% | |
| GPT-5.6 sol | Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available) | 58.3 | 8.2 | 64.6 | — | — | 70.9% |
| Control + compaction forced on (every 40 turns) | 58.3 | 8.7 | 65.0 | — | — | 68.5% | |
| Control + StateTape's graph as MCP tool + forced compaction | 60.4 | 9.2 | 66.6 | 6.6 | 83% | 65.4% | |
| StateTape (harness compaction off) | 62.5 | 10.2 | 69.5 | — | — | 14.4% |
Note: — means the added StateTape-graph server is not present in this configuration, so there are no calls to count. Under StateTape the agent never queries the graph; the tape consults it after every write to compute drift sets (Section 3.3.2). The harness's default code-graph MCP remains available in every row and is not counted in these two columns.
Exposing StateTape's own graph as a tool and forcing compaction leave the problem in place for every agent.
- (a) The context still rots. Together they cut the stale share by only 4.3 to 5.7 points (Sonnet 5: 71.8% to 66.4%) and recover 0.5 to 2.1 of the 1.5 to 6.2 points StateTape gains over Control, at most half of the gain in any of the 18 cells.
- (b) The agent uses the graph to find where to edit, not what its edit invalidated. The stale share alone cannot tell whether the agent ignores the tool or calls it for a different purpose; the two call columns separate these cases. The agent calls the tool 4.7 to 6.9 times per trajectory, so it is not ignored. But 82% to 88% of those calls are for localization: they query a symbol that no earlier write has changed. Only the remaining 12% to 18% query a symbol in the drift set of an earlier write, and so could ask what the agent's own edit invalidated. Re-fetching helps only when the agent knows a belief is stale (A1.1).
- (c) Compaction does not fill the gap; StateTape does. A summarizer cannot tell a stale read from a fresh one. StateTape cuts the stale share to 13.9% to 17.2% with zero agent-side calls, because it does not wait for a question the agent does not know to ask.
Response to Reviewer WmFk (3/4)
A2. "Lack of novelty: code graphs for localization exist (RepoGraph, LocAgent, CodexGraph) and context rot is known."
1. Prior code-graph work uses the graph at read time to find code; StateTape uses the graph's change at write time to invalidate what is already held. RepoGraph, CodexGraph and LocAgent build a graph at task start and traverse it to localize (Section 2); none re-derives it after the agent's own write. StateTape's new objects are the drift sets (Section 3.3.2) and the nomination rule (Eq. 7), which map a write to the held records it falsifies with no model call. Table 3 isolates this: "w/o tape (text-judged staleness)" keeps the graph, and CE falls from 6.18 to 2.86, within 0.1 of evict-all.
2. The claim is not that context rot exists, but that the agent's own writes are a cause that can be observed rather than inferred. Masking, summarization, pruning, AgentDiet, SWE-Pruner, AdaCoM and ContextCurator (Section 2) all judge staleness from the text of the history. We judge it from the write log, where 72.3% of unneeded held tokens originate (Table 5).
3. The contributions beyond the graph are the ones the reviewer's own summary lists. (i) The action-conditioned formulation of staleness with its monotonicity property (Eq. 1); (ii) TraceBench, the first benchmark that labels a context state at a decision point rather than a terminal patch (Table 6); (iii) the two-sided CE/RE objective (Eqs. 2 to 5), which Appendix E shows F1 cannot replace; (iv) the tape-plus-manager division of labour.
A3. "No AI or ML rigor: nothing is trained, the manager is an off-the-shelf small LLM with a prompt, figures are logged numbers, and the theory is about the code graph and does not connect to resolve rate."
1. Training-free is a design choice that every baseline in Table 1 shares. Observation masking, LLM summary, CORVUS, AgentDiet-style managers, AdaCoM and ContextCurator all keep the base model frozen and ask what to put in front of it. A trained component would make the method harness- and model-specific, the opposite of what a drop-in shim for six models across two vendors needs; rigor here is a well-posed objective, a sound procedure, a benchmark that measures the right quantity and controlled comparisons, and the paper supplies each.
2. The theorems are about the tape and the manager, not about a pre-existing code graph. Theorem 1(a) guarantees that under symbol locality (Assumption 1) every stale record is nominated at the write that falsifies it; 1(b) bounds the stale mass after any manager output by the records the manager kept; 2(a) bounds the tape's cost without model calls; 2(b) bounds the manager's token share; 2(c) shows stale tokens sent grow as Ω(δW²) without maintenance. These are properties of the drift sets and nomination rule of Section 3.3.2, not of anyone's graph, and their empirical counterparts are Table 4 (manager share 0.8% to 4.7%, matching 2(b)) and Table 2 (CE 6.18 against 2.75 for evict-all, matching 1(b)).
3. On "does not connect to resolve rate." Resolve rate is a property of a frozen black-box model's behaviour, so no context-maintenance paper proves a theorem about it; the honest intermediate quantity is how much of the context is false and how much of what is needed is present, which is exactly what Theorem 2(c) bounds. The connection to resolve rate is empirical: Table 3 removes components one at a time and CE and retrieval coverage fall monotonically with resolve rate on all three benchmarks, Table 2 shows the same metrics separate StateTape from 13 reference policies, and Section 3.1.2 shows the stale share growing with edit count as Ω(δW²) predicts.
4. "Most figures present logged numbers." The figures report measurements of a system on benchmarks, which is what an empirical paper's figures do. The additional analyses the reviewers asked for are in this response (graph-as-tool and compaction baselines in A1.3, the stale share measured with git available in Q2.2) and in our responses to Reviewers 1RuA (three-seed variance, Table R1) and Tgkp (cache-aware cost accounting, Table R2).
Response to Reviewer WmFk (4/4)
Q1. "Did you enable harness-native features such as compaction, git history, and code tools like grep and LSP find-references?"
1. Tools: yes, all of them, for every method. The shim intercepts only the model request; the harness, its tool registry, the container and the repository's git history are untouched (Appendix D), so the agent in every row of Table 1 can run grep, git diff, git show, the code-graph MCP server that ships with each harness and any language server it would normally run.
2. Compaction: yes, enabled for Control and every baseline; it is off only under StateTape. Control runs Claude Code and Codex exactly as shipped, with the harness's own compaction on, and it fired in 3 of 48 Control trajectories on SWE-EVO with Sonnet 5. StateTape is the one arm that runs without it, so that its context is maintained by the tape and the manager alone and its gains are not shared with the harness's summarizer (A1.2). The LLM-summary baseline applies a more aggressive compaction than the harness default and loses to StateTape in every cell, and A1.3 reports compaction forced on.
3. Under StateTape the context window limit is never hit, so running that arm without compaction costs it nothing. StateTape removes a large portion of the unneeded context (74.2% of it, Table 2), which contributes nothing to the resolve rate. Its context therefore never reaches the window limit in any run, and the harness's compaction would have had nothing to compact.
Q2. "What is the tape's advantage over git history management? git diff already tells the agent which files and lines a write changed, and the agent can read the previous version at any time."
1. git diff reports which lines changed; staleness travels along the edges of the code graph to lines that did not change. Take a call chain A → B → C: function A calls B, and B calls C. The agent has read all three and has run A and B, so it holds their outputs as evidence. It then modifies C. git diff shows the changed lines of C and nothing else, because the text of A and B is untouched. Yet the outputs of A and B that the agent holds are no longer valid, since both were computed through the old C. These results must be rerun or marked as stale, and no git command says so: git has no symbol table and no call edges, so it cannot link a write on C to A and B. StateTape knows that A calls B and B calls C. When the agent modifies C, StateTape follows these calls backwards from C, finds B and A, and checks whether the agent still holds anything it learned about them. It does, so those results are marked as stale or refreshed before the agent's next step (Section 3.3.2). git tells the agent what it changed; StateTape tells the agent what it can no longer trust.
2. The agent in every measured run already had git, and 72.3% of the unneeded tokens it held were stale anyway. Section 3.1.2 and Table 5 were measured on Claude Code with no shim and with git and the file tools available: across 48 trajectories, 74% to 80% of held context is unneeded, and 72.3% of that is stale because the agent's own later write falsified it (Figure 2). The agent could have run git diff after any of those writes; it did not, because nothing told it a held belief had become false. Table 1 is therefore already the requested comparison: Control has git, StateTape has git plus the tape, and StateTape is best in all 18 cells by 1.5 to 6.2 points; the last column of the A1.3 table gives each agent's stale share.
Response to Reviewer WmFk (1/4)
We thank the reviewer for the careful reading. We respond to each point below.
A1. "Current coding agents do not let the context rot: they have git, grep, LSP find-references, code-graph MCP, and compaction. Compaction was disabled, so Control is unfair. No baseline gets the code graph as a tool."
1. Retrieval tools do not prevent context rot. git, grep, and LSP re-fetch ground truth from the repository; context rot occurs inside the context window (stale conclusions, accumulated noise, attention dilution). Re-fetching only helps if the agent knows its belief is stale, which is precisely what a degraded context prevents. Hong et al. [Chroma, 2025] show performance degrades with input length across 18 frontier models even at fixed task difficulty; Laban et al. [arXiv:2505.06120] report a 39% average drop from single- to multi-turn settings, with no recovery after a wrong turn; Sinha et al. [arXiv:2509.09677] show models self-condition on their own prior errors, a degradation not fixed by scale. None of these is addressed by adding grep. Our paper targets this in-context mechanism: Section 3.1.2 and Figure 2 show the held context growing from 5.9k tokens before the first edit to 28.6k after the thirtieth, with 74% to 80% of it unneeded at every edit count, and 72.3% of that unneeded mass is stale because a later write falsified it. Those measurements were taken from Claude Code runs with grep, git, the file tools and the harness's default code-graph MCP server all available.
2. The "compaction is disabled" sentence is in the proof of Theorem 2(b) and describes StateTape runs only; compaction is on for Control and every baseline. Theorem 2(b) bounds StateTape's manager-to-agent token ratio, and Control appears in neither its statement nor its proof. Control is Claude Code and Codex at their shipped defaults, with the harness's own compaction on; no setting was modified or turned off (Appendix D). Compaction is switched off in the StateTape arm alone, so that the context there is maintained by the tape and the manager and by nothing else, and every gain in Table 1 is attributable to StateTape rather than to StateTape plus the harness's summarizer. The comparison is therefore the harness's full native context management against ours, and the only arm that gives up a harness feature is our own.
3. Every method in Table 1, Control included, already has the code graph as a tool. Control is Claude Code and Codex at their shipped defaults, and both harnesses ship with a code-graph MCP server in their default tool set, so every row of Table 1 could query the code graph at any step; "no baseline gets the code graph as a tool" does not describe our setup. The shim never removes a tool (Appendix D). To make the mechanism easier to follow, we additionally ran two configurations for the reviewer's reference: Control with compaction forced on every 40 turns, and the same with an added MCP server exposing the very symbol-level graph StateTape maintains (find-definition, find-callers, find-callees, symbols-changed-since-step) with no shim, so the agent can query the graph StateTape uses but nothing rewrites its context. We add the graph on top of forced compaction so that this is the strongest graph-access baseline we can build.