acceptodds
Under review as a conference paper at ICLR 2027

StateTape: Action-Conditioned Evidence Lifecycle Modeling for Long-Horizon Coding Agents

Abstract

Despite the recent success of coding agents built on large language models, it remains challenging to run them over long horizons, since every observation is appended to the context and the context grows with each one. History-based maintenance is a common remedy, which masks or summarizes old observations, or prunes what a model reads as useless, and bounds the context at little cost. However, it decides from the text of the history alone and sees nothing of how the code is connected. Since a coding agent edits code many times over a single task, and each write can change what code elsewhere means, such maintenance may keep records a write has falsified, drop ones that still hold, and miss code the agent needs next. To overcome these challenges, this paper proposes StateTape, a novel and scalable framework that rewrites a coding agent's context as the repository changes rather than as the context grows. The key idea of StateTape is to model the repository as a symbol-level code graph, whose dependencies and language rules expose which symbols a write can affect. Upon this graph, a tape marks the symbols each write changed, which turns staleness from an inference about text into an observation of the agent's writes. We propose a per-write procedure in which the tape nominates the records a write could have falsified while a small manager model settles what the write log cannot, and further provide a theoretical analysis and TraceBench, a benchmark that labels what an agent is holding against what is actually needed. Empirically, we demonstrate that StateTape can effectively clear falsified records and retrieve what is needed, and thus achieve a higher resolve rate in all experiments spanned by six coding agents and three edit-heavy benchmarks with little computational overhead.

Primary area:
foundation or frontier models, including LLMs
Keywords:
coding agents, context management, long-horizon coding task, code graph

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WmFkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that a coding agent's own writes make its earlier reads stale, and text-based context management cannot see this. StateTape keeps a symbol-level code graph updated at every write, marks which symbols each write changed, and flags every held record that touches them. A small prompted LLM, the manager, then refreshes, drops or keeps flagged records, and pulls in code within two call hops of the changed symbols. TraceBench labels 403 pre-edit points from Claude Code runs on SWE-EVO and reports 73.4% of held tokens unneeded and 58.5% of needed tokens missing. On six agents in Claude Code and Codex CLI and three benchmarks, StateTape has the best resolve rate in all 18 cells at 0.8% to 4.7% manager overhead.

Critical strengths
  1. The paper formulates the problem that a coding agent can be fooled by context its own writes have revised, and demonstrates this phenomenon with Figure 1.
  2. The paper evaluates coding agents in realistic setup, Claude Code and Codex CLI, with LLMs of different size (Haiku 4.5, Sonnet 5, Opus 4.8; GPT-5.6 luna, terra, sol), on three edit-heavy benchmarks.
Critical weaknesses
  1. Current coding agents do not let the context rot as the paper assumes. Claude Code, Codex CLI and Cursor keep edit history through git, and analyze code through file-system and code tools such as grep, LSP find-references and code-graph MCP. They also compact the context when the window is full. In the experiments compaction is disabled (proof of Theorem 2(b)), so Control is the harness with its own context management turned off. I consider this an unfair comparison. No baseline gives the agent the same code graph as a tool either.
  2. The paper lacks novelty. Code graph as a solution to context localization has been studied for years, producing papers such as RepoGraph, LocAgent and CodexGraph. Context rot in LLMs is also a known problem with a long line of research. So the paper identifies a known problem and uses code graph as the main solution. I also do not see AI or ML rigor in the paper. Nothing is trained, and the manager is an off-the-shelf small LLM with a prompt. Most figures present logged numbers. The theoretical analysis proves properties of the code graph, which is not proposed by the paper but used out of the box, and does not connect to resolve rate.
Questions for authors
  1. When applying coding agents as baseline, did you enable harness native features, such as compaction, git history, and code tools like grep and LSP find-references?
  2. What is the tape's advantage over git history management? In my view tape is a stripped-down version of it. git diff already tells the agent which files and lines a write changed, and the agent can read the previous version at any time.
4 comments
AuthorsOpenReview ↗

Response to Reviewer WmFk (1/4)

We thank the reviewer for the careful reading. We respond to each point below.


A1. "Current coding agents do not let the context rot: they have git, grep, LSP find-references, code-graph MCP, and compaction. Compaction was disabled, so Control is unfair. No baseline gets the code graph as a tool."

1. Retrieval tools do not prevent context rot. git, grep, and LSP re-fetch ground truth from the repository; context rot occurs inside the context window (stale conclusions, accumulated noise, attention dilution). Re-fetching only helps if the agent knows its belief is stale, which is precisely what a degraded context prevents. Hong et al. [Chroma, 2025] show performance degrades with input length across 18 frontier models even at fixed task difficulty; Laban et al. [arXiv:2505.06120] report a 39% average drop from single- to multi-turn settings, with no recovery after a wrong turn; Sinha et al. [arXiv:2509.09677] show models self-condition on their own prior errors, a degradation not fixed by scale. None of these is addressed by adding grep. Our paper targets this in-context mechanism: Section 3.1.2 and Figure 2 show the held context growing from 5.9k tokens before the first edit to 28.6k after the thirtieth, with 74% to 80% of it unneeded at every edit count, and 72.3% of that unneeded mass is stale because a later write falsified it. Those measurements were taken from Claude Code runs with grep, git, the file tools and the harness's default code-graph MCP server all available.

2. The "compaction is disabled" sentence is in the proof of Theorem 2(b) and describes StateTape runs only; compaction is on for Control and every baseline. Theorem 2(b) bounds StateTape's manager-to-agent token ratio, and Control appears in neither its statement nor its proof. Control is Claude Code and Codex at their shipped defaults, with the harness's own compaction on; no setting was modified or turned off (Appendix D). Compaction is switched off in the StateTape arm alone, so that the context there is maintained by the tape and the manager and by nothing else, and every gain in Table 1 is attributable to StateTape rather than to StateTape plus the harness's summarizer. The comparison is therefore the harness's full native context management against ours, and the only arm that gives up a harness feature is our own.

3. Every method in Table 1, Control included, already has the code graph as a tool. Control is Claude Code and Codex at their shipped defaults, and both harnesses ship with a code-graph MCP server in their default tool set, so every row of Table 1 could query the code graph at any step; "no baseline gets the code graph as a tool" does not describe our setup. The shim never removes a tool (Appendix D). To make the mechanism easier to follow, we additionally ran two configurations for the reviewer's reference: Control with compaction forced on every 40 turns, and the same with an added MCP server exposing the very symbol-level graph StateTape maintains (find-definition, find-callers, find-callees, symbols-changed-since-step) with no shim, so the agent can query the graph StateTape uses but nothing rewrites its context. We add the graph on top of forced compaction so that this is the strongest graph-access baseline we can build.

AuthorsOpenReview ↗

Response to Reviewer WmFk (2/4)

Resolve rates (SWE-EVO / SlopCode / SWE-bench Pro); the best cell per agent and suite is in bold.

Main agentMethod, resolve rate (%)SWE-EVOSlopCodeSWE-ProCalls to the added StateTape-graph server per trajectoryShare of graph calls for localizationStale share of held tokens
Haiku 4.5Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)25.04.626.3——74.6%
Control + compaction forced on (every 40 turns)25.05.126.5——72.9%
Control + StateTape's graph as MCP tool + forced compaction27.16.126.94.788%70.3%
StateTape (harness compaction off)29.28.127.8——17.2%
Sonnet 5Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)41.79.263.5——71.8%
Control + compaction forced on (every 40 turns)41.79.763.9——69.5%
Control + StateTape's graph as MCP tool + forced compaction43.810.765.06.185%66.4%
StateTape (harness compaction off)47.912.868.4——14.8%
Opus 4.8Control (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)58.39.769.2——70.2%
Control + compaction forced on (every 40 turns)58.310.269.5——67.8%
Control + StateTape's graph as MCP tool + forced compaction60.410.770.26.982%64.5%
StateTape (harness compaction off)62.512.871.4——13.9%
GPT-5.6 lunaControl (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)56.34.164.3——73.1%
Control + compaction forced on (every 40 turns)56.34.664.7——70.6%
Control + StateTape's graph as MCP tool + forced compaction58.35.165.85.287%67.7%
StateTape (harness compaction off)60.46.667.7——16.1%
GPT-5.6 terraControl (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)52.16.665.8——72.4%
Control + compaction forced on (every 40 turns)52.16.666.2——70.1%
Control + StateTape's graph as MCP tool + forced compaction54.27.167.05.885%67.0%
StateTape (harness compaction off)58.38.268.4——15.3%
GPT-5.6 solControl (harness defaults: compaction on; grep, read, Bash, git and the default code-graph MCP available)58.38.264.6——70.9%
Control + compaction forced on (every 40 turns)58.38.765.0——68.5%
Control + StateTape's graph as MCP tool + forced compaction60.49.266.66.683%65.4%
StateTape (harness compaction off)62.510.269.5——14.4%

Note: — means the added StateTape-graph server is not present in this configuration, so there are no calls to count. Under StateTape the agent never queries the graph; the tape consults it after every write to compute drift sets (Section 3.3.2). The harness's default code-graph MCP remains available in every row and is not counted in these two columns.

Exposing StateTape's own graph as a tool and forcing compaction leave the problem in place for every agent.

  • (a) The context still rots. Together they cut the stale share by only 4.3 to 5.7 points (Sonnet 5: 71.8% to 66.4%) and recover 0.5 to 2.1 of the 1.5 to 6.2 points StateTape gains over Control, at most half of the gain in any of the 18 cells.
  • (b) The agent uses the graph to find where to edit, not what its edit invalidated. The stale share alone cannot tell whether the agent ignores the tool or calls it for a different purpose; the two call columns separate these cases. The agent calls the tool 4.7 to 6.9 times per trajectory, so it is not ignored. But 82% to 88% of those calls are for localization: they query a symbol that no earlier write has changed. Only the remaining 12% to 18% query a symbol in the drift set of an earlier write, and so could ask what the agent's own edit invalidated. Re-fetching helps only when the agent knows a belief is stale (A1.1).
  • (c) Compaction does not fill the gap; StateTape does. A summarizer cannot tell a stale read from a fresh one. StateTape cuts the stale share to 13.9% to 17.2% with zero agent-side calls, because it does not wait for a question the agent does not know to ask.
AuthorsOpenReview ↗

Response to Reviewer WmFk (3/4)

A2. "Lack of novelty: code graphs for localization exist (RepoGraph, LocAgent, CodexGraph) and context rot is known."

1. Prior code-graph work uses the graph at read time to find code; StateTape uses the graph's change at write time to invalidate what is already held. RepoGraph, CodexGraph and LocAgent build a graph at task start and traverse it to localize (Section 2); none re-derives it after the agent's own write. StateTape's new objects are the drift sets (Section 3.3.2) and the nomination rule (Eq. 7), which map a write to the held records it falsifies with no model call. Table 3 isolates this: "w/o tape (text-judged staleness)" keeps the graph, and CE falls from 6.18 to 2.86, within 0.1 of evict-all.

2. The claim is not that context rot exists, but that the agent's own writes are a cause that can be observed rather than inferred. Masking, summarization, pruning, AgentDiet, SWE-Pruner, AdaCoM and ContextCurator (Section 2) all judge staleness from the text of the history. We judge it from the write log, where 72.3% of unneeded held tokens originate (Table 5).

3. The contributions beyond the graph are the ones the reviewer's own summary lists. (i) The action-conditioned formulation of staleness with its monotonicity property (Eq. 1); (ii) TraceBench, the first benchmark that labels a context state at a decision point rather than a terminal patch (Table 6); (iii) the two-sided CE/RE objective (Eqs. 2 to 5), which Appendix E shows F1 cannot replace; (iv) the tape-plus-manager division of labour.


A3. "No AI or ML rigor: nothing is trained, the manager is an off-the-shelf small LLM with a prompt, figures are logged numbers, and the theory is about the code graph and does not connect to resolve rate."

1. Training-free is a design choice that every baseline in Table 1 shares. Observation masking, LLM summary, CORVUS, AgentDiet-style managers, AdaCoM and ContextCurator all keep the base model frozen and ask what to put in front of it. A trained component would make the method harness- and model-specific, the opposite of what a drop-in shim for six models across two vendors needs; rigor here is a well-posed objective, a sound procedure, a benchmark that measures the right quantity and controlled comparisons, and the paper supplies each.

2. The theorems are about the tape and the manager, not about a pre-existing code graph. Theorem 1(a) guarantees that under symbol locality (Assumption 1) every stale record is nominated at the write that falsifies it; 1(b) bounds the stale mass after any manager output by the records the manager kept; 2(a) bounds the tape's cost without model calls; 2(b) bounds the manager's token share; 2(c) shows stale tokens sent grow as Ω(δW²) without maintenance. These are properties of the drift sets and nomination rule of Section 3.3.2, not of anyone's graph, and their empirical counterparts are Table 4 (manager share 0.8% to 4.7%, matching 2(b)) and Table 2 (CE 6.18 against 2.75 for evict-all, matching 1(b)).

3. On "does not connect to resolve rate." Resolve rate is a property of a frozen black-box model's behaviour, so no context-maintenance paper proves a theorem about it; the honest intermediate quantity is how much of the context is false and how much of what is needed is present, which is exactly what Theorem 2(c) bounds. The connection to resolve rate is empirical: Table 3 removes components one at a time and CE and retrieval coverage fall monotonically with resolve rate on all three benchmarks, Table 2 shows the same metrics separate StateTape from 13 reference policies, and Section 3.1.2 shows the stale share growing with edit count as Ω(δW²) predicts.

4. "Most figures present logged numbers." The figures report measurements of a system on benchmarks, which is what an empirical paper's figures do. The additional analyses the reviewers asked for are in this response (graph-as-tool and compaction baselines in A1.3, the stale share measured with git available in Q2.2) and in our responses to Reviewers 1RuA (three-seed variance, Table R1) and Tgkp (cache-aware cost accounting, Table R2).

AuthorsOpenReview ↗

Response to Reviewer WmFk (4/4)

Q1. "Did you enable harness-native features such as compaction, git history, and code tools like grep and LSP find-references?"

1. Tools: yes, all of them, for every method. The shim intercepts only the model request; the harness, its tool registry, the container and the repository's git history are untouched (Appendix D), so the agent in every row of Table 1 can run grep, git diff, git show, the code-graph MCP server that ships with each harness and any language server it would normally run.

2. Compaction: yes, enabled for Control and every baseline; it is off only under StateTape. Control runs Claude Code and Codex exactly as shipped, with the harness's own compaction on, and it fired in 3 of 48 Control trajectories on SWE-EVO with Sonnet 5. StateTape is the one arm that runs without it, so that its context is maintained by the tape and the manager alone and its gains are not shared with the harness's summarizer (A1.2). The LLM-summary baseline applies a more aggressive compaction than the harness default and loses to StateTape in every cell, and A1.3 reports compaction forced on.

3. Under StateTape the context window limit is never hit, so running that arm without compaction costs it nothing. StateTape removes a large portion of the unneeded context (74.2% of it, Table 2), which contributes nothing to the resolve rate. Its context therefore never reaches the window limit in any run, and the harness's compaction would have had nothing to compact.


Q2. "What is the tape's advantage over git history management? git diff already tells the agent which files and lines a write changed, and the agent can read the previous version at any time."

1. git diff reports which lines changed; staleness travels along the edges of the code graph to lines that did not change. Take a call chain A → B → C: function A calls B, and B calls C. The agent has read all three and has run A and B, so it holds their outputs as evidence. It then modifies C. git diff shows the changed lines of C and nothing else, because the text of A and B is untouched. Yet the outputs of A and B that the agent holds are no longer valid, since both were computed through the old C. These results must be rerun or marked as stale, and no git command says so: git has no symbol table and no call edges, so it cannot link a write on C to A and B. StateTape knows that A calls B and B calls C. When the agent modifies C, StateTape follows these calls backwards from C, finds B and A, and checks whether the agent still holds anything it learned about them. It does, so those results are marked as stale or refreshed before the agent's next step (Section 3.3.2). git tells the agent what it changed; StateTape tells the agent what it can no longer trust.

2. The agent in every measured run already had git, and 72.3% of the unneeded tokens it held were stale anyway. Section 3.1.2 and Table 5 were measured on Claude Code with no shim and with git and the file tools available: across 48 trajectories, 74% to 80% of held context is unneeded, and 72.3% of that is stale because the agent's own later write falsified it (Figure 2). The agent could have run git diff after any of those writes; it did not, because nothing told it a held belief had become false. Table 1 is therefore already the requested comparison: Control has git, StateTape has git plus the tape, and StateTape is best in all 18 cells by 1.5 to 6.2 points; the last column of the A1.3 table gives each agent's stale share.

Reviewer_1RuAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies context maintenance for long-horizon coding agents. Existing history-based methods decide only from the text of the history, so they cannot tell which records a write has made stale. This paper proposes StateTape, which rewrites the context as repo changes by using a symbol level code graph. A rule-based tape compares the graph before and after each write and flags records that may have become stale. A small manager model then decides what to refresh or drop and retrieves related code along call edges. It also proposes TraceBench, which annotates what an agent is holding against what it needs. Experiments show higher resolve rates than all baselines, with small overhead.

Critical strengths
  1. This paper studies an important and under-explored problem: a coding agent’s own writes can make its held context stale. Instead of inferring staleness from text, StateTape detects it directly from the diff of a symbol-level code graph at each write.

  2. TraceBench is a useful testbed. It makes it easy to measure both what the agent holds but does not need and what it needs but does not hold. The method is also described clearly and in detail.

Critical weaknesses
  1. The SWE-EVO evaluation contains only 48 instances, and no variance or multiple runs are reported. Many gains correspond to only 1/2 instances, so it is hard to tell whether the improvements are significant.

  2. TraceBench uses silver labels from an LLM, without human validation. It is also built from the same SWE-EVO instances used in the evaluation. Agreement with manual annotation on a subset would make it more convincing.

Questions for authors
  1. Can you report results over multiple runs over different seeds?
3 comments
AuthorsOpenReview ↗

Response to Reviewer 1RuA (1/2)

We thank the reviewer for the careful reading. We respond to each point below.


A1. "SWE-EVO contains only 48 instances with no variance or multiple runs; many gains correspond to 1 or 2 instances."

1. SWE-EVO is one of three suites, and the other two are much larger. SlopCodeBench scores 196 checkpoints and SWE-bench Pro 731 instances, so one instance moves the rate by 0.5 and 0.1 points. StateTape's average gains there (+2.7 and +3.3 points, up to +3.6 and +4.9 for Sonnet 5) correspond to roughly 5 to 7 checkpoints and 24 to 36 instances.

2. The evidence is the consistency across 18 cells, not any single cell. StateTape is the best of six methods in all 18 agent–benchmark cells of Table 1 and beats Control in every one. Under the null that the two are exchangeable, beating Control for all six agents on one benchmark has probability 2^-6 = 0.016 by a sign test, and this holds on all three benchmarks.

3. Multi-seed runs. Table R1 (Q1) reports mean ± standard deviation over three seeds for all six configurations on all three suites. StateTape's mean gain over Control is positive in all 18 cells, at least one seed-to-seed standard deviation in all 18 and at least two in 15 of them.


A2. "TraceBench uses silver labels from an LLM without human validation, and is built from the same SWE-EVO instances used in evaluation."

1. TraceBench labels are rule-anchored and LLM-annotated. All four gold rules in Appendix E are anchored to the reference patch and the code graph: G1 (edit site) and G3 (blast radius, 1,338 regions) are computed directly from the patch and call graph, and G2 (referenced definition) and G4 (contract, 595 regions) are annotated by gpt-5.4 (seed 0) with the patch, the graph neighbourhood and the rule definition in context. This is a narrow labelling task with the full evidence in context, the regime in which frontier models are highly reliable annotators.

2. No number in Table 1 depends on TraceBench labels. Resolve rate is scored by each benchmark's own hidden tests; TraceBench serves only the diagnostic metrics of Tables 2 and 3 and the motivation in Section 3.1. Nothing in StateTape is trained, tuned or thresholded on the labels: the tape is deterministic (Section 3.3.2), the manager is a prompted off-the-shelf model (Section 3.3.3) and retrieval is a fixed two-hop walk, so sharing instances with the evaluation cannot inflate Table 1.

AuthorsOpenReview ↗

Response to Reviewer 1RuA (2/2)

Q1. "Can you report results over multiple runs with different seeds?"

Yes. Table R1 reports all six configurations on all three suites over three seeds.

Table R1. Resolve rate (%) on SWE-EVO (48 instances), SlopCodeBench (196 scored checkpoints) and SWE-bench Pro (731 instances), mean ± sample standard deviation over three seeds.

Main agentConfigurationSWE-EVOSlopCodeSWE-Pro
Haiku 4.5Control25.0 ± 2.15.1 ± 0.526.3 ± 0.2
Obs. masking26.4 ± 3.25.1 ± 1.027.1 ± 0.3
LLM summary26.4 ± 2.46.1 ± 1.026.7 ± 0.3
CORVUS24.3 ± 3.24.9 ± 0.626.4 ± 0.1
Text-only mgr27.1 ± 2.17.3 ± 0.827.1 ± 0.3
StateTape29.2 ± 2.18.0 ± 0.827.9 ± 0.1
Sonnet 5Control44.4 ± 2.49.2 ± 1.063.5 ± 0.3
Obs. masking43.1 ± 1.29.7 ± 0.565.2 ± 0.2
LLM summary44.5 ± 3.29.7 ± 1.063.9 ± 0.2
CORVUS45.1 ± 3.211.9 ± 0.663.6 ± 0.1
Text-only mgr41.7 ± 2.110.7 ± 1.065.5 ± 0.2
StateTape50.0 ± 2.112.8 ± 1.168.5 ± 0.2
Opus 4.8Control55.6 ± 2.49.7 ± 1.069.2 ± 0.3
Obs. masking57.6 ± 3.211.0 ± 0.369.2 ± 0.3
LLM summary59.0 ± 3.211.2 ± 1.069.6 ± 0.2
CORVUS62.5 ± 2.110.2 ± 0.569.2 ± 0.2
Text-only mgr59.7 ± 2.411.7 ± 0.568.3 ± 0.2
StateTape63.2 ± 2.412.6 ± 0.871.2 ± 0.2
GPT-5.6 lunaControl54.9 ± 1.23.9 ± 0.864.3 ± 0.3
Obs. masking60.4 ± 2.14.1 ± 0.565.8 ± 0.3
LLM summary56.3 ± 2.15.1 ± 1.064.7 ± 0.3
CORVUS61.1 ± 2.45.4 ± 0.664.2 ± 0.1
Text-only mgr59.0 ± 3.24.6 ± 0.566.3 ± 0.2
StateTape62.5 ± 2.16.6 ± 1.167.9 ± 0.2
GPT-5.6 terraControl50.0 ± 2.16.8 ± 0.865.8 ± 0.3
Obs. masking55.6 ± 1.27.8 ± 0.667.7 ± 0.3
LLM summary54.9 ± 3.27.7 ± 0.666.1 ± 0.2
CORVUS50.0 ± 2.17.1 ± 0.665.7 ± 0.2
Text-only mgr54.2 ± 2.16.6 ± 0.567.7 ± 0.3
StateTape58.3 ± 4.28.7 ± 0.568.3 ± 0.2
GPT-5.6 solControl58.3 ± 4.28.0 ± 0.864.6 ± 0.3
Obs. masking58.3 ± 2.19.2 ± 0.566.1 ± 0.2
LLM summary58.3 ± 2.19.2 ± 0.567.9 ± 0.2
CORVUS57.0 ± 1.29.2 ± 0.567.4 ± 0.2
Text-only mgr58.3 ± 2.19.7 ± 1.068.0 ± 0.1
StateTape62.5 ± 2.110.0 ± 0.869.3 ± 0.2

Thank you for providing additional results. I'll maintain my original rating.

Reviewer_TgkpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes StateTape to combat the stale tokens in the context and also missing relevant context in coding tasks. StateTape models the repository as a symbol-level code graph, whose dependencies and language rules expose which symbols a write can affect. The authors also release TraceBench which is a benchmark that labels what an agent is holding against what is actually needed. The authors report higher resolve rate for StateTape comparing to other agents.

Critical strengths
  • This paper is working on improving the quality of agent context for coding tasks which is an important problem. Several studies has shown that LLMs perform better when their context is more relevant meaning it includes the information the LLM needs and does not include extra information.
  • The paper compares mainstream agents such as claude code and codex cli with their StateTape design.
  • The paper includes an ablation study which shows the call graph information is the most useful information.
Critical weaknesses
  • The paper doesn't consider token caching and the balance between token cost and precision. Does earning a few points on precision on some specific tasks really worth the cache misses introduced by StateTape?
  • The paper focuses on a very specific problem, where call graph info is relevant such as the example given in the paper. The TraceBench is also focused on those types of problems and StateTape also targets those problems, therefore a little increase in resolution rate is not surprising. It would be useful to know if the published result are on full set of problems in these benchmarks or a subset.
Questions for authors
  • Can you include the $ cost for these experiments so that we can understand whether this worth cache misses or not?
  • Can you specify whether the published results are on the full set of problems in the benchmarks or a subset?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Tgkp (1/3)

We thank the reviewer for the careful reading. We respond to each point below.


A1. "The paper doesn't consider token caching and the balance between token cost and precision."

1. A rewrite breaks the cache only in the round after it; every other round of the agentic loop is cached exactly as in Control. The tape runs only when the agent writes; reads, searches, tests and messages append to an unchanged history and reuse the cached prefix. After a write the nominated records are rewritten in place, so the next request re-processes the context once and re-caches it; the request after it hits again.

2. Write rounds are a small fraction of the total request rounds. On TraceBench a trajectory has a median of 71 steps and 11.5 edit bursts, so about one round in six follows a write and the other five append to an unchanged history. Table R1 gives the same share for each main agent in the Table 1 runs.

Table R1. Rounds in which StateTape rewrote the context, per trajectory, all three suites.

Main agentRounds following a write (context rewritten)All request roundsShare (%)
Haiku 4.517.69618.3
Sonnet 513.97817.8
Opus 4.810.86416.9
GPT-5.6 luna14.18317.0
GPT-5.6 terra11.26916.2
GPT-5.6 sol9.66115.7

3. The tokens a cache break re-processes are a small fraction of the input, so the cost of breaking is marginal. In Table R2, StateTape's uncached input exceeds Control's by 1.4% to 3.6% of StateTape's total input tokens (2.0% over all 18 cells), and uncached tokens remain 15% to 18% of its input against 10% for Control, so the cache still serves more than 80% input tokens. Against this, the rewritten context is shorter and StateTape reads 28% to 29% fewer input tokens than Control in every cell, an order of magnitude more than the extra uncached volume, which is why the bill in point 4 does not rise.

4. The dollar cost barely moves while the number of resolved problems rises substantially. In Table R2, summed over the 18 cells, StateTape's bill is 1.5% below Control's, and no single cell is more than 6.3% above it (13 of 18 are at or below), whereas StateTape resolves 3% to 78% more problems in every cell and 6.7% more overall. The cached input itself is substantially smaller: because the stale and unneeded tokens are removed from the context instead of being re-read from the cache in every later round, StateTape's cached input is 33% to 34% below Control's in every cell (3,955M against 5,919M tokens over the 18 cells), a saving 21 times the extra uncached volume of point 3.

AuthorsOpenReview ↗

Response to Reviewer Tgkp (2/3)

Table R2. Cache-aware token cost and resolved problems for each main agent and benchmark. Δ rows give StateTape relative to Control, in percent.

Main agentBenchmarkMethodCached input (M)Uncached input (M, cache writes included)Output (M)Cost ($)Resolved problems
Haiku 4.5SWE-EVOControl255.528.44.181.512
StateTape172.331.23.484.914
Δ StateTape vs Control (%)+4.2+16.7
SlopCodeBenchControl650.472.310.5207.99
StateTape437.279.510.0220.916
Δ StateTape vs Control (%)+6.3+77.8
SWE-bench ProControl1283.3142.520.9411.0192
StateTape862.4156.820.1416.4203
Δ StateTape vs Control (%)+1.3+5.7
Sonnet 5SWE-EVOControl229.425.53.7146.620
StateTape153.628.93.3144.923
Δ StateTape vs Control (%)−1.2+15.0
SlopCodeBenchControl711.179.011.5454.718
StateTape475.789.710.5431.625
Δ StateTape vs Control (%)−5.1+38.9
SWE-bench ProControl902.6100.314.7578.3464
StateTape605.1113.911.9555.9500
Δ StateTape vs Control (%)−3.9+7.8
Opus 4.8SWE-EVOControl95.810.61.5151.728
StateTape63.612.71.3146.830
Δ StateTape vs Control (%)−3.2+7.1
SlopCodeBenchControl116.412.91.9186.319
StateTape77.315.41.6182.025
Δ StateTape vs Control (%)−2.3+31.6
SWE-bench ProControl548.661.08.9878.0506
StateTape363.172.78.4872.5522
Δ StateTape vs Control (%)−0.6+3.2
GPT-5.6 lunaSWE-EVOControl86.89.61.45.827
StateTape57.112.11.15.529
Δ StateTape vs Control (%)−5.2+7.4
SlopCodeBenchControl117.613.11.97.98
StateTape77.116.41.78.213
Δ StateTape vs Control (%)+3.8+62.5
SWE-bench ProControl254.728.34.117.1470
StateTape167.235.43.717.2495
Δ StateTape vs Control (%)+0.6+5.3
GPT-5.6 terraSWE-EVOControl54.66.10.937.025
StateTape36.27.01.036.628
Δ StateTape vs Control (%)−1.1+12.0
SlopCodeBenchControl94.410.51.563.113
StateTape62.712.21.662.716
Δ StateTape vs Control (%)−0.6+23.1
SWE-bench ProControl186.220.63.0124.7481
StateTape123.624.03.2124.0500
Δ StateTape vs Control (%)−0.6+4.0
GPT-5.6 solSWE-EVOControl53.66.00.969.428
StateTape35.66.90.966.930
Δ StateTape vs Control (%)−3.6+7.1
SlopCodeBenchControl91.310.11.5117.016
StateTape60.611.71.6115.820
Δ StateTape vs Control (%)−1.0+25.0
SWE-bench ProControl186.420.73.0238.1472
StateTape124.124.02.9228.5508
Δ StateTape vs Control (%)−4.0+7.6
AuthorsOpenReview ↗

Response to Reviewer Tgkp (3/3)

A2. "The paper focuses on a very specific problem where call-graph info is relevant; TraceBench and StateTape both target those problems, so a small increase is not surprising."

1. The problem is long-horizon coding itself, not a problem class in which call graphs happen to matter. The only condition StateTape relies on is that the agent writes heavily: a write is the sole event that can falsify a held record, so the stale set grows with the number of writes, and that is what defines a long-horizon coding task. The call graph is a property of code rather than of particular tasks, since every repository has definitions and calls between them and the symbol is the unit at which a write changes meaning. Figure 1 illustrates the mechanism; it does not describe the problem class.

2. The three evaluation benchmarks were built by others, are defined by horizon rather than by any call-graph property, and are scored by their own hidden tests. SWE-EVO (Le et al., 2025), SlopCodeBench (Orlanski et al., 2026) and SWE-bench Pro (Deng et al., 2025) are third-party suites chosen because their tasks require many edits (Section 4.1); each is run on its full released set, so no instance was chosen for call-graph relevance or for any other reason (Q2), and every method sees the same instances in the same containers (Appendix D). StateTape has the highest resolve rate in all 18 agent–benchmark cells (Table 1), including SlopCodeBench and SWE-bench Pro, which share no instance with TraceBench.

3. TraceBench is the diagnosis, not the evaluation, and it was measured on an agent nothing of ours touched. It is built from unmodified Claude Code runs on 48 SWE-EVO instances, cut at 403 pre-edit decision points, with gold regions anchored on the reference patch rather than on the code graph (Section 3.1.1, Appendix E). On this data 73.4% of held tokens are unneeded, 72.3% of those are stale, and 58.5% of the tokens the next edit needs are missing (Table 5). That the missing evidence lies a call edge from the edit site (2,721 of 3,545 gold regions, Section 4.3) is a finding of TraceBench, not a design choice of it, and no number in Table 1 depends on a TraceBench label.

4. The gain is consistent across every setting and larger than that of every baseline built for the same long-horizon problem. Averaged over the six agents, StateTape adds 4.9, 2.7 and 3.3 points over Control on SWE-EVO, SlopCodeBench and SWE-bench Pro, while the best of the four baselines, all of which target long-horizon context and run in the same shim, adds 2.1, 1.6 and 2.0 (Table 1). StateTape beats Control in all 18 cells; under exchangeability, beating Control on all six agents of one benchmark has probability 2^-6, and this happens on all three. The ablation (Table 3) shows where the gain comes from: replacing symbols by files halves CE to 3.14, drops coverage to 44.3% and costs 3.4 points of resolve rate.


Q1. "Can you include the $ cost for these experiments?"

1. Yes; Table R2 in A1 reports it for every main agent and benchmark. For each of the 18 agent-benchmark cells it lists, for Control and StateTape, the cached input, uncached input (cache writes included) and output tokens of the Table 1 runs, the dollar cost at list prices with the manager's tokens included, and the number of resolved problems. Cache reads and cache writes are priced at their own rates, so the cache misses raised in W1 are in the bill.

2. The gains do not come at a higher cost. StateTape's bill is at or below Control's in 13 of the 18 cells and at most 6% above it in the other five (Haiku 4.5 on all three benchmarks, GPT-5.6 luna on SlopCodeBench and SWE-bench Pro). Per resolved problem StateTape costs 4% to 40% less than Control in every cell. We will add Table R2 and its pricing basis to the paper next to Table 4.


Q2. "Are the published results on the full set of problems or a subset?"

1. SlopCodeBench: full set. All 196 scored checkpoints of all 36 problems (Table 4 caption).

2. SWE-EVO: all 48 instances from seven repositories in the released evaluation split.

3. SWE-bench Pro: full set. All 731 instances of the public split, with no instance filtered out; the resolved counts in Table R2 are out of these 731.

4. Why edit-heavy suites. The paper's claim is about what happens when an agent writes repeatedly (Section 3.2). A suite on which the agent typically makes one write offers nothing for any context-maintenance method to do, including the baselines; this is why all five baselines also move on these suites. We will state the instance count of each suite in Section 4.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.