- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies stale references to files or file sections in the chat context of coding agents as a potential problem and proposes "LiveState," a context management tool to find and update stale file observations. LiveState replaces file reads in the context with a live updating panel containing one entry per file read and the latest text around the line ranges the agent has read. They obtain the latest updated state by relying on git to identify and replace stale files by reusing agent bash tool calls and adding hidden git commands to them (hidden in the sense that the agent never sees the git outputs) and updating the changed files among those that the agent has read. Their method is primarily positioned as an efficiency trick complementary to or stackable on top of relevance based compaction, since it might sometimes slightly hurt performance.
Critical strengths
- Sound experimentation. The authors evaluate the proposed method over multiple benchmarks and for strong frontier backbones
- Better efficiency. LiveState produces performance benefits in some cases (e.g. Gemini 3 Flash) and in almost all cases and reduces number of turns and tokens making it more efficient
Critical weaknesses
- Reported tokens/efficiency reductions won't translate. The main benefit of this paper is boosting efficiency by reducing number of tokens and turns used by the model but the authors themselves acknowledge that token savings might not translate to actual benefit since they lead to greater cache misses
- Missing crucial baselines/ablations. This paper explicitly positions itself as something that can be added on top of relevance based compaction. However there are no comparisons against it (performance vs token/turn reduction benefits) and whether once you stack it on top of compaction the token savings still persist or not. In general it is also not really compared to any other technique. All comparisons are just the base model with and without LiveState. It is hard to contextualize the benefits of this technique relative to past work.
- Lack of technical and analytical novelty. The core idea/approach of this paper is really simple context management. While in principle a simple inervention isn't inherently bad, but for them to be novel and interesting they should be surprising or lead to surprising outcomes and be accompanied with really insightful analysis. This paper misses on both counts. The results are not unexpected and the analysis doesn't uncover or explain any novel behavior shown by coding agents. This weakness is a bit more opinionated (and related to research taste) so feel free to discard this.
- Claudeisms in text. Uninformative AI sounding phrases like "accounting identity, not a causal split" spoil the reading experience and make it harder to understand the text. I highly encourage the authors to find these cases and rewrite them manually with more intentional and meaningful text.
Questions for authors
- Why don't you compare LiveState to compaction and/or do ablations to check if it stacks on top of it?
- Why don't you actually explore "realized efficiency," similar to TokenPilot (mentioned in the related work) to see if the logical token and turn savings actually translate to a meaningful benefit for deployment?