acceptodds
Under review as a conference paper at ICLR 2027

Context Rot in Agent Trajectories: Evolving-State Maintenance Is the Hidden Bottleneck of Long-Context Agents

Abstract

Long-context language models are increasingly deployed as agents whose context is not a static document but a self-generated multi-turn trajectory of (thought, action, observation) steps. We ask whether early key information is faithfully used as this trajectory grows. We introduce a controlled probe suite that holds task difficulty fixed while varying only trajectory length K in 1,...,128, across four task families that separate passive recall (key–value lookup, early constraints, multi-hop) from active evolving-state maintenance. Evaluating nine frontier models, we find a sharp dissociation: passive recall is largely robust (constraint and multi-hop stay at ceiling; KV recall degrades mildly), whereas evolving-state maintenance collapses catastrophically for every model once K >= 16 (early-information utilization drops from  1.0 to  0.0; all decay slopes lambda < 0). We further isolate an agent-specific corruption source: a model's own erroneous intermediate conclusion written into the trajectory (self-conflict) degrades performance significantly more than neutral filler of equal length (Delta = -0.150, 95% CI [-0.199,-0.102]); a controlled comparison shows this is driven by the misleading content rather than self-authorship per se, except for the strongest reasoner. Moreover 47.5% of these failures land exactly on the "snowball" value the model would obtain by faithfully continuing from its own error—an effect that worsens with reasoning strength. A two-phase mechanism emerges: shallow-K failures are dominated by self-generated-error snowballing (which a from-scratch recompute prompt partially repairs for weaker models, e.g. +0.333 for Claude Haiku), while deep-K failures are dominated by forgetting (which no training-free prompt repairs; a mechanism-targeted recompute recovers +0.211, 95% CI [+0.139,+0.289], in the shallow snowball regime but +0.007, not significant, at the deep cliff). We corroborate both the cliff and the self-error effect on genuine multi-turn rollouts in which each model self-generates its own trajectory: the collapse to EUR 0 reproduces, and rollouts containing a self-authored intermediate error fail 6.1x more often. Together these establish evolving-state maintenance as a stubborn, agent-specific long-context bottleneck and provide a first partial solution scoped to where it works.

Primary area:
datasets and benchmarks
Keywords:
long-context agents, benchmark, context rot, evolving-state maintenance, self-conflict, training-free mitigation, agent trajectory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_z9iMOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies context rot in agent trajectories across nine frontier models in four task families: three passive-recall tasks and one active state-tracking task. The authors find passive recall stays near the ceiling across lengths, but state tracking collapses near zero when K is larger than 16. Analysis indicates tool-reported errors disrupt models more than self-authored ones, and stronger reasoning models snowball more from their own mistakes. Prompt mitigations fail at deep context horizons (K >= 32).

Critical strengths
  1. Clear Problem Framing: Investigating performance degradation in self-generated agent trajectories is practically relevant, and separating passive recall from active state tracking offers a clear diagnostic framework.

  2. Insightful Error Analysis: Section 5 provides interesting counterfactual findings showing that tool-reported errors cause more disruption than self-authored ones, and reasoning models snowball more aggressively from early mistakes.

These two are the novelty and core contribution I particularly find in the paper

Critical weaknesses
  1. The task D is artificial framing; mental arithmetic is not how a real agent solves problems. The headline claims that "evolving-state maintenance is the hidden bottleneck of long-context agents" overgeneralizes what is essentially an in-context multi-step mental arithmetic stress test. it is neither a practical test nor utilizing any feature related to "agent". All of this conflates a known limitation in sequential numerical computation with a general theory of agent failure. The claim seemed to be too big to be supported by existing evidence.

  2. Lacking some Novelty and Deeper Theoretical/Mechanistic Insight: The performance cliff at deep K and the failure of prompt mitigations are predictable from existing theoretical frameworks: both from an agent cognitive load perspective (Shang et al., TMLR 2025) and from the architectural mechanics of how transformers track evolving states (Li et al., ICML 2025). Crucially, the paper merely documents this empirical drop without offering formal modeling or deeper theoretical insight into why this bottleneck occurs, leaving the contribution as an empirical report.

Shang, H., Liu, X., Liang, Z., Zhang, J., Hu, H., & Guo, S. (2025). United minds or isolated agents? Exploring coordination of LLMs under cognitive load theory. Transactions on Machine Learning Research (TMLR). arXiv:2506.06843.

Li, B. Z., Guo, Z. C., & Andreas, J. (2025). (How) do language models track state? In Proceedings of the 42nd International Conference on Machine Learning (ICML 2025). PMLR.

Questions for authors

Clarifying

  1. Beyond empirical reporting, can the authors provide some theoretical justification or explanation for this new phenomenon as suggested in the weaknesses? This could directly resolve part of my primary concern.
Reviewer_k1NSOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a controlled probe suite to test whether long-context agents continue to use early key information as their trajectories grow, separating passive recall from active evolving-state maintenance. Across nine frontier models and four task families, it finds that retrieval and constraint-following remain largely robust while state tracking collapses beyond a short horizon. The main contributions are the diagnostic suite, a large-scale evaluation of state maintenance as a key long-context failure mode, a proposed self-generated-error mechanism, and an evaluation of training-free remedies.

Critical strengths

The paper conceptually separates passive recall from active evolving-state maintenance and shows the dissociation across nine frontier models and four task families, with the core state-maintenance cliff reproduced on 22,317 genuine multi-turn rollouts. This is consequential because it surfaces a failure mode that static long-context benchmarks systematically under-measure, and the breadth of the evidence makes the phenomenon difficult to dismiss as a single-model or single-task artifact, even if its precise mechanism remains debated.

Critical weaknesses
  1. STATE-TRACK adds (K) arithmetic updates as (K) grows, whereas tasks A, B, and C require only retrieving a small fixed set of early facts. D therefore becomes a harder multi-step computation task, so its collapse may reflect running-sum/working-memory load rather than context rot or a distinct state-maintenance bottleneck. Without controls that separate computation from length, such as a last-event-only condition, restated totals, calculator access, or token-matched padding, the headline causal claim is not established.
  2. The main suite uses templated rather than model-generated trajectories, and no sufficiently matched static-document control is reported. It is therefore unclear whether the effect is agent-specific or simply long-context state tracking. The self-conflict condition also inserts a wrong total without establishing that it is the model’s own spontaneous error, and the rollout result is correlational.
Questions for authors
  1. Can you provide a STATE-TRACK control that separates computational load from trajectory length, for example (K) events where only the final event matters, a condition that restates the running total at each step, a calculator-enabled condition, or token-matched padding?
  2. How is the erroneous running total in the controlled self-conflict condition produced: by the same model in an independent rollout, by another model, or by a script? If it is not the model’s own spontaneous output, how do you justify the “self-authored” or “self-generated” interpretation?
  3. The real-rollout 6.1× comparison is correlational. Can you report a stratified or mixed-effects analysis controlling for model and (K), or a randomized error-injection experiment? Separately, what is the chance baseline for the 47.5% ±2 snowball hit rate?
  4. Beyond position and error value, are the self-conflict and external conditions matched on wording and length? Can you hold the wording constant and vary only the authority label, for example verified versus unverified tool, to separate source authority from role semantics?
  5. The metrics section says an LLM judge resolves ambiguous cases, while the LLM-use statement says no LLM judge was used. Which is correct? If a judge was used, please report the model/version, prompt, and agreement with human labels.
Reviewer_RisiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the phenomenon of "context rot", where agents' capability to recall and use information diminishes over the course of a trajectory. Specifically, the work studies the passive recall and evolving-state maintenance of nine frontier models. It finds that while the former is largely robust across context lengths, the latter collapses beyond a trajectory length of 16.

Critical strengths

The research question is interesting, and the proposed methodology of controlled task difficulty over varying lengths is largely well-motivated. The paper assesses three different types of task for the passive recall results, lending weight to the relevant findings.

Critical weaknesses

The paper is extremely challenging to read. The abstract and introduction repeatedly introduce terminology without prior definition (KV, snowball, recompute, etc.). Along similar lines, the plots are often unclear and messy. I would strongly recommend substantial revisions.

Further, the paper claims to hold task difficulty constant, but this is inconsistent with the fact that the fourth task family involves tasks whose difficulty scales with the trajectory length (i.e., adding more numbers together is presumably more difficult than adding less). This is a meaningful confounding factor that is not sufficiently addressed.

Lastly, figures in the text sometimes appear to contradict the plots. For example, section 4 suggests that Claude Sonnet and other models fall "to the floor by K=128", with multiple explicitly stated as going to 0.0, but figure 3 shows these models with EUR values above 0.6 at K=128.

Questions for authors
  • How do you control for the length confound described above for the state-tracking task family?
  • Can you clarify the apparent contradiction between the text and figure 3 in section 4?
  • Can you share data on the same experiments from more recent models that were available at the time of submission (e.g GPT-5.6 Sol, Claude Opus 5, etc.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.