acceptodds
Under review as a conference paper at ICLR 2027

Towards Robust Long-Lived Agents: Benchmarking Experience Lifecycles in Stateful Environments

Abstract

Long-lived agents must discover how an unfamiliar environment works and keep the resulting experience useful as conditions change. Existing benchmarks have expanded from historical information retrieval to interactive experience reuse and adaptation, yet maintaining experience under costly exploration and unannounced changes remains underexplored. We introduce ReScopeBench, a benchmark connecting experience acquisition, reuse, invalidation, and revision across 18 interactive environments. Our experience-first construction organizes 16-task streams around structural experience about resource relationships and rule experience about local operating conditions. Agents pursue varied goals under explicit exploration budgets, while silent changes invalidate selected dependencies and preserve others. Paired evaluations with and without retained experience measure benefits, disruption, and recovery through objective outcome scoring. Experiments with seven models show that experience acquired through interaction improves subsequent task performance, but silent environmental changes erode these gains and recovery varies substantially across models. Trace analyses further reveal a gap between retaining valid experience and applying it effectively. ReScopeBench makes maintaining actionable experience an explicit target for long-lived agent evaluation.

Primary area:
datasets and benchmarks
Keywords:
Language Agents, Experience Reuse, Continual Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_azezOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ReScopeBench to evaluate agents' ability to acquire, reuse, and revise experience in stateful environments while maintaining its validity. The benchmark contains 18 simulated environments, each with a sequence of 16 tasks, for a total of 288 tasks. Tasks are organized around shared structural and rule experience, and agents must discover relevant information within limited budgets of environmental resources. Each task stream undergoes a local change before task 9, without an announcement to the agent. The change invalidates some experience while leaving other dependencies intact, and subsequent tasks provide opportunities for revision and reuse.

Evaluation runs two task streams from the same initial state: one retains experience, while the other clears it before each task. Scores based on actual delivered outcomes measure the benefits of experience, the impact of changes, and subsequent recovery. The paper compares seven model deployment configurations. The results show that experience acquired through interaction can improve performance on later tasks, but environmental changes reduce these gains, and recovery varies across configurations. Analyses of tool use and limited content audits also show that fewer interactions do not necessarily indicate more effective experience reuse, and that retaining correct experience does not mean it is actually used during execution.

Critical strengths
  1. The paper makes the continued applicability of experience its main evaluation target. Exploration and verification consume environmental resources that are also needed to complete tasks, requiring agents to balance direct reuse against rechecking. Local changes require them to revise the parts that have become invalid while preserving unaffected experience. These design choices focus on maintaining usable experience during sustained operation, extending beyond retrieval, restatement, or the continued accumulation of historical information. I consider this the paper's strongest motivation.

  2. The authors make several useful distinctions in task construction and result interpretation. A successful reference execution does not establish that an agent can discover a solution through publicly available observations. Scoring should also be based on actual delivered outcomes and allow different valid completion paths. Accordingly, the paper describes discoverability checks, validation of recovery after changes, and outcome-based scoring. It also distinguishes a reduction in the benefits of experience after a change from experience becoming harmful relative to the condition without it. Likewise, retaining correct experience and using that experience during execution are separate issues. These distinctions help avoid overly simplistic interpretations of the role of experience based only on declining performance or memory contents.

Critical weaknesses
  1. The contribution beyond existing work is not sufficiently established. EvoArena already covers changes in external workflows, dependencies, interfaces, permissions, and other operating conditions. The need to update experience after an environmental change is therefore not, by itself, a sufficient distinction. Combining resource budgets, unannounced local changes, and selective revision could add evaluation value, but the paper lacks concrete task materials showing how these requirements arise in practice. The authors describe requirements for budget calibration, discoverability checks, and validation of recovery paths, but do not provide a complete account connecting actual tasks, observable feedback, invalidated and preserved dependencies, resource consumption, and scores. I therefore cannot readily verify whether repeated exhaustive exploration is genuinely unaffordable, whether local investigation and repair are feasible, or whether later tasks actually test reuse of the revised experience. Such materials would provide direct evidence that the additional evaluation requirements are implemented; their role goes beyond illustrating an example for readers. The current results do not yet show which capability differences the benchmark can identify that existing evaluations have difficulty distinguishing.

  2. The experience management mechanisms and control conditions are insufficiently specified, and the validation is limited, making it difficult to draw clear conclusions about experience maintenance. The condition with retained experience includes history, memory, and skills, and the behavioral analysis also mentions background consolidation. However, the paper does not clearly explain how these are generated, retrieved, and updated, or how context overflow is handled. The condition without retained experience still preserves environment-managed observation and capability records. Their contents, whether they can be queried, and how they differ from the cross-task information that is cleared also need to be specified. The authors acknowledge that the business states of the two task streams evolve separately, so the comparison measures the longitudinal benefit of retaining experience rather than the effect of intervening on memory alone at an identical intermediate state. I accept this evaluation objective, but it makes a clear definition of the experience system and its information retention boundaries particularly important for interpreting the comparison. In addition, each condition is run only once per environment, and the content audit covers only two environments and relies on a single pass of LLM-assisted annotation. These analyses offer some local observations, but they do not establish whether the experience gains and recovery trends are stable, nor do they connect conflict detection, experience revision, and subsequent reuse into a complete process. Researchers seeking to use this benchmark to compare experience maintenance methods still lack sufficiently detailed experimental specifications and validation evidence.

Questions for authors
  1. Compared with dynamic environment evaluations such as EvoArena, what additional capabilities are required by the combination of resource budgets, unannounced local changes, and selective revision? How do these requirements affect agents' decisions, and why are the resulting differences difficult to capture with existing evaluations? Please explain the contribution in these terms, rather than only listing differences in benchmark settings.

  2. Could the authors provide a representative task sequence from an actual benchmark environment spanning both sides of a change, connecting the public tasks, tool feedback, shared experience dependencies, invalidated and preserved components, resource budgets, and scores? What are the respective resource costs of informed execution, local diagnosis and repair, and repeated exhaustive exploration? How were these constraints checked during benchmark construction, and are the corresponding validation results available? This information would directly affect my assessment of whether the tasks realize the intended tradeoffs.

  3. What experience management procedure is used in each of the seven model deployment configurations? Which components generate interaction history, memory, and skills, and when are these retrieved and updated? Does background consolidation use the same model as task execution, and how is context overflow handled? Are these mechanisms consistent across configurations? What environment-managed observation and capability records remain in the condition without retained experience, can the agent query them, and how do they differ from the history and caches that are cleared? These details are needed to interpret and reproduce the experiments.

  4. Could the recovery analysis and content audit use existing trajectories to demonstrate a complete instance of experience maintenance: detecting that old experience is no longer valid, revising the affected parts, reusing the revision in a later task, and continuing to use unaffected experience? This would help distinguish maintenance and reuse from rediscovering information in each task. Are repeated-run results already available to assess the stability of the main experience gains and recovery trends? If not, limited reruns on a small set of representative environments would address this concern more directly than adding further models.

Reviewer_8iyXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ReScopeBench evaluates how agents acquire, reuse, and revise experience in 18 stateful environments, each with a stream of 16 tasks. An unannounced environment change occurs before task 9. Paired runs retain or clear experience while each run keeps its evolving business state. The paper measures the benefit of experience, its loss after the change, and subsequent recovery, and supplements the aggregate results with behavioral summaries and a trace analysis.

Critical strengths
  1. Costly discovery followed by an unannounced change is a useful test of experience reuse. Keeping some dependencies valid while changing others requires selective revision rather than simply replacing all stored information. Sections 3.2.3-3.2.4 also distinguish a valid reference solution from one an agent could discover through the permitted observations.

  2. The analysis makes useful distinctions. A loss of the earlier advantage does not necessarily mean experience has become harmful, and recovery is reported only for eligible cases. The common eligible subset in Section 4.2 and the distinction between retaining correct information and applying it successfully both help interpret the results.

Critical weaknesses
  1. The benchmark is difficult to assess from the available materials. The submission describes construction and verification principles, but I could not find an executable release or supplement containing the environment/task definitions, concrete budget settings, graders, and agent context/memory configuration. Section 3.2.4 describes checks for discoverability and repair without showing their outcomes or worked examples. For a benchmark whose difficulty depends on these details, this leaves too much of the construction unverifiable and is my main reason for leaning toward rejection.

  2. The lifecycle metrics also mix changes in experience usefulness with changes in tasks and business states. Gain loss compares tasks 8 and 9, and recovery is measured on further new tasks (Section 3.4). The paper acknowledges that these are descriptive comparisons. The construction checks are therefore important, but their reported evidence does not yet show how much ordinary task/state variation contributes to the observed drops. This limits the interpretation of the metrics as measures of invalidation and repair.

Questions for authors
  1. Can you provide the benchmark definitions and evaluation configuration, together with a worked example and results of the construction checks already described in Section 3.2.4? Please include per-model completion coverage, since Table 2 summarizes different environment sets.

  2. Is there evidence that the drop around task 9 exceeds ordinary variation along the stream? An existing construction check, or a small unchanged-environment control on a representative subset, would help. A full rerun of the benchmark is not needed to address this point.

Reviewer_8hPmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces ReScopeBench, a benchmark for evaluating the experience lifecycle of long-lived agents: acquisition, reuse, invalidation, and revision. The benchmark contains 18 stateful simulated environments, each with a 16-task stream, for a total of 288 tasks across nine application domains. The authors distinguish structural experience, which concerns how resources are organized, connected, and accessed, from rule experience, which concerns how local conditions affect whether actions are legitimate. Construction is experience-first: reusable dependencies that are discoverable through ordinary interaction are identified first, and tasks and changes are then jointly synthesized around these shared dependencies. Evaluation uses a paired protocol that runs a with-experience stream and a without-experience stream from the same initial state, scores final deliverables with private deterministic graders, and defines seven metrics: baseline performance BP, online gain OG, positive gain rate PGR, gain loss GL, experience harm EH, recovery level RL, and recovery efficiency RE.

Critical strengths

1: The problem setting is well chosen, fills a real evaluation gap, and makes selective revision measurable. The benchmark combines budgeted exploration, unannounced environment change, and agent-initiated revision within a single task stream. According to Table 1, this combination is not covered by existing memory and experience benchmarks such as LifelongAgentBench, MemoryArena, AgingBench, and EvoArena, yet it is precisely the situation that agents deployed over long horizons must face. The distinction between structural and rule experience separates knowing where a resource lives from knowing whether it is currently usable, so correcting outdated conclusions while retaining valid knowledge becomes an explicit evaluation target for the first time. This is the main reason I consider the work substantively valuable.

2: The measurement design is careful, the analysis goes beyond a leaderboard, and the claims are honestly framed. Defining gain loss GL separately from experience harm EH avoids misreading a shrinking advantage as evidence that experience hurts. Throughout, the paper qualifies the scope of its conclusions, noting for example that the paired contrasts do not establish causal harm from stale experience and that a small GL need not imply robustness. This honesty strengthens the paper's credibility.

Critical weaknesses

1: For a paper whose main contribution is a benchmark, the benchmark itself is nearly impossible to inspect or reproduce from the submission. The paper has nine pages of main text and no appendix, and it includes no complete example of any environment, such as its tool list, one structural and one rule experience, the task requests before and after the change, or the grader's decision rules; only seven of the eighteen environments are even named, in Section 4.3. The agent scaffold is likewise mentioned but never described, including the history retrieval and experience access or management tools, the background consolidation process, and the context policy, although these choices directly affect whether an agent can retain and revise experience and therefore shape every number in Table 2. Section 3.4 also states that the without-experience condition retains environment-managed observation and capability records but does not say what they contain or whether the agent can read them, so it is unclear whether the control condition is genuinely free of cross-task experience. The reproducibility statement points only to sections of the paper, and there is no commitment to release the environments, tasks, graders, or run manifests. For a benchmark contribution, the usable artifact is the contribution itself; with the current materials a reviewer cannot assess the quality of the 288 tasks, the presence of leakage or shortcuts, or whether the conclusions depend on one particular scaffold.

2: The experimental evidence is not rigorous enough to support the cross-model comparisons and the main quantitative conclusions. The environment sets aggregated in Table 2 differ across models; working backwards from the definition of PGR, the reported 87.01 for DeepSeek-V4-Pro-0813 is arithmetically consistent only with an average over 11 environments, the 81.51 for GPT-5.6 Terra only with 17, and the 65.31 for MiniMax-M3 only with 7 or 14, so direct cross-model statements in Section 4.2, such as 47.18 versus 47.36, compare averages taken over different environment subsets. In addition, each model and environment receives exactly one paired run, GL and EH are determined by the single boundary pair of tasks 8 and 9, and RL and RE depend on eligibility conditions and on L as a denominator, with eligible counts of 9, 18, 11, and 17 out of 18 for the four analyzed models and no repeated runs, variance estimates, or confidence intervals anywhere in the paper, which introduces both selection effects and numerical instability. Finally, the two conditions retain independently evolving business states, so per-task differences mix experience effects with diverging intermediate states. Taken together, beyond the directional findings about experience and change, most numerical conclusions and cross-model rankings cannot be confirmed from the reported evidence.

Questions for authors

Q1. Do you plan to release all environments, task streams, graders, and the run harness? Could you provide an anonymous link during the rebuttal, together with one complete worked example of a single environment, including its tool list, one structural and one rule experience, the task requests before and after the change, the grader's decision rules, and the budget setting along with how it was calibrated? A full trace showing public observation, experience acquisition, the silent change, diagnosis and revision, and later reuse would be especially convincing.

Q2. Please state the exact number and identity of the environments aggregated for each model in Table 2. Is the inference above correct that DeepSeek-V4-Pro-0813, GPT-5.6 Terra, and MiniMax-M3 are averaged over 11, 17, and 7 or 14 environments respectively? Please add a version of Table 2 computed on the subset of environments completed by all seven models, explain which runs were incomplete and why, and report RL and RE eligibility counts for all seven models together with recovery results on the common eligible subset.

Q3. Could you repeat the paired runs at least three times for at least one model on the common complete environments and report the dispersion of OG, GL, EH, RL, and RE across runs? Given that GL and EH come from a single task pair and that L serves as the denominator of RL and RE, I would like to see whether headline gaps such as RL of 35.51 versus 81.86 and direct reuse rates of 12.50 versus 75.00 percent exceed run-to-run noise.

Q4. Would it be feasible to branch from the same pre-change state snapshot and compare three arms, one that keeps but freezes the old experience, one that may update it, and one that clears it, together with a matched control in which no change occurs? Such a comparison would show whether the recovered advantage really involves revising invalidated dependencies while retaining valid ones rather than re-solving tasks from scratch.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.