acceptodds
Under review as a conference paper at ICLR 2027

AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?

Abstract

Large language model (LLM) agents can self-evolve by continually improving from their own accumulated experience. However, existing studies predominantly adopt independent evaluation. Consequently, the behavior of self-evolving agents in realistic streaming settings, where agents adapt to diverse and complex task streams, remains poorly understood. To address this gap, we introduce AgentStream, a unified framework that evaluates self-evolving agents spanning diverse evolution components by organizing agentic benchmarks into a configurable task stream. Specifically, we instantiate *Isolated*, *Sequential*, and *Interleaved* streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and is non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.

Primary area:
datasets and benchmarks
Keywords:
self-evolving, large language model, streaming evaluation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_rXzhOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AgentStream, a framework for evaluating self-evolving LLM agents under continuous task streams. It defines three scenarios—Isolated, Sequential, and Interleaved—and evaluates five self-evolving methods across three foundation models and six agent benchmarks. The results suggest that self-evolution is not uniformly beneficial: its effectiveness depends on model capability, method architecture, and stream composition. Context-integrated methods generally favor isolated streams, whereas retrieval-based methods appear more robust under interleaved streams.

Critical strengths
  • Studies an important limitation of conventional independent evaluation for self-evolving agents.

  • Covers a broad combination of models, methods, benchmarks, and streaming scenarios.

  • Reports negative evolution effects and cost increases rather than focusing only on successful configurations.

  • Provides per-seed results, cumulative performance curves, cost analysis, state-evolution analysis, and complete prompts.

  • Offers useful empirical observations about the interaction between method architecture and stream composition.

Critical weaknesses
  • Sequential evaluation uses only one fixed benchmark order, so its results may be specific to that ordering.

  • Many reported gains are small relative to the observed variance, but no paired confidence intervals or significance tests are provided.

  • The capability-gating and non-monotonicity claims are based on only three models and an average over heterogeneous benchmark metrics.

  • Isolated states contain at most 50 tasks, while shared states may accumulate 300 tasks. This confounds domain composition with state size, age, and possible context truncation.

  • Comparisons may be affected by method adaptations, the custom Harness implementation, and different model reasoning configurations.

Questions for authors
  • Why is Sequential evaluated with only one benchmark ordering? Do the conclusions hold under alternative orders?

  • What are the 45 statistical units in Table 2, and does the reported ± denote standard deviation, standard error, or a confidence interval?

  • Were the vanilla baselines repeated with the same random seeds and decoding settings as the evolving methods?

  • How do the authors justify capability gating and non-monotonicity from only three model observations?

  • Are state or token budgets normalized across scenarios? How are truncation and retrieval capacity handled as the state grows?

  • Does the cost analysis include reflection, state updates, embedding, retrieval, and skill or memory maintenance? Why is Claude omitted?

2 comments

Thanks for the additional results and response. I will raise my score.

Thank you for appreciating our work and for raising your score accordingly. We also value the effort you put into reviewing our submission.

Reviewer_nSBJOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AgentStream a new eval settings which focus on self-evolving LLM agents on streams of tasks. In a continual online setting where tasks are streaming in continually, the agent is allowed to keep an evolution state (refined context, memory entries, skills, or a revised harness). It updates this state after every task using its own feedback, with no ground-truth labels. The paper evaluated 5 methods covering harness which can update context, memory, skills and harness (ACE, A-Mem, ReasoningBank, AutoSkill etc) are crossed with three frontier models (GPT-5.4, Gemini 3.1 Pro, Claude Opus 4.7) over three seeds. Results are scored as the gain over the same model with no state. The main claims are: Self-evolution is not uniformly beneficial. Isolated is the most reliable scenario, and Interleaved surprisingly beats Sequential. The benefit depends on model capability. GPT-5.4 gets negative gains on average, and the gain is non-monotonic: Gemini gains more than the stronger Opus. No method dominates. Context-integrated methods (ACE, Harness) do best in Isolated, and retrieval-based methods do best in Interleaved. Sensitivity to method choice shrinks as models get stronger.

Critical strengths
  1. The experiment is well designed and the code checks out. The 3 settings (isolated, sequential, interleaved) are a make sense, since they separate learning within one domain, carrying experience over to the next domain, and dealing with all the domains mixed together, which are the cases an agent would actually run into in deployment. In their setup which run 5 methods which did address missing part from previous work (Streambench and Evo-memory)

  2. I also like that they don’t hide the negative result: self-evolution ends up below the vanilla baseline in a good chunk of the configs (11-17 out of 45 per setting), which is useful to know since most of the original method papers only show gains on a single benchmark. On top of that all the method prompts are in the appendix and the per-seed tables are there too, so it’s easy to check what’s going on.

Critical weaknesses

Major

  1. A missing study in terms of performance over different reasoning levels, since this is kinda like a long running task with many sub-task, can higher reasoning benefits from this?

Minor:

  1. I think the paper lacks a general model behavior study, ie why do the models perform badly in some tasks?

The explanations they give (weak models filling their memory with failed trajectories, context-based methods getting interference that retrieval avoids) sound reasonable but are never actually checked.

B.2 shows update dynamics but only as counts, not what is actually being written into the state, so we can’t tell whether a model that doesn’t gain is bad at writing the state or bad at using it. From what I remember ACE isn’t great at writing useful entries either, and Opus building 1251 ACE bullets and then dropping from 67.0 to 61.9 in interleaved could just be the playbook bloating up. Looking at the actual state content would answer most of this, and without it the “which method for which model and stream” guidance in the paper is hard to act on.

Questions for authors
  1. Why is Opus 4.7 run at high reasoning while GPT-5.4 and Gemini are on medium? Related to this, did you check GPT-5.4’s runs for answer extraction or early stopping issues in Exgentic? 2.0% on HLE and ~9.5 steps per task (vs 21.1 for Gemini) looks more like an integration problem than the model being weak, and the whole “weak model gets negative gains” finding rests on GPT-5.4.

  2. Did you try models from the same family, or the same model with different reasoning effort, ie GPT-5.4 vs GPT-5.4 mini, or GPT-5.4 low vs high? Right now, with 3 models from 3 families, it’s hard to tell capability apart from model family. If the same methods still win in the same settings, the capability claims in 5.2 and 5.3 would be much more convincing.

  3. Did you try several LLMs on the same streaming tasks and see if they improve? In Wu et al. (2024) the multi-LLM agent setting with shared memory did perform better than a single LLM. A simple version here would be letting one model write the state and another use it (eg GPT-5.4 running with a state written by Opus), which would tell whether GPT-5.4’s negative gains come from writing bad state or from not being able to use it. A full multi-LLM shared memory setup is probably more future work, but the swap experiment would directly change how I read Section 5.2.

Reviewer_AfgfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that self-evolving agents are mostly evaluated task by task, or on a stream drawn from a single benchmark with one kind of evolving component, so it is unclear whether reported gains hold up when experience from one domain is still in the agent's state when tasks from another domain arrive. AgentStream samples 50 tasks from each of six benchmarks (AppWorld, BFCL, BrowseComp-Plus, HLE, SWE-bench Verified, Tau2) and runs five evolution methods (ACE, A-Mem, ReasoningBank, AutoSkill and a harness-evolution method) on GPT-5.4, Gemini 3.1 Pro and Claude Opus 4.7. Three stream settings are compared: Isolated keeps a separate evolution state per benchmark, Sequential carries one state through a fixed benchmark order, and Interleaved shuffles all tasks into one stream with a shared state. The gain is the difference from the same model run with an empty state, averaged over three task orders.

The gains are small, around one point on average. Isolated has the most configurations above the vanilla baseline (34 of 45). GPT-5.4, the model with the lowest vanilla average, loses accuracy on average in all three settings, and Gemini gains more than Claude although Claude has the higher baseline. Methods that fold experience into the prompt (ACE, Harness) do best under Isolated, retrieval-based methods do best under Interleaved, and no method is best on all three models.

Critical strengths

The main strength is the factorial design. Model, evolved component and stream are varied on one shared task set, which, as the paper notes, earlier streaming evaluations did not do: they stream each benchmark separately and study one kind of evolving component at a time. Table 4 shows that the best method depends on the model, and Tables 5 and 6 suggest that prompt-integrated and retrieval-based methods prefer different streams. A-Mem's lead on GPT-5.4 holds in all three seeds, so at least part of this interaction is solid, and it matters for anyone choosing an evolution method for a given model.

Critical weaknesses

The takeaways are stated more firmly than the data allow. The paper calls Isolated the most reliable setting, says Interleaved generally outperforms Sequential, and says the non-monotonic effect "intensifies with streaming complexity". The first two do not hold in every seed, and the third rests on differences of about half a point between averages and is reversed in one seed. On top of that, describing GPT-5.4 as the weakest model relies heavily on an unexplained 2.0% HLE score. In a benchmark paper the wording of the findings is a large part of the contribution, because readers will cite the takeaway boxes. I would like the revision to present them as tendencies in this sample, with the seed-level spread next to each one.

Questions for authors
  1. Could you report seed-level intervals for the scenario contrasts in Tables 2 and 3, for example Isolated minus Sequential computed per seed? Given seed 44, I expect Isolated will not be reliably ahead, and the takeaway should then be softened.
  2. What explains GPT-5.4's 2.0% on HLE? Is it the judge, answer extraction, or failed submissions? If it is an artifact, please report the model ordering without it.
  3. Did you try a reversed or random benchmark order for Sequential? Without it, the comparison with Interleaved depends on one curriculum.
  4. For SWE-bench, does the feedback used for evolution include test outcomes? If so, the statement that evolution uses no ground-truth labels needs a qualification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.