acceptodds
Withdrawn from ICLR 2027

Generality on Demand: Progressive Behavioral Consolidation for Web Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Web agents often invoke a model again for routines they have already solved, paying for fresh action selection on every repetition. We study whether verified experience can instead become executable behavior that still recovers from deployment drift. Progressive Behavioral Consolidation (PBC) compiles repeated successes into parameterized workflows. At a live failure it returns control to an agent, then persists a recovery only if the resulting patch passes replay, regression, and held-out validation. On Gmail, PBC achieves 0.998 success, comparable to the strongest baselines (0.986–0.992), while reducing agent dependence from 1.000 to 0.306. Its final artifact executes 0.80/0.80 of isolated and cumulative drift states without a model. On GitLab, dependence falls to 0.111 and tokens to 0.34M per stream, with 0.948 success versus 1.000 for the full-solver baselines. Main-table results average three independent draws. Among the gate’s rejections, what blocks a recovery is preserving behavior the system has already verified, not the novelty of the fix. Verified recovery thus changes how later tasks execute: the agent handles unresolved cases, while the workflow serves behavior the system has learned to verify.

Primary area:
foundation or frontier models, including LLMs
Keywords:
web agents, LLM agents, behavioral consolidation, agent self-evolution, workflow automation, continual deployment, verified execution, agent dependence

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fd9cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a confounding factor in benchmarking LLM serving systems for multi-turn agents: identical initial tasks do not necessarily produce identical execution workloads. Differences in generated tokens can change subsequent prompts, tool calls, dependencies, and termination behavior. AgentReplay records input/output tokens, invocation dependencies, optional logical expert selections, and tool durations, then replays the recorded trajectory while retaining normal autoregressive model computation. Its configurable controls enforce output tokens, optionally restore MoE expert IDs, and emulate tool delays, while leaving batching, scheduling, caching, and physical expert placement to the system under evaluation. The paper also demonstrates replaying a trajectory collected with a more capable model on a smaller target model. Experiments show workload variation under greedy decoding, differences in MoE access patterns and execution costs between random-token length-wise replay and token-wise replay, and improved timing repeatability under fixed-trace replay.

Critical strengths
  1. The paper addresses a meaningful benchmarking problem and separates system efficiency from changes in agent behavior. Section 4.1 shows substantial variation in token counts, turns, and tool calls despite greedy decoding. AgentReplay fixes the recorded trajectory without replacing model execution with stored responses. It also appropriately preserves scheduling, cache-management, and physical-placement decisions as outcomes of the system being evaluated. The authors explicitly distinguish improved repeatability from speedup, avoiding the misleading interpretation that executing a different workload more quickly demonstrates a system improvement.

  2. The MoE experiments provide informative mechanistic evidence. Section 4.2 connects token content to expert concentration and then examines its interaction with expert parallelism, tensor parallelism, and offloading. The observed bias can have different directions: at batch size 32, length-wise replay yields a full EP prefill-phase time of 1.18 times the token-wise reference, but a TP decode MoE-kernel time of 0.84 times the reference. These results substantiate the narrower, important claim that equal sequence lengths do not imply equivalent MoE execution workloads.

Critical weaknesses
  1. Replay fidelity and measurement overhead are not sufficiently validated for a benchmarking framework. Section 3.2 describes a sensible token-replacement point, but the paper does not show implementation-level tests establishing that replay maintains the intended token state and prefix/cache keys across batch reordering and turn boundaries. This does not require different cache policies to achieve identical hit rates; rather, it requires the supplied token history and legal reuse opportunities to match the trace definition. Router replay is presented as a separate contribution, yet Figures 4–6 do not clearly specify whether it is enabled or report recorded-versus-enforced expert-ID agreement and an on/off comparison across configurations. The handling of gate weights after expert IDs are forced is also insufficiently specified. Finally, the overhead of trace lookup and token/router injection is not reported. These omissions do not demonstrate an incorrect implementation, but they limit confidence that the framework controls the intended workload without materially perturbing the measured execution.

  2. The experiments establish useful counterexamples more convincingly than they establish reliable practical serving-system comparisons. The length-wise baseline in Section 4.2 replaces content with random tokens. This proves that length matching alone is insufficient in that setting, but it does not quantify the bias of a more realistic baseline that retains recorded inputs and allows natural generation while enforcing output lengths. Some placement and offloading configurations are also deliberately constructed around the random-token hotspots; these are useful diagnostic cases, but their effect sizes should not be generalized without further evidence. In Figure 3(c), output tokens and tool durations are fixed together, so the reduction in cumulative-time CV cannot be attributed to token replay alone. Fixed tool delays can reduce CV both by removing tool-time variability and by adding a constant to its denominator, as the authors acknowledge. Although Figures 5–6 include full-phase measurements, the paper does not demonstrate how replay changes a paired comparison between actual serving configurations under the same agent trace and arrival policy. Missing hardware/software specifications and repeat counts for Figures 4–6 further limit reproducibility.

Questions for authors

1Can you provide replay-consistency tests covering batch reordering, turn boundaries, and prefix/cache-key updates, together with token/router injection overhead measurements, especially for small-batch decoding? The objective is to verify the intended reuse opportunities, not to require identical cache-hit outcomes across different policies.

2.Was router replay enabled in Figures 4–6? How are gate weights handled after expert IDs are forced, and how are routing events aligned across prefill and decode? Please report recorded-versus-enforced ID agreement and one on/off comparison using the existing TP/EP configurations.

3.With the recorded graph and inputs held fixed, what are the model-only and model-plus-tool timing means, standard deviations, and CVs when output-token replay and tool-duration replay are enabled separately and together? This would distinguish their contributions to repeatability.

4.Can you supplement the existing trace with a length-wise baseline that retains real inputs and enforces only output length, and report one paired phase-level or end-to-end cost comparison between actual serving configurations? Please also specify the hardware, software versions, and measurement repeat counts.

Reviewer_Vi7YOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper describes a problem in which web agents keep paying for general-purpose reasoning on routines they should have mastered. The authors propose progressive behavioral consolidation, which turns that experience into executable behavior and can consistently improve performance. The method includes a parameterized workflow, a recovery stage, a persistence gate, a verified envelope, and reconsolidation. The authors benchmarked on two subsets of Web-Arena infinity, Gmail and GitLab. The authors gained comparable results to other harnesses and settings.

Critical strengths
  • The framing of the problem is useful and seems very well-motivated. They found an actual gap in how self-evolving agents are evaluated.
  • The results have been reported honestly. Some of the experimental outcomes are unflattering and could be considered negative results. The authors didn't exaggerate the results in any manner.
  • The appendix is thorough with a lot of technical details.
Critical weaknesses
  • The biggest weakness is the empirical evidence. The method has not been fully evaluated on an actual benchmark. I don't believe subsets or domains from a web benchmark are sufficient. Similar methods are generally evaluated across multiple benchmarks.
  • Following Table 1, the success rate is generally the same or worse than the baselines. The results are mostly negative in terms of performance. The other metrics, where the proposed PBC shines, don't actually feel well-motivated and connected with the rest of the literature.
  • The method itself is more comparable to an engineering trick, where workflows are used with failure recovery. This is commonly used in several other domains, and the authors should draw connections to it.
  • The paper should be presented in a better way. In the current state, it's very hard to navigate. The third line of introduction refers to Appendix D.6. Most of the text reads in a very unnatural way. While the authors acknowledged the use of AI, the AI-generated paraphrasing reduces readability a lot. Similarly, the diagrams could benefit from more iterations.
Questions for authors
  • Can the authors differentiate the scientific contribution of the proposed method from the existing engineering methods, and also draw a connection with the current literature?
  • Would it be possible to provide stronger empirical evidence of the proposed method?
  • Could the authors better motivate the proposed metrics and their necessity in the broader scope of web agent works?
Reviewer_VBm9OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper looks at web agents that repeat the same routine many times but still call the LLM on every run. It proposes Progressive Behavioral Consolidation, which turns a routine into an executable workflow after a few verified successes, so later runs need no model. When the website changes and the workflow breaks, an agent takes over from the failure point, and its fix is added to the workflow only if it fixes the failed case, keeps old cases working, and generalizes to new inputs. Experiments on Gmail and GitLab show that the method greatly reduces how often the model is needed, with similar success on Gmail and a small success drop on GitLab.

Critical strengths
  1. The paper makes a useful distinction between solving a repeated task and no longer needing the model to solve it. Success rates alone cannot tell these apart, so the paper adds agent dependence and model-free coverage to measure the second one directly. On Gmail, these metrics show a large drop in model use while success stays close to the strongest baselines. It gives repeated web deployments a practical evaluation target that current benchmarks do not cover.

  2. A workflow edit is accepted only if it passes replay on the failed case, regression on cases that already worked, and validation on held-out inputs. The gate also logs why each candidate is rejected, and on Gmail most rejections come from the regression check rather than held-out validation. This matters because every update can be traced, and it points future work to a clear bottleneck, which is turning successful recoveries into edits that keep existing behavior intact.

Critical weaknesses
  1. The evaluation is too narrow for the paper's general claim. The main results cover two applications with three short routines each, such as archiving or labeling an email, and they use one model and a drift schedule the authors designed. A fourth GitLab workflow fails for every method and is left out of the main table. The paper argues that repeated web tasks can move out of the agent loop in general, and it is unclear whether this holds for longer workflows, real websites, or other models.

  2. The evidence for the two core components is thin. The envelope and gate ablations each come from a single run, and removing the envelope actually gives higher final coverage with similar success, so its benefit shows up only in lower model use and token cost. The checks behind a verified update are also small, since the regression check uses two reserved instances and the held-out check uses at most five. The envelope and the gate are what set this method apart from earlier workflow reuse, and the paper does not show that they are reliable.

Questions for authors
  1. How do the results change with a different model family? If agent dependence and gate acceptance stay similar, it would make the method's generality much more convincing.

  2. Can you run the envelope and gate ablations over multiple runs? Removing the envelope gives higher final coverage with similar success, so why the envelope should still be considered a core component.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.