acceptodds
Withdrawn from ICLR 2027

STEER: Scaling Long-Term Memory for Self-Evolving GUI Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

GUI agents accumulate experience through repeated interaction, but turning that experience into useful long-term memory requires more than retaining past trajectories. Successful procedures and failure-derived lessons need explicit applicability conditions, evidence for routine reuse, and selective activation in later tasks. We present calable ask-conditioned xperience volution and euse (**STEER**), a framework that evolves operational knowledge while keeping the execution agent frozen. STEER represents experience as outcome-conditioned behavioral hypotheses, giving generation, validation, and retrieval a shared operational interface. Consolidation and controlled trials govern admission to a compact deployed store, while TaskContract-based retrieval filters conflicting knowledge and composes complementary procedures and warnings. Both candidate testing and deployed reuse contribute to task execution and further learning. On MobileWorld, STEER raises Seed-2.0-Pro's five-pass cumulative success from **67.5%** without memory to **77.8%**. Independent evolution on OSWorld V2 also improves task progress. These results demonstrate the value of a complete knowledge lifecycle for improving GUI agents through continued interaction. An anonymized project page is available at [https://anonymous.4open.science/w/STEER-GUI/](https://anonymous.4open.science/w/STEER-GUI/).

Primary area:
applications to robotics, autonomy, planning
Keywords:
UI agents, long-term memory, self-evolving agents, experience evolution, selective retrieval, knowledge reuse

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TkJiOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this paper considers that gui agents keep meeting similar operations across tasks, and existing memory methods such as ExpeL [1] and Agent Workflow Memory [2] already store such experience but rarely check whether a recalled item actually applies to the task at hand. so the authors propose STEER, which keeps the executor frozen, turns trajectories, gates admission to a small deployed store. The novelty lies this admission gate plus operational compatibility filtering

Critical strengths

The representation is well designed. Each item carries applicability, completion conditions, scope, and provenance, so generation, validation, and retrieval share one interface (Sec. 3.1).

Critical weaknesses

reelated work misses earlier memory and skill-library efforts, e.g. Reflexion [3], the verified skill library of Voyager [4], the reflective memory stream of Generative Agents [5], and MemGPT [6]. IMO the paper should state what is new relative to their write and verify policies.

admission rule in Sec. 3.2 has no statistical grounding. i've also checked the supplementary part, where sec e.2 finds promoted vs rejected items differ by only +0.0278+0.0278 with p=1.0p = 1.0, so the gate is throttling rather than a quality test, tbh? sec c also shows the headline five-pass run used zero deployed reuse? i.e. the core mechanism stays untested at benchmark scale

all baselines are in-house append-all variants, as memory methods, e.g. ExpeL [1] and Agent Workflow Memory [2], are discussed but never run. the one comparison that isolates the lifecycle is not significant with p~0.42? and cumulative Pass@k over the same 117 tasks rewards retries. and only one benchmark, one executor, and one knowledge model, sec d shows Qwen generator drops Pass@3 by three tasks, the osworld run is unreplicated, sec i shows generation-token costs are untracked, and 12 deployed items after nine passes is a tiny memory, imo.

Questions for authors

see above comments

Reviewer_RDnXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how frozen GUI agents can accumulate long-term memory without suffering from unverified or conflicting instructions. The authors propose STEER, which represents extracted experiences as outcome-conditioned behavioral hypotheses with explicit applicability conditions. Instead of immediate reuse, candidate memories must pass controlled online trials before promotion to a compact deployed store. At inference time, TaskContract-based retrieval filters out operational conflicts, composes complementary procedures and warnings, or abstains. Experiments on MobileWorld and OSWorld V2 show that STEER improves multi-pass task performance while cutting retrieved-context tokens by ~70%.

Critical strengths
  1. Well-motivated problem. The paper addresses the important challenge of turning GUI interaction experience into reusable knowledge while limiting irrelevant or harmful memory use.
  2. Carefully engineered memory pipeline. The framework integrates experience extraction, candidate validation, controlled admission, and task-conditioned retrieval into a coherent lifecycle.
  3. Rigorous empirical evaluation. Repeated runs, controlled comparisons, and detailed ablations support the analysis, with transparent reporting of null results and limitations.
Critical weaknesses
  1. Zero deployed-memory reuse in the main evaluation: Although STEER centers its narrative around promoting validated candidates into a deployed store (MM) for routine reuse, Appendix C (Tables 9 and 10) reveals that deployed-memory reuse occurs 0 times across all 585 episodes in the primary five-pass MobileWorld experiment, with the very first item promoted only after Pass 5 concludes. Consequently, the headline Pass@5 gains (from 67.5% to 77.8%) stem entirely from unpromoted candidate trials rather than the proposed routine reuse of validated long-term memory.

  2. Insignificant performance gains across ablations for key components: In the controlled mechanism studies (Appendix E, Tables 12–14), none of the three core components yields a statistically significant success-rate improvement: removing outcome conditioning produces identical Pass@3 (19/30 vs. 19/30), promoted items perform no better than rejected items (p=1.0p = 1.0), and TaskContract activation fails to outperform similarity retrieval on Retrieval24 (p=0.625p = 0.625). Furthermore, two of the three full-scale replicated runs (R2: p=0.143p = 0.143; R3: p=0.302p = 0.302 in Table 4) as well as the source-disjoint cross-task evaluation (p=0.804p = 0.804 in Appendix B) do not reach statistical significance.

  3. Cumulative Pass@kk confounds memory evolution with best-of-kk sampling: Relying on cumulative Pass@kk (and cumulative best partial reward) conflates genuine cross-pass learning with stochastic multi-sampling, as evidenced by the memoryless No Memory baseline jumping +11.9 percentage points (from 55.6% to 67.5%) over five passes purely from repeated rollouts. Similarly, on OSWorld V2 (Table 19), STEER’s per-pass attempt score actually falls below the memoryless baseline in Passes 1, 4, and 5 (e.g., 7.18% vs. 7.62% at Pass 5), indicating that the upward cumulative curve largely reflects monotonic max-aggregation rather than consistent policy improvement.

Questions for authors

See the weakness.

Reviewer_W2ByOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes STEER, an external experience evolution and reuse framework for GUI agents. The method converts successful and failed trajectories into operational knowledge with explicit applicability conditions, behavioral roles, and provenance while keeping the execution policy frozen, governs admission to deployed memory through controlled candidate trials, and uses TaskContract-based retrieval to assess operational compatibility with the current task, selectively composing a primary procedure with a complementary warning. In the historical R1 experiment on MobileWorld, STEER increases five-pass cumulative success from 67.5% without memory to 77.8%. Across three independent runs, its mean cumulative success rate is 76.4%, compared with 70.4% for AppendAll + Similarity, while using 70.5% fewer retrieved-context tokens on average. Identity-matched replay experiments under controlled exposure further support the effectiveness of task-conditioned selection. Independent memory evolution on OSWorld V2 also improves cumulative partial reward.

Critical strengths
  1. Operationally incompatible experience in GUI tasks is not necessarily semantically irrelevant, but may conflict with the action direction, object, source role, or completion requirements of the current task. STEER explicitly incorporates these factors into TaskContract and the applicability conditions of knowledge items, directly addressing practical challenges in experience reuse.

  2. The paper not only reports overall task success, but also distinguishes between controlled candidate trials, deployed-memory reuse, candidate growth, and retrieved-context costs. It further provides independent runs, source-disjoint experiments, and model sensitivity analyses, offering a broad set of diagnostics.

Critical weaknesses
  1. Across the first five passes of historical R1, all 141 experience-bearing episodes were controlled candidate trials, and ordinary deployed-memory reuse was zero. The first deployed item was promoted only at the Pass-5 boundary, after the reported Pass@5 executions, so it could not have influenced these results. This means that the primary result of 77.8% cumulative success directly supports the use of candidate experience during controlled trials and the associated selection mechanisms; it does not directly establish that routine reuse of admitted knowledge produces this benefit. Correspondingly, the combination of high cumulative success and only one deployed item cannot be interpreted as evidence that this compact deployed store supported performance during the first five passes.

  2. The main experiment repeatedly executes the same set of 117 tasks, and the metric records whether each task has been solved at least once. This metric is suitable for measuring cumulative task coverage, but it cannot reveal: (1) whether per-pass success rates improve in later passes; (2) whether previously solved tasks regress; or (3) whether the learned experience improves performance on unseen tasks that were not involved in experience generation or validation. Although the source-disjoint experiment provides evidence beyond exact-source repetition, excluding tasks present in an item's provenance is not equivalent to evaluating frozen memory on a held-out task set.

  3. The main comparisons are against No Memory, AppendAll + Similarity, and AppendAll + Operational. These conditions are useful for isolating parts of the design, but the reported experiments do not include direct, matched-budget comparisons with the related experience-learning or memory-evolution methods discussed in the paper. The public model results in Figure 1 use different settings, and the authors explicitly present them as contextual references. Therefore, these results cannot substitute for controlled comparisons between memory methods. Even within the internal ablations, full STEER improves over AppendAll + Operational by only four tasks in historical R1. The replicated comparison primarily supports the overall benefit over AppendAll + Similarity, while the incremental value of controlled evolution and admission to deployed memory beyond operational selection has not yet been robustly established.

Questions for authors

Please see Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.