STEER: Scaling Long-Term Memory for Self-Evolving GUI Agents
Abstract
GUI agents accumulate experience through repeated interaction, but turning that experience into useful long-term memory requires more than retaining past trajectories. Successful procedures and failure-derived lessons need explicit applicability conditions, evidence for routine reuse, and selective activation in later tasks. We present calable ask-conditioned xperience volution and euse (**STEER**), a framework that evolves operational knowledge while keeping the execution agent frozen. STEER represents experience as outcome-conditioned behavioral hypotheses, giving generation, validation, and retrieval a shared operational interface. Consolidation and controlled trials govern admission to a compact deployed store, while TaskContract-based retrieval filters conflicting knowledge and composes complementary procedures and warnings. Both candidate testing and deployed reuse contribute to task execution and further learning. On MobileWorld, STEER raises Seed-2.0-Pro's five-pass cumulative success from **67.5%** without memory to **77.8%**. Independent evolution on OSWorld V2 also improves task progress. These results demonstrate the value of a complete knowledge lifecycle for improving GUI agents through continued interaction. An anonymized project page is available at [https://anonymous.4open.science/w/STEER-GUI/](https://anonymous.4open.science/w/STEER-GUI/).
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.