acceptodds
Under review as a conference paper at ICLR 2027

Retrieve and Compress: Why Naive Experience Sharing Fails for Web Agents

Abstract

Giving a web agent access to other agents' past trajectories is widely assumed to help. We show that unselective sharing does the opposite. On WebArena-Verified (812 tasks, 5 seeds, one independent memory pool per seed), injecting every accumulated trajectory into every subsequent task drops success from 30.0% to 12.6%, below an agent with no memory at all, as does a curated pool read at random (−9.9 pp) — in neither case is what gets injected chosen for the task in front of it. What recovers the gain is not sharing but discipline: deciding what may enter the pool, what survives compression, and which entry a task is shown. That reaches 41.4% against a 30.0% baseline (+11.4 pp, task-level paired p < 0.005, Holm-adjusted across the 13 memory conditions) — real but modest, Cohen's h = 0.24. Of those three decisions only one changes the sign: that design's own pool, read without the intent rule, scores 20.1% — 21.3 pp lower, below the no-memory baseline — while removing compression or admission leaves it above. The direction holds across task type, task difficulty and model family, and more weakly on a second benchmark (transfer results in the appendix). An oracle injecting each task's best historical trajectory reaches 53.4%, 12.0 pp above us — an upper bound, not headroom — and no configuration of ours approaches the strongest specialised systems on the benchmark.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.