AgentEAES: Effect-Aware Experience Selection for LLM Agents

Recent advances have enabled large language models (LLMs) to act as agents capable of solving increasingly complex tasks in scientific research, office productivity, and software development. Prior work often improves agent performance at inference time by retrieving relevant experiences from past interactions. Although such experience-based augmentation offers a promising path toward continual adaptation, we identify a fundamental limitation of similarity-based retrieval in agentic settings. Highly similar experiences can degrade performance, even on tasks that the agent would otherwise solve successfully. This finding suggests that experience utilization is a decision problem rather than merely a retrieval problem. We propose effect-aware experience selection (AgentEAES), a framework that learns which experience to use from experience-augmented rollouts. AgentEAES trains a model to predict the sign of each task-experience pair's effect and selects the experience with the largest positive predicted effect. Experiments cover two agentic benchmarks with backbones of different strength, allowing us to evaluate both a strong backbone on a more challenging benchmark and a smaller model on an easier one. On AppWorld, with Qwen-3.7-Max, AgentEAES improves task success rates on the challenging domain by 13.3 percentage points. On WorkBench, even with the smaller Qwen3-30B-A3B backbone, AgentEAES still yields an improvement of up to 21.5 percentage points. The code is available at https://anonymous.4open.science/r/AgentEAES-BDBF.