acceptodds
Under review as a conference paper at ICLR 2027

From Similar to Useful: Failure-Attributed Memory Selection for LLM Agents

Abstract

LLM-based agents increasingly rely on memory of past experience to adapt without updating their parameters. Existing work concentrates on the memory bank itself, addressing how agent trajectories should be processed and how the bank should be maintained. Less attention has been paid to a different question: why memories that appear relevant often fail to help, and what makes a memory useful to a given agent on a given task. We study this question in a controlled setting where the memory bank holds only verified successful trajectories, leaving selection as the only variable. For each task the agent fails without memory, we inject each retrieved candidate separately and examine how recovery relates to the agent's failed trajectory. Across three datasets covering code generation, embodied interaction and text-to-SQL, even among highly similar tasks, how the agent failed carries information about which memory will help, and memories that demonstrate the behavior a failure lacks are more likely to recover it. Building on this, we propose FAMA (Failure-Attributed Memory Allocation), an LLM-based selector that reads the agent's failed rollout, identifies what it lacks, and chooses the memory that addresses that gap. On the three datasets, under both single-agent and multi-agent architectures, fine-tuned FAMA recovers 8 to 11 more points of failed tasks than top-1 similarity retrieval. It also outperforms a cross-encoder reranker, a selector that does not observe the failure, and Reflexion under the same retry budget and partly transfers to an executor it was not trained on.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent memory, Experience retrieval, Memory selection, LLM-based agents, Failure attribution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HfAUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why a retrieved, verified-successful trajectory may be relevant to an LLM agent’s task yet fail to help it. It proposes FAMA, a selector that reads the agent’s failed rollout and chooses a memory for one retry. Across code generation, embodied interaction, and text-to-SQL, the fine-tuned selector recovers 8–11 percentage points more initially failed tasks than top-1 similarity retrieval.

Critical strengths
  1. The paper tests retrieved memories individually and compares FAMA with a selector that sees the same task and candidates but not the failure. This makes the central finding more convincing than an improvement over similarity retrieval alone.

  2. Fine-tuned FAMA recovers 8-11% more previously failed tasks than top-1 retrieval across code generation, embodied interaction, and text-to-SQL, demonstrating benefits that generalize beyond a single benchmark.

Critical weaknesses
  1. The cross-encoder does not see the failed rollout, and on KodCode FAMA can choose separate memories for two agents while some baselines must use one shared memory. This leaves uncertainty about how much of the reported gain comes from FAMA’s proposed method rather than extra information or selection flexibility.

  2. The experiments use verified successful, same-distribution memories and require a reliable signal that the first attempt failed. Without results for noisy banks or uncertain outcomes, it is unclear whether the gains extend to less controlled agent settings.

Questions for authors
  1. On KodCode, how much of FAMA’s gain remains if it must select one shared memory for both agents, matching the top-1 and cross-encoder baselines?

  2. How does FAMA compare with a failure-aware reranker trained or prompted using the same failed rollout and the same ten candidates? Can you separate the value of seeing the rollout from the value of the proposed diagnosis-and-selection procedure?

  3. What are the paired confidence intervals for the main recovery differences, and what are the added selector cost and end-to-end cost per recovered task?

2 comments

Dear reviewer,

Thank you for the precise questions. Please also see our General Response (top-level comment) for a table of all new experiments and the cost analysis. New results are recovery rates (% of memory-free failures solved on retry) on KodCode / ALFWorld / Spider, same executor and same ten candidates as Table 1.

Failure-aware reranker; rollout vs. procedure. We added the failed trajectory to the cross-encoder query (same off-the-shelf bge-reranker as in Table 1, not retrained): 39.3 / 31.5 / 36.5 vs. 39.6 / 39.0 / 36.2 without it. No gain: a reranker that sees the rollout but was not trained to interpret it does not benefit from it. To separate seeing the rollout from the diagnosis procedure, we fine-tuned the same model on the same instances to output the choice directly, with no diagnosis text: 50.8 / 45.1 / 37.0, vs. FAMA 49.7 / 48.4 / 41.0. So seeing the rollout is worth +8.9 / +3.8 / +0.5 over the selector w/o failure, and the diagnosis adds −1.1 / +3.3 / +4.0 on top. On KodCode the rollout alone suffices; on ALFWorld and Spider the procedure carries a real part of the gain. This matches Analysis I-a: on KodCode the failure's surface already carries the signal, on the other two it does not.

Cost. Please see the cost table in the General Response. Per retried task, FAMA costs 15–37% more than Top-1 (the selector call takes 1.2–1.8 s, less than one executor retry at 4–11 s). Per successful recovery, FAMA is on par with Top-1 and 2–6× cheaper than Reflexion, because it recovers a much larger share of the tasks it retries.

Uncertain outcomes. FAMA is triggered by a failure signal, which in our settings comes from the environment. How FAMA behaves when that signal is noisy or self-judged is a separate question from the selector itself, and we leave it to future work.

We thank you again for the careful review, and we look forward to your reply.

Thanks for the very detailed response and the further experiments. The new robustness, stability, geometry and numerical bound results go a long way to addressing my primary concerns and the controlled 2D example does provide some useful evidence that the geometry sensitive bound can be of value. The rebuttal does provide a lot of empirical backing, though the remaining looseness of the practically relevant bounds means that a large score increase is not really justifiable. I'd keep my rating.

Reviewer_gwUBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies experiential memory selection for LLM agents, highlighting that task similarity does not always reflect the actual utility of a memory for a given agent. Through empirical analyses, the authors find that the way an agent fails can provide useful information for identifying memories that help recover failed tasks, even when the tasks themselves are highly similar.

Based on this observation, the authors propose FAMA, a failure-aware memory selection method. When an agent fails its initial attempt, FAMA analyzes the failed trajectory and selects a potentially useful memory from the retrieved candidates to support a retry. The authors explore both prompting-based and LoRA fine-tuned implementations.

Experiments are conducted on KodCode, ALFWorld, and Spider. The results show that fine-tuned FAMA improves the recovery rate of failed tasks by approximately 8–11 percentage points over Top-1 similarity retrieval. The method also demonstrates some transferability across different executors.

Critical strengths
  1. This paper investigates the gap between task similarity and actual memory utility. Through analyses of failure similarity and missing-behavior matching, the authors show how an agent's failure patterns can influence memory selection. I find these empirical analyses to be one of the most valuable contributions of the paper.

  2. The authors evaluate the method on three datasets covering different types of tasks and include a selector without failure trajectories as a baseline. The cross-executor experiments also provide some evidence of generalization, which strengthens the empirical findings.

Critical weaknesses
  1. Limited methodological novelty.

The core idea of FAMA is to incorporate an agent's failed trajectory into the LLM-based memory selection process and select useful candidate memories by analyzing the failure. While this idea is reasonable, the overall approach is relatively straightforward. The proposed failure diagnosis, gap identification, and memory selection steps mainly rely on the reasoning capabilities of the LLM, with limited additional technical innovation.

  1. Limited evaluation of the method's applicability in more general settings.

Although the paper demonstrates the effectiveness of FAMA across multiple datasets, the experiments are mainly conducted with fixed memory banks containing only verified successful trajectories. In addition, the method requires determining whether the agent's initial attempt has failed before performing memory selection. While this controlled setting is useful for analyzing memory utility, it remains unclear whether FAMA can maintain its advantages with noisy memories, tasks without explicit success feedback, or shifts in task distributions. This limits the generalizability of the current findings.

Questions for authors
  1. Could the authors further clarify the advantages of FAMA over a simpler LLM-based selector that directly uses failed trajectories? For example, if an LLM is given the same task, failed trajectory, and candidate memories, can it achieve comparable performance by directly selecting the most useful memory without explicit failure diagnosis and gap identification?

  2. Have the authors considered evaluating FAMA with memory banks containing unverified or noisy trajectories? Also, how would the method handle tasks where explicit success or failure feedback is unavailable? Further experiments or discussion on these settings would help clarify the practical applicability of the proposed approach.

3 comments

Dear reviewer,

Thank you for the review and for valuing the empirical analyses. Please also see our General Response (top-level comment) for a table of all new experiments. New results below are recovery rates (% of memory-free failures solved on retry) on KodCode / ALFWorld / Spider, same executor and same ten candidates as Table 1.

Novelty. We agree the architecture is simple. What we claim is the finding, not the architecture: among verified-correct and near-identical candidates, how the executor failed informs which one helps (Analysis I-b, II), and a selector that reads the failure recovers 8–11 more points than similarity retrieval. The ablation below makes the claim more precise than "the LLM reasons well".

Simpler selector without explicit diagnosis. We ran it: the same model fine-tuned to output the choice directly from (task, failed trajectory, candidates), with no diagnosis text, gives 50.8 / 45.1 / 37.0, vs. FAMA 49.7 / 48.4 / 41.0. So the explicit diagnosis is not needed on KodCode and is worth +3.3 / +4.0 points on ALFWorld / Spider. This matches Analysis I-a: on KodCode the surface of the failure (the submitted code) already carries the signal; on ALFWorld and Spider it does not, and that is where the diagnosis step pays.

Noisy banks. We agree, and we state in Limitations that the bank holds only verified successful trajectories. We chose this on purpose: with every candidate known to be correct, a memory that fails to help can only be a mismatch with this executor on this task, so selection is the only variable. A noisy bank is a different experiment. Unverified trajectories change the distribution of the bank, lower the chance that a helpful memory is among the top-10 at all, and give the selector a second job on top of gap matching: judging whether a candidate is correct. The selector would likely need retraining with incorrect candidates in its training data. We think this is a good direction and will test it in future work, with banks containing 30% and 50% unverified trajectories.

Tasks without an explicit success signal. FAMA is triggered by a failure signal, which in our settings comes from the environment. How FAMA behaves when that signal is noisy or self-judged is a separate question from the selector itself, and we leave it to future work.

We thank you again for the careful review, and we look forward to your reply.

Thanks for the response and the additional experiments! I appreciate the authors' efforts to clarify the contributions of the paper. However, my main concerns about the limited methodological novelty and practical applicability still remain. I will therefore keep my original score.

Dear Reviewer,

Thank you very much for reading our response and the new results. We understand that concerns remain, and we would like to state our position briefly.

On novelty. We agree that the mechanics are standard: a teacher writes targets, and the selector is trained with SFT. What we consider new is the finding and the object it is about. Reflexion and related work let the model write feedback about its own failure, but do not connect that failure to selection from a memory bank, and existing selectors choose memories without looking at how the agent failed.

On the controlled bank. We hold the bank to verified successful trajectories by design. With a noisy bank, a failed retry has two possible causes: a flawed memory or an unsuitable one, and they cannot be separated. With every candidate known to be correct, a memory that fails to help can only be unsuitable for this agent on this task, which is the quantity the paper studies.

On the failure signal. We agree FAMA depends on an external signal that the first attempt failed. Adding a self-check would make FAMA both a verifier and a selector, and the results would no longer isolate selection. We therefore keep the signal external and state this as a limitation.

Thank you again for the review.

Reviewer_SSd6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper is interested in the LLM agents that can retrieve relevant trajectories from the previous experiences. Most of the works in this domain focus on the semantic similarity between the current task agent is solving and the stored experiences tasks. This work argues that it is also important to take into account agent's failure when performing retrieval of similar experiences. In order to do this authors propose to train FAMA (failure-attributed model allocation) selector that would select the relevant memory not only based on the semantic relevance, but also based on what kind of behaviour the retrieved experience can demonstrate to agent that it misses to solve the task.

The selector is based on Qwen3-4B-instuct, and takes as input the task, the failed trajectory, the top-k most similar trajectories from the memory and outputs the selected trajectory which is the added in the context for the new agent run on the same task. There is failure analysis and missing behaviour identification phase which is done in order to select the relevant trajectory, but this only impacts the selector output, and not downstream agent behaviour. The selector is trained on the failed and recovered trajectories from the backbone, but the exact training data jsutifying which trajectory to select has been created with Gemini 3.7 Flash. The selector is either LoRA finetuned model, or just prompt-based Qwen3-4B-Instruct.

Authors perform most of the experiments with Llama3.1-8B, on 3 benchamarks (AlfWorld, KodCode, Spider), and perfrom also selector transfer analysis from Llama3.1-8B based FAMA on Qwen2.5-7B backbone. It seems that different backbone can still benefit from the selector, which indicates that the main gain comes from the failed trajectory dependence rather than the backbone itself.

Overall the introduction is well written and easy to follow, the method description itself is a bit harder to get since it is mixed with different analysis authors perform, and sometimes I am lost between whether the information is related to the analysis, or to the method itself.

Critical strengths

I actually like the idea. I think the motivation is sound and the experimental design and analysis are nice. The initial results authors report support the hypothesis. However, in my opinion, the current experiments are a bit light to make it convincing enough (cf. critical weaknesses)

Critical weaknesses

Experimental settings are pretty light imo:

  • all the runs are done with the single seed, which is concerning, since we know that there is a high variance for such agents. It is important to perform multiple runs and report mean and std results across the runs
  • More recent and strong backbones would be interesting to test in this setting to make results more convincing
  • Some baseline could make the results more convincing: eg. - A very natural baseline would be to perform retrieval with the failed trajectory as query, against the index of successful trajectories : this would mean that you select from the memory the trajectory which is most semantically close to the failed one. Maybe this alone would be enough to distinguish among multiple possible trajectories existing in the memory for the same task. -Another interesting baseline would be to have top-2 or top-3 similarity (according to https://arxiv.org/abs/2603.18272 this boosts the performance quite significantly on AlfWorld);

Reproducibility might be an issuer: some important technical details are missing : eg.

  • From the motivation it seems to me that FAMA should be sort of ranking model, that would rank different candidates based on the failed trajectory, and select the most promising one. But that is not how it is implemented since it is based on LLM with some reflection in the middle. More details on the implementation and training would be helpful to better understand the work. What is the training loss ? How exactly the training data was built?
  • Again, based on the motivation, that FAMA should be backbone specific I would expect them to train it on the trajectories from the backbone model: eg. backbone generates trajectories for all the training tasks, keep only the failed ones, re-do the execution with top-k relevant trajectories from memory, and create relevant training data for FAMA: failed trajectory --> relevant memory trajectory that allows to train FAMA on these data. I don't really understand what is the role of Gemini 3.7 Flash as a teacher in this pipeline? (cf. line 307-308) If it is for creation of reflection and justification to train model on, it would be interesting to have an ablation on the usefulness of this step in the final results.
Questions for authors
  • You seem to claim that FAMA selector should be specific to the backbone, but it seems that it only depends on the failed trajectory in practice ? This is confirmed by the transfer to different backbone experiments you have in the paper. I would suggest that the gains are lower not because the selector is not adapted, but because the starting point (no memory performance) is much higher.

  • You seem to assume that there is a single way for the agent to fail? I would be curious to see whether it is indeed the case: eg. if you run the same agent several times with different seeds, how would the failure mode change? Would the memory selected for the seed 1 failure still be relevant for the seed 2 failure?

  • there might be the cases where the agent succeeds to solve the task, but once the memory is prepended it fails in solving it. Do you think such cases could also be interesting when learning you FAMA selector? How could they help?

  • If the trajectory is good at recovering any task, you discard it, but why just not keep it? What kind of trajectory is it? Is it some kind of borderline behaviour rather than useful recovery information?

  • There is no really any connection between the identified behaviours and FAMA definition? The behaviours are there just for the analysis purpose?

Writing questions:

  • what does excess agreement in figure 4 refer to?
3 comments

Dear Reviewer,

Thank you for the detailed review. Please also see our General Response (top-level comment) for a table of all new experiments and the cost analysis. Below, new results are recovery rates (% of memory-free failures solved on retry), same executor and same ten candidates as Table 1.

Stronger backbones. Table 2 already tests a second executor (Qwen2.5-7B-Instruct), where FAMA > selector w/o failure > Top-1 holds without retraining. We will add a more recent executor in the revision.

Retrieval with the failed trajectory as query. We added this baseline: the same retriever as Top-1, with the failed trajectory instead of the task description as the query, over the same ten candidates. It gives 39.7 / 34.3 / 34.3 on KodCode / ALFWorld / Spider, vs. Top-1 38.8 / 37.1 / 33.0 and FAMA 49.7 / 48.4 / 41.0. This matches Analysis I-a: surface similarity of failures carries a little signal on KodCode and none on ALFWorld or Spider. The same holds for the cross-encoder with the failed trajectory appended to its query (39.3 / 31.5 / 36.5, no gain over 39.6 / 39.0 / 36.2). Both baselines see the failure but treat it as text to match against; the gain appears only when a model reads it to identify what the failure lacks. What matters is what the failure lacks, not what it looks like.

Top-k. Top-2 gives 47.0 / 30.5 / 31.6, below Top-1 on ALFWorld and Spider. We think this is compatible with Ferraz et al. (2026): their agent starts near zero on ALFWorld (No-RAG 7.5% for Llama-3.1-8B) and they measure success on all tasks, so extra trajectories mainly supply format and procedure; we measure recovery of failures after a task-matched memory is already present. Their own Table 17 also shows that K=2 can fall far below K=1 depending on format (Qwen2.5-7B, textual: 41.8 → 14.9), and they note extra trajectories hurt when conflicting. On KodCode Top-2 reaches 47.0, which we read as KodCode benefiting from a second demonstration.

Success-then-fail cases. These are the 16–26% of solved tasks broken by Top-1 injection (Section 4.3). They are useful negatives, but supervised fine-tuning cannot use them directly; we will note this as future work.

Reproducibility. We will add a full appendix. Training is standard supervised fine-tuning: cross-entropy on the target, which is the failure analysis followed by the index of the chosen candidate. Data: run the executor on training tasks without memory; keep failures; retry each failure once per candidate to get the recovery label; for each (failure, recovering candidate) pair, the teacher writes the analysis; keep it only if the teacher’s choice matches the observed recovery.

Role of the teacher. The pipeline you describe is exactly ours: labels come from the executor’s actual recoveries, not from the teacher. The teacher has two roles. First, it writes the rationale text. Second, it provides attribution: a candidate can recover a task by coincidence rather than by supplying what the failure lacked. The teacher therefore reads the failed trajectory, identifies the gap, and selects the candidate that addresses it, without seeing the recovery labels; an instance is kept only when the teacher’s choice agrees with the observed recovery. This filters out recoveries that cannot be attributed to the gap, so training covers only failures with an identifiable cause (Section 3.3). We ran the ablation you asked for: fine-tuning on the same instances without the rationale (just failure + task + candidates → choice) gives 50.8 / 45.1 / 37.0 vs. 49.7 / 48.4 / 41.0 with it. So the rationale helps on ALFWorld and Spider and not on KodCode. This matches Analysis I-a: on KodCode the surface of the failure already carries the signal, on the other two it does not, and that is where the diagnosis step pays.

Backbone-specific? Agreed, and we will reword: FAMA is executor-specific only through the failed trajectory, not through its parameters, which is what the transfer result shows. On the smaller margins under Qwen2.5-7B: recovery is already conditioned on failure, but the residual failures are harder. We will add the oracle row to Table 2 to separate “harder failures” from “shifted failures” in the final version.

One way to fail? We do not assume this. Analysis I-b shows the opposite: for the same problem, pairs with dissimilar failures show near-zero agreement, pairs with alike failures +2.8 to +4.6.

Discarding trajectories. We believe this refers to the centering in Analysis I-a: tasks that almost any memory recovers would create agreement unrelated to the memory, so we subtract the mean recovery of the task’s other candidates. No trajectory is removed from the bank or from training.

Behaviours vs. FAMA. Analysis II uses a fixed regex vocabulary so the gap-matching signal can be tested without an LLM; it is dataset-specific and only covers behaviours on which candidates differ. FAMA replaces it with free-form gap identification for that reason. The no-rationale ablation above now links the two: where the behaviour is visible on the surface (KodCode) the diagnosis is dispensable; where it is not (ALFWorld, Spider) the diagnosis is where the gain comes from.

Writing. “Excess agreement” is a memory’s recovery on a task minus the mean recovery of that task’s other candidates (Eq. 5).

We thank you again for the careful review, and we look forward to your reply.

Reviewer_SSd6OpenReview ↗

thank you for your comments

Dear authors, thank you for your detailed replies. Most of my concerns are addressed. Therefore I am willing to increase the soundness score. As for the grade, i still ponder between weak accept and weak reject (leaving this for AC), as I wonder whether these findings will still be valid once we switch to much stronger recent models (Eg. qwen3 at least, qwen3.6 or qwen3.8 even better, but maybe less fair since it is too recent with respect to the submission date). Best of luck with your submission

Comments on the paper

2 comments

We thank all reviewers for their careful reading. Two concerns were shared across reviews: whether FAMA’s gain comes from its diagnosis procedure or simply from seeing the failed trajectory, and whether the comparison to baselines is fair. We ran the following additional experiments to address them. All numbers are recovery rates (% of memory-free failures solved on retry) with the same executor and the same ten candidates as Table 1.

MethodKodCodeALFWorldSpider
Random single candidate36.631.031.4
Top-1 similarity (Table 1)38.837.133.0
Top-2 similarity47.030.531.6
Top-1 similarity, failed trajectory as query39.734.334.3
Cross-encoder (Table 1)39.639.036.2
Cross-encoder, query = task + failed trajectory39.331.536.5
Selector w/o failed trajectory, fine-tuned (Table 1)41.941.336.5
Direct selection (fine-tuned, no diagnosis text)50.845.137.0
FAMA, fine-tuned (Table 1)49.748.441.0

Three takeaways.

  1. The failed trajectory is what matters. Giving it to a model that can read it (direct selection) raises recovery by +8.9 / +3.8 / +0.5 over the same model without it. Giving it to models that cannot (similarity to the failure, cross-encoder with the failure appended) does not help.
  2. The explicit diagnosis step helps where the failure’s surface does not carry the signal. On KodCode, direct selection matches FAMA (50.8 vs. 49.7); on ALFWorld and Spider, FAMA is 3.3 and 4.0 points higher. This is consistent with Analysis I-a, where the surface of the failure carries the signal on KodCode but not on the other two datasets.
  3. Injecting more memories is not a substitute for selecting one. Top-2 is below Top-1 on ALFWorld and Spider.

Cost. Measured on one A100-40GB with vLLM, batch 1, in GPU-seconds. “Per retried task” is the selector call plus one executor retry for every memory-free failure (Top-1 has no selector call); “per successful recovery” divides that by the recovery rate.

DatasetTop-1 (per retry)FAMA (per retry)Reflexion (per retry)Top-1 (per recovery)FAMA (per recovery)Reflexion (per recovery)
KodCode4.15.68.310.611.343.9
Spider6.47.69.019.418.535.0
ALFWorld11.012.816.929.626.4149.6

The selector call takes 1.2–1.8 s, less than one executor retry (4–11 s). Per successful recovery, FAMA is on par with Top-1 and 2–6× cheaper than Reflexion.

Dear AC and Reviewers,

We have uploaded a revised version of the paper based on the reviewers' advice.

We thank the reviewers for their suggestions and comments, and look forward to your reply.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.