acceptodds
Under review as a conference paper at ICLR 2027

CoMem: Compounding Retrieved and Parametric Memory for Reflective Language Agents

Abstract

Reflection lets LLM agents improve from their own attempts, but the memory it builds is local to a single task, so an agent reflecting on its own tends to revisit the same explanation for failure. Cross-task reflective memory addresses this in two forms: retrieved memory keeps past experience explicit and looks up related cases, which is specific but bounded by the bank, and parametric memory internalizes experience into parameters, which generalizes but is grounded in nothing beyond the task description. When both are used, existing agents combine them additively, feeding their outputs to the solver side by side, so retrieved experience never changes what the learned memory produces. We introduce CoMem (Compounding Memory), which makes retrieved experience an input to the learned memory itself, at training and at inference. The agent builds a bank of its own successful and failed reflections, learns from retrieved cases to write failure hypotheses, candidate explanations of how an attempt is likely to go wrong, and at test time reasons from newly retrieved cases to guide each attempt. Varying the retrieved cases gives the memory different perspectives, yielding reflective diversity that is not merely sampled but also grounded in retrieved experience. Across two backbones and five benchmarks in math, code, and multi-hop QA, CoMem achieves strong overall performance. Additional evaluations show that it generalizes to out-of-distribution tasks and enables smaller models to provide effective guidance to stronger solvers.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Language Agent, Reflection, Agent Memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8BuBOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper trains a memory model with LoRA, to retrieve the success and failure history each, and create the task-wise failure hypothesis to guide the frozen solver. This paper starts from an observation that that reflection-based agents repeat a same failure explanations, therefore retrying do not help. This paper proposes to compound the previous cases with learned parametric memory to compound and create a diverse and grounded reflection.

They propose a hybrid method of two memory types managed across task: retrieved memory from a bank by embedding similarity and parametric memory which is learned into model weight.

Critical strengths
  • The paper starts from a strong hypothesis: Failure analysis stay repetitive and superficial while diverse reflection leads to task success.
  • The method is easy to understand and intuitive.
Critical weaknesses
  • This paper does not show a direct evidence that diversity actually maps to true failure modes
  • Limited novelty compared to ParamAgent-plus. The difference seems to be mainly using the search pairs as a input to parametric memory during SFT. This is closer to retrieval-augmented fine-tuning.
  • Limited evidence on compounding vs additive. I need a clean ablation that compounding architecture is better than additive architecture.
Questions for authors

Is the improvement of the proposed method statistically significant compared to ParamMem? Can you report the result seed-wise and paired bootstrap on each problem case?

Reviewer_kAVfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors consider that in the reflection process, the model's past experience is referenced either through a retrieval memory, such as RAG, or through a parametric memory, such as RL or LoRA fine-tuning. The authors aim to combine these two, rather than treating them as two independent supplementary sources of information. In terms of methodology, they train a memory module. First, they use the base model to solve problems. If a problem is solved in one attempt, it is skipped. They set the number of reflections to K=3. If successful, it goes into positive experience; if it fails, it goes into negative experience. The second step is to prepare a fine-tuning dataset, so that the base model changes its output. The input is the question q, one relevant positive experience e, and one relevant negative experience e. The output is the potential pitfalls that may be encountered. Previously, the output was how to solve the problem; now it is changed to output the potential pitfalls. Here, the authors manually filter out vague and empty pitfalls to increase the quality of the dataset. Finally, they obtain the fine-tuning dataset and use LoRA to fine-tune a memory module. In the final experiments, after inputting a question, one relevant positive experience e and one relevant negative experience e are still attached, and the memory module is simultaneously made to output the potential pitfalls. This reflects the fusion of parametric memory from retrieval memory, and proposes a way to use parametric memory and experiential memory in a fused manner. In the experimental section, the authors validate on two models (Qwen3-4B and Qwen3-8B) and five benchmarks, including mathematical reasoning, code generation, and multi-hop question answering, comparing methods such as Base, Reflexion, DoT-Bank (retrieval memory), and ParamMem (parametric memory). The results show that CoMem outperforms retrieval memory and parametric memory methods alone on most tasks. For example, under the Qwen3-8B setting, compared with ParamMem, CoMem achieves performance of 94.72, 95.09, 59.67, 94.9, and 85.6 on MATH, HumanEval+, LiveCodeBench, HotpotQA, and MuSiQue, respectively, showing improvements or maintaining a lead over ParamMem on all of them.

Critical strengths

The advantage of this paper lies in that, on the basis of parametric reflective memory, it additionally introduces historical experience relevant to the current task into the input, so that reflection generation no longer relies solely on the general failure patterns learned internally by the model, but can instead combine task-specific success and failure trajectories to generate more targeted reflection hypotheses. The experimental results also show that this experience-conditioned reflection mechanism can improve the diversity and effectiveness of the reflection content, thereby further enhancing the agent's problem-solving ability on mathematical, code, and question-answering tasks.

Critical weaknesses

The main problem with this work is that its content strongly overlaps with the existing parametric memory method (ParamMem, ICML 2026). ParamMem has already proposed improving the diversity of reflection through a parametric memory module, so the improvement in reflection diversity mentioned in this work (CoMem) cannot really be regarded as an innovation. In comparison, the main difference of CoMem lies in using retrieved database experience as an additional input to the parametric memory module, thereby generating more relevant reflection hypotheses, which is more like an improvement or extension of the existing method.

Questions for authors

The main contribution claimed by the paper is the fusion of experiential memory and parametric memory, but from the experiments and the comparison with ParamMem, it is impossible to distinguish whether the performance improvement comes from the new mechanism itself or simply because the model has additional input and therefore performs better. Can it be proven that the improvement comes from the interactive fusion of the two memories rather than from the introduction of additional information?

Reviewer_wLNaOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CoMem, a reflective memory method that conditions a trainable hypothesis generator on retrieved cross-task experience, both when producing its training targets and at inference. The experience bank contains successful and unsuccessful reflection transitions from the backbone’s own attempts. For each task, the method retrieves positive and negative examples, uses the base model to generate task-specific failure hypotheses, and fine-tunes a LoRA memory module on these retrieval-grounded targets. At test time, multiple sampled reference pairs produce multiple hypothesis sets that guide a frozen solver alongside its task-local reflections; unresolved tasks receive an additional retry using a successful test-time trajectory when available. Experiments use Qwen3-4B/8B on MATH, HumanEval+, LiveCodeBench, HotpotQA, and MuSiQue, with three-seed results, plus out-of-distribution and weak-to-strong transfer evaluations. CoMem generally improves over Reflexion, DoT-Bank, and ParamMem, and the paper also reports increased hypothesis diversity under embedding-distance, clustering, and lexical metrics.

Critical strengths
  1. A clear and practically motivated integration of episodic and parametric memory. CoMem’s memory module consumes retrieved positive and negative experience at training and inference, rather than passing retrieved items and an independently generated reflection to the solver in parallel.
  2. Broad evaluation with multiple complementary checks. The paper evaluates five in-domain benchmarks across math, code, and QA, reports three-seed means and standard deviations, and includes OOD transfer, smaller-memory training, weak-to-strong transfer, and multiple diversity measurements.
Critical weaknesses
  1. The key “compounding” effect is not cleanly separated from the added inference-time budget and conditioning. CoMem generates eight hypothesis sets per test task (Algorithm 1), samples new references across attempts, and for unresolved tasks has an additional stage using a successful test-time trajectory. The shared second stage is a helpful control against DoT-Bank, but the paper does not make sufficiently clear whether the main comparisons equalize total solver attempts, generated guidance tokens, and model calls across ParamMem, DoT-Bank, and CoMem. Nor is there a factorial control that gives a retrieval-conditioned generator the same number of hypotheses and calls without the proposed retrieval-grounded training. This matters because the main performance gains are attributed to compounding rather than simply more or differently conditioned guidance. A compute- and attempt-matched ablation would materially strengthen that attribution.
  2. The operational diversity measure does not yet establish useful semantic coverage. The main diversity metric embeds an entire hypothesis set and averages pairwise cosine distances; this can increase through paraphrase or formatting variation without introducing distinct actionable failure modes. The QA lexical metrics are supportive but likewise do not assess correctness or utility of individual hypotheses.
Questions for authors
  1. For the main Table 1 comparisons, what are the exact per-task solver attempt limits, number of guidance generations, model calls, and total generated tokens for CoMem versus ParamMem and DoT-Bank? Are those quantities matched, and if not, could the authors provide a compute- and attempt-matched comparison or ablation? This could change how much of the observed improvement is attributable to retrieval-conditioned learning itself.
  2. Can the authors quantify how often independently sampled reference pairs at inference are actually distinct, and how often their resulting hypotheses add a new semantic failure category rather than a paraphrase? In particular, how does removing redundant hypotheses affect task performance at a fixed token/attempt budget?
  3. The QA results use Qwen3.6-35B-A3B as the semantic judge. Were judge prompts and decisions checked against human labels on a sample, and could judge disagreement or sensitivity affect the reported HotpotQA/MuSiQue differences (including CoMem’s lower Qwen3-4B MuSiQue score than ParamMem)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.