- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes CORA, a method for organizing and retrieving experiential memories for large language model agents. To address the limitations of semantic similarity, which does not adequately reflect or distinguish the practical usefulness of memories, and scalar utility scores, which ignore differences across task categories, CORA maintains a category-conditioned utility profile for each memory and clusters memories with similar performance patterns into capability-oriented regions. During recall, CORA combines individual and regional utility estimates to rerank memories. It also conditionally replaces retrieved failure memories with region-level failure summaries. The authors evaluate CORA on ALFWorld, BigCodeBench, and Lifelong Agent Bench. The results show that CORA outperforms existing memory baselines in most experimental settings.
Critical strengths
-
CORA distinguishes between frequently retrieved candidate memories and infrequently used reference memories according to their usage frequency across different task categories. Through regional information sharing, it allows low-frequency memories to benefit from the experience of related high-frequency memories.
-
Semantically similar memories are not necessarily useful for completing a task, and the same memory may perform differently across task categories. CORA replaces a single scalar utility score with category-conditioned utility profiles, thereby preserving these differences. The proposed design is well aligned with the limitations identified in the paper.
Critical weaknesses
-
The choice of the retrieval weight lambda is insufficiently justified. Equation (10) defines lambda as the parameter that balances semantic similarity and memory utility. However, the paper does not adequately explain how its value was selected. ALFWorld and LLB-OS use lambda=0.55, whereas LLB-DB and BigCodeBench use lambda=0.50. The authors do not explain why different datasets require different weights or whether these values were selected through validation experiments.
-
CORA relies on predefined task categories, which may limit its applicability to complex or ambiguous tasks. The datasets used in CORA either provide predefined categories or use task categories constructed by the authors. However, for complex or ambiguous tasks, it may be difficult to define a small and stable set of task categories. For example, LoCoMo contains memories from everyday human–machine conversations rather than model-generated trajectories or task experiences, making it difficult to partition memories into fixed categories. It remains unclear whether CORA would be effective in such settings.
-
The utility score may not reliably measure the causal contribution of an individual memory. CORA updates the category-specific utility of retrieved memories based on the final task outcome. However, task performance is also affected by the model’s inherent capabilities, task difficulty, and other retrieved memories. Therefore, a high utility score does not necessarily imply that the corresponding memory contributed substantially to the successful outcome. Can the authors design additional experiments to verify whether the utility scores reliably reflect the actual helpfulness of individual memories?
Questions for authors
-
Why do the authors use different backbone models for different datasets? In related work, it is common to evaluate the same dataset with multiple models in order to assess the robustness of the method across different backbones.
-
Table 4 reports 500 tasks in total with a 7:3 train/test split, which implies 350 training tasks. However, Table 8 reports results on 361 LLB-DB training tasks. Could the authors clarify this discrepancy?
-
The abstract submitted through the ICLR interface contains LaTeX commands such as \textbf{}.
6 comments
# Response to Reviewer jC7t - Part 2
The results are shown in the table below. All F1 values are official LoCoMo scores on held-out test questions, aggregated across the evaluation folds. Training questions are used only to learn the utility and region state for MemRL and CORA; they are not included in the reported test scores. The paired analysis uses the same held-out question for each method.
| Temporal | Adversarial | |
|---|---|---|
| F1 | ||
| RAG | 28.28 | 93.72 |
| MemRL | 28.60 | 93.95 |
| CORA | 29.61 | 94.17 |
| Paired comparison | ||
| CORA vs. RAG (Δ ± SE) | +1.33 ± 0.75 | +0.45 ± 0.55 |
On temporal questions, CORA reaches 29.61 F1, exceeding RAG's 28.28 and MemRL's 28.60. This improvement is consistent with CORA's design: temporal questions involve date-sensitive evidence and heterogeneous conversational facts, so category-conditioned utility avoids collapsing different retrieval requirements into one scalar, while capability-oriented regions share sparse feedback among memories with similar utility profiles. CORA also achieves the highest adversarial score (94.17 F1 vs. 93.72 and 93.95), showing that this category-conditioned organization remains effective when the conversation does not support an answer. Together, these results indicate that CORA can be applied to complex and ambiguous task without relying on a predefined question taxonomy to discover categories or route test queries. We have added these results and the accompanying analysis to Appendix C.3, “Applicability to Complex and Ambiguous Tasks,” with the new text highlighted in blue.
W3. Utility reflects the causal contribution of individual memories. MemRL provides a direct positive precedent for this interpretation. It treats selecting a particular memory m as the retrieval action and defines its memory-specific Q(s,m) as the expected return after that selection. Its analysis finds a strong relationship between memory-level Q-values and downstream success (Pearson r = 0.861), with success increasing from 21.5% in the lowest Q bin to 88.1% in the highest. High-utility failure reflections also transfer useful recovery strategies, with representative cases achieving 19/19 and 88/94 successful retrievals. Because the estimate is indexed by the individual memory, these results support interpreting high utility as evidence that the selected memory contributes to successful task completion.
Beyond MemRL's positive evidence, we independently test whether utility reflects individual-memory contribution by evaluating three conditions on 342 held-out BigCodeBench tasks using a frozen memory bank: keep the retrieved memories, remove one memory, or replace one with a semantically close, same-category memory with lower Q. Removing one memory reduces success by 0.9 percentage points on average, while replacing it with a semantically close, same-category memory with lower Q reduces success by 1.5 points. The larger decrease under replacement than removal indicates that the original higher-utility memory is more useful than a matched lower-Q alternative. Together, these results support the interpretation that utility reflects the contribution of individual memories to task completion.
# Response to Reviewer jC7t - Part 3
Q1. Different backbone models across datasets.
We use benchmark-specific backbones for two reasons. First, we follow the evaluation protocol of MemRL, our most direct utility-aware baseline. MemRL also uses different backbone models for different datasets. Matching its benchmark-specific assignments keeps our comparison aligned with the reference implementation. Within each benchmark, CORA and the other baselines use the same frozen backbone, benchmark interface, task split, and evaluation protocol. This isolates the effect of the memory mechanism.
Second, the benchmarks exercise different capabilities: ALFWorld emphasizes embodied planning and action selection, BigCodeBench emphasizes code generation and debugging, and LLB-OS/LLB-DB emphasize long-horizon interaction with their respective tool and data interfaces. A backbone suitable for one setting is not necessarily equally capable in another. A model that is too weak for code generation or tool use may fail even when it retrieves useful memories, while a model near a performance ceiling leaves little headroom for memory methods.
Q2. Stratified splitting explains the 361 training tasks.
We use a 7:3 split stratified by the complete skill_list combination. Because each skill group contributes max(1, floor(0.7n)) tasks to training, singleton groups are assigned entirely to training, resulting in 361 training and 139 test tasks out of 500. Thank you for pointing this out; we have accordingly revised Appendix A.2 to document this splitting rule, with the changes highlighted in blue.
Q3. LaTeX commands in the submitted abstract.
Thank you for pointing this out and we have revised the submitted abstract to remove the LaTeX commands.
We again sincerely thank you for engaging carefully with CORA and for the constructive suggestions. We sincerely hope that our responses and revisions address your concerns, and, if you find them satisfactory, we would be very grateful if you would consider raising the evaluation of our paper.
# Response to Reviewer jC7t - Part 1
We sincerely thank the reviewer for the careful reading and constructive comments, and we are especially grateful that the reviewer recognizes CORA's design as well aligned with the limitations of similarity-based and scalar-utility retrieval, while its regional information sharing allows infrequently used memories to benefit from related frequently used memories. We have carefully considered every concern and question, conducted the corresponding experiments, which took a few days to complete, and revised the manuscript accordingly, with changes highlighted in blue. Below, we address the reviewer's concerns point by point.
W1. Training-stage selection of the utility weight. Regarding the selection of the λ parameter, we initially used the balanced setting. During training, we found that giving the category-conditioned utility a slightly larger weight improved performance on ALFWorld, where task procedures are highly repetitive and related tasks expose the agent to similar workflows and outcomes. We therefore used λ=0.55 for ALFWorld. LLB-OS has the same characteristic: many tasks reuse related command workflows, so we used the same λ=0.55 setting. For LLB-DB and BigCodeBench, we retained the balanced λ=0.50 setting. These values were fixed at the benchmark level before final evaluation and were not adjusted per task or according to test outcomes. We have added this training-stage selection rationale to Appendix A.1, with the added text highlighted in blue.
W2. Applicability to complex and ambiguous tasks. To evaluate CORA's applicability to complex and ambiguous tasks, we conduct an evaluation on LoCoMo's long-context conversational question-answering task. We use the official observation facts as memories and compare CORA with semantic RAG and scalar-utility MemRL under the same evaluation setup. For this evaluation, CORA obtains the dimensions of its category-conditioned utility from KMeans clustering of training-question embeddings. Each cluster defines one latent category and one utility coordinate; each test question is assigned to the nearest frozen cluster centroid.
We report two complementary question types that directly probe memory selection in the scope of our paper. Temporal questions require identifying date-relevant facts across a long conversation, while adversarial questions test whether the system uses only information supported by the conversation and abstains when no answer is supported. Single-hop questions are largely a semantic-retrieval test, while open-domain questions additionally depend on world knowledge and generator behavior; these categories are less diagnostic of capability-conditioned memory organization.
The questions are split into three folds with stratification by conversation and official question category. Each fold contains 1,324 training questions and 662 held-out test questions, and every question serves as test exactly once across the three folds. MemRL and CORA use the same training questions and the same number of training passes: each retrieved training question produces an official LoCoMo F1 reward that updates MemRL's scalar utility or CORA's category-conditioned utility and region state. RAG uses the same held-out test questions without a utility-training stage. After training, the MemRL and CORA states are frozen and evaluated on the held-out test questions. Each held-out test question is evaluated once by each method, with no test-time utility updates or repeated test passes.