acceptodds
Under review as a conference paper at ICLR 2027

CORA: Capability-Oriented Region Aggregation for Experiential Memory in LLM Agents

Abstract

Large language model agents increasingly accumulate rich experiential memories, yet reusing this experience under task diversity remains a central challenge. Similarity-based retrieval selects memories by topical resemblance, but its ranking signal contains no information about previous task outcomes. Utility-aware retrieval incorporates outcome-based usefulness, but a scalar utility score combines results from different task categories into one value, while estimating utility independently for each memory makes sparse feedback noisy and prevents evidence sharing across memories with similar behavior. We therefore introduce Capability-Oriented Region Aggregation (CORA), which represents each memory by a category-conditioned utility profile and groups memories with similar profiles into capability-oriented regions. CORA comprises three mechanisms: Capability Profile Clustering discovers these regions; Hierarchical Utility Estimation combines individual and regional utility for category-specific reranking; and Region-Level Knowledge Aggregation consolidates recurring failure patterns within capability-similar regions into retrieval-time guidance. Experiments across four benchmarks and five frozen language-model backbones show that CORA achieves final-test gains of up to 6.43 percentage points over the strongest competing method, demonstrating the effectiveness of capability-oriented regions for organizing and reusing experiential memory across diverse tasks.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
experiential memory, capability-oriented regions, LLM agents, region-based aggregation.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jC7tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes CORA, a method for organizing and retrieving experiential memories for large language model agents. To address the limitations of semantic similarity, which does not adequately reflect or distinguish the practical usefulness of memories, and scalar utility scores, which ignore differences across task categories, CORA maintains a category-conditioned utility profile for each memory and clusters memories with similar performance patterns into capability-oriented regions. During recall, CORA combines individual and regional utility estimates to rerank memories. It also conditionally replaces retrieved failure memories with region-level failure summaries. The authors evaluate CORA on ALFWorld, BigCodeBench, and Lifelong Agent Bench. The results show that CORA outperforms existing memory baselines in most experimental settings.

Critical strengths
  1. CORA distinguishes between frequently retrieved candidate memories and infrequently used reference memories according to their usage frequency across different task categories. Through regional information sharing, it allows low-frequency memories to benefit from the experience of related high-frequency memories.

  2. Semantically similar memories are not necessarily useful for completing a task, and the same memory may perform differently across task categories. CORA replaces a single scalar utility score with category-conditioned utility profiles, thereby preserving these differences. The proposed design is well aligned with the limitations identified in the paper.

Critical weaknesses
  1. The choice of the retrieval weight lambda is insufficiently justified. Equation (10) defines lambda as the parameter that balances semantic similarity and memory utility. However, the paper does not adequately explain how its value was selected. ALFWorld and LLB-OS use lambda=0.55, whereas LLB-DB and BigCodeBench use lambda=0.50. The authors do not explain why different datasets require different weights or whether these values were selected through validation experiments.

  2. CORA relies on predefined task categories, which may limit its applicability to complex or ambiguous tasks. The datasets used in CORA either provide predefined categories or use task categories constructed by the authors. However, for complex or ambiguous tasks, it may be difficult to define a small and stable set of task categories. For example, LoCoMo contains memories from everyday human–machine conversations rather than model-generated trajectories or task experiences, making it difficult to partition memories into fixed categories. It remains unclear whether CORA would be effective in such settings.

  3. The utility score may not reliably measure the causal contribution of an individual memory. CORA updates the category-specific utility of retrieved memories based on the final task outcome. However, task performance is also affected by the model’s inherent capabilities, task difficulty, and other retrieved memories. Therefore, a high utility score does not necessarily imply that the corresponding memory contributed substantially to the successful outcome. Can the authors design additional experiments to verify whether the utility scores reliably reflect the actual helpfulness of individual memories?

Questions for authors
  1. Why do the authors use different backbone models for different datasets? In related work, it is common to evaluate the same dataset with multiple models in order to assess the robustness of the method across different backbones.

  2. Table 4 reports 500 tasks in total with a 7:3 train/test split, which implies 350 training tasks. However, Table 8 reports results on 361 LLB-DB training tasks. Could the authors clarify this discrepancy?

  3. The abstract submitted through the ICLR interface contains LaTeX commands such as \textbf{}.

6 comments
AuthorsOpenReview ↗

# Response to Reviewer jC7t - Part 1

We sincerely thank the reviewer for the careful reading and constructive comments, and we are especially grateful that the reviewer recognizes CORA's design as well aligned with the limitations of similarity-based and scalar-utility retrieval, while its regional information sharing allows infrequently used memories to benefit from related frequently used memories. We have carefully considered every concern and question, conducted the corresponding experiments, which took a few days to complete, and revised the manuscript accordingly, with changes highlighted in blue. Below, we address the reviewer's concerns point by point.

W1. Training-stage selection of the utility weight. Regarding the selection of the λ parameter, we initially used the balanced setting. During training, we found that giving the category-conditioned utility a slightly larger weight improved performance on ALFWorld, where task procedures are highly repetitive and related tasks expose the agent to similar workflows and outcomes. We therefore used λ=0.55 for ALFWorld. LLB-OS has the same characteristic: many tasks reuse related command workflows, so we used the same λ=0.55 setting. For LLB-DB and BigCodeBench, we retained the balanced λ=0.50 setting. These values were fixed at the benchmark level before final evaluation and were not adjusted per task or according to test outcomes. We have added this training-stage selection rationale to Appendix A.1, with the added text highlighted in blue.

W2. Applicability to complex and ambiguous tasks. To evaluate CORA's applicability to complex and ambiguous tasks, we conduct an evaluation on LoCoMo's long-context conversational question-answering task. We use the official observation facts as memories and compare CORA with semantic RAG and scalar-utility MemRL under the same evaluation setup. For this evaluation, CORA obtains the dimensions of its category-conditioned utility from KMeans clustering of training-question embeddings. Each cluster defines one latent category and one utility coordinate; each test question is assigned to the nearest frozen cluster centroid.

We report two complementary question types that directly probe memory selection in the scope of our paper. Temporal questions require identifying date-relevant facts across a long conversation, while adversarial questions test whether the system uses only information supported by the conversation and abstains when no answer is supported. Single-hop questions are largely a semantic-retrieval test, while open-domain questions additionally depend on world knowledge and generator behavior; these categories are less diagnostic of capability-conditioned memory organization.

The questions are split into three folds with stratification by conversation and official question category. Each fold contains 1,324 training questions and 662 held-out test questions, and every question serves as test exactly once across the three folds. MemRL and CORA use the same training questions and the same number of training passes: each retrieved training question produces an official LoCoMo F1 reward that updates MemRL's scalar utility or CORA's category-conditioned utility and region state. RAG uses the same held-out test questions without a utility-training stage. After training, the MemRL and CORA states are frozen and evaluated on the held-out test questions. Each held-out test question is evaluated once by each method, with no test-time utility updates or repeated test passes.

AuthorsOpenReview ↗

# Response to Reviewer jC7t - Part 2

The results are shown in the table below. All F1 values are official LoCoMo scores on held-out test questions, aggregated across the evaluation folds. Training questions are used only to learn the utility and region state for MemRL and CORA; they are not included in the reported test scores. The paired analysis uses the same held-out question for each method.

TemporalAdversarial
F1
RAG28.2893.72
MemRL28.6093.95
CORA29.6194.17
Paired comparison
CORA vs. RAG (Δ ± SE)+1.33 ± 0.75+0.45 ± 0.55

On temporal questions, CORA reaches 29.61 F1, exceeding RAG's 28.28 and MemRL's 28.60. This improvement is consistent with CORA's design: temporal questions involve date-sensitive evidence and heterogeneous conversational facts, so category-conditioned utility avoids collapsing different retrieval requirements into one scalar, while capability-oriented regions share sparse feedback among memories with similar utility profiles. CORA also achieves the highest adversarial score (94.17 F1 vs. 93.72 and 93.95), showing that this category-conditioned organization remains effective when the conversation does not support an answer. Together, these results indicate that CORA can be applied to complex and ambiguous task without relying on a predefined question taxonomy to discover categories or route test queries. We have added these results and the accompanying analysis to Appendix C.3, “Applicability to Complex and Ambiguous Tasks,” with the new text highlighted in blue.

W3. Utility reflects the causal contribution of individual memories. MemRL provides a direct positive precedent for this interpretation. It treats selecting a particular memory m as the retrieval action and defines its memory-specific Q(s,m) as the expected return after that selection. Its analysis finds a strong relationship between memory-level Q-values and downstream success (Pearson r = 0.861), with success increasing from 21.5% in the lowest Q bin to 88.1% in the highest. High-utility failure reflections also transfer useful recovery strategies, with representative cases achieving 19/19 and 88/94 successful retrievals. Because the estimate is indexed by the individual memory, these results support interpreting high utility as evidence that the selected memory contributes to successful task completion.

Beyond MemRL's positive evidence, we independently test whether utility reflects individual-memory contribution by evaluating three conditions on 342 held-out BigCodeBench tasks using a frozen memory bank: keep the retrieved memories, remove one memory, or replace one with a semantically close, same-category memory with lower Q. Removing one memory reduces success by 0.9 percentage points on average, while replacing it with a semantically close, same-category memory with lower Q reduces success by 1.5 points. The larger decrease under replacement than removal indicates that the original higher-utility memory is more useful than a matched lower-Q alternative. Together, these results support the interpretation that utility reflects the contribution of individual memories to task completion.

AuthorsOpenReview ↗

# Response to Reviewer jC7t - Part 3

Q1. Different backbone models across datasets.

We use benchmark-specific backbones for two reasons. First, we follow the evaluation protocol of MemRL, our most direct utility-aware baseline. MemRL also uses different backbone models for different datasets. Matching its benchmark-specific assignments keeps our comparison aligned with the reference implementation. Within each benchmark, CORA and the other baselines use the same frozen backbone, benchmark interface, task split, and evaluation protocol. This isolates the effect of the memory mechanism.

Second, the benchmarks exercise different capabilities: ALFWorld emphasizes embodied planning and action selection, BigCodeBench emphasizes code generation and debugging, and LLB-OS/LLB-DB emphasize long-horizon interaction with their respective tool and data interfaces. A backbone suitable for one setting is not necessarily equally capable in another. A model that is too weak for code generation or tool use may fail even when it retrieves useful memories, while a model near a performance ceiling leaves little headroom for memory methods.

Q2. Stratified splitting explains the 361 training tasks.

We use a 7:3 split stratified by the complete skill_list combination. Because each skill group contributes max(1, floor(0.7n)) tasks to training, singleton groups are assigned entirely to training, resulting in 361 training and 139 test tasks out of 500. Thank you for pointing this out; we have accordingly revised Appendix A.2 to document this splitting rule, with the changes highlighted in blue.

Q3. LaTeX commands in the submitted abstract.

Thank you for pointing this out and we have revised the submitted abstract to remove the LaTeX commands.

We again sincerely thank you for engaging carefully with CORA and for the constructive suggestions. We sincerely hope that our responses and revisions address your concerns, and, if you find them satisfactory, we would be very grateful if you would consider raising the evaluation of our paper.

Reviewer_BYVROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
3
Read review
Paper summary

This paper focuses on improving memory retrieval and reuse for LLM agents. The authors argue that existing methods often assign a single utility score to each memory, overlooking its effectiveness across different task types, while sparse feedback makes these scores unreliable. To address this, CORA groups memories based on their task-specific utility profiles and uses region-level information to improve utility estimation and aggregate common failure patterns. Experiments on four benchmarks show that CORA generally outperforms existing memory methods.

Critical strengths

The paper identifies a meaningful limitation of existing utility-based memory retrieval methods: a single utility score cannot capture how useful a memory is across different task categories. Organizing memories by capability profiles is a reasonable and interesting idea, and sharing information across memories provides a practical way to address sparse feedback.

Critical weaknesses
  1. Limited novelty. The proposed method appears to be a combination of several existing techniques, including clustering, hierarchical utility estimation, and failure aggregation. Although CORA achieves some performance improvements, these gains seem to come from combining several incremental modifications.

  2. Concerns about the effectiveness of the method. While CORA shows clear improvements on ALFWorld, the gains on other benchmarks are relatively small, and it slightly underperforms MemRL on BigCodeBench at the final epoch. Moreover, as reported in Appendix D.1.2 (Table 9), the improvements over Category-Q-only and embedding-space clustering on BigCodeBench are not statistically significant (p = 0.256 and p = 0.108, respectively).

Questions for authors
  1. In Section 4.3.2, the embedding-space baseline ends up with only one active region, even though multiple regions were initially identified. Could the authors explain why this happens? Does the collapse occur during clustering or during the later region assignment steps?

  2. Could the authors compare CORA with a simpler baseline that uses category-specific Q-values, global utility estimation, and global failure summaries without clustering? This would help clarify whether capability-based grouping is really necessary for the reported improvements.

  3. CORA relies on predefined task categories to construct capability profiles and estimate memory utility. Could the authors evaluate how the method performs when some category labels are incorrect or noisy? For example, randomly changing 10% of the task labels and assess how sensitive CORA is to category classification errors.

6 comments
AuthorsOpenReview ↗

# Response to Reviewer BYVR - Part 1

We sincerely thank the reviewer for the careful reading and constructive comments, and we are especially grateful that the reviewer recognizes CORA's identification of scalar utility's limitation, the value of capability-profile-based memory organization, and the practicality of sharing information across memories to address sparse feedback. We have carefully considered every concern and question, conducted the corresponding experiments, which took a few days to complete, and revised the manuscript accordingly, with changes highlighted in blue. Below, we address the reviewer's concerns point by point.

W1. Novelty and the source of the gains.

We would like to clarify that CORA's novelty lies in a capability-oriented framework for organizing and reusing experiential memory, rather than in any single primitive used to implement it. The framework uses outcome-derived capability profiles as a shared basis for memory organization, category-specific utility estimation, and failure-knowledge transfer. Clustering, hierarchical utility estimation, and failure aggregation are existing techniques that instantiate components of this framework; they are implementation means, whereas the contribution is their capability-oriented integration.

More broadly, this problem is important because the value of experiential memory depends on whether accumulated experience can be reused reliably as the memory bank grows and task distributions diversify. If memories are selected only by topical similarity, the agent may retrieve experiences that appear relevant but have repeatedly failed; if outcomes are collapsed into a scalar utility, category-specific strengths and weaknesses are obscured, while sparse per-memory feedback remains noisy. Consequently, simply accumulating more experience does not guarantee more reliable behavior. CORA addresses this bottleneck by preserving category-conditioned usefulness and sharing evidence among memories with similar capability profiles, allowing memory growth to translate into more dependable reuse across heterogeneous tasks.

The gains should therefore be attributed to the shared capability structure and its coordinated use throughout the framework, rather than to a simple accumulation of independent modifications. The Category-Q-only variant and global, subtask-conditioned, or inline failure summaries do not reproduce the behavior of the complete framework, supporting the framework-level contribution of CORA.

AuthorsOpenReview ↗

# Response to Reviewer BYVR - Part 2

W2. Effectiveness of the method.

We would like to respectfully clarify that CORA's gains extend beyond ALFWorld; the relative magnitude of the gains varies with the characteristics of each task distribution and the available headroom in each setting. On ALFWorld, CORA achieves E10 gains of 2.85 and 6.43 percentage points over the strongest competing method with Qwen3.6-35B-A3B and Qwen2.5-72B-Instruct, respectively. Beyond these gains, CORA also improves over MemRL (the most direct utility-aware baseline) on LLB-OS, raising E10 SR from 73.33% to 76.67% (+3.34 points). On LLB-DB, CORA improves over MemRL from 78.42% to 79.86% at E10 (+1.44 points). LLB-DB has less available headroom than ALFWorld; a positive 1.44-point gain in this regime still reflects CORA's effectiveness. ALFWorld requires multi-step embodied interaction, exposing more varied task trajectories and failure modes. In contrast, LLB-OS and LLB-DB evaluate more structured operating-system commands and database operations. These task properties create different amounts of opportunity for capability-conditioned memory reuse; the smaller absolute gain on LLB-DB therefore reflects the task setting's available headroom, while the direction remains positive.

As for the BigCodeBench result, CORA's E1--E10 mean is 51.75% versus 50.50% for MemRL, and its peak is 54.09% versus 52.34%. CORA's lower E10 value (49.71% versus 50.29% for MemRL) can be understood from the evaluation trajectory and generation process. CORA is ahead of MemRL at E9 (52.05% vs. 50.29%), reflecting CORA's stronger performance over the preceding epochs. From E9 to E10, however, CORA's SR decreases to 49.71%, while MemRL remains at 50.29%. CORA rewrites previously passing programs more often (76.4% vs. 68.5%), and the E10 fix rate among CORA's rewritten cases is 34.6%, compared with approximately 49.5% in its earlier epochs and 50.0% for MemRL. Thus, the final update removes CORA's E9 advantage because more programs are rewritten and fewer of those rewrites recover a passing solution in this round. E10 is CORA's lowest point in only this one of the five settings, while averaging the last two or three epochs gives a positive BigCodeBench difference of +0.58 or +1.85 percentage points. This preserves the overall conclusion that CORA improves performance over the full trajectory. We have added this analysis to §4.2, highlighted in blue.

We would like to respectfully clarify that the Table 9 rows are component ablations within the CORA framework rather than independent competing methods. The task-paired E10 favor full CORA by +2.34 points over Category-Q-only and +3.22 points over embedding-space regions, with asymmetric unique-win counts of 23/15 and 25/14, respectively (exact p=0.256 and p=0.108). Because each variant retains other CORA mechanisms, these comparisons measure the specific contribution of the replaced component within the coupled framework, rather than testing whether the complete method is effective.

Q1. Concentration into a single active region in the embedding-space baseline.

The concentration into a single active region occurs after initial clustering, during topology maintenance and subsequent region reassignment. The initial embedding-space HDBSCAN clustering is healthy: on ALFWorld, it produces 26 regions at epoch 1, with the maximum region share decreasing from 21.9% to 12.9% and the Gini coefficient from 0.595 to 0.346 by epoch 3. The concentration begins at the first split/merge cycle and reaches 100% membership in one active region by epoch 8. The reason is that, after initialization, the baseline repeatedly reassigns members as the topology evolves; under the embedding-space assignment geometry, these updates increasingly favor the current dominant region, while newly declared regions remain inactive.

Q2. Simpler no-clustering baseline with global utility and failure summaries.

We thank the reviewer for suggesting this comparison. We evaluate the requested baseline on the ALFWorld benchmark with one global pool, category-specific Q-values, global utility estimation, and a global failure summary, without clustering or regional reassignment. All other settings match CORA; evaluation uses temperature 0 on 140 ID and 134 OOD games.

The results are:

SettingUtility / failure summaryFinal IDFinal OOD10-epoch mean ID10-epoch mean OOD
No clustering (requested baseline)category-Q (global pool) / global RFS74.29%74.63%73.64%75.22%
CORAregional utility / regional RFS82.86%86.57%80.07%85.67%

The baseline obtains 74.29% ID and 74.63% OOD at the final epoch, versus 82.86% and 86.57% for CORA; its 10-epoch mean ID and OOD are 73.64% and 75.22%, versus 80.07% and 85.67% for CORA. This supports the value of capability-based grouping. We have added these results to §4.3 , highlighted in blue.

AuthorsOpenReview ↗

# Response to Reviewer BYVR - Part 3

Q3. Robustness to category-label randomization.

We thank the reviewer for suggesting this comparison. We evaluate the 10% category-label-randomization condition on ALFWorld. The results are:

SettingLabelsFinal IDFinal OOD10-epoch mean ID10-epoch mean OOD
CORAClean labels82.86%86.57%80.07%85.67%
CORA with 10% category-label randomization10% randomized labels80.71%83.58%79.32%84.25%

The 10% category-label-randomization condition maintains competitive performance: relative to CORA, the final change is -2.15 percentage points for ID and -2.99 points for OOD, while the 10-epoch mean changes by -0.75 and -1.42 points, respectively. These results indicate that CORA is robust to category-label errors. We have added these results to §4.3 , highlighted in blue.

We again sincerely thank you for engaging carefully with CORA and for the constructive suggestions. We sincerely hope that our responses and revisions address your concerns, and, if you find them satisfactory, we would be very grateful if you would consider raising the evaluation of our paper.

Reviewer_Aj6nOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper looks at experience reuse in LLM agents when individual memories receive too little feedback for reliable utility estimates and proposes CORA. CORA gives each memory a category-conditioned utility profile, groups similar profiles into regions, and uses both memory-level and region-level feedback during retrieval. A retrieved failure memory can also be replaced with a cached summary of failures from the same region. The paper evaluates CORA on ALFWorld, BigCodeBench, LLB-OS, and LLB-DB. Among five settings with full E1–E10 trajectories, CORA has the highest trajectory-mean success rate in all five and the highest final E10 rate in four. GPT-5-mini is reported separately using peak success rate. The largest final E10 gain over the strongest competing baseline is 6.43 percentage points on ALFWorld.

Critical strengths
  1. Proposes CORA, a memory management framework that groups memories based on task-conditioned utility patterns rather than relying solely on semantic similarity. This design enables memories with similar utility profiles to share sparse feedback while preserving category-specific utility information.
  2. Provides comprehensive evaluations across multiple agent domains. CORA outperforms MemRL in terms of mean E1–E10 success rate across all five complete settings, with notable improvements on ALFWorld. The ablation studies on region space and utility estimation further support the proposed design.
Critical weaknesses
  1. Insufficient analysis and discussion of experimental results. For example, the performance improvements are inconsistent across benchmarks, on BigCodeBench, although CORA achieves a higher mean success rate across E1–E10, its final E10 success rate is 0.58 percentage points lower than that of MemRL. These results require further analysis and explanation.
  2. CORA incorporates both neighbor Q-value propagation and training-time exploration, but the current ablation studies do not clearly distinguish their individual contributions. Moreover, the tested embedding-space clustering configuration groups all memories into a single active region. This only indicates the limitations of the specific configuration rather than demonstrating that semantic-space clustering is generally inferior to the proposed approach.
Questions for authors

Refer to the Weaknesses for details.

3 comments
AuthorsOpenReview ↗

# Response to Reviewer Aj6n - Part 1

We sincerely thank the reviewer for the careful reading and constructive comments, and we are especially grateful that the reviewer recognizes CORA's capability-based memory organization for sharing sparse feedback while preserving category-specific utility, and that CORA provides comprehensive evaluations across multiple agent domains. We have carefully considered every concern and question, conducted the corresponding experiments, which took a few days to complete, and revised the manuscript accordingly, with changes highlighted in blue. Below, we address the reviewer's concerns point by point.

W1. Insufficient analysis and discussion of experimental results. In the paper, we provide a descriptive analysis of the performance improvement from the region perspective. Section 4.2 and Figure 2 analyze how capability-space organization evolves and how its task-specific region profiles relate to test performance.

We would like to respectfully point out that our results are consistent across benchmarks. Across five complete benchmark--backbone settings spanning four benchmarks and four frozen backbones, CORA achieves the highest E1--E10 mean SR in every setting and the highest E10 SR in four settings.

The lower BigCodeBench E10 value can be understood from the evaluation trajectory and generation process, and does not contradict this consistent trajectory-level improvement. To further explain this result, we examine the E9-to-E10 transition and the generation process. CORA is ahead of MemRL at E9 (52.05% vs. 50.29%), reflecting CORA's stronger performance over the preceding epochs. From E9 to E10, however, CORA's SR decreases to 49.71%, while MemRL remains at 50.29%. CORA rewrites previously passing programs more often (76.4% vs. 68.5%), and the E10 fix rate among CORA's rewritten cases is 34.6%, compared with approximately 49.5% in its earlier epochs and 50.0% for MemRL. Thus, the final update removes CORA's E9 advantage because more programs are rewritten and fewer of those rewrites recover a passing solution in this round. E10 is CORA's lowest point in only this one of the five settings, while averaging the last two or three epochs gives a positive BigCodeBench difference of +0.58 or +1.85 percentage points. This explains why the E10 ordering differs from the trajectory-mean ordering while preserving the overall conclusion that CORA improves performance over the full trajectory. We have added this analysis to §4.2, highlighted in blue.

AuthorsOpenReview ↗

# Response to Reviewer Aj6n - Part 2

W2. Ablating propagation and exploration, and comparing semantic clustering. We isolate two auxiliary components--neighbor Q-value propagation and training-time exploration--below, and compare CORA with CLAG, an independent semantic-space clustering method.

2×2 ablation. We use p for neighbor Q-value propagation and e for training-time exploration. For the ablation, we report the E10 train, ID, and OOD success rates, all evaluated under the same ALFWorld protocol. The original main run is CORA; the other rows are the newly run ALFWorld cells.

CellpeE10 train / ID / OOD
CORA (main experiment)onon77.60 / 82.86 / 86.57
CORA_pron_expoffonoff79.76 / 80.00 / 84.33
CORA_proff_exponoffon83.15 / 86.43 / 91.79
CORA_proff_expoffoffoff78.33 / 80.00 / 84.33

Within the newly controlled cells, exploration improves CORA_proff_expon over CORA_proff_expoff by +4.82 / +6.43 / +7.46 points on E10 train / ID / OOD. The propagation comparison, CORA_pron_expoff versus CORA_proff_expoff, is +1.43 / 0.00 / 0.00 points.

The higher result for CORA_proff_expon should be understood as an interaction between CORA's core framework and its auxiliary training choices. CORA's contribution is the capability-profile clustering, hierarchical utility estimation, and region-level knowledge aggregation framework; propagation and training-time exploration are auxiliary implementation details. In the corresponding exploration-off comparison, the propagation-on configuration also showed higher training SR, so we retained propagation in the main setting during the experiments.

For ALFWorld's highly repetitive workflows, retaining uncertainty-based exploration while removing embedding-neighbor propagation appears to expose more varied memories during training, whereas propagation can introduce interference when semantically similar memories have different capability-conditioned outcomes. The observed pattern is consistent with CORA's design premise: semantic similarity alone cannot fully capture a memory's usefulness across task categories, so we introduce category-conditioned utility profiles to better capture each memory's utility across task categories. We have added these results and the accompanying analysis to Appendix C.4, “Propagation and Exploration Ablations,” with the new text highlighted in blue.

Semantic-space clustering. We compare CORA with CLAG, an independent semantic-space clustering method, using the paper's metrics (E1--E10 mean test SR / E10 test SR):

Benchmark / backboneCORACLAG
BigCodeBench / DeepSeek-V351.75 / 49.7148.60 / 47.66
ALFWorld / Qwen2.5-72B80.07 / 82.8669.93 / 71.43

CORA exceeds CLAG on both BigCodeBench and ALFWorld, supporting the benefit of capability-profile organization.

We again sincerely thank you for engaging carefully with CORA and for the constructive suggestions. We sincerely hope that our responses and additional analyses address your concerns, and, if you find them satisfactory, we would be very grateful if you would consider raising the evaluation of our paper.

We sincerely hope that our responses and revisions have addressed your concerns. We would be very grateful if you could let us know whether the issues you raised have been adequately resolved. If so, we would be deeply grateful if you would consider raising your evaluation of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.