acceptodds
Under review as a conference paper at ICLR 2027

MCMem: Metacognitive Memory for Self-Aware Reliance in Long-Horizon LLM Agents

Abstract

The memory mechanisms have long been recognized as a reusable knowledge reservoir that supports large language model (LLM) agents to self-evolve in open-ended environments. Nevertheless, the universal utility of a piece of memory across task environments is hardly determinable at the time of abstraction. In this paper, we argue that an ideal memory mechanism should also enable an agent to progressively recognize its own capability boundaries when augmented with accumulated memories, so that it could strike a balance between exploiting existing ones and exploring more in open-ended environments. Based on this insight, we propose a MetaCognitive Agentic Memory mechanism (MCMem) that augments an agent with adaptable and reliable memory management and utilization throughout self-evolution. It is built on a hierarchical memory graph that progressively abstracts episodic trajectories into reusable memories while preserving provenance links to the underlying evidence. On top of that, a metacognitive layer continuously calibrates the agent's reliance on each memory as new experiential evidence accumulates, enabling sustained utilization of memory over long task horizons. Experiments on long-horizon agent benchmarks validate MCMem's superiority in terms of long-term task success and exploration-exploitation balance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agentic memory, long-horizon, metacognition, hierarchical memory

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_BkmfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes MCMEM, a metacognitive memory mechanism for long-horizon LLM agents that calibrates reliance on retrieved memories using provenance-grounded evidence. The framework constructs a hierarchical memory graph that preserves provenance links to supporting evidence. It further introduces a metacognitive layer that disentangles correctness, adoption, and execution signals from episode-level evidence to revise memories and control their delivery. Experiments across multiple benchmarks and model scales demonstrate the effectiveness of MCMEM.

Critical strengths
  1. The paper is well motivated, addressing the important problem of memory reliability calibration. The hierarchical memory graph with explicit provenance edges and audited evidence provides clear traceability.
  2. The evaluation across four long-horizon benchmarks and multiple model sizes demonstrates the generality and effectiveness of the proposed approach.
Critical weaknesses
  1. The method requires additional model calls to verify and maintain memories, but the computational overhead is not evaluated or compared with baselines. The performance gains may partly stem from increased computation rather than the proposed mechanism itself.
  2. The paper is difficult to follow due to excessive formalization and overly complex definitions, which obscure the main ideas and reduce readability.
Questions for authors
  1. Could you provide a comparison of computational costs and evaluate whether MCMEM still outperforms the baselines under matched computation budgets?
  2. Does MCMEM use the same LLM for the assessor, applicability/evidence judges, adoption judge, and editor/merge judge, or are different models used?
  3. The method involves numerous hyperparameters that require task-specific configuration. How sensitive is performance to these settings?
  4. The paper does not separately ablate correctness, adoption, and execution beliefs. Could you provide ablations to clarify the individual contributions of these components?
7 comments

[W1 & Q1] Could you provide a comparison of computational costs and evaluate whether MCMEM still outperforms the baselines under matched computation budgets?

Thanks for the constructive comment. We agree the necessity of comparing total computation cost and conduct matched-total-budget experiments. We have additionally conducted the following experiments to address the reviewer's concern.

On SW 9B backbone setting, we have added an end-to-end cost table covering input and output tokens, all LLM calls and wall-clock time, with separate accounting for actor decision inference, within-step memory processing, and post-episode extraction and maintenance.

We also added two comparable computation budget setting for no-memory baseline and MCMem: 1) Enhanced no memory: we added additional self-review computation for the no-memory baseline to correct its potential decision errors; 2) MCMem, budget-comparable: we reduced the number of memory entries processed during agent retrieval, memory judging, and periodic review, and adopted batch processing for step-level applicability assessment and source-evidence verification instead of prompting the LLM separately for each entry.

MethodSuccess %Base actorWithin-step memoryPost-episode memoryTotal tokensLLM callsRuntime hh
No memory30.77.33007.331,9751.56
Enhanced No memory34.46.870014.23,7802.51
AWM41.110.000.056010.12,4302.36
A-Mem31.17.626.642.0216.38,58714.1
Mem-α50.012.513.90.43126.86,2737.84
Mem-T41.17.166.980.92415.110,2454.67
MCMA48.98.960.4844.7014.13,6642.96
Memora20.78.534.271.2214.07,2745.04
Dynamic Cheatsheet20.010.900.88411.82,2504.80
MemRL23.313.100.083013.22,3852.81
MCMem, default56.38.5229.39.1547.017,51813.8
MCMem, budget-comparable54.46.2110.83.1820.28,5296.33
DatasetEnhanced no-memory successEnhanced no-memory tokensMCMem successMCMem tokens
ScienceWorld34.4%14.2M54.4%20.2M
ALFWorld67.9%10.1M77.6%14.7M
BabyAI38.4%15.7M52.7%19.5M
PDDL36.7%12.5M36.7%15.7M

At comparable measured token consumption, MCMem still achieves superior performance, and beat enhanced no-memory baseline across all four datasets. These results support the value of allocating computation to memory management and utilization and demonstrate the superiority of our method design is not purely from higher computational cost.

[Q3] sensitivity analysis

Thanks for the comment. We apologize for listing hyperparameters without discussing their sensitivity.

First, we clarify that MCMem uses a single fixed hyperparameter configuration across all benchmarks and actor scales without task-specific tuning.

Second, despite many hyperparameters, they serve different purposes: some govern core memory decisions, such as evidence requirements, retrieval, and maintenance, while others serve as implementation safeguards or computational limits.

Third, we conduct further sensitivity experiments on ScienceWorld SWSW and ALFWorld AWAW with the 9B backbone, varying representative parameters governing reliability estimation, memory retrieval, maintenance, and context budgets, one at a time, while keeping other settings fixed within each experiment.

Parameter (tested values)Δ SW ppppΔ AW pppp
Reliability threshold ℓmin⁡\ell_{\min} 0.4,0.5,0.60.4, 0.5, 0.6−4.40 to +0.75−3.48 to +1.24
Retrieval width knormk_{\mathrm{norm}} 1,2,31, 2, 3−3.0 to −1.9-2.28 to -0.76
Episode retrieval width kepk_{\mathrm{ep}} 1,2,31, 2, 3−3.2 to −1.0-4.10 to -1.68
Periodic review capacity kswk_{\mathrm{sw}} 5,10,205, 10, 20−5.55 to +4.45+1.24 to +2.48
Memory context budget 512,1024,2048512, 1024, 2048+1.11 to +1.85−2.74 to +0.24
Execution threshold ρgap\rho_{\mathrm{gap}} 0.3,0.5,0.70.3, 0.5, 0.7−2.59 to −1.850.00 to +1.99

The results show that 11 core decision settings matter: lowering the reliability threshold reduces success on both benchmarks; narrower retrieval budget also yield lower success in the tested runs; 22 maintenance sensitivity is environment-dependent: decreasing review capacity does not consistently degrade performance; and 33 auxiliary settings are relatively stable within the tested ranges: halving the context budget preserves performance, and changes to the execution threshold have comparatively smaller effects.

We will incorporate these results and reorganize Appendix B.1 by functional role, distinguishing core decision parameters from auxiliary implementation controls.

[W2] The paper is difficult to follow due to excessive formalization and overly complex definitions, which obscure the main ideas and reduce readability.

Thanks for the valuable comment. We will revise the methodology section accordingly to enhance the delivery of key ideas and insights of our method.

[Q2] Does MCMEM use the same LLM for the assessor, applicability/evidence judges, adoption judge, and editor/merge judge, or are different models used?

Thanks for the question. MCMem uses the same frozen LLM as the actor for all memory management and utilization procedures, including the assessor, applicability/evidence judges, adoption judge, and editor/merge judge. This design aligns with our goal of enabling an LLM agent to progressively recognize its own memory-augmented capability boundaries and calibrate its reliance on accumulated experience, without relying on a separate, more powerful model for memory evaluation or management.

[Q4] The paper does not separately ablate correctness, adoption, and execution beliefs. Could you provide ablations to clarify the individual contributions of these components?

Thanks for the insightful comment. We conduct additional three-repeat ablation studies to more clearly isolate the effects of three beliefs. As shown below, all these modules contribute to the performance superiority and stability across datasets. We will add these results into the revised manuscript.

VariantsSW-9BAW-9B
MCMem56.3 ± 3.3981.6 ± 2.40
w/o correctness belief only53.3 ± 4.0176.9 ± 1.97
w/o adoption belief only53.0 ± 3.2177.9 ± 2.28
w/o execution belief only51.5 ± 4.4579.6 ± 0.860

Thank you for the additional experiments. However, some concerns remain. The sensitivity analysis shows that performance is considerably affected by hyperparameter choices, making the tuning protocol particularly important. How were the hyperparameters selected and were comparable tuning efforts applied to all baselines? Finally, the readability concern remains unaddressed, as no concrete revisions are provided. I maintain my original Weak Reject rating.

Reviewer_613VOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MCMem, a metacognitive memory framework that aims to improve the reliability and utilization of accumulated experience in long-horizon LLM agents. The framework introduces a hierarchical memory graph that links abstracted knowledge to its supporting episodes and a metacognitive mechanism that continuously evaluates memory reliability, distinguishes memory correctness from adoption and execution failures, and adjusts how memories are retrieved, trusted, revised, or withdrawn. The method is evaluated on four agent benchmarks (ALFWorld, ScienceWorld, BabyAI, and PDDL) using Qwen3.5 models of three different sizes, with comparisons against nine baselines. The results demonstrate competitive or superior task success rates across most settings. Additional ablation studies, analyses of exploration–exploitation behavior, experiments with deliberately incorrect memories, and robustness evaluations further investigate the effectiveness of the proposed memory management mechanism.

Critical strengths
  • The paper introduces a meaningful closed-loop mechanism that continuously evaluates memory reliability using accumulated evidence and adjusts how memories are retrieved, trusted, revised, or withdrawn. This goes beyond conventional memory storage and retrieval by explicitly connecting memory reliability to agent behavior.

  • The framework distinguishes memory correctness, adoption, and execution failures rather than relying solely on task outcomes. Experiments across four benchmarks and three model scales, together with ablations, incorrect-memory injection, and robustness analyses, provide substantial evidence for the effectiveness of the proposed approach.

Critical weaknesses
  • MCMem introduces multiple additional LLM calls for memory retrieval, evaluation, and maintenance, but the paper does not provide a comprehensive computational cost comparison with the baselines. This makes it difficult to determine how much of the performance improvement comes from the proposed memory mechanisms rather than increased inference computation.

  • Several ablation settings remove multiple interconnected mechanisms simultaneously, making it difficult to isolate their individual contributions. Moreover, procedural memory and the observation index are retained across all ablations, leaving their independent contributions unclear. This weakens the evidence for which components are primarily responsible for the reported improvements.

Questions for authors
  • The actual computational costs of MCMem and the baselines, including LLM calls, token consumption, and inference time, need to be reported. Given the additional evaluation and maintenance operations, how much of the performance improvement can be attributed to the proposed mechanism rather than increased computation?

  • Could the authors further isolate the contributions of the core metacognitive mechanisms? Several ablations remove multiple components simultaneously, while procedural memory and the observation index remain enabled throughout. A more controlled comparison would help establish which components primarily drive the improvements.

  • Authors should explain the difference between the ScienceWorld 9B results in Table 1 (56.3%) and the easy-to-hard setting in Table 9 (47.4%). Since both appear to use the same benchmark, backbone, and task ordering, clarifying the experimental differences would help assess the reliability and reproducibility of the results.

4 comments

[W1 & Q1] The actual computational costs of MCMem and the baselines, including LLM calls, token consumption, and inference time, need to be reported. Given the additional evaluation and maintenance operations, how much of the performance improvement can be attributed to the proposed mechanism rather than increased computation?

Thanks for the constructive comment. We agree the necessity of comparing total computation cost and conduct matched-total-budget experiments. We have additionally conduct the following experiments to address the reviewer's concern.

On SW 9B backbone setting, we have added an end-to-end cost table covering input and output tokens, all LLM calls and wall-clock time, with separate accounting for actor decision inference, within-step memory processing, and post-episode extraction and maintenance.

We also added two comparable computation budget setting for no-memory baseline and MCMem: 1) Enhanced no memory: we added additional self-review computation for the no-memory baseline to correct its potential decision errors; 2) MCMem, budget-comparable: we reduced the number of memory entries processed during agent retrieval, memory judging, and periodic review, and adopted batch processing for step-level applicability assessment and source-evidence verification instead of prompting the LLM separately for each entry.

MethodSuccess %Base actorWithin-step memoryPost-episode memoryTotal tokensLLM callsRuntime hh
No memory30.77.33007.331,9751.56
Enhanced No memory34.46.870014.23,7802.51
AWM41.110.000.056010.12,4302.36
A-Mem31.17.626.642.0216.38,58714.1
Mem-α50.012.513.90.43126.86,2737.84
Mem-T41.17.166.980.92415.110,2454.67
MCMA48.98.960.4844.7014.13,6642.96
Memora20.78.534.271.2214.07,2745.04
Dynamic Cheatsheet20.010.900.88411.82,2504.80
MemRL23.313.100.083013.22,3852.81
MCMem, default56.38.5229.39.1547.017,51813.8
MCMem, budget-comparable54.46.2110.83.1820.28,5296.33
DatasetEnhanced no-memory successEnhanced no-memory tokensMCMem successMCMem tokens
ScienceWorld34.4%14.2M54.4%20.2M
ALFWorld67.9%10.1M77.6%14.7M
BabyAI38.4%15.7M52.7%19.5M
PDDL36.7%12.5M36.7%15.7M

At comparable measured token consumption, MCMem still achieves superior performance, and beat enhanced no-memory baseline across all four datasets. These results support the value of allocating computation to memory management and utilization and demonstrate the superiority of our method design is not purely from more computation cost.

[W2 & Q2] Could the authors further isolate the contributions of the core metacognitive mechanisms? Several ablations remove multiple components simultaneously, while procedural memory and the observation index remain enabled throughout. A more controlled comparison would help establish which components primarily drive the improvements.

Thanks for the insightful comment. We conduct additional three-repeat ablation studies to more clearly isolate the effects of three beliefs, the label wording and the procedural memory. As shown below, all these modules contribute to the performance superiority and stability across datasets. We will add these results into the revised manuscript.

VariantsSW-9BAW-9B
MCMem56.3 ± 3.3981.6 ± 2.40
w/o correctness belief only53.3 ± 4.0176.9 ± 1.97
w/o adoption belief only53.0 ± 3.2177.9 ± 2.28
w/o execution belief only51.5 ± 4.4579.6 ± 0.860
w/o ordering while w/ label wording only56.3 ± 2.5773.9 ± 0.00
w/o procedural memory and the observation index55.9 ± 2.8075.9 ± 3.11

[Q3] Authors should explain the difference between the ScienceWorld 9B results in Table 1 (56.3%) and the easy-to-hard setting in Table 9 (47.4%). Since both appear to use the same benchmark, backbone, and task ordering, clarifying the experimental differences would help assess the reliability and reproducibility of the results.

Thanks for the valuable comment. We apologize for the confusion caused by our mistake in aligning the SW environments in Table 1 and Table 9. In fact, the Table 9 was conducted under the vanilla SW environment, in which we discovered an inappropriate design: The original action descriptions do not clearly distinguish focus on (submitting a choice) from look at (inspection) and failure traces show repeated inspection without submission for all tested methods. This could not only cause lower performance of all methods (all of MCMem, mem-α\alpha and AWM are lower then those in Table 1), but also causing a severe uncertainty in fair comparison —— those trials which discover the submission convention in an early episode and retain it for subsequent tasks will succeed in much more easier tasks in the early stage beyond reflecting the true capability of different methods. That is why all three methods in E→H order perform worse than in Table 1. Therefore, we make this env change to ensure that the comparison depends less on incidental discovery of submission semantics. It supplies no answers or solution steps and changes neither simulator dynamics nor scoring. However, we truly apologize for not making this change consistent in Table 9. To remedy this inconsistency, we have rerun the full results and the key conclusions still hold in the explain-focus SW environment. We will revise the paper accordingly.

MethodOrderLevel 1 3434Level 2 2727Level 3 2323Level 4 66All 9090
AWME→H65.8 ± 7.432.1 ± 7.726.1 ± 7.50.0 ± 0.041.1 ± 2.3
AWMH→E71.6 ± 13.937.0 ± 12.829.0 ± 9.10.0 ± 0.045.6 ± 6.2
Mem-αE→H82.4 ± 10.643.2 ± 11.324.6 ± 10.00.0 ± 0.050.0 ± 7.6
Mem-αH→E73.5 ± 15.639.5 ± 9.314.5 ± 5.00.0 ± 0.043.3 ± 8.7
MCMemE→H87.3 ± 1.755.6 ± 7.424.6 ± 2.55.6 ± 9.656.3 ± 2.6
MCMemH→E88.2 ± 7.850.6 ± 7.720.3 ± 2.50.0 ± 0.053.7 ± 1.7

Thank you for the detailed responses and additional experiments. The new ablations and clarification of the ScienceWorld settings adequately address my concerns. The computational cost analysis is also helpful, although the budgets are not strictly matched and some uncertainty remains about the impact of additional computation. Overall, the rebuttal has addressed most of my concerns, and I will maintain my original rating.

Thank you again for your time and effort. We will ensure that the raised problems are well addressed in the revised manuscript.

Reviewer_UwoyOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MCMem maintains episodes, abstract memories, provenance, and evidence ledgers, distinguishing claim correctness, actor adoption, and execution failure. Rules including a Wilson lower bound assign well-tested, tentative, disputed, and weak-match labels. Multiple LLM roles extract, assess, revise, and retrieve memories. Experiments compare 4B/9B/27B actors on AgentBoard's ALFWorld, ScienceWorld, BabyAI, and PDDL; appendices examine planted wrong memories, task ordering, and single-to-two-object transfer.

Critical strengths
  1. Separating incorrect memories, non-adoption, and execution failure is meaningful engineering. Provenance retention and the persistence of refutations despite task successes help avoid conflating relevance with reliability.
  2. The paper actively tests wrong memories rather than reporting success alone. Appendix B.4/Tables 7–8 examine delivery, incorrect actions, and correct twins; B.5 tests reverse order and task shift. These go beyond no-memory comparisons. Reporting denominators for planted memories that receive no judgment is also useful.
Critical weaknesses

W1: Ablations and budget controls do not isolate reliability calibration; some beliefs are better viewed as heuristics

  • Table 5, p. 39, shows that “without correctness belief” also removes withdrawal, all labels except weak match, grounded-procedure recall, and the refutation cap. “Without reliance labels” also removes some recall. Success decreases therefore do not cleanly isolate a single mechanism.
  • The plain-prefix variant retains beliefs but also changes label-based ordering, pp. 8/39. It tests wording and ordering together, not label wording independently.
  • The system invokes a query formulator each step, applicability/evidence judges per candidate, and multiple extraction/assessment/editing roles after episodes. Equal 30-step actor budgets do not imply equal total tokens, LLM calls, or wall-clock budgets. The inspected experimental appendix provides no end-to-end cost table. Matched-total-budget repeated no-memory reasoning or simplified controllers are needed; training-free does not mean low cost.
  • A Wilson bound on selectively sampled support/refutation outcomes is not automatically a calibrated interval for the probability that a memory is correct. Correlated episodes, judge errors, and inherited verdicts after revision weaken an independent-Bernoulli interpretation. This can be a heuristic, but empirical calibration or independent auditing is needed.
  • Page 16 attributes environment-refused adopted actions to actor execution failure and removes associated refutations. Incorrect memories can themselves recommend invalid actions. This gate could protect wrong memories; such cases should be measured rather than excluded by assumption.

W2: Claim coverage and independent validation are weaker than the wording; cross-experiment scores and filtered denominators require clarification

  • Page 7 states “globally best across all datasets,” but Table 1 reports ALFWorld27B MCMem87.6 below AWM89.8 and PDDL4B33.3 below no-memory38.3. The table contradicts a claim of winning every setting, although it does not negate average gains.
  • In Table 8, p. 40, among the 22 memories retained by additional judges, only 12 of 32 wrong-version plantings are decided (37.5%), and 14 of 34 correct-version plantings are decided (41.2%). The reported 83% is 10/12, not detection across all wrong plantings; the latter is 10/32≈31.3%. Undecided memories are not automatically errors, but coverage must accompany conditional accuracy, which should not be interpreted as general memory detoxification.
  • Language-model judges filter 30 correct twins down to 22 without human/environment-rule ground truth or judge-agreement evidence. A separate judge is not automatically reliable external truth.
  • Table 1 reports SW9B56.3, while Table 9 reports 47.4 for the same easy-to-hard 90-task setting. B.1 says experiments use their own runs, so this is not a direct contradiction. Nevertheless, the 8.9-point difference needs run/seed and interval information; shared constants alone do not establish stability. Temperature 0 does not eliminate all memory-path nondeterminism.
  • The three-condition/net-flips criterion on pp. 36–37 is a custom stability rule, not a standard significance test with stated sampling assumptions. With only 17 tasks, paired item-level outcomes, run details, and uncertainty would be useful.
Questions for authors

see weakness

5 comments

Thanks for the reviewer's detailed and constructive comments. Please find our point-to-point rebuttal as follows:

[W1-1] Table 5, p. 39, shows that “without correctness belief” also removes withdrawal, all labels except weak match, grounded-procedure recall, and the refutation cap. “Without reliance labels” also removes some recall. Success decreases therefore do not cleanly isolate a single mechanism.

[W1-2] The plain-prefix variant retains beliefs but also changes label-based ordering, pp. 8/39. It tests wording and ordering together, not label wording independently.

Thanks for the insightful comment. We conduct additional three-repeat ablation studies to more clearly isolate the effects of three beliefs as well as the label wording. As shown below, all these modules contribute to the performance superiority and stability across datasets. We will add these results into the revised manuscript.

VariantsSW-9BAW-9B
MCMem56.3 ± 3.3981.6 ± 2.40
w/o correctness belief only53.3 ± 4.0176.9 ± 1.97
w/o adoption belief only53.0 ± 3.2177.9 ± 2.28
w/o execution belief only51.5 ± 4.4579.6 ± 0.860
w/ label wording only56.3 ± 2.5773.9 ± 0.00

[W1-3] The system invokes a query formulator each step, applicability/evidence judges per candidate, and multiple extraction/assessment/editing roles after episodes. Equal 30-step actor budgets do not imply equal total tokens, LLM calls, or wall-clock budgets. The inspected experimental appendix provides no end-to-end cost table. Matched-total-budget repeated no-memory reasoning or simplified controllers are needed; training-free does not mean low cost.

Thanks for the constructive comment. We agree the necessity of comparing total computation cost and conduct matched-total-budget experiments. We have additionally conducted the following experiments to address the reviewer's concern.

On SW 9B backbone setting, we have added an end-to-end cost table covering input and output tokens (M), all LLM calls and wall-clock time, with separate accounting for actor decision inference, within-step memory processing, and post-episode extraction and maintenance.

We also added two comparable computation budget settings for no-memory baseline and MCMem: 1) Enhanced no memory: we added additional self-review computation for the no-memory baseline to correct its potential decision errors; 2) MCMem, budget-comparable: we reduced the number of memory entries processed during agent retrieval, memory judging, and periodic review, and adopted batch processing for step-level applicability assessment and source-evidence verification instead of prompting the LLM separately for each entry.

MethodSuccess %Base actorWithin-step memoryPost-episode memoryTotal tokensLLM callsRuntime hh
No memory30.77.33007.331,9751.56
Enhanced No memory34.46.870014.23,7802.51
AWM41.110.000.056010.12,4302.36
A-Mem31.17.626.642.0216.38,58714.1
Mem-α50.012.513.90.43126.86,2737.84
Mem-T41.17.166.980.92415.110,2454.67
MCMA48.98.960.4844.7014.13,6642.96
Memora20.78.534.271.2214.07,2745.04
Dynamic Cheatsheet20.010.900.88411.82,2504.80
MemRL23.313.100.083013.22,3852.81
MCMem, default56.38.5229.39.1547.017,51813.8
MCMem, budget-comparable54.46.2110.83.1820.28,5296.33
DatasetEnhanced no-memory N=2Enhanced no-memory N=4Enhanced no-memory N=6MCMem
ScienceWorld34.4% / 14.2M38.1% / 20.6M36.0% / 31.5M54.4% / 20.2M
ALFWorld67.9% / 10.1M74.6% / 15.0M71.6% / 20.5M77.6% / 14.7M
BabyAI38.4% / 15.7M41.1% / 24.8M39.3% / 34.2M52.7% / 19.5M
PDDL36.7% / 12.5M35.0% / 19.8M36.7% / 26.6M36.7% / 15.7M

Here, each unit is filled with Success Rate (%) / Token Consumption (M). N is the hyperparameter that controls the action review times. At comparable measured token consumption, MCMem still achieves superior performance, and beats the enhanced no-memory baseline across all four datasets. The performance variation in the enhanced no-memory baseline also shows that increasing the computation budget is not always better. These results support the value of allocating computation to memory management and utilization and demonstrate that the superiority of our method design is not purely from higher computational cost.

[W1-4] A Wilson bound on selectively sampled support/refutation outcomes is not automatically a calibrated interval for the probability that a memory is correct. Correlated episodes, judge errors, and inherited verdicts after revision weaken an independent-Bernoulli interpretation. This can be a heuristic, but empirical calibration or independent auditing is needed.

Thanks for the constructive comment. We clarify that the Wilson lower bound is used as a conservative measure of accumulated empirical support, rather than a calibrated probability of memory correctness. Although the independent-Bernoulli assumptions could be weakened, the Wilson lower bound is still useful in penalizing sparse evidence and regulate memory reliance.

To examine whether these potential evidence biases materially affect memory reliance decisions, we conducted an independent retrospective audit of the evidence supporting all 115 memories originally classified as well-tested, using powerful close-sourced LLM as a judge. i) Selective sampling: we examined 12,840 previously unassessed memory–episode pairs to detect refutations missed by selective retrieval, identifying only 10 omitted refutations affecting two ScienceWorld memories; ii) Judge errors: we also independently examined 2,982 directly assessed memories against agent actions and environment feedback, identifying only only 249 confirmed discrepancies; iii) Inherited verdicts: we examined inherited verdicts against revised memory wordings, identifying only 64 confirmed discrepancies among 311 inherited records.

We then incorporated the confirmed evidence corrections and recomputed the "well-tested" qualifications of the original 115 memory wordings, finding 102 of the original 115 wordings retained their well-tested labels. To further examine the effect of "correlated episodes", we deduplicated supporting evidence sharing the same normalized goal and action sequence, reducing the count by only one, from 102 to 101. These analyses validated that the Wilson score is empirically useful even though it is not a rigorous calibrated probability. We will also avoid this type of claims in the revised manuscript.

[W1-5] Page 16 attributes environment-refused adopted actions to actor execution failure and removes associated refutations. Incorrect memories can themselves recommend invalid actions. This gate could protect wrong memories; such cases should be measured rather than excluded by assumption.

Thanks for the constructive comment. We agree that "Incorrect memories can themselves recommend invalid actions." However, the motivation behind the execution-gap mechanism is not to attribute environment rejection categorically to actor failure, but to avoid treating potentially inconclusive evidence as a definitive refutation. When an actor adopts a memory but all associated actions are rejected by the environment, the resulting failure may reflect either an incorrect memory or unsuccessful execution. Without a powerful enough external judge, rather than assuming either explanation is correct, MCMem conservatively reclassifies the refutation from that episode as `not tested`, without converting it into support or removing refutations accumulated from previous episodes.

We further measured whether this gate could protect wrong memories; we examined two complete 9B runs on ScienceWorld and ALFWorld. The gate activated 8 times and we inspected each cancelled judgment against the memory’s applicability conditions, cited actions, and environment feedback. None of the eight original refutations was sufficiently supported by its evidence: the judgments relied on unmet conditions, departures from the prescribed procedure, or unsupported inferences from overall task failure. This partially supports that the memories caught by the gate are often controversial themselves. However, we agree a stronger judge that more reliably distinguishes incorrect memories from execution failures would further improve this mechanism.

[W2-1] Page 7 states “globally best across all datasets,” but Table 1 reports ALFWorld27B MCMem87.6 below AWM89.8 and PDDL4B33.3 below no-memory38.3. The table contradicts a claim of winning every setting, although it does not negate average gains.

Thanks for the valuable comment. We apologize for the ambiguity. By “globally best across all datasets,” we intended to convey that MCMem achieves the strongest overall performance among all baselines, ranking first on 10 datasets and second on 2. We will revise the wording to clarify this point.

[W2-2]

Thanks for the valuable comment. We will make the qualification in Lines 1918–1920 explicit in Table 8 and report coverage alongside conditional accuracy. Moreover, we make further clarification: **1) Motivation. **While Table 7 demonstrates that MCMem limits the behavioral influence of incorrect memories, we aim to further investigate whether the evidence ledger of MCMem plays an indispensable role; 2) Assessment coverage. To control computational cost as memory grows, MCMem selects a small number of memories for assessment against each completed episode. Selection depends on relevance ranking, so a memory may remain unassessed even when an episode contains feedback that could test it. That's why only 12 out of 32 wrong-version planting memories received non-zero evidence. 3) Supplementary assessment. To examine whether the original assessor could recognize additional incorrect memories when supplied with relevant experience, we searched the saved trajectories for each of the 20 untested planted wrong memories. We identified eight instances with environmental feedback contradicting the planted claim. In an offline assessment, we paired each memory with the corresponding saved episode and applied the original evidence assessor using the same prompt and decision rules. It refuted six of the eight incorrect versions 7575% and supported five of the seven paired original versions verified as procedurally valid. These results demonstrate that the evidence-based judgment mechanism can identify additional incorrect memories when they are paired with informative episodes.

[W2-3]

Thanks for the valuable comment. We clarify that the filtering employed a powerful proprietary LLM, following established practices in LLM-based evaluation [1,2]. Importantly, the judge assessed each candidate against the recorded actions and environment responses, without access to MCMem's verdicts or task-level outcomes.

To further substantiate the screening reliability, we first reran two rounds of LLM judge with no context interference, and found only 1 pair has differing judgements. Furthermore, we manually audited all 30 candidate pairs and consistent with the original judgement results except for the above controversial pair. Representative examples are shown below. Claims are condensed for readability.

Task IDMemory ClaimAgent ActionEnvironment FeedbackScreening Decision
ScienceWorld 25Navigation: When the target is in another location, navigate there before interacting with it.Attempt pick up dove egg before reaching its location; then execute go to outside and repeat the pickup.Before navigation: “No known action matches that input.” After navigation: “You move the dove to the inventory.”Retain: The comparison supports the prescribed navigation step under the stated condition that the target is elsewhere.
ScienceWorld 9Temperature measurement: To measure a heated substance’s temperature, use the thermometer on its container.Compare use thermometer on metal pot with use thermometer on gallium in separately reset replays with identical preparation and matched measurement steps.The container reads 33°C, whereas the gallium inside reads 12°C.Exclude: Measuring the container does not reliably measure its contents, so the original claim cannot serve as an unambiguous correct counterpart.
ScienceWorld 9Melting-point measurement: Transfer the solid into a metal pot, heat it on a stove, and then measure the substance’s temperature to determine its melting point.Pick up gallium, transfer it to the pot, place the pot on the stove, activate the stove, and immediately measure the gallium’s temperature.The thermometer reads 12°C, while gallium remains solid and its recorded melting point is 30°C; the immediate reading therefore does not identify the melting point.Originally retained; unresolved upon re-examination: The heating and measurement steps work, but the claim omits measurement timing and phase identification without explicitly equating the first reading with the melting point.

Moreover, this experiment is intended as a targeted mechanistic analysis rather than a comprehensive benchmark of wrong-memory detection. Its primary purpose is to investigate whether MCMem's ability to distinguish incorrect from correct memories helps explain its robustness against planted memories. The trajectory-grounded evaluation on the screened memory pairs provides complementary evidence for this mechanism, without making broader claims about general-purpose memory correctness assessment. We will revise the manuscript accordingly to avoid confusion.

[1] Peng et al., Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios? EMNLP 2026.

[2] Barke et al., AgentRx: Diagnosing AI Agent Failures from Execution Trajectories. EMNLP 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.