acceptodds
Under review as a conference paper at ICLR 2027

MLRMem: Test-Time Multi-Level Memory with Re-Verification for Long-Horizon Agents

Abstract

As large language model agents tackle long-horizon tasks, memory is essential for preserving experience and supporting effective decisions. Recent work demonstrates the promise of adaptive memory at test-time, using step-level strategies to guide agent decisions. However, each step depends on the current local state, with limited access to global task information, leading to repetitive action loops and task failure caused by inapplicable step-level strategy. To address this issue, we propose **MLRMem**, a test-time memory framework that combines multi-level memory integration and selection with a strategy re-verification mechanism based on subsequent state feedback. The former combines information from multiple memory levels to select strategy for the current step, while the latter re-verifies the selected strategy using subsequent execution outcomes and triggers re-verification when the selected strategy is no longer applicable. This enables the agent to select appropriate strategy for the current state at each step, without introducing extra inference time cost. Experiments on ALFWorld show that MLRMem consistently outperforms the evaluated agent memory frameworks, improving both task performance and interaction efficiency. Ablation studies further confirm the effectiveness of both multi-level memory and strategy re-verification, with the latter playing a more critical role in improving performance on unseen tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model Agents, Agent Memory, Adaptive Memory, Long-Horizon Tasks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rCVoOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MLRMem, a memory system for LLM agents. It combines task-level and step-level memories and checks whether the selected strategy remains useful after each action. The memory stays fixed during evaluation. On ALFWorld, success increases from 68.5% with AdaMEM-low to 73.5%. On WebShop, the average task score increases from 19.6 to 36.6. The paper also tests different backbones and removes individual components to study their effects.

Critical strengths

1.Better results. MLRMem improves over both tested memory baselines on ALFWorld and WebShop. On ALFWorld, the methods use the same source trajectories, which makes the comparison more useful.

2.Useful feedback checks. Removing re-verification lowers ALFWorld success from 73.5% to 52.9%. This supports checking whether a strategy still applies instead of keeping it until completion or expiry.

Critical weaknesses
  1. Unfair runtime comparison. Page 7 states that concurrency and timing settings differ across methods. Therefore, the reported runtime reduction does not establish that MLRMem itself is faster. The abstract also claims no extra inference time cost, while the conclusion acknowledges additional model calls. Matched runtime settings and model-call and token counts are needed to support this central claim. The innovation seems too simple.

  2. Limited evidence for task-level memory. In Figure 6(b), removing task-level memory changes overall success only from 73.5% to 73.4%. Task-level memory helps unseen tasks but hurts seen tasks. This supports a narrower benefit than the main framing suggests. The paper does not clearly show when the extra memory level is worth its added complexity and cost.

  3. The selected primary area is “foundation or frontier models, including LLMs,” but the main contribution is an agent memory system. The paper does not propose a new model or training method. An agents-focused area seems more suitable.

Questions for authors
  1. Under the same hardware, concurrency, and timing settings, does MLRMem still run faster than AdaMEM-low? Please include all checking calls and report token use and model-call counts.

  2. Is the unseen-task gain from task-level memory consistent across runs? How does its inference cost compare with the step-level-only version?

  3. How exactly are the selection score, penalties, and repeated lack of progress computed? These details affect which strategies are selected and withdrawn.

4 comments

Thank you for your careful review and thoughtful suggestions, and for recognizing the value of execution-feedback checks and our comparisons using shared source trajectories. Your comments helped us focus the revision on the evidence needed to understand MLRMem’s contribution: when task-level memory is useful, what inference costs it incurs, and how strategy selection and withdrawal operate. We have revisited these issues through additional analyses and implementation refinements, and respond to each point below.

W1 & Q1: Runtime comparison, inference costs, and methodological contribution.

Thank you for pointing out this inconsistency. We revisited the timing records to answer your question directly: the available runs do not establish that MLRMem is faster under the same hardware, concurrency, and timing settings. On 70 tasks matched by game identity, MLRMem-Efficient records 952.8 versus 744.1 seconds of cumulative inference time per task for AdaMEM-low, with different serving loads and timing boundaries. We will therefore remove the abstract’s “no extra inference time cost” statement and replace the runtime-reduction claim with a comparison of success rates and inference costs, explicitly accounting for the additional model calls.

We report model-call counts and input/output token use across all 274 ALFWorld tasks with Qwen3.5-4B, including failed tasks (Appendices E.2–E.3, Tables 7–8). The MLRMem totals include action generation, all candidate-checking and re-verification calls, fact extraction, and retries and fallbacks; embedding requests are excluded from these LLM totals.

MethodCalls/taskInput tokens (K/task)Output tokens (K/task)Total tokens (K/task)Success rate (%)
AdaMEM-low61.7105.358.1163.468.5
MLRMem127.9211.944.7256.773.5
MLRMem-Efficient75.1106.038.4144.573.0

To reduce the checking overhead, MLRMem-Efficient shares context across candidate checks and combines fact extraction with feedback verification while retaining the original validators (Appendices E.1 and E.4). Relative to original MLRMem, this reduces calls by 41.3% and total tokens by 43.7%, with success changing from 73.5% to 73.0%. Compared with AdaMEM-low, it uses 11.6% fewer total tokens and achieves 4.5 percentage points higher success, while requiring 21.7% more calls. These results support a better success–token trade-off; they do not establish lower runtime.

Regarding the methodological contribution, MLRMem addresses how retrieved experience should continue to guide execution as the task state changes. It combines task-level guidance with step-level strategies, then uses subsequent feedback to reassess their applicability and withdraw guidance that no longer applies. In Figure 6(a), removing multi-level memory while retaining re-verification reduces success from 73.5% to 65.9%; disabling re-verification reduces it to 52.9%. These controls support the contribution of integrating memory selection with feedback-driven revision during execution.

W2 & Q2: Contribution and cost of task-level memory.

Thank you for highlighting the need to clarify when task-level memory is beneficial. Our diagnosis identified overly restrictive matching of source-task objects and destinations, together with incomplete delivery of task advice to the action prompt. We therefore introduced role-conditioned advice and an independent advice section while keeping the memory library unchanged (Appendix D.4). With the same executor, step-level memory, and re-verification retained, the two-seed pooled comparison improves seen and unseen success by 2.86 and 2.99 percentage points, respectively, raising overall success from 72.63% to 75.55% (Figure 10). Task advice is incorporated into existing model prompts without a separate advice-generation call; its direct overhead consists of retrieval and additional context, controlled through advice expiry and the context budget. These results indicate that task-level memory provides useful procedural guidance when retrieved experience is adapted to the current goal and made available during execution.

W3: Research positioning and area selection.

Thank you for the suggestion. We agree that MLRMem is primarily an agent-centric framework rather than a new foundation model or model-training method. More specifically, our work investigates test-time memory and strategy management for LLM-based agents, enabling them to reuse task- and step-level experiences and re-verify strategy applicability during execution without updating model parameters. We selected “Foundation or frontier models, including LLMs” to reflect the paper’s focus on LLM-based agents and their inference-time memory mechanisms. We will further clarify this positioning in the paper. If the area assignment can still be revised, we would be happy to adjust our selection to a more appropriate area in accordance with the chairs’ guidance.

Q3: Selection score, penalties, and progress criteria.

Thank you for requesting these details, now specified in Appendices B.2–B.3. Candidates that pass the applicability and citation checks are ranked by pt,m=slocal+sgoal+sevid+sprog−sconf−sloop−sdup,p_{t,m} = s_{\mathrm{local}} + s_{\mathrm{goal}} + s_{\mathrm{evid}} + s_{\mathrm{prog}} - s_{\mathrm{conf}} -s_{\mathrm{loop}} - s_{\mathrm{dup}}, with unit weights and each component in [0,1][0,1]. The positive terms represent local relevance, goal alignment, confirmed-condition coverage, and progress potential; Appendix B.2 gives their exact mappings and fallback values. The conflict penalty is binary; the default looping penalty is 1 when the last three actions are identical and the candidate recommends repeating that action; redundancy is (nm−1)/max⁡(N−1,1)(n_m-1)/\max(N-1,1), where NN is the retrieved candidate count and nmn_m counts candidates sharing the same source lineage. For withdrawal, three accumulated checker reports of no progress terminate active guidance, and supported or uncertain feedback does not reset this counter; separately, when no guidance is active and the task remains incomplete, unchanged goal-state and exploration records across three observations can trigger reselection (Appendix B.3). These rules distinguish candidate ranking from feedback-driven withdrawal, allowing guidance to change without automatically invalidating the stored experience.

Beyond the points above, we have expanded evaluation to WebArena and LoCoMo, additional model families, and large API-accessed models (Table 1, Figure 3, and Appendix D.5). We have also added baseline diagnostics and re-verification integrations (Appendix D.3 and Figure 8), experiments on initial-memory scale and cross-domain memory reuse (Table 2 and Figure 6), and analyses of verification noise, checker reliability, and boundary conditions (Figure 5 and Appendices D.1–D.2). We hope this broader evidence helps clarify the framework’s scope and practical value, and appreciate your time in considering the revised work.

Thank you for the thorough rebuttal and additional analyses. I appreciate the authors' efforts in revising the manuscript and will update my score accordingly.

Thank you for carefully considering our rebuttal and additional analyses, and for your willingness to update your assessment. Your feedback helped us clarify MLRMem’s contribution and strengthen the experimental evidence!

Our central contribution is to combine task-level guidance and step-level strategies with feedback-driven re-verification, enabling agents to reuse applicable experience and revise their guidance as execution conditions change.

If these clarifications and additional results have addressed your concerns, we would be grateful if you would consider revising your score upward. Thank you again for your time and effort in helping us make MLRMem better!

Reviewer_dUUKOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed MLRMem, which is a training-free test-time memory framework for long-horizon LLM agents. The main contribution is building a task-level memory (goals, summaries, advice, etc.) and a step-level memory. Experiments ran in small open-source models. Some major issues definitely need to be fixed, and there are some contradictory findings throughout the whole paper. More details please refer to the strength and weakness sections.

Critical strengths
  • Agent memory is certainly an important issue to discover, and I appreciate the authors find this well-motivated topic. Agents that keep following a retrieved strategy after it stops applying is a practical problem.
  • The paper addressed a real failure mode of step-level memory. Table 4 explicitly showed some promising results: explicitly checking strategies against execution feedback, and distinguishing "conditions changed" from "the guidance itself is wrong" are sensible design that is supported by large ablation effects.
  • The experimental results are averaged over 3 runs with SD, use 3 backbones from 2 model families. The comparisons with baselines are more controlled than is typical.
Critical weaknesses

Several major weaknesses definitely reduced the paper's authority and credibility.

  • The gains are attributed to multi-level memory, but the evidence points almost entirely to re-verification. Therefore the key controls are missing: without the re-verification, MLRMem drops to 52.9 SR, very well below AdaMEM-low (68.5). Removing task-level memory leaves overall SR essentially unchanged (73.4 vs. 73.5 in Figure 6b), and the claimed unseen split benefit (3 points) is within run-to-run variation.
  • The memory structure itself does not appear to outperform existing methods: main gain comes from a per-step LLM feedback check, which is a generic mechanism that could be added to any baseline. It's unclear whether the proposed memory design contributes anything without baselines augmented with the same re-verification (let's say, AdaMEM-low or ReAct + re-verfication)
  • The cost and efficiency claims are inconsistent and not under controlled conditions. Abstract says "without introducing extra inference time cost", while the conclusion acknowledges "additional model calls". Without the number of LLM calls and input/output tokens per task, the Pareto-frontier claim (Figure 4) is not supported either.
Questions for authors
  • Please provide the exact definition of the selection score pt,mp_{t,m}, the penalty terms, and the criterion for "progress repeatedly stalls". How sensitive are the results to η\eta, the guidance lifetime, and top-kk? Will the code be released?
  • The current AdaMEM-high reproduction (56.9) is much lower than AdaMEM-low (68.5), which conflicts to the original paper. Were baseline hyperparameters adapted to your setup?
  • As I mentioned in weakness, what are the average number of LLM calls and the input/output tokens per task for MLRMem, AdaMEM-low/high, and ReasoningBank?
  • And how do AdaMEM-low or ReAct + re-verfication perform? This would isolate the contribution of the multi-level memory
  • How about evaluating this on harder and longer horizon environments (WebArena motivated this paper)? And how will the main argument and findings extend from small models (e.g., Qwen 4B/27B) to larger ones (100B), even close-source models?
6 comments

Thank you for the time and care you devoted to reviewing our work, and for recognizing the practical importance of revisiting memory guidance as execution conditions change. Your comments helped us identify where the original evidence and presentation needed to be strengthened. We have responded with additional ablations and baseline comparisons, a refinement of task-level memory use, explicit inference-cost accounting, and broader evaluations across tasks and model scales.

W1: Contributions of multi-level memory and re-verification.

Thank you for this careful and constructive assessment; to strengthen the paper’s empirical support and credibility, we have expanded the ablation analysis and added a task-memory diagnosis and refinement study (Section 4.3 and Appendix D.4).

Re-verification is a major contributor, but retaining it while removing multi-level memory also lowers success from 73.5% to 65.9%, supporting a contribution from memory beyond verification alone (Figure 7(a)). We acknowledge that the original task-level contribution is small overall and that the unseen mean difference alone does not establish a statistically reliable gain.

Our diagnosis identified restrictive matching of source-task objects and destinations and incomplete delivery of task advice to the action prompt; we therefore introduced role-conditioned advice and an independent advice section while keeping the memory library unchanged. With the same executor, step-level memory, and re-verification retained in both arms, the reported two-seed comparison improves seen and unseen success by 2.86 and 2.99 percentage points, respectively, raising overall success from 72.63% to 75.55% (Appendix D.4, Figure 10). These results support complementary contributions from memory and re-verification, while showing that task-level memory benefits depend on adapting and exposing relevant experience appropriately.

W2 & Q4:The performance of re-verification in other frameworks

Thank you for this valuable suggestion. To isolate the contribution of multi-level memory, we integrated re-verification into AdaMEM-low and retrieval-augmented ReAct on ALFWorld using Qwen3.5-4B, considering two integration strategies for AdaMEM-low: re-verification with self-evaluation and re-verification-only memory refresh. As shown in Figure 8, both AdaMEM-low variants experienced performance degradation (55.5% and 54.7% overall success), whereas ReAct benefited from re-verification (50.4% overall success). These mixed results suggest that re-verification is not universally effective when directly applied to existing memory structures, as persistent strategies or isolated retrieved insights provide limited targets for feedback-based verification. In contrast, MLRMem's task-level guidance and step-level strategies provide explicit targets for verification and dynamic revision, supporting the importance of multi-level memory in enabling effective re-verification, consistent with the ablation results in Figure 7.

W3 & Q3:Inference Costs and Efficiency

Thank you for pointing out this inconsistency. We have revised the abstract to remove the unsupported claim of "no extra inference time cost" and clarified our efficiency claims. We also report the average LLM calls and input/output tokens per task across all 274 ALFWorld tasks, including failed tasks (Appendices E.2–E.3, Tables 7–8).

MethodCalls/taskInput tokens (K)Output tokens (K)Total tokens (K)SR (%)
ReasoningBank31.349.628.478.061.4
AdaMEM-high39.675.731.2107.056.9
AdaMEM-low61.7105.358.1163.468.5
MLRMem127.9211.944.7256.773.5
MLRMem-Efficient75.1106.038.4144.573.0

To address the additional overhead, we introduced MLRMem-Efficient, which shares context across candidate checks and combines fact extraction with feedback verification (Appendices E.1 and E.4). Compared with original MLRMem, it reduces calls by 41.3% and total tokens by 43.7%, with only a 0.5 percentage-point decrease in success. Compared with AdaMEM-low, it achieves 4.5 percentage points higher success with 11.6% fewer tokens, despite requiring 21.7% more calls.

We have accordingly revised Figure 4 to show the success–token trade-off, where MLRMem-Efficient lies on the observed Pareto frontier. On 70 matched tasks, it achieves 75.7% versus 62.9% success with fewer interaction steps (26.4 versus 29.6), but records 1.28 times AdaMEM-low's cumulative inference time (952.8 versus 744.1 s/task). These results support a better success–token trade-off.

Q1: Selection rules, parameter sensitivity, and reproducibility.

Thank you for requesting these clarifications. Appendix B.2 now defines the selection score as four positive terms—local relevance, goal match, evidence support, and progress—minus conflict, looping, and redundancy penalties, all with unit weights.

Appendix B.3 specifies the distinct criteria for accumulated no-progress feedback and task-level stagnation, and explains how guidance lifetime extends with strategy length.

Appendices A.1–A.2 clarify that the reported constant adoption-check output leaves selection unchanged for η<0.9\eta < 0.9; our empirical sensitivity analysis focuses on retrieval depth, with increasing kk from 1 to 5 improving LoCoMo QA F1 from 42.6 to 48.6.

The implementation and configuration files are provided in the supplementary material. These explicit rules make memory selection and retention interpretable and reproducible, while allowing guidance to be revised as task evidence changes.

Q2: Baseline adaptation and AdaMEM-high performance.

Thank you for highlighting this discrepancy. We retained the original high/low strategy-retention rules while adapting both variants to Qwen3.5-4B and our shared frozen memory library, with matched retrieval, sampling, and interaction settings and a 4,096-token limit for both action and strategy generation (Appendix D.3). High discards its strategy after the current action, whereas low retains it until an update. Our analysis of 822 episodes per variant shows that strategy text is available in only 10.2% of high’s action-generation steps, compared with 98.5% for low (Table 6). This suggests that sparse guidance across successive actions may help explain high’s lower performance under our backbone and memory configuration. Low outperforms high in all three runs, supporting our use of the stronger AdaMEM-low baseline in the main comparisons.

Q5: Generalization to harder tasks and larger models.

Thank you for this valuable suggestion. We have added WebArena for realistic multi-step web interaction and LoCoMo for long-term conversational memory, where MLRMem achieves the highest overall performance among the compared methods with Qwen3.5-4B (Table 1). We further expanded cross-model evaluation across all four benchmarks, observing consistent benefits with Qwen and Gemma executors at different scales (Figure 3). Beyond 100B parameters, we evaluated API-accessed GLM-5.3-Flash (320B) and GLM-5.3 (approximately 743B), with MLRMem achieving ALFWorld success rates of 98.9% and 98.2%, respectively, exceeding both baselines in the reported runs (Appendix D.5, Figure 11). MLRMem operates through model inputs and outputs without requiring access to model weights, making its interface compatible with closed-source APIs as well. Together, these results support our central argument that structured memory use complements model capability across tasks, architectures, and scales.

Beyond the responses above, we have added experiments on initial-memory scale (Table 2), robustness to re-verification noise (Figure 5), and cross-domain memory reuse (Figure 6), together with analyses of checker reliability and boundary conditions (Appendices D.1–D.2). We hope these additions strengthen your confidence in the work; if the revised evidence addresses your concerns, we would be grateful if you would consider revising your score upward.

Reviewer_zC2COpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces MLRMem, a training-free memory framework for long-horizon language agents that combines task-level and step-level memories with strategy re-verification. During strategy selection, an LLM Checker evaluates whether candidate step-level strategies align with the task goal and satisfy execution conditions, using the interaction history and task-level guidance. It produces structured judgments supported by observational evidence, which rules then use to filter and rank candidates. After each action, a feedback check reassesses the active strategy’s applicability. Together with completion and expiry checks, this determines whether to retain the guidance or repeat memory selection. Withdrawal reasons inform subsequent selection to reduce repeated use of ineffective strategies. Model parameters and memory stores remain fixed during evaluation. Compared with AdaMEM-low, MLRMem increases ALFWorld overall success from 68.5% to 73.5% and WebShop average task score from 19.6 to 36.6. Ablations identify re-verification as the main contributor, particularly on unseen ALFWorld tasks.

Critical strengths
  1. MLRMem combines task-level and step-level memories so that local strategy selection considers both broader task guidance and current execution conditions. The LLM Checker evaluates whether candidate strategies are relevant to the current state, align with the task goal, and satisfy their execution conditions. This design helps reduce the use of experience that appears locally reasonable but is unsuitable for the overall task, providing a clear mechanism for coordinating strategies during long-horizon execution.

  2. Re-verification is one of the paper’s most compelling design choices. After executing an action, the agent uses environmental feedback to reassess strategy applicability and task progress, rather than continuing to rely on the judgment made during retrieval. When execution conditions change, feedback contradicts the guidance, or progress repeatedly stalls, the system withdraws the active guidance and selects another strategy. This feedback loop helps prevent ineffective strategies from continuing to dominate decision-making.

  3. MLRMem achieves test-time adaptation through prompt context, structured checks, and memory-management rules, without updating model parameters. In the ALFWorld experiments, all methods construct their memories from the same pool of 159 successful trajectories, and MLRMem achieves higher task success. This supports its ability to use existing experience more effectively. Its advantages concern training-free adaptation and effective use of fixed memory; overall computational efficiency must also account for the cost of verification calls.

  4. The framework distinguishes changes in execution conditions from evidence contradicting the strategy itself. In the former case, it withdraws the active guidance while retaining the source for later reassessment. In the latter, when the agent followed the strategy under valid conditions, it excludes the associated source and duplicate candidates for the remainder of the task. This treatment is more selective than uniformly discarding memories and uses withdrawal reasons to reduce repeated selection of ineffective strategies under unchanged conditions.

Critical weaknesses
  1. Candidate screening occurs when a new strategy must be selected, while post-action re-verification introduces ongoing checks. Both require additional model calls. Although fewer repetitive actions and failures may offset these costs, the reported runtime comparisons differ in concurrency and timing settings. Consequently, they do not conclusively establish a runtime advantage attributable to the method itself. Environment interactions, model inference costs, and end-to-end task latency should be evaluated separately.

  2. Important decisions depend on the LLM Checker’s judgments about goal alignment, execution conditions, and feedback. Incorrect judgments may cause useful guidance to be withdrawn prematurely or ineffective guidance to remain active. Evidence citations and an unknown judgment help mitigate these risks, but the paper does not directly evaluate false withdrawals, missed withdrawals, or incorrect failure attribution. The central concern is the reliability of these judgments across tasks, rather than an assumption that hallucinations necessarily cause the framework to fail.

  3. Freezing the memory stores helps control experimental variables, but the current experiments do not establish how newly acquired experience can be retained and reused across subsequent tasks. The system already records withdrawal reasons within a task, so it does use failure feedback. The open question is how these records can become reliable cross-task memories. Initialization performance also remains unclear when successful trajectories are scarce or source experience differs substantially from the target domain.

  4. MLRMem handles candidate strategies through condition screening, scoring, and exclusion rules. However, the complete scoring procedure, conflict penalties, and threshold sensitivity require further specification. In tasks involving multiple objectives, ambiguous feedback, or partial conflicts between task-level guidance and step-level strategies, there is limited evidence about whether fixed rules might exclude useful experience prematurely. This is an insufficiently tested boundary of applicability, rather than an established failure of the rule-based design.

Questions for authors
  1. Does MLRMem retain its end-to-end efficiency advantage under identical hardware, concurrency, and timing settings? How many model calls, tokens, and seconds are attributable to candidate checking and re-verification?

  2. Under what conditions are false withdrawals, missed withdrawals, and incorrect failure attribution most common? How much do evidence citations and the unknown mechanism improve these judgments?

  3. Does the method remain effective with no memory, few successful trajectories, or experience transferred from another domain? Can failure information discovered within one task be safely reused in later tasks?

  4. How are scoring thresholds and conflict penalties determined? Can the system avoid prematurely excluding valid strategies when conflicts are partial, premises are ambiguous, or feedback is delayed?

5 comments

Thank you for your positive review and constructive comments. We greatly appreciate your recognition of MLRMem's multi-level memory mechanism, the design of its re-verification approach, and the experimental analysis. Below, we provide detailed responses to each of your comments , hoping to address your concerns and clarify any remaining questions.

W1 & Q1: Inference overhead and efficiency.

We found that the additional checks in our original configuration increased LLM calls and request time. To reduce this overhead, MLRMem-Efficient shares context across candidate checks and combines fact extraction with feedback verification while retaining the original validators (Appendices E.1 and E.4).

MethodCalls/taskInput tokens/taskOutput tokens/taskTotal tokens/taskSR
MLRMem-Efficient75.1106.0K38.4K144.5K73.0%

On all 274 ALFWorld tasks, it reduces calls by 41.3% and total tokens by 43.7% relative to original MLRMem, with success changing from 73.5% to 73.0% (Appendix E.2, Table 7). On 70 seen tasks, it achieves 75.7% versus 62.9% success and uses fewer environment steps (26.4 versus 29.6), at approximately 1.28 times AdaMEM-low’s recorded cumulative inference time (952.8 versus 744.1 s/task). Appendix E.3 (Table 8) details candidate-checking and re-verification calls and tokens; their recorded cumulative request times in Efficient are approximately 505 and 115 s/task, respectively, with joint fact extraction included in the latter. These results show that MLRMem can retain most of its task-performance gains while reducing repeated context processing and verification calls, making structured memory selection and feedback-driven strategy correction more economical and achieving a better success–token trade-off than AdaMEM-low.

W2 & Q2: Reliability and robustness of the LLM Checker.

Thank you for highlighting this concern. We analyzed 200 stratified verification records from ALFWorld and WebShop (Appendix D.1). For example, labeling guidance as incorrect and proposing its withdrawal after the required size has been successfully selected constitutes a false withdrawal. Retaining guidance to select an unavailable size constitutes a missed withdrawal. Blaming the strategy when an object is absent from its expected location constitutes incorrect failure attribution. Replay experiments show that retaining unknown and no_progress reduces false withdrawals and incorrect failure attribution, whereas citation requirements have little effect on judgment error rates and primarily enable programmatic checks of the supporting evidence (Table 5).

We further evaluate downstream robustness at six verdict-retention levels on 140 ALFWorld seen tasks using Qwen3.5-4B, with one seed per setting (Section 4.2, Figure 5). Selected verdicts are replaced uniformly with supported or conflict, with attribution fields cleared.

Verdict retention100%90%80%70%60%50%
MLRMem success rate (%)75.778.675.774.375.776.4

The results show that MLRMem’s performance is robust to noise in re-verification judgments. The success rate remains largely stable across verdict-retention rates of 50–100%, indicating that our approach is not overly sensitive to checker noise under the tested perturbations. This robustness supports the use of MLRMem in tasks where re-verification judgments are noisy or uncertain.

W3 & Q3: Memory initialization and cross-task reuse.

Thank you for highlighting the dependence on initial experience and the reuse of newly acquired memories.

On ALFWorld, the variant without multi-level memory achieves 65.9% success, compared with 73.5% using the full memory stores (Section 4.3, Figure 7(a)). To further examine the role of initial experience, we evaluate ReasoningBank, AdaMEM-low, and MLRMem using 159, 795, and 1,590 successful source trajectories, with three runs per setting (Table 2). Across all three scales, MLRMem consistently outperforms both baselines on seen and unseen tasks, while maintaining an overall success rate of 72.9–74.2%.

For cross-domain initialization, we initialize our memory-construction variant with unmodified WebShop memories and evaluate it on WebArena shopping with online updates, obtaining approximately 15% success (Section 4.2, Figure 6). Newly acquired WebArena memories are selected for 77.0% of tasks, demonstrating that target-domain experience can be retained and retrieved across tasks. These results show that MLRMem uses initial experience effectively across the tested memory scales, while its online memory-construction variant enables newly acquired experience to guide subsequent tasks.

W4 & Q4: Scoring rules, conflict penalties, and premature exclusion.

Thank you for raising this concern. To give a concrete answer to your question about prematurely excluding valid strategies, we examined both the rules and their behavior in actual task runs, focusing on partial conflicts, ambiguous premises, and stalled progress. Candidate scoring uses fixed unit weights, with a penalty of 1 for a conflict with task-level guidance; the complete scoring procedure and eligibility checks are specified in Appendices B.1–B.2. We analyzed 822 ALFWorld episodes across three seeds using Qwen3.5-4B, with results reported in Appendix D.2 and Figure 9. Offline analysis showed that, under the deployed constant-output gate, adoption thresholds in [0, 0.9) left decisions unchanged, while varying the conflict penalty by ±25% preserved the selected candidate in at least 95.6% of the 780 decisions where competing candidates differed in conflict score.

ConditionEpisodesSuccess rate (%)Corresponding overall rate (%)
Partial conflicts45571.973.5
Ambiguous premises (unknown)12375.674.8
Stalled progress (no_progress)21370.973.5

Success rates under these conditions remain within 2.6 percentage points of their corresponding overall references. The ambiguous-premise group uses one annotated run; the other groups pool three runs, and the groups can overlap. The no_progress group measures stalled progress; actual feedback delay has not been tested.

To reduce the risk of prematurely excluding valid strategies, the system distinguishes stopping current guidance from excluding its source memory. Partial conflicts suspend the affected clauses and withdraw the associated strategy while preserving the remaining valid guidance. Unconfirmed premises defer current adoption while leaving the source available for reassessment. In 106 of 822 episodes, the same memory source was re-adopted after earlier guidance had ended (Appendices B.4 and D.2). Together, these observations show that ending current guidance preserves opportunities to reconsider and reuse potentially valid strategies as conditions change.

Beyond the responses above, we have added standard benchmark evaluations on WebArena and LoCoMo, together with broader comparisons across Qwen, Gemma, and large GLM executors (Table 1, Figure 3, and Appendix D.5). We have also added task-level memory diagnosis and refinement (Appendix D.4, Figure 10), re-verification integrations into AdaMEM-low and retrieval-augmented ReAct (Figure 8), baseline reproduction diagnostics (Appendix D.3), and a retrieval-depth comparison on LoCoMo (Appendix A.2). These additions broaden the evidence for the framework’s applicability and help clarify how its components contribute to performance. Thank you again for your thoughtful feedback and consideration of the revised work.

Thank you for the authors’ additional clarification. Regarding the response to Q2, claiming robust performance based on results from a single random seed may not be sufficiently rigorous. Could the authors provide additional experiments with multiple random seeds to further substantiate the robustness of their method?

Comments on the paper

1 comment

Comment:

Dear Reviewers,

Thank you for your careful reading of our paper and your constructive feedback. Your questions helped us clarify when task-level memory is useful, how reliably MLRMem can determine that a strategy no longer applies, and how much these checks cost during execution.

During the rebuttal period, we conducted additional experiments and revised the manuscript. Changes are highlighted in blue in the revised PDF. We summarize the main updates below.

  • Broader evaluation. We added WebArena and LoCoMo to evaluate MLRMem on web navigation and long-term conversational memory. We also expanded the evaluation across Qwen and Gemma models and tested two larger GLM models on ALFWorld (Table 1, Figure 3, and Appendix D.5).
  • Inference cost and efficiency. We introduced MLRMem-Efficient and reported model calls, token usage, and inference time. Across 274 ALFWorld tasks, the efficient variant used 41.3% fewer calls and 43.7% fewer tokens than the original MLRMem, with success rates of 73.0% and 73.5%, respectively. We also corrected the runtime comparison in the main text (Section 4.2 and Appendix E).
  • Contributions of memory and re-verification. We expanded the ablations of task-level and step-level memory, tested re-verification within AdaMEM-low and retrieval-augmented ReAct, and examined how adapting task-level advice to the current goal affects success (Section 4.3, Figures 7–8, and Appendix D.4).
  • Reliability of the LLM checker. We analyzed 200 verification records from ALFWorld and WebShop, tested robustness to noisy judgments, and examined cases involving partial conflicts, ambiguous premises, and delayed feedback (Figure 5 and Appendices D.1–D.2).
  • Memory scale and transfer. We compared three initial memory sizes, ranging from 159 to 1,590 successful source trajectories, and tested whether experience collected in WebShop transfers to WebArena (Table 2 and Figure 6).
  • Selection rules and baseline implementation. We clarified how MLRMem scores candidates, withdraws guidance, and selects a new strategy. We added a sensitivity analysis of the adoption threshold and conflict penalty and examined how often AdaMEM-high and AdaMEM-low provide strategy guidance during execution (Appendices A.2, B.1–B.3, and D.3).

We hope these revisions address your concerns and make MLRMem’s benefits, limitations, and computational costs easier to assess. We appreciate your consideration of the revised manuscript and would be happy to clarify any remaining questions.

Sincerely, The MLRMem Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.