acceptodds
Under review as a conference paper at ICLR 2027

DUAL-MEM: Cross-Task Memory Formation and Within-Task Active Memory Control for Tool-Using LLM Agents

Abstract

Tool-using large language model (LLM) agents can encode past interaction experiences into procedural memory. However, memory retrieval alone does not determine how retrieved memory should be adapted during execution. When execution stalls, the agent must decide whether to revise the current strategy or replace the retrieved procedure. We propose DUAL-MEM, a dual lifecycle framework that separates Strategy adaptation within a task from procedural memory evolution across tasks. During execution, an event-driven Controller decides whether to regenerate the Strategy while preserving the current Card or to retrieve a different Card when the procedure no longer provides suitable guidance. After each task, observed usage outcomes update Card history, while successful unmatched trajecto- ries can contribute new procedural memory for later tasks. Across four interactive benchmarks, DUAL-MEM improves average success rate from 46.30% to 49.30% over the strongest baseline. On ALFWorld, success improves by 2.98%–3.73% across three different model backbones, while average online input tokens are reduced by 4.10% to 4.50%. Over a 120-task continual stream, fixed probe success increases from 68.91% to 73.88%. Execution traces indicate a clear timescale separation between the two levels. Active Cards have an RMST@40 2.27 times that of Strategies, suggesting that Strategy revision often occurs without replacing the underlying procedural memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agents,Agent Memory, Procedural Memory,Lifelong Learning,Adaptive Memory Control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EJqYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper separates persistent procedural Cards from task-specific Strategies for tool-using LLM agents. At execution stalls, a contextual-bandit controller either regenerates the Strategy while retaining the Card or retrieves a different Card. Completed tasks update Card usage statistics and can add new procedures. The evaluation reports a 3.00 percentage-point improvement over MemCon averaged across four interactive benchmarks, additional QA results, and ALFWorld analyses of controller choices, memory evolution, and token consumption. The proposed distinction is sensible, and the experiments provide encouraging evidence, but attribution of the gains to the specific lifecycle design remains incomplete.

Critical strengths
  1. The separation between a reusable procedure and its current instantiation addresses a concrete failure mode: a locally unsuccessful plan need not invalidate the underlying procedural knowledge. Table 2(a) goes beyond a no-memory comparison by including fixed interventions, Coupled Refresh, non-contextual UCB, and LinTS. The 68.91% result for LinUCB versus 64.93% for Coupled Refresh is useful evidence that selecting the intervention level can matter.

  2. The empirical coverage is reasonably broad. Table 1 reports improvements on all four interactive benchmarks with three evaluation runs, while the ALFWorld efficiency study includes three backbones. Table 2(b) separately examines memory growth and usage history. The paper also appropriately states that the longer Card lifetimes are descriptive consequences of the controller structure, rather than independent causal validation of the hierarchy (Section 5.3).

Critical weaknesses
  1. The incremental benefit over existing adaptive memory control is not isolated cleanly enough. MemCon already learns context-dependent memory operations; the critical question is what the Card/Strategy distinction and segment-level feedback add under otherwise matched conditions. The internal ablations are helpful, but Coupled Refresh changes the intervention-selection rule as well as how the two levels are refreshed. Meanwhile, Appendix A specifies GPT-5.6 Sol for DUAL-MEM's Card extraction but does not establish an equivalent construction model and budget for each baseline. Sharing source trajectories alone does not eliminate this possible confound. A comparison using the same Cards, extractor, Watcher, execution prompts, and construction resources would make the modest full-suite gain much more persuasive.

  2. Important parts of the control and continual-evaluation protocol remain ambiguous. Table 5 describes progress in terms such as a task-relevant state or a verified intermediate condition, without specifying the exact rules or prompts that implement those judgments. These signals determine both intervention timing and controller reward. In addition, Appendix B states that LinUCB parameters persist across tasks, whereas the slow-lifecycle experiment describes a fixed Fast Lifecycle and frozen probe state. It is unclear whether the controller matrices and reward vectors are identical across the four Table 2(b) variants, or whether some variants also benefit from additional controller learning on the 120-task stream. This distinction directly affects attribution of the 68.91% to 73.88% improvement to persistent-memory evolution.

Questions for authors
  1. Clarification of the main comparison: Which model and construction budget produced each baseline's memory, especially MemCon's? Is there a matched comparison that changes only the lifecycle/controller design while sharing the extracted Cards and runtime components?

  2. Clarification of Table 2(b): What happens to the LinUCB matrices and reward vectors during the 120-task stream and the held-out probe? Please specify the initialization, updates, and resets for every variant, and provide results with matched controller state if those states currently differ.

  3. Clarification of the progress signal: Please provide the executable adapter rules or prompts underlying Table 5, including how task relevance and verification are determined in AppWorld, tau-bench, and GAIA. Evidence that these adapters use the same permitted observations and feedback across compared methods would materially strengthen the result.

1 comment

We thank the reviewer for the thorough feedback and for acknowledging the value of our intervention ablations and the breadth of the evaluation. We appreciate the emphasis on experimental attribution and reproducibility, and we clarify the two comparison settings and the continual-evaluation protocol below.

R1. Baseline comparability and contribution of the two-level design (W1 / Q1)

Our paper uses two complementary comparisons. In the main six-benchmark study (Table 1 and Appendix A), memory-based methods share the benchmark-specific source trajectories, execution backbone, task order, tool interface, and evaluator. Each baseline retains its native memory representation and construction procedure. This evaluates complete systems under shared source experience and execution conditions. The interactive-benchmark average increases from 46.30% (MemCon) to 49.30% (DUAL-MEM); Appendix A, Table 4, also reports 44.05% (DUAL-MEM) versus 41.87% (ReMe) on AppWorld.

Table 2(a) then isolates the intervention-policy choice within a shared representation: all variants use the same frozen 60-Card Bank, retrieval index, Watcher, and Usage State. With these components fixed, contextual LinUCB reaches 68.91%, compared with 64.93% for Coupled Refresh. Our additional deterministic R→Q baseline achieves 64.68% under the same protocol. The new rule comparison reinforces the value of choosing the intervention level from execution context while holding the underlying procedural-memory system constant.

R2. Continual-evaluation protocol and Controller updates (W2 / Q2)

Section 4.2 and Appendix A specify a shared 60-Card initial Bank, a separate 120-task continual stream, and evaluation on the same 134-task held-out ALFWorld probe. Table 2(b) keeps the Fast Lifecycle design fixed and varies Bank Evolution and Usage History. The results show distinct contributions:

VariantSuccess (%) ↑Input tokens/task ↓
Frozen Persistent State68.91 ± 1.5552.60k
History Only70.15 ± 1.4944.13k
Bank Evolution Only71.64 ± 1.4943.73k
Full DUAL-MEM73.88 ± 1.4941.90k

Appendix B (Eqs. 20–21) specifies LinUCB initialization (A_a = I₉, h_a = 0) and segment-level parameter updates. It also specifies that persistent memory updates are applied after task termination. We appreciate the request for an explicit variant-by-variant account of the Controller state and will clarify when the LinUCB matrices and reward vectors are initialized, updated, or frozen during the continual stream and held-out probe. This will make the Table 2(b) protocol easier to reproduce and interpret.

R3. Observable progress and benchmark adapters (W2 / Q3)

Appendix B, Eq. (17), defines segment progress as a binary event: Δp = 1 when at least one observable task-relevant progress event occurs after intervention. Table 5 identifies the evidence used for each environment:

BenchmarkObservable evidence of progress
ALFWorldConfirmed object/receptacle state or completed manipulation advancing the task
AppWorldStructured tool response confirming an application state, side effect, or intermediate condition
ScienceWorldIncreased task score or observed satisfaction of an experimental condition
τ-benchTool-confirmed transaction state or resolution of a pending user constraint
GAIAVerified artifact, evidence item, or subgoal-resolving state transition

The binary definition gives the Controller a common segment-level feedback scale across environments with different native signals. As described in Appendix B, adapters use observations available through the environment or tool interface. Segment utility then combines progress with completion, intervention cost, and stagnation (Eqs. 18–19). We will expand the adapter descriptions with precise conditions and examples, especially for AppWorld, τ-bench, and GAIA, as suggested.

Thank you again for these detailed comments. They have helped us improve the clarity of the experimental protocol and the presentation of the controlled evidence.

Reviewer_zNvYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DUAL-MEM, a dual-lifecycle memory framework for tool-using large language model (LLM) agents. The framework distinguishes between "strategy adaptation" within a task and the "evolution of procedural memory" across tasks, utilizing a controller to determine whether to revise strategies or replace memory entries. Experimental results demonstrate that this framework achieves a higher task success rate compared to existing memory-based baseline methods.

Critical strengths
  1. The paper presents the dual-lifecycle framework and its key components in a relatively clear and structured manner, making the overall approach easy to follow.

  2. The method demonstrates consistent performance improvements across multiple benchmarks.

Critical weaknesses

A core contribution of this paper is the "Slow Lifecycle" mechanism, which enables the evolution of procedural memory across tasks, allowing the system to refine its memory as it accumulates experience. However, in Appendix B (lines 894–913), the authors state that existing "Cards" cannot be modified or deleted during the evaluation process, and that the addition of any new Card must satisfy the "semantic novelty constraint" defined in Eq. (25).

A key concern of mine is whether this design truly allows for the continuous refinement of existing procedural knowledge, or if it merely results in the monotonic accumulation of new memory entries. Consider a scenario where the applicability conditions for an existing Card remain unchanged, but the internal procedure itself becomes obsolete or incorrect due to environmental shifts (e.g., an update to a tool API). In such a case, the system appears unable to directly modify or delete the problematic Card. Furthermore, a corrected version of the procedure might be rejected by the "novelty gate" if its applicability description is too similar to that of the existing, outdated Card. While the system might be able to bypass an incorrect Card for a specific task via the "REACQUIRE" mechanism, this does not constitute a correction of the underlying, flawed knowledge.

Consequently, it remains unclear how DUAL-MEM handles outdated or erroneous existing memories, and whether this "append-only" memory mechanism can genuinely support long-term memory evolution.

Another core claim of this paper is the use of a "Fast Lifecycle" to distinguish between failures at the strategy level and the card level, with interventions carried out via either RENEW or REACQUIRE, respectively. A key concern I have is that the existing experimental results do not definitively demonstrate that the controller can accurately determine whether a failure originates from the strategy or the card. Table 2(a) primarily reports overall task success rates under various control strategies; however, a higher success rate does not equate to an accurate identification of the source of failure at different levels. For instance, when a card remains valid but the strategy fails, can the controller reliably choose to RENEW? Conversely, when a card is no longer applicable, can it promptly opt for REACQUIRE? While Figure 5 shows that the lifespan of a card is 2.27 times that of a strategy, this is—to some extent—a direct consequence of the mechanism design itself (since RENEW retains the card while regenerating the strategy). Consequently, this observation cannot serve as independent proof that the controller’s assessments are correct. The current results do not sufficiently demonstrate that the performance gains come from the Controller's ability to distinguish between different failure types, rather than simply from a more flexible intervention strategy.

Questions for authors
  1. Appendix B explicitly states that existing Cards cannot be modified or deleted. Given the "semantic novelty constraint" in Eq. (25), how does DUAL-MEM correct an erroneous memory—for instance, when the applicable scenario for an existing Card remains unchanged, but its internal procedure has become outdated or ineffective? Do the authors have direct experimental results demonstrating the system's ability to successfully recover from such scenarios?

  2. A core motivation of the paper is to distinguish between "strategy-level" failures and "card-level" failures. However, Table 2 primarily reports overall task success rates. What evidence exists to prove that the Controller genuinely selects the appropriate intervention for different types of failures, rather than simply benefiting from more flexible, dynamic retries? Could the authors provide a breakdown of "intervention selection accuracy" and "recovery performance" for each of the two failure categories?

  3. Figure 5 shows that the Memory Bank grows from 60 to approximately 77 Cards during continuous adaptation. To what extent are the observed performance gains attributable to the proposed "dual-lifecycle design," as opposed to a simple increase in the number of stored procedural memories? If compared to an append-only memory baseline that uses the same success trajectory, has similar computational overhead, and the same memory capacity, will DUAL-MEM still be able to maintain its current performance advantage?

1 comment

We sincerely appreciate the reviewer's close reading of the Slow and Fast Lifecycles. Your examples helped us see where the description of memory evolution and the interpretation of intervention decisions can be made more precise.

R1. How procedural memory evolves (W1 / Q1)

The Slow Lifecycle maintains two persistent states: the Memory Bank B and the Card-specific Usage State U. Appendix B (Eqs. 22–24) describes how completed execution segments update Card reliability and evidence support. Progress or successful completion contributes positive evidence; ineffective use followed by Card abandonment or unsuccessful termination contributes negative evidence. A failed local Strategy that is renewed leaves the Card's reliability unchanged. These statistics then inform the Controller in subsequent tasks.

In parallel, successful trajectories that reveal a memory gap can yield new Cards through the novelty-controlled formation rule (Eq. 25). Together, the two updates expand procedural coverage and adapt subsequent use of existing knowledge. Existing Card text is immutable in the current implementation: an obsolete procedure can be bypassed through REACQUIRE when an alternative is available, while direct rewriting or deletion of that Card is outside the current update operation. We will clarify this scope of memory evolution and the stale-procedure case in the revision.

R2. Choosing the appropriate intervention level (W2 / Q2)

Our Controller learns which recovery action is more useful in the current execution context. Its inputs include stagnation, Strategy/Card age, prior RENEW attempts, semantic applicability, and historical Card usage (Appendix B, Eq. 10). Following each intervention, the resulting segment provides a utility signal combining progress, task completion, intervention cost, and continued stagnation (Eq. 19). This makes the recovery decision responsive to observed execution outcomes.

Table 2(a) compares intervention policies under the same frozen Memory Bank, retrieval index, Watcher, and Usage State. To make the comparison with simple retry logic more direct, we additionally evaluated a deterministic R→Q controller:

Intervention policySuccess (%) ↑Input tokens/task ↓
Always Renew62.94 ± 1.1459.70k
Coupled Refresh64.93 ± 1.4959.80k
R→Q Rule (new)64.68 ± 0.4358.81k
Contextual LinUCB68.91 ± 1.5552.60k

LinUCB improves success by 4.23 pp and saves 10.55% of online input tokens relative to R→Q, even though both use the same RENEW/REACQUIRE operations. We will describe this contribution more precisely as context-dependent selection of the recovery level from post-intervention utility, which is the learning objective of the Controller.

R3. Continual gains and Memory Bank growth (Q3)

The Slow Lifecycle ablation in Table 2(b) separately varies Bank Evolution and Usage History after a shared 120-task stream:

Persistent-state variantBank EvolutionUsage HistorySuccess (%) ↑
Frozen Persistent StateNoNo68.91 ± 1.55
History OnlyNoYes70.15 ± 1.49
Bank Evolution OnlyYesNo71.64 ± 1.49
Full DUAL-MEMYesYes73.88 ± 1.49

History Only gains 1.24 pp without Bank expansion, and combining usage evidence with Bank Evolution adds 2.24 pp over Bank Evolution Only. Appendix D, Table 12, provides a useful complementary observation: when the mean final Bank size grows from 76.67 to 96 Cards, success decreases from 73.88% to 71.64%. These results illustrate the importance of both procedural coverage and the evidence used to guide memory adaptation, rather than memory quantity alone.

We are grateful for the concrete questions; they help us articulate the mechanism and its empirical behavior more clearly.

Reviewer_EG7WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how tool augmented LLM agents should recover when execution fails. The key idea is that failure may come either from an incorrect task specific strategy or from unsuitable long term procedural memory, and these two cases should be treated differently. To address this, the paper proposes DUAL MEM, which separates persistent procedural memory into Cards and temporary task level plans into Strategies. When the agent gets stuck, a controller decides whether to renew the current Strategy or reacquire a new Card. The framework also updates memory across tasks by adding new useful Cards and tracking their reliability. Experiments on several benchmarks show that this design improves success rates over prior memory based agent methods.

Critical strengths

The paper identifies an interesting and relevant gap in current agent research, namely the need to distinguish strategy failure from memory failure during execution. This is a meaningful framing that connects in task control with cross task memory evolution.

Critical weaknesses
  1. The paper does not show that the improvement truly comes from distinguishing strategy failure from memory failure, as opposed to coming from additional engineered control components. A simpler rule based controller is not tested, so the necessity of the learned controller remains unclear.

  2. The lifetime analysis is used to support the hierarchical design, but it is largely induced by the framework definition itself. This makes the interpretation weaker than the paper sometimes suggests.

  3. The token comparison excludes parts of the slow lifecycle cost, especially memory formation and post task processing. This makes the efficiency advantage less convincing.

  4. The framework depends on many manually chosen thresholds, feature definitions, and benchmark specific progress rules. This raises concerns about robustness, portability, and whether the contribution is more system engineering than methodological novelty.

Questions for authors

see weanknesses above

1 comment

We sincerely thank the reviewer for recognizing the importance of distinguishing task-specific Strategy failures from unsuitable procedural memory. Your comments are especially helpful in clarifying which gains come from the intervention policy and how we should present the lifecycle analysis.

R1. Effectiveness of the learned Controller (W1)

Following your suggestion, we added a Renew-then-Reacquire (R→Q) rule-based baseline. At the first lifecycle event, it keeps the current Card and regenerates the Strategy. If execution stalls again without observable progress, it reacquires a different Card. The rule resets when progress is observed or a new Card is activated. It introduces no learned parameters.

We evaluated this baseline using the same frozen 60-Card Bank, retrieval index, Watcher, Usage State, Qwen3.5-27B backbone, 134 ALFWorld tasks, and execution budget as Table 2(a), with three independent runs.

ControllerSuccess (%) ↑Steps/task ↓Input tokens/task ↓
Always Renew62.94 ± 1.1425.4059.70k
Always Reacquire61.44 ± 1.1426.3065.80k
Coupled Refresh64.93 ± 1.4925.1059.80k
R→Q Rule (new)64.68 ± 0.4324.8758.81k
Non-contextual UCB65.17 ± 1.1424.3056.90k
LinTS66.92 ± 1.1423.6054.80k
LinUCB (DUAL-MEM)68.91 ± 1.5523.0052.60k

Compared with R→Q, LinUCB improves success by 4.23 percentage points, reduces execution by 1.87 steps/task, and saves 10.55% of online input tokens. The rule and Coupled Refresh achieve similar success rates, whereas contextual LinUCB performs better across all three metrics. Since the memory representation and execution components are held fixed, this comparison highlights the contribution of learning when to preserve the current Card and when to replace it.

R2. Meaning of the Card–Strategy lifetime analysis (W2)

Thank you for raising this interpretation. As noted in Section 5.3 and Appendix C, the lifetime statistics describe the execution dynamics of our hierarchy. Across 402 episodes, we observe 607 RENEW and 342 REACQUIRE events; 53.10% of active Card spans contain multiple successive Strategies. The 2.27× RMST ratio summarizes how frequently local guidance changes while procedural memory remains active. We will make this descriptive role clearer in the abstract and main discussion, and foreground the controlled comparisons in Table 2(a) when discussing effectiveness.

R3. Scope of the token-efficiency comparison (W3)

Figure 4 measures online execution input tokens, including Actor calls, Strategy generation, and retrieval/reacquisition prompts. Initial Bank construction and post-task processing are outside this explicitly defined metric (Section 4.3 and Appendix A). Under the same online accounting, DUAL-MEM reduces input tokens by 4.10%–4.50% relative to MemCon across three backbones while improving task success. We will state the online/offline accounting boundary more prominently in the figure caption and discussion.

R4. Robustness and implementation details (W4)

Appendix B details the nine-dimensional Controller context, the Watcher, Card suitability, and segment utility. Appendix D (Tables 10–13) varies the three main control parameters on ALFWorld:

ParameterDefaultRegion within 1 pp of default success
Watcher stall threshold, h43–5
Card admission threshold, ε0.850.80–0.90
LinUCB exploration, α0.500.20–0.80

These results show a stable operating region around each default. The Card suitability threshold (0.60) and segment-reward coefficients are also explicitly specified in Appendix B, with the same default configuration used in the controlled evaluations. We appreciate the suggestion to make these implementation choices easier to locate and will strengthen the presentation accordingly.

Thank you again for the careful and constructive review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.