acceptodds
Under review as a conference paper at ICLR 2027

Cross-Task Experience Memory through Selective Strategy Revision

Abstract

Experience memory lets language-model agents reuse strategies from earlier tasks. As related tasks reveal new conditions, the memory system must decide which parts of a strategy to preserve and which to revise. We introduce Cross-Task Experience Memory (CTEM), which makes this decision using the relationship between new trajectory evidence and a stored strategy. CTEM links compact strategy items to source fragments: confirming evidence is attached without rewriting, while refinements, counterexamples, and unresolved conditions guide selective updates. The resulting strategy fields provide concise guidance for later tasks, with source context retained for future revisions. On WebArena, CTEM improves macro accuracy over ReasoningBank by 2.1–7.7 percentage points across three backbones; on AppWorld, dense scores improve by 8.4 and 8.8 points. With query enhancement disabled, expanding the update policy from support-only to support+refine and then full relations raises WebArena macro accuracy from 35.1 to 36.3 to 38.2. These results support selective revision as a useful design for maintaining reusable experience across tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, experience memory, cross-task learning, memory consolidation, selective strategy revision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PBWoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CTEM, a cross-task experience memory framework that maintains shared strategies across an online task stream. Instead of storing each completed trajectory as an independent memory, CTEM links compact strategy items to source trajectory fragments and classifies new evidence as support, refinement, contradiction, uncertainty, or no match. The corresponding relation determines whether the strategy is preserved, revised, qualified with exceptions, or separated into a new memory. Experiments on WebArena and AppWorld show consistent improvements over ReasoningBank and other memory baselines.

Critical strengths

1.The paper studies an important problem in long-running agents: how to update reusable strategies without repeatedly overwriting useful knowledge.

2.The distinction between actor-facing compact strategies and source evidence retained for later revision is intuitive and well motivated.

3.The evaluation covers two different interactive environments and multiple language-model backbones. CTEM-direct also outperforms ReasoningBank, suggesting that the gains are not solely due to query enhancement.

4.The relation-update ablation provides useful evidence that selective refinement is more effective than simply accumulating supporting experiences.

Critical weaknesses

1.The methodological novelty is moderate. Selective memory editing, consolidation, evidence retention, and applicability-aware updates have substantial overlap with prior work such as ExpeL, A-MEM, RecMem, and related experience-memory systems.

2.The evaluation uses the same fixed task order across runs. Since CTEM explicitly relies on cross-task accumulation, performance may depend strongly on which related tasks appear earlier, but task-order sensitivity is not evaluated.

3.The strongest evidence supports support and refinement, while contradiction and uncertainty are extremely rare in the accumulated memory. It is therefore unclear how much these additional relation types actually contribute to the reported improvement.

4.Several components remain entangled. In particular, the current experiments do not isolate the value of source-fragment access from the relation-aware updater itself, nor compare against a generic updater given the same evidence.

5.Reproducibility is incomplete because some important implementation details, including the embedding backend, memory-pipeline model, token limit, and AppWorld memory-reset policy, are not fully specified.

Questions for authors

1.How sensitive are the reported gains to task ordering? Would the relative advantage persist under shuffled or adversarially reordered task streams?

2.Since contradiction and uncertainty account for only a very small fraction of relation records, how much of the full-policy improvement can actually be attributed to these two relations?

3.Can the authors clarify whether source-fragment access itself contributes substantially to performance, independently of relation-aware updating?

4.What memory-reset policy is used across repeated AppWorld evaluations, and could accumulated memory from one evaluation affect another?

5.Can the authors clarify the embedding model and memory-pipeline model used in the reported experiments?

Reviewer_AkvNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces Cross-Task Experience Memory (CTEM), an online memory framework for LLM agents that consolidates task trajectories into shared, evolving strategies. Its core contributions and validation are:

  1. CTEM separates compact, actor-facing strategy items in a Memory Library (title, description, content, counterexamples, unresolved) from raw trajectory fragments and anchors stored in a Mirror Environment. After each episode, an internal LLM judge labels the outcome and extracts up to three "environment slices" (each linking a trajectory span, an evidence anchor, and a retrieval anchor).
  2. Rather than storing separate per-task memories or unconditionally rewriting rules, CTEM retrieves the closest existing strategy for each new slice and classifies their relationship into one of five operations: Support (attach evidence without rewriting text), Refine (update applicability conditions/boundaries), Contradict (log concrete exceptions in counterexamples), Uncertain (log edge cases in unresolved), or No match (create a new strategy). At inference time, it optionally uses retrieved historical anchors to enhance the search query (CTEM vs. CTEM-direct).
  3. Evaluated on online task streams in WebArena (655 tasks across 4 domains with GPT-5.4-mini, GPT-5.4, and Qwen3-32B) and AppWorld (test_normal, 168 tasks with the two GPT backbones), both CTEM and CTEM-direct outperform No Memory, AWM, ReasoningBank, and A-MEM across all benchmark–backbone aggregates (+2.1 to +7.7 macro accuracy points over ReasoningBank on WebArena; +8.4 to +8.8 dense score points on AppWorld) with only ~0.5%–1.8% total token overhead over ReasoningBank and much higher memory reuse coverage (83.1% of 136 consolidated items retrieved vs. 3.6% of ReasoningBank's 1,930 items). Ablations on WebArena show accuracy steadily improves as the update policy expands from Support-only (35.1%) to Support + refine (36.3%) to Full (38.2%).
Critical strengths
  1. By separating actor-facing strategy summaries (Memory Library) from linked raw trajectory spans (Mirror Environment) and treating support matches as no-op attachments, CTEM prevents the severe memory bloat seen in append-only baselines. On WebArena Shopping, CTEM consolidates experience into 136 items (with 83.1% retrieved by subsequent tasks) compared to 1,930 items for ReasoningBank (where 96.4% are never retrieved), allowing accuracy to scale positively as retrieval count kk increases (Figure 4a, Figure 5).
  2. Across three LLM backbones on WebArena and two on AppWorld, both full CTEM and CTEM-direct (without query enhancement) consistently outperform No Memory, AWM, ReasoningBank, and A-MEM while reducing average interaction steps. Moreover, the relation-subset ablation (Figure 3 / Table 6) directly validates the progressive benefit of adding refine (+1.2 macro pp) and contradict/uncertain (+1.9 macro pp) over support-only, with only a 0.5%–1.8% increase in total tokens per task over ReasoningBank (Table 4).
Critical weaknesses
  1. In Section 3.3 (line 365) and Appendix F (lines 852–854), the paper explicitly states that key details of its own experiments are "not documented" or "unavailable", including the identifiers for the memory-pipeline LLM and embedding backend, the numerical memory-block token limit, which WebArena domains are actually covered in the Table 4 token profile, and the AppWorld memory reset policy across the three repeated evaluations per task. Not knowing whether memory was reset between repeated trials of the same task in AppWorld (test_normal) raises serious concerns about potential within-task leakage across trials, and omitting the official strict task-success metric on AppWorld in favor of only the dense score makes comparison with prior work difficult.
  2. While the relation-subset ablation (Figure 3) compares Support-only (which never rewrites a strategy after creation), Support + refine, and Full, it does not compare CTEM's 4-way relation classifier and specialized writers against a generic LLM memory updater that simply merges/rewrites a matched strategy using the same retrieved source slices without explicit relation classification (a gap noted in Limitations, line 473). Furthermore, contradict (1.9% of updates) and uncertain (0.4% of updates) occur only 12 times total on Shopping (Figure 8), making it unclear why enabling them accounts for a larger macro accuracy jump (+1.9 pp from Support + refine to Full) than enabling refine (+1.2 pp, 30.2% of updates).
  3. All online streams are evaluated in a single fixed ascending task-ID order (3 runs with temperature 0.0 on WebArena), where adjacent tasks often share templates or entities. Because Support-only (which freezes each strategy's text after the very first task that creates it) already achieves most of the gain over No Memory (e.g., 27.8% →\to 35.1% macro accuracy, vs. 38.2% for Full), the paper does not test whether selective revision remains robust under shuffled task orders, nor whether restricting each anchor to update at most one candidate memory (kupdate=1k_{\text{update}} = 1) causes fragmentation when tasks span multiple strategies.
Questions for authors
  1. Why does Appendix F (lines 852–854) state that the embedding model, memory-pipeline LLM, prompt budget, and AppWorld memory reset policy are "unavailable"? Please specify these details, confirm whether memory was reset between the 3 repeated trials per AppWorld task, and provide AppWorld's standard strict task-success rate.
  2. Why do contradict (1.9% of updates) and uncertain (0.4%) yield a larger accuracy gain (+1.9 macro pp in Table 6) than refine (30.2% of updates, +1.2 pp)? Also, how does CTEM compare to a single-prompt generic updater that rewrites matched memories using the same source slices without relation classification?
  3. Since WebArena tasks are run in fixed ascending task-ID order (where similar tasks are grouped together), how does CTEM perform under randomized task orderings? Also, what causes the run-to-run variance in Table 5 given that task order is fixed and both acting and construction temperatures are set to 0.0?
Reviewer_YWUROpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Cross-Task Experience Memory (CTEM), a memory system that consolidates experience from related agent tasks into shared strategies. It links compact strategy items to source trajectory fragments and classifies new evidence as support, refine, contradict, or uncertain to decide whether and how to update a strategy. On WebArena and AppWorld, CTEM outperforms the compared memory baselines on the reported aggregate metrics. Ablations suggest that strategy refinement and the full update policy improve WebArena accuracy.

Critical strengths
  1. CTEM explicitly distinguishes confirming evidence from new conditions, counterexamples, and unresolved cases. This provides a clear and practical strategy for maintaining reusable experience instead of simply appending memories.
  2. CTEM improves over ReasoningBank across the reported WebArena backbones and both AppWorld settings. The update-policy ablation also shows a consistent increase in WebArena macro accuracy from 35.1 to 36.3 to 38.2.
Critical weaknesses
  1. The online evaluations use a fixed task order, but memory formation and revision are likely order-dependent. The paper does not test different task orders or distribution shifts, so it is unclear how robust the gains are when earlier experiences differ.
  2. The contribution of the individual update relations is not fully established. Contradict and uncertain evidence are rare in the reported memory statistics, and the ablation evaluates them together as part of the full policy. The paper also lacks a comparison with a generic updater given the same evidence, making it difficult to determine how much improvement comes specifically from relation-aware revision.
Questions for authors
  1. How sensitive are the results to task order? Can the authors report results across multiple randomized task orders, rather than repeated runs with the same order?
  2. Can the authors isolate the contribution of contradict and uncertain updates, and compare CTEM with a generic memory updater that receives the same source evidence?
  3. What exact models and embedding backend are used for the memory pipeline, and what are the memory-block token limit and AppWorld memory-reset protocol? These details appear necessary for reproduction but are not fully specified.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.