acceptodds
Withdrawn from ICLR 2027

Composing Experience Across Tasks: Hierarchical Process Experience Memory for Interactive Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

12 views

Abstract

Interactive agents repeatedly face tasks that share reusable subprocesses, but memories based on complete trajectories, task-specific reflections, or standalone routines do not always preserve how subprocesses combine or whether locally plausible guidance helps the final task. We introduce Hierarchical Process Experience Memory (HPEM), an external plug-in memory that organizes task-parameter-independent experience into leaf processes and composed workflows. Each node records an objective, interaction stage, subgoals, applicability and failure boundaries, evidence, maturity, and ordered child relations. HPEM learns process abstractions from successful interactions, composes compatible experience into higher-level workflows, and updates node reliability using both local feedback and downstream task outcomes. At inference time, HPEM retrieves a relevant leaf process or composed workflow as optional guidance; the host agent retains full control of reasoning and environment actions. Memory is frozen before evaluation to separate experience acquisition from cross-task reuse. Extensive experiments show gains in task success and interaction efficiency in multiple settings with official host agents, while frozen evaluations confirm retrieval of learned high-level workflows.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
LLM Agents, Agent Memory, Cross-task Experience

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vkd5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces HPEM, an external memory that stores reusable processes and workflows with shared subprocess identities. Evidence from interactions determines retrieval eligibility, and memory is frozen before evaluation.

Critical strengths
  1. The method produces clear success gains on WebShop and ALFWorld.
  2. Reusing the same memory across model sizes is useful, and the paired evaluations and frozen-memory checks strengthen the results.
Critical weaknesses
  1. The main concern is whether the gains come from hierarchical experience memory or the authored process templates. A fixed-template baseline is missing.

  2. The structural comparisons are mixed: Full improves strict success over Flat, but Flat achieves higher reward at lower cost. The shared-identity ablation needs more implementation detail, while the composition ablation’s confidence interval includes zero.

  3. The feedback ablation changes both memory construction and retrieval coverage, leaving the source of its benefit unclear. Transfer is mostly tested on new instances of familiar task families, and the AndroidWorld gain is uncertain.

Questions for authors
  1. How does HPEM compare with the same templates without experience acquisition?

  2. What changes in the shared-identity ablation besides node sharing?

  3. Can downstream failures lower trust once local evidence takes priority?

Reviewer_Vm9qOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Hierarchical Process Experience Memory, a hierarchical process experience memory framework for interactive agents. HPEM abstracts interaction experiences into process nodes with explicit goals, applicability conditions, success signals, and failure boundaries, and organises reusable sub-processes into a DAG. It uses local verification and downstream task outcomes to update the reliability and maturity of stored experiences. For new tasks, HPEM retrieves suitable workflows or composes multiple experiences to cover different subgoals. As an external memory system, HPEM guides existing agents without replacing their reasoning or action selection.

Critical strengths
  • The paper addresses a practical challenge in cross-task experience reuse: complete task trajectories may not transfer directly to new tasks with different objectives and subgoals. HPEM decomposes experiences into reusable process components and lets multiple workflows share child nodes, providing a structured way to compose and reuse experiences.
  • The framework considers both local process verification and downstream task outcomes when updating the reliability and maturity of experiences. Its retrieval mechanism also considers task objectives, interaction stages, task families, subgoal coverage, and context tags, aiming to balance experience quality with relevance to the current task.
  • The paper evaluates HPEM on interactive task benchmarks including WebShop, ALFWorld, and AndroidWorld, comparing different base agents, model scales, and memory structures. The results show notable success-rate improvements on WebShop and ALFWorld. Comparisons between Full and variants such as Flat and Trajectory also provide an initial investigation into the contribution of hierarchical process representations.
Critical weaknesses
  • HPEM integrates process abstraction, hierarchical workflow composition, outcome feedback, and structured retrieval into a unified framework, but these mechanisms substantially overlap with existing experience memory and hierarchical skill methods. The paper does not sufficiently establish its substantive methodological novelty over prior work.
  • Process abstraction relies on predefined semantic vocabularies, environment adapters, and verification rules, while node composition, variant matching, maturity assessment, and memory retrieval rely on fixed thresholds, manually specified weights, and deterministic rules. The framework primarily integrates existing memory and workflow mechanisms into a complex engineering system, without introducing new algorithms for process discovery, abstraction, or composition. The paper also does not adequately justify the necessity of these specific design choices.
  • HPEM organises reusable processes into an ordered DAG and composes and filters nodes according to predefined rules, but does not explicitly learn conditional branches, loops, or state-dependent subprocess selection. The retrieved experiences are ultimately provided as textual guidance to the original agent, leaving the actual contribution of the hierarchical representation to complex task planning unclear.
  • The ablation studies do not sufficiently isolate the contributions of individual design choices. The comparison between Full and Flat does not disentangle the effects of root-level scoring, child-node filtering, and temporary composition. The downstream feedback ablation also does not independently validate the specific reliability formula or maturity thresholds. The paper does not establish that these complex rule-based designs are necessary compared with simpler alternatives.
Questions for authors
  • How does HPEM differ methodologically from existing experience memory and hierarchical skill methods? Could the authors clarify which components constitute the main methodological contributions beyond integrating process abstraction, workflow composition, outcome feedback, and structured retrieval?
  • Why are predefined semantic vocabularies, environment adapters, deterministic composition rules, and manually specified thresholds necessary? Could the authors clarify whether the framework can discover and compose reusable processes with less reliance on hand-designed rules?
  • How does the hierarchical DAG support complex task execution beyond organising and retrieving ordered process components? Can HPEM represent conditional branches, loops, or state-dependent subprocess selection, or does it rely on the host agent to handle these aspects?
Reviewer_A8UUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposed HPEM (Hierarchical Process Experience Memory), a plug-in advisory memory for interactive agents. Try to store experience as a DAG of process nodes: leaf, plus composed workflows that reference shared child nodes. Benchmark-specific deterministic adapters map successful interactions to process observations and compose workflows. At inference level, a hand-specificied weighted matcher can retrieve a stored root or a temporary compostion, rendered as optional guidance for an unchanged host agent.

Lastly, memory was "frozen" before evaluation - Qwen3 VL 8B and Qwen 3.5 27B cover WebShop, ALFWorld, and AndroidWorld, reporting paired gains in success and reduced cost.

Critical strengths
  • Memory is frozen and verified by artifact hashes, and comparisons are paired on the same task instances, and results are reported with paired CIs with correction. So I would say the whole eval protocol is well-set.
  • The method plugs into several different hosts and benchmarks, without changing action interfaces.
  • Great improvements in ReAct (+13.6) and ALFWorld (+15.7 and +17.9), reducing token and steps
  • transferrable skills proven from 8B to 27B models
Critical weaknesses
  • I highly suggest authors improve their writing - the whole storytelling is so vague and I found a bunch of parts very hard to understand or follow. Many dense and seemingly unreasonable symbols. If there could be a concrete, great guidance example, the whole paper should be easier to take. Multiple "protocol-specific Full" conditions are involved, which is hard to follow and match back and forth.
  • stored "experience" mostly is human-authored, so the source of the gains are unclear. The semantic vocabulary and mapping rules are hand-written (1.50-51). The paper lacks a control that gives the host adapter's templates as static guidance without any acquisition. Therefore the gains cannot be attributed to learned experience rather than benchmark-specific prompt engineering.
  • (I am not 100% sure) the central "composition across tasks" claim is somewhat weakly supported. Especially for the held-out work - new instances of seen tasks templates rather than unseen tasks, and AndroidWorld reuses the same 116 task classes. Moreover, the hosts are weak, no other memory method is compared head-to-head in the main results.
  • Many parameters are subjective (according to Appendix A), like weights (0.38/0.16/0.08/0.30/0.08) etc. So they all come from human experience, no scientific reasonable inference.
  • If I understand correctly, we have to re-write adapter for each single benchmark. So the generalizability and extension remain questionable.
Questions for authors
  1. How sensative are the results to the hand-set retrieval weights and maturity thresholds? How to choose?
  2. I don't understand why are the host baselines much lower than published numbers - do the gains persist with stronger backbones or less restrictive host configurations?
  3. Can you evaluate on task templates or families that are absent from acquisition to support the "across tasks" claim?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.