acceptodds
Under review as a conference paper at ICLR 2027

Causal Experience Instantiation: Grounding Procedural Memory with Temporal Graphs for Tool-Use Agents

13 views

Abstract

Tool-use agents need to reuse past procedures without treating task-specific values from those executions as facts about a new task. Retrieving a successful trajectory can import stale identifiers, while abstracting it into a workflow leaves open when and where its arguments should be grounded. We introduce Causal Experience Instantiation, which couples a Static Procedural Bank of abstract workflow graphs with an online Dynamic Fact Ledger. The bank retains action and argument dependencies without episode-specific values. As the agent acts, the ledger adds typed facts from the current task to a temporal graph, recording when each value became available and which observation supplied it. At each decision, a Presence Head estimates whether evidence for a required argument is available; a Source Ranking Head selects among compatible observed sources when it is. Otherwise, the readout leaves the argument unresolved rather than supplying an unsupported value. On AppWorld, the method completes 82/168 tasks versus ReMe’s 73/168, while reducing the invalid-call rate from 1.10% to 0.21%. Relative to ExpeL, gains also appear on ALFWorld, a stratified ScienceWorld panel, and all five evaluated AppWorld language-model backbones. Component ablations show lower AppWorld success when either memory path or the evidence-checking readout is removed.

Primary area:
applications to robotics, autonomy, planning
Keywords:
LLM agents, tool use, procedural memory, temporal graphs, evidence grounding, experience retrieval

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_quALOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets two failure modes of experience-reusing tool-use agents: (i) copying stale, episode-specific literals (IDs, phone numbers) from retrieved demonstrations into live API calls, and (ii) issuing state-changing calls before the required argument has been observed in the current episode. The method splits memory into a Static Procedural Bank of abstract workflow graphs (literals replaced by typed slots) and an online Dynamic Fact Ledger that records typed values from the current task with timestamp and provenance, encoded as a temporal graph. A trained dual-head readout then, per slot, estimates whether supporting evidence is present (Presence Head, threshold 0.75), emitting an UNRESOLVED "sense first" directive if not, and otherwise ranks type-compatible ledger candidates (Source Ranking Head); the result is injected into the frozen LLM's prompt. On AppWorld test-normal (168 tasks) the method solves 82 tasks vs. 73 for ReMe and 70 for ExpeL, with invalid-call rate 0.21% vs. 1.10%/0.72%; it also reports gains on ALFWorld, a 90-episode ScienceWorld panel and five AppWorld backbones (vs. ExpeL), plus ablations, mechanism-level error counts (e.g., old-value copies 14 → 2) and a threshold sweep.

Critical strengths
  1. The paper identifies a concrete and practically relevant gap in experience reuse for tool-use agents, and states it precisely. The prefix-causal grounding criterion (§3.2) turns "reuse the procedure, not the values" into a checkable requirement on each dynamic argument, and the split into missing evidence (Presence Head) and choice among observed candidates (Source Ranking Head) is a clean decomposition of the problem. This framing is useful independently of the specific architecture, and it is the main reason I see potential in the paper.
Critical weaknesses
  1. The paper does not show that its gains come from the mechanism it proposes. The evidence for the motivating failures is thin: all examples are schematic (Figs. 1–2, Appendices C and H), and over 168 tasks ExpeL makes only 14 old-value copies and 18 state changes without evidence. These errors also cannot account for the readout's gain: from the matched static + dynamic control to the full method, success rises by 14 tasks while the two error types fall by 4 and 6, at most 10 tasks. Simpler remedies are not compared (Question 3): the control without any directive already cuts copies from 14 to 6, and neither a non-learned readout nor a rule-based pre-dispatch check is evaluated. The central claim, that the learned readout is what makes grounding work, is therefore unsupported.

  2. The gains may be within run-to-run variation. The direction is consistent across environments and backbones, but each condition is a single run, and in the paper's own two-attempt protocol (Table 16) ExpeL's outcome changes on 34 of 168 tasks, more than the method's 12-task gain over ExpeL and 9-task gain over ReMe (Question 4). Whether the gains hold with the official AppWorld agent and faithfully reproduced baselines is also unclear (Question 6).

Questions for authors
  1. Is the directive in Listings 2–3 the prompt used in the reported experiments? Listing 2 shows a fully specified lookup (apis.venmo.search_transactions(query="Mark")), and Listing 3 fills {sensing_api} and {search_key}. I could not find this template in the supplementary code. For an unresolved slot, experience_instantiation_runtime.py (around l. 414–429) lists only the GET routes from which the value was read in the historical trajectory; these routes come from a fixed dataflow rule in experience_instantiation_data.py (around l. 303–311), not from the readout, and the emitted text states "No query parameters are inferred". Which code path and prompt variant produced the main tables?
  2. Can you show recorded (not schematic) traces of stale-value copying and premature state changes by the evaluated backbones? For tasks solved by the full method but not by ExpeL, what caused ExpeL's failures?
  3. What success rate does a non-learned readout achieve with the same prompt and directive? How does ExpeL perform with a rule-based pre-dispatch check that blocks calls whose ID-typed arguments do not appear in the current goal or observations? What exactly does the "Static + dynamic, no final readout" arm place in the prompt? The base prompt (Listing 1, rule 4) already tells the agent not to copy IDs and to query unobserved entities first; do all arms use it, and what fraction of UNRESOLVED directives is followed by a lookup before the state-changing call?
  4. Can you report several independent runs with confidence intervals? Is attempt 2 in Table 16 independent of attempt 1?
  5. How were "confirmed old-value copies" and "state changes without evidence" identified: automatically or manually?
  6. Is the AppWorld agent the official ReAct implementation, and why was a 40-step horizon chosen when the official agent allows 100 LLM calls? The no-memory agent solves 28.6% of test-normal, below the 48.8% TGC the AppWorld paper reports for GPT-4o; do the gains persist with the official setup? Relatedly, ExpGraph gains +11.3 pp (AppWorld) and +5.2 pp (ALFWorld) over no-memory here, versus roughly +23 to +27 pp and +23 to +40 pp in its own paper (arXiv 2605.30712, listed there as "ExpHarness"). Was its full system, including the trained retrieval copilot, implemented?
  7. By which criterion, and on which held-out set, was τ_pres = 0.75 chosen (§4.4)? In the test-set sweep (Table 11), 0.75 coincides with the best success count; please confirm that the selection did not use test outcomes.

Suggestions: Figures 1–2 are very dense and would benefit from simplification, and the "causal" terminology should either be justified or dropped, since the paper itself states that its graph edges carry no causal meaning.

Reviewer_JA4LOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Causal Experience Instantiation, a framework for grounding procedural memory in tool-use agents. It separates reusable workflows from task-specific values. A temporal graph encoder supports two readout heads: the Presence Head determines whether a required argument has current-task evidence, while the Source Ranking Head selects among observed candidates. Missing evidence triggers a directive to gather additional information. Experiments cover AppWorld, ALFWorld, and a 90-episode ScienceWorld panel, with five language-model backbones evaluated on AppWorld. The method achieves 82/168 AppWorld successes compared with 73/168 for ReMe, alongside fewer invalid calls. Component ablations further support the usefulness of the complete framework.

Critical strengths
  1. Separating reusable workflows from current-task values addresses a practical failure mode in tool-use agents. The two-head design provides a sensible way to distinguish missing evidence from ambiguity among available sources. This is a useful extension of procedural memory, particularly for tasks requiring precise entity binding.

  2. The evaluation covers multiple environments and language-model backbones, with consistent improvements across the settings. The component ablations are also informative: the full method solves 82 AppWorld tasks, compared with 68 without the final readout, 74 without Presence, and 76 without Source Ranking (Table 5). Similar trends appear on the other benchmarks.

Critical weaknesses
  1. Several central training and execution steps remain unclear. Section 4.4 does not explain how supported source sets are constructed or how supporting evidence is masked across the text history, ledger, and graph. Algorithm 1 also assumes that the next subgoal and required slots have been selected, while Listing 3 provides a sensing API and search key without specifying how they are obtained. These details are needed to reproduce the grounding pipeline and understand what supervision and action guidance the system receives.

  2. The contribution of temporal graph encoding is not clearly established. Tables 5 and 13 show that the final readout improves performance. However, removing the readout removes the grounding functionality, while “ExpeL + same-scale TGN” uses a different memory representation. Neither comparison isolates the graph encoder. A control using the same workflows, ledger, supervision, and readout with a non-graph encoder would clarify how much the temporal graph contributes beyond explicit evidence tracking and source selection.

Questions for authors
  1. How are supported source labels constructed, including cases with multiple valid sources? When creating negative prefixes, is supporting evidence removed consistently from the text history, ledger, and graph?

  2. How are the next subgoal, required slots, sensing API, and search key selected? For AppWorld Python blocks containing multiple API calls, how are the readout’s bindings aligned with the actual calls?

  3. How does the method perform when only the temporal graph encoder is replaced with a simpler non-graph encoder? Please also specify the decision and selection rules used when presence or source ranking is removed.

Reviewer_vhVXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets the two failure modes of experience reuse for tool-use agents: copying episode-specific literals from retrieved demonstrations into a new task and issuing stage-changing calls before the prerequisite evidence has been observed. The paper proposed "Causal Experience Instantiation" couples a "Static Procedural Bank", which retains actions and argument dependencies without episode-specific values, and "Dynamic Fact Ledger" which adds typed facts from the current task to a temporal graph recording when each value became available and which observation supplied it. At each decision, a 2-layer MLP "Presence Head " estimates whether a required slot has current-task evidence. If a "Source Ranking Head" picks among type-compatible ledger entries. If not the readout leaves the argument unresolved and asks the LLM to gather evidence. It's validated on AppWorld, the method completes 82/168 tasks versus ReMe’s 73/168, while reducing the invalid-call rate from 1.10% to 0.21%. Besides AppWorld, ALFWorld and ScienceWorld are also evaluated

Critical strengths

1, the paper has a clear definition of the failure mode for experience reuse tool-use agents and builds methods to address it. The method separates procedure from value, which makes it more than another memory-retrieval variant. 2, broad evaluation was conducted on multiple backbones and external baselines showed consistent result across backbones and environments

Critical weaknesses

1, the evidence is more directional and less attributive, for example, the full method 82 success@1 vs static + dynamic, no final readout 68 success@1, more detailed study about the per-task link will have more clarity for the win. 2, not critical but still worth noting, the baseline coverage is uneven across different tables

Questions for authors

1, can you have a detailed ablation for the AppWorld tasks that were solved by the full method but not the control? e.g. how many involved an unresolved or source ranking? And any other difference? 2, for the five backbones study, was one readout(gpt-5.6-luna traces) reused, or retrained per backbone? 3, Did all baselines receive the same system prompt?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.