acceptodds
Under review as a conference paper at ICLR 2027

Beyond Context: Executable Memory for Embodied Agents

Abstract

Long-horizon robotic tasks require memory because the same current observation can call for different actions depending on earlier interactions. Existing memory-augmented systems improve how history is stored and retrieved, but often pass it to a learned model whose interpretation determines the next decision—an interface we call Context-Memory. Yet recalling the relevant past does not by itself determine its role in execution: identifying a previously used target and deciding whether to repeat an interaction require different operations on history. Our insight is that these operations can be made explicit, allowing remembered information to update task state rather than merely provide context for inferring it. We introduce Execute-Memory, which couples persistent entity and event memory with compositional task programs that resolve historical references, maintain progress, and determine the next execution requirement. By keeping uncertain world beliefs separate from accepted events, it links progress updates to observed interactions without assuming perfect perception. The resulting requirement is grounded in the current scene and passed as a local subgoal to a frozen vision-language-action policy. Experiments on RoboMME and challenging stress subsets derived from it show substantial gains over memory-augmented and policy baselines, supporting a shift from remembered context to executable memory.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied agents, agent memory, long-horizon task execution, evidence-grounded reasoning, execution monitoring, vision-language-action models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FyqCOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Execute-Memory, which couples persistent entity and event memory with compositional task programs that resolve historical references, maintain progress, and determine the next execution requirement. By keeping uncertain world beliefs separate from accepted events, this work links progress updates to observed interactions without assuming perfect perception.

Critical strengths
  1. The paper rigorously distinguishes Context-Memory from Execute-Memory, grounding this distinction in RoboMME’s empirically validated memory demand categories.
Critical weaknesses
  1. In the method section, the paper introduces a large number of symbols without providing concrete evaluation traces, or demonstrating their use beyond isolated equations. This undermines reproducibility and conceptual clarity.
  2. The core execution logic depends entirely on hand-authored event contracts (Table 6) and task-specific program compositions (Table 9), which are tightly aligned with RoboMME’s predefined task families. This renders the approach non-scalable to novel events or tasks outside this scope and regresses from learned VLA generalization to a rule-based executor.
  3. As the sole source of evidence for all programmatic reasoning, the admission criteria, key construction, and robustness of “Verified Ledger” are neither empirically quantified nor validated, making the entire execution chain critically dependent on an unexamined, potentially brittle component.
Questions for authors

The reviewer thinks that method in this article is not an enhancement of the VLA, but rather a combination of the VLA and a rule engine, whose upper limit is determined by the scope of human prior knowledge. This method lacks open-world adaptability and is also difficult to scale from a methodological standpoint directly. The reviewer is more concerned with how the authors ensure that their method continues to perform well on tasks other than RoboMME.

Reviewer_1Uj7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Execute-Memory uses task-family programs over entity/event memory to resolve historical references and advance progress, then re-grounds subgoals for frozen π0.5 (pp.4–5). It reports 72.4% success across 16 RoboMME tasks, versus 44.7% for the strongest tested memory baseline (p.6).

Critical strengths
  1. Separating reusable history from completion evidence is clear and useful (pp.4–5,19–21).

  2. The main evaluation uses 90 episodes per task with a fixed local policy and interface. In the separate 10-episode-per-task ablation, Structured Memory+Qwen achieves 31.3% versus 72.5% for the full method (Table 3, p.8). Stress tests reuse checkpoints without level-specific fine-tuning.

Critical weaknesses
  1. Both contract-removal ablations fall to 6.3%, but the missing failure analysis makes it hard to identify what these contracts add beyond guarded execution (p.8).

  2. The evaluated providers, thresholds, stride, and mapping from boundary claims to canonical events are insufficiently specified, limiting reproducibility and interpretation of the ablations (pp.15,19–25).

Questions for authors
  1. Please provide the evaluated configuration and example traces linking event admission, persistence, and phase updates. What errors explain the two 6.3% ablation results?

  2. Which rules are shared across task families, and what recipe/guard information does Qwen receive in Table 3? Can existing traces isolate the contracts’ incremental benefit?

Reviewer_jYsuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Execute-Memory couples persistent entity beliefs and an accepted-event ledger with registered task programs. Typed queries resolve historical references and repetition counts. Phase-scoped consumption prevents an old occurrence from repeatedly completing an obligation. The resulting requirement is grounded in the current scene and executed by a frozen pi0.5 policy through GroundSG. Evaluation covers the 16-task RoboMME suite, longer-history variants of four task families, and two qualitative SO101 demonstrations.

Critical strengths
  1. Distinguishing reusable historical evidence from consumable completion evidence addresses a concrete failure mode in repeated manipulation. The system also separates uncertain perception from accepted events and current geometry from historical coordinates. These are useful execution semantics, rather than simply a larger history buffer. The authors appropriately clarify that an accepted event is not guaranteed physical truth and that the compiler supports registered task families.
  2. The evaluation provides substantial evidence for the complete interface. Execute-Memory reaches 72.4% on RoboMME, compared with 44.7% for FrameSamp+Modulator and 33.0% for GroundSG+Qwen (Table 1). More importantly, Structured Memory+Qwen retains the same structured memory, grounding interface, and frozen policy but reaches 31.3%, versus 72.5% for the program-based system (Table 3). The longer-history tests reuse checkpoints without stress-level fine-tuning, supporting robustness within the registered families rather than merely better performance on the original counts.
Critical weaknesses
  1. The contribution is narrower than the broad Context-Memory versus Execute-Memory distinction suggests. Section 2 already acknowledges programmatic planning and guarded execution. Here, template-constrained parsing, task-family measurement providers, and task-dependent realization rules remain central (Appendices B.3 and J). Giving Qwen the same ledger is useful, but does not establish that it receives equivalent procedural knowledge to the registered programs. The comparison therefore combines execution semantics with explicitly supplied task structure. The stress tests change counts or swaps within existing families, not composition coverage. This does not invalidate the system result, but without a matched controller or specification-matched planner, the reusable methodological advance is not sufficiently established for acceptance.
  2. The gate-removal results establish dependence on the contracts, but do not explain their practical benefit under imperfect perception. Both removals reduce success to 6.3% on only ten episodes per task. Bypassing admission or reuse restrictions can make the remaining runtime consume inputs outside its intended assumptions; a catastrophic drop is not itself evidence that the proposed gate is better than a simpler consistent event/counter design. Although Appendices K-L describe diagnostic receipts, the paper does not report quantitative false-admission rates, duplicate completions, abstention coverage, or evidence-to-dispatch delay. Table 4 also leaves observation stride and several provider configurations unspecified. These omissions limit independent assessment of where reliability comes from.
Questions for authors
  1. Which program rules, event thresholds, detector/tracker configurations, and intent adapters are shared across tasks, and which require task-specific design or calibration? Can the authors provide one matched comparison giving a learned planner the same procedural specification, or a simpler guarded controller the same measurements?
  2. For the existing Table 3 runs, what fractions of failures arise from false events, repeated consumption, unresolved grounding, or policy execution? Please report available decision-coverage and delay statistics, and identify the exact observation stride and provider configurations. These records would be more informative than adding another uncontrolled task.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.