acceptodds
Under review as a conference paper at ICLR 2027

ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control

Abstract

A robot may lose sight of an object it must later retrieve, need to recall what a person demonstrated earlier, or track which steps of a task it has already completed. Current vision-language-action (VLA) policies often fail once the information needed for action disappears from the current observation, making memory critical for long-horizon robot behavior. Existing approaches typically provide longer histories or learn implicit memory from observation-action trajectories. But action supervision tells a policy how to act, not what to remember: it does not specify which past facts should persist or how they should change as new evidence arrives. We therefore separate maintaining an evidence-grounded account of the past from learning how to act on it. This insight motivates Explicit Concept Memory (ECoMEM), which represents task-relevant history with a reusable library of grounded concepts. An evidence-based Writer selects and updates these records, while a learned Reader turns them into memory tokens that directly condition the VLA. Across 16 RoboMME tasks, ECoMEM leads the evaluated robot policies on 15 tasks. On two new real-robot tasks, the same memory library either transfers directly or requires only one new concept, achieving 86.1% success versus 8.6% for a no-memory VLA. These results show that explicit concepts provide a reusable and extensible memory interface for robot control.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Robotic Manipulation, Vision-Language-Action Models, Memory-Augmented Models, Neuro-Symbolic Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_5EVQOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This manuscript proposes ECoMEM, which decouples memory construction from memory utilization within VLA policies. The Writer selects and maintains explicit records (entities, states, events, progress, and temporal sequences) from a shared concept library based on visual and robot-state evidence. The Reader encodes this structured memory into tokens to condition the VLA alongside image and language inputs. ECoMEM achieves the highest average success rate across 16 memory-dependent tasks in the RoboMME benchmark.

Critical strengths

This paper wons a clear problem formulation. ECoMEM decouples memory construction from strategy learning and employs explicit, evidence-linked conceptual records to enhance inspectability and task relevance, thereby avoiding reliance on shortcuts associated with purely implicit memory.

ECoMEM achieves state-of-the-art performance across multiple tasks, demonstrating significant gains in real-world robotic tasks involving repetitive counting and progress tracking. Additionally, ablation studies validate the necessity of structured extraction, task-conditioned selection, and language grounding.

Critical weaknesses

Details regarding the concept library, Writer update rules, and Reader training and deployment are insufficient.

The real-world robot experiments primarily compare against the memoryless π0.5, omitting strong memory-based baselines (such as keyframe, VLM, or scene graph approaches) that utilize the same backbone. The inference latency is not reported.

Questions for authors
  1. How many concepts does the current concept library contain? On average, how much human design, annotation, and validation is required to add a new concept?

  2. How are the evidence accumulation threshold, event confirmation, and counting rules configured for Writer? Is it robust against noise, occlusion, false positives, and missed detections?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 5EVQ (1/3)

We thank the reviewer for the positive assessment and the constructive questions. We answer each below and will incorporate the details into the revision.

W1: Details regarding the concept library, Writer update rules, and Reader training and deployment are insufficient.

These are in the supplementary material: the concept library in Appendix C (complete inventory, Table 4); the Writer in D–E (perception, recognition) and F.1–F.3 (evidence merging, log-odds state estimation, event history and progress); the Reader in F.4–F.6 (record interface, architecture, causal training alignment); training and deployment in A and I.3–I.4. We agree they should be more visible: we will move the update rules (F.2–F.3) and the record interface (Table 5) into the main text, and add the parameter values and construction statistics given under Q1 and Q2 below.

W2 (baselines): The real-world experiments compare primarily against the memoryless π0.5, omitting strong memory-based baselines (keyframe, VLM, scene graph) on the same backbone.

We agree this is a limitation of the real-robot study, whose goal is to test library reuse and extension. Memory-based alternatives are compared on RoboMME with the same backbone: keyframe retrieval with a VLM (MemER) and visual memory banks (SAM2Act+, FrameSamp+Modul). To add the closest analogue of a textual world-state or scene-graph memory, we ran a new comparison in which the same Writer records are rendered as text and appended to the instruction, with the Writer, data, training recipe, and evaluation protocol unchanged:

TaskRecord tokens (ours)Textual records in instruction
VideoUnmask84.073.3
PatternLock60.063.3
PickXtimes62.030.7

Text is comparable on PatternLock, where memory is fixed once observed, and worse where memory changes during execution. It also costs ≈21 tokens per record instead of 1 and is 27% slower per inference (96.2 vs. 75.7 ms). We will add this comparison and state the scope of the real-robot study explicitly.

W2 (latency): The inference latency is not reported.

Thank you for pointing this out; we now report both components.

  • Policy. 75.7 ms per 20-step action chunk on one L40S. Memory adds 5.1 ms (+7%) over the no-memory policy (70.6 ms).
  • Writer on RoboMME. Per control cycle (16 actions, 0.8 s of motion), median over cycles and then summarized over the 16 tasks: mean 0.51 s, median 0.34 s. Policy inference 79 / 78 ms.
  • Writer on the real robot. Memory is updated every 6 control steps (ScoopPour) or 3 (CupSwitch) at a nominal 30 Hz. In an offline replay of the six held-out ScoopPour demonstrations, the frozen recognizer processed 1,346 updates in 45 s on CPU, ≈33 ms per update, within the 200 ms budget.
  • Initialization. One-time, median 2.23 s (VideoUnmask) and 4.38 s (VideoRepick) on RTX 3090 (Appendix G.2).
AuthorsOpenReview ↗

Response to Reviewer 5EVQ (2/3)

Q1: How many concepts does the library contain? On average, how much human design, annotation, and validation is required to add a new concept?

The base library contains 20 executable concepts, each a recognizer with an explicit update rule. They cover all 16 RoboMME tasks and emit the 48 predicate, event, and role codes of Table 4. The real-robot tasks add one concept.

Base libraryCupSwitchScoopPour
New concepts2001 (event.scoop)
Coding agentsClaude Opus 5, GPT-6-astra–GPT-6-astra (Codex)
Author effort≈42 person-hours, 6 working days–supervision and feedback
Revisions5 library versions–5 recognizer versions
Development demonstrations–26
Validation–6 held-out demos33 logic tests; 18 evidence-removal replays (all 0 counts); 15/15 held-out cycles agree with visual review

For the new concept, design consisted of identifying the missing distinction (a scoop that actually carries beans) and reusing grounding, persistence, counting, and goal tracking unchanged. Annotation required no dense manual labels: development used agent-assisted visual review of the 6 demonstrations with author feedback. Validation is the last row, run after freezing the recognizer. We will add this table to the paper.

AuthorsOpenReview ↗

Response to Reviewer 5EVQ (3/3)

Q2: How are the evidence accumulation threshold, event confirmation, and counting rules configured? Is the Writer robust against noise, occlusion, false positives, and missed detections?

Configuration. The Writer accumulates signed evidence in log-odds with exponential decay (Eq. 8). One setting serves all predicates; the two slowly varying states use a slower one:

ParameterAll predicatesactivated, lifted
Confirmation threshold (log-odds)4.01.2
Retention threshold (log-odds)1.50.5
Retraction threshold (log-odds)−4.0−4.0
Decay half-life2 s60 s
Log-odds bound88
Per-frame evidence cap0.1 s0.1 s

Robustness rules.

  • Occlusion and missed detections. A missed detection is never evidence against a state. A saturated state (log-odds 8) that receives no evidence stays confirmed for ≈4.8 s (2 s × log₂(8/1.5)), then reverts to unknown, never to false. An occluded object keeps its last supported location (Section 4.2, Appendix D.2).
  • Noise and false positives. A single frame contributes at most 0.1 s × its channel weight, so confirmation needs several consistent frames; lifted, for example, needs at least three to reach 4.0. Attachment, holding, and lifting additionally require independent physical evidence such as tracked object motion, not gripper closure alone.
  • Repeated detections. An event is recorded once, when first confirmed; later matching evidence updates that record (Appendix F.3). A count is the number of distinct recorded events, so repeated detections cannot inflate it.

Empirical evidence. Offline, on 95 held-out videos, the Writer reaches 95–100% on entity, state, and event concepts and 88.2–100% on temporal concepts (Table 2); replacing instruction-specific entity phrases with generic queries drops target-location agreement from 98% to 20% (Appendix G.3), so the Writer follows the evidence rather than a task prior. On the real robot, under camera noise, lighting changes, and occlusion by arm and tool, ECoMEM succeeds in 86.1% of 79 trials. Two recoveries arose without any retry rule: when a scoop spilled, the wrist camera showed an empty scoop, the count did not advance, and the policy scooped again; when a pour was incomplete, no cycle was confirmed and the policy poured again (Figure 7). For the new concept, all 18 evidence-removal replays give zero counts.

We have not run a dedicated noise-injection study that drops or corrupts detections at controlled rates. We will add one to the revision, together with the parameter table above.

Thank you again for the valuable feedback. We are happy to answer further questions during the discussion period.

Thank you for the clarification. I maintain my previous assessment of the paper.

Reviewer_kUKnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes Explicit Concept Memory (ECoMEM) for robotic tasks in which the current observation alone is insufficient to determine the correct action. ECoMEM separates memory construction from memory utilization: a fixed Writer uses pretrained vision models, robot states, and explicit update rules to maintain structured records of entities, spatial relations, events, progress, and temporal order, while a learnable Reader encodes these records into tokens that are provided to the π0.5\pi_{0.5} policy together with image and language inputs. The paper reports strong empirical results. On the 16 RoboMME tasks, ECoMEM achieves an average success rate of 82.42%, outperforming the strongest baseline reported in the table by 37.91 percentage points and leading on 15 of the 16 tasks.

Critical strengths
  1. The paper structurally decomposes the information required by memory-dependent tasks into fine-grained concepts and maintains memory through a reusable concept library.

  2. The experimental results are strong, with substantial performance improvements over the baselines on RoboMME.

Critical weaknesses
  1. Incremental novelty. The main contribution appears to be an integration of existing components—symbolic predicates, object grounding, event counting, and memory-token conditioning—rather than a substantially new memory-learning principle. The paper does not clearly isolate which capability is uniquely enabled by ECoMEM compared with prior structured or grounded memory systems.
  2. Under-specified concept-library construction. The paper does not identify the coding agent or specify how candidate concepts are generated, selected, and validated; the reported demonstrations and logic tests provide only limited evidence for the reliability of newly constructed concepts. The origin of the initial concepts and the extent of human intervention also remain unclear.
  3. Unclear extensibility of the Reader. Although the paper introduces a new SCOOP concept while using a fixed 11-field record interface and embedding tables, it does not explain how new predicate values are added, initialized, or learned by the Reader. The experiments therefore demonstrate an offline, task-specific extension rather than general plug-and-play concept expansion.
  4. Limited real-robot validation. The real-robot evaluation covers only two closely related repetitive-cycle tasks on a single robot platform, with one task reusing existing concepts and the other adding only one local concept. This limited setting does not establish broad transfer across substantially different tasks, objects, sensors, or robot platforms.
Questions for authors

see weakness above

3 comments
AuthorsOpenReview ↗

Response to Reviewer kUKn (1/2)

We thank the reviewer for the review. We address each weakness below. Since the reviewer found the paper difficult to follow, we would be grateful to know which sections were hardest, so that we can revise them specifically.

W1: Incremental novelty. The contribution appears to be an integration of existing components rather than a new memory-learning principle, and the paper does not isolate which capability is uniquely enabled by ECoMEM compared with prior structured or grounded memory systems.

We should state the contribution more sharply, and will do so in the revision. The contribution is not a new way to learn memory. It is the separation of what to remember, maintained by a fixed, evidence-based Writer over a shared concept library, from how to use it, learned by a Reader that feeds records directly to a low-level VLA. Three capabilities are isolated experimentally against the prior alternatives:

  • Against VLM-based memory: reliable temporal composition. A fine-tuned VLM extracting the same concepts from video reaches 5.9–11.8% on ordered object–destination sequences; the Writer reaches 100% (Section 5.4, Table 2).

  • Against textual world-state memory: acting on memory that changes during execution. New in this rebuttal, we feed the same Writer records through the interfaces prior structured memories use, text in the instruction and a counter, with data, initialization, and training recipe unchanged:

    TaskRecord tokens (ours)Textual recordsCounter
    VideoUnmask84.073.3–
    PatternLock60.063.3–
    PickXtimes62.030.726.0

    The counter supplies exactly the count needed to stop, yet reaches 26%. Record tokens are also ≈21× more compact and 27% faster per inference (75.7 vs. 96.2 ms).

  • Against task-specific memory: transfer by composition. New real-robot tasks need 0 (CupSwitch) or 1 (ScoopPour) new concept, and removing any one concept family drops success from 60–84% to 12–18% (Section 5.3).

AuthorsOpenReview ↗

Response to Reviewer kUKn (2/2)

W2: Under-specified concept-library construction: the coding agent, how candidates are generated, selected, and validated, the origin of the initial concepts, and the extent of human intervention.

We agree these should be stated, and will add them.

  • Agents and human intervention. The 20-concept base library was built with Claude Opus 5 and GPT-6-astra under author supervision, in ≈42 person-hours over 6 working days, through 5 library versions, 8 perception-layer versions, and 39 task-program rewrites. ScoopPour's single new concept was built with GPT-6-astra (Codex) on 6 development demonstrations through 5 recognizer versions, with authors providing supervision and feedback.
  • Generation and selection. Candidate concepts are proposed by the coding agents and reviewed by the authors. A candidate is kept only if it can change the correct action when the instruction and current observation are identical (Eq. 1, Section 4.1), and only if existing concepts cannot already express the distinction.
  • Validation. Logic tests, evidence-removal replays, and a held-out check on the frozen recognizer (Appendix I.2). For ScoopPour: 33 logic tests, 18 replays all giving 0 counts, and 15/15 held-out cycles agreeing with visual review.

W3: Unclear extensibility of the Reader: how new predicate values are added, initialized, or learned. The experiments demonstrate an offline, task-specific extension rather than plug-and-play expansion.

The reviewer is right that this mechanism is not described; we will add it to the appendix. The Reader's predicate embedding table reserves 64 rows, of which the base library uses 48. A new predicate value is appended as a new id; existing ids and the 11-field interface are unchanged. Reserved rows are randomly initialized with the table, receive no gradient until used, and are learned during fine-tuning together with the rest of the Reader (Appendix F.5). Four groups were added this way during RoboMME development (manner:*, route:*, side:*, lifted); ScoopPour appended 7.

We agree that this is not plug-and-play expansion without training, and we do not claim it. The claim (Section 5.5) is that the library transfers through reuse and local extension; we will make this distinction explicit.

W4: Limited real-robot validation: two closely related tasks on a single platform.

We agree, and will state the scope explicitly. The real-robot study answers one question: can the concept library transfer to new tasks through reuse and extension? The two tasks test the two forms: CupSwitch reuses the library unchanged, and ScoopPour adds one concept that changes what counts as progress, so that only scoops carrying beans advance the count. The paper does not claim transfer across platforms or sensors.

We hope these clarifications, together with the new same-evidence comparison, resolve the concerns, and we welcome further questions.

We are grateful for the reviewer’s careful reading and comments. Our responses above include new experiments and clarifications on each point. We hope these are helpful in addressing all concerns, and we remain available for any further questions during the discussion period.

Reviewer_9RGBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a VLA with explicit concept memory that separates memory construction from action generation. The benchmark improvements are substantial, and the real-world reuse and retry behavior are promising. My main concerns are the engineering effort required, the attribution of the gains, and the scope of generalization.

Critical strengths
  • The empirical improvements are substantial and span diverse memory-dependent tasks. ECoMEM achieves 82.42% average success on RoboMME and leads the evaluated policies on 15 of 16 tasks. The real-world experiments further demonstrate library reuse and recovery from unsuccessful attempts. Although stronger controlled comparisons are needed.
Critical weaknesses
  • The contribution of the proposed memory architecture is not sufficiently isolated. The system combines engineered recognizers, update rules, and structured progress information, but lacks comparisons using the same evidence with simpler counters, finite-state machines, or textual records. The real-world comparison uses only a memory-free policy. Consequently, the large gains do not establish whether the proposed concept-memory architecture and Reader are necessary.

  • The engineering requirements and scope of generalization remain insufficiently established. The effort and task-specific knowledge required to construct the concept library are not quantified, and the real-world policies are fine-tuned on task demonstrations. These results support library reuse, but provide limited evidence for generalization to unseen concept combinations or for inexpensive extension.

Questions for authors
  • How much task-specific engineering was required to build and extend the concept library? Please quantify the concepts, thresholds, development demonstrations, and manual revisions, and clarify which Reader and policy parameters were reused or retrained for the real-world tasks.

  • Are there controlled comparisons using the same recognizers and evidence with simpler counters, finite-state machines, or textual memory? Such results would materially affect whether I attribute the gains to the proposed memory architecture rather than to the structured information supplied.

  • Please provide Appendices as review materials. The main text frequently refers to these appendices, which are needed to assess recognizer rules, training settings, and evaluation procedures.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 9RGB (1/3)

We thank the reviewer for the thorough review, and especially for stating which experiment would change their assessment. We ran that experiment; it is reported under W1 & Q2.

Q3: Please provide the Appendices as review materials.

We apologize for the inconvenience. The appendices (A–I) were submitted as part of the supplementary material at the first submission, pages 15–24 of the supplementary PDF. The three items the reviewer lists are found here:

Requested detailAppendix
Recognizer rulesC (complete concept inventory, Table 4); D (language-grounded perception, tracking); E (concept selection, multimodal recognition, evidence composition, Fig. 8); F.1–F.3 (evidence merging, log-odds state estimation, event history and progress)
Training settingsA (shared configuration, main training, controlled comparisons); I.3 (real-robot training); F.4–F.6 (record interface, Reader architecture, causal training alignment)
Evaluation proceduresB (baselines and protocol); G (interventions with matched controls, Tables 6–7); H (VLM extraction protocol); I.4–I.5 (real-robot deployment and outcome annotation, Tables 8–9)
AuthorsOpenReview ↗

Response to Reviewer 9RGB (2/3)

W1 & Q2: The contribution of the memory architecture is not isolated. Comparisons using the same evidence with simpler counters, finite-state machines, or textual records are missing; the real-robot comparison uses only a memory-free policy.

We agree this is the key question, and we thank the reviewer for specifying the experiment. We ran it.

Setup. We keep the Writer, its records, the training data, the initialization (pretrained π0.5), the training recipe, and the evaluation protocol fixed, and change only how the records reach the policy:

  • Record tokens (ours): the Reader embeds the 11 record fields, one token per record.
  • Textual records: the same active records are rendered as text and appended to the instruction, with no Reader. Example: "Belief: inside, object 2, with object 1, true, confirmed, confidence 9/9, age 6."
  • Counter (PickXtimes): the instruction receives only the progress state, i.e. the output a hand-written finite-state machine would produce: "Progress: 2 cycles completed, 0 milestones completed."

Table 1. Success rate (%) with identical Writer records and different interfaces.

TaskRecord tokens (ours)Textual recordsCounter
VideoUnmask84.073.3–
PatternLock60.063.3–
PickXtimes62.030.726.0

Textual records: mean over three policy-sampling seeds, 50 validation episodes each. Record tokens: the matched full-memory controls of Figure 4.

Table 2. Cost of the same memory under each interface (PickXtimes memory, 18 records; one L40S; median of 50 calls).

InterfaceTokens per recordmemory, max tokensLatency per 20-step chunk
No memory––70.6 ms
Record tokens (ours)111275.7 ms (+5.1)
Textual records≈212,51196.2 ms (+25.6)
Counter––70.6 ms

What this isolates.

  • The structured information is not sufficient. The counter contains exactly the fact needed to decide when to stop, and nothing else, yet success collapses to 26%. The same information delivered through the instruction channel does not work; the gain is therefore not explained by the supplied information alone.
  • The interface matters most when memory changes during execution. On PickXtimes the policy must stop after the k-th placement. Through the instruction, each progress update edits one token inside a long, otherwise unchanged prompt; a Writer event instead adds a separate record token. On PatternLock the memory is fixed once the pattern is observed and acts as an extended instruction, and there text is comparable (63.3 vs. 60.0, within seed spread).
  • The information is still necessary. Removing one concept family from otherwise identical records drops success from 84→18%, 62→14%, and 60→12% (Section 5.3); a fine-tuned VLM extracting the same concepts from video reaches 5.9–11.8% on ordered sequences versus 100% for the Writer (Section 5.4). The Writer determines whether the right facts exist; the record-token Reader determines whether the policy can act on them.
  • Why not a hand-written state machine per task. It would replace one shared library with 16 task-specific programs and output formats. The library covers all 16 tasks with 20 concepts and transfers to a new real-robot task with zero or one new concept.

Real-robot comparison. We agree that the real-robot study compares only against a memory-free policy. Its purpose is to test whether the library transfers through reuse (CupSwitch) and extension (ScoopPour), not to compare memory designs; those comparisons are made on RoboMME against MemER, SAM2Act+, FrameSamp+Modul, and now the interfaces above. We will state this scope explicitly in the revision and add Tables 1–2.

AuthorsOpenReview ↗

Response to Reviewer 9RGB (3/3)

W2 & Q1: How much task-specific engineering was required to build and extend the concept library? Quantify concepts, thresholds, development demonstrations, and manual revisions; clarify which Reader and policy parameters were reused or retrained for the real-world tasks.

We thank the reviewer for asking for this; the numbers were not in the submission and should have been.

Table 3. Construction and extension of the concept library.

Base library (16 RoboMME tasks)CupSwitchScoopPour
New executable concepts2001 (event.scoop)
Coding agentsClaude Opus 5, GPT-6-astra– (reuse only)GPT-6-astra (Codex)
Author effort≈42 person-hours over 6 working days–supervision and feedback
Revisions5 library versions; 8 perception-layer versions; 39 task-program rewrites–5 recognizer versions
Development demonstrations–26
Held-out check after freezing–6 demos6 demos; 15/15 cycles agree with visual review
Tests––33 logic tests; 18 evidence-removal replays, all 0 counts
New Reader predicate ids4807 appended

Thresholds. Every predicate uses one evidence-filter setting; only the two slowly varying states (activated, lifted) use a slower one. Confirmation 4.0 / 1.2 log-odds, retention 1.5 / 0.5, retraction −4.0, decay half-life 2 s / 60 s, log-odds bound 8, per-frame evidence cap 0.1 s (Eq. 8, Appendix F.2).

Reused on the real robot. The concept library, the Writer and its update rules, the frozen perception models (OWLv2, SAM2), and the 11-field record interface.

Retrained. Both real-robot policies, ECoMEM and the no-memory baseline, are fully fine-tuned from the same pretrained π0.5 weights on the same demonstrations (Appendix I.3); the Reader is trained jointly. ScoopPour appends 7 predicate ids to reserved rows of the Reader's 64-entry table; existing ids are unchanged, and the new rows are learned during fine-tuning. The transfer claim in Section 5.5 therefore concerns the memory library and Writer, not a frozen policy.

Generalization. The one-cycle ScoopPour target does not appear in the fine-tuning data (Appendix I.1); on it ECoMEM succeeds in 16/20 trials versus 4/12 without memory. We agree the study gives limited evidence for unseen concept combinations. We will add Table 3, make the reused/retrained split explicit, and narrow the wording of the transfer claim.

We hope the same-evidence comparison and the quantified construction cost address the concerns about attribution and engineering effort. We are happy to answer further questions.

We thank the reviewer again for the time and the constructive discussion. In our responses, we have added the requested experiments and clarified the points raised. We hope these have addressed all concerns. If anything remains unclear, we would be very happy to provide further clarification.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading. They recognized a clear problem formulation [9RGB, 5EVQ], substantial gains across diverse memory-dependent tasks [9RGB, kUKn, 5EVQ], a reusable library of fine-grained concepts [kUKn, 5EVQ], ablations validating structured extraction, selection, and grounding [5EVQ], and real-robot reuse and recovery from unsuccessful attempts [9RGB, 5EVQ].

Two main concerns were shared across reviews:

  • Whether the gains come from the proposed memory interface or merely from the structured information it carries [9RGB, kUKn, 5EVQ]. We ran the controlled experiment Reviewer 9RGB requested: the same Writer records are fed to the policy through simpler interfaces (textual records, a counter). These are substantially worse where memory changes during execution and costlier in tokens and latency, which isolates the contribution of the interface from that of the information.
  • The engineering cost of building and extending the concept library [9RGB, kUKn, 5EVQ]. We now quantify it: concepts, coding agents, person-hours, revisions, demonstrations, and validation tests, for the base library and for each real-robot extension.

We also clarify the Reader extension mechanism [kUKn, 9RGB], the scope of the real-robot study, which tests transfer rather than compares memory designs [9RGB, kUKn, 5EVQ], the Writer parameters and robustness rules, inference latency, and the location of the appendices in the supplementary material [5EVQ, 9RGB].

Details, tables, and new results are in the individual responses below. All will be incorporated in the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.