acceptodds
Under review as a conference paper at ICLR 2027

DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policies

Abstract

Robotic manipulation is inherently history-dependent, yet most pretrained robotic policies condition on only the current observation or a short temporal window. Equipping such policies with long-term memory remains challenging: existing approaches either feed the backbone multi-frame observation windows, which substantially increase inference cost, or rely on pre-defined semantic features, which limit task generality and may also require the retraining of the backbone to adapt to the memory. We introduce DRAM (Delta-rule Recurrent Associative Memory), a plug-and-play memory module that can be attached to a wide range of pretrained robotic policies, endowing them with long-horizon memory without architectural modification or backbone retraining, requiring only task-specific post-training of the memory module and action expert. DRAM maintains a fixed-size associative memory using gated delta-rule linear attention, with a modified update that incorporates all tokens within each frame in parallel. An architecture-agnostic readout integrates historical context into action prediction across different policy architectures. Experiments show that DRAM consistently improves frozen pretrained policies over short-context baselines and alternative compact memory designs, validating its effectiveness as a fixed-size, post-hoc memory module trained with the backbone frozen.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Embodied AI, vision language action models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_9geaOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DRAM, a fixed-size recurrent memory module for pretrained robot policies with frozen backbones. It combines frame-wise joint delta-rule updates, patch-level historical retrieval, and flexible policy integration. LIBERO experiments across two backbones, integration and memory-reset tests, and real-robot evaluations demonstrate improved success, particularly on longer manipulation tasks.

Critical strengths

Practical integration with pretrained policies. DRAM adds persistent, fixed-size memory while keeping the vision/VLM backbone frozen. Improvements across both DiT and π0.5 support its usefulness beyond a single policy architecture.

Evidence of benefit on multi-stage manipulation. The real-world full-task success rate improves from 0% to 85% over 20 trials per setting, alongside gains on LIBERO-10. These results support the practical value of historical context for extended manipulation.

Critical weaknesses

The central memory design is not isolated experimentally. Comparisons mainly establish gains over no memory; they do not isolate joint delta-rule writing, read-before-write, or patch-level access against matched alternative memory modules. This leaves unclear whether the proposed mechanism offers benefits beyond adding recurrent capacity.

The efficiency claim lacks empirical support. Fixed-size state bounds storage with episode length, but each update requires a per-head, per-layer matrix solve. Without measured latency, memory usage, and control throughput, the practical advantage over other history-conditioning approaches remains unsubstantiated.

Questions for authors

Clarification: Under matched backbone, training, and parameter budgets, how does DRAM compare with simpler recurrent memory and sequential delta-rule updates? What evidence attributes the gains specifically to frame-synchronous joint writing rather than additional capacity?

Clarification: What are DRAM’s measured inference latency, memory overhead, and control frequency relative to the backbone alone and alternative memory modules, including the per-layer Cholesky solves?

3 comments
AuthorsOpenReview ↗

Response to Reviewer 9gea (Weakness 1 and Question 1)

W1 & Q1. Isolated comparisons of joint delta-rule writing , read-before-write, and patch-level access against matched alternative memory modules. What evidence attributes the gains specifically to frame-synchronous joint writing rather than additional capacity?
A1. Thank you for your question. We have added comparisons of DRAM with simpler recurrent memory mechanisms (Pooled Gated Recurrent Unit(GRU) Memory and Recurrent Memory Transformer(RMT)), DRAM sequential delta-rule writing (Sequential Write), and an adapter without history (Stateless Adapter) on RMBench under the same setting to better isolate the contributions of DRAM’s design choices.

  1. Stateless adapter, an additional adapter projects current VLM key/value sources layer-wise to the action expert without history state, adding 256 extra tokens to the action expert using the current state VLM latent embedding only.
  2. Pooled GRU separately pools the final-layer native K/V features, updates a 512-dimensional hidden state, and projects the previous hidden state into one prefix token at each action-expert layer.
  3. RMT, maintains a fixed-size set of persistent memory tokens as its recurrent state and updates them with Transformer blocks as new observations arrive. Our external RMT baseline operates on frozen VLM features and maintains 18 independent memory banks, aligned with the VLM’s 18 integration layers. Each bank contains 256 memory tokens of dimension 512.
  4. DRAM (write first) , swap the read and write operation from default DRAM, the delta rule update is happend before the memory query tokens read out memory state first.
  5. DRAM (seq. write) replaces DRAM's joint update with an ordered token-by-token delta recurrence; Stateless Adapter projects current-frame VLM K/V features into additional conditioning tokens without carrying information between frames.

Each variant is evaluated over 100 episodes per task across nine tasks, totaling 900 episodes. The group success rate averages follow RMBench's M(1) and M(n) partition.

MethodOverall %M(1) %M(n) %
Stateless Adapter14.0016.4011.00
Pooled GRU25.3324.0027.00
RMT32.423.2044.00
DRAM (seq. write)55.1156.8053.00
DRAM (write first)58.2257.659
DRAM59.1152.8067.00

The results show that DRAM outperforms other variants in most of the situations, directly demonstrates the advantages of DRAM. Specifically, DRAM exceeds the tested stateless adapter and pooled GRU by 45.11 and 33.78 percentage points, respectively. Against DRAM (seq. write), it improves overall success by 4.00 points. The difference is concentrated in M(n), which improves from 53.00% to 67.00%. DRAM (seq. write) performs better on M(1), at 56.80% versus 52.80%; we report this task-group dependence explicitly.

We further emphasize that for comparison of DRAM (seq. write) and DRAM, the setup is aligned to frame-level decay with no further human bias. The sequential and joint variants share the memory architecture, parameter count, state dimensions, readout interface, and frame-level retention parameterization. Both apply retention once before writing the frame. While the FLA sequential kernel receives a zero tokenwise log-decay input, so it adds no further multiplicative token-retention gate. Its successive delta corrections remain order-dependent. Joint writing fits all frame associations together. This comparison examines the implemented joint-write operators; it does not identify serialization order as the sole explanation for their performance difference.

The following table shows that neither the simple recurrent mechanism nor parameter capacity really contributed to the success rate on RMBench. For the stateless adapter, it is basically equivalent to adding more parameters to the action expert, but it still performs poorly. Also, for pooled GRU, it applies a recurrent mechanism, but the parameter capacity is not enough to capture long-term history. The RMT experiment provides a clearer illustration of the issue: the RMT ablation has higher parameter capacity, the same number of memory tokens, and the same history-state size, but it still performs worse than our DRAM.

MethodParametersFP32 statePrefix tokens
Stateless Adapter61.49M0256
Pooled GRU6.56M2 KiB1
RMT373.07M9 MiB256
DRAM (seq. write) / DRAM (write first) / DRAM288.05M9 MiB256
AuthorsOpenReview ↗

Response to Reviewer 9gea (Weakness 2 and Question 2)

W2 & Q2. What are DRAM’s measured inference latency, memory overhead, and control frequency relative to the backbone alone and alternative memory modules, including the per-layer Cholesky solves?
A2. We evaluate the the inference overhead of pi0.5 and pi0.5 + DRAM over historical window HH, the main overheads of DRAM integrated is arised in cholesky solve in all DRAM layer, which introduce extra 35ms of compuation overhead, also, for action expert, extra memory tokens raise the action expert inference cost by 10%. However, comapring to the just packing a fixed window of history frame into VLM, the DRAM inference obverhead is much less than naive approach and reatin the constant computation overhead across the entire trajectories. We use first 10 steps from action chunks, so without inference optimization the control speed is about 6.25Hz. For this computation overhead, even without highly inference optimization, the inference cost of pi0.5 with DRAM could be covered by using technique like RTC (real time chunking).

Stage / Metricspi0.5pi0.5pi0.5 + DRAMpi0.5 + DRAM
H132132
VLM (ms)43.454421.0343.2543.75
DRAM (ms; includes Cholesky solves in all layers)——35.8335.80
Action Expert (ms)72.22166.6980.1180.09
Full Path Overhead (ms)115.684594.82159.19159.64
Peak GPU memory(GiB)9.0058.3810.2310.24
Presistent memory state (MiB)009.009.00
AuthorsOpenReview ↗

Code Release and Kind Request for Further Discussion

Dear Reviewer 9gea,

Thank you again for your thoughtful review. As a further update, we have anonymously released our model inference code at https://anonymous.4open.science/r/DRAM-5ABC/, along with the model weights for RMBench.

As the discussion period draws to a close, we would greatly appreciate the opportunity for further discussion. If you have any remaining questions or concerns, we would be happy to discuss them with you at any time.

Reviewer_eJjXOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Delta-rule Recurrent Associative Memory, DRAM, for robot manipulation policies, which builds on delta-rule memory management where instead of unconditionally adding each new association, DRAM first retrieves the value currently associated with a key and then writes the residual between this prediction and the new value. Thus, DRAM can continuously incorporate new observations into a fixed-size recurrent state while controlling redundant and conflicting writes. The paper also develops a read-before-write mechanism for visual patches which is needed for robotic memory. They also develop a lightweight residual interface so it is easy to incorporate into existing policy backbones. The simulation experiments are done in LIBERO and the real-world experiments are on two tasks.

Critical strengths

The paper develops a novel DRAM mechanism for robotic manipulation policies.

Critical weaknesses

The experimental evidence seems limited. The basic LIBERO benchmark is really quite saturated. The real-world experimentation is restricted to two tasks with 20 trials. There seem to be no baselines tested for the real-world experiments as its mostly an ablation between DiT and DiT + DRAM.

Questions for authors

Why does DRAM perform better than other memory mechanisms? Would this approach scale well if you are rewriting previous memories with residuals?

5 comments
AuthorsOpenReview ↗

Response to Reviewer eJjX (Weakness 1)

W1. The basic LIBERO benchmark is really quite saturated. The real-world experimentation is restricted to two tasks with 20 trials. There seem to be no baselines tested for the real-world experiments as its mostly an ablation between DiT and DiT + DRAM.
A1. Thank you for your comment. To further validate the advantages of DRAM, we expanded the evaluation to RMBench, which contains nine memory-dependent manipulation tasks spanning M(1) and M(n). The results are summarized below. DRAM with pi0.5 achieves average success rate of 59.11%, with 100 evaluation episodes per task. Its averages are 52.80% on M(1) and 67.00% on M(n).

PolicyOverall success %
pi0.510.40
X-VLA9.80
MemER8.70
MemoryVLA19.40
Mem-042.00
pi05 + DRAM59.11

The baseline values are taken from RMBench Table I. They retain the original training and evaluation protocols. Architectures and supervision also differ across these policies. We therefore use the published results as benchmark context and the component comparisons below to examine the DRAM design. The additional evaluation includes 98% success on Battery Try, 90% on Blocks Ranking Try, and 92% on Rearrange Blocks. Substantial difficulties remain on Observe and Pick Up (6%) and Press Button (8%). These results broaden the evidence beyond LIBERO and expose remaining limitations on tasks explicitly designed around memory requirements.

For the real-world evaluation, we conducted 20 trials per task–policy setting, consistent with prior work [1,2,3]. Given the limited rebuttal period, we were unable to conduct additional real-world trials. Nevertheless, we have substantially expanded our evaluation on RMBench and hope that these additional results help address your concerns. Besides, we will organize the inference code and corresponding model weights for RMBench and LIBERO and make them available via an anonymous link before November 18 at 23:59 UTC.

[1] Gong Z, Ding P, Lyu S, et al. Carp: Visuomotor policy learning via coarse-to-fine autoregressive prediction[C]//2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025: 13460-13470.
[2] Fu Y, Chen N, Zhao J, et al. Metis: Multi-source egocentric training for integrated dexterous vision-language-action model[J]. arXiv preprint arXiv:2511.17366, 2025.
[3] Yan H, Li Q, Yang J, et al. Progressvla: Progress-guided diffusion policy for vision-language robotic manipulation[J]. arXiv preprint arXiv:2603.27670, 2026.

AuthorsOpenReview ↗

Response to Reviewer eJjX (Question 1)

Q1. Why does DRAM perform better? Would this approach scale well if you are rewriting previous memories with residuals?
A1. Linear attention provides an efficient recurrent memory by compressing historical key–value associations into a fixed-size state. Schlag et al. [1] identified limitations of purely additive accumulation and introduced delta-rule updates, which correct the existing key–value mapping using prediction residuals. This enables targeted modification of stored associations. Yang et al. [2] subsequently developed hardware-efficient parallelization of the delta-rule recurrence, making these models practical to train at scale. Gated DeltaNet [3] further combined adaptive retention gates with residual corrections, supporting both rapid forgetting and content-selective updates.

DRAM combines gating for adaptive retention with key-conditioned residual corrections for selective memory updates, following [3]. Joint writing preserves this content selectivity: updates are confined to the subspace addressed by the current keys, while orthogonal memory components are preserved subject only to frame-level retention. Unlike sequential delta writing, which couples later corrections to earlier writes and introduces serialization-dependent recency, DRAM treats all patches within a frame as simultaneous observations and jointly fits their associations. This removes artificial within-frame order dependence while preserving spatial information and selective associative revision. Compared with sequential writing, joint writing improves overall success from 55.11% to 59.11% and M(n) success from 53.00% to 67.00%.

Regarding scalability, residual rewriting updates a fixed-size associative state without retaining or reprocessing the complete observation history. For fixed frame-token counts and memory dimensions, persistent storage and per-frame inference update cost are independent of episode length. Selective correction supports continued memory revision within this bounded state, although finite capacity still permits interference and forgetting; it does not imply unlimited history retention.

[1] Schlag I, Irie K, Schmidhuber J. Linear transformers are secretly fast weight programmers[C]//International conference on machine learning. PMLR, 2021: 9355-9366.
[2] Yang S, Wang B, Zhang Y, et al. Parallelizing linear transformers with the delta rule over sequence length[J]. Advances in neural information processing systems, 2024, 37: 115491-115522.
[3] Yang S, Kautz J, Hatamizadeh A. Gated delta networks: Improving mamba2 with delta rule[C]//International Conference on Learning Representations. 2025, 2025: 29687-29707.

Thank you for expanding to RMBench as well. However, the tasks seem somewhat simple such as "cover blocks" and "press button". However, I do want to main my weak reject rating due to the lack of the real-world results in the original paper unless there are significantly more results in simulation.

Reviewer_s9ZeOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DRAM is a recurrent memory module for adding historical context to pretrained robot policies while keeping the vision/VLM backbone frozen. It maintains a fixed-size associative state, retrieves from the previous state before writing the current observation, and updates all tokens within a frame jointly through a regularized least-squares objective. This avoids the within-frame order dependence of sequential delta-rule writes. The memory, policy-specific adapter, and action expert are trained using the control objective with truncated BPTT. The paper evaluates integration with π0.5 and a DiT-based policy on LIBERO, and with the DiT policy on two real-robot manipulation tasks. The largest reported gains occur for the DiT backbone and the longer real-world task.

Critical strengths
  • Treating all tokens from one observation as a joint write is a sensible design choice. It avoids assigning different recency to visual patches simply because of their serialization order. The objective in Eq. (15) gives a well-defined update, and Appendix C clearly establishes its connection to the single-token gated delta rule. Keeping the vision/VLM backbone frozen while learning memory through the control objective also makes the approach relevant to adapting existing policies.

  • The within-backbone comparisons in Table 2 are the strongest empirical evidence. DiT improves from 84.31% to 94.63% on average, including a gain from 75% to 91% on LIBERO-10. The real-world Full task improves from 0/20 to 17/20 successful trials. These are substantial improvements, and showing gains with both DiT and π0.5 makes the integration approach more convincing than a result on a single host policy.

Critical weaknesses
  • The experiments do not isolate the value of the proposed memory update. Table 2 shows that adding the DRAM system helps, but there is no matched comparison against a simpler recurrent memory, a sequential delta-rule update, or an adapter with comparable capacity but no history.

  • The experimental reporting leaves important questions about reproducibility and practical cost. The main LIBERO tables do not specify rollout counts or variability across training seeds; the 300-rollout protocol is stated specifically for the reset experiment. Key settings such as the TBPTT window, instantiated memory dimensions, and training budget are also missing. This makes the smaller π0.5 improvement, from 97.5% to 99.0%, difficult to assess.

Questions for authors
  1. Can you provide a controlled comparison against a simple recurrent-memory baseline and an adapter-only, no-history control, keeping the backbone, data, and action-expert training fixed? A joint-versus-sequential delta-write ablation would be particularly helpful for determining whether the proposed update contributes beyond the added memory capacity.

  2. What are the rollout counts, training-seed and checkpoint-selection protocols, TBPTT window, memory dimensions, and training budget for Table 2? Can you provide training results, particularly for the π0.5 comparison?

2 comments
AuthorsOpenReview ↗

Response to Reviewer s9Ze

W1 & Q1. Comparisons of DRAM with simpler recurrent memory mechanisms , sequential delta-rule writing , and an adapter without history.
A1. Thank you for your question. We have added comparisons of DRAM with simpler recurrent memory mechanisms (Pooled Gated Recurrent Unit(GRU) Memory and Recurrent Memory Transformer(RMT)), DRAM sequential delta-rule writing (Sequential Write), and an adapter without history (Stateless Adapter) on RMBench under the same setting to better isolate the contributions of DRAM’s design choices.

  1. Stateless adapter, an additional adapter projects current VLM key/value sources layerwise to action expert without history state, adding 256 extra tokens to action expert using current state VLM latent embedding only.
  2. Pooled GRU separately pools the final-layer native K/V features, updates a 512-dimensional hidden state, and projects the previous hidden state into one prefix token at each action-expert layer.
  3. RMT, maintains a fixed-size set of persistent memory tokens as its recurrent state and updates them with Transformer blocks as new observations arrive. Our external RMT baseline operates on frozen VLM features and maintains 18 independent memory banks, aligned with the VLM’s 18 integration layers. Each bank contains 256 memory tokens of dimension 512.
  4. DRAM (write first) , swap the read and write operation from default DRAM, the delta rule update is happend before the memory query tokens read out memory state first.
  5. DRAM (seq. write) replaces DRAM's joint update with an ordered token-by-token delta recurrence; Stateless Adapter projects current-frame VLM K/V features into additional conditioning tokens without carrying information between frames.

Each variant is evaluated over 100 episodes per task across nine tasks, totaling 900 episodes. The group success rate averages follow RMBench's M(1) and M(n) partition.

MethodOverall %M(1) %M(n) %
Stateless Adapter14.0016.4011.00
Pooled GRU25.3324.0027.00
RMT32.423.2044.00
DRAM (seq. write)55.1156.8053.00
DRAM (write first)58.2257.659
DRAM59.1152.8067.00

The results show that DRAM outperforms other variants in most of the situations, directly demonstrates the advantages of DRAM. Specifically, DRAM exceeds the tested stateless adapter and pooled GRU by 45.11 and 33.78 percentage points, respectively. Against DRAM (seq. write), it improves overall success by 4.00 points. The difference is concentrated in M(n), which improves from 53.00% to 67.00%. DRAM (seq. write) performs better on M(1), at 56.80% versus 52.80%; we report this task-group dependence explicitly.

We further emphasize that for comparison of DRAM (seq. write) and DRAM, the setup is aligned to frame-level decay with no further human bias. The sequential and joint variants share the memory architecture, parameter count, state dimensions, readout interface, and frame-level retention parameterization. Both apply retention once before writing the frame. While the FLA sequential kernel receives a zero tokenwise log-decay input, so it adds no further multiplicative token-retention gate. Its successive delta corrections remain order-dependent. Joint writing fits all frame associations together. This comparison examines the implemented joint-write operators; it does not identify serialization order as the sole explanation for their performance difference.

W2 & Q2. What are the rollout counts, training-seed and checkpoint-selection protocols, TBPTT window, memory dimensions, and training budget for Table 2? Can you provide training results, particularly for the π0.5 comparison?
A2. For Libero evaluation, we evaluate 50 episodes for each subtask across 4 suites. The training hyperparameters for pi0.5-based DRAM experiments are shown below; we also add more detail in the appendix. As for the checkpoint-selection protocols, we select the latest checkpoint.

Regarding to the training results in Table 2, we will organize the inference code and corresponding model weights for RMBench and LIBERO and make them available via an anonymous link before November 18 at 23:59 UTC.

SettingValue
Pretrained policypi0.5-base
Memory tokens budgets128 tokens
Training budget50,000 steps
OptimizerAdamW
Peak learning rate1.5e-4
Warmup5,000 steps
Learning-rate decayCosine; 50,000-step schedule horizon
Global batch size32 trajectory windows per optimizer update per GPU
TBPTT window32
Mixed precisionbfloat16 autocast; FP32 memory states and joint solves
AuthorsOpenReview ↗

Code Release and Kind Request for Further Discussion

Dear Reviewer s9Ze,

Thank you again for your thoughtful review. As a further update, we have anonymously released our model inference code at https://anonymous.4open.science/r/DRAM-5ABC/, along with the model weights for RMBench.

As the discussion period draws to a close, we would greatly appreciate the opportunity for further discussion. If you have any remaining questions or concerns, we would be happy to discuss them with you at any time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.