acceptodds
Withdrawn from ICLR 2027

EvoMem-VLA: State-Evolution Memory for Long-Horizon Robot Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Most vision-language-action (VLA) models rely on current observations and lose task-relevant evidence once it leaves view, limiting performance on long-horizon, memory-dependent tasks. Existing efforts incorporate compressed historical features or sparse visual keyframes. However, isolated snapshots can leave the policy uncertain about what changed during past interactions and which action should follow. To overcome this limitation, we propose EvoMem-VLA, which constructs state-evolution memory by explicitly encoding and retaining observed changes between historical states. These change representations preserve evidence of interaction outcomes, allowing the policy to track task progress beyond isolated snapshots. Specifically, we introduce conditional delta tokenization to encode ordered frame pairs into directional, source-conditioned delta tokens, each associated with its corresponding state evidence. A shared VLM backbone supports task-adaptive routing: normal long-horizon tasks follow a direct action route, whereas multi-stage tasks use a subtask route that generates an executable subtask as an additional input for action generation. With a single jointly trained policy for each simulation benchmark, EvoMem-VLA achieves success rates of 80.7% on RMBench, 82.0% on RoboMME and 83.8% across four real-world tasks spanning two robot embodiments. These results represent substantial improvements over the previous state of the art in all three evaluation settings.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, long-horizon manipulation, state-evolution memory, delta tokenization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AjzzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EvoMem-VLA adds a memory of visual changes to a VLA for memory-dependent manipulation. A frozen DeltaTok-style tokenizer on DINOv3 features turns each ordered frame pair into one delta token. These tokens are placed next to the initial frame, recent frames and EventVLA-style keyframes. A shared Qwen3-VL-4B backbone predicts actions directly, or first generates a subtask and then predicts actions. Which route a task uses is set by hand. The paper reports persistent improvement over various tasks from RMBench and RoboMME. RMBench ablations are used to argue that delta tokens are the main contributor

Critical strengths
  • Large, consistent gains on two memory benchmarks and on real robots with two embodiments, especially on the hardest tasks.
  • A cheap change representation (one token per frame pair) that fits existing VLA pipelines. The learned version clearly beats naive feature differencing.
Critical weaknesses
  • The gains cannot be attributed to the proposed memory. All baseline numbers are copied from prior papers, which used different backbones, budgets and protocols. EvoMem-VLA also uses subtask supervision and a hand-set route per task, which EventVLA (VA only) does not appear to use. The authors’ own ablation shows this matters. Without the subtask route, M(n) drops to 33.0, below EventVLA’s 54.0. Without the route, delta tokens add only 2 points on M(n). RoboMME, where the margin is largest, has no ablation.
  • The temporal design is fixed and untested, and it does not test long-horizon memory. Every setting uses the same recent schedule: anchors at t−30, t−15 and t, plus three delta pairs at a fixed 15-step interval covering only the last 45 steps. This holds even though action-chunk lengths (50, 20 and 30 steps) and task horizons differ widely. The paper never justifies these choices or ablates them: not the interval, not the number of delta groups, and not a longer window. The right temporal scale could also depend on the task. In addition, keyframe memory is disabled on RMBench, where all ablations are run, so the gains there may come from short-term motion context rather than long-horizon memory. The event-local delta tokens tied to keyframes are never ablated.
Questions for authors
  • How were the RMBench subtask labels created? Do they encode progress, for example counts in Press Button? If so, the subtask history acts as a supervised progress counter.
  • Why compare against EventVLA (VA only) rather than a variant that uses language?
  • Can you train a baseline in your own framework with the same subtask supervision (for example EventVLA-style memory, or your state-only variant with keyframes)? Can you also run ablations on RoboMME?
  • How did you choose the 15-step interval, the three delta groups and the 45-step window? How sensitive are the results to these choices, including longer windows or more groups? Would the best setting differ across tasks or control frequencies, and could it be set adaptively instead of fixed?
  • Does adding the raw source frames (t−45) without delta tokens recover the gain?
  • Please report variance across seeds and the inference latency.

Suggestion (presentation)

The paper is not self-contained about its baselines. Several methods in Tables 1 and 2 are never described, including Mem-0, SimpleSG, GroundSG, TokenDrop, FrameSamp, TTT and RMT. Variant labels such as “(Modul)”, “(Expert)”, “(QwenVL)” and “(VA only)” are not explained either. Appendix B.3 lists only where the scores came from. Please add a short description of each baseline’s memory approach and backbone, in the main text or in the appendix, so readers can tell what kind of memory each comparison represents.

Reviewer_7jQUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the issue of insufficient utilization of historical information in long-term robot operations by proposing EvoMem-VLA. This method encodes ordered image frame pairs into Delta Tokens through Conditional Delta Tokenization and constructs a State-Evolution Memory by combining initial observations, recent history, and keyframes to explicitly model the state evolution process. Simultaneously, it designs two paths based on a shared VLM: direct action prediction and sub-task conditional action prediction. Multiple benchmark tests and real-world robot experiments validate the effectiveness of the proposed method, and ablation experiments further demonstrate the performance contribution of key modules.

Critical strengths
  1. Addressing the difficulty of existing methods in fully utilizing state change information in historical interactions, this paper employs Conditional Delta Tokenization to encode ordered frame pairs and combines visual state information to assist action prediction. This design is rationally motivated, explicitly modeling the state evolution process, and has the potential to be integrated with existing VLA architectures.
  2. EvoMem-VLA demonstrates superior performance compared to baselines in RMBench, RoboMME, and real-world robot tasks. Ablation experiments further validate the effectiveness of Delta Tokens and sub-task paths, and show that the learned Delta representation is more advantageous than direct feature differencing, providing experimental support for the design of key modules.
Critical weaknesses

This paper constructs a State-Evolution Memory by combining historical visual states with Delta Tokens to enhance the model's understanding of state evolution. However, existing component ablation experiments were only conducted on RMBench without keyframe memory enabled, primarily validating the recent effectiveness of Delta Tokens. For RoboMME employing the complete memory structure, the paper has not yet provided corresponding ablation analysis, making it difficult to clarify whether Delta Tokens can still provide additional benefits on top of the existing historical keyframe memory.

Given that the combination of state change representation and historical state memory is the core design of this paper, it is recommended that the authors supplement RoboMME with a control experiment that retains keyframe memory but removes only Delta Tokens, to further validate the independent contribution of Delta Tokens and the validity of the complete memory structure.

Questions for authors
  1. Current ablation experiments on RMBench have demonstrated the performance contribution of Delta Tokens, but keyframe memory is disabled in this benchmark. Have the authors considered experiments on RoboMME that retain keyframe memory while only removing Delta Tokens?

  2. The Conditional Delta Tokenizer is trained by reconstructing target frame features under source frame conditions. Could the authors further explain how this conditional compressed representation helps the model utilize task-related state change information?

Reviewer_oGCROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EvoMem-VLA, a memory-augmented vision-language-action (VLA) policy for long-horizon robot manipulation. Its main addition is "delta tokens". A frozen DeltaTok encoder produces one token per pair of stored frames and camera view, describing the change between the two frames. These are added to an EventVLA-style frame memory. For multi-step tasks, the model first predicts the next subtask in text, following WLA-0. The paper reports the following success rates:

  • RMBench: 80.7% (vs. 67.8% for EventVLA)
  • RoboMME: 82.0% (vs. 44.5% for the strongest baseline)
  • Four real-robot tasks: 83.8% (vs. 30.0% for pi-MEM) It presents an RMBench ablation as evidence that delta tokens are the main contribution.
Critical strengths
  1. Strong results: statistically significant improvements on RMBench, RoboMME and the real robots.
  2. A clean 2x2 ablation study (delta tokens x subtask prediction, 900 episodes per variant) with convincing results.
  3. Clear writing and careful reporting.
Critical weaknesses
  1. The improvement over prior work is not shown to come from delta tokens.
  • EventVLA (67.8%) uses the same backbone (Qwen3-VL-4B), the same stored frames (initial, t-30, t-15 and current) and the same number of training steps (80,000). It is essentially this model without delta tokens and without subtask prediction.
  • With delta tokens but without subtask prediction, the model reaches 61.3%, which is 6.5 percentage points below EventVLA.
  • It surpasses EventVLA only when subtask prediction is added. Subtask prediction is trained on subtask text annotations that EventVLA does not use.
  • This does not support the claim that the ablations "validate the central contribution of conditional delta tokenization" (line 485).
  1. The ablation baseline is much weaker than the published equivalent. Without delta tokens and subtask prediction, the model reaches 44.3%. That is 23.5 percentage points below EventVLA in an almost identical setup. The paper does not explain this gap, and all delta-token gains in the ablation are measured within this weaker setup.
  2. Only RMBench is ablated, and keyframe memory is disabled there. As a result:
  • The long-range memory (keyframes paired with delta tokens) is never tested.
  • The RoboMME results have no ablation.
  • The real-robot results have no ablation, although the paper attributes them to delta tokens (lines 423-425).
  1. The role of the change information is not isolated.
  • The naive feature-difference control (47.1%) performs worse than using no delta tokens at all (52.3%).
  • There is no control that adds the same number of tokens without change information.
  • All results come from a single training seed, and no variance is reported.
  1. Novelty is limited: the method mainly combines EventVLA, WLA-0 and DeltaTok. Minor (not affecting my score):
  • The paper and the released code differ: three vs. two recent delta-token groups, and 100 vs. 50 RoboMME evaluation episodes per task.
  • The six "franka_wrist_*" videos show the AgileX robot from an overhead camera. Overall: the system results are strong, but the paper's main claim, that delta tokens drive the improvement, is not supported by the current ablations (Weaknesses 1–3).
Questions for authors
  1. Why does the variant without delta tokens and subtask prediction (44.3%) fall so far below EventVLA (67.8%)? Can you add delta tokens to an EventVLA-level model?
  2. Can you report RoboMME ablations without delta tokens, without subtask prediction and without keyframe memory?
  3. Can you add a control with the same number of tokens but no change information?
  4. How was the pi-MEM baseline implemented, given that its weights do not appear to be public?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.