- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EvoMem-VLA adds a memory of visual changes to a VLA for memory-dependent manipulation. A frozen DeltaTok-style tokenizer on DINOv3 features turns each ordered frame pair into one delta token. These tokens are placed next to the initial frame, recent frames and EventVLA-style keyframes. A shared Qwen3-VL-4B backbone predicts actions directly, or first generates a subtask and then predicts actions. Which route a task uses is set by hand. The paper reports persistent improvement over various tasks from RMBench and RoboMME. RMBench ablations are used to argue that delta tokens are the main contributor
Critical strengths
- Large, consistent gains on two memory benchmarks and on real robots with two embodiments, especially on the hardest tasks.
- A cheap change representation (one token per frame pair) that fits existing VLA pipelines. The learned version clearly beats naive feature differencing.
Critical weaknesses
- The gains cannot be attributed to the proposed memory. All baseline numbers are copied from prior papers, which used different backbones, budgets and protocols. EvoMem-VLA also uses subtask supervision and a hand-set route per task, which EventVLA (VA only) does not appear to use. The authors’ own ablation shows this matters. Without the subtask route, M(n) drops to 33.0, below EventVLA’s 54.0. Without the route, delta tokens add only 2 points on M(n). RoboMME, where the margin is largest, has no ablation.
- The temporal design is fixed and untested, and it does not test long-horizon memory. Every setting uses the same recent schedule: anchors at t−30, t−15 and t, plus three delta pairs at a fixed 15-step interval covering only the last 45 steps. This holds even though action-chunk lengths (50, 20 and 30 steps) and task horizons differ widely. The paper never justifies these choices or ablates them: not the interval, not the number of delta groups, and not a longer window. The right temporal scale could also depend on the task. In addition, keyframe memory is disabled on RMBench, where all ablations are run, so the gains there may come from short-term motion context rather than long-horizon memory. The event-local delta tokens tied to keyframes are never ablated.
Questions for authors
- How were the RMBench subtask labels created? Do they encode progress, for example counts in Press Button? If so, the subtask history acts as a supervised progress counter.
- Why compare against EventVLA (VA only) rather than a variant that uses language?
- Can you train a baseline in your own framework with the same subtask supervision (for example EventVLA-style memory, or your state-only variant with keyframes)? Can you also run ablations on RoboMME?
- How did you choose the 15-step interval, the three delta groups and the 45-step window? How sensitive are the results to these choices, including longer windows or more groups? Would the best setting differ across tasks or control frequencies, and could it be set adaptively instead of fixed?
- Does adding the raw source frames (t−45) without delta tokens recover the gain?
- Please report variance across seeds and the inference latency.
Suggestion (presentation)
The paper is not self-contained about its baselines. Several methods in Tables 1 and 2 are never described, including Mem-0, SimpleSG, GroundSG, TokenDrop, FrameSamp, TTT and RMT. Variant labels such as “(Modul)”, “(Expert)”, “(QwenVL)” and “(VA only)” are not explained either. Appendix B.3 lists only where the scores came from. Please add a short description of each baseline’s memory approach and backbone, in the main text or in the appendix, so readers can tell what kind of memory each comparison represents.