- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates long-term visual memory for vision-language-action (VLA) models by directly incorporating hundreds of historical images into robot action prediction. Built on Qwen3.5-4B, the proposed LongContextVLA combines streaming cache, parallel narration decoding, and grouped action decoding to reduce inference latency. The model is also trained to narrate historical events through latent reasoning tokens, encouraging it to extract task-relevant information from long visual histories without autoregressive reasoning at inference time. Experiments on RoboMME and three real-world robotic manipulation tasks demonstrate strong performance on memory-dependent tasks and generalization to unseen scenarios.
Critical strengths
-
Important research problem. This paper addresses the important problem of long-term visual memory in VLAs and explores the advantages of retaining dense visual histories over memory compression and keyframe selection.
-
Efficient inference with strong empirical results. The proposed framework combines Streaming Cache, Parallel Narration Decoding, and OAT-based Grouped Action Decoding to reduce inference latency. It further uses historical event narration to supervise latent reasoning, improving the model's ability to utilize historical information without autoregressive text generation. The method achieves an average success rate of 84.9% across 16 RoboMME tasks and demonstrates generalization to unseen event sequences, object assignments, and task scales in real-world experiments.
Critical weaknesses
-
Limited benchmark coverage and incomplete SOTA comparisons. The simulation evaluation is limited to RoboMME, and additional experiments on RoboMemArena [1], RMBench [2], or MIKASA-Robo [3] would help establish the method's effectiveness across more diverse memory-intensive tasks. More importantly, the paper reports only the ablation variants of SimpleMemVLA [4], while omitting its full model, which reports 88.3% success on RoboMME compared with 84.9% in this paper. A fair comparison with the complete method is necessary to support the claimed improvement over existing approaches.
-
Insufficient analysis of history scaling and inference efficiency. The paper does not systematically examine how task success changes with the number of historical frames, making it unclear how many additional frames are needed to achieve meaningful improvements and when the performance gains begin to diminish. Furthermore, while Parallel Narration and Grouped Action Decoding increase computational parallelism, they do not fundamentally reduce the computational complexity of the corresponding decoding operations. The efficiency claims would be more convincing with end-to-end latency comparisons against optimized single-frame and few-frame VLA baselines under the same computational resources.
[1] Lei et al., 2026. RoboMemArena: A Comprehensive and Challenging Robotic Memory Benchmark.
[2] Chen et al., 2026. RMBench: Memory-Dependent Robotic Manipulation Benchmark with Insights into Policy Design.
[3] Cherepanov et al., 2026. Memory, Benchmark & Robots: A Benchmark for Solving Complex Tasks with Reinforcement Learning. ICLR 2026.
[4] Yin et al., 2026. SimpleMemVLA: A Simple but Effective Native-Video Memory for Vision-Language-Action Models.
Questions for authors
-
Benchmark coverage: Could the authors provide additional results on memory-intensive benchmarks such as RoboMemArena [1], RMBench [2], or MIKASA-Robo [3] to demonstrate the generality of the proposed approach?
-
SOTA comparison: Why is the full SimpleMemVLA [4] model, which reports 88.3% on RoboMME, omitted from Table 1 while its ablation variants are included? Could the authors provide a fair comparison under consistent evaluation settings?
-
History length scaling: Could the authors evaluate task success rates with different numbers of historical frames? This would help determine how many additional frames yield meaningful performance improvements and at what history length further gains begin to diminish.
-
Inference efficiency: Could the authors compare the end-to-end inference latency of LongContextVLA with optimized single-frame and few-frame VLA baselines under the same computational resources? This would help quantify the additional inference cost of maintaining long visual histories.
-
Understanding the decision process: The paper does not sufficiently analyze how historical visual information and latent reasoning affect action decisions. Could the authors investigate the relationships between historical visual tokens, latent reasoning tokens, and action tokens, for example through token-level attention or attribution analysis, to clarify their respective roles in action prediction?
2 comments
Q4: Inference efficiency: Could the authors compare the end-to-end inference latency of LongContextVLA with optimized single-frame and few-frame VLA baselines under the same computational resources? This would help quantify the additional inference cost of maintaining long visual histories.
Thank you for this suggestion. We evaluated history length scaling using our full model on 150 unseen five-swap videos and 100 unseen six-swap videos. We kept the checkpoint and evaluation protocol fixed, sampled observations at 2 Hz, and varied the maximum number of recent frames.
The table reports offline cup-location accuracy (%).
| Maximum history frames | 5 swaps | 6 swaps |
|---|---|---|
| 1 | 38.0 | 34.0 |
| 4 | 32.0 | 31.0 |
| 8 | 30.0 | 35.0 |
| 12 | 35.3 | 38.0 |
| 16 | 80.7 | 31.0 |
| 20 | 83.3 | 40.0 |
| 24 | 84.0 | 64.0 |
| Full history | 84.0 | 65.0 |
For five swaps, accuracy increases substantially from 35.3% at 12 frames to 80.7% at 16 frames, with smaller gains thereafter, reaching 84.0% with full history. For six swaps, the largest observed improvement occurs later: accuracy increases from 40.0% at 20 frames to 64.0% at 24 frames, close to the full-history accuracy of 65.0%.
These results demonstrate the importance of retaining sufficient temporal history, with longer swap sequences requiring more historical observations. Increasing the budget beyond the full available history provides no additional benefit on these videos, because the input remains unchanged. At 2 Hz, the available histories contain 18–26 frames for five swaps and 22–32 frames for six swaps. Thus, the observed diminishing gains apply to these recordings and do not establish saturation for longer videos.
Q5: Understanding the decision process: The paper does not sufficiently analyze how historical visual information and latent reasoning affect action decisions. Could the authors investigate the relationships between historical visual tokens, latent reasoning tokens, and action tokens, for example through token-level attention or attribution analysis, to clarify their respective roles in action prediction?
Thank you for this suggestion. We examine the roles of historical visual information and latent reasoning through history-length ablations, model ablations, and real-robot evaluation.
Historical visual information affects the model’s decisions. Keeping the checkpoint and evaluation protocol fixed, we vary the number of recent observations sampled at 2 Hz. Five-swap cup-location accuracy increases from 35.3% with 12 frames to 80.7% with 16 frames, reaching 84.0% with full history. For six swaps, accuracy increases from 40.0% with 20 frames to 64.0% with 24 frames, close to the full-history accuracy of 65.0%. These results show that retaining earlier observations substantially improves location prediction, with longer swap sequences requiring more historical observations.
The latent-reasoning variant improves both location prediction and physical task completion. We compare three model variants below. Offline evaluation measures cup-location accuracy on 150 five-swap and 100 six-swap videos. Real-robot evaluation measures full-task success over 30 trials per setting, requiring both selecting the correct cup and completing the grasp.
| Model | Offline: 5 swaps | Offline: 6 swaps | Real robot: 5 swaps | Real robot: 6 swaps |
|---|---|---|---|---|
| OAT + subtask | 55.3% | 50.0% | 53.3% | 36.7% |
| OAT + subtask + narration | 51.3% | 47.0% | 60.0% | 33.3% |
| Full model, including latent reasoning | 84.0% | 65.0% | 76.7% | 70.0% |
Adding narration alone does not consistently improve performance. The full model achieves higher accuracy and task completion than both ablations. In particular, compared with the narration-only variant, six-swap location accuracy increases from 47.0% to 65.0%, while real-robot success increases from 33.3% to 70.0%.
Together, these experiments show that access to historical observations is important for identifying the correct target, and that the full model’s improvements extend from semantic decisions to successful physical execution.
Q1. Benchmark coverage: Could the authors provide additional results on memory-intensive benchmarks such as RoboMemArena [1], RMBench [2], or MIKASA-Robo [3] to demonstrate the generality of the proposed approach?
Thank you for suggesting these benchmarks. Our current evaluation covers the sixteen RoboMME tasks, which span four memory categories, together with three real-robot tasks. During the rebuttal period we prioritized the core experimental results, including a new real-robot drawer task and larger real-robot sample sizes, and could not complete evaluations on additional benchmarks in the same window. We agree that they would further demonstrate generality, and we hope to include results on them in the final version.
Q2. SOTA comparison: Why is the full SimpleMemVLA [4] model, which reports 88.3% on RoboMME, omitted from Table 1 while its ablation variants are included? Could the authors provide a fair comparison under consistent evaluation settings?
Thank you for pointing this out. Omitting the full result was an oversight, and we have added it to Table 1 (88.3%, compared with 84.9% for our method). All SimpleMemVLA numbers in Table 1 are taken from its paper.
For a comparison under consistent settings, we trained SimpleMemVLA ourselves on our real-robot tasks, where it is the strongest baseline. LongContextVLA achieves higher success in all three settings, with 30 trials each:
SimpleMemVLA performs comparably to our subtask-only ablation. This is expected, because the two are closely related: both condition on the full video history and predict a subtask before acting, and neither is trained to narrate its history. Our main finding is that narration improves generalization to unseen scenarios, which is where LongContextVLA outperforms SimpleMemVLA.
For context, we would also like to respectfully note that SimpleMemVLA is concurrent work: it appeared on arXiv on September 2, 2026, two weeks before the submission deadline. The ICLR reviewer guidelines (https://iclr.cc/Conferences/2027/ReviewerGuidelines) consider a paper contemporaneous if it was published within the two months before the full paper deadline, that is, on or after July 17, 2026, and authors are not required to compare against such work. The guidelines also note that arXiv is not a peer-reviewed venue, so authors are not required to compare against papers available only on arXiv. They add that reviewers may make authors aware of such work, but that "the lack of such comparisons cannot be a basis for rejection." We are grateful that you brought the full result to our attention, we agree that the comparison is informative, and we are glad to include it. We would kindly ask that this be taken into consideration.
Q3. History length scaling: Could the authors evaluate task success rates with different numbers of historical frames? This would help determine how many additional frames yield meaningful performance improvements and at what history length further gains begin to diminish.
Thank you for this question. We expect the policy to be robust to history length and sampling density because of its backbone. The Qwen3.5 family supports video inputs of up to two hours within its 256K-token context, at varying frame rates and resolutions. The backbone family also handles sparse and irregular frame sequences: MemER (Sridhar et al., 2026) [5] obtains useful memory from a small set of disjoint keyframes with a backbone from the same family.
In our early experiments, changing the history sampling density did not visibly affect performance, and performance did not degrade as the history grew to more than 250 images at some decision steps.
During the rebuttal period we prioritized the core experimental results, and could not complete a systematic sweep in the same window. We will include a controlled sweep of history length and sampling rate in the final version.
[5] Ajay Sridhar, Jennifer Pan, Satvik Sharma, and Chelsea Finn. Scaling up memory for robotic control via experience retrieval. International Conference on Learning Representations (ICLR), 2026.