- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Latent Transition Memory (LTM), a memory mechanism for VLAs that represents interaction history using compact latent transitions between consecutive observations. A frozen latent action encoder extracts a transition representation from each observation pair, which is projected into a single memory token and provided to the policy as an ordered memory sequence. For video-conditioned tasks, sparse visual anchors are additionally used to provide visual grounding.
Critical strengths
- The paper addresses an important limitation of current VLAs under partial observability.
- The proposed memory representation is compact and inference-efficient.
Critical weaknesses
-
The performance seems marginal. In particular, the proposed method performs worse than PonderPounce on the RoboMME benchmark.
-
Several recent works [1-4] already compress past observations into compact representations in a simpler way and compress the observations significantly. This paper does not include a comparison to these approaches or an in-depth discussion of why the latent action provides better context to VLAs.
-
A large gain seems to come from temporal cues rather than transition-specific content. Replacing the latent transitions with fixed random representations still achieves 44.50% success, compared with 57.67% for the full model. In several counting tasks, the random-token variant is even competitive with or better than the full model. This suggests that sequence length or temporal position alone explains a substantial portion of the improvement, making the contribution of the learned transition semantics less clear.
-
In real-world experiments, pi0.5 significantly underperform even in x=1 problem, but the proposed method gets 95%. This unusually large gap suggests that factors beyond long-term memory, such as training stability or temporal conditioning, may contribute substantially to the reported improvement.
-
Minor issue: The main result tables are difficult to interpret because they contain too many task-level scores without sufficiently summarizing the key trends. Moreover, the table captions provide limited information about the experimental settings under which these results are obtained, making the comparisons difficult to follow without repeatedly referring back to the main text.
[1] Jang et al., ContextVLA: Vision-Language-Action Model with Amortized Multi-Frame Context, 2025
[2] Torne et al., MEM: Multi-Scale Embodied Memory for Vision Language Action Models, 2026
[3] Kwon et al., Exploring High-Order Self-Similarity for Video Understanding, 2026
[4] Physical intelligence, π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities, 2026
Questions for authors
Please answer the weaknesses part.
Response to Reviewer 1Ybh
Our primary contribution is to propose using a Latent Action Model (LAM) as the memory representation for VLAs, along with several inference optimizations, including lagged inputs and KV-cache reuse. In fact, our experimental baselines already include methods that use pooled visual tokens to represent historical information, such as Frame Sampling and Token Dropping, both of which perform relatively poorly (with the best result reaching only 44.51%). We additionally evaluated a transition representation based on the video compression encoding approach proposed in HiF-VLA, which achieved a slightly better result of 51.3%, but still fell substantially short of our method.
We also observed this phenomenon in our experiments. For certain tasks, such as counting, feeding fixed random transitions according to timestamps can achieve relatively good performance. Upon further analysis, we found that this behavior reflects a dataset-specific shortcut. Since the manipulation tasks in the training set are performed at approximately the same speed and contain no external disturbances, the model can exploit temporal cues alone, learning a shortcut that associates the number of manipulations with their duration. However, after introducing random disturbances and variations in execution speed into both the training and test sets to invalidate this temporal shortcut, the success rate of the ablation relying solely on temporal cues dropped to 24% on the PickXtimes and SwingXtimes tasks, whereas the model using correct transitions maintained a success rate of 94%. This demonstrates that transitions provide meaningful information beyond temporal cues alone.
In fact, we have already discussed the latency details in the paragraph analyzing the effect of lag in our ablation study. With lag = 5, the transition encoding latency can be largely hidden through asynchronous inference, overlapping with policy execution and computation, while introducing almost no degradation in performance. Please refer to the corresponding section of our paper for further details.
We sincerely thank you for your careful review and constructive feedback. We will further refine and improve the paper accordingly.