- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LAC, which learns both which visual tokens to recompute and how many tokens to reuse across consecutive frames. A selector and a cache-ratio predictor are initialized from attention and then optimized with the VLA task loss using straight-through differentiable decisions. Simulation and real-robot experiments report meaningful wall-clock savings while maintaining or improving task success.
Critical strengths
- Well-motivated
- The paper measures end-to-end CUDA latency and evaluates multiple VLA backbones, simulation benchmarks, and physical tasks. This is important for embodied communities.
- The recovery mechanism is a practical idea for limiting error accumulation over long rollouts. This point is interesting for me.
Critical weaknesses
- In my understanding, differential token pruning is relatively not a new thing for VLMs and VLAs. Beyond general token pruning methods devised for VLMs, Authors are expected to introduce the difference between your method and latest works, and compare the proposed method with more related and latest works published in 2025 and 2026. Here are some recommendations to refer to [1-3].
- Code will be open-sourced to ensure its reproducibility?
- The training objective includes a cache-ratio reward, while the selector and ratio predictor are optimized through straight-through estimates. More evidence is needed that the learned policy remains stable across unseen scenes and different frame rates. For me, there are more latest benchmark beyond the early LIBERO suites and SIMPLER.
[1] EfficientVLA, NeurIPS'25
[2] LightVLA, ICRA'26
[3] Dynamic-LLaVA, ICLR'25
Questions for authors
See above
6 comments
Response to Reviewer WNqs (Part 2 of 2: W3)
W3: Stability and Generalization of the Learned Caching Policy
Thank you for highlighting the stability and generalization of the learned caching policy. Our design addresses these concerns through task-supervised optimization, observation-dependent compute allocation, and continual cache refresh, supported by the following results:
1. Training and Inference Stability:
We optimize (Eq. 9), where the cache-ratio reward encourages efficiency while the task loss penalizes reuse decisions that impair action prediction. Before joint end-to-end optimization, we initialize the selector through Stage I attention alignment and keep the pretrained VLA backbone completely frozen throughout both stages. This initialization provides a strong saliency prior: as shown in Table 2, removing Stage I initialization (w/o Stage I Init) reduces LIBERO-Spatial success from 85.6% to 79.2%. During Stage II, Straight-Through Estimation (STE) enables gradient-based training of discrete decisions, whereas at inference, the ratio predictor and token selector execute deterministic hard and decisions without Gumbel sampling noise. Complementarily, Stochastic Recovery separately refreshes a small random subset of cached states at each step; enabling the default improves success from 83.4% to 85.6% while increasing latency only from 29.04 ms to 29.32 ms (Table 2 & Appendix Table 6), effectively bounding accumulated cache staleness over long horizons.
2. Adaptation to Unseen Scene Changes:
LAC conditions both token selection and the reuse budget on the current RGB observation and inter-frame optical flow . Consequently, recomputation is dynamically allocated according to observed appearance and motion rather than a fixed spatial mask. Cached tokens remain accessible through their historical KV states across all Transformer layers, and stochastic recovery ensures that previously inactive regions periodically receive fresh computation. Our experiments provide direct evidence beyond a single scene configuration:
- On SIMPLER Variant Aggregation (Table 3)—which specifically evaluates zero-shot robustness across unseen backgrounds, lighting conditions, table textures, distractors, and camera viewpoints—LAC achieves 63.0% average success versus 61.3% for dense CogAct, together with a speedup.
- Physical robot experiments (Table 4 & Appendix D.4) further demonstrate reliable operation under real-world sensing noise, lighting variations, and physical interactions, improving average success from 33.0% to 38.0% ( speedup) on single-view tasks and from 50.0% to 60.0% ( speedup) in a dual-camera (main + ego-centric wrist camera) setup.
3. Adaptation to Different Operating Frequencies (Frame Rates):
Rather than imposing a fixed reuse schedule, LAC makes a fresh caching decision at each observation. Because inter-frame optical flow directly captures the displacement magnitude between consecutive observations while RGB provides complementary appearance cues, the learned predictor automatically adjusts the reuse budget as temporal spacing changes, while the selector directs fresh computation toward newly displaced regions. Our evaluations span OpenVLA, CogAct, and (Tables 1 & 3), which operate under distinct action-chunking horizons and control frequencies and thus expose the caching module to different temporal sampling regimes. Across all three backbones, LAC preserves task success (76.9% vs. 75.0% on OpenVLA; 67.3%/63.0% vs. 64.9%/61.3% on CogAct; 96.4% vs. 96.85% on ) while delivering consistent end-to-end speedups (, , and ).
4. Benchmark Coverage:
We agree that evaluating on emerging benchmarks can further broaden coverage. Our current evaluation already spans 4 LIBERO suites (40 tasks, 2,000 rollout episodes), 2 SIMPLER protocols (Visual Matching and OOD Variant Aggregation), 3 distinct VLA architectures, and physical robot manipulation, providing complementary support for the adaptability of our learned caching mechanism. We will incorporate this expanded discussion of policy stability and frequency adaptability into the revised manuscript.
Reply to Authors
I appreciate your detailed reply. If authors add corresponding comparison with more state-of-the-art pruning methods instead of only stating the comparison with VisionZip, FastV and other counterparts (a bit out-of-date), I'm willing to raise my rating.
Best
Response to Reviewer WNqs (Part 1 of 2: W1–W2)
We thank the reviewer for recognizing that our work is well-motivated, measures genuine end-to-end CUDA latency across multiple VLA backbones and physical tasks, and introduces a practical stochastic recovery mechanism for long-horizon rollouts.
W1: Methodological Differences and Comparisons with Latest 2025–2026 Works (
EfficientVLA,LightVLA,Dynamic-LLaVA)We appreciate the pointers to [1] EfficientVLA (NeurIPS'25), [2] LightVLA (ICRA'26), and [3] Dynamic-LLaVA (ICLR'25). We will cite and discuss all three works in Section 2 and Appendix D.2. Below, we clarify the fundamental methodological distinctions and provide empirical comparisons:
1. Methodological Distinction — Spatial Token Pruning vs. Non-Destructive Temporal KV Caching:
Dynamic-LLaVA [3] & LightVLA [2] (Single-Frame Differentiable Token Pruning): Both methods use learnable predictors with Gumbel-Softmax/STE to permanently prune (drop) visual tokens spatially within a single forward pass to shorten sequence length. While effective for static VLM question answering or single-frame compression, spatial dropping is risky in closed-loop manipulation: an ungrasped target receptacle or secondary object that is pruned in early layers is completely inaccessible to deeper Transformer layers. Furthermore, neither method exploits cross-frame KV state reuse or defines a budget predictor + stochastic recovery loop across control steps.
EfficientVLA [1] (Training-Free Pruning & Diffusion Step Caching): EfficientVLA accelerates diffusion-based VLAs via a training-free pipeline combining redundant language-layer pruning, task-relevance/diversity visual token pruning, and intermediate diffusion denoising feature caching. Its visual token selection relies on static attention/similarity heuristics decoupled from the action loss LVLA, and its diffusion-step caching is specific to iterative denoising heads rather than general cross-frame visual prefilling.
Ours (LAC — Task-Learned Cross-Frame KV Reuse with Stochastic Refresh): Instead of discarding spatial tokens, LAC maintains a full-context cross-frame KV cache (N tokens) at every layer and learns which token states to refresh (fsel) and what global budget to allocate (fpred) based on Vt=[It;Ot] and ∇LVLA, protected by Stochastic Recovery (precover). Thus, static objects remain continuously visible in the attention context while active manipulation regions are freshly recomputed.
2. Empirical Comparison on LIBERO (OpenVLA Backbone):
As shown in Table 1 and our extended comparisons with recent 2025–2026 VLA accelerators (including
SP-VLA[ICLR'26] andFLASH-VLAon the exact same OpenVLA backbone across the 4 LIBERO suites):Notably, spatial token pruning/merging baselines (
VisionZip,FastVLM,SparseVLM) suffer severe accuracy drops on LIBERO-Long (dropping from 53.2% to 4.6%, 36.6%, and 39.4%, respectively) because permanently dropping tokens destroys spatial memory over long-horizon multi-stage tasks, whereas LAC improves LIBERO-Long to 59.2% (+6.0 pp over dense OpenVLA).W2: Code Release and Reproducibility
Yes, we will fully open-source our code. In addition, as detailed in our earlier responses to Q1–Q2 and Appendix B, we have provided the complete architectural specifications (5-layer CNN, 0.42M parameters), mathematical backward surrogates, and hyperparameter settings to ensure full reproducibility.