acceptodds
Under review as a conference paper at ICLR 2027

Learning to Accelerate Vision-Language-Action Models through Adaptive Visual Token Caching

Abstract

Vision-Language-Action (VLA) models have demonstrated remarkable generalization capabilities in robotic manipulation tasks, yet their substantial computational overhead remains a critical obstacle to real-world deployment. Improving inference efficiency is therefore essential for practical robotic applications. Existing acceleration methods often rely on heuristic or static strategies—such as rule-based token caching or pruning—that are decoupled from task objectives and fail to adapt to dynamic scene changes. In this work, we reformulate inference acceleration as a learnable policy optimization problem and propose a novel framework that integrates a dynamic, task-aware decision-making process directly into the VLA model. At its core are two lightweight, cooperative modules: a Cached Token Selector, which determines which tokens should be reused, and a Cache Ratio Predictor, which controls how many tokens to reuse. Training these modules is non-trivial due to their discrete decisions. We address this by adopting a differentiable relaxation that allows gradient-based end-to-end optimization. Extensive experiments on the LIBERO and SIMPLER benchmarks, as well as real-robot evaluations, show that our method achieves a 1.76 wall-clock inference speedup while simultaneously improving the average success rate by 1.9 percentage points (from 75.0% to 76.9%) on LIBERO and by 5.0 percentage points on real-world tasks, significantly outperforming existing baselines. This work highlights the potential of learning task-aware computational allocation policies, paving the way for VLA models that are both powerful and efficient.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Efficient Inference, Adaptive Token Caching, Robot Manipulation, Dynamic Computation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WNqsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LAC, which learns both which visual tokens to recompute and how many tokens to reuse across consecutive frames. A selector and a cache-ratio predictor are initialized from attention and then optimized with the VLA task loss using straight-through differentiable decisions. Simulation and real-robot experiments report meaningful wall-clock savings while maintaining or improving task success.

Critical strengths
  • Well-motivated
  • The paper measures end-to-end CUDA latency and evaluates multiple VLA backbones, simulation benchmarks, and physical tasks. This is important for embodied communities.
  • The recovery mechanism is a practical idea for limiting error accumulation over long rollouts. This point is interesting for me.
Critical weaknesses
  • In my understanding, differential token pruning is relatively not a new thing for VLMs and VLAs. Beyond general token pruning methods devised for VLMs, Authors are expected to introduce the difference between your method and latest works, and compare the proposed method with more related and latest works published in 2025 and 2026. Here are some recommendations to refer to [1-3].
  • Code will be open-sourced to ensure its reproducibility?
  • The training objective includes a cache-ratio reward, while the selector and ratio predictor are optimized through straight-through estimates. More evidence is needed that the learned policy remains stable across unseen scenes and different frame rates. For me, there are more latest benchmark beyond the early LIBERO suites and SIMPLER.

[1] EfficientVLA, NeurIPS'25

[2] LightVLA, ICRA'26

[3] Dynamic-LLaVA, ICLR'25

Questions for authors

See above

6 comments
AuthorsOpenReview ↗

Response to Reviewer WNqs (Part 1 of 2: W1–W2)

We thank the reviewer for recognizing that our work is well-motivated, measures genuine end-to-end CUDA latency across multiple VLA backbones and physical tasks, and introduces a practical stochastic recovery mechanism for long-horizon rollouts.


W1: Methodological Differences and Comparisons with Latest 2025–2026 Works (EfficientVLA, LightVLA, Dynamic-LLaVA)

We appreciate the pointers to [1] EfficientVLA (NeurIPS'25), [2] LightVLA (ICRA'26), and [3] Dynamic-LLaVA (ICLR'25). We will cite and discuss all three works in Section 2 and Appendix D.2. Below, we clarify the fundamental methodological distinctions and provide empirical comparisons:

1. Methodological Distinction — Spatial Token Pruning vs. Non-Destructive Temporal KV Caching:

  • Dynamic-LLaVA [3] & LightVLA [2] (Single-Frame Differentiable Token Pruning): Both methods use learnable predictors with Gumbel-Softmax/STE to permanently prune (drop) visual tokens spatially within a single forward pass to shorten sequence length. While effective for static VLM question answering or single-frame compression, spatial dropping is risky in closed-loop manipulation: an ungrasped target receptacle or secondary object that is pruned in early layers is completely inaccessible to deeper Transformer layers. Furthermore, neither method exploits cross-frame KV state reuse or defines a budget predictor + stochastic recovery loop across control steps.

  • EfficientVLA [1] (Training-Free Pruning & Diffusion Step Caching): EfficientVLA accelerates diffusion-based VLAs via a training-free pipeline combining redundant language-layer pruning, task-relevance/diversity visual token pruning, and intermediate diffusion denoising feature caching. Its visual token selection relies on static attention/similarity heuristics decoupled from the action loss LVLA\mathcal{L}_{\text{VLA}}, and its diffusion-step caching is specific to iterative denoising heads rather than general cross-frame visual prefilling.

  • Ours (LAC — Task-Learned Cross-Frame KV Reuse with Stochastic Refresh): Instead of discarding spatial tokens, LAC maintains a full-context cross-frame KV cache (NN tokens) at every layer and learns which token states to refresh (fself_{\text{sel}}) and what global budget to allocate (fpredf_{\text{pred}}) based on Vt=[It;Ot]V_t = [I_t; O_t] and ∇LVLA\nabla \mathcal{L}_{\text{VLA}}, protected by Stochastic Recovery (precoverp_{\text{recover}}). Thus, static objects remain continuously visible in the attention context while active manipulation regions are freshly recomputed.

2. Empirical Comparison on LIBERO (OpenVLA Backbone):

As shown in Table 1 and our extended comparisons with recent 2025–2026 VLA accelerators (including SP-VLA [ICLR'26] and FLASH-VLA on the exact same OpenVLA backbone across the 4 LIBERO suites):

  • Baseline (OpenVLA): 75.0% average success | 1.00×1.00\times speedup (51.9 ms)
  • SparseVLM: 64.7% average success | 0.62×0.62\times speedup
  • FastV: 73.3% average success | 0.97×0.97\times speedup
  • FastVLM: 66.65% average success | 1.16×1.16\times speedup
  • VisionZip: 33.05% average success | 1.21×1.21\times speedup
  • FLASH-VLA: 73.7% average success | 1.56×1.56\times speedup
  • VLA-Cache: 74.7% average success | 1.63×1.63\times speedup (31.8 ms)
  • SP-VLA (2026): 74.9% average success | 1.35×1.35\times speedup
  • Ours (LAC): 76.9% average success (+2.0 pp over SP-VLA, +2.2 pp over VLA-Cache) | 1.76×1.76\times speedup (29.5 ms)

Notably, spatial token pruning/merging baselines (VisionZip, FastVLM, SparseVLM) suffer severe accuracy drops on LIBERO-Long (dropping from 53.2% to 4.6%, 36.6%, and 39.4%, respectively) because permanently dropping tokens destroys spatial memory over long-horizon multi-stage tasks, whereas LAC improves LIBERO-Long to 59.2% (+6.0 pp over dense OpenVLA).


W2: Code Release and Reproducibility

Yes, we will fully open-source our code. In addition, as detailed in our earlier responses to Q1–Q2 and Appendix B, we have provided the complete architectural specifications (5-layer CNN, 0.42M parameters), mathematical backward surrogates, and hyperparameter settings to ensure full reproducibility.

AuthorsOpenReview ↗

Response to Reviewer WNqs (Part 2 of 2: W3)

W3: Stability and Generalization of the Learned Caching Policy

Thank you for highlighting the stability and generalization of the learned caching policy. Our design addresses these concerns through task-supervised optimization, observation-dependent compute allocation, and continual cache refresh, supported by the following results:

1. Training and Inference Stability:

We optimize LVLA−λE[r]\mathcal{L}_{\mathrm{VLA}} - \lambda \mathbb{E}[r] (Eq. 9), where the cache-ratio reward encourages efficiency while the task loss penalizes reuse decisions that impair action prediction. Before joint end-to-end optimization, we initialize the selector through Stage I attention alignment and keep the pretrained VLA backbone completely frozen throughout both stages. This initialization provides a strong saliency prior: as shown in Table 2, removing Stage I initialization (w/o Stage I Init) reduces LIBERO-Spatial success from 85.6% to 79.2%. During Stage II, Straight-Through Estimation (STE) enables gradient-based training of discrete decisions, whereas at inference, the ratio predictor and token selector execute deterministic hard argmax\mathrm{argmax} and top-k\mathrm{top}\text{-}k decisions without Gumbel sampling noise. Complementarily, Stochastic Recovery separately refreshes a small random subset of cached states at each step; enabling the default precover=10p_{\text{recover}} = 10\\% improves success from 83.4% to 85.6% while increasing latency only from 29.04 ms to 29.32 ms (Table 2 & Appendix Table 6), effectively bounding accumulated cache staleness over long horizons.

2. Adaptation to Unseen Scene Changes:

LAC conditions both token selection and the reuse budget on the current RGB observation and inter-frame optical flow Vt=[It;Ot]V_t = [I_t; O_t]. Consequently, recomputation is dynamically allocated according to observed appearance and motion rather than a fixed spatial mask. Cached tokens remain accessible through their historical KV states across all Transformer layers, and stochastic recovery ensures that previously inactive regions periodically receive fresh computation. Our experiments provide direct evidence beyond a single scene configuration:

  • On SIMPLER Variant Aggregation (Table 3)—which specifically evaluates zero-shot robustness across unseen backgrounds, lighting conditions, table textures, distractors, and camera viewpoints—LAC achieves 63.0% average success versus 61.3% for dense CogAct, together with a 1.41×1.41\times speedup.
  • Physical robot experiments (Table 4 & Appendix D.4) further demonstrate reliable operation under real-world sensing noise, lighting variations, and physical interactions, improving average success from 33.0% to 38.0% (1.15×1.15\times speedup) on single-view tasks and from 50.0% to 60.0% (1.12×1.12\times speedup) in a dual-camera (main + ego-centric wrist camera) setup.

3. Adaptation to Different Operating Frequencies (Frame Rates):

Rather than imposing a fixed reuse schedule, LAC makes a fresh caching decision at each observation. Because inter-frame optical flow Ot=RAFT(It−1,It)O_t = \mathrm{RAFT}(I_{t-1}, I_t) directly captures the displacement magnitude between consecutive observations while RGB ItI_t provides complementary appearance cues, the learned predictor automatically adjusts the reuse budget rt∈Rr_t \in \mathcal{R} as temporal spacing changes, while the selector directs fresh computation toward newly displaced regions. Our evaluations span OpenVLA, CogAct, and π0.5\pi_{0.5} (Tables 1 & 3), which operate under distinct action-chunking horizons and control frequencies and thus expose the caching module to different temporal sampling regimes. Across all three backbones, LAC preserves task success (76.9% vs. 75.0% on OpenVLA; 67.3%/63.0% vs. 64.9%/61.3% on CogAct; 96.4% vs. 96.85% on π0.5\pi_{0.5}) while delivering consistent end-to-end speedups (1.76×1.76\times, 1.41×1.41\times, and 1.22×1.22\times).

4. Benchmark Coverage:

We agree that evaluating on emerging benchmarks can further broaden coverage. Our current evaluation already spans 4 LIBERO suites (40 tasks, 2,000 rollout episodes), 2 SIMPLER protocols (Visual Matching and OOD Variant Aggregation), 3 distinct VLA architectures, and physical robot manipulation, providing complementary support for the adaptability of our learned caching mechanism. We will incorporate this expanded discussion of policy stability and frequency adaptability into the revised manuscript.

Reviewer_WNqsOpenReview ↗

Reply to Authors

I appreciate your detailed reply. If authors add corresponding comparison with more state-of-the-art pruning methods instead of only stating the comparison with VisionZip, FastV and other counterparts (a bit out-of-date), I'm willing to raise my rating.

Best

Reviewer_4fEoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LAC, an adaptive visual token caching method for accelerating the inference of Vision-Language-Action (VLA) models. The method aims to reduce redundant visual computation across consecutive robotic observations and address the limitations of existing heuristic-based token caching strategies in adapting to dynamic scenes. Specifically, LAC introduces two lightweight learnable modules: the Cached Token Selector, which identifies visual tokens that need to be recomputed, and the Cache Ratio Predictor, which dynamically determines the caching ratio. Both modules make decisions based on RGB images and optical flow information, and the caching strategy is jointly optimized using the downstream task loss of the VLA model through a two-stage training procedure. In addition, the authors introduce a stochastic recovery mechanism to mitigate the issue of stale information caused by long-term caching. Experiments are conducted on three VLA models, OpenVLA, π0.5\pi_{0.5}, and CogAct, and evaluated on LIBERO, SIMPLER, and real-world robotic tasks. The results show that LAC can reduce inference overhead while largely maintaining or improving task success rates.

Critical strengths
  • The experimental evaluation is relatively comprehensive and demonstrates the applicability of the method across different VLA architectures. The authors conduct experiments on three VLA models with different action generation mechanisms, namely OpenVLA, π0.5\pi_{0.5}, and CogAct, covering two simulation benchmarks, LIBERO and SIMPLER, as well as real-world robotic tasks. The method is also compared with various token pruning and caching approaches. The experimental results show that LAC achieves a certain degree of inference acceleration across different models and evaluation environments while largely maintaining or improving task success rates.
  • Integrating visual token caching strategies with downstream task objectives offers a certain degree of novelty. Unlike existing caching methods that mainly rely on attention scores or other heuristic rules, LAC directly optimizes token selection and caching ratios through the VLA task loss, allowing caching decisions to better adapt to task requirements.
  • The ablation studies and inference efficiency analyses are relatively thorough. The authors evaluate the effects of the Token Selector, Cache Ratio Predictor, two-stage training, optical flow input, and stochastic recovery mechanism. They also analyze the effects of different caching ratios and recovery probabilities on task performance.
Critical weaknesses
  • There is a substantial discrepancy in inference acceleration between simulation and real-world robotic experiments, without sufficient explanation. With the same OpenVLA backbone, LAC achieves a 1.76× inference speedup on LIBERO, whereas the speedup in real-world robotic experiments is only approximately 1.15× (from 37.38 ms to 32.47 ms). The paper does not sufficiently explain the reasons for this discrepancy.
  • The speedup is significantly affected by the model architecture. The speedup on the π0.5\pi_{0.5} model is only 1.22×, suggesting that the actual acceleration benefits of the proposed method may be significantly influenced by the model architecture and deployment environment. The authors should provide an explanation for these differences.
  • A systematic evaluation of the robustness of the caching strategy in dynamic scenes is lacking. LAC relies on optical flow to identify scene changes and reduces computational overhead by reusing historical KV caches. However, in scenarios where objects themselves are moving rapidly, the caching strategy may fail to correctly identify the tokens that need to be updated, potentially affecting action prediction. The paper lacks direct validation in such scenarios.
Questions for authors
  • The authors describe LAC as a task-aware caching strategy, but its decision modules do not explicitly receive language instructions. For different manipulation tasks in the same visual scene, can the caching strategy effectively distinguish the visual information required by different tasks?
  • The paper considers the Cache Ratio Predictor as one of its core modules. However, in the existing ablation studies, adding this module improves the success rate while also increasing FLOPs and inference latency. Therefore, it remains unclear whether the performance improvement comes from optimizing the dynamic caching ratio or simply from recomputing more tokens. The authors are encouraged to compare dynamic and fixed caching ratio strategies under the same computational budget and show how the caching ratio changes across different stages of a task.
  • Details regarding the training cost of the caching strategy are insufficient.
  • The authors should disclose the GPUs used in the experiments.
8 comments
AuthorsOpenReview ↗

Response to Reviewer 4fEo (Part 1 of 3: W1–W2)

We thank the reviewer for appreciating our comprehensive evaluation across three VLA architectures, the novelty of aligning token caching with downstream task objectives, and our thorough ablations.


W1 & W2: Explanation of Speedup Differences Between Simulation (1.76×1.76\times) and Real World (1.15×1.15\times), and Across Architectures (π0.5\pi_{0.5} at 1.22×1.22\times)

The differences in wall-clock speedup across deployment environments and backbones arise from two quantifiable factors: (1) adaptive safety-first cache ratios learned under different visual/task regimes, and (2) Amdahl's law governing the ratio of visual prefilling cost to fixed non-prefill overheads.

1. Simulation (1.76×1.76\times) vs. Physical Robot (1.15×1.15\times) with OpenVLA:

  • Higher Recomputation Budget in Noisy Real-World Scenes: In LIBERO simulation, background lighting and camera extrinsics are perfectly noiseless, allowing LAC to cache ~40% of visual tokens on average and reduce total FLOPs by 25.3% (1.864T→1.392T1.864\text{T} \to 1.392\text{T}). In physical Franka experiments (Table 4), real-world sensor noise, specular reflections, and workspace clutter induce higher frame-to-frame variation. Consequently, the Cache Ratio Predictor conservatively selects lower cache ratios to safeguard control accuracy, reducing FLOPs by 17.1% (1.893T→1.569T1.893\text{T} \to 1.569\text{T}). Notably, VLA-Cache exhibits the exact same phenomenon on the physical robot, achieving only a 1.12×1.12\times speedup (37.38 ms→33.36 ms37.38\text{ ms} \to 33.36\text{ ms}).

  • Hardware Scaling & Fixed Module Overhead: Simulation profiling was conducted on an NVIDIA A100 (51.91 ms51.91\text{ ms} dense baseline), whereas the real robot was controlled by a local workstation with an NVIDIA RTX 4090 (37.38 ms37.38\text{ ms} dense baseline due to higher single-stream boost clock speeds). Because the fixed overhead of RAFT-small + Selector/Predictor (∼4.4 ms\sim 4.4\text{ ms}) represents a larger fraction of the shorter 37.38 ms37.38\text{ ms} baseline and operates at a more conservative cache ratio, the net end-to-end latency reduction is 37.38 ms→32.47 ms37.38\text{ ms} \to 32.47\text{ ms} (1.15×1.15\times), while simultaneously boosting real-robot success from 33.0% to 38.0% (+8.0 pp over VLA-Cache).

2. OpenVLA (1.76×1.76\times) vs. π0.5\pi_{0.5} (1.22×1.22\times) on LIBERO:

  • Conservative Caching to Preserve Near-Ceiling Accuracy: On LIBERO, the dense π0.5\pi_{0.5} baseline achieves an exceptionally high 96.85% average success rate (Table 1). Because aggressive approximation risks degrading fine-grained trajectory precision at this performance ceiling, joint optimization with LVLA\mathcal{L}_{\text{VLA}} converges to a conservative operating point that retains 88.1% of normalized FLOPs (compared to 74.7% on OpenVLA), successfully preserving a 96.4% success rate.

  • Action-Chunking Flow-Matching Decoder Cost: Unlike OpenVLA, which autoregressively decodes a single 7-DoF action step, π0.5\pi_{0.5} employs a flow-matching action expert that performs multi-step iterative velocity integration over an action chunk of 10 steps. While LAC accelerates the visual conditioning prefix, the multi-step flow-matching ODE solver incurs a constant output-side cost per query. Under Amdahl's law, an 11.9% total FLOPs reduction with visual KV reuse yields the measured 1.22×1.22\times end-to-end speedup. We will include this detailed architectural breakdown in Section 4.1.

AuthorsOpenReview ↗

Response to Reviewer 4fEo (Part 2 of 3: W3 & Q1)

W3: Robustness of the Caching Strategy in Dynamic Scenes with Rapidly Moving Objects

We address the robustness of LAC under rapid object motion from three perspectives:

1. Direct Validation in Dynamic Object Manipulation Tasks:

Our evaluation explicitly includes tasks characterized by rapid object motion and impulsive contact dynamics—most notably KnockCrisp and KnockBottle on the physical robot (Table 4) and MoveNear / DrawerApple in SIMPLER (Table 3), where objects undergo rapid tipping, rolling, or sliding upon contact. Rather than failing when objects move rapidly, LAC achieves its largest performance improvement on KnockBottle (64.0% vs. 44.0% for dense OpenVLA and 40.0% for VLA-Cache).

2. Why Rapid Object Motion Triggers Recomputation Rather Than Cache Staleness:

In LAC, tokens are cached only when their predicted saliency score st(i)s_t^{(i)} falls in the lowest kt=N⋅rtk_t = N \cdot r_t percentile. When an object moves rapidly, it generates a strong optical flow magnitude ∣Ot∣2\\|O_t\\|_2 across both its current and displaced spatial patches:

  • First, the Cached Token Selector takes Vt=[It;Ot]V_t = [I_t; O_t] and assigns high saliency scores st(i)→1s_t^{(i)} \to 1 to high-flow regions, immediately routing those tokens to the active recomputation branch (Mt(i)=1M_t^{(i)}=1).
  • Second, the Cache Ratio Predictor observes the elevated global motion entropy in VtV_t and dynamically lowers the overall cache ratio rtr_t, expanding the active recomputation budget across the frame.
  • Third, Stochastic Recovery (precover=10p_{\text{recover}}=10\\%) continuously refreshes a random subset of cached tokens at every step (costing only 0.28 ms0.28\text{ ms}, Appendix Table 6), preventing any newly disturbed region from remaining stale.

3. Graceful Degradation Under Extreme Global Motion:

As shown in Appendix A and D.4, even if optical flow is completely removed (RGB-only LAC in Table 2), the appearance pathway alone still achieves 83.0% success on LIBERO-Spatial; when rapid motion occurs across the scene, LAC automatically shifts toward dense recomputation to prioritize action accuracy.


Q1: Task-Awareness Without Explicit Language Instructions

Please also refer to our detailed response in Part 2, Q3. In summary:

1. How Different Tasks in the Same Scene Are Distinguished:

  • Step t=1t=1 (Full Multimodal Encoding): At the first frame of every episode, no tokens are cached; the VLA encodes all 256 visual tokens together with the language instruction Ilang\mathcal{I}_{\text{lang}}.
  • Steps t≥2t \ge 2 (Instruction-Driven Visuomotor Feedback Loop): In closed-loop manipulation, different language instructions immediately drive the robot arm along different trajectories, creating task-specific motion patterns in OtO_t and local appearance changes in ItI_t around the manipulated target.
  • Non-Destructive Context Retention: Because cached tokens retain their full t=1t=1 KV states in every Transformer layer (unlike token pruning, which deletes tokens), static target objects required by the language instruction remain accessible to cross-attention throughout the episode, and are freshly recomputed as soon as the gripper enters their interaction zone.

2. Direct Empirical Ablation (Motion vs. Language Conditioning):

We tested a Language-Guided Policy variant on LIBERO-Spatial where the decision modules explicitly receive the embedded language instruction alongside the visual input. As reported in our ablation study, the language-guided variant achieves 83.0% success, whereas our motion-guided LAC achieves 85.6% success. Lightweight language conditioning struggles to ground fine-grained spatial token coordinates without incurring heavy cross-modal attention overhead, whereas optical flow provides an exact, pixel-aligned cue of physical interaction at only 3.2 ms3.2\text{ ms}.

AuthorsOpenReview ↗

Response to Reviewer 4fEo (Part 3 of 3: Q2–Q4)

Q2: Dynamic vs. Fixed Caching Ratios Under Matched Computational Budget & Stage-Wise Ratio Evolution

1. Dynamic vs. Fixed Caching Ratio Under the Same Computational Budget:

In Table 2, Selector only uses our default fixed cache ratio of r=40r = 40\\% (1.283T1.283\text{T} FLOPs, 28.48 ms28.48\text{ ms}), whereas + Reuse Predictor dynamically selects rt∈R=0.1,0.2,…,0.8r_t \in \mathcal{R} = \\{0.1, 0.2, \dots, 0.8\\} around 4040\\%. Because the predictor lowers the cache ratio during contact-rich grasping steps, its episode-average cache ratio on LIBERO-Spatial is rˉ=37.8\bar{r} = 37.8\\%, resulting in slightly higher FLOPs (1.325T1.325\text{T}, 29.04 ms29.04\text{ ms}) than the fixed 4040\\% default in Table 2.

To isolate dynamic ratio adaptation from simply recomputing more tokens, we evaluate Fixed Ratio (Selector only) across r \in \\{40\\%, 35\\%, 30\\%\\} to match or exceed the budget of our dynamic predictor on LIBERO-Spatial, both without and with Stochastic Recovery (precover=10p_{\text{recover}} = 10\\%):

Strategy (LIBERO-Spatial)Cache Ratio rtr_tSuccess (%) ↑\uparrowFLOPs (T) ↓\downarrowLatency (ms) ↓\downarrow
Fixed Ratio (Selector only)Fixed 4040\\%82.201.28328.48
Fixed Ratio (Selector only)Fixed 3535\\%82.601.33229.08
Fixed Ratio (Selector only)Fixed 3030\\%82.801.39629.85
Dynamic Ratio (+ Reuse Predictor)Dynamic (Avg. 37.837.8\\%)83.401.32529.04
Fixed Ratio + Stochastic RecoveryFixed 3535\\%84.201.38429.36
Full LAC (Dynamic Ratio + Recovery)Dynamic (Avg. 37.837.8\\%)85.601.37729.32
  • Without Stochastic Recovery (Selector only vs. + Reuse Predictor): Lowering the fixed cache ratio from the default 4040\\% to 3535\\% (1.332T1.332\text{T} FLOPs, 29.08 ms29.08\text{ ms}) to match the budget of + Reuse Predictor (1.325T1.325\text{T} FLOPs, 29.04 ms29.04\text{ ms}) only improves success from 82.2082.20\\% to 82.6082.60\\%, remaining 0.800.80\\% below Dynamic Ratio (+ Reuse Predictor) (83.4083.40\\%). Even at a 3030\\% fixed cache ratio (1.396T1.396\text{T} FLOPs, 29.85 ms29.85\text{ ms}), it only reaches 82.8082.80\\%.

  • With Stochastic Recovery (Fixed Ratio + Stochastic Recovery vs. Full LAC): At the matched budget (1.384T1.384\text{T} vs. 1.377T1.377\text{T} FLOPs, ∼29.3 ms\sim 29.3\text{ ms}), Full LAC (85.6085.60\\%) outperforms Fixed Ratio + Stochastic Recovery (84.2084.20\\%) by +1.40+1.40\\%, confirming that the gain stems from dynamically reallocating computation across task phases rather than recomputing more tokens overall.

2. How the Cache Ratio Changes Across Different Stages of a Task:

Across LIBERO-Spatial rollouts, the Cache Ratio Predictor fluctuates around 4040\\% (episode mean 37.837.8\\%), adapting to each stage:

  • Stage 1 — Free-Space Reaching: While the gripper moves through open space and background objects are stationary, the predictor maintains an above-average cache ratio of 4343\\%\text{–}47\\% (predominantly selecting rt∈0.4,0.5r_t \in \\{0.4, 0.5\\}), reusing static background tokens.

  • Stage 2 — Object Grasping & Contact: As the gripper aligns with the object and closes its fingers—where fine spatial precision around the contact region is critical—the predictor lowers the cache ratio to 3030\\%\text{–}34\\% (rt∈0.3,0.4r_t \in \\{0.3, 0.4\\}) to refresh tokens around the gripper–object interface.

  • Stage 3 — In-Hand Transport & Placement: Once grasped, the cache ratio rises back to 4040\\%\text{–}44\\% during steady transport, then dips to 3232\\%\text{–}35\\% during final placement alignment.

We will include this comparison table and stage-wise trajectory in the revised manuscript.


Q3 & Q4: Training Cost Details and GPU Hardware Disclosure

1. Training Cost of the Caching Strategy:

Because the pretrained VLA backbone (e.g., 7B parameters in OpenVLA) is kept completely frozen, only the lightweight 5-layer CNN Cached Token Selector and Cache Ratio Predictor are trained:

  • Trainable Parameters: 0.42M parameters (<0.006< 0.006\\% of the 7B OpenVLA backbone), requiring <2 MB< 2\text{ MB} of storage.
  • Training Time: On a single NVIDIA A100 (80GB) GPU, Stage I (5 epochs) takes ~2.5 GPU hours and Stage II (10 epochs) takes ~5.5 GPU hours, totaling ~8 GPU hours per suite with a peak memory footprint of 18.4 GB.

2. GPU Hardware Disclosure:

  • Simulation Training & Inference Profiling (LIBERO, SIMPLER): Single NVIDIA A100-SXM4-80GB GPU.
  • Real-Robot Deployment (Franka Emika Panda): Single NVIDIA GeForce RTX 4090 (24GB) GPU.

We will explicitly state these hardware specifications and training costs in Section 4 and Appendix B.

Reviewer_bEjaOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LAC, a learnable visual-token caching framework for accelerating Vision-Language-Action models. A token selector predicts which visual tokens should be recomputed, while a cache-ratio predictor determines how many tokens to reuse. Both modules use RGB observations and optical flow and are trained through attention-based initialization followed by task-loss optimization with differentiable surrogates. Historical KV states are reused for cached tokens, with stochastic recovery to reduce staleness. Experiments cover OpenVLA and π0.5 on LIBERO, CogAct on SIMPLER, and four real-robot tasks.

Critical strengths

Learning caching decisions through the downstream action objective is a useful direction for improving the efficiency–performance trade-off. Separating token selection from reuse-budget prediction provides a practical way to adapt computation, and the component ablations show benefits from initialization, adaptive ratios, and recovery.

The evaluation covers heterogeneous VLA architectures, simulation benchmarks, and physical manipulation. The reported OpenVLA result combines a 1.76× speedup with an increase in average LIBERO success from 75.0% to 76.9%. Appendix C.1 explicitly includes optical-flow and decision-module overheads, which strengthens the efficiency evaluation.

Critical weaknesses

The training procedure lacks essential details, including the task loss for each backbone, training data, cache construction across timesteps, candidate cache ratios, and optimization settings. Equations (9)–(11) do not explain how task-loss gradients propagate through the discrete ratio-to-token-count and top-k operations to train the cache-ratio predictor.

The improvements over VLA-Cache are modest, while their statistical reliability is unclear. The additional speedup is approximately 8% on OpenVLA, 3–5% on CogAct, and 2.7% in real-robot experiments. Success-rate uncertainty and latency variability are not reported. The real-robot improvement amounts to five additional successes across 100 trials, with the entire net gain coming from the bottle task.

Questions for authors

What task loss and training data are used for each backbone? Are the caching modules trained separately for each task, each suite, or jointly across tasks?

How are historical caches constructed during training, and how does the task-loss gradient propagate through the discrete cache-ratio selection and top-k mask?

Since both decision modules receive only RGB and optical flow, how can their caching decisions differ for the same scene under different language instructions?

Are the improvements over VLA-Cache consistent across independent runs? Please report evaluation seed counts, success-rate uncertainty, hardware, and latency variability.

Why is VLA-Cache described as static when it already includes dynamic token selection, task-relevance filtering, and layer-adaptive reuse? What advantage does learned caching provide at matched actual recomputation budgets?

9 comments
AuthorsOpenReview ↗

Response to Reviewer bEja (Part 1 of 3: Q1)

We thank the reviewer for the thorough reading and insightful questions regarding our training formulation, gradient flow, and empirical comparisons.


Q1: Task Loss, Training Data, Granularity, Candidate Ratios, and Optimization Settings

1. Backbone-Specific Task Loss (LVLA\mathcal{L}_{\text{VLA}}) and Training Data:

In Stage II, the VLA backbone remains strictly frozen, and LVLA\mathcal{L}_{\text{VLA}} is the native downstream action supervision loss of each respective architecture:

  • OpenVLA (LIBERO & Real Robot): Cross-entropy loss over discretized 7-DoF action tokens, LVLA=−∑d=17log⁡PVLA(at,d∗∣K~t,V~t,Ilang)\mathcal{L}_{\text{VLA}} = -\sum_{d=1}^{7} \log P_{\text{VLA}}(a_{t,d}^{*} \mid \tilde{\mathbf{K}}_t, \tilde{\mathbf{V}}_t, \mathcal{I}_{\text{lang}}).

We train on the standard LIBERO demonstration splits (50 demonstrations per task) for simulation, and 50 teleoperated demonstrations per task for the physical Franka setup.

  • CogAct (SIMPLER): Denoising score-matching MSE loss of the diffusion action head, \mathcal{L}_{\text{VLA}} = \mathbb{E}_{\epsilon, k}\\!\left[\left\\|\epsilon - \epsilon_\theta(\mathbf{a}_t^{(k)}, k \mid \tilde{\mathbf{K}}_t, \tilde{\mathbf{V}}_t)\right\\|_2^2\right], trained on manipulation trajectories following the official CogAct protocol.

  • π0.5\pi_{0.5} (LIBERO): Conditional flow-matching velocity regression loss over an action chunk of horizon H=10H=10, \mathcal{L}_{\text{VLA}} = \mathbb{E}_{\tau \sim \mathcal{U}(0,1)}\\!\left[\left\\|v_\theta(\mathbf{A}_t^{(\tau)}, \tau \mid \tilde{\mathbf{K}}_t, \tilde{\mathbf{V}}_t) - (\mathbf{A}_t^1 - \mathbf{A}_t^0)\right\\|_2^2\right].

2. Training Granularity (Joint Suite-Level Training):

The caching modules are not trained per individual task. Instead, we train a single unified selector and predictor jointly across each benchmark suite (e.g., all 10 tasks in a LIBERO suite share one LAC policy; all 4 tasks in SIMPLER share one LAC policy). This ensures that the decision modules learn generalizable visuomotor saliency rather than overfitting to a single trajectory layout.

3. Architecture, Candidate Cache Ratios, and Optimization Hyperparameters:

  • Architecture: Both fself_{\text{sel}} and fpredf_{\text{pred}} share a lightweight 5-layer convolutional stem (0.42M trainable parameters in total; channels [32,64,128,128,64][32, 64, 128, 128, 64], 3×33\times 3 kernels with stride 2 and BatchNorm/GELU) operating on the 5-channel input Vt=[It;Ot]∈RH×W×5V_t = [I_t; O_t] \in \mathbb{R}^{H\times W\times 5}, followed by a 1×11\times 1 projection head for token saliency St∈[0,1]NS_t \in [0,1]^N (N=256N=256) and a global average pooling + 2-layer MLP head for ratio logits Lt∈RCL_t \in \mathbb{R}^C.

  • Candidate Ratio Set: We use C=8C=8 discrete candidate cache ratios R=0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8\mathcal{R} = \\{0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8\\}.

  • Optimization Settings: We use AdamW (weight decay 10−410^{-4}, batch size 16 trajectory snippets). Stage I trains fself_{\text{sel}} for 5 epochs with learning rate 3×10−43\times 10^{-4} using Lalign\mathcal{L}_{\text{align}} against the layer-averaged cross-attention map from the action query tokens to visual tokens. Stage II jointly optimizes fself_{\text{sel}} and fpredf_{\text{pred}} for 10 epochs with learning rate 1×10−41\times 10^{-4}, cosine annealing, trade-off weight λ=0.05\lambda = 0.05, and Gumbel/sigmoid temperatures annealed from 1.01.0 to 0.20.2.

AuthorsOpenReview ↗

Response to Reviewer bEja (Part 2 of 3: Q2–Q3)

Q2: Historical Cache Construction & Gradient Propagation (Equations 9–11)

1. Online Historical Cache Construction Across Timesteps:

During Stage II training, we sample contiguous T=8T=8-frame windows from the demonstration buffer and unroll sequentially: at t=1t=1, all NN visual tokens are encoded to initialize the layer-wise cache K1l,V1ll=1L\\{\mathbf{K}_1^l, \mathbf{V}_1^l\\}_{l=1}^L; at each step t∈2,…,Tt \in \\{2, \dots, T\\}, the binary mask Mt∈0,1N\mathbf{M}_t \in \\{0,1\\}^N (11: active, 00: reuse) routes active tokens through the Transformer while retrieving cached entries from t−1t-1.

2. Differentiable Coupling Between Ratio Predictor and top-k Mask:

To allow ∇LVLA\nabla \mathcal{L}_{\text{VLA}} to update both θsel\theta_{\text{sel}} and θpred\theta_{\text{pred}}, backpropagation uses two differentiable bridges:

  • Step A (Coupling p~t\tilde{p}_t to the top-k threshold): For each candidate ratio rj∈Rr_j \in \mathcal{R}, let kj=⌊N⋅rj⌋k_j = \lfloor N \cdot r_j \rfloor and let θkj=sg(St(kj))\theta_{k_j} = \mathrm{sg}(S_t^{(k_j)}) denote the kjk_j-th lowest score in StS_t. Instead of evaluating Eq. (11) at a detached scalar θk\theta_k, the backward surrogate forms an expected soft mask weighted by the Gumbel-Softmax distribution p~t\tilde{p}_t: M~t(i)=∑j=1Cp~t(j),σ!(st(i)−θkjτs),M^t=Mt+M~t−sg(M~t).\tilde{M}_t^{(i)} = \sum_{j=1}^{C} \tilde{p}_t^{(j)} \\, \sigma\\!\left(\frac{s_t^{(i)} - \theta_{k_j}}{\tau_s}\right), \qquad \hat{M}_t = M_t + \tilde{M}_t - \mathrm{sg}(\tilde{M}_t).

  • Step B (Coupling M^t\hat{M}_t to KV Cache Update): At layer ll, token ii's merged KV state is a convex combination: K~tl,(i)=M^t(i)Kt,newl,(i)+(1−M^t(i))Kt−1l,(i),V~tl,(i)=M^t(i)Vt,newl,(i)+(1−M^t(i))Vt−1l,(i).\tilde{\mathbf{K}}_t^{l,(i)} = \hat{M}_t^{(i)} \mathbf{K}_{t,\text{new}}^{l,(i)} + (1 - \hat{M}_t^{(i)}) \mathbf{K}_{t-1}^{l,(i)}, \quad \tilde{\mathbf{V}}_t^{l,(i)} = \hat{M}_t^{(i)} \mathbf{V}_{t,\text{new}}^{l,(i)} + (1 - \hat{M}_t^{(i)}) \mathbf{V}_{t-1}^{l,(i)}.

Thus, the differentiable mask propagates LVLA\mathcal{L}_{\text{VLA}} gradients to both modules. The cache-ratio predictor's gradient is ∂LVLA∂lt(j)=∑i=1N∑q=1C∂LVLA∂M^t(i)∂M~t(i)∂p~t(q)∂p~t(q)∂lt(j)\frac{\partial \mathcal{L}_{\text{VLA}}}{\partial l_t^{(j)}} = \sum_{i=1}^{N}\sum_{q=1}^{C} \frac{\partial \mathcal{L}_{\text{VLA}}}{\partial \hat{M}_t^{(i)}} \frac{\partial \tilde{M}_t^{(i)}}{\partial \tilde{p}_t^{(q)}} \frac{\partial \tilde{p}_t^{(q)}}{\partial l_t^{(j)}}, where summing over qq captures the coupling among all candidate probabilities through the Gumbel-Softmax normalization. This gradient provides task-dependent feedback on the reuse budget, balancing the efficiency incentive from λLratio\lambda \mathcal{L}_{\text{ratio}} against action-prediction accuracy. We will expand Section 3.5 with this derivation.

Q3: How Caching Decisions Adapt Across Tasks Without Explicit Language Input

1. Why "Task-Aware" Refers to Task-Objective Optimization Rather Than Text Input:

Here, task-aware contrasts with task-agnostic image heuristics relying on raw pixel differences or pre-trained vision self-attention without control supervision. LAC's task awareness has three sources:

  • Closed-Loop Visuomotor Embodiment: At t=1t=1, all visual tokens are fully encoded with the language instruction Ilang\mathcal{I}_{\text{lang}} (zero caching). Once the VLA decodes the initial action a1a_1 conditioned on Ilang\mathcal{I}_{\text{lang}}, the gripper moves toward the instruction-specific target. For t≥2t \ge 2, different instructions in the same initial scene induce distinct end-effector trajectories and local optical-flow fields OtO_t around the target workspace.

  • Non-Destructive KV Preservation of Static Targets: Unlike token pruning (which permanently deletes unchosen tokens), LAC retains the full t=1t=1 KV states of all static objects in the cache across layers. Even if a static destination object is cached during initial reaching, the language-conditioned decoder attends to its stored KV state at every layer, and once the gripper approaches it, local motion and appearance interaction trigger active recomputation.

  • Stage I Distillation & Stage II Task Gradients: fself_{\text{sel}} is distilled from SVLAS_{\text{VLA}} (computed inside the language-conditioned VLA) and optimized via LVLA\mathcal{L}_{\text{VLA}}, teaching the CNN appearance pathway (ItI_t) to assign higher baseline saliency to manipulable foreground objects than to background surfaces.

2. Empirical Comparison with a Language-Guided Variant:

We evaluated a Language-Guided Policy variant on LIBERO-Spatial that injects the language embedding (via lightweight FiLM conditioning) into the selector and predictor alongside VtV_t. This variant achieves 83.0% success, underperforming our optical-flow + RGB formulation (85.6%). Heavy cross-modal fusion inside the per-frame selector adds latency overhead that defeats acceleration, whereas lightweight text conditioning provides weaker spatial localization than pixel-level optical flow OtO_t.

AuthorsOpenReview ↗

Response to Reviewer bEja (Part 3 of 3: Q4–Q5)

Q4: Comparison with VLA-Cache, Real-Robot Gains, Hardware, and Statistical Reliability

1. Performance Gains Over VLA-Cache Across Benchmarks and Tasks:

We respectfully clarify a key comparison in Table 4 (Real Robot): the reviewer noted a net gain of 5 successes coming from the bottle task, which compares LAC (38.0%) against the dense OpenVLA baseline (33.0%). When compared against the accelerated baseline VLA-Cache (30.0%), LAC achieves an +8.0 percentage point improvement (8 additional successes across 100 trials) and outperforms VLA-Cache on three out of four physical tasks:

  • KnockCrisp: 52.0% (LAC) vs. 48.0% (VLA-Cache) [+4.0 pp]
  • CoverBanana: 24.0% (LAC) vs. 20.0% (VLA-Cache) [+4.0 pp]
  • KnockBottle: 64.0% (LAC) vs. 40.0% (VLA-Cache) [+24.0 pp]
  • PickMango: 12.0% (LAC) vs. 12.0% (VLA-Cache) [tied]

Similarly, on LIBERO (Table 1), while VLA-Cache degrades average success relative to dense OpenVLA (75.075.0\\% \to 74.7\\%), LAC improves average success to 76.9% (+2.2 pp over VLA-Cache) and achieves a +6.4 pp gain on the hardest long-horizon suite, LIBERO-Long (59.2% vs. 52.8%), while simultaneously reducing latency from 31.83 ms31.83\text{ ms} (1.63×1.63\times) to 29.51 ms29.51\text{ ms} (1.76×1.76\times).

2. Evaluation Protocol, Hardware Disclosure, and Variability:

  • Hardware: All simulation benchmarks and latency profiling were conducted on a single NVIDIA A100 (80GB) GPU with PyTorch FP16/BF16 inference and CUDA event synchronization. Physical Franka robot experiments were executed on a local workstation equipped with an NVIDIA RTX 4090 (24GB) GPU (whose higher single-stream boost clock speed explains the shorter dense baseline latency of 37.38 ms37.38\text{ ms} vs. 51.91 ms51.91\text{ ms} on A100).

  • Evaluation Trials & Consistency: Following standard LIBERO protocols, each suite comprises 10 tasks evaluated over 50 rollout episodes per run (500 episodes per suite, 2,000 episodes across the 4 suites) across 3 independent random seeds. Across seeds, LAC exhibits low variance (±0.4\pm 0.4\\% average success std on LIBERO and ±0.3 ms\pm 0.3\text{ ms} CUDA latency std), consistently outperforming VLA-Cache across all runs. We will add the full uncertainty intervals to Tables 1–4.


Q5: Characterization of VLA-Cache and Advantage at Matched Recomputation Budgets

1. Refining the Characterization of VLA-Cache:

We agree with the reviewer that describing VLA-Cache simply as "static" is imprecise, as VLA-Cache performs dynamic token selection via patch similarity, attention-based task-relevance filtering, and layer-adaptive reuse ratios. Our intended distinction is between hand-crafted heuristic rules (whose selection criteria and thresholds are fixed prior to deployment and decoupled from LVLA\mathcal{L}_{\text{VLA}}) and end-to-end learnable policies. We will revise Sections 1 and 2 to accurately describe VLA-Cache as a training-free heuristic caching method with hand-crafted similarity and attention rules.

2. Advantage of Learned Caching Under Matched Recomputation Budgets:

Table 1, Table 3, and Figure 4 directly demonstrate LAC's advantage under matched actual recomputation budgets:

  • Matched Suite-Level Budgets (Tables 1 & 3): LAC operates at nearly identical FLOPs to VLA-Cache (74.7% vs. 72.7% on LIBERO; 81.8% vs. 81.0% on SIMPLER Visual Matching), yet achieves both higher success rates (+2.2 pp on LIBERO, +6.4 pp on LIBERO-Long, +1.1 pp on SIMPLER) and lower wall-clock latency (29.51 ms29.51\text{ ms} vs. 31.83 ms31.83\text{ ms} on LIBERO; 37.82 ms37.82\text{ ms} vs. 39.63 ms39.63\text{ ms} on SIMPLER) because LAC makes one frame-level decision via a 1.2 ms CNN rather than computing layer-wise attention-ranking heuristics inside the Transformer forward pass.

  • Controlled Token-Count Sweep (Figure 4, LIBERO-Spatial at 50, 100, and 200 cached tokens out of 256): Across every matched token budget in Figure 4, LAC consistently achieves a higher success rate and lower CUDA latency than VLA-Cache, with the performance gap widening at 200 cached tokens (77.0% for LAC vs. 68.4% for VLA-Cache, +8.6 pp). As visualized in Figure 3, heuristic filtering in VLA-Cache repeatedly caches the initially static target basket and fails to refresh it when the gripper approaches for placement, whereas LAC's task-loss-trained selector preserves critical interaction cues.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chairs and Reviewers,

We thank the reviewers for recognizing LAC's strengths: (1) learning token caching via VLA task losses, (2) end-to-end CUDA latency profiling across OpenVLA, OpenVLA-OFT, CogAct, π0.5\pi_{0.5}, and physical robots, and (3) stochastic recovery to bound staleness. Below we summarize our new results and revisions:

1. New Experiments & Results (Tables 1–4, 7; App. D–E)

  • 4th Backbone (OpenVLA-OFT) & Five 2025–2026 SOTA Baselines: Tables 1 & 3 add OpenVLA-OFT (8-step chunk, dual-camera, 2,000 trials) and five 2025–2026 methods (Dynamic-LLaVA [ICLR'25], SP-VLA [ICLR'26], LightVLA [ICRA'26], SpecPrune-VLA [ICML'26], MoLe-VLA [AAAI'26]), plus EfficientVLA [NeurIPS'25] (Sec. 2, App. D.2):

    • LIBERO / OpenVLA: LAC gets 76.9% Avg. SR (+1.9 pp vs. dense, +2.2 pp vs. VLA-Cache, +2.0 pp vs. SP-VLA [74.9%, 1.35x], +13.0 pp vs. Dynamic-LLaVA [63.9%, 1.07x]) and 1.76x speedup (29.51 ms), with +6.4 pp vs. VLA-Cache and +5.0 pp vs. SP-VLA on LIBERO-Long (59.2%).
    • LIBERO / OpenVLA-OFT: LAC reaches 97.60% Avg. SR (vs. 96.85% dense) and 1.66x speedup, beating LightVLA (96.25%, 1.54x; +1.35 pp) and SpecPrune-VLA (96.10%, 1.46x; +1.50 pp) on all 4 suites (+2.0 pp on Long: 95.6% vs. 93.6%).
    • SIMPLER (VM & VA) / CogAct: LAC is the only method beating dense CogAct in both VM (75.5% vs. 74.8% dense, 74.2% SpecPrune-VLA [1.28x], 71.9% MoLe-VLA; 1.44x) and VA (63.0% vs. 61.3% dense, 61.2% MoLe-VLA; 1.41x).
  • Zero-Shot Cross-Suite Transfer: Trained only on LIBERO-Spatial and tested zero-shot on unseen Object (96.8%), Goal (97.8%), and Long (94.4%), LAC (Spatial-trained) gets 97.00% Avg. SR and 1.58x speedup on OpenVLA-OFT, beating suite-adapted LightVLA and SpecPrune-VLA.

  • Matched-Budget & Visuomotor Task-Awareness: (1) At matched compute on LIBERO-Spatial (approximately 1.33T/1.38T FLOPs, approximately 29.0–29.3 ms; Tables 2, 7), dynamic ratios beat fixed r\in\\{40\\%,35\\%,30\\%\\} without (83.4% vs. 82.6%) and with Recovery (85.6% vs. 84.2%, +1.4 pp) across Reaching (43–47%)→\toGrasping (30–34%)→\toTransport/Placement (40–44%$\to$32–35%). (2) We refine "task-aware" (Abstract, Sec. 1, 3.3) to control-objective-driven (visuomotor task-aware) (via LVLA\mathcal{L}_{\text{VLA}}, no direct language input to policy): at t=1t=1, all 256 tokens encode with IlangI_{\text{lang}}; for t≥2t\ge 2, closed-loop motion induces task-specific flow OtO_t while KV cache keeps static context. Language-Guided Policy (FiLM) gets 83.0% vs. 85.6% for RGB+flow (Table 2).

  • Dynamic-Scene Robustness & Boundary (App. A, D.1, E.4): Under pre-contact motion in ego-centric wrist views on dual-camera Franka (N=512N=512, App. D.1), LAC refreshes the fast wrist view while caching the static 3rd-person view (SR: 50.0% to 60.0%, +10.0 pp, 1.13x), matching dual-camera 8-step chunks on OpenVLA-OFT (97.60%, 1.66x). App. A & E.4 distinguish robot/ego-centric dynamics from external targets.

2. Methodological & Analytical Revisions

  • Training & Counterfactual STE Gradient: Sec. 3.4–3.5 & App. B.2 detail LVLA\mathcal{L}_{\text{VLA}} (Eqs. 11–13), T=8T=8 rollouts, suite-level training, R=0.1,…,0.8\mathcal{R}=\\{0.1,\dots,0.8\\}, and soft mask M~t(i)\tilde{M}_t^{(i)} (Eqs. 8–10). Skipping is inference-only (Sec. 3.6); in training, Kt,newl,Vt,newlK_{t,\text{new}}^l, V_{t,\text{new}}^l are computed for all NN tokens and merged via STE M^t(i)=Mt(i)+M~t(i)−sg(M~t(i))\hat{M}_t^{(i)} = M_t^{(i)} + \tilde{M}_t^{(i)} - \text{sg}(\tilde{M}_t^{(i)}). Forward matches sparse inference (M^t(i)=0\hat{M}_t^{(i)}=0 for reused); backward gives an exact counterfactual gradient via (Kt,newl,(i)−Kt−1l,(i))(K_{t,\text{new}}^{l,(i)} - K_{t-1}^{l,(i)}), penalizing stale reuse.

  • Gains Over VLA-Cache & Reliability: (1) Sec. 1–2 refine VLA-Cache to training-free heuristic. (2) While 3 saturated suites dilute the mean (+2.2 pp, >5σ>5\sigma), LAC gets +6.4 pp on LIBERO-Long (59.2% vs. 52.8%, 16σ16\sigma), +8.6 pp at 200 cached tokens (77.0% vs. 68.4%, Fig. 4), and +8.0 pp on physical tasks (38.0% vs. 30.0%, 3/4 tasks, +24.0 pp on KnockBottle: 64.0% vs. 40.0%, Table 4). (3) Above OpenVLA's approximately 18–20 ms ViT+AR floor, saving 2.32 ms cuts prefill cost via a 1.2 ms CNN (1.63x to 1.76x). (4) 3-seed std: ±0.4\pm 0.4\\% SR, ±0.3 ms\pm 0.3\text{ ms}.

  • Speedup Scaling, Cost & Code: (1) Sim (1.76x, A100, 51.91 ms) vs. Real (1.15x, RTX 4090, 37.38 ms): noise prompts a 17.1% FLOPs cut (vs. 25.3% sim) for +8.0 pp SR over VLA-Cache (1.12x), with approximately 4.4 ms flow+policy a larger share of 37.38 ms. (2) π0.5\pi_{0.5} (1.22x, multi-step ODE, 88.1% FLOPs) vs. parallel OpenVLA-OFT (1.66x; Sec. 4.1–4.2). (3) Cost: Frozen VLA; trains 0.42M params (less than 2 MB) in approximately 8 A100 GPU hours per suite (18.4 GB VRAM; App. B.2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.