acceptodds
Under review as a conference paper at ICLR 2027

MemCache: Memory-Augmented Adaptive Token Caching for Efficient Vision-Language-Action Inference

Abstract

Vision-Language-Action (VLA) policies facilitate generalized robotic manipulation by mapping visual scenes and task instructions to low-level robotic actions. Nevertheless, their heavy computations fail to meet the stringent real-time control constraints and limited computational budgets of physical robots. As an efficient inference paradigm, VLA caching alleviates computational burdens by exploiting temporal redundancy across consecutive robotic observations and reusing static token states. However, existing methods exhibit several limitations: their caching decisions generally depend on hand‑crafted proxy signals often misaligned with control policy demands, and they rely only on adjacent frames without exploiting longer-term temporal context. To this end, we present MemCache, a novel learnable memory-augmented caching framework for high-efficiency and reliable VLA inference. Specifically, we leverage learnable query banks to generate token‑level importance scores. This eliminates dependence on hand‑crafted proxy supervision and externally defined signals, enabling the token‑selection strategy to be fully optimized with the control objective. Additionally, we integrate a recurrent visual memory state to capture long-range temporal dependencies in manipulation behaviors, providing history-aware corrections for caching decisions. We further develop a linear-warmup training schedule to coordinate the above two components and facilitate joint optimization. Equipped with these merits, MemCache delivers reliable and efficient VLA inference for robotic manipulation tasks, e.g., higher task performance with speedups over VLA caching baselines on the LIBERO benchmark.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision‑Language‑Action, robotic manipulation, token caching, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ufQbOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MemCache, a learnable token-caching framework for efficient Vision-Language-Action inference. It combines query-based token selection with a recurrent visual memory that refines caching decisions using historical context. A memory-consistency objective and warm-up schedule support joint optimization. Experiments on LIBERO and SIMPLER evaluate the framework across OpenVLA, OpenVLA-OFT, and CogACT, reporting reduced inference latency while maintaining or improving task success rates.

Critical strengths

The paper addresses a practically relevant problem by exploiting temporal redundancy to reduce VLA inference costs. The framework learns caching decisions under the action objective, and its recurrent memory provides a plausible mechanism for incorporating historical context. Evaluation covers three VLA architectures and two simulation benchmarks. The single-GPU latency measurements provide useful evidence of practical acceleration.

Critical weaknesses

W1. Insufficient evidence for the central contribution of long-term memory. The ablation results do not convincingly establish the benefit of long-term temporal context. In Appendix Table 7, adding memory increases average success rates only from 76.3% to 76.5% for OpenVLA and from 97.3% to 97.6% for OpenVLA-OFT. On the OpenVLA Long suite, performance decreases from 56.8% to 56.2%. Although the full framework performs better after introducing the training strategy, these results do not isolate the contribution of long-term memory. Comparisons involving memory resets, different history lengths, and simpler temporal aggregation mechanisms are missing. W2. The efficiency comparison with token-pruning methods is incomplete. Table 2 compares pruning and caching ratios, but these quantities are not directly equivalent measures of computational savings. Caching retains the full KV context, whereas pruning shortens the token sequence. A higher caching ratio therefore does not necessarily imply lower FLOPs or latency than a lower pruning ratio. Without measured latency and FLOPs for these baselines, the table does not sufficiently substantiate the claimed superiority in the performance–efficiency trade-off. W3. Limited validation under real-world conditions and cache-failure scenarios. All experiments are conducted in simulation, including those on SIMPLER. The evaluation does not systematically examine viewpoint changes, occlusions, abrupt scene changes, or prolonged reuse of stale cached states. These conditions are particularly relevant to a method that relies on temporal redundancy and historical KV reuse. Consequently, the robustness of the caching mechanism under practical deployment conditions remains insufficiently established.

Questions for authors

Q1 Can you isolate the benefit of long-term memory by comparing the full method with periodic memory resets, different history lengths, and simple alternatives such as exponential moving averages or short-window feature aggregation? Does the recurrent memory provide consistent improvements when caching budgets and training schedules are matched? Q2 Can you report measured inference latency and FLOPs for the pruning baselines in Table 2 under the same hardware and timing protocol? Do the claimed advantages remain when methods are compared at matched latency or computational budgets rather than token ratios? Q3 How does MemCache behave under viewpoint changes, temporary occlusions, abrupt object displacement, and extended cache reuse? Is there a mechanism to detect or refresh stale cached states, and can its effectiveness be evaluated through controlled perturbation experiments or real-robot trials?

8 comments

We thank the reviewer for the insightful and constructive comments. As suggested, we have: (1) Compared our method against memory-reset variants over a range of history lengths and against a simpler temporal aggregation mechanism (EMA), demonstrating the importance of long-term memory for caching decisions and our method's ability to retain it (Contribution of long-term memory, W1&Q1);

(2) Added comparisons of FLOPs and latency speedup against token-pruning baselines to validate our superior performance–efficiency trade-offs (Efficiency comparison with token-pruning methods, W2&Q2);

(3) Conducted real-robot experiments to demonstrate the effectiveness of our method in real-world applications, and conducted experiments on the extended LIBERO benchmarks, including LIBERO-Occ and LIBERO-Plus, to validate our effectiveness in robustness under temporary occlusions and different types of perturbations (Real-robot & Robustness experiments, W3&Q3).

W1&Q1: Insufficient evidence for the central contribution of long-term memory

To validate the long-term contribution of our memory mechanism, we compare our method with memory-reset variants across different history lengths. To further verify the effectiveness of our memory update mechanism, we compare it with a simpler temporal aggregation mechanism, the exponential moving average (EMA) [1].

We evaluate on two LIBERO benchmark suites [2], Spatial and Long: the former includes an average of 104 steps per episode, whereas the latter has averaging 280 steps per episode, enabling us to assess performance under both short- and long-horizon regimes.

We evaluate both OpenVLA [3], in which each frame prefill yields a single action step, and OpenVLA-OFT [4], which uses action chunking to produce eight steps per frame prefill. All comparisons are conducted with identical training schedules and caching budgets, except for the base models, which are not cached.

MethodOpenVLA SpatialOpenVLA LongOpenVLA-OFT SpatialOpenVLA-OFT Long
Base Model85.2%53.4%98.20%94.60%
+ Queries85.4%56.8%98.6%95.0%
+ Memory (2 frames)85.6%57.2%98.8%95.4%
+ Memory (8 frames)86.2%57.4%99.0%95.6%
+ Memory (32 frames)86.4%58.2%--*95.9%
EMA (Full, α=0.2)83.6%57.9%99.0%95.7%
EMA (Full, α=0.5)84.4%57.1%98.8%92.9%
EMA (Full, α=0.8)87.2%58.1%97.6%95.3%
Ours (Full)87.6%59.0%99.2%96.2%

*For OpenVLA-OFT (action chunk size 8), 32 frames yield 256 steps, exceeding the horizon of LIBERO-Spatial.

As shown in the table, resetting every two frames restricts our memory state to only adjacent frames. Consequently, its success rate is comparable (0.2%–0.4% higher) to that of query-based methods (no memory). Resetting every 8 frames improves the success rate by 0.4%–0.8%, while resetting every 32 frames endows our caching decisions with longer-term memory, further improving the success rate by 0.9%–1.4% over the query-based variant. Our method without reset achieves the best success rate (0.6%–2.2% over the query-based variant), demonstrating both the importance of long-term memory for caching decisions and our method's ability to retain it effectively.

Furthermore, compared with EMA, which mixes historical memory and the current observation in a proportional manner (new memory = α × history + (1 − α) × current) and therefore depends on the choice of α, our method consistently achieves higher success rates under the same caching ratio. This verifies the effectiveness of our memory updating mechanism, which learns end-to-end to refresh the memory state, aiming to accumulate useful historical context and enable better caching decisions.

W2&Q2: Efficiency comparison with token-pruning methods

We have added comparisons of FLOPs and latency speedup against token-pruning baselines. Since IAprune [5] is not open-sourced, we take its latency results directly from its paper. We measure latency for the remaining open-source baselines (VLA-ADP [6] and SpecPrune-VLA [7]) using their official repositories, and for our method, under the same evaluation protocol. All evaluations run on the same GPU type (NVIDIA A100) as IAprune, so that the results are directly comparable.

MethodSpatialObjectGoalLongAvg.Ratio↑FLOPs (T)↓Latency (ms)↑
OpenVLA-OFT97.897.697.694.296.80.04.013112.35
VLA-ADP99.498.096.491.296.30.503.21596.03
VLA-ADP97.698.497.484.294.40.702.92592.14
SpecPrune-VLA97.495.897.793.496.1–1.726105.91
EfficientVLA96.591.196.072.188.90.70––
IAprune97.399.198.295.297.50.50–89.93 (Paper)
IAprune96.698.898.094.897.10.70–80.00 (Paper)
Ours99.298.898.096.298.10.751.66083.16

The new results show that our method still achieves a favorable performance–efficiency trade-off. Specifically, on LIBERO with OpenVLA-OFT, our method surpasses IAprune by an average of 0.6% in success rate at a 1.1× speedup, and by 1.0% in success rate at comparable latency (83.16 ms vs. 80.0 ms). We also achieve a 2.0% higher success rate with a 1.27× speedup over SpecPrune-VLA. This is because its dynamic operations, including multi-stage token selection and explicit attention-map computation, introduce additional computation, data movement, and synchronization overhead, which offset the latency savings from FLOPs reduction.

Moreover, to enable a more direct and controlled comparison between caching and pruning, we conduct an additional experiment in which we replace our caching operation with pruning at inference time while keeping all other components and settings unchanged. At the same computational ratio of 0.25, caching incurs only 3% higher latency and 1% more FLOPs than pruning.

We have incorporated the new results into the revised manuscript (Table 4 and Section 4.3).

W3&Q3: Limited validation under real-world conditions and cache-failure scenarios

Mechanism to detect or refresh stale cached states

We learn a memory update module that adopts a preserve gate ztz_t to refresh stale memory states. Formally, the module reads the previous memory state Mt−1M_{t-1} together with the current observation StS_t and predicts a per-entry gate ztz_t and a candidate update Mt′M_t' (Sec. 3.3.1 "Recurrent State Update", lines 238–247 of P5 in our manuscript):

[at,Mt′]=fupd([LN(Mt−1),LN(St)]),zt=σ(at)[a_t, M_t'] = f_{upd}([LN(M_{t-1}), LN(S_t)]),\quad z_t=\sigma(a_t)

Mt=zt⊙Mt−1+(1−zt)⊙tanh⁡(Mt′)M_t = z_t\odot M_{t-1}+(1-z_t)\odot\tanh(M_t')

Since σ()\sigma() and tanh()tanh() act element-wise, zt∈RK×dz_t\in\mathbb{R}^{K\times d} sets the retention strength of each memory entry independently: larger values preserve historical entries, smaller ones replace them with the candidate state derived from the current observation. Since the gate is conditioned jointly on Mt−1M_{t-1} and StS_t, the update is both content-dependent and per-entry. Staleness is thus handled implicitly by the learned gate rather than by a hand-designed detector: entries that the current observation no longer supports are overwritten, with no explicit staleness signal required.

Limited validation under real-world conditions

We extend our evaluation to a physical Realman manipulator across four manipulation tasks to assess performance under real-world conditions. We conduct experiments on PickCube, KnockBottle, PickBox, and PutCup, collecting 150–200 trajectories per task to fine-tune OpenVLA-OFT with LoRA and validating on 25 trials per task. As detailed in the table, our method outperforms the baseline in both task performance and inference efficiency, improving the average success rate by 2.0% over the base OpenVLA-OFT, while reducing FLOPs by 49.8% and attaining a 1.53× inference speedup on an NVIDIA RTX 4090, demonstrating our benefits in real-world deployment.

MethodPickCubeKnockBottlePickBoxPutCupAverage ↑FLOPs (T) ↓Latency (ms) ↓
OpenVLA-OFT88%72%92%80%83%3.96076.16
Ours92%76%96%76%85%1.98649.71

To further assess the robustness of our method, we test both the base OpenVLA-OFT model and our method on the LIBERO-Plus benchmark [8], which builds on LIBERO to evaluate VLA robustness under seven types of perturbations: camera viewpoints, robot initial states, language instructions, lighting, background textures, sensor noise, and object layouts. We evaluate on the Long suite, and the results are summarized in the table: our method achieves a 0.6% higher success rate with 55.7% fewer FLOPs, demonstrating robustness to these perturbations.

MethodCameraRobotLanguageLightBackgroundNoiseLayoutAverage ↑FLOPs (T) ↓
OpenVLA-OFT92.8%30.3%85.8%94.9%93.9%89.3%77.6%79.5%4.194
Ours93.3%30.5%86.3%95.3%94.2%88.9%77.9%80.1%1.857

Moreover, to evaluate performance under temporary occlusions, we test both the base OpenVLA-OFT model and our method on the LIBERO-Occ benchmark [9], a variant that injects occlusions into the Spatial, Object, Goal, and Long suites and measures how reliably policies complete tasks when visual information is partially obstructed. As shown in the table, our method achieves an average of a 0.5% higher success rate with 60.3% fewer FLOPs, demonstrating its robustness to occlusions.

MethodSpatialObjectGoalLongAverage ↑FLOPs (T) ↓
OpenVLA-OFT95.0%97.8%88.6%60.6%85.5%4.013
Ours94.8%98.6%88.2%62.4%86.0%1.593

[1] Timothy P Lillicrap, et al. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015.

[2] Bo Liu, et al. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems, 36, 2024a.

[3] Moo Jin Kim, et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning, pp. 2679–2713, 2024.

[4] Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645, 2025.

[6] Xiaohuan Pei, et al. Action-aware dynamic pruning for efficient vision-language-action manipulation. arXiv preprint arXiv:2509.22093, 2025.

[8] Hanzhen Wang, et al. Specprune-vla: Accelerating vision-language-action models via action-aware self-speculative pruning. arXiv preprint arXiv:2509.05614, 2025a.

[9] Taishan Li, et al. LIBERO-Occ: Evaluating and Improving Vision-Language-Action Models under Scene-Induced Occlusion via Viewpoint Imagination. arXiv preprint arXiv:2606.10862. 2026.

[10] Senyu Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models. arXiv preprint arXiv:2510.13626, 2025.

[5] Jintao Cheng, et al. Training-free interaction-aligned visual token pruning for efficient embodied manipulation. arXiv preprint arXiv:2603.22991, 2026.

Reviewer_bhJmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MemCache, a learnable token-caching method for efficient vision-language-action inference. Following prior VLA caching methods, it reuses the previous step’s KV states for static visual tokens and recomputes only dynamic ones. The key idea is to replace hand-crafted or external proxy signals with a bank of learnable queries that score the importance of tokens, trained end-to-end under the action loss via a Gumbel-Softmax straight-through estimator and a budget loss. Additionally, a recurrent visual memory state provides history-conditioned corrections to the caching decisions, trained with a linear-warmup schedule and a memory-consistency loss. The paper reports that MemCache reduces FLOPs and inference latency while maintaining or slightly improving success rate relative to the base models and caching baselines (VLA-Cache, LAC), including OpenVLA and OpenVLA-OFT on the LIBERO benchmark and CogACT on SIMPLER.

Critical strengths
  1. Learning caching decisions end-to-end using the action loss is a sensible approach. Removing the need for external components, such as the optical-flow network used by LAC, also makes the method more self-contained and lightweight.
  2. The paper provides a broad evaluation across three VLA architectures and two benchmarks (LIBERO and SIMPLER), with comparisons against eight caching and pruning baselines.
  3. The results demonstrates the proposed method's efficiency gain, with 44-46% fewer FLOPs than caching baselines on LIBERO benchmark and 17-31% fewer than LAC on SIMPLER benchmark while maintaining success rate.
  4. The paper is clearly written and easy to follow.
Critical weaknesses
  1. Latency comparisons may use different hardware, and the reuse of baseline results is not disclosed: Section 4.1 states that latency is measured on a single RTX 4090 "to ensure fair comparisons." However, the baseline values in Table 1 (OpenVLA, SparseVLM, FastV, VLA-Cache, and LAC) and Table 4 (CogACT, VLA-Cache, and LAC) exactly match those reported in [1], which does not specify the GPU used. The paper neither discloses that these baseline results are reused nor identifies the hardware on which they were obtained, while its AI Use Statement section claims that "all reported numbers come from our own experiments." Consequently, the reported speedups, including 1.87$\times$ over OpenVLA and 1.07-1.19$\times$ over LAC, may be based on latency measurements obtained using different hardware and timing setups. Only the FLOPs comparisons, which are hardware-independent, can therefore be considered reliable.

  2. The gains are marginal, and no variance is reported: Compared with LAC, MemCache improves success by only 1.2 percentage points on OpenVLA (Table 1) and performs comparably on SIMPLER (−0.1 / +0.3, Table 4). No standard deviations, random seeds, or episode counts are reported for any success rate. Meanwhile, reproduction differences on this benchmark are of a similar magnitude: the OpenVLA baseline in Table 1 (75.0%) is 1.5 percentage points below the 76.5% reported in the original OpenVLA paper. Without estimates of variability, the reported improvements cannot be distinguished from evaluation noise.

  3. The recurrent memory has only a small measured effect, and its claimed long-range benefits remain untested: In the main ablation (Table 3), adding memory improves success rates by only 0.8 percentage points for OpenVLA and 0.4 percentage points for OpenVLA-OFT, with evaluation limited to a single benchmark suite (LIBERO Spatial). For OpenVLA, the training schedule contributes more to the improvement (+1.4) than the memory itself. Moreover, no experiment examines how far back the memory can effectively retain or use information, even though capturing long-term context beyond adjacent frames is one of the two main problems the paper claims to address.

Minor Weaknesses:

  • No hyperparameter analysis: The method introduces several hyperparameters, most importantly the budget schedule (r0→r∗r_0 \rightarrow r_{*}), the memory warmup coefficient γs\gamma_s, and the budget-loss weight λbudget\lambda_{budget}, but the main paper does not analyze their sensitivity. For example, λbudget\lambda_{budget} is set to 1.0, 0.1, and 0.2 for OpenVLA, OpenVLA-OFT, and CogACT respectively (Appendix A.4). These differences suggest that model-specific tuning may be necessary, yet the paper provides no guidance on how to tune these hyperparameters for new VLAs.
  • No real-world experiments: The paper is motivated by real-time control on physical robots, yet all results are in simulation.

[1] Wei, Yujie, et al. "Learning to accelerate vision-language-action models through adaptive visual token caching." arXiv preprint arXiv:2602.00686 (2026).

Questions for authors
  1. Were the baseline latencies in Tables 1 and 4 re-measured on your RTX 4090, or taken from [1]? If the latter, could the authors re-measure all baselines (at least the vanilla models and LAC) on the same hardware and report the resulting speedups? Please also clarify the source of the VLA-Cache results on OpenVLA-OFT.

  2. Could the authors report standard deviations over multiple evaluation seeds, with the number of episodes per suite, for Tables 1, 3 and 4? Are the gains over LAC statistically distinguishable from evaluation noise?

  3. Could the authors provide an analysis of the effective temporal horizon of the recurrent memory, for instance by resetting the memory state every k steps at inference and reporting the resulting success rates?

[1] Wei, Yujie, et al. "Learning to accelerate vision-language-action models through adaptive visual token caching." arXiv preprint arXiv:2602.00686 (2026).

8 comments

We thank the reviewer for the constructive feedback on our manuscript. As suggested, we have:

(1) Clarified the source of the baseline latency results and noted that latency comparisons use the same GPU type, except for LAC, which neither specifies the GPU type nor provides a publicly available implementation; Re-measured OpenVLA, OpenVLA-OFT, and our caching baseline VLA-Cache, confirming cross-machine consistency on the same GPU type and our method retaining its speedup over these baselines (Latency results comparisons and clarifications, W1&Q1);

(2) Conducted experiments with three different random seeds, confirming that our superior performance–efficiency trade-offs hold beyond seed noise (Standard deviations over multiple random seeds, W2&Q2);

(3) Compared our method against memory-reset variants and a simpler temporal aggregation mechanism (EMA), demonstrating our long-term memory benefits (Long-range benefits, W3&Q3);

(4) Analyzed the sensitivity to the budget-loss weight and provided guidance on tuning hyperparameters for new VLAs (Hyperparameter analysis, Minor Weakness 1);

(5) Conducted real-robot experiments to demonstrate our effectiveness in real-world applications (Real-robot experiments, Minor Weakness 2).

W1&Q1: Latency results comparisons and clarifications

We would like to clarify that the latency comparisons (except LAC [1], which neither specifies the GPU type nor provides a publicly available implementation) are under the same GPU type. Specifically, the latency results for OpenVLA [2], OpenVLA-OFT [3], SparseVLM [4], FastV [5], and VLA-Cache [6] reported in our paper were taken from the VLA-Cache paper, which explicitly reports using a single NVIDIA RTX 4090 for latency measurement. The LAC latency was taken from its own paper, which also adopts the latency results from the VLA-Cache paper for direct comparison. Therefore, we conducted our latency measurements on a 4090 GPU to align with most of our baselines. We have clarified this and revised our AI Use Statement to avoid confusion.

To address potential variations across machines, we re-evaluated OpenVLA, OpenVLA-OFT, and VLA-Cache on our own machine, strictly following the measurement protocol provided in the released VLA-Cache code. LAC is not re-measured since its implementation is not publicly available. As we can see, the re-measured latencies of these baselines deviate by no more than 5% from their originally reported values, demonstrating good consistency across different machines of the same type of GPU. Besides, our method maintains its speedup over these re-evaluated baselines.

MethodOpenVLA (LIBERO)OpenVLA-OFT (LIBERO)
Base Model (VLA-Cache Paper)51.9179.05
Base Model (Re-measured)52.7577.86
VLA-Cache (VLA-Cache Paper)31.8362.59
VLA-Cache (Re-measured)30.5663.74
Ours27.6943.23
MethodVisual Matching (CogACT)Visual Aggregation (CogACT)
Base Model (VLA-Cache Paper)54.2953.54
Base Model (Re-measured)52.6351.86
VLA-Cache (VLA-Cache Paper)39.6337.90
VLA-Cache (Re-measured)41.3338.94
Ours31.7134.23

W2&Q2: Standard deviations over multiple evaluation seeds

To assess the sensitivity of MemCache to random seeds, we run experiments with 3 random seeds on both the LIBERO [7] and SIMPLER [8] benchmarks. Each LIBERO suite is evaluated over 500 episodes (ten subtasks, 50 episodes each), following the standard setup of OpenVLA and OpenVLA-OFT for consistency. For SIMPLER, the visual matching setting comprises 864 trials and the visual aggregation setting includes 1,992 trials, following the standard setup of CogACT [9].

As shown in the tables, across all three seeds, we outperform LAC by 0.8%–1.8% in success rate at 44.2% fewer FLOPs on OpenVLA, and outperform VLA-Cache [6] by 0.4%–1.2% at 46.4% fewer FLOPs on OpenVLA-OFT with LIBERO. On CogACT with SIMPLER, the success rate changes by −0.3% to +0.5% at 31.5% fewer FLOPs under the visual matching setting, and by −0.2% to +0.6% at 17.4% fewer FLOPs under the visual aggregation setting, relative to LAC. These results indicate that our gains hold beyond seed noise. We would like to emphasize that the key advantage of MemCache lies in its better performance–efficiency trade-offs rather than in task performance alone.

Main results on the LIBERO benchmark

MethodSuccess Rate (Average) ↑FLOPs (T) ↓Latency (ms) ↓
OpenVLA (Original Paper)76.5 ± 0.60%1.864--
OpenVLA (VLA-Cache Paper)75.0%1.86451.91
OpenVLA (Reproduced)75.1 ± 0.55%1.86452.75
SparseVLM64.7%1.40783.39
FastV73.3%1.86453.28
VLA-Cache74.7%1.35531.83
LAC76.9%1.39229.51
Ours78.2±0.50%0.77727.69
OpenVLA-OFT (Original Paper)97.1%4.013--
OpenVLA-OFT (VLA-Cache Paper)96.8%4.01379.05
OpenVLA-OFT (Reproduced)96.6±0.38%4.01377.86
VLA-Cache97.4%3.09762.59
Ours98.2±0.41%1.66043.23

Noted that our reproduced OpenVLA and OpenVLA-OFT results are close to those reported in the VLA-Cache paper. Although our reproduced baselines are lower than those reported in the original papers, equipping them with our caching method still yields a higher success rate (1.7% on OpenVLA and 1.1% on OpenVLA-OFT) than the originally reported numbers, demonstrating our effectiveness.

Main results on the SIMPLER benchmark

SIMPLERMethodSuccess Rate (Average) ↑FLOPs (T) ↓Latency (ms) ↓
Visual MatchingCogACT74.8%1.84754.29
Visual MatchingVLA-Cache74.4%1.49639.63
Visual MatchingLAC75.5%1.51137.82
Visual MatchingOurs75.6±0.36%1.03531.71
Visual AggregationCogACT61.3%1.80753.54
Visual AggregationVLA-Cache62.3%1.49339.11
Visual AggregationLAC63.0%1.50637.90
Visual AggregationOurs63.2%±0.411.24434.23

Ablation studies

Applying query-based caching improves the average success rate by 0.8% on our reproduced OpenVLA baseline and by 0.6% on the vanilla OpenVLA-OFT. As for the integration of our recurrent memory module, the vanilla method that directly trains our memory-conditioned correction alongside query-based caching yields only an additional 0.3% improvement on both OpenVLA and OpenVLA-OFT over the query-based variant. When equipped solely with the linear warmup training strategy or the consistency loss, we improve the success rate by 1.4% or 1.7% on OpenVLA and by 0.4% or 0.7% on OpenVLA-OFT under the same training budget and caching ratio. Combining both components achieves the best performance, i.e., 2.3% and 1.0% higher success rates on OpenVLA and OpenVLA-OFT, respectively, over the query-based variant, demonstrating the effectiveness of each component.

MethodQueryMem.Vanilla TrainWarm-upConsis. LossSpatial ↑Object ↑Goal ↑Long ↑Avg. ↑Caching Ratio ↑
OpenVLA–––––85.0% ± 0.47%86.1% ± 0.57%76.3% ± 0.61%53.1% ± 0.55%75.1% ± 0.55%–
OpenVLA (LoRA)–––––84.7% ± 0.58%86.4% ± 0.52%76.2% ± 0.54%53.3% ± 0.47%75.2% ± 0.54%–
Query-Based Variant✓–✓––84.9% ± 0.52%87.0% ± 0.44%75.3% ± 0.41%56.4% ± 0.45%75.9% ± 0.45%0.65
Memory-Conditioned Variant✓✓✓––85.9% ± 0.56%86.4% ± 0.35%76.5% ± 0.54%56.0% ± 0.41%76.2% ± 0.47%0.65
Memory-Conditioned Variant✓✓–✓–87.2% ± 0.45%86.9% ± 0.54%77.4% ± 0.42%57.7% ± 0.57%77.3% ± 0.50%0.65
Memory-Conditioned Variant✓✓––✓86.8% ± 0.54%87.3% ± 0.56%77.7% ± 0.44%58.5% ± 0.41%77.6% ± 0.49%0.65
Ours✓✓–✓✓87.2% ± 0.56%87.9% ± 0.46%78.1% ± 0.35%59.4% ± 0.62%78.2% ± 0.50%0.65
OpenVLA-OFT–––––98.0% ± 0.36%97.4% ± 0.45%96.6% ± 0.37%94.4% ± 0.33%96.6% ± 0.38%–
Query-Based Variant✓–✓––98.4% ± 0.33%98.3% ± 0.35%97.2% ± 0.44%94.7% ± 0.40%97.2% ± 0.38%0.75
Memory-Conditioned Variant✓✓✓––98.6% ± 0.40%98.5% ± 0.34%97.5% ± 0.51%95.4% ± 0.42%97.5% ± 0.42%0.75
Memory-Conditioned Variant✓✓–✓–98.6% ± 0.44%98.7% ± 0.41%97.7% ± 0.40%95.5% ± 0.35%97.6% ± 0.40%0.75
Memory-Conditioned Variant✓✓––✓99.0% ± 0.38%98.7% ± 0.38%97.9% ± 0.42%95.8% ± 0.30%97.9% ± 0.37%0.75
Ours✓✓–✓✓99.4% ± 0.33%98.9% ± 0.44%98.2% ± 0.44%96.2% ± 0.42%98.2%± 0.41%0.75

W3&Q3: Long-range benefits remain untested

To validate our long-term benefits, we compare our method with memory-reset variants across different history lengths. To further verify the effectiveness of our memory update mechanism, we compare it with a simpler temporal aggregation mechanism, the exponential moving average (EMA) [10]. We evaluate on two LIBERO benchmark suites, Spatial and Long: the former includes an average of 104 steps per episode, whereas the latter has averaging 280 steps per episode, enabling us to assess performance under both short- and long-horizon regimes. We evaluate both OpenVLA, in which each frame prefill yields a single action step, and OpenVLA-OFT, which uses action chunking to produce eight steps per frame prefill.

All experiments use the default random seed and evaluation protocols of the OpenVLA and OpenVLA-OFT repositories. All comparisons are conducted with identical training schedules and caching budgets, except for the base models, which are not cached.

MethodOpenVLA SpatialOpenVLA LongOpenVLA-OFT SpatialOpenVLA-OFT Long
Base Model85.2%53.4%98.2%94.6%
+ Queries85.4%56.8%98.6%95.0%
+ Memory (2 frames)85.6%57.2%98.8%95.4%
+ Memory (8 frames)86.2%57.4%99.0%95.6%
+ Memory (32 frames)86.4%58.2%--*95.9%
EMA (Full, α=0.2)83.6%57.9%99.0%95.7%
EMA (Full, α=0.5)84.4%57.1%98.8%92.9%
EMA (Full, α=0.8)87.2%58.1%97.6%95.3%
Ours (Full)87.6%59.0%99.2%96.2%

*For OpenVLA-OFT (action chunk size 8), 32 frames yield 256 steps, exceeding the horizon of LIBERO-Spatial.

As shown in the table, resetting every two frames restricts our memory state to only adjacent frames. Consequently, its success rate is comparable (0.2%–0.4% higher) to that of query-based methods (no memory). Resetting every 8 frames improves the success rate by 0.4%–0.8%, while resetting every 32 frames endows our caching decisions with longer-term memory, further improving the success rate by 0.9%–1.4% over the query-based variant. Our method without reset achieves the best success rate (0.6%–2.2% over the query-based variant), demonstrating both the importance of long-term memory for caching decisions and our method's ability to retain it effectively.

Furthermore, compared with EMA, which mixes historical memory and the current observation in a proportional manner (new memory = α × history + (1 − α) × current) and therefore depends on the choice of α, our method consistently achieves higher success rates under the same caching ratio. This verifies the effectiveness of our memory updating mechanism, which learns end-to-end to refresh the memory state, aiming to accumulate useful historical context and enable better caching decisions.

Minor Weakness 1: Hyperparameter analysis

Budget-loss weight: It is selected based on the relative gradient contributions of the action loss and budget loss to the parameters of our caching module. The goal is to keep their gradient magnitudes within the same order of magnitude, preventing either objective from dominating the optimization. This also explains why different VLA backbones may require different values of λ, as their loss scales and gradient magnitudes can vary.

We further ablate the budget-loss weight with OpenVLA-OFT on the LIBERO Long suite, with quantitative results reported in the table. As we can see, a higher weight suppresses the supervision from the action loss and thus lowers performance by 0.5% (λ = 0.5) and 1.1% (λ = 1). Conversely, a lower weight causes the caching gate to cache fewer tokens during training, so that the gate fails to make reliable decisions under more aggressive caching ratios during inference, and task performance drops by 0.9% (λ = 0.01).

lambda0.010.1 (Ours)0.51.0
Success Rate95.3%96.2%95.7%95.1%

When it comes to a new VLA backbone, we recommend monitoring the gradient magnitudes of the two losses during early training and adjusting the budget-loss weight accordingly.

Budget schedule and memory warmup coefficient: These hyperparameters are largely model-independent and can be applied directly across different VLA backbones. We share the same values for OpenVLA, OpenVLA-OFT, and CogACT. Specifically, the budget schedule adopts a linear decay from a high recomputation ratio to a lower one, promoting training stability in the early stages while progressively encouraging more token reuse and higher cache ratios. The memory warmup coefficient starts from a small value and gradually increases, allowing the memory branch to progressively influence gate decisions while preventing potentially unstable memory representations during early training from compromising gate accuracy.

Reviewer_tuiGOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MemCache learns which visual tokens to recompute or reuse during VLA inference. Its gate combines learned query-based importance features with corrections from a recurrent visual memory. Budget annealing, memory consistency, and a warm-up schedule support training. Experiments cover OpenVLA and OpenVLA-OFT on LIBERO and CogACT on SIMPLER, reporting lower latency while maintaining or modestly improving task success.

Critical strengths

The evaluation spans three action-generation architectures and two benchmarks. In particular, OpenVLA-OFT latency decreases from 79.05 to 43.23 ms while reported success increases from 96.8% to 98.1%. The frozen-backbone OFT and CogACT experiments provide useful evidence beyond an additional policy fine-tuning effect.

The paper reports measured latency alongside FLOPs and acknowledges their imperfect correspondence. This makes the efficiency contribution more convincing than a purely theoretical computation reduction.

Critical weaknesses

The OpenVLA comparision is confounded by additional policy adaptation: Appendix A.4 enables rank-64 LoRA in the final eight decoder layers. A similarly trained, uncached baseline is needed to attribute the accuracy improvement to caching.

Evidence for long-range memory is limited. In Table 7, adding memory improves average success by only 0.2–0.3 percentage points before the combined training modifications. No uncertainty estimates or memory-reset/history-length controls are reported, and warm-up and consistency are introduced together. The current ablations therefore leave the central mechanism somewhat unclear.

Questions for authors

What happens when the OpenVLA baseline receives the same additional LoRA training without caching? Were competing learned methods given comparable adaptation budgets?

Can you separate the effects of recurrent history, memory consistency, and warm-up at matched caching budgets, with evaluation counts and uncertainty estimates? A memory-reset control would be particularly informative.

4 comments

We thank the reviewer for the positive evaluation of our work and for taking the time to provide constructive suggestions. As suggested, we have:

(1) Compared the original OpenVLA, our LoRA-finetuned OpenVLA (uncached), and our full method to isolate the effect of LoRA fine-tuning. The results show that the gain stems primarily from our caching mechanism rather than from LoRA adaptation (LoRA adaptation on OpenVLA, W1&Q1);

(2) Conducted additional ablations to disentangle the effects of warm-up training and the consistency loss in integrating the memory module, which clarifies how each component contributes (Ablation study of warm-up training and consistency loss, W2&Q2);

(3) Compared our method against memory-reset variants over a range of history lengths and against a simpler temporal aggregation mechanism (EMA), which demonstrates the importance of long-term memory for caching decisions and our method's ability to retain it (Limited evidence for long-range memory, W2&Q2).

W1&Q1: LoRA adaptation on OpenVLA

To isolate the effect of LoRA fine-tuning on OpenVLA's performance, we compare (i) the original OpenVLA [1], (ii) our LoRA-finetuned OpenVLA (uncached), and (iii) our full method. As shown in the table, LoRA fine-tuning alone changes the average success rate by only 0.1%, whereas combining it with our caching mechanism yields an average improvement of 2.6% at even the 65% caching ratio. This indicates that the gain primarily stems from our caching mechanism rather than from LoRA adaptation.

MethodSpatialObjectGoalLongAverageFLOPs (T) ↓
OpenVLA (Base)85.2%86.6%76.6%53.4%75.5%1.864
OpenVLA (Finetuned)85.2%86.9%76.4%53.7%75.6%1.864
LAC85.6%86.2%76.6%59.2%76.9%1.392
Ours87.6%87.6%78.2%59.0%78.1%0.777

We would also like to clarify that LoRA fine-tuning is not intended to boost OpenVLA's task performance, but to provide stronger gradient signals to our cache modules. As explained in Appendix A 4.1 in our manuscript, OpenVLA adopts a cross-entropy loss over discrete action tokens, so gradients to the caching gate must traverse the vocabulary head and the causal Transformer, yielding weak signals. We therefore enable LoRA only on the last 8 decoder layers and optimize it jointly with our caching module for less than 50% of the original fine-tuning steps. This limited layer coverage and reduced optimization cost explain why LoRA alone cannot improve OpenVLA's performance.

Adaptation budgets for competing learned methods

As our learning-based baseline, LAC [2] keeps the backbone frozen but trains its caching selector and predictor in two stages (attention-map imitation for initialization, followed by task-driven joint optimization). In contrast, our method trains the caching module in a single end-to-end stage. We note that LAC neither reports its number of trainable parameters nor releases its code, so an exact parameter-level comparison is not feasible. However, we restrict LoRA to only the last 8 layers to reduce adaptation costs, and our method still achieves better performance-efficiency trade-offs than LAC. These benefits also hold for other models, including OpenVLA-OFT and CogACT, where LoRA is entirely disabled.

W2&Q2: Ablation study of warm-up training and consistency loss

We conduct ablation experiments using both OpenVLA [1] and OpenVLA-OFT [3] on the LIBERO benchmark. Each suite (Spatial/Object/Goal/Long) is evaluated over 500 episodes, and we run 3 random seeds per suite to account for uncertainty.

As shown in the table, query-based caching improves the average success rate by 0.7% over our LoRA-finetuned OpenVLA baseline and by 0.6% over the vanilla OpenVLA-OFT baseline. Directly training the memory-conditioned corrections alongside query-based caching yields only an additional 0.3% higher success rate on both baselines over the query-based variant. Under the same training budget and caching ratio, the linear warmup strategy alone or the consistency loss alone improves the success rate by 1.4% or 1.7% on OpenVLA and by 0.4% or 0.7% on OpenVLA-OFT, respectively. Combining both achieves the best performance, i.e., 2.3% and 1.0% higher success rates on OpenVLA and OpenVLA-OFT, respectively, over the query-based variant.

MethodQueryMem.Vanilla TrainWarm-upConsis. LossSpatial ↑Object ↑Goal ↑Long ↑Avg. ↑Caching Ratio ↑
OpenVLA–––––85.0% ± 0.47%86.1% ± 0.57%76.3% ± 0.61%53.1% ± 0.55%75.1% ± 0.55%–
OpenVLA (LoRA)–––––84.7% ± 0.58%86.4% ± 0.52%76.2% ± 0.54%53.3% ± 0.47%75.2% ± 0.54%–
Query-Based Variant✓–✓––84.9% ± 0.52%87.0% ± 0.44%75.3% ± 0.41%56.4% ± 0.45%75.9% ± 0.45%0.65
Memory-Conditioned Variant✓✓✓––85.9% ± 0.56%86.4% ± 0.35%76.5% ± 0.54%56.0% ± 0.41%76.2% ± 0.47%0.65
Memory-Conditioned Variant✓✓–✓–87.2% ± 0.45%86.9% ± 0.54%77.4% ± 0.42%57.7% ± 0.57%77.3% ± 0.50%0.65
Memory-Conditioned Variant✓✓––✓86.8% ± 0.54%87.3% ± 0.56%77.7% ± 0.44%58.5% ± 0.41%77.6% ± 0.49%0.65
Ours✓✓–✓✓87.2% ± 0.56%87.9% ± 0.46%78.1% ± 0.35%59.4% ± 0.62%78.2% ± 0.50%0.65
OpenVLA-OFT–––––98.0% ± 0.36%97.4% ± 0.45%96.6% ± 0.37%94.4% ± 0.33%96.6% ± 0.38%–
Query-Based Variant✓–✓––98.4% ± 0.33%98.3% ± 0.35%97.2% ± 0.44%94.7% ± 0.40%97.2% ± 0.38%0.75
Memory-Conditioned Variant✓✓✓––98.6% ± 0.40%98.5% ± 0.34%97.5% ± 0.51%95.4% ± 0.42%97.5% ± 0.42%0.75
Memory-Conditioned Variant✓✓–✓–98.6% ± 0.44%98.7% ± 0.41%97.7% ± 0.40%95.5% ± 0.35%97.6% ± 0.40%0.75
Memory-Conditioned Variant✓✓––✓99.0% ± 0.38%98.7% ± 0.38%97.9% ± 0.42%95.8% ± 0.30%97.9% ± 0.37%0.75
Ours✓✓–✓✓99.4% ± 0.33%98.9% ± 0.44%98.2% ± 0.44%96.2% ± 0.42%98.2%± 0.41%0.75

We would also like to clarify that the warm-up schedule and the consistency objective are integral components of our method, specifically designed to enable effective learning of the recurrent memory. The modest gain of the "+ Memory" variant in our original manuscript results from vanilla training, which fails to fully unleash its benefits under the same training budget, and does not indicate that our memory module itself is ineffective. We apologize for any confusion caused by our previous wording and have updated the ablation study table and clarified this point in the revised manuscript.

W2&Q2: Limited evidence for long-range memory

To validate the long-term contribution of our memory mechanism, we compare our method with memory-reset variants across different history lengths. To further verify the effectiveness of our memory update mechanism, we compare it with a simpler temporal aggregation mechanism, the exponential moving average (EMA) [4]. We evaluate on two LIBERO benchmark suites, Spatial and Long: the former includes an average of 104 steps per episode, whereas the latter has averaging 280 steps per episode, enabling us to assess performance under both short- and long-horizon regimes. We evaluate both OpenVLA [1], in which each frame prefill yields a single action step, and OpenVLA-OFT [3], which uses action chunking to produce eight steps per frame prefill.

All experiments use the default random seed and evaluation protocols of the OpenVLA and OpenVLA-OFT repositories. All comparisons are conducted with identical training schedules and caching budgets, except for the base models, which are not cached.

MethodOpenVLA SpatialOpenVLA LongOpenVLA-OFT SpatialOpenVLA-OFT Long
Base Model85.2%53.4%98.2%94.6%
+ Queries85.4%56.8%98.6%95.0%
+ Memory (2 frames)85.6%57.2%98.8%95.4%
+ Memory (8 frames)86.2%57.4%99.0%95.6%
+ Memory (32 frames)86.4%58.2%—*95.9%
EMA (Full, α=0.2)83.6%57.9%99.0%95.7%
EMA (Full, α=0.5)84.4%57.1%98.8%92.9%
EMA (Full, α=0.8)87.2%58.1%97.6%95.3%
Ours (Full)87.6%59.0%99.2%96.2%

*For OpenVLA-OFT (action chunk size 8), 32 frames yield 256 steps, exceeding the horizon of LIBERO-Spatial.

As shown in the table, resetting every two frames restricts our memory state to only adjacent frames. Consequently, its success rate is comparable (0.2%–0.4% higher) to that of the query-based method (no memory). Resetting every 8 frames improves the success rate by 0.4%–0.8%, while resetting every 32 frames endows our caching decisions with longer-term memory, further improving the success rate by 0.9%–1.4% over the query-based variant. Our method without reset achieves the best success rate (0.6%–2.2% over the query-based variant), demonstrating both the importance of long-term memory for caching decisions and our method's ability to retain it effectively.

AuthorsOpenReview ↗

Follow-up on your concerns

Dear Reviewer,

As the discussion period draws to a close, I would like to check whether we have addressed all of your concerns satisfactorily.

We appreciate your insights and would be glad to follow up on any remaining issues. If there is any additional feedback you would like us to consider, please let us know.

Thank you for your time and effort in reviewing our paper.

AuthorsOpenReview ↗

Kind reminder: discussion period closing

Dear Reviewer tuiG,

As the discussion period closes today, we would like to kindly check whether our responses have addressed your remaining concerns, and we would be happy to provide any further details or additional results if needed.

Thank you again for your time and constructive feedback. We would greatly appreciate your final evaluation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank the AC and reviewers for their constructive suggestions. We have addressed them as follows:

(1) Effectiveness of long-term memory (Reviewer tuiG W2&Q2, bhJm W3&Q3, ufQb W1&Q1): We compare our method against memory-reset variants under various history lengths and against a simpler temporal aggregation mechanism (EMA), demonstrating the importance of long-term memory for caching decisions and our ability to retain it;

(2) Real-robot experiments (Reviewer bhJm minor weakness 2, ufQb W3&Q3): We conducted real-robot experiments to validate our method in real-world applications;

(3) LoRA adaptation on OpenVLA (Reviewer tuiG W1&Q1): We compared vanilla OpenVLA, our LoRA-finetuned OpenVLA (uncached), and our full method to isolate the effect of LoRA fine-tuning. The results show that the gain stems primarily from our caching mechanism rather than LoRA adaptation;

(4) Ablation on the training strategy (Reviewer tuiG W2&Q2): We added ablations disentangling the effects of warm-up training and the consistency loss on memory integration, clarifying the contribution of each component;

(5) Latency comparison and clarification (Reviewer bhJm W1&Q1): We clarified the source of baseline latencies and note that all comparisons use the same GPU except LAC, which specifies neither the GPU type nor a public implementation. We re-measured OpenVLA, OpenVLA-OFT, and VLA-Cache on the same GPU, confirming cross-machine consistency and that our method retains its speedup;

(6) Different random seeds (Reviewer bhJm W2&Q2): We ran experiments with three random seeds, confirming that our superior performance–efficiency trade-offs hold beyond seed noise;

(7) Hyperparameter analysis (Reviewer bhJm minor weakness 1): We analyzed sensitivity to the budget-loss weight and provided hyperparameter tuning guidance for new VLAs;

(8) Comparison with token-pruning baselines (Reviewer ufQb W2&Q2): We added FLOPs and latency-speedup comparisons against token-pruning baselines to validate our superior performance–efficiency trade-offs;

(9) Robustness of our method (Reviewer ufQb W3&Q3): We evaluated on extended LIBERO benchmarks (LIBERO-Occ, LIBERO-Plus) to validate robustness under temporary occlusions and various perturbations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.