acceptodds
Under review as a conference paper at ICLR 2027

WAM-Cache: Staleness-Bounded KV Reuse for Efficient World Action Models

Abstract

World Action Models (WAMs) enable generalist robot manipulation by conditioning an action expert on representations from a pretrained video Diffusion Transformer (DiT). In closed-loop control, the video DiT runs at every chunk to encode the current observation into layerwise key–value (KV) pairs that the action expert queries. This prefill dominates the per-chunk computational cost, yet existing training-free accelerations leave it fully dense. We present WAM-Cache, a training-free framework that retains layerwise key–value representations across chunks and recomputes only a sparse refresh set of tokens. Crucially, we find that the intuitive heuristic of refreshing visually drifted tokens plateaus far below the dense baseline, even with an oracle predicting ground-truth KV drift. Downstream action accuracy is instead governed by where the action expert attends, not by what moved. WAM-Cache therefore selects the refresh set by uniting the action expert's cross-attention with visual latent surprise, complemented by a strict age bound that suppresses compounding error. On Fast-WAM, WAM-Cache cuts video DiT prefill FLOPs by 32–42% across RoboTwin 2.0, LIBERO, and real-world experiments, while staying within 0.7–1.8 percentage points of the dense policy in simulation and 2.5 points on a real robot.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World-Action Models, Efficient Inference, Robotics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_NrmDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WAM-Cache accelerates Fast-WAM's video DiT prefill without retraining by reusing layerwise visual KV states across control cycles and refreshing tokens selected by latent changes, previous action attention, and cache age. The evaluated inference path encodes current observations without generating future video. The method reduces prefill FLOPs by 32–42% while largely preserving task success in simulation and on a real robot.

Critical strengths
  1. The method reduces prefill FLOPs by 32–42% with small losses in task success and can approach dense performance at higher refresh budgets while retaining substantial computational savings.
  2. At comparable compute budgets, it outperforms the evaluated baselines, and the combined signals achieve higher task success with fewer refreshed tokens than either signal alone.
  3. The paper is well written, with carefully designed figures that clearly explain the refresh mechanism.
Critical weaknesses
  1. TTF-VLA[1] uses current visual tokens where pixel changes are large or previous attention is high, reuses historical tokens elsewhere, and replaces all tokens with current features at periodic keyframes. Given these similar design principles, the additional methodological contribution of the proposed refresh policy remains unclear. The author could provide additional comparison to validate their contribution if possible.
  2. Treating the KV-drift oracle as a performance upper bound for all drift prediction rules requires further justification. Even if the true KV changes are known exactly, background regions with larger changes may be less important to refresh than a grasp target with smaller changes. Ranking tokens by drift magnitude does not necessarily reflect their impact on actions, so it remains unclear whether the oracle’s task success rate provides an upper bound for other rules.
  3. The refresh policy may spend its budget on irrelevant changes such as flickering lights and fail to update tokens that matter for the current action. The reliability of this selection strategy in such cases remains unverified.

[1] Liu C, Zhang J, Li C, et al. Ttf-vla: Temporal token fusion via pixel-attention integration for vision-language-action models[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2026, 40(22): 18452-18459.

Questions for authors

Please see Weaknesses 1–3 above.

6 comments
AuthorsOpenReview ↗

Rebuttal 1: W1

We are grateful for your insightful review. We are delighted by your appreciation of our work as achieving "substantial computational savings", being "well written", and featuring "carefully designed figures".

You also raised some points of weakness, which we found very enlightening. We have thoroughly analyzed the issues you mentioned and provide the following responses. Changes are highlighted in blue in the revision.{\color{blue}\text{Changes are highlighted in blue in the revision.}}

W1: TTF-VLA [1] uses current visual tokens where pixel changes are large or previous attention is high, reuses historical tokens elsewhere, and replaces all tokens with current features at periodic keyframes. Given these similar design principles, the additional methodological contribution of the proposed refresh policy remains unclear. The author could provide additional comparison to validate their contribution if possible.

Thank you for your careful review and for bringing TTF-VLA [a] to our attention. We agree that TTF-VLA and WAM-Cache are related at a high level, and we have cited it in the related work of the revision (Section 2{\color{blue}\text{Section 2}}).

Nevertheless, the two methods differ substantially. Table D1 summarizes the differences, and we discuss them below in order of importance.

Table D1. Comparison between TTF-VLA and WAM-Cache.

AspectTTF-VLA [a]WAM-Cache (Ours)
GoalHigher success rateLower inference cost
What is reusedVision-encoder output tokens at the LLM inputLayerwise K/V states inside the video DiT
Compute savedNone (encoder and backbone run in full)32–42% of prefill FLOPs
Change signalPixel difference between consecutive framesLatent drift since the token's last refresh
Staleness controlGlobal keyframe (all tokens every K steps)Per-token age counter
Attention sourceText-to-vision (reported results and code default)Action expert to cached video K/V

1. Different objectives: TTF-VLA improves accuracy, whereas WAM-Cache reduces inference cost. TTF-VLA fuses historical visual tokens into the current input to exploit temporal information and improve the success rate of a VLA. Its vision encoder and LLM backbone still process every frame in full. The paper reports a runtime overhead of less than 2% and names direct KQV reuse as a future direction. In contrast, WAM-Cache selects tokens whose layerwise K/V states can be reused, and forms queries only for the refresh set, which removes 32–42% of the prefill FLOPs. Consequently, stale content plays opposite roles in the two methods. In TTF-VLA, historical tokens are a beneficial signal, and every selection variant in its ablation improves on the baseline on average. In WAM-Cache, a stale K/V pair is an approximation error that propagates across layers and chunks, so the selection rule is decisive: at a matched budget, success ranges from 26.9% to 88.4% (Table 3).

2. Different change signal: latent drift since the last refresh, rather than pixel difference. TTF-VLA thresholds the grayscale pixel difference between consecutive frames. WAM-Cache scores the normalized latent distance between a token's current embedding and its embedding at its last refresh (Eq. 5). The score therefore measures the displacement from the observation that the cached K/V pair actually represents, so drift that accumulates over several reuses is detected, whereas a consecutive-frame difference may miss it. It is also computed on the latent that the prefill requires anyway. We note that a change signal alone is insufficient in either form (60.40% vs. 60.62% in Table 3), which is precisely what motivates our action-conditioned signal.

3. Different staleness control: a per-token age bound, rather than a global keyframe. TTF-VLA replaces all tokens at periodic keyframes, regardless of their state. This is free in TTF-VLA because its encoder always runs in full, but in a compute-skipping setting a keyframe amounts to a periodic dense prefill. WAM-Cache instead maintains an age counter for each token, which is reset whenever the token is refreshed by any signal (Eqs. 7–8). A forced refresh therefore applies only to tokens that neither content signal has selected for GG consecutive chunks, and its amortized cost is bounded by 1/(G+1)1/(G+1) (Appendix B).

[a] Liu, et al. TTF-VLA: Temporal Token Fusion via Pixel-Attention Integration for Vision-Language-Action Models. AAAI (2026).

AuthorsOpenReview ↗

Rebuttal 2: W1 (continued)

4. Different attention source: action attention, which the WAM architecture provides directly. WAM-Cache uses the action expert's attention to the cached video K/V, averaged over all action queries, layers, and heads. It measures how strongly the policy consumes the very K/V pairs whose refresh is being decided. TTF-VLA describes an action-to-vision variant (its Eq. 8, using the first action token at a single layer), but its reported experiments use text-to-vision attention, which is also the default in its released code. Moreover, the WAM architecture makes action attention inexpensive to obtain: the action expert attends over the full cache at every chunk, regardless of which tokens are refreshed. A text-based score would instead come from the video DiT's text cross-attention, where visual tokens act as queries, so it is available only for recomputed tokens unless additional computation is spent on the reused ones. Empirically, action attention is the most important component (81.46% alone vs. 60–65% for change-based rules, Table 3).

5. Complementary rather than alternative. Since TTF-VLA operates on input tokens and WAM-Cache operates on K/V states, the two methods can be combined. TTF-VLA itself is applied on top of VLA-Cache [55] and raises its LIBERO average from 71.3% to 74.0%. Following your suggestion, we conducted the analogous experiment by applying TTF on top of WAM-Cache on LIBERO.

Table D2. Combining TTF with WAM-Cache on LIBERO (success rate, %).

MethodSpatialObjectGoalLongAvg.
Fast-WAM (dense prefill)96.8099.4096.0093.8096.50
+ WAM-Cache96.2099.2097.6090.4095.85
+ WAM-Cache + TTF97.0099.4097.6092.4096.60

The results indicate that TTF further improves WAM-Cache from 95.85% to 96.60% on average, with the largest gain on Long (+2.0), and brings it on par with the dense prefill (96.50%). This confirms that TTF-VLA contributes temporal fusion for robustness, whereas WAM-Cache contributes compute reduction, and that the two are complementary.

We hope this explanation resolves your concern and clearly conveys the contribution of WAM-Cache.

[55] Xu, et al. VLA-Cache: Efficient Vision-Language-Action Manipulation via Adaptive Token Caching. arXiv (2025).

AuthorsOpenReview ↗

Rebuttal 3: W2

W2: Treating the KV-drift oracle as a performance upper bound for all drift prediction rules requires further justification. Even if the true KV changes are known exactly, background regions with larger changes may be less important to refresh than a grasp target with smaller changes. Ranking tokens by drift magnitude does not necessarily reflect their impact on actions, so it remains unclear whether the oracle's task success rate provides an upper bound for other rules.

Thank you for this insightful comment. We fully agree that ranking tokens by drift magnitude does not necessarily reflect their impact on actions. In fact, this is exactly the central message of our paper, and the K/V-drift oracle is designed to demonstrate it. We apologize that our wording "performance ceiling" in the Introduction caused this confusion.

1. Purpose of the oracle: to show that tokens cannot be selected simply by predicting their K/V drift. A natural approach to cache refresh is to predict which tokens' K/V states drift most and refresh those tokens. Pixel difference and latent surprise are two such predictors. If their low success rates were caused by inaccurate drift prediction, a better predictor would close the gap. The oracle removes prediction error entirely: it ranks tokens by the exact K/V drift, measured against a dense prefill of the new frame. As shown in Table D3, it still reaches only 65.34%, about 25 points below the dense model, whereas action attention, which uses no drift information at all, reaches 81.46%.

Table D3. Refresh rules grouped by the information they use (from Table 3, clean RoboTwin 2.0, content budget 0.5, G=2G = 2).

Selection ruleDrift informationSR (%)ρˉ\bar{\rho}
Pixel differenceEstimated60.400.625
Surprise onlyEstimated60.620.615
Oracle K/V driftExact65.340.599
Action attention onlyNone81.460.643
Attention ∪ surprise (Ours)Estimated88.360.584
Fast-WAM (dense prefill)–90.161.000

The results indicate that the bottleneck lies in the drift-ranking criterion itself, rather than in the accuracy of drift prediction: even exact knowledge of how much each token drifts does not reveal which tokens matter for the action. Your example of a background region with large changes and a grasp target with small changes is precisely the failure case we describe in Section 4.3, which motivates our action-conditioned selection.

2. Clarification of our wording and revision. In Section 5.3 of the submission, we described the oracle as "an upper bound for any rule that predicts cache drift". This referred to drift prediction: since the oracle ranks tokens by the exact K/V drift, no drift predictor can rank tokens more accurately. It was not intended as an upper bound on task success. We agree that, since drift ranking does not determine action impact, a less accurate drift predictor may occasionally refresh more action-relevant tokens and achieve a higher success rate. Our conclusion does not rely on such a bound: it rests on the comparison in Table D3, in which exact drift falls far behind attention without any drift information. We have therefore revised the paper:

  • Section 1{\color{blue}\text{Section 1}}: "This performance ceiling reveals that ..." → "This reveals that ..."
  • Section 5.3: we have removed the sentence "It is an upper bound for any rule that predicts cache drift, not a practical method."
  • Appendix C{\color{blue}\text{Appendix C}}: we have added the definition of the oracle score (cosine distance between the cached and the exact keys and values, summed over all layers), together with the statement that it does not bound task success.

We hope this clarification resolves your concern. Thank you again for helping us make this point more precise.

Reviewer_5PMZOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

WAM-Cache accelerates world-action models without retraining by reusing video-DiT key-value states across action chunks. It combines visual change, action attention, and bounded cache age to select tokens for refresh. Simulation benchmarks, real-world tasks, and ablation studies show reduced prefill computation while largely preserving task performance.

Critical strengths

Addresses a practical inference bottleneck without retraining. Targeting video-DiT prefill makes the approach useful alongside methods that accelerate action decoding.

Informative ablations support the refresh strategy. Comparisons with change-based and oracle-drift selection show the value of action attention, while age-bound ablations demonstrate the importance of preventing prolonged reuse. These experiments provide a clear empirical rationale for the design.

Critical weaknesses

The contribution over existing caching methods is insufficiently isolated. WAM-Cache combines action-guided selection with periodic refresh, but the VLA-Cache comparison changes both components. Adding the same age bound to existing methods would clarify whether the gains require action-conditioned selection or mainly reflect avoiding indefinite reuse.

Practical acceleration remains incompletely validated. Evaluation emphasizes prefill computation and latency rather than end-to-end control efficiency, and stronger scaling results use enlarged contexts without corresponding policy evaluation. Testing only Fast-WAM further limits evidence that the benefits generalize across WAM architectures.

Questions for authors

How does VLA-Cache perform with the same age bound and matched recomputation budget? Does action-conditioned selection retain a meaningful advantage under this control? What end-to-end latency improvement remains at native input sizes when observation encoding, cache selection and updates, and action decoding are all included under matched execution settings?

4 comments
AuthorsOpenReview ↗

Rebuttal 1: W1 & Q1 (first part)

Thank you for your thoughtful and valuable feedback. It is very encouraging to us that you find that our work "addresses a practical inference bottleneck without retraining", that the "informative ablations support the refresh strategy", and that the experiments "provide a clear empirical rationale for the design".

You also raised weaknesses and questions that we found very helpful. We have analyzed each of them and respond below. Changes are highlighted in blue in the revision.{\color{blue}\text{Changes are highlighted in blue in the revision.}}

W1: The contribution over existing caching methods is insufficiently isolated. WAM-Cache combines action-guided selection with periodic refresh, but the VLA-Cache comparison changes both components. Adding the same age bound to existing methods would clarify whether the gains require action-conditioned selection or mainly reflect avoiding indefinite reuse.

Q1 (first part): How does VLA-Cache perform with the same age bound and matched recomputation budget? Does action-conditioned selection retain a meaningful advantage under this control?

Thank you for this valuable suggestion. Following it, we have added the same age bound to VLA-Cache at a matched recomputation budget, and included the results in the revision (Appendix F{\color{blue}\text{Appendix F}}).

1. VLA-Cache with the same age bound.

Table C1. VLA-Cache with and without the age bound on clean RoboTwin 2.0 (50 tasks, 100 episodes per task).

MethodAge boundSR (%) ↑ΔSR (pp)ΔFLOPs (%)
Fast-WAM (dense prefill)–90.160.00.0
+ VLA-Cachenone86.16−4.0−40.8
+ VLA-Cache with age boundG=2G = 286.46−3.7−40.9
+ WAM-Cache (Ours)G=2G = 288.36−1.8−41.6

The age bound raises VLA-Cache by only 0.3 points. Under this control, WAM-Cache remains 1.9 points above it at a slightly larger FLOPs reduction, so action-conditioned selection retains its advantage. The gains therefore do not mainly reflect avoiding indefinite reuse.

2. The same conclusion from the ablation in the paper. Every rule in Table 3 runs with the same age bound G=2G = 2 and the same content budget, so the rules differ only in which tokens they select. Action attention alone reaches 81.46%, against 60.40–65.34% for the three change-based rules.

3. The age bound is not a matter of extra recomputation either. The age bound also adds recomputation, so the same FLOPs could instead be spent on larger content budgets. We compare the two on randomized RoboTwin 2.0 at a matched recompute ratio.

Table C2. Age bound against a larger content-only budget on randomized RoboTwin 2.0 (50 tasks, 100 episodes per task).

Method(ba,bs)(b_a, b_s)Age boundρˉ\bar{\rho}ΔFLOPs (%)SR (%) ↑ΔSR (pp)
Fast-WAM (dense prefill)––1.0000.089.660.0
+ Content-only selection(0.6, 0.2)none0.630−37.085.40−4.3
+ WAM-Cache (Ours)(0.4, 0.3)G=2G = 20.630−37.089.56−0.1

At the same recompute ratio, content-only selection with larger budgets is 4.2 points below the age-bounded setting. The two components therefore contribute separately: action-conditioned selection decides which tokens to refresh, and the age bound removes the tokens that rank-based selection would otherwise reuse indefinitely (Section 4.4).

AuthorsOpenReview ↗

Rebuttal 2: W2 & Q1 (second part)

W2: Practical acceleration remains incompletely validated. Evaluation emphasizes prefill computation and latency rather than end-to-end control efficiency, and stronger scaling results use enlarged contexts without corresponding policy evaluation. Testing only Fast-WAM further limits evidence that the benefits generalize across WAM architectures.

Q1 (second part): What end-to-end latency improvement remains at native input sizes when observation encoding, cache selection and updates, and action decoding are all included under matched execution settings?

Thank you for these insightful questions. We answer the three parts in turn.

1. End-to-end latency at the native input size. Table C3 breaks one inference chunk into its stages at the native size (Sv=120S_v = 120, N=2N = 2 action-denoising steps).

Table C3. Latency of one inference chunk by stage (ms).

StageLatency (ms)
Text encoding20.1
VAE encoding7.7
Video DiT prefill, dense77.9
Video DiT prefill, WAM-Cache64.3 (1.21×)
Action expert, N=2N = 280.1
Complete observation-to-action, with WAM-Cache172.2

WAM-Cache changes only the prefill stage, which drops from 77.9 ms to 64.3 ms and saves 13.6 ms per chunk. At the native input size the end-to-end improvement is therefore modest, and we state this limitation in the Conclusion. With only 120 visual tokens, the prefill latency is dominated by fixed per-layer overheads, so the latency gain trails the 41.6% FLOPs reduction. Our main contribution, WAM-Cache, targets the computation of the prefill, which accounts for 83.5% of the DiT FLOPs at N=2N = 2 (Appendix D), and its latency benefit grows with the visual context (Figure 3). The cost of cache selection and updates is small: extracting the action attention adds about 0.0009 TFLOPs per chunk, 0.09% of the dense prefill, and keeping the cache across chunks takes about 66 MB of memory.

A further finding that matters for end-to-end latency. Fast-WAM uses N=10N = 10 action-denoising steps, at which the action expert alone takes 390 ms per chunk and dominates the latency. We find that N=2N = 2 is sufficient: it cuts the action denoising to 80.1 ms, a 4.9× reduction, while staying within 1.5 points of N=10N = 10 on RoboTwin 2.0 and 0.6 points on LIBERO (Appendix D). Together with WAM-Cache, one complete observation-to-action chunk takes 172.2 ms, against 495.7 ms for the original Fast-WAM configuration with a dense prefill and N=10N = 10.

2. Enlarged contexts. These measurements are meant to show how the efficiency of WAM-Cache scales with the visual context, rather than to report policy accuracy. Policy evaluation at these sizes would require a Fast-WAM checkpoint trained at the larger resolution, which is not available (Section 5.2), and WAM-Cache is training-free and does not retrain the base model. We therefore replay the refresh ratios measured at Sv=120S_v = 120 and measure cost only. All accuracy results in the paper are obtained at the native sizes.

3. Other WAM architectures. Following your suggestion, we have applied WAM-Cache to two other WAMs, Cosmos Policy [21] and OpenWAM [50], and added the results to the revision (Appendix G{\color{blue}\text{Appendix G}}). Cosmos Policy uses a different video backbone (Cosmos-Predict2) and has no separate action expert: actions are denoised as latent frames by the same DiT. OpenWAM couples a Wan2.2-5B video DiT and an action DiT by joint self-attention and denoises the future frames and the action chunk jointly. We use the setting of Fast-WAM on LIBERO, (ba,bs)=(0.5,0.1)(b_a, b_s) = (0.5, 0.1) and G=2G = 2, without retuning.

Table C4. WAM-Cache on other WAMs on LIBERO (success rate, %). ρˉ\bar{\rho} is the computation on the visual tokens relative to the dense model.

Base modelMethodρˉ\bar{\rho} ↓SpatialObjectGoalLongAvg.
Cosmos PolicyDense1.00097.499.697.896.897.90
Cosmos Policy+ WAM-Cache0.65497.099.897.896.697.80
OpenWAMDense1.00099.299.699.697.899.05
OpenWAM+ WAM-Cache0.66299.099.899.295.698.40

The results show that WAM-Cache removes 34.6% and 33.8% of the visual-token computation on the two models, while the average success rate stays within 0.1 and 0.7 points of the dense model.

We hope these responses address your concerns. If you have any further questions, please don't hesitate to let us know. Thank you again for your time and expertise.

[21] Kim, et al. Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning. arXiv (2026).

[50] Wang, et al. OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining. arXiv (2026).

Dear Reviewer 5PMZ,

We greatly appreciate the time and effort you’ve invested in reviewing our work. With less than two days remaining before the discussion deadline, we wanted to reach out regarding the rebuttal we submitted earlier.

If possible, we would be grateful for any feedback you might be able to provide, as this would give us the opportunity to engage in a productive discussion before the deadline. We’re eager to address any remaining questions or concerns you may have about our paper.

We understand that you likely have a busy schedule, and we truly appreciate your valuable insights and expertise in this process.

Thank you for your time and consideration.

Best regards,

Authors of Submission 8350

Dear Reviewer 5PMZ,

I apologize for reaching out again after my previous message. With the discussion deadline now less than one day away, I wanted to very gently follow up on our earlier correspondence regarding our paper and rebuttal.

We completely understand if you haven't had the opportunity to review our rebuttal yet, given the many demands on your time. However, if you have had the chance to look it over, we would be immensely grateful for any feedback you could provide. Your insights, no matter how concise, would be valuable in helping us engage in a constructive discussion before the deadline closes.

We are sincerely thankful for your continued involvement in this process, and we respect your time and professional commitments. Thank you for your understanding and consideration.

Best regards,

Authors of Submission 8350

Reviewer_BfXmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes WAM-Cache, a training-free method for reducing the video DiT prefill cost in world action models. It retains visual key-value representations across action chunks and refreshes tokens selected by latent change, attention from the preceding action chunk, or a maximum reuse age. Refreshed tokens attend to the full cache, including retained entries. Experiments on Fast-WAM cover RoboTwin 2.0, LIBERO, and two real-robot tasks. The default settings reduce prefill FLOPs by roughly 32 ~ 42%, with relatively small decreases in task success. The paper also compares refresh criteria and reports prefill latency measurements.

Critical strengths

The paper demonstrates a useful compute–accuracy tradeoff without retraining the policy. On clean RoboTwin 2.0, WAM-Cache reduces prefill FLOPs by 41.6% while success drops from 90.16% to 88.36%; a larger refresh budget nearly recovers dense performance. The evaluation covers two simulation benchmarks and two real-robot tasks, and Appendix D confirms that the benefit also holds at ten denoising steps. I found the comparison of refresh signals particularly informative: action-conditioned selection substantially outperforms even the KV-drift oracle. This provides meaningful evidence for the design beyond the aggregate benchmark results.

Critical weaknesses
  • Native-context prefill speedups are 1.23× and 1.16×, but complete observation-to-action timings are missing, including a clear accounting of attention extraction and cache-management overhead. The larger-context measurements test cost without validating policy accuracy.

  • The contribution of the age bound is not sufficiently isolated. Adding it also increases recomputation, and neither VLA-Cache with the same age bound nor a larger content-only budget is evaluated at matched FLOPs.

  • Ranking tokens by exact representation drift does not establish an upper bound on task success for every drift-based selector.

Questions for authors
  • What are the absolute prefill and complete observation-to-action latencies under the same execution backend?

  • At matched FLOPs, how does WAM-Cache compare with VLA-Cache using the same G = 2 age bound and with content-only selection using a larger refresh budget?

  • How is the KV-drift oracle score normalized and aggregated across layers?

  • What justifies treating its task success as an upper bound for all drift-based selection rules?

5 comments
AuthorsOpenReview ↗

Rebuttal 1: W1 & Q1

We sincerely thank you for your careful and thorough review. We truly appreciate your recognition that the paper "demonstrates a useful compute–accuracy tradeoff without retraining the policy", that the comparison of refresh signals is "particularly informative", and that it "provides meaningful evidence for the design".

You also raised weaknesses and questions that we found very helpful. We have analyzed each of them and respond below. Changes are highlighted in blue in the revision.{\color{blue}\text{Changes are highlighted in blue in the revision.}}

W1: Native-context prefill speedups are 1.23× and 1.16×, but complete observation-to-action timings are missing, including a clear accounting of attention extraction and cache-management overhead. The larger-context measurements test cost without validating policy accuracy.

Q1: What are the absolute prefill and complete observation-to-action latencies under the same execution backend?

Thank you for these thoughtful and detailed questions. We answer the three parts in turn.

1. Absolute prefill and observation-to-action latencies. Table B1 breaks one inference chunk into its stages at the native size (Sv=120S_v = 120, N=2N = 2 action-denoising steps).

Table B1. Latency of one inference chunk by stage (ms).

StageLatency (ms)
Text encoding20.1
VAE encoding7.7
Video DiT prefill, dense77.9
Video DiT prefill, WAM-Cache64.3 (1.21×)
Action expert, N=2N = 280.1
Complete observation-to-action, with WAM-Cache172.2

WAM-Cache changes only the prefill stage, which drops from 77.9 ms to 64.3 ms. This is a 1.21× speedup, and it saves 13.6 ms per chunk. At the native size the end-to-end gain is therefore modest, for two reasons. First, with only 120 visual tokens the prefill latency is dominated by fixed per-layer overheads, so the latency gain trails the 41.6% FLOPs reduction (Figure 3 and the Conclusion). Second, the other stages are untouched. WAM-Cache targets the computation of the prefill, which accounts for 83.5% of the DiT FLOPs at N=2N = 2 (Appendix D), and its latency benefit grows with the visual context.

A further finding that matters for end-to-end latency. Fast-WAM uses N=10N = 10 action-denoising steps, at which the action expert alone takes 390 ms per chunk and dominates the latency. We find that N=2N = 2 is sufficient: it cuts the action denoising to 80.1 ms, a 4.9× reduction, while staying within 1.5 points of N=10N = 10 on RoboTwin 2.0 and 0.6 points on LIBERO (Appendix D). Together with WAM-Cache, one complete observation-to-action chunk takes 172.2 ms, against 495.7 ms for the original Fast-WAM configuration with a dense prefill and N=10N = 10.

2. Attention extraction and cache management.

  • Attention extraction. The action-to-video attention used for selection is computed at the first action-denoising step, from the query and key projections that the action expert already produces. It adds only the attention logits and the softmax over the video keys, 2LaSa(Sv+Sa)d≈0.00092 L_a S_a (S_v + S_a) d \approx 0.0009 TFLOPs per chunk, which is 0.09% of the dense prefill.
  • Cache management. Keeping the video keys, values, and hidden states across chunks takes about 66 MB of memory in bfloat16, which is negligible relative to the 12 GB model. The surprise score, the two top-kk selections, and the age counters each cost less than a single projection in one module (Appendix B).

3. Larger-context measurements. Thank you for this careful observation. The purpose of these measurements is to show how the efficiency of WAM-Cache scales with the visual context, rather than to report policy accuracy. The native input has only 120 visual tokens, at which fixed overheads hide most of the FLOPs reduction, so we enlarge the context to reach the regime in which the prefill dominates. Policy evaluation at these sizes would require a Fast-WAM checkpoint trained at the larger resolution, which is not available (Section 5.2), and WAM-Cache is training-free and does not retrain the base model. We therefore replay the refresh ratios measured at Sv=120S_v = 120 and measure cost only. All accuracy results in the paper are obtained at the native sizes.

AuthorsOpenReview ↗

Rebuttal 2: W2 & Q2

W2: The contribution of the age bound is not sufficiently isolated. Adding it also increases recomputation, and neither VLA-Cache with the same age bound nor a larger content-only budget is evaluated at matched FLOPs.

Q2: At matched FLOPs, how does WAM-Cache compare with VLA-Cache using the same G = 2 age bound and with content-only selection using a larger refresh budget?

Thank you for this valuable suggestion. We have run both comparisons at matched FLOPs and added them to the revision (Appendix F{\color{blue}\text{Appendix F}}).

1. VLA-Cache with the same age bound. We add the age bound G=2G = 2 to VLA-Cache on clean RoboTwin 2.0 at the same prefill FLOPs.

Table B2. VLA-Cache with and without the age bound on clean RoboTwin 2.0 (50 tasks, 100 episodes per task).

MethodAge boundSR (%) ↑ΔSR (pp)ΔFLOPs (%)
Fast-WAM (dense prefill)–90.160.00.0
+ VLA-Cachenone86.16−4.0−40.8
+ VLA-Cache with age boundG=2G = 286.46−3.7−40.9
+ WAM-Cache (Ours)G=2G = 288.36−1.8−41.6

The age bound raises VLA-Cache by only 0.3 points, and WAM-Cache remains 1.9 points above it at a slightly larger FLOPs reduction. Avoiding indefinite reuse therefore does not explain the advantage of WAM-Cache, which comes from which tokens are selected. Table 3 of the paper isolates the same factor: every rule there runs with G=2G = 2 and the same content budget, and action attention reaches 81.46% against 60.40–65.34% for the change-based rules.

2. A larger content-only budget against the age bound. The age bound also adds recomputation, so the same FLOPs could instead be spent on larger content budgets. We compare the two on randomized RoboTwin 2.0 at a matched recompute ratio.

Table B3. Age bound against a larger content-only budget on randomized RoboTwin 2.0 (50 tasks, 100 episodes per task).

Method(ba,bs)(b_a, b_s)Age boundρˉ\bar{\rho}ΔFLOPs (%)SR (%) ↑ΔSR (pp)
Fast-WAM (dense prefill)––1.0000.089.660.0
+ Content-only selection(0.6, 0.2)none0.630−37.085.40−4.3
+ WAM-Cache (Ours)(0.4, 0.3)G=2G = 20.630−37.089.56−0.1

At the same recompute ratio, content-only selection with larger budgets reaches 85.40%, which is 4.2 points below the age-bounded setting and 4.3 points below the dense prefill. The gain of the age bound therefore does not come from the additional recomputation. Larger content budgets still select by rank, so a token outside the cutoff can be reused indefinitely (Section 4.4), and only the age bound removes this failure.


AuthorsOpenReview ↗

Rebuttal 3: W3 & Q3 & Q4

W3: Ranking tokens by exact representation drift does not establish an upper bound on task success for every drift-based selector.

Q3: How is the KV-drift oracle score normalized and aggregated across layers?

Q4: What justifies treating its task success as an upper bound for all drift-based selection rules?

Thank you for these careful and insightful comments. We fully agree that ranking tokens by exact drift does not establish an upper bound on task success for every drift-based selector, and we have revised the paper accordingly. We first describe the revision, and then explain how the oracle is computed and what it is intended to show.

1. Clarification of our wording and revision (W3 & Q4). We apologize for the confusion caused by our wording. In Section 5.3 of the submission, we described the oracle as "an upper bound for any rule that predicts cache drift". This referred to drift prediction: since the oracle ranks tokens by the exact K/V drift, no drift predictor can rank tokens more accurately under this measure. It was not intended as an upper bound on task success. As you point out, drift ranking does not determine action impact, so a less accurate drift predictor may occasionally refresh more action-relevant tokens and achieve a higher success rate. Our conclusion does not rely on such a bound: it rests on the comparison in Table B4 below, in which exact drift falls far behind attention without any drift information. We have therefore revised the paper:

  • Section 1{\color{blue}\text{Section 1}}: "This performance ceiling reveals that ..." → "This reveals that ..."
  • Section 5.3: we have removed the sentence "It is an upper bound for any rule that predicts cache drift, not a practical method."
  • Appendix C{\color{blue}\text{Appendix C}}: we have added the definition of the oracle score given below, together with the statement that it does not bound task success.

2. How the oracle score is computed (Q3). At each chunk tt, the oracle first runs a dense prefill of the new frame to obtain the exact keys and values at every layer. Let ul[i]u^l[i] denote the concatenation of the cached key and value of token ii at layer ll, and utl,∗[i]u^{l,\ast}_t[i] the concatenation of the exact ones. The score of token ii is the cosine distance between the two, summed over all LL layers:

dt[i]=∑l=1L(1−cos⁡(ul[i], utl,∗[i])).d_t[i] = \sum_{l=1}^{L} \Big(1 - \cos\big(u^l[i],\, u^{l,\ast}_t[i]\big)\Big).

  • Normalization. The cosine distance is invariant to the scale of the representations, so every layer contributes a term in [0,2][0, 2] regardless of its activation norm, and no layer dominates the score. No further normalization is applied.
  • Aggregation. The per-layer distances are summed with equal weights over all LL layers.

The oracle refreshes TopK(dt,0.5)\text{TopK}(d_t, 0.5) together with the tokens that have reached the age bound G=2G = 2, as in the other rows of Table 3. The dense prefill is used only for ranking and is then discarded.

3. Purpose of the oracle (W3 & Q4). A natural approach to cache refresh is to predict which tokens' K/V states drift most and refresh those tokens. Pixel difference and latent surprise are two such predictors. If their low success rates were caused by inaccurate drift prediction, a better predictor would close the gap. The oracle removes prediction error entirely by ranking tokens with the exact drift dtd_t. As shown in Table B4, it still reaches only 65.34%, about 25 points below the dense model, whereas action attention, which uses no drift information at all, reaches 81.46%.

Table B4. Refresh rules grouped by the information they use (from Table 3, clean RoboTwin 2.0, content budget 0.5, G=2G = 2).

Selection ruleDrift informationSR (%)ρˉ\bar{\rho}
Pixel differenceEstimated60.400.625
Surprise onlyEstimated60.620.615
Oracle K/V driftExact65.340.599
Action attention onlyNone81.460.643
Attention ∪ surprise (Ours)Estimated88.360.584
Fast-WAM (dense prefill)–90.161.000

The results indicate that the bottleneck lies in the drift-ranking criterion itself, rather than in the accuracy of drift prediction: even exact knowledge of how much each token drifts does not reveal which tokens matter for the action. This is the failure case described in Section 4.3, which motivates our action-conditioned selection.

We hope this clarification resolves your concerns. If you have any further questions, please don't hesitate to let us know. Thank you again for helping us make this point more precise.

Reviewer_4fjVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WAM-Cache, a training-free framework to reduce the inference cost of World Action Models by reusing key-value (KV) features across consecutive observation chunks. Instead of recomputing all visual tokens, the method selects tokens for refresh based on visual changes, action attention, and a fixed age limit to prevent stale features from accumulating errors. The approach is evaluated on Fast-WAM using RoboTwin 2.0, LIBERO, and real-world robot tasks. Experimental results show that WAM-Cache reduces video DiT prefill FLOPs by 32–42%, while maintaining performance close to the original model. The paper also compares the method with several token reduction and caching baselines and provides ablation studies to examine the role of each selection strategy.

Critical strengths
  1. The paper addresses an important inference bottleneck in World Action Models. The proposed method reduces repeated computation through KV reuse without requiring model retraining.

  2. The refresh strategy combines visual changes, action attention, and a fixed age limit. This provides a reasonable way to balance computational cost and feature freshness.

  3. The ablation studies provide useful evidence for the proposed design. In particular, they show that action attention is more effective than using visual changes alone to select tokens for refresh.

  4. The method is training-free and does not require changes to the original model parameters. This makes it relatively easy to apply to existing WAM inference pipelines.

Critical weaknesses
  1. The method is only evaluated on Fast-WAM. Its effectiveness on other WAM architectures remains unclear, which limits the evidence for its general applicability.

  2. The reduction in prefill FLOPs does not lead to an equally large latency improvement. At the original visual token count, the reported speedup is only 1.23×, so the actual inference benefit needs further investigation.

  3. The action-conditioned refresh strategy relies on attention information from the previous observation chunk. This information may become outdated when the task changes quickly, but the paper provides limited analysis of this issue.

  4. The real-world evaluation is relatively small, covering only two tasks with 20 trials per task. More experiments are needed to demonstrate the reliability of the method across different manipulation tasks.

Questions for authors
  1. Can WAM-Cache be applied to other WAM architectures besides Fast-WAM? Have the authors tested whether the method works with different video backbones or action experts?

  2. Why does the actual latency improvement remain relatively small (1.23×) when the prefill FLOPs are reduced by more than 40%? Can the authors provide further latency results under different hardware settings or larger visual inputs?

  3. The paper mentions that action attention from the previous chunk may become outdated when the task changes its phase. Can the authors provide more analysis of how this affects task performance, especially on tasks with frequent state changes?

  4. Can the authors provide more real-world experiments or additional trials to show that the method remains reliable across different manipulation tasks?

6 comments
AuthorsOpenReview ↗

Rebuttal 1: W1 & Q1

Thank you for your constructive comments. We are greatly encouraged by your assessment that our work "addresses an important inference bottleneck in World Action Models", that it is "training-free" and "relatively easy to apply to existing WAM inference pipelines", and that the ablation studies "provide useful evidence for the proposed design".

You also raised weaknesses and questions that we found very helpful. We have analyzed each of them and respond below. Changes are highlighted in blue in the revision.{\color{blue}\text{Changes are highlighted in blue in the revision.}}

W1: The method is only evaluated on Fast-WAM. Its effectiveness on other WAM architectures remains unclear, which limits the evidence for its general applicability.

Q1: Can WAM-Cache be applied to other WAM architectures besides Fast-WAM? Have the authors tested whether the method works with different video backbones or action experts?

Thank you for this valuable suggestion. Following it, we have applied WAM-Cache to two other WAMs, Cosmos Policy [21] and OpenWAM [50], and added the results to the revision (Appendix G{\color{blue}\text{Appendix G}}).

The two models differ from Fast-WAM in both respects you mention:

  • A different video backbone and no separate action expert. Cosmos Policy fine-tunes the Cosmos-Predict2 video model into a policy without architectural changes. Actions are encoded as latent frames and denoised by the same DiT as the visual latents.
  • A different coupling between video and action. OpenWAM pairs the Wan2.2-5B video DiT with a 1B-parameter action DiT. The two streams are coupled by joint self-attention at every layer and denoise the future frames and the action chunk jointly.

We apply WAM-Cache to the visual tokens of each model on LIBERO, with the setting used for Fast-WAM on this benchmark, (ba,bs)=(0.5,0.1)(b_a, b_s) = (0.5, 0.1) and G=2G = 2, without retuning.

Table A1. WAM-Cache on other WAMs on LIBERO (success rate, %). ρˉ\bar{\rho} is the computation on the visual tokens relative to the dense model. The Fast-WAM rows are from Table 2 of the paper.

Base modelMethodρˉ\bar{\rho} ↓SpatialObjectGoalLongAvg.
Cosmos PolicyDense1.00097.499.697.896.897.90
Cosmos Policy+ WAM-Cache0.65497.099.897.896.697.80
OpenWAMDense1.00099.299.699.697.899.05
OpenWAM+ WAM-Cache0.66299.099.899.295.698.40
Fast-WAMDense1.00096.899.496.093.896.50
Fast-WAM+ WAM-Cache0.67796.299.297.690.495.85

The results show that WAM-Cache removes 34.6% and 33.8% of the visual-token computation on Cosmos Policy and OpenWAM, while the average success rate stays within 0.1 and 0.7 points of the dense model. Most of the loss on OpenWAM falls on Long, as it does on Fast-WAM. The same budgets and age bound therefore carry over to a different video backbone and to a model without a separate action expert.

[21] Kim, et al. Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning. arXiv (2026).

[50] Wang, et al. OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining. arXiv (2026).

AuthorsOpenReview ↗

Rebuttal 2: W2 & Q2

W2: The reduction in prefill FLOPs does not lead to an equally large latency improvement. At the original visual token count, the reported speedup is only 1.23×, so the actual inference benefit needs further investigation.

Q2: Why does the actual latency improvement remain relatively small (1.23×) when the prefill FLOPs are reduced by more than 40%? Can the authors provide further latency results under different hardware settings or larger visual inputs?

Thank you for raising this important point. We answer the three parts of your question in turn.

1. Why the gain is small at the native size. At the native resolution the prefill processes only Sv=120S_v = 120 visual tokens, of which about 70 are refreshed per chunk. At this size, the latency of each layer is dominated by fixed overheads such as kernel launches, normalization, and cache gather/scatter. These overheads do not shrink with ρˉ\bar{\rho}, so a FLOPs reduction above 40% yields only a 1.23× latency gain. We state this limitation in the Conclusion.

2. A different GPU. Following your suggestion, we repeated the measurements on an NVIDIA RTX A6000 (48 GB).

Table A2. Prefill speedup over the dense prefill on clean RoboTwin 2.0. The bound 1/ρˉ1/\bar{\rho} is 1.71× on both GPUs.

SvS_vRTX PRO 6000 (paper)RTX A6000
1201.23×1.20×
4801.47×1.52×
10801.67×1.65×

Table A3. WAM-Cache on the RTX A6000 with the default budgets and G=2G = 2.

Benchmark(ba,bs)(b_a, b_s)SR (%)ρˉ\bar{\rho}FLOPs (T)ΔFLOPs (%)
RoboTwin 2.0, clean(0.4, 0.1)88.060.5850.616−41.5
RoboTwin 2.0, randomized(0.4, 0.2)88.820.6070.639−39.3
LIBERO, Spatial(0.5, 0.1)96.600.6800.584−32.0
LIBERO, Object(0.5, 0.1)99.000.6730.578−32.7
LIBERO, Goal(0.5, 0.1)97.800.6860.589−31.4
LIBERO, Long(0.5, 0.1)90.600.6690.575−33.1
LIBERO, Avg.(0.5, 0.1)96.000.6770.582−32.3

The results show that the speedup follows the same trend on both GPUs. The success rates and FLOPs reductions on the A6000 are also close to those in the paper (88.36% and 88.32% on RoboTwin 2.0, 95.85% on LIBERO).

3. Larger visual inputs. As Table A2 shows, the speedup rises steadily as the number of visual tokens grows. From Sv=120S_v = 120 to 480 and 1080, it increases from 1.23× to 1.47× and 1.67× on the RTX PRO 6000, and from 1.20× to 1.52× and 1.65× on the RTX A6000. With more tokens, compute rather than fixed overhead dominates each layer, so the speedup approaches the 1/ρˉ1/\bar{\rho} bound and reaches 98% and 96% of it at 1080 tokens.

AuthorsOpenReview ↗

Rebuttal 3: W3 & Q3

W3: The action-conditioned refresh strategy relies on attention information from the previous observation chunk. This information may become outdated when the task changes quickly, but the paper provides limited analysis of this issue.

Q3: The paper mentions that action attention from the previous chunk may become outdated when the task changes its phase. Can the authors provide more analysis of how this affects task performance, especially on tasks with frequent state changes?

Thank you for this insightful question. We have carried out a per-chunk analysis of this effect and added it to the revision (Appendix H{\color{blue}\text{Appendix H}}).

Setup. We run closed-loop episodes with (ba,bs)=(0.5,0.1)(b_a, b_s) = (0.5, 0.1) and G=2G = 2 on two tasks with explicit stage switches: LIBERO-Long "put both the alphabet soup and the tomato sauce in the basket", where the robot releases the soup and turns to the sauce, and RoboTwin 2.0 handover_block, where the left arm hands the block to the right arm. At each chunk, WAM-Cache selects tokens with the action attention inherited from the previous chunk (the stale attention). Purely for measurement, we also compute the fresh attention on the current frame with a full prefill, and record three quantities:

  • Focus overlap: the fraction of the fresh top-kk tokens that are also in the stale top-kk, for k=5k = 5 and 1515.
  • Worst stale rank: the lowest rank, in the stale attention, of any fresh top-15 token. A rank beyond the attention budget (49 of 98 tokens on LIBERO, 60 of 120 on RoboTwin 2.0) means that the attention signal alone would miss that token.
  • Coverage: which signal refreshes each fresh top-15 token (attention, surprise, or age bound), or whether it is missed.

Table A4. Stale versus fresh action attention at stage switches. Episode means are in parentheses.

Stage switch (chunk)Top-5 overlapTop-15 overlapWorst stale rank / budgetFresh top-15 refreshed by attention / surprise / age / missed
LIBERO: soup released (20)0.20 (0.74)0.93 (0.78)18 / 4915 / 0 / 0 / 0
handover_block: right arm takes block (8)0.20 (0.53)0.87 (0.73)60 / 6015 / 0 / 0 / 0
handover_block: left arm releases (9)0.60 (0.53)0.80 (0.73)83 / 6013 / 1 / 1 / 0

1. The attention does become stale at stage switches. At the LIBERO release and at the RoboTwin 2.0 handover, the top-5 overlap drops to 0.20, the lowest value in each episode.

2. Two cases arise, and both are covered. When the relevant region stays the same and only the ranking within it shifts (LIBERO release, handover), the new focus tokens still rank high in the stale attention, so the attention signal refreshes them itself. When a previously ignored region suddenly becomes relevant (left-arm release, where a new focus token ranked 83rd), the stale attention misses it, and the surprise signal and the age bound refresh it instead. No fresh top-15 token is missed at any switch. Over whole episodes only 0.9–1.1% are missed, and they are not concentrated at switches.

3. Impact on task performance. In closed loop, this configuration solves the LIBERO task in 49/50 trials (dense: 45/50). Staleness does hurt when the attention budget is narrow and reuse is long: with ba=0.1b_a = 0.1–0.20.2 and G=3G = 3, multi-stage LIBERO-Long tasks drop sharply (45/50 → 7/50 on this task), while single-stage tasks are unaffected. This matches the mechanism above, and it is why WAM-Cache pairs a wide attention budget with the surprise signal and the age bound.


From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.