acceptodds
Withdrawn from ICLR 2027

SparseWAM: Relevance-Guided Asymmetric Sparsification for World Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

14 views

Abstract

World Action Models (WAMs) use world-model representations to condition action generation, but compressing these representations can compromise closedloop robustness. We study where sparsification should be applied, how task relevance should guide token selection, and how world-side compression interacts with action-side acceleration. Our analysis shows that sparsifying exported conditioning representations after dense world-model computation preserves task success substantially better than pruning visual tokens before representation formation, while aggressive diversity-based pruning becomes fragile under distribution shift. Motivated by these observations, we introduce Sparse- WAM, a training-free framework for relevance-guided asymmetric sparsification. On the world side, SparseWAM follows a relevance-before-redundancy principle: instruction-conditioned scores first restrict the candidate set, after which diversity-based selection enforces a fixed conditioning budget. On the action side, it adapts similarity-based layer skipping to selected Action-DiT denoising steps. At 50% conditioning retention with ActionSkip, SparseWAM achieves 95.00% on LIBERO and 50.66% on LIBERO-Plus with 369.20 ms inference latency, compared with 455.73 ms for the dense model and 378.40 ms for ActionSkip alone. Under a more aggressive 25% budget, semantic guidance improves LIBEROPlus success from 42.83% to 44.16% with world-side sparsification alone and from 42.50% to 44.22% when combined with ActionSkip, partially recovering the degradation caused by compression. Controlled real-robot shirt-folding trials further show that moderate sparsification retains more successful executions than aggressive joint compression. These results identify compression location and token-selection strategy as important factors in the efficiency–robustness trade-off of WAM inference.

Primary area:
applications to robotics, autonomy, planning
Keywords:
World Action Models, Embodied AI, Future Representation, Token Pruning, Efficient Inference, Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LFghOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SparseWAM, a training-free inference acceleration method for Fast-WAM. It sparsifies World-to-Action Conditioning KV tokens using semantic relevance and diversity-based selection, and applies dynamic layer skipping in the Action DiT to reduce inference cost. Experiments cover LIBERO, LIBERO-Plus, and real-robot manipulation tasks. With 50% KV retention and ActionSkip, inference latency decreases from 455.73 ms to 369.20 ms, while achieving success rates of 95.00% on LIBERO and 50.66% on LIBERO-Plus.

Critical strengths

S1. Useful empirical analysis of sparsification locations in WAMs.

The paper compares input visual token pruning with downstream conditioning KV pruning and examines their effects on control performance. These results provide concrete empirical observations that help inform the choice of compression location in WAM architectures.

S2. Relatively comprehensive internal ablations and efficiency evaluation.

The authors evaluate different KV retention ratios, token selection orders, and ActionSkip both independently and in combination. The latency breakdown, LIBERO-Plus perturbation results, and controlled real-robot trials help characterize the behavior of different compression configurations.

Critical weaknesses

W1. Limited technical novelty and insufficient evidence for the independent contributions of key design choices.

SparseWAM's core acceleration mechanisms largely build on existing QK-affinity-based semantic scoring, DivPrune, and the dynamic layer-skipping strategy of Flash-Unified. The new design primarily involves adapting and combining these mechanisms within Fast-WAM, including grouped KV selection and joint scheduling. However, the current evidence does not sufficiently establish the independent value of these design choices.

In Table 2, Semantic → DivPrune improves over DivPrune alone by only 0.80 and 1.33 percentage points on LIBERO and LIBERO-Plus. Without a random filtering baseline matched for the intermediate candidate budget, it remains unclear whether these gains arise from semantic relevance or merely from modifying DivPrune's candidate set.

W2. Limited incremental efficiency gains from the joint method and insufficient explanation of the main speedup results.

Table 3 shows that ActionSkip alone reduces inference latency from 455.73 ms to 378.40 ms while nearly preserving task success. Adding 50% KV sparsification provides only a further 2.43% latency reduction, while decreasing LIBERO and LIBERO-Plus success rates by 1.80 and 1.27 percentage points.

The more aggressive 25% KV joint configuration achieves approximately 1.28× speedup over dense Fast-WAM, but reduces success rates by 6.65 and 7.78 percentage points. Since ActionSkip alone accounts for most of the acceleration, the incremental value of world-side sparsification remains insufficiently demonstrated. The lack of comparable evaluations against other competitive training-free acceleration methods further limits the assessment of its accuracy–efficiency advantages.

Additionally, the reported ActionSkip speedup requires further explanation. According to the execution schedule in Section 3.4, ActionSkip reduces the nominal number of Transformer block executions by only 12 out of 300 (4 steps × 3 blocks out of 10 steps × 30 blocks, 4%), whereas Table 10 reports a reduction in action-stage latency from 379.66 ms to 322.13 ms (15.15%). Although these quantities need not be strictly proportional, the paper does not provide sufficient computational cost or execution-path analysis to explain this discrepancy, leaving the source of the main speedup unclear.

Questions for authors

Q1. Could the authors provide a Random Filtering → DivPrune baseline with the same intermediate candidate budget (93%) and final KV retention (25%) to evaluate the independent contribution of semantic selection? Please also clarify the intermediate token budget and selection rules used in the DivPrune → Semantic reverse-order ablation.

Q2. Could the authors clarify the actual number of Transformer blocks executed at each ActionSkip denoising step and provide the corresponding FLOPs analysis or finer-grained latency breakdown to explain the discrepancy between the reduction in block executions and the measured latency gains?

Q3. Could the authors provide a broader comparison with representative training-free acceleration methods for VLA/WAMs to better establish SparseWAM's competitiveness in the accuracy–efficiency trade-off?

Reviewer_31kFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SparseWAM, a training-free inference-acceleration framework for pretrained Fast-WAM. It keeps Video-DiT computation dense while sparsifying the conditioning K/V exported to the Action DiT, using instruction-guided candidate filtering followed by DivPrune under a fixed token budget. On the action side, it adapts similarity-based layer skipping at selected denoising steps. Experiments cover LIBERO, LIBERO-Plus, and small-scale real-robot shirt folding. The 50%-retention joint configuration achieves 95.00% and 50.66% simulation success, reducing latency from 455.73 ms to 369.20 ms. More aggressive compression provides further acceleration at a larger success-rate cost.

Critical strengths
  1. The method distinguishes world-representation formation from conditioning transmission. At 25% retention, direct visual pruning achieves 24.90% LIBERO success, whereas conditioning-only pruning after dense representation formation achieves 89.85%, providing clear evidence that compression location matters for closed-loop control.

  2. Fixed-budget comparisons of semantic filtering, diversity selection, and selection order, together with a separate ActionSkip evaluation, make component trade-offs interpretable. At 25% retention, semantic filtering improves LIBERO-Plus from 42.83% to 44.16%, without tuning its parameter on that shifted test set.

Critical weaknesses
  1. The method primarily combines existing DivPrune with adapted Flash-Unified skipping, and the added design offers limited practical gains. Relative to ActionSkip alone, the 50% joint configuration saves only 9.20 ms (2.43%) while losing 1.80/1.27 percentage points on LIBERO/Plus; the 75% joint configuration is slower. The main efficiency results do not establish a compelling incremental advantage over simpler action-side skipping.

  2. The 1.33-percentage-point semantic gain on LIBERO-Plus at 25% retention lacks repeated-evaluation intervals and a random-filtering control, leaving its reliability and semantic specificity unclear. The 50% joint configuration also lacks a diversity-only counterpart. Evaluation covers one Fast-WAM, while real-robot testing uses ten trials per configuration and omits ActionSkip alone and the 50% joint setting, limiting validation of the recommended configuration’s generalization and deployment benefits.

Questions for authors
  1. Under repeated evaluation and a matched candidate-filtering ratio, does semantic guidance consistently outperform random filtering? What is its independent contribution over DivPrune at 50% conditioning retention?

  2. Is the 9.20 ms saving over ActionSkip alone stable under real observations and repeated timing? Does existing evidence establish deployment value relative to the accompanying success-rate loss?

  3. Are there existing real-robot results for ActionSkip alone and the 50% joint setting, or validation on another WAM? If not, please delimit conclusions to the current Fast-WAM and synthetic-input timing protocol.

Reviewer_3PpaOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces SparseWAN, a training-free compression method that combines relevance-based selection as the first step with diversity-based selection. The authors investigate how early vs late compression affects performance and report promising results on a number of benchmarks, as well as a small scale robot task completion investigation.

Critical strengths

The paper addresses a highly practically relevant problem.

The idea of comparing late vs early compression is interesting and would be of interest to many members of the community.

Critical weaknesses

Stylistically, the paper is not very clearly written, and includes multiple borderline redundant or irrelevant statements. Often stating what was not done more clearly than what was.

For example, the 234-236: “This score averages raw QK affinities, rather than softmax-normalized attention probabilities. It serves as an instruction-conditioned ranking heuristic; it is not a calibrated estimate of a token’s causal contribution to control. No additional scorer or trainable parameter is introduced.”

Most of this paragraph is self-evident from the formula & description above. Similar statements are common throughout the paper.

At the same time, many other architectural decisions & differences with what was done before are not described in enough depth.

More importantly, the quality of the experimental support is, unfortunately, not sufficient.

There are very few baseline comparisons, there are no uncertainty bounds in the estimates, and most experiments do not help to form a clear conclusion about the value of the proposed approach.

For example, in the actual robotic task experiment, even setting the small sample size aside, we see a decline in quality as more and more compression is introduced. This is expected. What, therefore, does such an experiment add to our understanding of the method?

I do believe that the results on early/late compression are potentially interesting and useful, but it’s also a very intuitive result. To make a paper based on them, it’s necessary to perform a deeper, compute-normalized investigation, thoroughly comparing with the alternatives

Questions for authors

Suggestions/questions:

1: In my view, the first part of the abstract goes a bit too deep into details which are hard to understand without more context.

I suggest dramatically condensing or even removing the part before “we introduce”. As well as condensing the second part of the abstract that again seems to go in a bit too much detail.

2: Do you think your method could be applied during training too?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.