acceptodds
Withdrawn from ICLR 2027

AC-Sparse: Action-Centric Dynamic Activation Sparsity for Efficient Vision-Language-Action Model Inference

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models enable robotic manipulation but require substantial computation during inference. Existing methods for efficient VLA inference often rely on coarse-grained layer skipping or token optimization, while computation reuse depends on temporal similarity between observations. Activation sparsification offers a fine-grained alternative by reducing linear computation while preserving the network structure and token sequence. However, existing sparsification criteria often overlook how perturbations at different tokens affect action generation, potentially degrading task performance at high sparsity levels. We propose AC-Sparse, a training-free dynamic activation sparsification method for VLA inference. Specifically, AC-Sparse combines attention weights from action representations with activation magnitudes and weight-column norms to estimate the cost of removing each token–channel activation. We formulate an attention-weighted expected squared local output perturbation objective and derive a separable upper bound that AC-Sparse minimizes under a fixed sparsity budget for each linear projection. We further prove that the derived sparsification criterion exactly minimizes the original objective when the weight columns are mutually orthogonal. Experiments across different VLA architectures and manipulation benchmarks demonstrate that AC-Sparse better preserves task success than the evaluated sparsification baselines at high sparsity levels, achieving an average success rate only 0.7 percentage points below dense inference on LIBERO at 65% activation sparsity.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Efficient Inference, Training-Free Activation Sparsification, Action-Conditioned Token Relevance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dSKsOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AC-Sparse, a training-free activation sparsification method for efficient VLA inference. The method combines action attention, activation magnitude, and weight norms to estimate the importance of each token-channel activation, and uses a shared sparsity budget to dynamically remove less important activations.

Critical strengths

The paper is easy to follow, and the proposed method is simple and training-free, making it easy to apply to existing pretrained VLA models without additional training.

Critical weaknesses
  • The novelty of the proposed method is quite limited. AC-Sparse is essentially an extension of WINA with action attention and a global Bottom-K selection strategy. Both action-aware importance estimation and activation sparsification are well-established techniques, and many previous works have explored similar ideas. I find it difficult to identify a substantial methodological contribution beyond combining these existing techniques.

  • The paper does not demonstrate actual inference acceleration. As the authors acknowledge in the Limitations section, the proposed method only reduces theoretical FLOPs and does not provide any wall-clock latency measurements. Since fine-grained dynamic sparsity requires specialized sparse kernels to achieve real speedups, it remains unclear whether AC-Sparse can actually improve inference efficiency in practice. This raises concerns about the practical value of the proposed method.

  • The paper lacks real-world robot experiments. All evaluations are conducted in simulation, making it difficult to assess whether the proposed method is effective for real-world robotic control. Given that efficient inference is particularly important for real-time robot deployment, I would like to see experiments on physical robots to better demonstrate its practical usefulness.

Questions for authors

Please see Weaknesses.

Reviewer_KW82OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AC-Sparse, a training-free dynamic activation sparsification method for VLA inference. For each linear projection, it scores every token–channel activation by combining three factors: activation magnitude, weight-column norm, and the attention that token receives from action representations. It then removes the lowest-scoring activations under a sparsity budget shared across all tokens. On VLA-Adapter and π0.5 (LIBERO, LIBERO-Plus), AC-Sparse degrades much less than TEAL and WINA at 65–75% sparsity.

Critical strengths
  1. The action-centric sparsity allocation is intuitive and well motivated for VLA models, where different tokens may have substantially different contributions to action prediction.
  2. The experimental results show promising task-success preservation at high sparsity levels, especially compared with existing activation sparsification baselines.
  3. The method is training-free and is evaluated on different VLA architectures.
Critical weaknesses
  1. Lack of actual inference speedup evaluation. My primary concern is that the paper is motivated by efficient VLA inference, yet it does not report any wall-clock latency, throughput, or actual inference speedup measurements. While the paper demonstrates substantial theoretical FLOPs reductions, it remains unclear whether these savings translate into practical acceleration. This is a significant limitation because improving inference efficiency is the central motivation of the paper. Without empirical runtime measurements, the practical efficiency benefits of AC-Sparse remain unverified.
  2. Although the paper motivates local output perturbation as a tractable surrogate for final action discrepancy, the analysis in Section 4.3 only examines next-layer perturbations with fixed attention weights. The relationship between this surrogate and final action sensitivity remains insufficiently validated.
  3. All results are in simulation, with no real-robot experiments, which is essential for demonstrating the true effectiveness of th method.
Questions for authors
  1. Could you report wall-clock latency on a realistic deployment GPU? It should include score computation, top-K selection, and attention extraction, along with end-to-end FLOPs for the full model.
  2. Figure 3 shows that AC-Sparse mainly shifts budget toward visual tokens. Does the gain come from token-level attention or from modality-level reallocation? Two baselines would separate these: WINA with modality-specific sparsity ratios matched to your modality shares, and WINA with modality normalization but no attention.
  3. Can you provide evidence that the proposed local perturbation surrogate reliably reflects final action prediction errors?
  4. Could the authors add real-world evaluations to demonstrate the effectiveness of the method?
Reviewer_46ohOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AC-Spaer study the problem about training-free activation-sparsity action-representation attention. The result with two VLA architectures shows strong performance preservation at high sparsity, but practical end-to-end acceleration remains unverified.

Critical strengths

The result that at 65% sparsity, VLA-Adapter reaches 93.40% on LIBERO, only 0.70 points below dense is promising.

Critical weaknesses
  1. The practical end-to-end acceleration remains unverified.
  2. The paper should cover more results on different benchmarks, such as robotwin and robodojo.
Questions for authors
  1. Could you report end-to-end latency and memory of different variant model?
  2. Could you report more results on different benchmarks?
  3. How were TEAL/WINA and token-pruning baselines adapted and tuned?

Comments on the paper

1 comment

Dear Reviewers,

Thank you for the review. The authors have responded to the points you raised, and the response period closes on 18 November (AoE). I would be grateful if you could post a brief reply on whether your concerns are resolved, partly resolved, or still open, so that the authors have a fair chance to address them. If anything still needs clarifying, please ask now while they can still respond.

Regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.