acceptodds
Under review as a conference paper at ICLR 2027

AIA-VLA: Adaptive Inference Acceleration for Vision-Language-Action Models

Abstract

Vision-language-action models (VLAs) combine the semantic understanding of pretrained vision-language models with diffusion-based action generation, demonstrating strong capabilities in robotic manipulation. However, their computational cost and inference latency hinder deployment on resource-constrained robotic platforms. Acceleration strategies with fixed computational budgets cannot readily accommodate varying computational demands across observations and inference stages, potentially compromising task performance. We present AIA-VLA, a training-free adaptive inference acceleration framework that uses intermediate inference signals to jointly regulate visual token budgets, language model execution depth, and feature reuse in diffusion-based action models. Specifically, AIA-VLA determines visual token budgets through cumulative task-relevant attention, uses recent inter-layer representation stability to trigger subsequent layer skipping, and uses feature drift between consecutive fully computed diffusion steps to decide whether to reuse attention and MLP branch outputs in the next step. These modules can operate independently or in combination, enabling input-dependent computation allocation without modifying model parameters. Evaluated on the SIMPLER benchmark with CogACT as the base model, AIA-VLA reduces floating-point operations by 50% and achieves a 1.54× inference speedup, while improving task success rate by 1.16 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot learning, efficient inference, adaptive computation, robotic manipulation, embodied AI

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_j85zOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AIA-VLA, a training-free framework for accelerating diffusion-based Vision-Language-Action inference. It combines adaptive visual token pruning, language-model layer skipping, and diffusion feature reuse, using intermediate attention and representation-drift signals to regulate computation. Evaluated on CogACT with SIMPLER, the default configuration reduces FLOPs by approximately 49.4% and achieves a 1.54× inference speedup, with an observed success-rate increase of 1.16 percentage points over the original model.

Critical strengths
  1. The paper addresses the practical problem of VLA inference latency without requiring additional training or changes to pretrained parameters.
  2. The three modules target distinct stages of the inference pipeline and can operate independently or jointly.
  3. The evaluation includes all eight module combinations, multiple acceleration settings, and paired task outcomes against the original CogACT, providing useful information about the performance–efficiency trade-off.
Critical weaknesses

W1. Missing comparison with the closely related EfficientVLA. The paper explicitly identifies EfficientVLA as a training-free framework combining visual token selection, language-model layer pruning, and diffusion feature caching, closely matching the three stages targeted by AIA-VLA. However, EfficientVLA is absent from the experimental comparisons. Without this baseline, the incremental benefits of the proposed adaptive mechanisms over an existing framework covering the same inference stages remain unclear. W2. The benefit of adaptive computation over static acceleration is not convincingly established. At approximately matched FLOPs, AIA-VLA has higher latency than the static baseline, increasing from 566.6 ms to 635.4 ms, while success rate increases from 78.13% to 79.17%. This represents approximately 12.1% higher latency for a 1.04-percentage-point difference in success rate. The paper does not provide paired statistical analysis against the static baseline or comparisons at matched latency. Consequently, the results do not yet establish that adaptive allocation offers a compelling practical advantage over simpler fixed schedules. W3. The supporting analyses do not sufficiently validate the predictive decision rules. The layer-skipping analysis shows that omitting different layers produces different action errors, but does not directly establish that low drift across recently executed layers predicts that the next two layers can be safely skipped. Similarly, the diffusion analysis does not clearly establish whether drift measured between preceding fully computed steps reliably predicts the action error caused by feature reuse at the next step. The analyses motivate adaptive decisions, but provide insufficient validation of the specific signals and thresholds used to make them. W4. Narrow evaluation limits conclusions about applicability and deployment. The evaluation is restricted to CogACT on four task categories in the SIMPLER simulation benchmark. There are no experiments on additional VLA architectures or physical robots. This limited scope leaves both cross-model applicability and practical deployment benefits unverified. In particular, the paper does not demonstrate whether its inference savings improve end-to-end responsiveness or task execution under real-world conditions.

Questions for authors

Q1, Could you compare AIA-VLA directly with EfficientVLA under the same hardware and evaluation protocol, including matched latency or FLOPs budgets, to isolate the benefits of adaptive computation? Q2, Is the success-rate difference between AIA-VLA and the static baseline statistically supported by paired outcomes? Does AIA-VLA retain an advantage when the static baseline is tuned to match its latency rather than its FLOPs? Q3, Can you directly evaluate whether each decision signal predicts the error of the subsequent intervention? For example, could you report action-error distributions and the frequency of high-error interventions when the layer-skipping or feature-reuse criterion is satisfied, together with threshold sensitivity on held-out observations? Q4 Do the proposed rules transfer to another VLA architecture without extensive threshold retuning? Can you provide real-robot evaluation measuring end-to-end control latency, task completion time, and responsiveness to environmental changes?

Reviewer_uoeROpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors introduce a training-free method to accelerate the diffusion-based VLAs by adapting computation to intermediate inference signals. To achieve this, they propose three parts. They first rank visual tokens by the text-to-image attention weights after the first LLM layer, and only preserve the smallest subset with respect to a specific cumulative attention mass. Then, they propose a skipping strategy to skip the next two layers when three consecutive drift measurements are sufficiently small. Last but not least, they measure the MLP-feature drift between two consecutive denoising steps, if the drift is sufficiently small, then they reuse the previous step’s attention and MLP outputs at the next step. They conduct experiments using CogACT on four SIMPLER tasks (a totoal of 864 paired evaluation cases) with a 1.54x speedup and a slight success rate enhancement.

Critical strengths
  1. The authors propose an interesting pipeline that contains visual token pruning, LLM layer skipping and reusing diffusion features, all without updating the model parameters.

  2. They also demonstrated a significant speed up while slightly enhancing the task performance in 864 paired evaluation cases. The ablation experiments are extensively conducted with all eight combination of the proposed three parts with the signal analyses that help characterize each part's contribution and the trade-off between computation and task performance.

Critical weaknesses
  1. My major concerns is how much AI used in the paper writing and implementation. For example, Figure 1 is clearly AI-generated with low resolution and apparent artifacts. Although it is not a problem, at least, the authors should draw the figure themselves to remove those artifacts. More importantly, Table 2 reports 78.56% success, which cannot appear from an integer number of successes over 864 trials under the apparent pooled evaluation protocol. So, I am wondering how much we can trust about this paper and the reported numbers. There are more evidence that supports my concerns. For example, In Appendix B.2, they promise a computational breakdown without numerical measurements. Table 7 leaves its correlation target inadequately defined. To me, the current paper looks like a result from auto-research without sufficient human evolvement to check its correctness.

  2. The novelty of this paper is very limited. All token pruning, layer skipping and feature caching are standard tricks in CV community. Engineering them together does not demonstrate a sufficient contribution without any theoretical motivation to support such design.

  3. There is no statistical significance demonstrated in all experiments. Moreover, the paired outcomes against unaccelerated CogACT yield an exact McNemar about 0.5, assuming independent evaluation pairs. Therefore, I believe the so called better reliability or matching of task difficulty is over-claimed.

  4. Small changes in previously computed representations do not imply that subsequent layers or denoising computations can safely be ignored. Similarly, the diffusion analysis does not clearly state that past feature drift can predict the next step’s reuse error. Thus, I believe it is important to conduct more direct comparisons and evaluations should against more simple schedules to justify the contributions of the proposed design choice.

  5. The experiments only cover only one backbone without any cross-architecture or validations on the real robot. This largely reduces the significance of the proposed work especially the claimed contributions are around deployment efficiency.

Questions for authors

See my concerns in Critical Weaknesses.

1 comment

The authors did not provide any rebuttal during the response period. Thus, I have decided to keep my score.

Reviewer_kdSKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AIA-VLA is a training-free framework for reducing the inference cost of diffusion-based vision-language-action (VLA) models. The method targets three parts of inference. First, it uses text-to-visual attention to select an input-dependent number of visual tokens. Second, it monitors changes in hidden representations to decide when to skip language-model layers. Third, it uses cross-step MLP feature drift to decide when to reuse cached attention and MLP outputs during diffusion-based action generation. The method does not modify pretrained model parameters. Evaluated on CogACT with the SIMPLER benchmark, the method reports a 49.4% reduction in FLOPs, a 1.54× inference speedup, and an observed increase in task success from 78.01% to 79.17%.

Critical strengths

S1. The method needs no training and no change to the model weights. It can be applied to a pretrained VLA directly.

S2. The paper reports substantial computational savings. Table 1 reports a 49.4% reduction in FLOPs and a 1.54× inference speedup over the original CogACT. Table 2 evaluates all eight combinations of the three modules and reports them openly, including cases where a module lowers success. This gives useful evidence about the effect of each module on efficiency and task performance.

S3. The evaluation protocol is careful in some ways. All methods use the same 864 instances, random seeds, and initial states. Table 8 gives paired outcomes.

S4. The paper is honest about some limits. It states that the static baseline is faster. It also says the tests cover one model, in simulation only.

Critical weaknesses

Major:

W1. The contribution over existing work is not sufficiently demonstrated. EfficientVLA (Yang et al., 2025) already combines visual token selection, language-model pruning, and diffusion feature caching on CogACT. AIA-VLA differs by using online inference signals, but the paper does not directly compare these strategies. Table 1 includes CogACT, a static baseline, and VLA-Cache, but not EfficientVLA. A controlled comparison is needed to establish the benefit of the proposed adaptive decisions.

W2. The claim that adaptive computation beats static computation is not supported. In Table 1, the static baseline reaches 78.13% success and AIA-VLA reaches 79.17%. The paper reports no statistical test and no repeated runs. The static baseline is also faster (566.6 ms vs 635.4 ms), so the adaptive method costs more time for a small, untested gain. The matched-FLOPs design is reasonable, but the paper tests only one static plan. It does not give the layer skip and reuse schedules or say how they were chosen. The authors should compare against several static plans with matched FLOPs and latency, and report several seeds with confidence intervals.

W3. The ablation does not clearly support the claim that the modules "work synergistically", a term the paper does not define. In Table 2, visual pruning alone (M100) gives the highest success (80.09%), and the full method (M111) gives 79.17%. Layer skipping alone (M010) gives 76.97%, below the baseline (78.01%). The FLOPs savings are sub additive, and adding diffusion reuse to M110 changes FLOPs only from 0.511 to 0.506. M111 beats each two-module combination by only 0.6 to 0.8 points (78.36–78.59%). Differences across the table are small (about 1 to 3 points) on 864 instances, with no confidence intervals or repeated runs, so they may be noise. The table shows that the modules can be combined while keeping success close to the baseline. It gives no evidence of a benefit beyond the sum of the parts.

Minor:

The paper lists three threshold presets (Section 4.1) but does not say how the values, the window size, or the skip length were chosen, or whether they were tuned on the 864 SIMPLER instances.

Section 4.2 contains a missing space in "points.These", section 4.3 contains a missing space in "Table 2,employing" and section 4.4 contains a missing space in "latency.The".

The abstract says "50%" FLOPs reduction. Section 4.2 says "49.4%". Use one value.

Section 4.1 says "As discussed in Section 4" while it is already inside Section 4. Cite the subsection.

Questions for authors

Q1. How does AIA-VLA compare with EfficientVLA under the same model, benchmark, and hardware settings? What specific advantage do the online adaptive decisions provide?

Q2. How were τv, τl, and τd, the window size, and the skip length chosen? Were they tuned on the SIMPLER evaluation instances? Please show a sensitivity analysis.

Q3. Why does diffusion reuse alone cut latency by about 23% while FLOPs change by only 0.4% (Table 2)? Please give hardware, number of runs, and a stage-wise breakdown including the decision overhead.

Q4. How often does each module reach its maximum? Does the amount of computation follow input difficulty?

Suggestions for improvement:

Add confidence intervals and several seeds to Tables 1 and 2.

Add a stage-wise latency table (encoder, language model, diffusion head, decision overhead).

Plot computation used against per-input difficulty to support the claim of input-dependent allocation.

Compare the first-layer attention score with other pruning scores, for example layer-2 attention.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.