acceptodds
Under review as a conference paper at ICLR 2027

VLA-HWBench: Where the Time Goes When VLA Models Run on Edge Hardware

Abstract

Vision–language–action (VLA) models increasingly run on edge devices aboard robots, where inference latency constrains closed-loop robot control. However, it remains unclear how to design VLA models, their inference software and edge hardware so that robots can run VLA inference in real time. Most VLA models report only task success, and prior studies cover few of the ways recent VLAs generate actions, split a call into at most three phases and do not compare inference optimizations on the same models. Therefore, we introduce **VLA-HWBench**, a benchmark that measures 18 open VLA models on their released software and on alternative backends, on an NVIDIA Jetson AGX Thor and an AMD Ryzen AI Max+ PRO 395, with a seven-stage breakdown of each call. Our study reveals three insights. First, where the time goes depends on how a model generates actions, not on its size. Action generation is the largest stage in 14 of the 19 configurations on the Jetson, latency spans on the Jetson and on the AMD platform, and parameter count explains at most 15% of its variance. Models that generate long action chunks in a few flow-matching steps keep up with control, whereas token-by-token decoding does not. Second, kernel fusion with 4-bit weights helps most, running and GR00T N1.7 2.6–2.9 faster than PyTorch with torch.compile, but it exists for only these two models and only on the Jetson. Optimization also moves the time to the stages it shortens least, as CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call. Third, after optimization, memory bandwidth limits only the stages that process few tokens on the Jetson, such as the denoising steps, whereas software limits all other stages, from unfused kernels to serial preprocessing on the host CPU. Overall, low-latency edge VLAs should use few action-generation iterations, inference software should extend kernel fusion and low-bit weights to more models and platforms, and edge hardware needs high memory bandwidth and a fast host CPU.

Primary area:
datasets and benchmarks
Keywords:
VLA, benchmark, latency

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_rSRPOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VLA-HWBench characterizes inference latency for 18 open VLA models, comprising 19 configurations, on two edge hardware platforms. It decomposes inference into seven stages, compares compilation and specialized backends, and analyzes bottlenecks using kernel profiling and roofline estimates. Measurements use a shared real LIBERO observation, supplemented by frame variation experiments and optimization checks. The study finds that sequential action generation often dominates latency, specialized backends substantially accelerate two supported models, and optimization exposes CPU preprocessing and other remaining bottlenecks.

Critical strengths
  1. Broad and detailed characterization of heterogeneous VLA implementations. The study covers four action-generation forms across two platforms and includes preprocessing and postprocessing in its timing boundary. Tables 7–9 show substantial differences in inference latency and stage-level costs that parameter count alone cannot explain. This makes the benchmark a useful resource for identifying where optimization effort is likely to matter.

  2. Concrete diagnosis beyond reporting speedups. Section 5.3 and Appendix G connect latency to identifiable implementation choices, including repeated context projections, inefficient patch embedding, and CPU worker-thread overhead. The thread-count experiment in Figure 6b provides particularly useful evidence that host configuration can materially affect inference latency.

Critical weaknesses
  1. The breadth of measurements is not matched by the depth or generality of the insights. Insight 1 substantially overlaps with Finding #1 of Zhou et al., leaving its additional contribution unclear. Insight 2 compares all models without optional optimizations in Figure 4, but derives its optimization recommendations primarily from two FlashRT-supported models in Figure 5. Figure 4 should use compiled configurations as the baseline for all models where supported. This would provide a stronger basis for Insight 3 to analyze how architectural differences affect the benefits of kernel fusion and the resulting stage-level bottlenecks, rather than further diagnosing only the same two models. The CPU preprocessing analysis is also limited to comparing 1 and 14 threads on Jetson, without a broader thread-count sweep across both platforms. These limitations weaken the paper’s broader model, software, and hardware design recommendations.

  2. Task performance after optimization is insufficiently validated. Table 14 measures action agreement on only 11 frames, despite reported gripper-command flips and larger deviations under NVFP4. Without task-success measurements, it remains unclear whether the headline speedups preserve policy effectiveness.

Questions for authors
  1. Does a broader thread-count sweep on both platforms support the CPU preprocessing conclusions, and what thread configuration was used for AMD in Figure 6b?

  2. Given that Insight 1 identifies sequential action generation as a major bottleneck, why does the main optimization comparison focus on compilation, specialized backends, and precision rather than also evaluating algorithmic methods that reduce action-generation iterations? Could including such methods at comparable task success change the optimization recommendations in Insight 2?

  3. Since both platforms share physical memory between the CPU and GPU, what accounts for the reported host-to-device latency: explicit buffer copies, synchronization, or other runtime overhead? Would a shared-buffer or zero-copy implementation reduce this cost on other platforms?

6 comments
AuthorsOpenReview ↗

Response to Reviewer rSRP (Part 1)

We thank the reviewer for the detailed and constructive feedback, and for recognizing the breadth of our characterization and the concrete diagnoses in Section 5.3 and Appendix G. We ran new experiments for each point and answer them below. Code and data are available at https://anonymous.4open.science/r/VLA_HWBench-4B77/.

W1. Relation of Insight 1 to Zhou et al.

Finding #1 of Zhou et al. [1] states that model size does not predict latency. Insight 1 goes further and identifies what does. Over 11 configurations spanning flow matching, token-by-token decoding and reasoning, iterations × active weight bytes explains 64% and 88% of the log-latency variance on Thor and AMD, and predicts held-out checkpoints with a Q² of 0.50 and 0.84, against 0.04 and 0.22 for parameter count. Insight 1 also connects latency to control. Only 10 of the 19 configurations keep up with the 20 Hz control rate of LIBERO on Thor, and the slower forms of action generation do not report higher success. We will cite Finding #1 and state this difference in the paper.

W1. Compiled baselines in Figure 4 and the effect of architecture.

We added compiled baselines for six models on both platforms, with 120 interleaved eager and compiled calls per model. The table gives the speedup of the median latency over eager execution, with the same weights and matmul precision in both runs.

Compiled regionThorAMD
π0.5 full model call, two views and 10 actions1.79×1.78×
π0.5 full model call, three views and 50 actions1.64×1.75×
SmolVLA full model call2.76×1.88×
OpenVLA-OFT language model1.03×1.14×
VLA-Adapter language model1.17×1.03×
SimpleVLA-RL language model1.09×1.08×
DeepThinkVLA masked CoT, language model and vision1.11×1.12×

The gain depends on how much of the call is compiled. Compiling the full model call of π0.5 and SmolVLA places vision, prefill and every denoising step in one graph and speeds them up 1.6 to 2.8 times. Compiling only the language model gains 1.03 to 1.17 times, close to the 1.02 to 1.15 times of compiling only the action head of GR00T N1.7 in Section 5.2. The bottleneck left after optimization also depends on the architecture. With 574 to 712 prefix tokens, prefill grows from 14% to 34% of the π0.5 call, while GR00T N1.7 prefills only 156 tokens and is left with CPU preprocessing as its largest stage at 39%. We will add the compiled points to Figure 4.

W1 and Q1. Thread sweep on both platforms.

The broader sweep supports our conclusions. We measured the processor and collator time of GR00T N1.7, its CPU preprocessing path, with 1, 2, 4 and 8 intra-op threads and the PyTorch default on both platforms, with 200 calls per setting. The AMD bar in Figure 6b uses the default of 16 threads.

Threads1248Default
Thor, default 145.8 ms6.2 ms7.1 ms6.9 ms11.5 ms
AMD, default 161.9 ms1.7 ms1.8 ms1.7 ms2.1 ms

On Thor, waking the idle worker threads doubles this time at the default setting, while the AMD host is insensitive to the thread count and remains 3 times faster even with one thread. The best setting also depends on how often the model is called. With NVFP4 on Thor, 4 threads give 37.5 ms per call against 45 to 47 ms with 14 threads when calls run back to back, while both settings give 44 ms with 50 ms of idle time after each call.

AuthorsOpenReview ↗

Response to Reviewer rSRP (Part 2)

W2. Task performance after optimization.

We now report LIBERO-10 success for the compiled, FlashRT FP16, FP8 and NVFP4 levels of Section 5.2, with 50 episodes per level.

ThorEagerCompiledFlashRT FP16FP8NVFP4
π0.5 success44/5042/5045/5040/5043/50
π0.5 latency317 ms124 ms90 ms61 ms55 ms
GR00T N1.7 success49/5046/5043/5046/5045/50
GR00T N1.7 latency113 ms96 ms81 ms57 ms38 ms

In larger runs with 200 episodes per configuration, the headline speedups stay within 3 points of native success. In the same runs, NVFP4 is 3.6 times faster than the native GR00T N1.7 runtime and two denoising steps shorten its call by 19%. We also ran this policy in real time, with the simulator advancing at 20 Hz while the policy runs.

GR00T N1.7 on ThorPaused, of 200Real time, 400 ms replanning, of 100Real time, 100 ms replanning, of 100
Native, 4 steps1838987
Two denoising steps1778688
NVFP41779387

Compiled π0.5, whose actions can flip a gripper command, succeeds in 169 of 200 episodes on AMD against 170 natively at 1.8 times lower latency.

Q2. Methods that reduce iterations.

Reducing iterations is an algorithmic change to the model, while Section 5.2 studies system optimizations that keep the weights and the algorithm fixed. This is why Insight 1 addresses it as guidance for model design. To show how the two levels relate, we also tested the simplest algorithmic change, fewer denoising steps on the released checkpoints. For GR00T N1.7, W2 shows that two steps stay close to native success in both settings but shorten the call by only 19%. Going from 10 to 5 steps cuts the π0.5 call on AMD from 634 to 533 ms with 168 against 170 successes. Fewer steps shorten only the denoising loop, so at the same success of 177 episodes NVFP4 is still 2.9 times faster than two-step sampling. The two levels are therefore complementary, and the recommendations of Insight 2 hold. Algorithmic reduction pays off most where generation dominates the call, as in VLA-0 and the reasoning models with 91 to 99%.

Q3. Host-to-device time on shared memory.

This stage is small. On AMD it takes 0.79 ms of the 299 ms GR00T N1.7 call, or 0.3%. It consists of 12 explicit HIP copies issued by the PyTorch pipeline, which keeps GPU tensors in a separate memory pool, together with dtype and layout conversion and the synchronization at the stage boundary. Reusing preallocated buffers lowers it to 0.73 ms, and eliminating the stage would save at most 0.79 ms per call in this configuration. The larger host-side cost is CPU preprocessing, discussed in W1 and Q1.

References

[1] K. Zhou et al. Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment. ICML 2026. https://proceedings.mlr.press/v306/zhou26e.html

Thank you for the additional experiments and clarifications. Most of my concerns have been addressed. I have two remaining concerns:

  1. Insight #1 reflects an established execution-cost analysis as novelty, but the relationship between repeated weight accesses and latency may depend on subsequent optimizations, including FlashRT’s CUDA Graphs, buffer reuse, and kernel fusion. Would these optimizations affect the conclusions drawn from Insight 1? If so, could build a more explicit cost model. For example, offline operator profiling combined with an estimate of the model’s theoretical performance limit?
  2. The simulation-frequency setup resembles the RTC experiments, but the difference for the native implementation is small: success decreases from 91.5% with paused simulation to 89% without paused. This is similar to the differences observed with compilation and does not demonstrate that more frequent replanning improves success. Indeed, success rates with 100 ms replanning is even lower than that of NVFP4 with paused simulation. Does this suggest that quantization is more effective than more frequent replanning in this setting?
Reviewer_tHvQOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces VLA-HWBench to benchmark latency on edge for many popular open-sourced VLA models, in particular on HW Jetson AGX Thor and Ryzen AI Max+ PRO 395. For each model, the study looks at different stages in an inference: from CPU preprocessing to postprocessing. Three main questions are answered in this paper: 1) Where does the time go in the reference configurations?, 2) How much do compilation, specialized backends (FlashRT, TensorRT) and low precision help? This is studied on π0.5 and GR00T N1.7 only. 3) What limits the fastest configurations? Its main findings are: i) Latency follows the number of action-generation iterations, not parameter count. ii) Fused kernels with 4-bit weights give the largest gain, but exist for only two models and only on Jetson. iii) After optimization, bandwidth limits the few-token stages and software limits the rest.

Critical strengths
  1. The study covers all four ways VLAs generate actions on both AMD and NVIDIA platforms, using published codes.

  2. Detailed timing at different stages of inference calls, from CPU preprocessing to postprocessing, action generation.

Critical weaknesses

First, regarding novelty, which is limited. Prior work already showed that action generation dominates the action expert is memory-bound ([1] Zhou et al.). It would be better if the paper should state clearly which conclusions are new.

Second, latency is evaluated without task success. In particular, every recommendation trades off against success rate: fewer denoising steps, parallel decoding, longer chunks, NVFP4. Would it be a stronger study if a latency–success Pareto analysis is provided.

Thirsd, the hardware recommendations are not supported by the experiments. However, the performance gap to AMD mostly reflects software maturity, not hardware? The CPU bottleneck is largely a PyTorch thread-pool setting? Power and energy, which are often concerns at the edge, are not reported.

Fourth, the optimization study is too narrow to generalize. Question 2, section 5.2, covers only two models. No runtime is evaluated on AMD, and the C++ runtimes cited in related work are not tested. The claim that kernel fusion with 4-bit weights helps most is therefore based on n = 2.

References:

[1] Zhou et al. Characterizing vision-language-action models across XPUs: Constraints and acceleration for on-robot deployment. ICML 2026.

Questions for authors

Q1: How much latency variance does iterations × weight bytes explain?

Q2: Can you report LIBERO-10 success for each configuration, and closed-loop success for the compiled and FP8/NVFP4 variants?

Q3: Does the CPU bottleneck persist with a single thread or with preprocessing moved to the GPU?

2 comments
AuthorsOpenReview ↗

Response to Reviewer tHvQ (Part 1)

We thank the reviewer for the constructive feedback and for recognizing our coverage of all four forms of action generation on both platforms and our detailed stage-level timing. We answer each question below with new experiments. Code and data are available at https://anonymous.4open.science/r/VLA_HWBench-4B77/.

W1. New conclusions relative to Zhou et al.

Zhou et al. [1] show for five VLAs with two forms of action generation that the action expert is memory-bound. We will cite this result and state our new conclusions explicitly in the paper. First, we refine their view of the backbone. After optimization, vision encoding and long prefills reach only 6 to 42% of the hardware bound because of unfused kernels and missing low-precision support, so they are limited by software rather than by compute. Second, across flow matching, token-by-token decoding and reasoning, latency follows iterations × active weight bytes, and the slowest heads do not report higher success. Third, compilation pays off when it captures the whole model call, with 1.6 to 2.8 times for the full call of π0.5 and SmolVLA against 1.02 to 1.17 times when only the language model or the action head is compiled. Fourth, optimization moves the bottleneck to the host, where CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call.

W2 and Q2. LIBERO-10 success and the latency-success trade-off.

We ran LIBERO-10 for the compiled, FlashRT FP16, FP8 and NVFP4 levels of Section 5.2, with 50 episodes per level.

ThorEagerCompiledFlashRT FP16FP8NVFP4
π0.5 success44/5042/5045/5040/5043/50
π0.5 latency317 ms124 ms90 ms61 ms55 ms
GR00T N1.7 success49/5046/5043/5046/5045/50
GR00T N1.7 latency113 ms96 ms81 ms57 ms38 ms

In larger runs with 200 episodes per configuration, the fastest configurations stay within 3 points of native success. In the same runs, NVFP4 is 3.6 times faster than the native GR00T N1.7 runtime and two denoising steps shorten its call by 19%. We also ran this policy in real time, with the simulator advancing at 20 Hz while the policy runs.

GR00T N1.7 on ThorPaused, of 200Real time, 400 ms replanning, of 100Real time, 100 ms replanning, of 100
Native, 4 steps1838987
Two denoising steps1778688
NVFP41779387
π0.5 on AMDNative, 10 stepsCompiled5 steps2 steps
Latency634 ms361 ms533 ms472 ms
Success of 200170169168165

Across the 19 configurations, the reported LIBERO success has little rank correlation with latency, with Spearman ρ of −0.05 on Thor and −0.09 on AMD. On Thor, X-VLA, the parallel decoder VLA-Adapter and InternVLA-A1.5 form the latency-success frontier, while token-by-token and reasoning heads are up to 82 times slower without reporting higher success. We will add this latency-success plot to the paper.

W3 and Q3. CPU preprocessing with one thread or on the GPU.

The bottleneck persists in both cases. With one thread, the processor and collator of GR00T N1.7 on Thor take 5.8 instead of 11.5 ms, because idle worker threads no longer have to be woken up. This serial work still takes 3 times longer than on the AMD host, which needs 1.9 ms with one thread. Moving the image transforms after cropping to the GPU saves only 0.8 ms, since the rest of the preprocessing remains serial work on the host. The thread-pool setting thus explains only part of the cost, and a fast host core still matters, as recommended in the paper.

AuthorsOpenReview ↗

Response to Reviewer tHvQ (Part 2)

W3. Software maturity, hardware recommendations and energy.

Our stage analysis attributes most of the AMD gap to software, as stated in Insight 3. For example, the ROCm library runs the patch embedding of GR00T N1.7 as 512 small matrix-vector products, and replacing it with a single matrix multiplication cuts the AMD call by 23% and its energy per call by 31%. Our hardware recommendations target the limits that remain after software optimization. On Thor, the fused denoising steps already read their weights at more than half of the peak memory bandwidth, so they are bounded mainly by memory bandwidth, and the serial preprocessing above depends on single-core speed.

We added energy measurements with three 60-second blocks per configuration on each board. On Thor under continuous inference, NVFP4 lowers the energy of a GR00T N1.7 call 3.1 times, from 6.88 to 2.23 J, at nearly the same mean power of 63 W. We will add the energy results to the paper.

W4. Breadth of the optimization study.

We extended compilation to six models on both platforms, with 120 interleaved eager and compiled calls each. Compiling the full model call of π0.5 and SmolVLA speeds them up 1.6 to 2.8 times, while compiling only part of the model of OpenVLA-OFT, VLA-Adapter, SimpleVLA-RL and DeepThinkVLA gains 1.03 to 1.17 times. On AMD, we also ran the C++ runtime vla.cpp, which takes 765 ms per OpenVLA-OFT call against 494 ms for PyTorch, so a C++ runtime alone did not speed up this model. Fused low-bit backends exist for only π0.5 and GR00T N1.7 among the 18 models, and only on Thor, so our study covers every model for which they exist. This scarcity is itself part of Insight 2.

Q1. Variance explained by iterations × weight bytes.

We fitted log latency on 11 configurations that cover flow matching, token-by-token decoding and reasoning. Iterations × active weight bytes explains 64% of the variance on Thor and 88% on AMD, and it also predicts held-out checkpoints best.

Predictor of log latencyR² ThorR² AMDLeave-one-out Q² ThorLeave-one-out Q² AMD
Parameter count0.330.450.040.22
Iteration count0.530.550.320.35
Iterations × active weight bytes0.640.880.500.84

Across all 19 configurations, parameter count explains at most 15% of the variance, as shown in Figure 1.

References

[1] K. Zhou et al. Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment. ICML 2026. https://proceedings.mlr.press/v306/zhou26e.html

Reviewer_iRsPOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper argues that the latency of open VLA models on edge hardware is set mainly by how many sequential action-generation iterations a model runs, not by its parameter count, and that after optimization most stages are limited by software (unfused kernels, missing low-precision support, serial CPU preprocessing) rather than by the hardware, with only the few-token denoising steps bound by memory bandwidth. To show this, VLA-HWBench runs 18 released VLA models on an NVIDIA Jetson AGX Thor and an AMD Ryzen AI Max+ PRO 395, each on its own released software and one LIBERO-10 observation, and splits every call into seven timed stages, including CPU preprocessing. Latency spans 82× on the Jetson, and a fit of log latency on log parameter count explains at most 15% of the variance on either platform. For two models, π0.5 and GR00T N1.7, the paper then adds compilation, a specialized backend and lower precision one level at a time and places the stages of the fastest configurations on rooflines. It concludes that action generation should use fewer iterations, that fused low-bit backends should reach more models and platforms, and that edge hardware needs memory bandwidth and a fast host CPU.

Critical strengths
  1. Broad and careful stage-level measurement of released VLAs on edge hardware. The benchmark covers 18 recent models across all four forms of action generation on two vendors' platforms, and times seven stages, including CPU preprocessing and host-to-device copies, rather than one end-to-end number.
  2. Optimization effects are reported together with their effect on the actions. The paper compares the actions before and after compilation and lower precision and reports where they diverge, for example a compiled π0.5 that can flip a gripper command. It also shows that the stages optimization does not shorten take a larger share afterwards: CPU preprocessing rises from 12% to 39% of the GR00T N1.7 call.
Critical weaknesses
  1. The benchmark is not available for verification during review. The submission provides neither a code or data link nor a reproducibility statement. Since the main contribution is a benchmark, access to the measurement scripts, configurations, and raw timing data is essential for assessing and reusing the work. Without these artifacts, reviewers cannot rerun the benchmark or independently verify the reported measurements.
  2. The conclusions offer limited new insight. Recommendations such as extending fused low-bit backends to more models and platforms, increasing memory bandwidth, and using a faster host CPU appear fairly generic and could apply to a broad range of deep learning inference workloads. The measurements provide useful VLA-specific numbers, but the paper does not clearly establish what new understanding of VLA inference emerges beyond these familiar optimization principles. This limits the conceptual contribution beyond the benchmark itself.
  3. The input setting is too narrow to establish applicability to real robots. The main analysis uses a single LIBERO simulation observation consisting of two 256 × 256 images. Although the paper also examines 24 frames from one episode, this does not address variation in camera resolution or the number of views. Real-robot deployments may use higher-resolution inputs and more cameras, which could increase CPU preprocessing costs and, depending on the model’s input processing, vision-encoding costs. The relative stage costs and bottleneck conclusions therefore may not carry over to these settings. Experiments varying resolution and camera count would help establish the scope of the findings.
Questions for authors
  1. Can the authors provide an anonymized repository containing the benchmark code, exact configurations, and raw timing data needed to reproduce the results?
  2. Which findings are specific to VLA inference, and how do they go beyond established observations about iterative decoding, memory bandwidth, kernel fusion, and host-side overhead?
  3. How do the stage-level latency breakdowns change with higher-resolution camera inputs and additional views? Do the main bottleneck conclusions remain valid under these settings?
7 comments
AuthorsOpenReview ↗

Response to Reviewer iRsP

We thank the reviewer for the constructive feedback and for recognizing our broad and careful stage-level measurements and the joint report of speedups and action changes. We answer each question below with new experiments on both platforms.

W1 and Q1. Benchmark code and data.

The anonymized repository is available at https://anonymous.4open.science/r/VLA_HWBench-4B77/. It contains the measurement harness with one adapter per model and runtime, the configuration of every measured row, the input observations, the raw per-call timing records of all 19 configurations on both platforms, and scripts that regenerate every number, table and figure of the paper. It also contains the code and records of all new experiments in this response, each with a script that recomputes the reported numbers. We will add a reproducibility statement to the paper.

W2 and Q2. Findings specific to VLA inference.

Our main findings follow from how VLAs generate actions, which general inference principles do not capture.

  1. Action chunks decide whether latency can be hidden. A VLA keeps up with control when its latency per executed action stays below the 50 ms control period, as in Figure 4b. We verified this in real time, with the LIBERO-10 simulator advancing at 20 Hz while GR00T N1.7 runs on the same Thor and executes each new chunk asynchronously. Each call generates 16 actions covering 800 ms, so the native runtime, two denoising steps and NVFP4 all keep 86 to 93% success at replanning periods of 400 ms and 100 ms. Nine of the 19 configurations do not keep up on Thor, and for them latency must be removed rather than hidden.
  2. Slow forms of action generation do not report higher success. The LIBERO success reported for the 19 configurations has little rank correlation with their latency, with Spearman ρ of −0.05 on Thor and −0.09 on AMD, and token-by-token and reasoning heads are up to 82 times slower than X-VLA without reporting higher success.
  3. Compilation pays off when it captures the whole model call. Compiling sample_actions of π0.5 and SmolVLA, with vision, prefill and every denoising step in one graph, speeds them up 1.6 to 2.8 times, while compiling only the language model or only the action head of GR00T N1.7 gains 1.02 to 1.17 times.
  4. Camera inputs do not map directly to encoder work. With one valid view, π0.5 still runs two encoder passes, and the released two-view setup of Evo-1 encodes three image slots, so comparisons at a given camera setup must measure the work the encoder actually executes.

These findings make our recommendations specific to VLAs, for example sizing the action chunk and the number of iterations to the control period.

W3 and Q3. Higher camera resolution and more views.

The main conclusions hold, and where the bottleneck shifts, the stage breakdown locates the cause.

Resolution. We rendered the cameras of π0.5 natively at three resolutions.

π0.5 camera resolution256 px512 px720 px
CPU preprocessing on Thor1.7 ms4.4 ms6.4 ms
CPU preprocessing on AMD0.5 ms2.7 ms3.9 ms
Vision encoding on Thor112 ms112 ms112 ms
Vision encoding on AMD190 ms191 ms191 ms

Vision encoding stays constant because the processor resizes each view to 224 × 224, so for fixed-size encoders such as that of π0.5, higher-resolution cameras mainly load the host CPU.

Views. Vision encoding and prefill of Evo-1 take 106, 187 and 262 ms for one, two and three views on Thor, about 78 ms per added view.

Visual tokens. For encoders whose token count grows with resolution, we varied InternVLA-A1.5 from 128 to 800 visual tokens.

InternVLA-A1.5Vision, 128 tokensVision, 800 tokensAction generation, 800 tokens
Thor21 ms102 ms870 ms
AMD98 ms625 ms556 ms

On Thor, action generation remains the largest stage. On AMD, vision overtakes action generation at 800 tokens. The stage breakdown traces this to the patch projection, and replacing it with a single matrix multiplication brings vision back to 202 ms.

We will add these results to the paper.

Thank you for the rebuttal and code release. Could you clarify how your action-chunk synchronization analysis differs from prior work? Measuring it on Thor alone appears to offer limited novelty. The rebuttal recommends choosing the action chunk length and number of denoising steps to fit the control period, but the paper only provides the feasibility condition T/H < 50 ms and uses each model’s released H. Could you vary both parameters for at least one model and report latency per executed action and closed-loop success to demonstrate how this trade-off should be chosen? The new resolution experiment uses π0.5 in its native runtime, where vision encoding takes 112 ms on Thor, making the CPU preprocessing increase from 1.7 to 6.4 ms relatively small. However, the paper shows that preprocessing becomes more important after optimization, increasing from 12% to 39% of the GR00T N1.7 call. In the fastest optimized configuration, such as GR00T N1.7 with NVFP4, what fraction of total call latency does preprocessing account for with 720 px inputs or three camera views, and do the main bottleneck conclusions still hold?

AuthorsOpenReview ↗

Follow-up response to Reviewer iRsP (Part 1)

We thank the reviewer for the follow-up and are glad that the rebuttal and the code release were useful. Below we answer the three remaining questions with new experiments.

Q1. Difference from prior work on action-chunk execution.

Real-time chunking [1] and asynchronous inference [2, 3] propose execution methods, which decide when the next chunk of a given model is computed and how the robot switches from one chunk to the next. Our analysis is about the model itself. Whether a VLA keeps up with control is set by its own design, namely how many actions one call returns and how many iterations produce them, together with the latency this design has on edge hardware. VLA-HWBench measures this for 19 configurations on both platforms, and only 10 keep up on Thor and 9 on AMD, as Figure 4b shows. Our real-time test executes each new chunk asynchronously in the simplest way, only to check this condition, and execution methods such as [1, 2, 3] are complementary to it. The new experiments in Q2 vary the chunk length and the number of denoising steps of GR00T N1.7 and π0.5 and show how these design choices decide closed-loop success, which gives guidance for designing new models.

References

[1] K. Black, M. Y. Galliker, S. Levine. Real-Time Execution of Action Chunking Flow Policies. NeurIPS 2025.

[2] M. Shukor et al. SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics. arXiv:2506.01844, 2025.

[3] Z. Yang et al. Jetson-PI: Towards Onboard Real-Time Robot Control via Foresight-Aligned Asynchronous Inference. CoRL 2026. arXiv:2607.12659.

Q2. Choosing the chunk length and the number of denoising steps.

We varied two parameters for GR00T N1.7 and π0.5 on Thor, the number of denoising steps in each call and the number of actions the robot executes from each chunk before calling the policy again. As in the official clients, the robot stops while the policy computes the next chunk, so the latency of every call adds to the task time and counts against the step limit of LIBERO-10. The policy runs alone on Thor, and each setting has 100 episodes.

GR00T N1.7Latency per callSuccess, 2 actions per callSuccess, 8 actions per call, officialSuccess, 16 actions per call
4 denoising steps, default108 ms41%90%93%
2 denoising steps78 ms50%86%85%
1 denoising step63 ms36%90%78%
NVFP4, 4 denoising steps30 ms73%91%91%
π0.5Latency per callSuccess, 10 actions per callSuccess, 25 actions per callSuccess, all 50 actions
10 denoising steps, default390 ms82%93%84%
5 denoising steps320 ms86%94%88%
2 denoising steps275 ms87%92%90%

Executing too few actions per call makes the robot wait too long. The latency per executed action is the latency per call divided by the actions executed per call. For native GR00T N1.7 executing 2 actions it is about 55 ms, longer than the 50 ms control period, so the robot waits longer than it moves, tasks take 2.5 times longer, and success drops to 41%. All failures in these runs are timeouts.

Executing too many actions per call also lowers success, because the robot follows one chunk for longer without a new observation, as for π0.5 executing all 50 actions.

Fewer denoising steps cost little success at a moderate number of actions per call. At the official 8 actions, GR00T N1.7 succeeds in 90%, 86% and 90% of episodes with 4, 2 and 1 steps, and π0.5 executing 25 actions in 93%, 94% and 92% with 10, 5 and 2 steps. With the simulator paused, X-VLA stays at 92 to 96% from 10 steps down to one. Combined with long open-loop execution, fewer steps do lower success, from 93% to 78% for GR00T N1.7 executing 16 actions with 4 and 1 steps.

The trade-off is therefore chosen in two steps. First, execute enough actions per call that the latency per executed action stays well below 50 ms on the target hardware, but not so many that the robot runs open loop for too long. Second, reduce the denoising steps to shorten each call, keeping the actions per call moderate. The actions per call cannot exceed the chunk length the model is trained to output, so new models should choose their chunk length for the latency of the target hardware. We will add these results to the paper.

Reviewer_zVMtOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The authors propose a test bench to understand how to build a proper VLA system on the edge. To reach their goal, they tested a lot of different algorithms on two different edge platforms: nvidia and amd. They have three claims: They showed that the time budget is related to the way the model generates the actions and not the size of the model. They also demonstrated that kernel fusion is something that is very efficient on edge. Finally, they observed that memory bandwidth is a limiting factor only for stages using few tokens.

Critical strengths

18 models is a very good coverage, and it provides a good support for the claims Originality to test this on edge. Interesting approach on the memory bandwidth estimation.

Critical weaknesses

I think regarding what the authors want to claim, two test platforms are not sufficient. We cannot generalize based on this setup.

TBD-VLA (https://tbd-vla.github.io) already showed with its comparison with 11 models that flow matching models have a better latency and throughput compared to auto-regressive models.

Low bit weight that favor speed is obvious, it is something that we observe on LLM and ViT, not a new concept.

Kernel fusion is also a well known approach to speed up models on edge, I do not see the novelty here.

Questions for authors

What new insights does this study provide beyond established inference optimization practices and previous VLA benchmarking studies, can you give some comparison please?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zVMt (Part 1)

We thank the reviewer for the constructive feedback and for recognizing our model coverage, the originality of measuring on edge hardware, and our memory-bandwidth analysis. We answer each question below with new experiments on both platforms. Code and data are available at https://anonymous.4open.science/r/VLA_HWBench-4B77/.

Q1. New insights beyond established practice and prior VLA benchmarks.

The table compares VLA-HWBench with TBD-VLA [1] and Zhou et al. [2].

TBD-VLA [1]Zhou et al. [2]VLA-HWBench
HardwareRTX A40 server GPUGPUs, NPUs, JetsonJetson AGX Thor, AMD Ryzen AI Max+ 395
Models105 VLAs18
Forms of action generation324
TimingEnd to endThree phasesSeven stages including CPU preprocessing
Optimizations of released modelsNoCompile, 4-bitCompile, TensorRT, FlashRT, FP8, NVFP4, fewer steps
Success after optimizationOwn modelOwn methodsCompiled, FlashRT, FP8 and NVFP4 levels

Beyond this coverage, our measurements give four insights specific to VLA inference on edge hardware.

  1. Latency is set by iterations, and the slowest heads do not report higher success. Across 11 configurations spanning flow matching, token-by-token decoding and reasoning, iterations × active weight bytes explains 64% and 88% of the log-latency variance on Thor and AMD, against 33% and 45% for parameter count on the same configurations. Reported LIBERO success has little rank correlation with latency, with a Spearman ρ of −0.05 on Thor, and token-by-token and reasoning heads are up to 82 times slower than X-VLA without reporting higher success.
  2. Compilation pays off when it captures the whole model call. Compiling sample_actions of π0.5 and SmolVLA, which places vision, prefill and every denoising step in one graph, speeds them up 1.6 to 2.8 times on both platforms. Compiling only the language model of the other models, or only the action head of GR00T N1.7, gains 1.02 to 1.17 times.
  3. Optimization moves the bottleneck to the host. CPU preprocessing grows from 12% to 39% of the GR00T N1.7 call.
  4. Software costs differ across platforms. On AMD, the ROCm library runs the patch embedding of GR00T N1.7 as 512 small matrix-vector products (Section 5.3). Replacing it with a single matrix multiplication on the same weights cuts the AMD call by 23% and its energy by 31% at unchanged success, while the same change saves less than 2 ms on Thor.

W2. Relation to TBD-VLA.

TBD-VLA [1] finds that flow matching is faster than autoregressive decoding, consistent with our Insight 1. We will cite it and add this comparison to the paper. Beyond this, VLA-HWBench finds that the form of action generation alone does not determine latency. On Thor, flow-matching models span 16 times in latency, from 77 ms for X-VLA to 1.2 s for InternVLA-A1.5, and the parallel decoder VLA-Adapter is faster than every flow-matching model except X-VLA. Across flow matching, token-by-token decoding and reasoning, latency follows the number of iterations times the weight bytes each iteration reads. Splitting each call into seven stages on edge hardware further reveals bottlenecks that end-to-end timing cannot show, such as CPU preprocessing growing to 39% of the GR00T N1.7 call after optimization.

W3 and W4. Low-bit weights and kernel fusion.

We agree that both techniques are established, and we use them as tools rather than claim them as contributions. What VLA-HWBench adds is their measured effect on complete VLA calls on edge hardware. With compilation, the fused backend and low-bit weights make π0.5 7.2 times and GR00T N1.7 2.6 times faster on Thor. GR00T N1.7 with NVFP4 keeps its LIBERO-10 success when the simulator runs in real time, with 93 of 100 successful episodes against 89 natively at a 400 ms replanning period. The measurements also show the limits of today's software. Fused low-bit backends exist for only 2 of the 18 models and only on Thor, so optimization widens the latency gap between the two platforms from about 2 times to 6 to 8 times. This motivates our recommendation to extend them to more models and platforms.

AuthorsOpenReview ↗

Response to Reviewer zVMt (Part 2)

W1. Choice of two platforms.

VLA-HWBench targets edge devices that run VLA models on board the robot. Jetson AGX Thor and Ryzen AI Max+ PRO 395 are the flagship edge platforms of NVIDIA and AMD, and both share 128 GB of memory between CPU and GPU, enough to run all 18 models on the robot. They also cover contrasting designs. One pairs a Blackwell GPU with Arm cores and the other an RDNA 3.5 GPU with x86 cores, and they differ 4.4 times in 16-bit throughput but only 1.1 times in memory bandwidth, which helps separate the roles of compute, memory and the host CPU. The main conclusions hold on both platforms. The latency ranking of the 19 configurations agrees across them with a Spearman correlation of 0.91, and on the same 11 configurations iterations × weight bytes explains 64% and 88% of the latency variance on Thor and AMD. Where the platforms differ, the stage breakdown identifies the cause, as in the patch-embedding example above. VLA-HWBench is open source and designed to add platforms, and we will extend it to more edge devices.

References

[1] TBD-VLA. arXiv:2606.07895, https://tbd-vla.github.io

[2] K. Zhou et al. Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment. ICML 2026. https://proceedings.mlr.press/v306/zhou26e.html

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision

We thank all reviewers for their constructive feedback. We have uploaded a revised paper, with new and changed text in blue. The main changes are as follows.

  1. We add the comparison with Characterizing Vision-Language-Action Models across XPUs [1] and TBD-VLA [2] to Section 2.
  2. We state the measurement setting in Section 4.4, together with the extended inputs and how to add new models.
  3. We add compiled baselines, the latency predictor, the LIBERO-10 success after optimization and closed-loop results to Sections 5.1 and 5.2 and Appendices H to J.
  4. We add the cost model, the effect of host settings and the generalization of the bottleneck analysis to Section 5.3 and Appendices K and L.
  5. We add results for higher camera resolution, more views, more visual tokens and energy in Appendices M and N.
  6. We add a reproducibility statement with the anonymous repository.

References

[1] K. Zhou et al. Characterizing Vision-Language-Action Models across XPUs: Constraints and Acceleration for On-Robot Deployment. ICML 2026.

[2] S.-W. Lee, X. Kang, Y.-L. Kuo. TBD-VLA: Temporal Block Diffusion Vision Language Action Model. arXiv:2606.07895, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.