acceptodds
Under review as a conference paper at ICLR 2027

VLA-Duo: Periodic Cross-Inference Scheduling for Mobile Vision-Language-Action Inference

Abstract

Mobile SoCs are an attractive platform for onboard vision-language-action (VLA) inference in robots with tight space and power constraints. However, their limited compute capacity makes it difficult to generate action chunks fast enough to support high-frequency control. We present VLA-Duo, a periodic GPU–NPU scheduling system that overlaps successive VLA inferences to increase action-chunk throughput at the cost of higher observation-to-action latency. For a fixed task configuration, successive VLA inferences use the same operator sequence and tensor shapes, allowing the same execution plan to be repeated. We generate the plan offline from GPU–NPU performance profiles, accounting for inter-accelerator interference. At runtime, the scheduler follows the plan to coordinate the two accelerators and overlap successive inferences. Short-run evaluation with π₀ and π₀.₅ on Snapdragon 8 Elite and 8 Elite Gen 5 shows that our system achieves 36.1–78.3% higher throughput than ExecuTorch/QNN, our strongest external NPU baseline. Output periods of 394–499 ms support action throughput equivalent to 10.0–12.7 Hz control when five actions are executed per chunk. In separate LIBERO simulations, π₀.₅ achieves 94.5% average success under a ten-control-step observation-to-action delay using prefix inpainting without additional training or gradient computation.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Vision-Language-Action Model, Heterogeneous Inference, Mobile Computation

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_BGATOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a system for running chunked VLA policies on mobile SoCs by overlapping successive inferences across the GPU and NPU. An offline scheduler uses measured interference between operators to select a periodic, single-cut execution plan. Combined with specialized kernels, the system improves throughput by 36.1–78.3% over ExecuTorch/QNN; scheduling alone contributes 17.9–25.0% over the authors’ optimized NPU runtime. Separate LIBERO experiments evaluate prefix inpainting under delayed observations.

Critical strengths

The work addresses a concrete deployment bottleneck and demonstrates gains on two mobile SoC generations and two policies. Comparing against both external runtimes and an optimized internal baseline makes the scheduling contribution reasonably clear.

The scheduling model is evaluated quantitatively. Prediction errors of 0.9–1.7% and cut-selection regret below 0.5%, together with the solo-profile comparison in Appendix B.3, support the usefulness of modeling concurrent execution.

Critical weaknesses

Measurements use cooled phones and only 20 measured inferences per run. Sustained throughput, thermal throttling, power consumption, and timing variability are therefore unresolved. These matter directly for the proposed onboard use case.

The hardware timings and manipulation experiments are evaluated separately using a fixed injected delay. This does not establish performance under actual runtime jitter. Moreover, Table 5 shows that deployment-precision π₀ achieves 76.5%, slightly below naive asynchronous execution at 77.2%, so the benefit of prefix inpainting is not consistent across configurations.

Questions for authors

Can you provide sustained measurements, including temperature, power, output-period variability, and missed deadlines? Does the selected schedule remain effective after thermal stabilization?

Have measured mobile latency traces been used in the closed-loop evaluation? Please also explain the π₀ result in Table 5 and provide uncertainty estimates for the comparison with naive execution.

3 comments
AuthorsOpenReview ↗

Response to Reviewer BGAT (1/2): W1, Q1

We appreciate your careful review and positive assessment. Sustained experiment is an important part we missed for this work. We ran new experiments for your questions, and will update the paper to cover all of them below.

This part (1/2) answers W1 and Q1.

Weakness 1 & Question 1 (Sustained measurements)

Sustained experiment at a fixed period, with the NPU core temperature: [Figure.T_sus].

We measured the sustained output period TsusT_{sus} of VLA-Duo and our optimized NPU-only runtime. We find TsusT_{sus} by binary search over [Test,4Test)[T_{est}, 4T_{est}), where TestT_{est} is the estimated best output period. A period counts as sustainable if the phone outputs action chunks at that period steadily for 5 minutes, and TsusT_{sus} is the shortest such period. A missed deadline (overrun) is a chunk that completes after its release time. The phones are plugged into a charger and cooled by a 40W air cooler.

SoCModelRuntimeTsusT_{sus} (ms)TinferT_{infer}, start / end (ms)LL, mean / max (ms)NPU core, start / end (°C)Overruns / periods
8 Eliteπ0\pi_0NPU-only792605 / 709669 / 71874 / 940 / 417
8 Eliteπ0\pi_0VLA-Duo700480 / 562974 / 100271 / 920 / 470
8 Eliteπ0.5\pi_{0.5}NPU-only896664 / 753714 / 76271 / 920 / 368
8 Eliteπ0.5\pi_{0.5}VLA-Duo792529 / 6171083 / 112073 / 920 / 415
8 Elite Gen 5π0\pi_0NPU-only729505 / 666614 / 68878 / 960 / 417
8 Elite Gen 5π0\pi_0VLA-Duo603394 / 476823 / 92973 / 970 / 536
8 Elite Gen 5π0.5\pi_{0.5}NPU-only828566 / 762717 / 78481 / 980 / 398
8 Elite Gen 5π0.5\pi_{0.5}VLA-Duo668441 / 565982 / 103972 / 950 / 489
  • Missed deadlines: none at TsusT_{sus} in any run.
  • Output-period variability: none at TsusT_{sus}. Each chunk is released at a fixed time after its observation (TsusT_{sus} for NPU-only, 2Tsus2T_{sus} for VLA-Duo). Heat slows inference by 13–35% within the five minutes, but LL max stays below the release time in every run, so every chunk is released on time.
  • Temperature: the NPU core rises to 92–98 °C.
  • Power (whole phone on battery with the screen off, π0\pi_0 on the 8 Elite): about 5.9 W for NPU-only and 6.7 W for VLA-Duo at their TsusT_{sus}, against 0.1 W when idle. The energy per action chunk is the same, about 4.7 J.
  • After thermal stabilization: VLA-Duo keeps 13% higher throughput than NPU-only on the 8 Elite and 21–24% on the 8 Elite Gen 5.

We will add the sustained experiment and the power measurement to Section 6.3.

AuthorsOpenReview ↗

Response to Reviewer BGAT (2/2): W2, Q2

This part (2/2) answers W2 and Q2.

Weakness 2 & Question 2 (Closed loop with measured timing, π0\pi_0 in Table 5)

(1) Measured timing in the closed loop

VLA-Duo completes LIBERO tasks 9–19% sooner than our optimized NPU-only runtime, at a success rate 3–7 points lower because of its longer delay.

The closed loop uses the measured timing as the delay. No chunk is late in the sustained runs of part (1/2), so each runtime applies its chunks at a constant delay, Δ=5\Delta=5 control steps for NPU-only and Δ=10\Delta=10 for VLA-Duo, and one control step lasts Tsus/5T_{sus}/5. We ran LIBERO with these delays on the four suites (three seeds, 1,200 episodes per row). Both runtimes use the same deployment precision, controller and prefix-inpainting rule.

ModelRuntimeΔ\DeltaSuccess (%)Steps to successTime to success, 8 Elite (s)Time to success, 8 Elite Gen 5 (s)
π0.5\pi_{0.5}NPU-only595.8144.325.923.9
π0.5\pi_{0.5}VLA-Duo1092.8145.423.019.4
π0\pi_0NPU-only589.4144.622.921.1
π0\pi_0VLA-Duo1082.5149.320.918.0

Steps to success average the episodes solved by both runtimes. The revised Section 6.2 will include this table.

We also connected the phone to the simulator, so that the phone computes every action chunk of π0.5\pi_{0.5} (50 episodes, 1,675 on-device inferences). The chunks match the simulated policy with a mean cosine similarity of 0.9998.

(2) The π0\pi_0 result in Table 5 It comes mainly from one layer. In the π0\pi_0 LIBERO checkpoint the action output projection amplifies the 4-bit quantization error: quantizing this layer alone changes the action output by 8.4%, against 0.8% for π0.5\pi_{0.5}. We therefore keep the action projection layers at 16 bits in the revised configuration. They hold less than 0.1% of the parameters and operations, so the effect on TT is negligible. With this change:

MethodΔ\DeltaW4A16 (Table 5)Revised configurationChange
Reference087.288.0+0.8
Naive1077.281.2+4.0
RTC1068.582.4+13.9
VLA-Duo1076.582.5+6.0

In the Table 5 configuration the error of this layer enters every method, so the 0.7-point gap to naive (76.5% against 77.2%) is not meaningful there. In the revised configuration VLA-Duo is above naive.

(3) Uncertainty estimates We repeated the Δ=10\Delta=10 comparison in the revised configuration with three seeds (1,200 episodes per entry, mean ± standard deviation over seeds):

ModelNaiveRTCVLA-Duo
π0.5\pi_{0.5}91.3 ± 0.493.0 ± 1.192.8 ± 0.9
π0\pi_081.2 ± 2.682.4 ± 1.482.5 ± 0.6

VLA-Duo is comparable to RTC, but needs no extra computation. We will update Table 5 and Appendix B.2 accordingly.

We are glad to provide further detail on any of these points. Thanks for your review!

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your review!

As the discussion period comes to an end, we'd like to check if you have any feedback on our response and the revised paper.

We will be glad to answer any questions or additional concerns.

Best, The Authors

Reviewer_rfmzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VLA-Duo, a mobile vision-language-action inference system that overlaps successive inferences across a GPU and an NPU. Under a fixed task configuration, the system exploits the repeated computation graph and tensor shapes to construct an offline execution plan. Each inference is divided into two contiguous segments, allowing the prefix of a newer inference to execute concurrently with the suffix of the preceding inference. A pairwise interference model predicts concurrent execution times and selects the cut point, accelerator order, and execution period. The system additionally incorporates shape-specific operator and memory optimizations.

The evaluation covers π₀ and π₀.₅ on two Snapdragon generations. Short-run measurements show output periods of approximately 394–499 ms, corresponding to throughput improvements of approximately 36%–78% over ExecuTorch/QNN and 18%–25% over the authors’ optimized NPU-only implementation. Separate LIBERO experiments evaluate training-free prefix inpainting under an injected observation-to-action delay. The central trade-off is higher action-chunk throughput at the cost of increased observation-to-action

Critical strengths

The single-cut periodic schedule is coherent under the stated execution assumptions, and the hardware measurements support its short-run throughput benefits. However, the realization of the latency bound under runtime variability is insufficiently evaluated. The experiments also do not directly establish the closed-loop benefit of the proposed throughput–latency trade-off under the actual timing characteristics of the competing runtimes.

The deployment-precision degradation observed for π₀ and the limited specification of prefix inpainting introduce additional concerns about the applicability and reproducibility of the closed-loop results.

Critical weaknesses

The paper provides useful empirical insights into mobile GPU–NPU execution of VLA models. In particular, it demonstrates that overlapping successive inferences can improve action-chunk throughput beyond single-accelerator optimization, and that concurrent execution profiles improve period prediction and cut selection.

The contribution is primarily a practical systems design and implementation, supported by measurements across two models and two mobile platforms.

Questions for authors
  1. Sustained execution and timing guarantees. Can the authors report output-period and observation-to-action latency distributions during sustained execution under a specified cooling and power configuration, including period-overrun rates? How does the runtime respond when either segment exceeds the planned period, and is the deployed period obtained directly from the performance model or calibrated on the device?

  2. Closed-loop evaluation using measured hardware timing. Can VLA-Duo and the optimized NPU-only runtime be compared using their respective measured output periods, observation-to-action latencies, and timing variability, while holding deployment precision and chunk-transition strategy constant? Replaying measured hardware timing traces in LIBERO would help determine whether the proposed throughput–latency trade-off improves task success or task-completion time.

  3. Deployment-precision results for π₀. What explains the 11.5-percentage-point reduction for π₀ between Tables 1 and 5, and the absence of an improvement over naive asynchronous execution at deployment precision? Please clarify the quantization and calibration procedures, whether the evaluation reproduces the mobile runtime’s numerical implementation, and whether repeated evaluations or confidence intervals support the observed differences.

  4. Definition and provenance of prefix inpainting. Please provide the update rule or pseudocode for prefix inpainting, including how the prefix length is selected, how committed actions are constructed at each flow-matching step, and when these actions become available during overlapping inferences. RTC already discusses a gradient-free inpainting baseline that overwrites part of the action chunk at each denoising step in Appendix A.4. Is the implementation here an adoption of this type of method or a modification of it? Clarifying this distinction would improve reproducibility and establish the contribution boundary.

  5. Profiling cost and the practical benefit of interference modeling. Please report the number of unique atom types and profiled atom pairs, total profiling time, schedule-search time, and the time quantum (\tau). From Table 6, the measured throughput improvement over solo-profile scheduling is approximately 0.8%–3.3%, calculated as (T_{\mathrm{solo}}/T_{\mathrm{Duo}}-1). The larger improvement is in prediction accuracy. Does this accuracy improvement also reduce period violations or the safety margin required by the runtime?

4 comments
AuthorsOpenReview ↗

Response to Reviewer rfmz (1/3): W1,Q1,Q2

We appreciate your careful review. We ran new experiments for your questions, and will update the paper to cover all of them below.

This part (1/3) answers W1, Q1 and Q2. Part (2/3) answers W2 and Q3, and part (3/3) answers Q4 and Q5.

Question 1 (Sustained execution and timing guarantees)

Sustained experiment at a fixed period: [Figure.T_sus].

(1) Period, latency and overruns in sustained execution We measured the sustained output period TsusT_{sus} of VLA-Duo and our optimized NPU-only runtime. We find TsusT_{sus} by binary search over [Test,4Test)[T_{est}, 4T_{est}), where TestT_{est} is the estimated best output period. A period counts as sustainable if the phone outputs action chunks at that period steadily for 5 minutes, and TsusT_{sus} is the shortest such period. The phones are plugged into a charger and cooled by a 40W air cooler. The traces are shown in [Figure.T_sus].

SoCModelRuntimeTsusT_{sus} (ms)LL, mean / max (ms)Release offset (ms)Overruns / periods
8 Eliteπ0\pi_0NPU-only792669 / 7187920 / 417
8 Eliteπ0\pi_0VLA-Duo700974 / 100214000 / 470
8 Eliteπ0.5\pi_{0.5}NPU-only896714 / 7628960 / 368
8 Eliteπ0.5\pi_{0.5}VLA-Duo7921083 / 112015840 / 415
8 Elite Gen 5π0\pi_0NPU-only729614 / 6887290 / 417
8 Elite Gen 5π0\pi_0VLA-Duo603823 / 92912060 / 536
8 Elite Gen 5π0.5\pi_{0.5}NPU-only828717 / 7848280 / 398
8 Elite Gen 5π0.5\pi_{0.5}VLA-Duo668982 / 103913360 / 489

Chunks are released at a fixed offset from their observation, so with no overrun the output period stays at TsusT_{sus}.

(2) Response to an overrun When a segment finishes after the period boundary, the runtime holds the next admission until the accelerator is free and keeps at most two inferences in flight (Section 3.3), so the following chunk is delayed by the overrun. The controller continues along the committed chunk in the meantime, since each chunk carries 50 actions and five are consumed per period.

(3) Deployed period It is calibrated on the device: the profiling model supplies the cut and TestT_{est}, and the binary search above finds TsusT_{sus} on the phone.

Question 2 (Closed-loop evaluation using measured hardware timing)

VLA-Duo completes LIBERO tasks 9–19% sooner than our optimized NPU-only runtime, at a success rate 3–7 points lower because of its longer delay.

We ran this comparison on the four LIBERO suites (three seeds, 1,200 episodes per row). Both runtimes use the same deployment precision (the revised configuration of Q3), the same controller, and the same prefix-inpainting rule; only the timing differs. No chunk is late in the sustained runs of Q1, so each runtime applies its chunks at a constant delay, Δ=5\Delta=5 control steps for NPU-only and Δ=10\Delta=10 for VLA-Duo, and one control step lasts Tsus/5T_{sus}/5.

ModelRuntimeΔ\DeltaSuccess (%)Steps to successTime to success, 8 Elite (s)Time to success, 8 Elite Gen 5 (s)
π0.5\pi_{0.5}NPU-only595.8144.325.923.9
π0.5\pi_{0.5}VLA-Duo1092.8145.423.019.4
π0\pi_0NPU-only589.4144.622.921.1
π0\pi_0VLA-Duo1082.5149.320.918.0

Steps to success average the episodes solved by both runtimes. The revised Section 6.2 will include this table.

AuthorsOpenReview ↗

Response to Reviewer rfmz (2/3): W2, Q3

Question 3 (Deployment-precision results for π0\pi_0)

(1) The 11.5-point reduction between Tables 1 and 5 It comes mainly from the quantization error of one layer, the action output projection. Relative error of the action output against the f16 model:

Configurationπ0\pi_0π0.5\pi_{0.5}
action_output_projection at 4 bits, other layers f168.4%0.8%
W4A16 (Table 5 configuration)8.9%4.7%
W4A16 with the action projection layers at 16 bits5.0%4.7%

The π0\pi_0 LIBERO checkpoint predicts actions relative to the robot state, which are about 4× larger, so the error of its output layer is amplified. The action projection layers hold less than 0.1% of the parameters and operations, so keeping them at 16 bits has a negligible effect on TT. With this change:

MethodΔ\DeltaW4A16 (Table 5)Revised configurationChange
Reference087.288.0+0.8
Naive1077.279.5+2.3
RTC1068.583.5+15.0
VLA-Duo1076.582.5+6.0

The remaining 5.5 points to Table 1 are in line with the 4.5 points that 4-bit weights cost the synchronous reference.

(2) No improvement over naive at deployment precision In the Table 5 configuration the error of this layer enters every method, so the 0.7-point gap to naive (76.5% against 77.2%) is not meaningful there. In the revised configuration VLA-Duo is above naive (table above and (4)).

(3) Quantization, calibration, and the mobile runtime Weights use the Q4_0 format of llama.cpp: blocks of 32 weights, 4 bits per weight and one 16-bit scale per block, with round-to-nearest. It is data-free, so there is no calibration set. Activations, SigLIP (vision) and the action projection layers are f16. To check that the evaluation reproduces the mobile runtime, we let the phone compute every action chunk of π0.5\pi_{0.5} in a closed LIBERO loop (50 episodes, 1,675 on-device inferences): the chunks match the simulated policy with a mean cosine similarity of 0.9998.

(4) Repeated evaluations We repeated the Δ=10\Delta=10 comparison in the revised configuration with three seeds (1,200 episodes per entry; mean ± standard deviation over seeds):

ModelNaiveRTCVLA-Duo
π0.5\pi_{0.5}91.3 ± 0.493.0 ± 1.192.8 ± 0.9
π0\pi_081.2 ± 2.682.4 ± 1.482.5 ± 0.6

We will update Table 5 and Appendix B.2 accordingly.

AuthorsOpenReview ↗

Response to Reviewer rfmz (3/3): Q4 & Q5

Question 4 (Definition and provenance of prefix inpainting)

(1) Update rule For VLA-Duo (prefix length 10, see (2)), inference kk samples its chunk as follows:

# x[1..50]   : the action chunk, initialized to noise eps
# ref[1..10] : committed actions (actions 11..15 of inferences k-2 and k-1)
# ref[11..20]: actions 16..25 of inference k-1, i.e. its plan for the same time steps
# r[j]       : release level of action j
#              r = 0 for j = 1..5, rises linearly to 0.33 over j = 6..10,
#              and from 0.4 to 1.0 over j = 11..20

for t in 1.0, 0.9, ..., 0.1:              # 10 flow-matching steps
    x = x - 0.1 * action_expert(x, t)     # standard step; the noise level is now s
    s = t - 0.1
    for j in 1..20:
        if t > r[j]:                      # action j is not released yet
            x[j] = s * eps[j] + (1 - s) * ref[j]   # reset to the reference at noise level s

execute x[11..15]

Actions 1–5 are reset after every step and end exactly at the committed actions. Actions 6–20 are reset only in the early steps.

(2) Prefix length It is determined by the pipeline. A chunk is released two periods after its observation and 5 actions are executed per period, so the prefix length is 10. Similarly, the prefix length for NPU-only is 5.

(3) Committed actions at each step The references are fixed before sampling and are the same at every step: ref[1..10] are the actions the robot executes while inference kk runs, and ref[11..20] is the plan of inference k−1k-1 for the following steps. At each step they are written at the current noise level, as in (1).

(4) Availability during overlapping inferences Inference kk needs the output of inference k−1k-1. Inferences start one period apart, so inference k−1k-1 completes about one period before inference kk does. The references are used only by the flow-matching stage (the action expert), the last stage of an inference, which takes about 110 ms (less than 20% of the inference time). When this stage starts, the output of inference k−1k-1 has therefore been ready for most of a period. As a safeguard, the rule can fall back to the plan of inference k−2k-2 for the same time steps; it has not been needed in our runs.

(5) Relation to RTC Appendix A.4 Our rule modifies the baseline in RTC Appendix A.4 in two ways, both shown in (1): the reference is written at the current noise level, and each action has its own release level. RTC's guidance needs a backward pass through the action expert at every flow-matching step; our rule only overwrites action values, so it adds no computation. At the 2T2T delay it reaches a success rate comparable to RTC (Q3 (4)). The revised Section 6.2 will include this listing.

Question 5 (Profiling cost and the benefit of interference modeling)

(1) Profiling cost

ItemValue
Atom types20
Atoms per inference1,352
Atom pairs800 (20×2020 \times 20 types, both directions)
Profiling timeabout 145 minutes (8 Elite Gen 5)
Schedule-search time0.3 s
Time quantum τ\tau1 ms

Several short atoms can complete within one quantum, so τ\tau only sets the resolution of the period; with τ=0.1\tau = 0.1 ms the predicted period changes by at most 1.4 ms. Section 3.2 will state this.

(2) Effect on the safety margin Yes, for short runs. We replayed the plans of Table 6 at fixed periods on 8 Elite (20 inferences from a cool start, two runs per setting) and searched for the shortest period without a late chunk:

ModelPlan fromPredicted period (ms)Period with no late chunk (ms)Margin
π0\pi_0Solo profiles42747812%
π0\pi_0VLA-Duo4474746%
π0.5\pi_{0.5}Solo profiles47052612%
π0.5\pi_{0.5}VLA-Duo4945246%

Both plans need about 5% above their measured period in Table 6; the rest of the margin is the prediction error, which co-execution profiles reduce from about 6% to 0.9% (Table 6).

For sustained operation the period also depends on heat, so TsusT_{sus} is measured on the device (Q1). The model remains a reliable reference for the cut: it ranks all 2,704 candidate plans in 0.3 s, so only a few cuts near its choice need to be tested on the phone.

We are glad to provide further detail on any of these points. Thanks for your review!

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your review!

As the discussion period comes to an end, we'd like to check if you have any feedback on our response and the revised paper.

We will be glad to answer any questions or additional concerns.

Best, The Authors

Reviewer_zeUaOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VLA-Duo proposes to take advantage of platforms that have an NPU and a GPU in order to compute VLA faster. To solve the problem, the authors developed a scheduler and divided the VLA into functional parts that they called atoms. With different experiments, they showed the efficiency of the method compared to the NPU and GPU alone, but also against different inference platforms that work on the edge.

Critical strengths

The concept is interesting, and the action rate is better with the proposed technique. I am surprised that not much time is lost due to synchronization and data transfers between the NPU and GPU. This may be explained by their shared memory architecture and efficient pipelining.

Critical weaknesses

I think the most important limitation is to not apply this to a real robot, even a simple arm would be enough. The second phase on simulation is not enough to prove it will work on a real system. The atomization is not clearly defined, it don't help to reproduce this work. For instance we don't know if the atoms are manually defined.

I am not sure that the term "scheduler" is entirely appropriate here, since the proposed method only considers a single GPU–NPU transition. It seems closer to a pipeline partitioning algorithm than to a general scheduling approach.

Questions for authors

The techniques you developed and the scheduler do not seem to be VLA-specific. Can you apply them to other types of models, such as ViTs or LLMs?

You tested the method with the Pi family. Do you think other families of VLAs could benefit from the same technique?

Why not include the CPU as well? Did you obtain negative results when using it?

Why did you restrict the scheduler to a single cut? I understand that multiple cuts would introduce additional synchronization and memory-layout conversion overhead, but did you evaluate whether the improved load balancing could compensate for these costs? The question of one cut versus many is still open.

Can you be more explicit about where the cut occurs in the VLA architecture?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zeUa (1/2): W1–W3

We appreciate your careful review and positive assessment, especially your suggestions on extending the method and your questions on multi-cut. We ran new experiments for your questions, and will update the paper to cover all of them below.

This part (1/2) answers W1–W3. Part (2/2) answers Q1–Q5.

Weakness 1 (Real robot)

We agree that a real robot is the strongest validation, but we do not have one currently. To narrow the gap, we connected the phone to the LIBERO simulator, the phone computes every action chunk with VLA-Duo, and the simulated arm executes it.

We ran this closed loop with π0.5\pi_{0.5} on both phones: 50 episodes over the four LIBERO suites, 1,675 on-device inferences. The chunks computed on the phone match the simulated deployment-precision policy with a mean cosine similarity of 0.9998. In a repeated run, the closed loop succeeds in 46/50 episodes, against 47/50 with same policy runs on desktop. Example rollouts: [1] [2] [3] [4].

We will add this experiment and its per-task results to the appendix.

Weakness 2 (Atomization)

Atoms are defined manually, with one template for every Transformer block. The grouping is mainly for removing branches in computation graph and enabling operator fusion. Each block of SigLIP, PaliGemma, and the action expert is split into six atoms (Figure 3):

norm → QKV projection → attention → output projection + residual + norm → FFN first projection + activation → FFN second projection + residual

SigLIP adds one embedding atom and one norm-and-projector atom. In total there are 20 atom types and m=6⋅27+6⋅18+6⋅18⋅10+2=1352m = 6\cdot27 + 6\cdot18 + 6\cdot18\cdot10 + 2 = 1352 atoms per inference. We will add the template and the full atom table to the appendix.

Weakness 3 (Scheduler/Pipelining)

We appreciate your suggestion. We will describe our method as periodic pipeline partitioning in the revision.

AuthorsOpenReview ↗

Response to Reviewer zeUa (2/2): Q1–Q5

This part (2/2) answers Q1–Q5.

Question 1 & 2 (Other models and other VLA families)

Thank you for proposing this, other models and VLA families are a good test of the generality of our method. We tested SmolVLM2-2.2B and OpenVLA-7B on Snapdragon 8 Elite, with the same runtime and no model-specific tuning (NPU-only and VLA-Duo: 3 runs, 20 measured inferences each):

ModelGPU-only (ms)NPU-only (ms)VLA-Duo (ms)Speedup over NPU-only
SmolVLM2-2.2B20247755691.36×
OpenVLA-7B5848134711891.13×

The selected cut differs per model. For SmolVLM2 the NPU runs the vision encoder and the prefill and the GPU generates the tokens, since token generation is as fast on the GPU as on the NPU. For OpenVLA the GPU runs the vision encoders and the NPU runs the language model, which is 88% of the inference, so the gain is smaller. We will add these results to the appendix.

Question 3 (CPU)

The CPU is far slower on this workload: one π0\pi_0 inference takes 41.4 s on the Snapdragon 8 Elite CPU (llama.cpp CPU backend, 4 threads), against 0.56 s on our NPU-only runtime (Table 2). Any atom moved to the CPU would lengthen the period, and the CPU cores already drive both accelerators and share memory bandwidth with them. We therefore keep the CPU for coordination.

Question 4 (Single cut)

We developed a DP algorithm that finds the best schedule with exactly kk-cuts per inference, k∈[1,10)k \in [1, 10). It uses the same profiling data and performance model. The best period on Snapdragon 8 Elite:

Number of cuts kk123456789
π0\pi_0 period, predicted (ms)449.4455.9462.8465.9464.2479.5482.2498.9496.8
π0\pi_0 period, measured (ms)454.1460.6467.2473.6482.3481.7489.4494.0502.4
π0.5\pi_{0.5} period, predicted (ms)494.5504.8509.5519.4512.5525.5538.1556.5552.3
π0.5\pi_{0.5} period, measured (ms)496.3510.0508.9535.2517.7527.1555.9557.5558.6

The measured periods differ from the predictions by −1.0% to +3.9%. Single-cut solution is the fastest among k∈[1,10)k \in [1, 10). As it already balances the workload of the two accelerators, further cuts have little imbalance to improve. Each extra cut adds a handoff (about 2 ms per extra NPU segment and 10ms per extra GPU segment).

We will add this experiment and the DP pseudocode to the appendix.

Question 5 (Cut location)

The cut lies inside the SigLIP vision encoder, in 22 to 24-th of its 27 transformer blocks. The GPU runs the patch embedding and the first SigLIP blocks, and the NPU runs the rest. The GPU gets this short prefix because it is slower than the NPU on these models (Table 2). We will state the location next to Figure 6.

We are glad to provide further detail on any of these points. Thanks for your review!

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your review!

As the discussion period comes to an end, we'd like to check if you have any feedback on our response and the revised paper.

We will be glad to answer any questions or additional concerns.

Best, The Authors

Reviewer_ekKWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents VLA-Duo, a mobile GPU–NPU execution system for chunked VLA policies. It partitions each inference into two contiguous segments and overlaps successive inferences, using offline pairwise performance profiles to select the cut and execution period. Shape-specific kernels and graph optimizations support the runtime. Experiments on two Snapdragon SoCs report shorter action-chunk output periods. The pipeline trades observation-to-action latency for throughput. Separate LIBERO experiments evaluate prefix inpainting under an injected ten-step delay, including tests at deployment precision.

Critical strengths
  1. The runtime improves throughput beyond an optimized single-accelerator implementation. Table 2 evaluates two policies on two SoCs and includes ExecuTorch/QNN alongside the authors' own GPU-only and NPU-only runtimes. The periodic pipeline adds approximately 18–25% throughput over their optimized NPU-only runtime. For pi_0, Appendix B.1 separates graph/memory optimization, kernel optimization, and heterogeneous scheduling, making it possible to distinguish the scheduling gain from the larger implementation gains.

  2. Co-execution profiling is tested against both a simpler predictor and measured cuts. Appendix B.3 compares the same cut search with and without interference modeling. Pairwise profiles reduce period-prediction error from 6.2–12.3% to 0.9–1.7%, and the selected cuts are within 0.5% of the best tested periods. This supports the performance model under the measured conditions.

Critical weaknesses
  1. The control evaluation does not establish the benefit of the throughput–latency trade-off. Section 6.2 deliberately separates policy evaluation from the mobile runtime by imposing the same ten-step delay on naive asynchronous execution, RTC, and VLA-Duo. This is useful for studying delay compensation, but it does not compare periodic execution with the optimized NPU-only runtime under their respective observation-to-action latencies and chunk-arrival times. Appendix B.2 does test deployment precision: at the same ten-step delay, Table 5 reports 93.5% versus 91.8% for VLA-Duo and naive execution on pi_0.5, but 76.5% versus 77.2% on pi_0. These averages do not establish consistent gains from prefix inpainting across both models, nor does the small difference on pi_0 establish significant degradation. They remain tests of delay compensation rather than a comparison of the runtimes' throughput–latency trade-offs.

  2. On-device validation of the latency bound is incomplete. The 2T bound in Sections 3–4 follows when both segments finish within the scheduled period, including the handoff allowance in Section 4.3. Tables 2–3 and 6 report output periods and prediction errors, but not measured observation-to-action latencies or deadline misses. Their accuracy therefore does not establish that the bound holds in the actual runtime. Section 6.4 also defines cut-selection regret under eager admission without explaining its relationship to fixed-period replay. Sections 6.1 and 6.3 explicitly limit timing tests to three short runs on cooled devices, each with twenty measured inferences after warm-up. This supports the reported burst performance, but leaves continuous operation under timing and thermal variation untested.

Questions for authors
  1. Could the authors compare periodic and optimized NPU-only execution on representative LIBERO tasks using the same deployment-precision policy, controller, and delay-compensation rule, but each runtime's measured chunk-arrival and observation-to-action timing? A hardware-connected or trace-driven evaluation could address this without requiring a new real-robot benchmark. Please also explain the different effect of prefix inpainting on pi_0 and pi_0.5 in Table 5.

  2. Could the authors clarify how the eager-admission measurements relate to fixed-period replay, and report measured observation-to-action latencies and any period overruns? A sustained comparison with the optimized NPU-only runtime through thermal steady state would help assess the intended onboard use. Please specify how overruns are handled and under what conditions the 2T bound remains applicable.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ekKW (1/2): W2&Q2

We appreciate your careful review. We ran two experiments to address your two questions, and we will update the paper to cover all the weaknesses and questions below.

We first answer Q2/W2 here, then Q1/W1 in the next comment (2/2).

Question 2 & Weakness 2 (eager-admission, fixed-period replay, overruns are handled)

Sustained experiment under thermal steady state: Figure.T_sus.

(1) Eager admission vs. fixed-period replay

  • We use eager admission for measurement. The next inference starts as soon as the first accelerator becomes free, which shows the maximum improvement VLA-Duo can achieve.
  • We use fixed-period replay for deployment. Actions are executed at a fixed period to keep the control frequency stable.

(2) Overrun handling When a segment finishes after the period boundary, the runtime holds the next admission until the accelerator is free and keeps at most two inferences in flight (Section 3.3), so the following chunk is delayed by the overrun. The controller continues along the committed chunk in the meantime, since each chunk carries 50 actions and five are consumed per period.

(3) When the 2T2T bound applies The bound holds when both segments of the plan finish within the replay period, handoff included, that is, when the replay period is at least the plan's eager period in the current device state. The offline model predicts that period within 0.9 to 1.7% on a cooled device (Table 3), and TsusT_{sus} is the shortest fixed period that holds over a 5-minute run.

(4) Sustained comparison with the optimized NPU-only baseline We measured the sustained output period TsusT_{sus} for both settings. We find TsusT_{sus} by binary search over [Test,4Test)[T_{est}, 4T_{est}), where TestT_{est} is the estimated best output period. A period counts as sustainable if the phone outputs action chunks at that period steadily for 5 minutes, and TsusT_{sus} is the shortest such period. The traces are shown in Figure.T_sus.

SoCModelRuntimeTsusT_{sus} (ms)LL, mean / max (ms)Release offset (ms)Overruns / periods
8 Eliteπ0\pi_0NPU-only792669 / 7187920 / 417
8 Eliteπ0\pi_0VLA-Duo700974 / 100214000 / 470
8 Eliteπ0.5\pi_{0.5}NPU-only896714 / 7628960 / 368
8 Eliteπ0.5\pi_{0.5}VLA-Duo7921083 / 112015840 / 415
8 Elite Gen 5π0\pi_0NPU-only729614 / 6887290 / 417
8 Elite Gen 5π0\pi_0VLA-Duo603823 / 92912060 / 536
8 Elite Gen 5π0.5\pi_{0.5}NPU-only828717 / 7848280 / 398
8 Elite Gen 5π0.5\pi_{0.5}VLA-Duo668982 / 103913360 / 489

The mobile phone is tested with a 40W air cooler.

(5) Scope of the new measurements The sustained runs cover five minutes on Snapdragon 8 Elite/8 Elite Gen 5 with active cooling. The revision adds a sustained-performance subsection with this table, the traces, the measured LL under fixed-period replay, and the overrun policy.

AuthorsOpenReview ↗

Response to Reviewer ekKW (2/2): W1&Q1

Weakness 1 & Question 1

(1) Periodic versus optimized NPU-only execution, each with its own timing

VLA-Duo completes LIBERO tasks 9–19% sooner than our optimized NPU-only runtime, at a success rate 3–7pts lower because of its longer delay.

We ran this comparison on the four LIBERO suites (three seeds, 1,200 episodes per row). Both runtimes use the same deployment-precision policy (see sub-answer (2)), the same controller, and the same prefix-inpainting rule; only the timing differs. The timing comes from the TsusT_{sus} in W2/Q2, one control step lasts Tsus/5T_{sus}/5.

ModelRuntimeΔ\DeltaSuccess (%)Steps to successTime to success, 8 Elite (s)Time to success, 8 Elite Gen 5 (s)
π0.5\pi_{0.5}NPU-only595.8144.325.923.9
π0.5\pi_{0.5}VLA-Duo1092.8145.423.019.4
π0\pi_0NPU-only589.4144.622.921.1
π0\pi_0VLA-Duo1082.5149.320.918.0

Steps to success average the episodes solved by both runtimes. The revised Section 6.2 will include this table.

(2) Effect of prefix inpainting on π0\pi_0 and π0.5\pi_{0.5} in Table 5

The π0\pi_0 rows of Table 5 are dominated by the quantization error of one layer, action output projection. With the action projection layers kept at 16 bits, VLA-Duo is no longer below naive asynchronous execution on π0\pi_0.

Relative error of the action output against the f16 model:

Configurationπ0\pi_0π0.5\pi_{0.5}
action_output_projection at 4 bits, other layers f168.4%0.8%
W4A16 (Table 5 configuration)8.9%4.7%
W4A16 with the action projection layers at 16 bits5.0%4.7%

The π0\pi_0 LIBERO checkpoint predicts actions relative to the robot state, which are about 4× larger, so the error of its output layer is amplified. The action projection layers hold less than 0.1% of the parameters and operations, so the effect on TT is negligible.

LIBERO success rates (%) for π0\pi_0 in the revised configuration (three seeds, 1,200 episodes per entry):

MethodΔ\DeltaW4A16 (Table 5)Revised configurationChange
Reference087.288.0+0.8
Naive1077.281.2+4.0
RTC1068.582.4+13.9
VLA-Duo1076.582.5+6.0

We will update Table 5, Appendix B.2 accordingly.

We are glad to provide further detail on any of these points. Thanks for your review!


Correction update: The naive and RTC data in (2) were first posted from a single seed. We forgot to update in the text. They are now the three-seed means (naive 79.5 → 81.2, RTC 83.5 → 82.4). The values change by 1–2 points, and the conclusion is unchanged.

AuthorsOpenReview ↗

Follow-up on Our Response

Dear Reviewer,

Thank you again for your review!

As the discussion period comes to an end, we'd like to check if you have any feedback on our response and the revised paper.

We will be glad to answer any questions or additional concerns.

Best, The Authors

Comments on the paper

3 comments
AuthorsOpenReview ↗

Paper Revision Notes

Paper Revision Notes

We thank all reviewers for their comments. We uploaded a revised PDF, with changes marked in blue.

Summary

  • Schedule -> Pipeline: the method is described as a periodic pipeline (except title)
  • T_sus Experiment: Section 6.3 adds the sustained period experiment, to evaluate long-term execution improvement.
  • Success rate: Section 6 now reports deployment precision (Q4) instead of 16-bit.
  • Mobile control validation: in Appendix C, the phone computes every action chunk that controls the robot in the LIBERO environment.
  • Implementation details: the appendix adds pseudocode, profiling details and extensive experiments.

Detailed Content

Reviewer ekKW

  • W1 / Q1 (periodic vs. NPU-only in closed loop): Table 1 adds NPU-only at Δ=5\Delta=5 next to VLA-Duo at Δ=10\Delta=10, both at deployment precision.
  • Q1 (inpainting on π0\pi_0 and π0.5\pi_{0.5}): Appendix B.2 explains the π0\pi_0 drop, which comes from the action output projection. Table 5 adds the deployment precision.
  • W2 / Q2 (eager admission, fixed-period replay, overruns): Section 3.3 states the overrun policy. Section 6.3 reports TsusT_{sus}, latency and power. Appendix H lists each sustained run and explains the two admission modes.
  • Differences from our reply: the reference in our reply (88.0) was single-seed, and the revision uses the three-seed mean 87.7.

Reviewer zeUa

  • W1 (real robot): Appendix C adds a phone-in-the-loop evaluation on Snapdragon 8 Elite.
  • W2 (atom definition): Appendix E gives the atom template and the atom counts.
  • W3 ("scheduler"): the method is described as a periodic pipeline. (except title)
  • Q1 / Q2 (other models and VLA families): Appendix I reports SmolVLM2 and OpenVLA.
  • Q3 (CPU): answered in our reply, but didn't add to the paper.
  • Q4 (single cut): Appendix D gives the DP algorithm and experiments for exactly kk cuts, k∈[1,10)k\in [1,10)
  • Q5 (cut location): Section 6.4 gives the location of the selected cuts, next to Figure 6.

Reviewer rfmz

  • Q1 (sustained execution and timing): Section 3.3 states the overrun policy. Section 6.3 reports TsusT_{sus}, latency and power. Appendix H lists each sustained run.
  • Q2 (closed loop with measured timing): Table 1 adds NPU-only at Δ=5\Delta=5 next to VLA-Duo at Δ=10\Delta=10, both at deployment precision.
  • Q3 (π0\pi_0 at deployment precision): Appendix B.2 explains the π0\pi_0 drop, which comes from the action output projection. Table 5 adds the deployment precision with three-seed means.
  • Q4 (prefix inpainting): Appendix G gives the update rule and its relation to RTC Appendix A.4.
  • Q5 (profiling cost): Appendix F gives the profiling method, its cost and the time quantum τ\tau.
  • Differences from our reply: the reference in our reply (88.0) was single-seed, and the revision uses the three-seed mean 87.7. Our reply pointed to Section 3.2 for τ\tau, which is now in Appendix F.

Reviewer BGAT

  • W1 / Q1 (sustained measurements): Section 6.3 reports TsusT_{sus}, core temperature and power. Appendix H lists each sustained run, with no overrun.
  • W2 / Q2 (measured timing in the closed loop): Table 1 adds NPU-only at Δ=5\Delta=5 next to VLA-Duo at Δ=10\Delta=10, both at deployment precision.
  • Q2 (π0\pi_0 in Table 5, uncertainty): Appendix B.2 explains the π0\pi_0 drop, which comes from the action output projection. Table 5 adds the deployment precision with three-seed means.
  • Differences from our reply: the reference in our reply (88.0) was single-seed, and the revision uses the three-seed mean 87.7.
AuthorsOpenReview ↗

Code Release

Dear Reviewers and Area Chairs,

We release the code of VLA-Duo in an anonymous repository:

https://github.com/iclr27submission45957/VLA-Duo

The repository helps reproduce the experiments, including the burst and sustained runs, profiling, the cut-selection study and the k-cut runs. You can follow the "Quick start" section in the README.

Thank you for your time.

Best regards, The authors.

Dear Reviewers,

The authors have submitted their rebuttal. Please review it and share your comments. If anything is unclear or requires clarification, please let the authors know as soon as possible. The author–reviewer discussion period ends on 18 November.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.