acceptodds
Under review as a conference paper at ICLR 2027

VLA-ULAP: Interleaving Cloud VLA Calls with Ultra-Lightweight Local Action Prediction at the Edge

Abstract

Billion-parameter vision–language–action (VLA) policies run either onboard, consuming substantial power, or on remote servers, adding communication latency. To address these drawbacks and better balance latency and onboard energy consumption, we propose VLA-ULAP. It partitions inference across decision times, interleaving remote VLA calls with predictions from an Ultra-Lightweight Local Action Predictor (ULAP). A single ULAP has 7.4M parameters including the frozen vision encoder. It combines current views, proprioception, and executed action history to predict chunks in one pass. Trained independently, it requires no VLA hidden states, online verification, or server round trips. On Jetson Orin Nano, ULAP takes 20.7 ms and 0.122 J of idle-subtracted energy per inference, versus 289.3 ms and 40.46 J for GR00T on RTX A6000. Across four simulated base-policy/benchmark pairs, VLA-ULAP removes 45.3–77.3% of VLA calls while retaining 95.0–98.5% of baseline success rates at selected operating points. On VLA-JEPA, VLA-ULAP also surpasses local acceleration alternatives, using an estimated 49.4% less inference time and 51.5% less GPU energy per successful episode than ACT, and 77.5% less time and 80.7% less energy than SP-VLA at higher success rates. In physical SO-101 trials, it similarly removes 70.3–71.9% of VLA calls without observed success-rate loss at seen or held-out placements. Measured device costs imply 64.6–66.4% less inference time and 70.1–71.7% less idle-subtracted energy per successful episode at these call counts. Beyond these savings, faster responses help VLA-ULAP exceed 's success rate by 11.0 and 15.5 percentage points (pp) on two tasks in latency-aware LIBERO-Safety simulation while approximately halving VLA calls.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot learning, cloud-edge inference, edge AI, efficient inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_HctgOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Vision–language–action (VLA) policies with billions of parameters are too costly to run on a mobile robot's own hardware and add communication and inference delay when queried from a remote server. The authors propose VLA-ULAP, which leaves the VLA unchanged on the server and lets a small on-device predictor, trained by behavior cloning on task demonstrations or on successful rollouts of the VLA, take a fixed fraction of the action-chunk decisions from fresh camera observations and proprioception. The claimed contributions are the split-inference framework itself, a reduction in server queries of roughly half to three quarters at small or no loss in task success across simulated benchmarks and two physical tasks, improved success over the base policy when VLA inference is delayed, and an estimated reduction in per-episode inference time and energy derived from separately profiled per-call costs.

Critical strengths
  1. The question is relevant to deployment of large robot policies: whether a small onboard model that reads fresh observations can take over a fraction of the decisions of a large remote policy. The method requires no access to VLA internals, and the authors apply it to six base policies with different action heads (GR00T N1.7, VLA-JEPA, Cosmos-Policy, X-VLA, π0.5\pi_{0.5}, and SmolVLA in Appendix E).

  2. The simulation protocol is careful. Training and evaluation initial states are disjoint (with an explicit audit for VLA-JEPA and LIBERO-Safety), call reduction in simulation is counted over all episodes including failures, full sweeps over ρ\rho are shown rather than single operating points, and per-suite, per-task, and per-depth results are reported (Figures 12–15). The authors also report a negative result (adaptive scheduling gives no consistent gain, Appendix D) and design ablations on the state branch, action history, backbone scale, and token pooling (Appendix C).

  3. The comparisons in Figure 7 and Appendix E address the main alternatives for reducing VLA calls: a different local model under the same schedule and corpus (ACT), action extrapolation without fresh observations (SP-VLA), standalone retrieval (RT-Cache), and longer open-loop execution of each VLA chunk (SmolVLA). ULAP gives the best observed success–cost frontier against all four.

Critical weaknesses
  1. The latency and onboard-energy trade-off that motivates the paper is not measured. The abstract and Figure 1 frame the problem as remote inference adding communication delay and onboard inference draining the battery, but apart from separate per-call profiles of each policy on an RTX A6000 and of ULAP on a Jetson Orin Nano, the experiments measure neither communication delay and energy, nor the robot's energy per episode, nor the cost of onboard VLA inference. Execution in Sections 4.1–4.3 is synchronous. Section 4.5 is presented as the one experiment in which the simulator keeps stepping during a modeled VLA delay, which implies that elsewhere it does not, and physical control is synchronous at a target 30 Hz while GR00T's profiled per-call latency is 289 ms (Appendix B.5). The success curves in Figures 3–9 are therefore independent of latency, and only the number of VLA calls is measured. The time and energy per successful episode in Figures 1 and 10 are computed afterward as call counts multiplied by separately profiled per-call costs, and the authors state that camera acquisition, communication, and queueing are outside the estimate (Appendix B.5). Communication is the cost the paper attributes to remote inference in its first paragraph. The energy metric also sums idle-subtracted A6000 board energy with Jetson module energy. Of the 209.2 J per successful Ping-pong episode reported for VLA-ULAP, 207.6 J (5.13 calls × 40.46 J) is server GPU energy and about 1.6 J is onboard compute. The reported 70–72% energy reduction is thus a reduction in server GPU energy proportional to the call count. This is a valid measure of server load, but it is not the onboard energy named in the paper's central question. On the robot, remote-only inference spends radio energy on each call (excluded) and no compute energy, while VLA-ULAP adds about 1.6 J of compute per episode and spends radio energy on fewer calls. The paper's own reading of Huang et al. (2025) puts Wi-Fi image transfer at hundreds of millijoules per frame, the same order as or larger than the 0.122 J per ULAP inference, so the onboard comparison depends on the excluded term. In addition, the physical rollouts ran ULAP on an Apple GPU (Mac MPS) rather than the Jetson (Section 4.4), and the onboard-VLA alternative in Figure 1(b) is not evaluated. The paper argues against onboard inference with the naive π0.5\pi_{0.5} latency of 457.9 ms on Jetson Thor quoted from Yang et al. (2026), a work whose stated contribution is onboard real-time control through asynchronous inference, while a vendor tutorial page reports 54 ms per π0.5\pi_{0.5} inference on the same module with TensorRT FP8, and 49 ms with FP8 and NVFP4 [e], under two control periods at 30 Hz. To support the claims, the authors would need to measure onboard energy (Jetson module power plus radio power) and wall-clock episode duration over a real wireless link for remote-only VLA, VLA-ULAP, and an onboard VLA baseline.

  2. The latency-aware experiment (Section 4.5) is the only test of the "more responsive control" benefit, and its design limits what it can show. First, the two tasks were "selected by an earlier latency sweep over the suite's five tasks" (Appendix B.6), so the tasks were chosen on the effect being tested; results on all five obstacle-avoidance tasks are needed. Second, the three-step delay corresponds to π0.5\pi_{0.5} inference on a local A6000 (147.5 ms) with no network, whereas a remote deployment adds round-trip time and jitter; a sweep over delay would show how the gain behaves at the delays that motivate the paper. Third, the π0.5\pi_{0.5} baseline uses the naive asynchronous strategy (continue the previous chunk during the delay, then skip the elapsed prefix of the new chunk). The paper cites recent asynchronous and cloud–edge VLA preprints (Yang et al., 2026; Peng et al., 2026), but it neither cites nor compares with established methods for executing action chunks under inference delay [a] or for combining delayed remote predictions with fresh on-device observations [c]. Fourth, the protocol in Appendix B.6 issues each VLA request before the current interval ends. A five-action ULAP chunk has no actions beyond its fifth step, so avoiding a three-step stall requires issuing the next VLA request three steps before the ULAP chunk ends, which conditions the VLA on an observation three steps old and requires the schedule to be known three steps in advance. The fixed schedule provides this, but a boundary-time scheduler such as the one in Appendix D, which was evaluated only at zero delay, could not issue an unscheduled VLA call under delay without a three-step stall; the compatibility of the framework with adaptive scheduling under delay is therefore untested. Two controls are also missing: π0.5\pi_{0.5} with zero delay, which bounds how much success the delay costs, and ULAP under the same protocol both after one initial VLA call (ρ=1\rho = 1) and with no VLA call at all, the standalone protocol applied to RT-Cache in Section 4.2. ULAP for this experiment is trained on the 70% training splits (about 258 and 293 episodes) of 368 and 418 successful rollouts of the same two tasks, so without the local-only results it is not possible to attribute the gain to responsiveness rather than to a task-specific student.

  3. The paper presents its first contribution as a split inference framework and contrasts it with model-internal partitioning (HiRT) and recent VLA acceleration methods, but overlooks established work on partitioning inference between devices and servers and on remote inference delay. This includes offloading and partitioning decisions between mobile devices and the cloud [b], split computing and early exiting [d], real-time execution of action chunks under inference delay [a], and on-device correction of delayed remote predictions [c]. The binary per-chunk choice discards the remote model's output during local decisions, except through the executed-action history, and that history no longer contains VLA actions after the first local chunk in a run of consecutive local decisions. At least four of the base policies compute actions beyond the executed prefix (GR00T computes 40 and returns 16, Cosmos-Policy generates 32 and executes 16, π0.5\pi_{0.5} generates 10 and executes 5, SmolVLA generates 50 and executes 20), and those actions are discarded. Conditioning ULAP on the most recent VLA chunk, including its unexecuted tail, would add no communication and is the natural alternative to binary switching [c]; the authors should compare against it or explain why it is excluded.

  4. The retention metric understates the cost of bypass near saturation, and the evaluated distributions leave the VLA's contribution unclear. On LIBERO with GR00T, 88.50% success at 95.0% retention means the failure rate rises from 6.85% to 11.50%, a factor of 1.7; with VLA-JEPA, 95.35% at 96.5% retention means failures rise from about 1.2% to 4.65%, a factor of about 3.9. Failure rates, or success with confidence intervals, should accompany retention. At ρ=1\rho = 1, where the only VLA call is the first chunk, ULAP reaches 78.2% on LIBERO with GR00T and about 89% with VLA-JEPA, and roughly 92–94% on the Spatial, Object, and Goal suites (Figure 13). On these suites, a predictor with 1.9M trainable parameters, trained on successful rollouts collected from 100 attempts per task under the same initial-state distribution, solves most episodes without the VLA, so the call reduction there mainly reflects how closely training matches evaluation. The settings that require the VLA show smaller reductions: RoboCasa with held-out objects falls to about 42–45% success at ρ=1\rho=1 (baseline 65.75%), CALVIN falls to about 22% (baseline 69.0%), and both retain about 98% of baseline success at 45–49% call reduction. On the physical robot, ULAP is trained on 80 of the same 100 demonstrations used to fine-tune GR00T. The ensemble at ρ=0.75\rho = 0.75 exceeds GR00T at the training placements on Ping-pong (48/50 versus 41/50), while the single predictor at ρ=0.60\rho=0.60 reaches 40/50 (Table 3). Because failed physical runs were stopped early, the physical call reduction is computed per successful episode (Appendix B.5), over different subsets of trials for the two policies, and the number of trials discarded and recollected after infrastructure failures is not reported. Table 3 stops at ρ=0.75\rho = 0.75 and Table 1 at ρ=0.50\rho = 0.50; no result with one initial VLA call, with no VLA call, or with ACT alone is reported on the robot or in the latency-aware experiment, so success cannot be attributed to the VLA calls. Finally, ULAP is trained on rollouts or demonstrations of the evaluated tasks: the CALVIN and RoboCasa predictors transfer to a new environment and to held-out objects, but no experiment tests tasks or instructions absent from ULAP training, and the paper notes that its physical OOD condition covers "unseen object placements, not unseen object categories or tasks" (Appendix B.5). Generalization to new tasks without task-specific data is the main reason to deploy a VLA, so this limitation should be stated and tested.

  5. The authors withhold the source code, the trained ULAP checkpoints, and the rollout datasets used to train them. The submission has no supplementary material, the manuscript has no anonymous repository link, and the reproducibility statement does not commit to a release. This prevents verification of the delay-injection logic in Appendix B.6, the scheduler, and the cost reconstruction in Appendix B.5. Since the code will not fit into the rebuttal text, the authors should provide it through an anonymous repository (e.g., https://anonymous.4open.science) during the rebuttal period.

References:

[a] Black et al., "Real-Time Execution of Action Chunking Flow Policies," NeurIPS 2025.

[b] Kang et al., "Neurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge," ASPLOS 2017.

[c] Jacobellis et al., "DeDelayed: Deleting Remote Inference Delay via On-Device Correction," CVPR 2026.

[d] Matsubara et al., "Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges," ACM Computing Surveys, 2022.

[e] NVIDIA Jetson AI Lab, "OpenPI on Thor," https://www.jetson-ai-lab.com/tutorials/openpi_on_thor/

Questions for authors
  1. What are the onboard energy per successful episode (Jetson module power plus radio power during observation upload and action download) and the wall-clock episode duration for remote-only GR00T and for VLA-ULAP when ULAP runs on the Jetson and the VLA is reached over a wireless link? In the physical trials, was GR00T reached over a network, and if so, which link and what round-trip time?

  2. What are the LIBERO-Safety results on all five obstacle-avoidance tasks, and how do they change with π0.5\pi_{0.5} at zero delay, ULAP after one initial VLA call (ρ=1\rho = 1), ULAP with no VLA call, a real-time chunk-execution baseline such as [a], and VLA delays longer than three control steps?

  3. In Appendix B.6, when the decision preceding a VLA call is a five-action ULAP chunk, at which step is the VLA request issued, and on which observation is it conditioned? How could a boundary-time adaptive scheduler such as the one in Appendix D issue an unscheduled VLA call under delay without a three-step stall?

  4. What success does ULAP after one initial VLA call (ρ=1\rho = 1) and with no VLA call achieve on the two SO-101 tasks, and what does ACT trained on the same 80 demonstrations achieve?

  5. Have the authors tried conditioning ULAP on the most recent VLA chunk, including the actions the VLA computed beyond the executed prefix? If so, how does it compare with binary switching?

  6. How does VLA-ULAP perform on tasks or instructions for which no ULAP training data was collected, for example a LIBERO task held out from ULAP training?

  7. Will the authors provide an anonymous repository with the training, evaluation, delay-injection, and cost-reconstruction code during the rebuttal period?

9 comments
AuthorsOpenReview ↗

Response to Reviewer Hctg (1/3)

Thank you for the detailed suggestions. We added the following measurements, controls, and artifacts.

Q1 — Physical network, onboard energy, and episode duration

Thank you for requesting measurements that more closely reflect the intended deployment. In response, we integrated ULAP into the physical control loop on a Jetson Orin Nano and measured latency and onboard energy. Because institutional security restrictions prevent a direct Jetson–server Wi-Fi connection, we used USB Ethernet to a Mac SSH relay with a Wi-Fi uplink. The transport/RPC residuals are 165.8/167.1 ms for Glue-stick and 175.9/178.9 ms for Ping-pong (VLA-only/hybrid VLA requests), averaging episode means over all 75 trials per condition. These residuals are calculated by subtracting measured server processing time from measured client RPC round-trip time. They include network transfer, relay, and RPC overhead outside the server timer, rather than pure network RTT (Appendix B.5; Table 15 gives the Glue-stick breakdown). Averaging equally across Glue-stick and Ping-pong, using the 56 and 49 pairs respectively completed by both methods, mean decision latency decreases from 534.9 to 182.3 ms (65.9%), time to success from 19.61 to 13.81 s (29.6%), and measured Jetson module energy from 130.6 to 94.2 J per episode (27.9%) (§4.3; Appendix B.5).

To address the intended direct-Wi-Fi deployment as fully as possible within this constraint, we also constructed latency and energy estimates from measured components. The reconstruction uses measured compute, losslessly compressed observations, 100 Mbps goodput, a 4.28 ms laptop–server RTT proxy, and transmit/receive/idle radio power. The 4.28 ms value is a laptop–server RTT proxy, not a directly measured Jetson–server Wi-Fi RTT. Under these common deployment assumptions, estimated robot-side energy decreases from 138.7 to 104.4 J per successful Glue-stick episode and from 129.3 to 94.3 J on Ping-pong (revised Figure 9; Appendix B.5). Varying goodput over 50–200 Mbps preserves the estimated onboard-energy advantage on both tasks (Appendix B.5, Table 17). Actions and outcomes are held fixed in these retrospective estimates.

Q2 — Complete Safety suite and additional controls

We now evaluate all five L1 tasks on 200 fresh paired initial states per task. The earlier delay setting accounted for model inference but omitted communication between the robot and server and other deployment overheads. We therefore use fixed VLA/local delays of five/one steps, informed by measured computation and deployment overheads together with a direct-Wi-Fi transfer estimate. All counts below are successes out of 1,000 trials.

ConditionSuccesses
VLA-only, zero delay828
VLA-only, five-step delay750
RTC, five-step VLA delay721
Alternating VLA/ULAP, five/one-step delays825
Same hybrid, five/five-step delays755
One initial VLA call, then ULAP851
ULAP with no VLA call864
ACT-only with temporal ensembling, one-step delay402

Faster local responses improve task success. Holding the predictor, alternating schedule and VLA delay fixed, reducing ULAP delay from five to one step raises success rate by 7.0 percentage points (95% CI [4.3, 9.7]). At equal five-step delays, VLA-ULAP and VLA-only have similar success rates (+0.5 points, 95% CI [−2.4, 3.4]). Together, these controls support a benefit from faster local responses beyond introducing the task-trained predictor alone.

With one-step ULAP responses, VLA-ULAP reaches 825/1,000 successes versus 750/1,000 for delayed VLA-only, approaching zero-delay VLA's 828/1,000 while making 53.0% fewer VLA calls. Thus, it recovers most of the success rate lost to delay while reducing remote inference (§4.4, Figure 10; Appendix B.7).

At the evaluated RTC boundary, the five directly weighted overlap actions lie in the elapsed prefix of the ten-action prediction, and execution starts at action six. Guidance can still affect the executed suffix through the coupled denoiser. This may limit the overlap benefit, but the experiment does not isolate why RTC underperforms here or imply that RTC is generally ineffective.

Local-only ULAP performs strongly in aggregate on these known tasks, but the benefit of recurring VLA intervention varies by task. For example, on Book, the hybrid achieves 179/200 successes versus 152/200 for local-only ULAP.

AuthorsOpenReview ↗

Response to Reviewer Hctg (2/3)

Q3 — Request timing and adaptive scheduling

In the original three-step protocol, we issued the next VLA request after two actions of the five-action ULAP interval, using the observation at that time. In the revised five-step protocol, the request is issued at the start of the ULAP interval, before its first action. The environment continues executing the active chunk; the elapsed prefix of the arriving prediction is discarded. ULAP's one-step delay is handled by requesting it one step before the switch (Appendix B.7).

This uses a fixed schedule. Boundary-time adaptive requests require waiting or fallback if buffered actions run out; Appendix D does not establish stall-free adaptive scheduling under delay.

Q4 — Physical one-call, zero-call, and ACT controls

With recurring VLA calls, VLA-ULAP achieves 67/75 successes on Glue-stick and 62/75 on Ping-pong, compared with 57/75 and 58/75 when the VLA supplies only the first chunk, and 63/75 and 58/75 for VLA-only (§4.3, Table 3). On held-out Ping-pong placements, recurring calls improve success from 13/25 to 20/25, supporting continued VLA intervention for transfer to new placements (Appendix B.5).

We have not evaluated SO-101 control with zero VLA calls. At ρ=1, the VLA supplies only the first 16 actions; across Glue-stick trials this gives 75 VLA calls and 2,070 local decisions. Due to time constraints, we have not completed the same-demonstration SO-101 ACT comparison. Existing ACT controls cover VLA-JEPA/LIBERO (§4.2) and the same Safety training data (§4.4), but do not replace a physical ACT comparison.

Q5 — Conditioning on the latest VLA chunk

Thank you for this suggestion. In response, we tested whether supplying the latest received VLA chunk, including its unexecuted actions, improves ULAP on RoboCasa. We append individual action tokens with source and relative-time information and compare against our default VLA-ULAP method, which does not receive these additional tokens, and a control that retains the tokens but zeros their action values.

With recent-chunk training and alternating execution (ρ=0.5), each ULAP decision immediately follows a VLA decision, so future actions remain in the received chunk. The action-token model achieves 786/1,200 successes, versus 769/1,200 for default VLA-ULAP and 725/1,200 for the zero-content control. The gains are +1.42 percentage points (95% CI [−0.67, 3.33]) and +5.08 points [2.58, 7.50], respectively: a modest observed gain over the default and a larger observed difference from zero-content tokens. Each token variant uses one training seed, so these intervals capture evaluation-state variability rather than training variability; a training-seed contribution to the difference remains possible.

We also trained each model once on a mixture of recent and older chunks, covering both alternating execution and consecutive ULAP decisions, then evaluated at ρ=0.5 and 0.8. Action tokens show no consistent advantage over the zero-content control and only small differences from default VLA-ULAP. Thus, a fresh VLA chunk can help the immediately following local decision, but this benefit does not consistently extend to repeated local decisions as the chunk ages. These results support retaining our default method without additional received-action inputs (Appendix C.4, Table 22).

Q6 — Tasks without ULAP training data

We added two splits, each excluding two tasks from each of two LIBERO suites from local training and checkpoint selection. At ρ=0.5, successes are 193/200 versus 198/200 for VLA-only on Goal/Spatial, and 195/200 versus 197/200 on Object/LIBERO-Long, with 42.1% and 47.6% fewer VLA calls. Paired differences are −2.5 points (95% CI [−5.5, 0.5]) and −1.0 point [−3.5, 1.5]. This supports task transfer with recurring VLA intervention; consecutive local chunks substantially reduce success (§4.1, Table 2; Appendix B.2.1).

We also compare against a simple action-history extrapolation baseline (Ridge), which predicts the next action chunk from the previous seven executed actions, under the same alternating VLA schedule. VLA-ULAP achieves 194/200 versus 164/200 successes on Goal/Spatial in a paired re-evaluation, and 195/200 versus 189/200 on Object/LIBERO-Long, with fewer VLA calls on both splits.

Q7 — Anonymous artifacts

We released training/evaluation code, ULAP weights and configurations for GR00T × all four LIBERO suites and π0.5 × LIBERO-Safety with ten-action prediction chunks, including the delay-injection executor. The physical measurement protocol provides timing boundaries, energy-integration equations and Python reconstruction examples (Reproducibility Statement).

AuthorsOpenReview ↗

Response to Reviewer Hctg (3/3)

Additional weaknesses — Related work, measurement scope, and reporting

Section 2 and Appendix A now discuss Neurosurgeon, split computing/early exiting, DeDelayed and RTC. Our contribution is independent local action prediction between VLA decisions, rather than model-layer partitioning or delayed-feature fusion. We added both RTC and received-plan controls.

We agree that optimized embedded deployment can make onboard VLA inference practical on higher-power modules such as Thor; we do not compare against that implementation. Our target is inexpensive, low-power devices unable to host the evaluated base VLA. The onboard A6000 point in Figure 9 is a same-checkpoint reference reconstructed from local replay, not an embedded-VLA baseline (Appendix B.5).

Using your failure-rate-ratio metric on development-selected operating points confirmed on new states, the hybrid/VLA-only ratios are 1.34 for GR00T (156 versus 116 failures out of 2,000), 1.61 for VLA-JEPA (61 versus 38/2,000), 1.01 for RoboCasa (449 versus 445/1,200), and 1.04 for CALVIN (330 versus 316/1,000). Absolute success-rate decreases are 0.3–2.0 points. Section 4.1 and Appendix B.6 report the corresponding paired intervals alongside retention. The ρ=1 endpoint retains the initial VLA call. The physical evaluation uses shared demonstrations (80/10/10 for ULAP) and distinguishes held-out placements from new tasks. Across both physical tasks, 16 attempts were excluded and repeated (nine for Glue-stick and seven for Ping-pong); ordinary task failures, including collisions, and timeouts remain in the evaluation. We report costs including all failed trials and separately compare time and energy on matched trials where both methods succeed (§4.3; Appendix B.5).

Reviewer_xiWCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

VLA-ULAP uses a fixed schedule to alternate between a cloud-based policy and a local action predictor with approximately 7.4M parameters. The local predictor combines visual observations, robot state, and action history to generate action chunks without accessing the base policy’s internal features or requiring joint training. At selected operating points across four main simulation settings, the method reduces base-policy calls by 45.3–77.3% while retaining 95.0–98.5% of baseline success rates. Real-robot experiments, Jetson profiling, and dynamic-task evaluations provide additional evidence of feasibility.

Critical strengths
  1. Practical design that is independent of the base policy’s internal architecture. ULAP generates actions from fresh observations without remote features or verification. Its approximately 20.7 ms ensemble inference latency on Jetson supports the feasibility of low-power deployment.
  2. Broad empirical validation. Experiments cover multiple base policies, held-out RoboCasa objects, cross-environment long-horizon CALVIN tasks, and a physical robot. Together with several ablations, these results support the central claim that substantial call reductions can preserve most of the baseline success rate.
Critical weaknesses
  1. The sources of the performance gains are not fully isolated. ACT and ULAP differ in visual backbone, action-history inputs, and inference precision, making individual contributions difficult to separate. Moreover, some main experiments execute only a prefix of the base policy’s output, such as 16/32 steps for Cosmos and 16/40 steps for real-robot GR00T. The longer-prefix comparison on SmolVLA does not establish ULAP’s advantage over longer execution prefixes in these settings.
  2. Evidence for system-level and dynamic-task benefits remains limited. Real-robot ULAP runs on Mac MPS, while Jetson costs are estimated by combining separate profiling with invocation counts, excluding communication, queueing, and device idle energy. Cost comparisons also consider only each method’s successful episodes. The dynamic evaluation reports only two previously selected tasks and lacks matched-delay controls, limiting conclusions about end-to-end benefits and the causes of the success-rate improvements.
Questions for authors
  1. Could you add a longer-execution-prefix baseline in one main setting to establish ULAP’s advantage at comparable call counts?
  2. Does the advantage over a lightweight action-prediction baseline persist when visual features, action-history inputs, and inference precision are matched?
  3. Are end-to-end latency and energy measurements available? Does the efficiency ranking remain unchanged when failed attempts are included and total inference cost is divided by the number of successes?
  4. How were the Safety tasks selected, and did selection involve the final evaluation states? Could you provide matched-delay controls and confidence intervals accounting for repeated initial states to substantiate the dynamic-task gains?
2 comments
AuthorsOpenReview ↗

Response to Reviewer xiWC

Thank you for identifying the need to isolate predictor-design effects, control the longer-prefix comparison, and strengthen system-level evaluation.

Q1 — Longer execution prefix at comparable call counts

We added a full-chunk X-VLA/CALVIN control, using one checkpoint to generate 30 actions and executing either 20 or all 30. Compared with the 30-action baseline, VLA-ULAP at ρ=0.5 completes 4.176 versus 4.022 instructions on average: +0.154 (paired 95% CI [0.047, 0.259]), with 17.9% fewer calls (12,266 versus 14,932). Full-chain success counts are close: 676/1,000 versus 674/1,000. At the closer call budget of ρ=0.4, average completion improves by +0.144 [0.045, 0.245], with 672/1,000 versus 674/1,000 full chains.

Q2 — Matched visual features, history, and precision

Yes. We trained a comparable-size MLP with the same frozen Theia features, executed-action history, proprioception, task identity, and action-chunk output. Training data, normalization, loss, schedule, budget, checkpoint-selection criterion, and BF16 inference are matched. At ρ=0.8, ULAP achieves 1,905/2,000 successes versus 1,638/2,000, a paired improvement of 13.35 percentage points (95% CI [11.70, 15.00]). The input- and precision-matched MLP also falls below the extended ACT chunk baseline (1,692/2,000), so input information and inference precision alone do not explain ULAP’s advantage. This is a direct architecture control against MLP; we have not trained an ACT variant with all ULAP inputs matched (§4.2; Appendix B.2).

Q3 — End-to-end measurements and failure-inclusive costs

Yes. ULAP now runs on the Jetson Orin Nano in the physical control loop. Averaging equally across Glue-stick and Ping-pong, using the 56 and 49 pairs respectively completed by both methods, mean decision latency decreases from 534.9 to 182.3 ms (65.9%), time to success from 19.61 to 13.81 s (29.6%), and measured Jetson module energy from 130.6 to 94.2 J per episode (27.9%) (§4.3; Appendix B.5). The two decision paths have distinct costs: in the Glue-stick hybrid, VLA decisions average 528.8 ms and local decisions 49.2 ms (Table 15). The reported mean mixes actual decisions along these paths; it is not the latency of every decision. Decision latency includes client preparation and the server request–response round trip, including inference, but not camera acquisition or action execution. Institutional security restrictions prevented a direct Wi-Fi connection from the Jetson to the server, requiring a Mac relay. To address your request as fully as possible within this constraint, we estimated direct-Wi-Fi latency and energy using measured component timings and payload sizes, with explicit communication assumptions and sensitivity analysis (Appendix B.5).

Including failures and timeouts in the evaluated episodes, control time per success decreases from 25.24 to 17.53 s and Jetson energy per success from 166.9 to 119.2 J on Glue-stick (Table 14). On Ping-pong, failure-inclusive Jetson energy per success decreases from 203.8 to 130.2 J (Appendix B.5). The revised Figure 7 shows that ULAP retains its advantage even after accounting for the inference costs of failed simulation trials. We divide total inference time and GPU-board energy across all trials by the number of successes. Under this failure-inclusive metric, every evaluated ACT, SP-VLA and RT-Cache point has a ULAP point with at least as many successes and lower estimated time and energy per success. The advantage also persists against extended ACT training with temporal ensembling (Figure 14).

Q4 — Safety selection, matched delays, and repeated states

The gains vary by task: relative to delayed VLA-only, the fast hybrid improves Mug (144 versus 73/200) and Basket (172 versus 154/200), matches Book (179/200), and trails on Moka (130 versus 144/200); Bowl is 200/200 for both. We originally selected Moka and Mug because injected delay substantially reduced VLA-only success without a floor effect. That sweep reused official states later used in evaluation. To improve coverage and fairness beyond these delay-sensitive tasks, we now evaluate all five L1 obstacle-avoidance tasks on 200 fresh initial states per task, verified not to overlap states used for training or data collection. All conditions share these states, with one rollout per state.

With VLA delay fixed at five steps, changing only the same hybrid's local delay from five to one step increases successes from 755/1,000 to 825/1,000: +7.0 points (95% CI [4.3, 9.7]). The fast hybrid exceeds delayed VLA-only by +7.5 points [4.9, 10.2], whereas the equal-delay hybrid differs by +0.5 points [−2.4, 3.4]. The full task breakdown and violation counts are reported (§4.4, Figure 10; Appendix B.7, Tables 19–21).

Thank you for the detailed response and additional experiments. The full-chunk control, input- and precision-matched MLP baseline, Jetson deployment, and cost analysis including failed trials provide further clarification and supporting evidence regarding several concerns raised in my original review.

Two points still require clarification. The matched MLP comparison supports the effectiveness of ULAP’s architecture, but does not fully explain its advantage over ACT. The delay controls support the benefit of faster local responses; however, given the ULAP-only results, claims about the additional value of recurring VLA calls should remain qualified. Please also clearly distinguish measured benefits under synchronous, relay-based deployment from estimates for direct Wi-Fi deployment.

Reviewer_Kv9vOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a hybrid inference scheme for VLA policies: VLA-ULAP. A frozen, remotely hosted VLA is called only at some chunk boundaries, and an Ultra-Lightweight Local Action Predictor produces the remaining action chunks on the robot. The experiments are tested on Jetson Orin Nano. VLA-ULAP removes 45.3–77.3% of VLA calls while retaining 95.0–98.5% of baseline success rates at selected operating points.

Critical strengths
  1. The evaluations used five base policies with different architectures, four simulated benchmarks, a real robot, and a real edge device.

  2. A simple, model-agnostic design that fits a real deployment constraint.

Critical weaknesses
  1. The Introduction section only illustrates latency varies across different hardware platforms, without discussing any technical issues limitations. This paper should fall within the field of robotics not ML.

  2. The author only discussed "Can hybrid inference achieve a better balance?", which can be done with some evaluations without any technical skills.

  3. The evidence of the Ultra-Lightweight is not clear. The author only claims the proposed method has low-power, low-latency. But those do not mean Lightweight.

  4. The latency and energy claims rest on a reconstructed cost model that leaves out the factor motivating the paper, and the latency-benefit experiment is narrow.

Questions for authors

Q1: We do not give citations in the "contributions" summary section; such formatting is reserved exclusively for AI-generated documents.

Q2: What is the technical challenge of this paper?

Q3: The headline "95–98.5% retention" is strong evidence for in-distribution call reduction. It is weak evidence for "retaining VLA capability",

Q4: How is the generalization beyond ULAP's training tasks? Generalization beyond ULAP's training tasks

Q5: How were the reported ρ values chosen? Please describe a validation-based selection protocol and report confidence intervals at the selected points. Also report ULAP-alone performance (ρ = 1) explicitly in the main text for every benchmark, including the real robot.

Q6: ACT baseline fairness. ACT was trained with ULAP's 120-epoch recipe, without temporal ensembling and without action history. The original ACT recipe typically trains much longer. Was ACT tuned?

3 comments
AuthorsOpenReview ↗

Response to Reviewer Kv9v (1/2)

Thank you for the questions about the contribution, generalization, operating-point selection, and baseline fairness.

Q1 — Citations in the contribution summary

We removed citations from the contribution bullets and placed the relevant citations in the surrounding discussion (§1).

Q2 — Technical challenge

Thank you for highlighting that the technical difficulty needs to be explicit. Replacing complete VLA calls introduces three coupled difficulties, each addressed by the design and evaluated in the revision (§§1–3).

  • Local actions change subsequent inputs. A predictor must continue controlling states affected by its own errors, beyond fitting actions on fixed trajectories. ULAP therefore predicts each chunk from current observations, with periodic VLA calls retaining access to the base policy's task capabilities. On tasks excluded from ULAP training, alternating VLA and ULAP maintains high success rates, whereas relying on ULAP after the initial VLA call sharply reduces them. This supports the combined system's generalization and the role of periodic VLA calls (Q3–Q4; §4.1).

  • The models must cooperate without internal coupling. ULAP independently produces executable action chunks, while the unchanged VLA receives current observations through its normal inference interface. Neither joint training nor access to VLA intermediate features is required. Application across GR00T, VLA-JEPA, Cosmos-Policy, X-VLA, and π0.5 demonstrates the practicality of this interface (§4).

  • Useful prediction must fit limited local computation. Compact visual processing, shallow fusion with a separate state branch, and a one-pass action head address this constraint. With comparable size and matched inputs and outputs, ULAP achieves 1,905/2,000 successes versus 1,638/2,000 for an MLP: +13.35 points (95% CI [11.70, 15.00]). At ρ=0.8, ULAP trained for 120 epochs also exceeds the larger ACT model trained for 600 epochs with temporal ensembling (1,905/2,000 versus 1,709/2,000; §4.2, Appendix B.2). Jetson Orin Nano deployment demonstrates shorter synchronous execution and lower energy per episode while maintaining high success rates (§4.3).

Together, these results support the compact predictor and its integration with an unchanged VLA, rather than only an evaluation of hardware placement.

Q3 — Generalization across environments, placements, and tasks

We agree that in-distribution retention alone is insufficient evidence for retaining broader VLA capabilities. Our existing evaluations test transfer to a held-out environment in CALVIN and to held-out objects in RoboCasa (§4.1; Appendices B.3–B.4), as well as to held-out object placements on the physical robot (§4.3; Appendix B.5). In response to your concern, we also tested whether VLA-ULAP transfers to tasks excluded from ULAP training. The new task-held-out results are:

Held-out splitVLA-onlyVLA-ULAP (ρ=0.5)Paired difference, pp (95% CI)VLA-call reduction
Goal/Spatial198/200193/200−2.5 [−5.5, 0.5]42.1%
Object/LIBERO-Long197/200195/200−1.0 [−3.5, 1.5]47.6%

These results support call reduction with high success rates on tasks excluded from local training (§4.1, Table 2; Appendix B.2.1).

Q4 — Tasks absent from ULAP training

In the evaluation described in Q3, two tasks per suite are held out and one shared ULAP is trained per split. Alternating VLA and ULAP retains 193/200 and 195/200 successes. With three consecutive ULAP chunks (ρ=0.75), these fall to 42/200 and 158/200; with only the initial VLA call (ρ=1), they fall to 0/200 and 38/200. The high held-out-task success therefore belongs to the combined system with periodic VLA access.

We also compare against a simple action-history extrapolation baseline (Ridge), which predicts the next action chunk from the previous seven executed actions, under the same alternating VLA schedule. VLA-ULAP achieves 194/200 versus 164/200 successes on Goal/Spatial in a paired re-evaluation, and 195/200 versus 189/200 on Object/LIBERO-Long, with fewer VLA calls on both splits. The Goal/Spatial gain supports learned local prediction beyond simple extrapolation (Appendix B.2.1, Table 12).

AuthorsOpenReview ↗

Response to Reviewer Kv9v (2/2)

Q5 — Selection, uncertainty, and ρ=1

Thank you for highlighting the need to separate operating-point selection from evaluation. In response, we added a confirmation evaluation on new initial states. We used the original sweeps as development data, selected the fewest-call setting retaining at least 95% of baseline development successes, then froze it before this confirmation evaluation.

Base / benchmarkρVLA-onlyVLA-ULAPULAP 95% CI (%)Paired difference, pp (95% CI)Call reduction
GR00T / LIBERO0.61884/20001844/200091.10–93.30-2.00 [-3.35, -0.65]53.9%
VLA-JEPA / LIBERO0.81962/20001939/200096.20–97.65-1.15 [-1.90, -0.40]77.6%
Cosmos / RoboCasa0.5755/1200751/120060.75–64.50-0.33 [-2.42, 1.75]49.0%
X-VLA / CALVIN (full chains)0.5684/1000670/100064.10–70.00-1.40 [-4.60, 1.90]45.6%

Section 4.1 and Appendix B.6 (Table 18) report the rule, paired baseline comparisons, and intervals. Section 4.1 also reports the development-cohort ρ=1 endpoints: 1,564/2,000, 1,775/2,000, 501/1,200, and 220/1,000, respectively. Here ρ=1 retains the mandatory initial VLA call.

For the robot experiments, ρ=0.75 was fixed in the deployment protocol rather than selected by the simulation development-sweep rule. For Glue-stick, this fixed deployment setting ρ=0.75 achieves 67/75 (95% CI 82.7–94.7%), versus 57/75 (66.7–85.3%) at ρ=1 (§4.3; Table 3; Appendix B.5). On Ping-pong, the corresponding counts are 62/75 and 58/75.

Q6 — ACT fairness

Using the same teacher corpus and episode splits, we extended ACT training from 120 to 600 epochs, selected checkpoints by validation error, and evaluated both chunk execution and temporal ensembling. At ρ=0.8, ACT obtains 1,692/2,000 and 1,709/2,000 successes, respectively, versus 1,905/2,000 for ULAP trained for 120 epochs. Among the extended-training controls, ACT at 600 epochs and ρ=0.5 reaches 95.45% (1,909/2,000), versus ULAP’s 95.25% (1,905/2,000): a 0.2-point difference, with ULAP using 49.8% less estimated inference time and 51.6% less GPU-board energy per success. Figure 14 compares the success-rate–cost trade-off using actual forward counts, including TE's per-step inference (§4.2; Appendix B.2). Separately, an MLP with comparable parameter count and the same visual features, executed-action history, state, task identity, action-chunk output, and inference precision reaches 1,638/2,000; ULAP's paired advantage is 13.35 points (95% CI [11.70, 15.00]).

Additional weaknesses — ML contribution, lightweight design, and deployment evidence

Our ML contribution concerns learning a compact policy that remains useful under the action-dependent distribution changes of closed-loop execution, tested through predictor-design controls and held-out-task transfer. Robotics is the evaluation domain, and is explicitly included in the ICLR 2027 call for papers.

ULAP uses 1.84–2.12M trainable parameters per predictor and shares a frozen visual encoder with 5.52M parameters (§3; Appendix B, Tables 4–5). A single ULAP model paired with VLA-JEPA contains 7.40M parameters including the frozen encoder, with 29.6 MB of FP32 stored tensors (not runtime memory). This compact model also performs well against larger and similarly sized alternatives. At ρ=0.8 on LIBERO, VLA-ULAP succeeds on 1,905/2,000 trials, compared with 1,709/2,000 for ACT after extended training and temporal ensembling. The profiled ACT model has approximately 51.6M parameters including its visual encoder. A similarly sized MLP given the same input information succeeds on 1,638/2,000 trials, showing that ULAP’s predictor design makes better use of a comparable parameter budget (§4.2; Appendix B.2).

“Lightweight” is also supported by direct component profiles: on the same RTX A6000, ULAP takes 17.0 ms and 1.67 J per call, versus 212.2 ms and 27.3 J for VLA-JEPA (about 12.5× lower latency and 16.3× lower GPU-board energy; Appendix B.2). On Jetson Orin Nano, the complete local decision with the two-model ensemble takes 49.2 ms (Appendix B.5, Table 15). These measurements supplement the parameter counts; they do not equate device power with model size.

On the physical SO-101, we now run ULAP on a Jetson Orin Nano and measure decision latency and Jetson module energy in the control loop. Averaging equally across Glue-stick and Ping-pong, using the 56 and 49 pairs respectively completed by both methods, mean decision latency decreases from 534.9 to 182.3 ms (65.9%), time to success from 19.61 to 13.81 s (29.6%), and measured Jetson module energy from 130.6 to 94.2 J per episode (27.9%) (§4.3; Appendix B.5). We also evaluate the full LIBERO-Safety L1 suite. With both models, the alternating schedule, and VLA delay fixed, reducing ULAP’s response delay improves task success by 7.0 percentage points (§4.4; Appendix B.7).

I thank the authors for the detailed response and the substantial additional experiments. I have read the rebuttal, the responses to the other reviewers, and Reviewer Hctg’s follow-up comments. However, I think the technical challenge and ML contribution is still the same. The rebuttal lists three difficulties. The first is the standard covariate-shift problem of any behavior-cloned policy. The second is a design choice rather than a challenge the method overcomes. The third is met with standard components. None is a unique problem that raises new issues for interleaving a local predictor with a remote VLA. So I keep my score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overall response

We thank the reviewers for identifying where stronger controls and deployment measurements were needed. We clarify the technical contribution: a compact predictor that generates actions from current observations, integrated with an unchanged VLA through periodic calls. The design addresses closed-loop state changes, cooperation without VLA intermediate features, and limited local computation (§§1–3). The resulting system maintains high task success with fewer VLA calls and, in synchronous physical deployment, shortens task execution and reduces onboard energy (§4). The principal additions are:

  • Synchronous real-robot deployment. ULAP runs on Jetson Orin Nano. VLA-ULAP achieves 67/75 versus 63/75 successes on Glue-stick and 62/75 versus 58/75 on Ping-pong. Averaging equally across the 56 and 49 pairs respectively completed by both methods, time to success decreases from 19.61 to 13.81 s (29.6%) and measured Jetson energy from 130.6 to 94.2 J per episode (27.9%). Institutional security restrictions require a Mac relay, so we also estimate direct-Wi-Fi deployment from measured components. At 100 Mbps, averaging both tasks equally, VLA-ULAP reduces mean decision latency from 477.4 to 167.5 ms (64.9%) and Jetson-plus-radio energy per episode from 134.0 to 99.4 J (25.8%) versus remote VLA-only, with actions and outcomes held fixed (§4.3; Appendix B.5).
  • Server energy, evaluated separately. Including failed trials and timeouts, estimated A6000 GPU-board energy per success decreases from 2.285 to 1.270 kJ on Glue-stick and from 2.872 to 1.427 kJ on Ping-pong. These are retrospective estimates from energy counters aligned to the control interval (§4.3; Appendix B.5).
  • Predictor-design controls. With comparable size and matched inputs and outputs, ULAP improves success rate over an MLP by 13.35 percentage points (95% CI [11.70, 15.00]). Its success-rate–inference-cost advantage also persists with extended ACT training and temporal ensembling (§4.2; Appendix B.2).
  • Generalization and delayed execution. On two task-held-out splits, alternating VLA and ULAP retains near-VLA-only success with 42.1–47.6% fewer VLA calls. Using the VLA only for the first action chunk and ULAP for all subsequent chunks sharply lowers success rates, showing that periodic VLA calls help the combined system generalize to tasks excluded from ULAP training (§4.1). Separately, on the full LIBERO-Safety suite, holding both models and VLA delay fixed while reducing ULAP delay from five to one step improves success rate by 7.0 points (95% CI [4.3, 9.7]; §4.4).

We also add operating points confirmed on new states, uncertainty estimates, failure-inclusive costs, received-plan controls, and an anonymous code/weights/configuration release.

Area_Chair_cb7YOpenReview ↗

Discussion Reminder

Dear Reviewers,

I hope this message finds you well. The rebuttal is now available in the discussion forum. I would like to kindly remind you to check the rebuttal and leave comments for the authors, which would be very helpful.

Thank you very much for your time and effort. Please let me know if you have any questions.

Best

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.