acceptodds
Under review as a conference paper at ICLR 2027

LATENT BRIDGE: FEATURE DELTA PREDICTION FOR EFFICIENT DUAL-SYSTEM VISION-LANGUAGE-ACTION MODEL INFERENCE

Abstract

Dual-system Vision-Language-Action (VLA) models achieve state-of-the-art robotic manipulation but are bottlenecked by the VLM backbone, which must execute at every control step while producing temporally redundant features. We propose Latent Bridge, a lightweight model that predicts VLM output deltas be- tween timesteps, enabling the action head to operate on predicted outputs while the expensive VLM backbone is called only periodically. We instantiate Latent Bridge on two architecturally distinct VLAs: GR00T-N1.6 (feature-space bridge) and π0.5 (KV-cache bridge), demonstrating that the approach generalizes across VLA designs. Our task-agnostic DAgger training pipeline transfers across benchmarks without modification. Across four LIBERO suites, 24 RoboCasa kitchen tasks, and the ALOHA sim transfer-cube task, Latent Bridge achieves 95–100% perfor- mance retention while reducing VLM calls by 50–75%, yielding 1.55–1.73× net per-episode speedup.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLA, Efficiency, Latent world model

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_SMrFOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper aims to address inference cost in dual-system VLAs, where the VLM backbone dominates per-call latency. The authors proposes Latent Bridge, a small DiT that predicts changes in the representations consumed by their action heads between consecutive inference calls. It predicts feature deltas for GR00T and per-layer, pre-RoPE KV deltas for π0.5\pi_{0.5} with supervised training followed by DAgger refinement. Longer-horizon GR00T tasks additionally use action-head LoRA and adaptive refresh scheduling. On LIBERO, RoboCasa and ALOHA-sim report 95-100% average success retention with 50-75% fewer backbone calls and a 1.55-1.73× net episode speedup.

Critical strengths
  • Predicting the full per-layer KV interface rather than only a final feature tensor makes the π0.5\pi_{0.5} result useful. The large improvement over matched period stale reuse in Table 1 validates the value of learned prediction beyond simply reducing VLM frequency.

  • Speedup is reported as wall-clock with an episode-length correction and per-component latencies, and the weak pruning baselines are explained by an Amdahl argument.

Critical weaknesses
  • Appendix H uses an identical refresh mask at every layer and describes savings as limited to K/V projections, but the original VLA-Cache includes task-relevant token exclusion and layer-adaptive reuse and its default similarity threshold is 0.996 instead of the 0.90 criterion described here. Its applicability is not limited to single-system architectures since it also evaluates CogACT. These differences may affect both success and latency.

  • Section 4.1 calculates speedup from inference latency multiplied by mean successful episode length, which means it measures episode inference compute but not the demonstrated end-to-end robot completion time (it excludes failures and physical execution time).

Questions for authors
  • How does each bridge behave when the scene is perturbed between VLM calls (e.g., the target object is displaced)?

  • For GR00T LIBERO-10, what are the speed and success of scheduling alone, caching with the same action-head adaptation, and the full bridge under matched refresh schedules? I'm curious about how much benefit comes from delta prediction.

7 comments

Dear Reviewer SMrF,

Thank you for the careful reading of Appendix H and of the speedup definition. A revised PDF is uploaded together with this response (changes listed at the end of Part 2/2). In brief:

  • W1 (VLA-Cache port). The Table 1 port is a simplified one. A faithful port with all three components keeps the success rate (95.7 vs 96.2 for sync) and gives no speedup. Both ports are now in Appendix H.
  • W2 (speedup definition). Your reading is correct. The metric is now named net inference-time speedup and defined in Section 4.1.
  • Q1 (scene perturbation). New test: with an object displaced by 3 cm on a bridge step, the bridge reaches 90.3% against 91.5% for sync and 30.8% for stale reuse.
  • Q2 (GR00T LIBERO-10). New runs: scheduling alone lifts stale reuse to 27%, the same action-head adaptation to 31%; the bridge under the same schedule reaches 84%.

W1: VLA-Cache adaptation. Appendix H is corrected on three points. (i) The port used for Table 1 is a simplified one: it uses the released static-patch selection, removes the selected tokens at the first pruning layer with a fixed reuse fraction, reads their K/V from the previous step, and ports neither task-relevant token exclusion nor the entropy-based per-layer schedule. (ii) Its cosine threshold is 0.90, chosen from a sweep over 0.95 / 0.90 / 0.80 (0.95 gave higher success with under 5% latency reduction); the released default is 0.996, which the faithful port below uses. (iii) VLA-Cache also evaluates CogACT.

To settle what these differences cost, we re-implemented VLA-Cache on π0.5 from the released code with all three components (static patches at the default threshold 0.996 with the two-view top-k of the released OpenVLA-OFT setting, task-relevant exclusion by the previous call's layer-15 text-to-vision attention, entropy-based per-layer reuse proportions, pruning at the released layers; text tokens never reused) and evaluated it on the same stack as sync (3 seeds × 20 episodes per task):

π0.5, every stepSpatialObjectGoalLIBERO-10MeanPrefix tokens computedPer-call time vs. sync
Sync99.098.395.891.796.2100%1.00×
VLA-Cache, faithful port98.796.896.590.895.794–95%1.18× (slower)
VLA-Cache, port of Table 193.291.092.286.890.8–0.86×

The faithful port loses no success rate (0.5 pp from sync, within seed noise) and gives no speedup: the exclusion and the entropy schedule reuse only a fraction of the 137–166 reusable tokens per call, so 94–95% of the prefix is still computed and the attention statistics the method needs cost more than the 5% it saves (1.18× sync per call on the same H200; per-step action deviation from sync 0.01). The simplified port reuses more and trades success for a 14% latency reduction. Either way the conclusion holds for a structural reason: a method that keeps the backbone on every step is bounded by the untouched action head (at most 1.33× per step on π0.5 even at the 1.7× VLA-Cache reports as its best), whereas the bridge skips the backbone call (1.55×). The revision reports both ports in Appendix H and marks the Table 1 row as the simplified port.

W2: what the speedup measures. The reviewer's reading is correct, and the revision says so: the metric is inference time per completed episode, per-call latency × calls per successful episode, now named net inference-time speedup. Physical execution time is set by the control rate and is common to all methods. Failed episodes run to the step cap under every method, so counting them would measure the cap; the metric charges the bridge for its longer successful episodes. With the GR00T schedule corrected and RoboCasa and ALOHA added, the net speedup is 1.47–1.67× across the four settings.

Q1: scene perturbed between VLM calls. The two instantiations differ. The π0.5 bridge runs SigLIP on every step and takes the embedding difference as input, so a displaced object changes the bridge input immediately; on held-out real streams its advantage over stale reuse grows with motion, from +0.002 cosine in the slowest tercile to +0.012 in the fastest (Appendix K). The GR00T bridge has no fresh vision on bridge steps: it predicts from the last features, state and previous action, so a displacement is seen at the next backbone call, at most f−1 calls later. In both cases the exposure is bounded by f. We ran this test on π0.5 (LIBERO-Goal, f=4, 3 seeds × 20 episodes per task): before the sixth policy call, a bridge step, a movable task object is displaced by 3 cm in a random horizontal direction, identically for every method. Sync reaches 91.5%, the bridge 90.3% and stale reuse 30.8% (97.0 / 97.3 / 68.8 without the displacement, Table 1). The bridge sees the displaced object on the same call; stale reuse acts on pre-displacement features for three more calls (Appendix F).

Q2: GR00T LIBERO-10 decomposition. We ran the two configurations the reviewer asks for, scheduling alone and caching with the same action-head adaptation. They were run on a second machine, so reference rows were re-run there and are shown next to the paper's values (3 seeds × 20 episodes per task; sync and bridge R1 at f=3: one seed).

ConfigurationVLM callsSR, paperSR, re-run
Sync100%93.0095.0
Stale reuse, f=333%42.339.8
Stale reuse + LoRA, f=333%–12.2
Bridge R1, f=333%67.1766.5
Bridge R1 + LoRA, f=333%73.17–
Stale reuse, phase-aware40%–27.3
Stale reuse + LoRA, phase-aware39%–30.7
Bridge R1, phase-aware42%88.83–
Bridge R1 + LoRA, phase-aware42%89.1784.0

Both columns give the same ordering, and the bridge is well above stale reuse at every matched setting. In the re-run:

  • Scheduling alone lifts stale reuse from 9.8 to 27.3 at 40% of the calls.
  • The same action-head adaptation adds 2–3 points to stale reuse under either schedule.
  • The full bridge under the same schedule and adaptation reaches 84.0, +53 points over stale reuse (30.7); at fixed f=3 it is +54 points over stale reuse + LoRA (66.5 vs 12.2).

Per-step latency follows the call rate: 48–49 ms at f=3 and 52–55 ms under the phase-aware schedule for both methods, against 90 ms for sync (the bridge network adds 2 ms on skipped steps). On π0.5, which uses a fixed f=4 and no adaptation stage, the bridge reaches 93.67 on LIBERO-10 against 52.67 for stale reuse at the same call budget (Table 1).

Section 3 now states the text-token handling on π0.5 precisely: the text-token KV entries are zero-filled on bridge steps (they are carried over on GR00T), and the action expert reads the instruction through the image-token KV. A controlled replay shows the comparisons do not depend on this (action deviation from sync 0.277 vs 0.275 for stale reuse and 0.075 vs 0.063 for the bridge, zero-filled vs carried over; Appendix F), and no reported number changes.

Changes in the revised PDF.

  • Appendix H: the faithful VLA-Cache port on π0.5 (components, success rates, token fractions, timing) in its own table; the threshold of the Table 1 port stated; the Table 1 footnote marks its rows as the simplified port.
  • Section 3: the text-token handling on π0.5 corrected (text KV zero-filled on bridge steps; the action expert reads the instruction through the image-token KV).
  • Section 4.1 and Table 1: the speedup defined as net inference-time speedup, per-call latency × calls per successful episode.
  • Appendix F: the scene-perturbation test (object displaced on a bridge step; sync / stale reuse / bridge).
  • Appendix F: GR00T LIBERO-10 with stale reuse under the phase-aware schedule and with the action-head LoRA, next to re-run reference rows.
  • Related Work: Leap-VLA, Acting While Understanding and AC²-VLA cited.
  • Figures 1–3 redrawn as vector figures in one style.

Dear Reviewer SMrF,

Thank you again for the constructive review.

As the discussion period closes today, we wanted to confirm that our rebuttal addresses your concerns. We are happy to expand any of the above if that would help the discussion. If we have resolved your issues, we sincerely hope that you can raise the rating score.

Best regards,

The authors

Reviewer_85GDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Latent Bridge speeds up dual-system VLAs by running the VLM backbone only every f steps; in between, a small DiT predicts how the backbone output changes (image features for GR00T-N1.6, per-layer K/V for π0.5), with one DAgger round to correct drift. On LIBERO it reports 1.73× (GR00T) and 1.55× (π0.5) net speedup with near-sync success. All closed-loop experiments are in simulation.

Critical strengths

The bridge clearly beats reusing stale features at the same call rate: for π0.5 on LIBERO at f=4 it reaches 96.92% vs. 96.96% for sync, while caching drops to 56.38% (Table 1), and similar gaps hold on RoboCasa and ALOHA (Table 2). This supports the core idea that the per-step update is learnable.

Critical weaknesses

A simpler alternative is not compared on speed: App. F shows sync π0.5 with a 10-action chunk keeps 97.50% success while halving all inference calls, which could match or exceed the bridge's 1.55×, but no latency is reported for it. The speedup claims are also narrower than stated: net speedup counts only successful episodes and is measured only on LIBERO, although the abstract extends it to RoboCasa and ALOHA.

Questions for authors
  1. Please report latency and net speedup for sync and the bridge with a 10-action chunk.
  2. Which f schedule produced GR00T's 49 ms in Table 1, and what are the speedups on RoboCasa and ALOHA?
  3. How does the method relate to HiRT (CoRL 2024), which also runs the VLM at low frequency?
2 comments

Dear Reviewer 85GD,

Thank you for the precise questions. A revised PDF is uploaded together with this response (changes listed at the end). In brief:

  • Q1 (10-action chunk). Chunk 10 alone gives 1.94×, and the bridge multiplies it: 3.02× with both, at 95.83% SR.
  • Q2 (the 49 ms; RoboCasa and ALOHA). With the per-suite schedules the GR00T average is 53 ms and the net speedup 1.59×, not 1.73×. RoboCasa and ALOHA are now measured at 1.67× and 1.47×. Table 1 and the abstract are corrected.
  • Successful episodes only. The metric is now defined explicitly in Section 4.1.
  • Q3 (HiRT). HiRT reuses a stale latent inside a jointly trained policy; we leave the VLA frozen and predict the change. HiRT is now in Related Work.

Q1: latency and speedup with a 10-action chunk. Per-call latency does not depend on the chunk length, so inference time per episode is calls × per-call cost. With the calls per successful sync episode recorded on Object, Goal and LIBERO-10, and bridge calls scaled by the +6% of Table 1:

π0.5, f=4Calls per episodems per calls per episodevs sync, chunk 5
Sync, chunk 535.2762.681.00×
Sync, chunk 1018.1761.381.94×
Bridge, chunk 537.5461.721.55×
Bridge, chunk 1019.3460.893.02×

Chunk 10 alone gives 1.94×, and the two multiply rather than compete: the bridge contributes the same 1.55× at either chunk length, and a bridge re-trained for the length-10 chunk is 1.55× faster than sync at chunk 10, at 95.83% SR. Chunk 10 is the full action horizon of this checkpoint, so chunking cannot go further, whereas f can (f=8 keeps at least 94.5% on three suites). Stale reuse falls to 38.92% at chunk 10 while the bridge holds. The table is added to Appendix F.

Q2: the 49 ms, and RoboCasa/ALOHA. Thank you for catching this. The 49 ms is the component-level average at f=3 applied to all suites. With the per-suite schedules (f=3 on Spatial and Object, f=2 on Goal, phase-aware on LIBERO-10 with 57.6% fewer calls) the averages are 49/49/60/55 ms, mean 53 ms, and the net speedup is 1.59× rather than 1.73×; Table 1 and the abstract are corrected. RoboCasa and ALOHA latency were not measured in the submission; they are now:

Settingms per call, sync → bridgeCalls per episode, sync → bridgeNet speedup
RoboCasa, GR00T, f=392 → 5125 → 271.67×
ALOHA, π0.5, f=374 → 4633 → 361.47×

The speedup transfers for a structural reason: a bridge step removes the backbone call whatever the task, so per-call latency falls by 38–45% on every platform and the 8–9% longer episodes offset only a small part of it. The range over the four settings is 1.47–1.67×.

Successful episodes only. Failed episodes run to the step cap under every method, so counting them measures the cap rather than the method, and a failed episode has no practical value however quickly it runs. The metric is the inference cost of completing the task, and it charges the bridge for its longer successful episodes; the revision states this definition explicitly.

Q3: HiRT. Both run the VLM at a lower rate; the mechanisms differ. HiRT caches a single pooled latent from the last VLM call, reuses it unchanged and conditions a new policy trained jointly with a LoRA-tuned VLM, so it is a new architecture; Latent Bridge leaves a trained VLA frozen and predicts how its representation changes. HiRT reports a drop from 94.2/76.0 to 63.4/52.5 (seen/unseen) at a VLM interval of 6; our sweep at f=6 gives 98.0/96.5/98.5/87.5 on the four LIBERO suites. The benchmarks differ, so this is context rather than a head-to-head result. HiRT is now in Related Work.

Changes in the revised PDF.

  • Section 4.1 defines the speedup as net inference-time speedup, per-call latency × calls per successful episode; the Table 1 caption uses that name.
  • Table 1 footnote: the GR00T latency average is stated as the average over the per-suite schedules (49/49/60/55 ms).
  • Table 1 footnote: the VLA-Cache rows are marked as the simplified port, with the faithful port in Appendix H.
  • Figures 1–3 redrawn as vector figures in one style, with Figure 1(c) built from the Table 1 numbers.
  • Table 1: the GR00T net speedup corrected to 1.59×. Abstract: the range over LIBERO, RoboCasa and ALOHA stated as 1.47–1.67×.
  • Appendix F: inference time per episode at chunk lengths 5 and 10 for sync and the bridge.
  • Related Work: HiRT cited and positioned relative to the bridge.

Dear Reviewer 85GD,

Thank you again for the constructive review.

As the discussion period closes today, we wanted to confirm that our rebuttal addresses your concerns. We are happy to expand any of the above if that would help the discussion. If we have resolved your issues, we sincerely hope that you can raise the rating score.

Best regards,

The authors

Reviewer_CW1GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose Latent Bridge, an inference acceleration framework for dual-system vision-language-action models. Instead of running the expensive VLM backbone at every control step, the method introduces a lightweight network that predicts changes in the backbone's hidden representations across consecutive observations. The authors implement this idea using feature prediction for GR00T-N1.6 and KV cache prediction for π0.5. To reduce accumulated prediction errors, they introduce a training pipeline involving supervised learning and DAgger refinement, with optional action head adaptation and phase-aware scheduling. Experiments on LIBERO, RoboCasa, and ALOHA simulation demonstrate that Latent Bridge can reduce VLM calls by 50 to 75% while maintaining relatively high task success rates. The authors report 1.55 to 1.73× net inference speedup compared with the original policies.

Critical strengths

The paper addresses an important problem in efficient VLA inference, and the idea of predicting intermediate representations instead of repeatedly computing the entire VLM backbone is interesting. I appreciate that the authors consider two different VLA architectures and provide detailed analysis of feature prediction quality, action deviation, and closed-loop task success. The comparison with naive feature caching and Taylor extrapolation helps demonstrate that learned feature updates can be more effective than simply reusing previous representations. The experiments also cover multiple benchmarks and include ablations on DAgger training, bridge capacity, visual inputs, and VLM refresh frequency. The reported speedups are encouraging, especially considering that the method largely preserves the original task performance.

Critical weaknesses

My main concern is whether the proposed method provides sufficient novelty and practical advantages over existing temporal feature reuse and efficient VLA inference approaches. Previous work such as EAGLE has already explored learned feature prediction for efficient inference, while VLA-Cache, VLA-ADP, SpecPrune-VLA, and other recent methods exploit temporal redundancy or action dynamics to reduce redundant computation. The authors propose extending feature prediction to consecutive robot control steps, but I find that the core design mainly combines established feature forecasting, residual prediction, and DAgger training. The phase-aware scheduling strategy is also relatively straightforward, using action magnitude to determine how frequently the backbone should execute. More importantly, the experimental comparisons are not entirely convincing. Although the authors compare against FastV and VLA-Cache, these methods were originally designed for different architectures and require substantial adaptation to the dual-system VLAs considered in this paper. The relatively poor performance of these adapted baselines makes it difficult to assess the advantage over stronger methods specifically designed for temporal computation reuse. My second concern is the lack of real-world closed-loop validation. All reported task success rates come from simulation, while the real robot experiments only measure feature reconstruction and action prediction errors on recorded observations. These results do not establish that the method remains reliable when prediction errors influence future observations and robot states. This is particularly important because Latent Bridge autoregressively predicts hidden representations, and even small errors may accumulate during contact-rich or dynamic manipulation. My third concern is the limited generalization of the learned bridge. Appendix G shows that transferring a bridge trained on LIBERO-Spatial to Goal and LIBERO-10 reduces success, suggesting that the method learns checkpoint-specific dynamics rather than a broadly transferable representation update rule. Furthermore, the method requires additional data collection, bridge training, and DAgger refinement for each checkpoint. Appendix A estimates approximately 8 to 12 GPU hours per training pipeline and a break-even point of tens of thousands of episodes. These additional costs are not fully reflected in the headline inference speedups. Overall, I find the results promising, but the methodological novelty, comparison fairness, and practical generalization remain insufficiently convincing.

Questions for authors

Could the authors provide a more systematic comparison with recent learned caching and adaptive computation approaches under matched VLA architectures and computational budgets? In particular, how much improvement comes specifically from predicting feature deltas rather than using a simpler learned predictor or adaptive VLM refresh strategy? I would also appreciate a clearer discussion of how the phase-aware scheduling differs from existing action-aware computation allocation methods such as VLA-ADP. Second, can the authors demonstrate the method in a real robot closed-loop setting, especially on tasks involving rapid visual changes or contact-rich manipulation? The current offline evaluations on real observations cannot fully capture the impact of accumulated prediction errors during execution. Third, Appendix G suggests substantial degradation when transferring the bridge across LIBERO suites. Can the authors clarify whether a single bridge can generalize across different checkpoints or tasks without additional training? Finally, could the authors report a more complete efficiency comparison that accounts for bridge training, DAgger data collection, and deployment on hardware beyond the A100 GPU? These results would help establish whether the proposed acceleration framework provides practical benefits beyond the evaluated simulation settings.

3 comments

Dear Reviewer CW1G,

Thank you for the thorough review. A revised PDF is uploaded together with this response (changes listed at the end of Part 2/2). We acted on each concern:

  • Novelty. Prior methods keep the backbone on every step or reuse its stale output; the bridge predicts what the skipped call would have produced. The closest work is now in Related Work.
  • Comparison fairness. A faithful VLA-Cache port on π0.5 keeps the success rate (95.7 vs 96.2 for sync) and gives no speedup.
  • Cost and hardware. The one-off cost (8–12 GPU-hours per bridge) is now stated as a limitation; the gain holds on an edge SoC (1.52–1.65× net).
  • Delta prediction. New control: the same network predicting the next KV directly loses 11.3 pp of mean SR.
  • Real robot. New closed-loop study on a physical arm: bridge 28/30, sync 25/30, stale reuse 8/30.
  • Generalization. One bridge trained on all four suites reaches 96.3% at f=3, against 96.8 for the per-suite bridges and 96.2 for sync.

Novelty. Feature forecasting, residual prediction and DAgger are each established. The contribution is using them to predict the next control step's backbone representation so that the backbone call is skipped in a closed loop.

MethodWhat is reused or predictedFull backbone on every step?Requires
EAGLEnext-token feature within one decode; every draft is verified by the full model, which makes it losslessyes (verification)trained draft head
VLA-Cacheprevious-step K/V of static image tokens, no predictionyestraining-free
VLA-ADP (ICLR 2026)subset of visual tokens, on/off from the previous chunk's displacementyes, all layerstraining-free
SpecPrune-VLAtwo-level token pruning with a velocity-based controlleryestraining-free
HiRTstale pooled latent of the last VLM callasynchronousnew policy, trained jointly with the VLM
Latent Bridgelearned per-step delta of backbone features or per-layer KV, chainedno: skipped on f−1 of f stepsbridge training; VLA frozen

Our search found no prior work that learns the next-step backbone representation in order to skip the VLM. The closest recent methods reuse a stale cache and train the action module to tolerate it (Leap-VLA, Acting While Understanding), learn when a cached representation can be reused (AC²-VLA), or forecast VLA features with the backbone still running (AHEAD): they decide when to reuse or how to tolerate staleness, we predict what the skipped call would have produced, and a learned refresh policy could be combined with the bridge. All are now in Related Work. We also corrected our description of SpecPrune-VLA (π0 results in its preprint, code public since June 2026).

Phase-aware scheduling vs. VLA-ADP. Both read recent motion, on orthogonal axes: VLA-ADP decides how many visual tokens the backbone sees on each step, our scheduler decides how often the backbone runs at all, and the two compose. We agree the rule is simple; it is an optional stage (Section 3.4), used only on GR00T LIBERO-10, and the novelty claim does not rest on it.

Comparison fairness. The VLA-Cache port used for Table 1 is simplified (fixed reuse fraction, no task-relevant exclusion, no entropy schedule); Appendix H now says so. We re-implemented VLA-Cache on π0.5 from the released code with all three components and evaluated it on the same stack as sync (3 seeds × 20 episodes per task): mean SR 95.7 (sync 96.2) at 1.18× the per-call time of sync, i.e. no speedup, because with the released exclusion and schedule the method recomputes 94–95% of the prefix tokens and the attention statistics it needs cost more than it saves. The simplified port of Table 1 (mean 90.8, 0.86× per call) is the version that trades success for latency. Any method that keeps the backbone on every step is bounded by the untouched action head: at the best ratio VLA-Cache reports on any model (1.7×), Amdahl's law gives at most 1.40× per step on GR00T and 1.33× on π0.5, below the bridge; SpecPrune-VLA independently measures VLA-Cache at 1.07× on OpenVLA-OFT.

Cost. The one-off cost is in Appendix A: 8–12 GPU-hours per bridge end to end, break-even after 34k–51k episodes (15k–28k when only the marginal compute is charged); the limitations now state that the net speedup excludes it. The target quantity is per-call latency at deployment, which decides whether a policy fits a control period, and training compute cannot substitute for it at run time. After correcting the GR00T schedule and adding RoboCasa and ALOHA, the net speedup is 1.47–1.67× across the four settings. On other hardware the gain scales with the backbone's share of a call (61% for π0.5 and 70% for GR00T on the A100): on the GPU of a Qualcomm Dragonwing IQ-9075 evaluation kit (Snapdragon-class SoC) the share is 63% (backbone 134 of 213 ms; bridge step 91 ms), giving 1.52× net at f=3 and 1.65× at f=4.

How much comes from delta prediction? Three controls isolate it.

  • Direct prediction, same network and pipeline (π0.5, f=4, 3 seeds × 20 episodes per task):
π0.5, f=4SpatialObjectGoalLongMean SR
Residual delta (ours)99.0097.6797.3393.6796.92
Direct next-KV95.6793.1781.8371.6785.58
Stale reuse46.6757.3368.8352.6756.38

The gap grows with task length (−15.5 pp on Goal, −22.0 pp on LIBERO-10): predicting the whole KV is harder than predicting the per-step change from a representation already at 0.95 cosine.

  • Simpler learned predictors, trained offline on the protocol of Table 5 with only the predictor changed: with a delta output a per-token MLP without attention reaches 0.975 next-step KV cosine and a per-token linear map 0.969, against 0.976 for our DiT; the same DiT predicting the next KV directly reaches 0.968, stale reuse 0.952. The gain comes from predicting deltas from the fresh vision input, not from the architecture.
  • Reuse and extrapolation at matched f: the bridge is +40.5 pp over stale reuse on π0.5, and Taylor extrapolation reaches 46.67.

Adaptive refresh is complementary rather than the source of the gain: π0.5 uses a fixed f=4 without any adaptive schedule, and on GR00T LIBERO-10 the phase-aware rule adds +21.7 pp on top of the bridge.

Real-robot closed loop. A closed-loop study on a physical arm is now in Appendix L (videos in the supplementary material).

  • Hardware: an AgileX Piper 6-DoF arm with a parallel gripper, an overhead third-person RGB camera and a wrist RGB camera (480×640), controlled at 20 Hz with end-effector velocity actions. The policy runs on a remote GPU server; each call returns a 10-step chunk, of which the arm executes 5 steps (10 when it is stationary).
  • Policy: π0.5 fine-tuned on 39 teleoperated demonstrations of two tasks, "put the pen into the box" (24) and "take the pen out of the box and bring it back" (15), one checkpoint for both.
  • Bridge: R0 only, trained on KV computed by that checkpoint along 53 recorded robot rollouts (no simulator, no DAgger).
  • Protocol: same checkpoint, execution protocol and scene for all arms; 15 episodes per arm and task.
Armputtakebackbone call ratepolicy calls per successful episode (put / take)
Sync15/1510/15100%82 / 66
Stale reuse, f=46/152/1525–26%87 / 80
Bridge, f=414/1514/1525–26%73 / 67

The bridge stays at the sync level with a quarter of the backbone calls, and successful episodes are about as long. Stale reuse at the same schedule drops to 8/30, and 19 of its 22 failures are failed grasps: the policy either never commits to closing the gripper (12 of 30 episodes) or closes on features two or three calls old and rarely succeeds (1/7, against 7/11 when the features are at most one call old). The bridge grasps at any feature age (27/29). Held-out KV cosine at f=4 is 0.964–0.966 for the bridge against 0.933–0.934 for stale reuse, and the chunk displacement error is 7–8 mm against 13–14 mm. The bridge network takes 21 ms where the backbone pass it replaces takes 51 ms.

Generalization (Appendix G). The degradation in Appendix G is a bridge trained on one suite applied to another, a data-coverage effect of one-bridge-per-suite training; the bridge is tied to a checkpoint, not a task (on RoboCasa one bridge serves 24 tasks and retains 91.6% on held-out tasks when trained on 12). One π0.5 bridge trained on all four suites at once, with a quarter of the per-suite data per suite, reaches a mean of 96.3% at f=3 against 96.8 for the per-suite bridges and 96.2 for sync in the same run (95.5 vs 95.8 at f=4; same VLM calls per episode; per-suite table in Appendix G). Per-suite bridges retrained at the joint bridge's data budget score 94.0% on Goal and 92.8% on LIBERO-10 against 95.0% and 93.2% for the joint bridge. We do not claim transfer across checkpoints; a new checkpoint needs a new bridge.

Changes in the revised PDF.

  • Related Work: Leap-VLA, Acting While Understanding and AC²-VLA cited and positioned relative to the bridge.
  • Appendix G: one bridge for four suites (table at f=3 and f=4) and the per-suite control at equal data.
  • Appendix H: the faithful VLA-Cache port on π0.5 with its own table; the Table 1 footnote marks the Table 1 rows as the simplified port.
  • Limitations (Section 5): the one-off bridge cost is excluded from the net speedup; the edge-SoC measurement.
  • Related Work and Section 4.1: the description of SpecPrune-VLA corrected.
  • Figures 1–3 redrawn as vector figures in one style.
  • Appendix L: closed-loop real-robot study (sync / stale reuse / bridge at f=4, two tasks, 15 episodes each); videos in the supplementary material.

Dear Reviewer CW1G,

Thank you again for the constructive review.

As the discussion period closes today, we wanted to confirm that our rebuttal addresses your concerns. We are happy to expand any of the above if that would help the discussion. If we have resolved your issues, we sincerely hope that you can raise the rating score.

Best regards,

The authors

Reviewer_Uu9nOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the asymmetric latency bottleneck in dual-system Vision-Language-Action (VLA) models, where expensive Vision-Language Model (VLM) backbones must execute at every control step despite producing temporally redundant representations. To bypass this overhead, the authors propose Latent Bridge, a lightweight delta predictor (instantiated as a feature-space bridge for GROOT and a pre-RoPE KV-cache bridge for π0.5\pi_{0.5}) that forecasts feature or key-value deltas between timesteps. This allows the VLM backbone to run only periodically (f=2f = 2 to 88), while intermediate steps rely on autoregressively chained bridge predictions decoded by the action head. The authors introduce a task-agnostic three-stage training pipeline incorporating supervised sync data collection (R0R_0) and online DAgger refinement (R1R_1) to mitigate autoregressive distribution shift.

Critical strengths

Reviewer found the following strengths:

  • The paper demonstrates a framework across two structurally distinct dual-system VLA paradigms (the feature-vector DiT interface in GROOT and the interleaved per-layer KV-cache interface in 0.5), showing that latent delta prediction is a broadly applicable design pattern rather than a one-off hack.

  • The ideas of including DAgger refinement to correct the compounding errors of autoregressive chaining (t'2) are well-motivated and empirically proven to add significant performance gains (+3 to +13 pp). Furthermore, comprehensive metrics connecting open-loop feature/KV cosine fidelity, action RMSE, and closed-loop success rates provide deep diagnostic insight.

  • Regarding Efficiency: Latent Bridge achieves an impressive 1.55–1.73 net episode wall-clock speedup while retaining 95–100% of synchronous task success rates (SR) across four LIBERO suites, 24 RoboCasa kitchen tasks, and ALOHA sim.

Critical weaknesses

The paper stil has the following major weakness:

  • About DAgger Training Horizon and Distribution Shift: The authors collect DAgger data using a fixed VLM refresh period of f=3, while the trained bridge is evaluated across longer refresh periods and adaptive schedules. Since DAgger is introduced to address the distribution shift caused by autoregressive predictions, it remains unclear whether training with a fixed, relatively short prediction horizon adequately covers the errors encountered during longer prediction chains. An ablation comparing fixed-period and mixed-period DAgger training under a matched data budget would clarify the robustness of this training strategy.

  • Secondly, while Latent Bridge outperforms feature caching and Taylor extrapolation, it remains unclear how much of this improvement stems specifically from the proposed residual delta prediction architecture versus learned temporal feature prediction more generally. The paper lacks a comparison against a direct next-feature predictor trained under the same inputs, supervision, and DAgger procedure. Furthermore, the 19M-parameter bridge achieves nearly identical success rates to the 148M-parameter model (Table 3), raising questions about whether a heavy DiT architecture is necessary over simpler learned predictors.

  • Simulation-Only Closed-Loop Evaluation and Missing Real-World Generalization (critical) (this is one of the main weakness in my opinion): In particular, all closed-loop policy evaluations are performed entirely in simulation. While open-loop tests on real-world camera streams (DROID) verify feature- and action-level fidelity, whether minor per-step prediction errors compound catastrophically under chaotic real-world physical dynamics remains unmeasured.

  • The R1 refinement stage relies heavily on an active simulation environment with a digital twin to collect on-policy rollouts paired with VLM oracle targets. Additionally, the trained bridge is tightly bound to specific fine-tuned VLA checkpoints, meaning any base model updates require complete retraining from scratch. Finally, latency gains are benchmarked exclusively on a high-end enterprise A100 80GB GPU, leaving real-world wall-clock speedups on resource-constrained edge robotics hardware unverified, compounded by interleaved architecture execution floors like the 0.5 30 ms action expert bottleneck

  • Other points Authors seem to be abusing Gen-AI too much to generate the figure; this is very small and does not explicitly reflect the true components in the paper. Furthermore, the styles of different figures are significantly different, making the whole paper inconsistent.

Questions for authors

Q1. (Regarding DAgger Horizon & Distribution Shift): The authors collect DAgger training data using a fixed VLM refresh period of f=3f = 3, but evaluate the bridge across larger periods (up to f=12f = 12) and adaptive schedules. Could the authors clarify why f=3f = 3 was chosen? How does training with a fixed, short horizon cover error compounding during longer autoregressive chains, and how would mixing multiple refresh horizons (e.g., f=2,4,6f = 2, 4, 6) in the DAgger budget impact robustness?

Q2 (Regarding Residual Formulation vs. Direct Prediction & Scaling): It remains unclear how much performance gain stems specifically from residual delta prediction versus general learned temporal feature prediction. Could the authors provide an ablation comparing against a direct next-feature predictor under the same inputs and DAgger pipeline? Furthermore, given that the 19M-parameter variant matches the 148M model (Table 3), why is a heavy DiT architecture necessary over simpler lightweight predictors?

Q3 (Regarding Simulation-Only Closed-Loop Evaluation): While open-loop DROID stream tests provide helpful proxy metrics, all closed-loop policy rollouts are strictly simulation-bound. Do the authors have qualitative insights, hardware logs, or plans regarding how the small per-step action deviations compound under real-world contact physics, friction variations, and unmodeled hardware jitter?

Q4 (Regarding Deployment Bottlenecks and Edge Hardware Constraints): The R1R_1 stage requires an active simulator, and checkpoints are tightly coupled to specific VLA instances. How feasible is it to adapt this pipeline for zero-shot real-world deployment using purely offline R0R_0 data? Additionally, since latency gains were benchmarked exclusively on an enterprise A100 GPU, what are the expected net wall-clock speedups on memory-constrained edge hardware, particularly given interleaved execution floors like the π0.5\pi_{0.5} 30 ms action expert bottleneck?

3 comments

Dear Reviewer Uu9n,

Thank you for the detailed review and for catching the citation error. A revised PDF is uploaded together with this response (changes listed at the end of Part 2/2). We acted on every point:

  • Ethics flag. The DreamerV3 entry had the wrong venue. It is corrected, and every reference was checked.
  • W1/Q1 (DAgger at f=3). One checkpoint runs the whole sweep f=2–12, and the chained error grows smoothly with the chain length.
  • W2/Q2 (residual vs. direct; DiT). New ablation: direct prediction loses 11.3 pp of mean SR. A per-token MLP matches the DiT, and we now recommend the 19M bridge.
  • W3/Q3 (real robot). New closed-loop study on a physical arm: bridge 28/30, sync 25/30, stale reuse 8/30.
  • W4/Q4 (deployment). R0 needs no simulator, one bridge serves four suites, and the gain holds on an edge SoC (1.52–1.65× net).
  • Figures. All three were redrawn from scratch in one style.

Ethics flag: the DreamerV3 reference. The paper exists; the error was in the venue field. DreamerV3 is arXiv:2301.04104 (2023), published in Nature 640:647–653 (2025), not at ICLR 2023. We apologize for the error. We have since checked every bibliography entry against publisher and arXiv records: each is a real paper, and the revised PDF carries the verified list.

W1/Q1: DAgger at a fixed f=3. f=3 is the middle of the deployed range f=2–4, and DAgger has one purpose there: before it, the bridge has only seen clean VLM features as input, while at deployment its input from the second bridge step on is its own prediction. DAgger adds that input type to training. Changing f at inference changes the chain length, not the kind of input, and the accumulated error is what we measure:

  • chained KV cosine degrades smoothly, 0.998/0.996/0.994/0.989 at offsets 1/2/3/5, against 0.993/0.988/0.983/0.974 for stale reuse (Table 11);
  • one checkpoint runs the whole sweep f=2…12 (Figure 4, Table 12): Spatial, Object and Goal stay at or above 94.5% through f=8 (a chain of seven), and LIBERO-10 reaches 84.5% (sync 94.0%).

One trained checkpoint serving every operating point is a deliberate choice; a training period per inference period would make the method much harder to use. On mixing horizons: on GR00T LIBERO-10 a bridge trained on mixed f∈{2,3,4} rollouts did not outperform the f=3 bridge. A matched-budget comparison with longer periods is listed as a limitation.

W2/Q2: residual vs. direct prediction. We agree this ablation belongs in the main text and have added it (Tables 3 and 5). Same network, inputs, data and pipeline including the DAgger round; only the output changes:

π0.5, f=4SpatialObjectGoalLongMean SR
Residual delta (ours)99.0097.6797.3393.6796.92
Direct next-KV95.6793.1781.8371.6785.58
Stale reuse46.6757.3368.8352.6756.38

Predicting the whole KV is harder than predicting the per-step change from a representation already at 0.95 cosine, and the gap grows with task length (−15.5 pp on Goal, −22.0 pp on LIBERO-10).

W2/Q2 (continued): is a DiT necessary? A DiT is sufficient but not required. On the protocol of Table 5's KV-cosine column (same 19 training and 6 held-out episodes, optimizer and schedule) we trained further predictors with one script; only the predictor changes (Table 15, Appendix F):

Predictor, same inputsOutputParamsNext-step KV cosChained, f=4
DiT, 10 blocks (paper)delta148M0.9760.969
DiT, 3 blocksdelta19M0.9730.965
Per-token MLP, no attentiondelta28M0.9750.968
Per-token linear mapdelta19M0.9690.959
DiT, 10 blocksdirect148M0.9680.964
Stale reuse––0.9520.936

The output form matters more than the architecture: a per-token MLP with no token mixing reaches 0.975 against 0.976 for the DiT, and the DiT with a direct output falls to the level of the linear delta map. What carries the prediction is the fresh vision input with the delta form (removing the vision input costs 11.2 pp on Goal and 28.3 pp on LIBERO-10, Table 3). The table is open-loop; in closed loop the 19M bridge matches the 148M bridge in SR (Table 3), and the revision recommends it.

Figures. We agree and adopted the suggestion. All three figures were redrawn from scratch with matplotlib as vector graphics in one consistent style, built directly from the paper's components: Figure 1 shows the dual-system call with the Table 1 costs, the bridge timeline and retention vs. speedup; Figure 2 the inference timeline with the inputs of Eqs. 2, 3 and 5; Figure 3 the three training stages of Section 3.3. Every label names a component or quantity defined in the text.

W3/Q3: real-robot closed loop. We ran a closed-loop study on a physical arm (Appendix L; videos in the supplementary material).

  • Hardware: an AgileX Piper 6-DoF arm with a parallel gripper, an overhead third-person RGB camera and a wrist RGB camera (480×640), controlled at 20 Hz with end-effector velocity actions. The policy runs on a remote GPU server; each call returns a 10-step chunk, of which the arm executes 5 steps (10 when it is stationary).
  • Policy: π0.5 fine-tuned on 39 teleoperated demonstrations of two tasks, "put the pen into the box" (24) and "take the pen out of the box and bring it back" (15), one checkpoint for both.
  • Bridge: R0 only, trained on KV computed by that checkpoint along 53 recorded robot rollouts (no simulator, no DAgger).
  • Protocol: same checkpoint, execution protocol and scene for all arms; 15 episodes per arm and task.
Armputtakebackbone call ratepolicy calls per successful episode (put / take)
Sync15/1510/15100%82 / 66
Stale reuse, f=46/152/1525–26%87 / 80
Bridge, f=414/1514/1525–26%73 / 67

With three of every four backbone calls replaced by the bridge, success stays at the sync level (28/30 vs 25/30; the take-the-pen gap is within the variation of a 15-episode test) and successful episodes are about as long as under sync. The same schedule with stale reuse drops to 8/30, and 19 of its 22 failures are failed grasps: in 12 episodes the policy never commits to closing the gripper, and when it closes on features two or three calls old it rarely succeeds (1/7, against 7/11 when the features are at most one call old). The bridge grasps at any feature age (27/29); its held-out KV cosine at f=4 is 0.964–0.966 against 0.933–0.934 for stale reuse, and its chunk displacement error is 7–8 mm against 13–14 mm. The bridge network takes 21 ms where the backbone pass it replaces takes 51 ms.

W4/Q4: deployment.

  • Simulator. Neither stage needs one. R0 uses rollouts of the unmodified policy, which a robot produces by running it. DAgger on a robot: run the bridge policy, log the frames, and compute the VLM targets on the logged frames offline. We ran DAgger in simulation only. R0 alone retains at least 93% of sync on 7 of 8 suite × architecture configurations (Tables 1 and 4), except GR00T LIBERO-10.
  • Checkpoint coupling. A new checkpoint needs a new bridge (8–12 GPU-hours, Appendix A). Within a checkpoint the bridge is not tied to a task: on RoboCasa one bridge serves all 24 tasks and retains 91.6% on 12 held-out tasks when trained on the other 12. For LIBERO we have now trained one π0.5 bridge on all four suites at once with a quarter of the per-suite data per suite (8 instead of 30 sync episodes per task, same R0 recipe). In the same run as the per-suite bridges and sync (3 seeds × 20 episodes per task) its mean is 96.3% at f=3 against 96.8 for the per-suite bridges and 96.2 for sync (95.5 vs 95.8 at f=4), with the same number of VLM calls per episode. Per-suite bridges retrained at the joint bridge's budget score 94.0% on Goal and 92.8% on LIBERO-10 against 95.0% and 93.2% for the joint bridge, so one bridge for four suites costs nothing at equal data (Appendix G).
  • Hardware. The gain comes from skipping the backbone, so it scales with the backbone's share of a policy call: 46 of 76 ms for π0.5 and 63 of 90 ms for GR00T on the A100 (Table 1); the action expert costs the same in every variant and bounds the per-step gain at 1/(1−share) for any method that keeps it. On the GPU of a Qualcomm Dragonwing IQ-9075 evaluation kit (Snapdragon-class SoC, Adreno 663) a synchronous π0.5 call takes 213 ms with a 134 ms backbone (63%, against 61% on the A100) and a bridge step 91 ms (bridge network 12 ms): 1.62× per step at f=3 and 1.75× at f=4, 1.52× and 1.65× net. The share, and with it the gain, is the same on the edge device as on the A100.

Changes in the revised PDF.

  • Figures 1–3 redrawn from scratch in one consistent style, following the reviewer's suggestion.
  • Residual vs. direct prediction added to the main text (Table 3 row and Table 5).
  • Section 4.4: the 19M bridge recommended for deployment (it matches the 148M bridge, Table 3).
  • Limitations (Section 5): fixed-period DAgger without a matched-budget mixed-period comparison; the one-off bridge training cost; the edge-SoC latency measurement.
  • Appendix G: one bridge trained on all four suites, with the per-suite comparison at f=3 and f=4 and the same-budget control.
  • Reference list: the DreamerV3 entry corrected to its Nature publication; all entries re-checked.
  • Appendix L: closed-loop study on a physical arm (sync / stale reuse / bridge at f=4, two tasks, 15 episodes each); videos in the supplementary material.

Dear Reviewer Uu9n,

Thank you again for the constructive review.

As the discussion period closes today, we wanted to confirm that our rebuttal addresses your concerns. We are happy to expand any of the above if that would help the discussion. If we have resolved your issues, we sincerely hope that you can raise the rating score.

Best regards,

The authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised PDF, updated code and real-robot videos uploaded; every reviewer answered

Dear Reviewers,

We sincerely thank you for the time and care you put into reviewing our work. Your comments led to new experiments and a clearer paper. Detailed responses are posted under each review. Three things are now available:

  • Revised PDF. It contains every change described in the responses, and each response ends with the list of changes relevant to that reviewer. The main additions are a closed-loop study on a physical robot (Appendix L), the residual vs. direct prediction ablation (Tables 3 and 5), a faithful VLA-Cache port on π0.5 (Appendix H), one bridge trained on all four LIBERO suites (Appendix G), a scene-perturbation test (Appendix F), the net speedup now reported on all four settings (1.47–1.67×) and redrawn Figures 1–3.
  • Code. The anonymized supplementary archive is updated with the scripts behind every new result, including the real-robot pipeline.
  • Real-robot videos. The same archive contains 18 videos of the closed-loop real-robot study, three per method and task (sync, stale reuse, Latent Bridge), with a README giving the protocol and the success rates.

We are happy to answer any further questions during the discussion.

AuthorsOpenReview ↗

Revised PDF updated: all changes highlighted in blue

Following reviewerb SMrF's suggestion, the revised PDF now marks every change from the original submission in blue (new sentences, changed words, changed table cells, new tables and appendices). Figures 1–3 were redrawn and are not coloured.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.