acceptodds
Under review as a conference paper at ICLR 2027

Making Computation Count: Speculative State Recycling for Flow-Based VLAs

Abstract

Speculative inference accelerates flow-based vision-language-action models (Flow-VLAs) by bypassing vision-language model (VLM) prefill and full flow integration on speculative rounds. However, lightweight drafting and approximate verification can compromise action quality, while additional computation to improve reliability can erode the resulting speedup. Moreover, rejection can erase these savings by triggering full-path inference after the draft and verification costs have already been incurred. These limitations point to a broader inefficiency: speculative states are often discarded once they are no longer directly usable, even though they may still contain useful predictive information. We propose Speculative State Recycling (SSR), which improves this quality–efficiency trade-off by recycling previously computed states across changes in conditioning. Before refresh, retained deep VLM features serve as semantic memory for a shallow predictor, augmenting the current observation with prior high-level representations. The same shallow computation also produces a draft-independent Context Risk estimate of how much refreshing the conditioning would change the full model's actions, guiding execution and refresh. When refresh is needed, saved proposals initialize short fresh-conditioned Action Expert integration, reducing regeneration from noise. On LIBERO, SSR with Triton nearly matches the success rate of full-generation with compiled inference while achieving a speedup per executed action and 74.1% fewer full AE calls per executed action. On LIBERO-Plus, SSR improves mean success by 5.77 points over FLASH at only 0.15 ms/action additional cost. Our project website is available at https://ssracc.github.io/speculative-state-recycling.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, vision-language-action models, speculative inference, flow matching, real-time control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xECtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a speculative inference acceleration method, Speculative State Recycling (SSR), for flow-based VLA models, aiming to improve the trade-off between task success rate and inference efficiency in existing speculative inference methods. Built upon the FLASH framework, the method mainly addresses the underutilization of previously computed results during speculative inference and the potential inconsistency between cached VLM conditioning and current observations.

SSR consists of three main components: (1) Semantic Memory: Reuses deep features obtained from previous full VLM inference to assist a lightweight model in predicting current actions; (2) Context Risk: Predicts the differences between full-policy outputs under historical cached conditioning and the latest VLM conditioning, and combines these predictions with action verification results to dynamically determine the executable action prefix and VLM refresh timing; (3) Proposal Recovery: After speculative actions are rejected, uses the original action proposal to initialize short-step Action Expert integration under the latest conditioning, reducing the computation required to regenerate actions.

The authors conduct experiments based on the π0.5\pi_{0.5} model on LIBERO, LIBERO-Plus, and three real-world robotic manipulation tasks, and analyze the contributions of different components through ablation studies. Experimental results show that SSR largely maintains the task success rate of the full model on standard LIBERO while reducing the average inference time per action. Compared with FLASH, SSR achieves a higher average task success rate on LIBERO-Plus and demonstrates practical execution benefits from inference acceleration in real-world robotic tasks.

Critical strengths
  • The overall research motivation is clear, and the problem addressed is practically meaningful. The existing FLASH method reduces the number of full VLA inference calls through lightweight action prediction and verification. However, when an action proposal is rejected, the computation already spent on prediction and verification is difficult to reuse. In addition, cached VLM conditioning may not accurately reflect the latest observations. This paper approaches the problem from the perspective of computational reuse, attempting to make better use of historical VLM features and rejected action proposals to reduce inference costs.
  • Context Risk provides a meaningful mechanism for dynamically refreshing cached conditioning. Conventional action verification mainly determines whether predicted actions are consistent with Action Expert outputs under cached conditioning, but cannot directly reflect the discrepancy between cached conditioning and the latest observations. This paper predicts changes in full-policy outputs under historical and updated VLM conditioning to help determine the executable action prefix and cache refresh timing. Compared with fixed periodic refresh, this action-discrepancy-based mechanism is reasonably motivated and can share lightweight computation with action verification, avoiding excessive additional inference overhead.
  • The experimental analysis is relatively comprehensive. The authors evaluate SSR on standard LIBERO, LIBERO-Plus, and three real-world robotic tasks, comparing it with the full π0.5\pi_{0.5} model and FLASH. Experiments on standard LIBERO show that, under the same compiled inference backend, SSR reduces the average inference time per action compared with FLASH while maintaining a similar task success rate. The LIBERO-Plus experiments further demonstrate the performance advantages of the method under scene and instruction perturbations. The real-world robotic experiments also provide additional support for the practical value of improved inference efficiency.
Critical weaknesses
  • The technical novelty of individual components is relatively limited, particularly the strong similarity between Proposal Recovery and existing speculative verification mechanisms. Proposal Recovery uses rejected action proposals to initialize short-step flow integration, and its core operation is similar to the endpoint reconstruction in FLASH. Although the two use different conditioning information and follow different execution procedures, the independent technical contribution remains insufficiently distinctive. The authors are encouraged to further clarify the substantive innovations of each component relative to existing methods, particularly the differences between Proposal Recovery and existing endpoint reconstruction mechanisms.
  • The inference acceleration gains are relatively limited, and some results are affected by inference backend optimizations. The paper emphasizes that SSR can achieve up to approximately 1.89× inference acceleration. However, this result compares the full π0.5\pi_{0.5} model using a compiled inference backend with SSR optimized using Triton, and therefore does not fully reflect the acceleration attributable to SSR itself. Furthermore, on LIBERO-Plus, the average inference time increases from 4.38 ms/action for FLASH to 4.53 ms/action for SSR, indicating that the method does not simultaneously improve task success rate and inference efficiency in more challenging scenarios. The authors are encouraged to more clearly distinguish the contributions of algorithmic improvements and inference backend optimizations, and to analyze the actual acceleration gains under different task complexities and degrees of scene variation.
  • The prediction reliability of Context Risk and its impact on cache refresh decisions are not sufficiently analyzed. Context Risk predicts action discrepancies under historical and updated VLM conditioning to determine whether cached conditioning should continue to be reused. However, the paper mainly evaluates this component through training loss, validation error, and final task success rate, without systematically analyzing how prediction errors affect actual execution decisions. For example, when Context Risk underestimates changes in conditioning, the system may accept actions for which the conditioning should have been refreshed. Conversely, overestimating these changes may trigger unnecessary full inference, reducing the acceleration gains. The paper does not sufficiently analyze the frequency of such errors or the relationships among misclassification rates, task success rates, and inference costs under different Risk thresholds. The authors are encouraged to further evaluate the calibration of Context Risk predictions and the impact of incorrect refresh decisions on system performance.
Questions for authors
  • Insufficient comparison with simpler cache refresh strategies. The paper proposes Context Risk to predict action discrepancies under old and updated VLM conditioning, thereby dynamically determining when to refresh the cache. Could simpler indicators be used to decide whether to refresh, such as the feature difference between the current image and the image at the last refresh?
  • Additional training costs. SSR requires additional training for some components. What is the computational cost of this training?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xECt

We thank Reviewer xECt for the constructive review and for recognizing SSR’s clear motivation around computational reuse, the shared-computation design of Context Risk, and our comprehensive evaluation across standard LIBERO, LIBERO-Plus, and real-world robotic tasks. Below we address the distinct computational role of Proposal Recovery, same-backend speedups and per-suite trade-offs, Context Risk decision errors, comparison with image-feature indicators, and additional training costs.

[W1] Substantive advances within the framework and decoupling context refresh from full regeneration.

As work such as EAGLE (Li et al., 2024) and EAGLE-2 illustrates in LLM speculative decoding, establishing a draft–verify framework still leaves a central efficiency question: how much executed output is realized from the drafting and verification compute already spent. Jointly generated continuous action chunks and reusable conditioning create distinct opportunities in Flow-VLAs, around which SSR unifies deep-semantic reuse, action-aware refresh, and post-rejection proposal recycling (Eqs. 2 and 18).

Specifically, Proposal Recovery is motivated by the insight that updating visual-language conditioning need not require discarding valid action structure and restarting 10-step integration from pure noise. While both FLASH verification and Proposal Recovery apply short-step integration along the learned velocity field, their outputs serve fundamentally different decisions and execution flows: FLASH reconstructs endpoints under cached conditioning strictly as a verification reference for the draft, whereas SSR uses the saved proposal to initialize short fresh-conditioned integration whose output itself serves as an executable candidate. When an update exceeds the displacement bound, full AE10 generation reuses the same fresh KV without a second prefill.

Both design elements are directly supported empirically and conceptually:

  • Value of proposal initialization (Table 4): Under the same one-step AE budget, initializing from the saved proposal achieves 96.65% mean success versus 95.00% from pure noise; shifting initialization closer still (σ0\sigma_0 = 0.9) raises adoption to 98.4% but drops success to 95.50%, highlighting the balance between structural reuse and fresh adaptation.
  • Displacement gating and control continuity: In a controlled bound sweep on Goal-Plus holding other settings fixed, relaxing ρ\rho from 0.15 to 0.30 lowers success from 1887/2591 to 1723/2591 (-6.3 percentage points), showing that the gate effectively confines one-step recovery to local refinements while delegating large trajectory shifts to full generation. This also aligns with RTC (Black et al., 2025c), which shows that preserving compatible trajectory structure across updates helps prevent undesirable mode switches between consecutive action chunks.

[W2] Separating algorithmic and backend speedups and analyzing per-suite trade-offs.

Under the identical torch.compile backend on standard LIBERO, SSR lowers mean model time from 3.52 ms/action (FLASH) to 3.05 ms/action—a 1.44× algorithmic speedup over compiled full π₀.₅ (4.39 ms/action)—while improving mean success from 95.85% to 96.65% (Table 1). The 1.89× figure (2.32 ms/action) reflects adding Triton optimization on top of SSR, and we will distinguish algorithmic and backend contributions explicitly in the Abstract and Section 5.2.

On LIBERO-Plus, SSR improves four-suite mean success by +5.77 points (75.22% → 80.99%) as mean model time moves from 4.38 to 4.53 ms/action (Table 2). Per-suite results (Table 13) clarify how this trade-off varies across task complexity and perturbation types: on multi-stage Long-Plus, SSR improves both success (64.9% → 76.5%) and speed (3.79 → 3.74 ms/action), and Spatial-Plus likewise reduces model time (5.34 → 5.13 ms/action) at comparable success. On Goal-Plus and Object-Plus, stronger perturbations leave fewer proposals suitable for local one-step recovery, incurring modest fallback overhead in exchange for substantial accuracy gains.

AuthorsOpenReview ↗

Response to Reviewer xECt

[W3] From Context Risk prediction errors to closed-loop refresh and execution decisions.

Beyond the closed-loop comparison against periodic refresh on Goal-Plus in Figure 5(b), characterizing decision reliability requires linking threshold-level classification errors to their execution consequences: overestimating discrepancy (false positive) adds refresh overhead, whereas underestimating it (false negative) can delay immediate refresh at h=1h = 1, while misses at later positions (h>1h > 1) affect motion only if they survive Lexec=min⁡(LA,LC,mt)L_{\mathrm{exec}} = \min(L^{A}, L^{C}, m_t).

We evaluated paired fresh/stale full-policy outputs under identical robot state and flow noise (Eq. 5) across 795 held-out closed-loop states (7,950 horizon positions) on standard LIBERO Long.

At the deployed threshold, Context Risk maintains a low FPR of 11.0% across the horizon (548/4,974) and 11.3% for immediate refresh at h=1h = 1 (57/505), with a horizon-wide FNR of 35.4% (1,053/2,976). Tightening the immediate-refresh cutoff to 0.8λ0.8\lambda lowers first-step FNR to 11.7% (34/290).

Crucially, downstream execution further limits how these misses affect control. Prefix truncation (LAL^{A}, LCL^{C}) and refresh/fallback routing filter missed above-threshold positions before they reach execution:

Execution-level measureResult
Missed above-threshold positions blocked before execution68.2% (718/1,053)
Unblocked misses as a share of executed cached-draft actions in the evaluated states10.9% (335/3,067)

The misses that reach execution are mild near-boundary cases: their normalized discrepancy r∗/λr^{\ast}/\lambda has a median of 1.12 and a 90th percentile of 1.30. Moreover, 99.4% (333/335) are followed by a fresh VLM refresh; the median wait among these cases is 6 executed actions.

Across 50 paired Long episodes, the corresponding success and call costs are:

SettingSuccessPrefill/100 actionsAE10/100 actions
Risk on47/50 (94.0%)5.382.13
Risk off45/50 (90.0%)2.661.95

Enabling Risk yields higher success with more frequent refreshes, while full AE10 calls remain nearly unchanged: one-step recovery absorbs 91.3% (432/473) of recovery attempts.

[Q1] Why action-discrepancy prediction is preferable to image-feature distance.

Feature distance from the last refresh image is a lightweight heuristic, but Context Risk offers two advantages:

  • Control sensitivity: Background motion or lighting can change many visual features without affecting the grasp, while a small target–gripper offset near contact may require immediate correction. This distinction also motivates VLA-Cache’s selective recomputation of task-relevant tokens (Xu et al., 2025, Sec. 3.3).
  • Horizon-wise decisions: A frame-level scalar cannot locate where cached conditioning becomes unsuitable along the action horizon. Trained on per-step fresh/stale policy-output differences (Eqs. 6–7), Context Risk distinguishes immediate refresh from shortening an otherwise valid prefix, controlling both refresh timing and execution length through the shared shallow forward pass (Eq. 8), without extra visual encoding.

Empirical verification: On the same 795 held-out LIBERO Long states as W3, we compare Context Risk with mean and top-10% visual-patch cosine distances between the current and last refresh images. Thresholds are calibrated on a separate split to match the same immediate-refresh alarm budget (48/198).

Here, FNR (false-negative rate) is the fraction of states with above-threshold fresh/stale action discrepancy that are classified as not requiring immediate refresh; FPR (false-positive rate) is the fraction of states within the discrepancy threshold that are classified as requiring refresh. The test set contains 290 above-threshold and 505 within-threshold states. Context Risk improves over both visual indicators:

Visual indicator compared againstFNR reductionFPR reductionTest alarm-rate reduction
Mean patch cosine distance23.8 pp16.6 pp1.8 pp
Top-10% patch cosine distance22.1 pp17.2 pp2.9 pp

All reductions are in percentage points (pp), computed as the visual indicator’s rate minus Context Risk’s rate.

At a slightly lower test alarm rate (25.7% versus 27.5%/28.6%), Context Risk reduces FNR by 22–24 percentage points and FPR by approximately 60% relative to both visual indicators. This simultaneous reduction in misses and false alarms supports the greater selectivity of action-discrepancy supervision: the improvement comes from more effective refresh decisions, rather than simply refreshing more often.

AuthorsOpenReview ↗

Response to Reviewer xECt

[Q2] Additional training pipeline and computational cost.

SSR reuses standard demonstrations without additional data collection. Proposal Recovery directly uses the frozen teacher Action Expert and requires no additional training. The added computation has two stages:

  1. One-time offline precomputation: The frozen 3.62B π₀.₅ teacher caches input features, action targets, and same-noise fresh/stale action-discrepancy labels (Eq. 5). Goal label-mining logs record 760,596 historical/current pairs at 68.8 GPU-hours, summed across worker ranks.
  2. Lightweight shallow joint optimization: Only 211M parameters are trained: one shallow Gemma block, two query formers, and linear heads, with Action and Risk sharing the shallow forward pass. Optimization reads cached tensors; the full teacher, image encoder, deep VLM, and 10-step integration are absent from the training loop.

On a 512-frame Goal cache subset, we benchmarked 2× NVIDIA A40 with micro-batch 32/GPU, accumulation 2, and global batch 128, timing 100 optimizer steps after 20 warmup steps. These measurements are separate from the submission’s RTX 4090 experiments:

Training resourceResult
Time per optimizer step2.33 s
Throughput55.0 current frames/s
Peak allocated memory per GPU22.5 GiB
Estimated optimization cost for 48 epochs22.9 A40 GPU-hours (~11.4 h on 2 GPUs)

The optimization estimate scales to Goal’s 47,190 training frames over 48 epochs (selected epoch 47 with zero-based indexing, Table 7), excluding validation and checkpointing. Offline precomputation is a one-time cost, while subsequent optimization runs entirely on cached features and labels.

Reviewer_bZf2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the quality–efficiency trade-off in speculative inference for flow-based vision-language-action models. It proposes Speculative State Recycling (SSR), which reuses previously computed information in three ways: cached deep VLM features support a shallow action predictor; a draft-independent Context Risk predictor estimates how much refreshing the conditioning would change the full policy’s actions; and saved proposals initialize a short integration under fresh conditioning when a refresh is required. The method is evaluated on LIBERO, LIBERO-Plus, and three real-robot tasks, with additional component analyses. Under the same compiled backend, SSR improves LIBERO success from 95.85% to 96.65% over FLASH while reducing reported inference time from 3.52 to 3.05 ms per executed action. On LIBERO-Plus, it improves success with a small reported increase in inference cost.

Critical strengths
  1. I find the distinction between draft quality and context staleness useful. A draft can agree with the cached policy even when refreshed conditioning would lead to a different action. Context Risk targets this gap directly. This is a sensible addition to action verification when deciding how much of a proposal to execute.

  2. Table 1 shows a useful improvement over FLASH with compiled inference for both methods: success increases from 95.85% to 96.65%, while reported inference time falls from 3.52 to 3.05 ms per executed action. On LIBERO-Plus, the success gain is larger, at 5.77 percentage points, with a reported additional cost of 0.15 ms/action. The timed robot tasks also provide useful evidence of SSR’s practical benefit.

Critical weaknesses
  1. My main concern is the timing estimate for LIBERO-Plus Goal. Appendix B.9 combines current call densities with historical direct-full and recovery-trigger proportions. It is unclear how sensitive the reported mean LIBERO-Plus overhead of 0.15 ms/action is to this approximation. This estimate needs a check using statistics from the evaluated run. The headline 1.89× speedup also includes a backend change, although Table 1 still shows an improvement under the same compiled backend.

  2. Some Goal ablations change the Risk threshold along with the recovery setting. Appendix B.7 uses λ = 1.2 without recovery and λ = 1.0 with recovery; Appendix B.8 also changes λ across recovery configurations. Since λ affects refresh frequency, some of the differences in success and cost may come from the threshold. These comparisons would be easier to interpret if λ were held fixed.

Questions for authors
  1. Can you recompute the LIBERO-Plus Goal timing using routing statistics from the corresponding evaluation run and report whether the mean overhead of 0.15 ms/action changes? A direct measurement of model inference time, including rejected drafts, recovery attempts, and fallbacks, would also help check the estimate.

  2. On Goal, λ changes between the recovery-off and recovery-on settings. Does the benefit of recovery remain when the same λ is used for both?

  3. How is current proprioception handled when the π0.5 VLM cache is reused? Equations (1) and (5) show the current state s_t alongside cached conditioning K_τ. Does the cached prefix contain state-related tokens, and are these updated on speculative rounds or is the current state supplied separately?

2 comments
AuthorsOpenReview ↗

Response to Reviewer bZf2

We thank Reviewer bZf2 for the clear assessment and for highlighting the distinction between draft quality and context staleness, our same-backend gains on standard LIBERO, the robustness improvement on LIBERO-Plus, and the timed real-robot results. Below we address same-backend speedups alongside LIBERO-Plus trade-offs and timing provenance, fixed-λ\lambda comparisons for Proposal Recovery, and how current proprioception is handled when reusing the π₀.₅ VLM cache.

[W1 & Q1] Same-backend gains, per-suite trade-offs on LIBERO-Plus, and timing provenance.

  • 1) Separating algorithmic and backend speedups. Under the same torch.compile backend on standard LIBERO, full π₀.₅, FLASH, and SSR take 4.39, 3.52, and 3.05 ms per executed action, respectively (Table 1)—giving a 1.44× algorithmic speedup over compiled full π₀.₅ and a 13.4% reduction relative to FLASH. The headline 1.89× figure (2.32 ms/action) additionally incorporates Triton optimization; we will distinguish these two levels more clearly in the paper.

  • 2) LIBERO-Plus trade-offs and recovery. SSR raises mean success from 75.22% to 80.99% (+5.77 points), with model time moving from 4.38 to 4.53 ms/action (Table 2). On Long-Plus, success and speed improve together: 64.9% → 76.5% and 3.79 → 3.74 ms/action; Spatial-Plus also runs faster at comparable success (Table 13). Goal-Plus and Object-Plus gain robustness with some added compute. Recovery saves time when useful action structure survives a refresh; otherwise, a rejected one-step attempt adds a small overhead before full AE10 generation. We will clarify this per-suite trade-off in Section 5.2.

  • 3) Timing provenance on Goal-Plus. Model time combines calibrated path latencies with closed-loop call statistics (Appendix B.5). Goal-Plus uses measured prefill and AE10 call densities; only the split between direct-full and recovery-triggered sub-routes uses historical proportions (Appendix B.9). The other three Plus suites use complete statistics from their evaluated runs, and all success rates come directly from rollouts. This split has limited timing impact because one-step recovery contributes little to path cost. We will clarify its provenance and cross-check archived logs when available.

[W2 & Q2] Operating-point selection versus fixed-λ\lambda component ablations.

We apologize for mixing λ\lambda operating points in the Goal ablations. Suite-dependent action and discrepancy distributions mean that the same λ\lambda need not yield the same refresh budget. Our main comparisons therefore select competitive quality–cost points at comparable compute, while component ablations fix λ\lambda within each suite. Object, Spatial, and Long already follow this protocol; below we align Goal.

  • 1) Table 12 (Appendix B.8): fixed λ\lambda = 0.8. We replace the middle Goal row, inadvertently taken from the λ\lambda = 1.0 main-table setting, with the λ\lambda = 0.8 result already reported as Point A in Table 11:

    Recovery settingσ0\sigma_0SuccessPrefill/100AE10/100Adoption
    Whole0.896.4%5.963.6957.2%
    Prefix (Table 11, Point A)0.897.4%6.102.7084.7%
    Prefix0.997.0%6.042.2898.1%
  • 2) Table 10 (Appendix B.7): matched λ\lambda = 1.0 comparison. We add the λ\lambda = 1.0 no-recovery result alongside the main-table recovery-on setting, retaining the original λ\lambda = 1.2 point for context:

    Goal settingλ\lambdaSuccessPrefill/100Full-generated action share
    Recovery off (matched-λ\lambda comparison)1.097.0%4.6246.2%
    Recovery on (main-table default)1.097.2%4.9023.8%
    Recovery off (original Table 10 point)1.297.2%4.1541.5%

    At fixed λ\lambda = 1.0, recovery nearly halves the full-generated action share (46.2% → 23.8%) while maintaining comparable success and prefill rates. The original λ\lambda = 1.2 no-recovery row was selected for its stronger quality–cost trade-off: on near-saturated Goal, more full-policy actions do not necessarily improve success.

AuthorsOpenReview ↗

Response to Reviewer bZf2

[Q3] Routing of current proprioception under cached VLM conditioning.

Reusing KτK_{\tau} preserves visual-language conditioning without freezing the other policy inputs. In our LIBERO π₀.₅ configuration, the cached VLM prefix contains no state tokens: discrete state input is disabled, and the teacher Action Expert does not separately encode numerical state. Current proprioception follows a separate path: every speculative round, the joint shallow drafter/Risk predictor encodes the latest robot state into a dedicated token StS_t, alongside current observation features CtC_t in Eq. (8). Thus, reusing historical KV does not feed stale proprioception to the predictor.

Equations (1), (5), and (9) express the general conditional Flow-VLA interface, explicitly including the currently available state sts_t; whether a backbone consumes this argument depends on its architecture. Specifically, Eq. (5) defines offline Risk target generation: with current inputs and flow noise ϵt\epsilon_t held identical, we run full AE integration under historical KτK_{\tau} and fresh KtK_t. Only the conditioning cache changes, isolating its effect on policy outputs.

Reviewer_nRkbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose a speculative inference based VLA method that leverages drafting to bypass VLM prefill and full flow integrations. In particular, the authors propose SSR, which basically tries to reuse information from prior hidden state for better memory drafting and rejected draft proposals for shorter fresh-conditioned integration instead of full flow integration for better efficiency. A context risk prediction mechanism is constructed to dynamically indicate when a refresh is needed. Experiments are conducted on both sim and latency-sensitive real world VLA tasks.

Critical strengths
  • The method is well motivated, with reasonable designs for reusing VLM features, deciding when to refresh, and recovering rejected proposals. The background and related-work discussion are informative and help explain how the approach builds on FLASH.

  • The experiments show improvements over FLASH in simulation, particularly in success rate on LIBERO-Plus. The three real-world tasks are also a strength, demonstrating practical benefits over full-generation π₀.₅ when latency affects task performance.

  • The paper includes useful ablations of its main components and design choices. It is generally well organized, though some baseline configurations and execution rules take effort to understand.

Critical weaknesses
  • From the paper, it is clear that FLASH is a major method this work is built on and comparing with. The contribution appears somewhat incremental over FLASH. SSR retains FLASH’s main speculative inference pipeline and adds memory-conditioned drafting, learned refresh decisions, and proposal recovery. These are reasonable extensions, but the evidence for a substantial advance is mixed: standard LIBERO gains are modest (+0.8 points in success and 13.4% lower inference cost), while the stronger LIBERO-Plus improvement comes with slightly higher cost. The real-world experiments compare only against π₀.₅, leaving unclear how much benefit SSR offers over FLASH in practice.

  • The reliability of refresh and recovery decisions is not sufficiently examined. Context Risk is evaluated through regression error, but the paper does not report how often it misses discrepancies above the deployed threshold. Recovery introduces another concern: a one-step update is accepted when it stays close to the rejected draft, which does not establish that it has incorporated enough fresh information. Comparisons with full generation at the same refresh states, especially when substantial action changes are needed, would help resolve this.

  • The isolated benefit of memory drafting remains unclear. The paper does not specify whether the FLASH drafter was trained with the same data, teacher checkpoint, and training budget as SSR’s drafter. Consequently, the FLASH versus “+ PF2” comparison cannot conclusively attribute the gains to memory. Removing memory from a model trained to use it is informative, but does not replace a separately trained, matched no-memory baseline.

Questions for authors

Was the FLASH drafter trained with the same data, teacher checkpoint, and training budget as SSR’s drafter? If not, could the authors provide a matched comparison to isolate the benefit of memory?

Could the authors report false-negative and false-positive rates for Context Risk at the deployed thresholds, especially on Long and LIBERO-Plus? Average regression error does not show how often a needed refresh is missed.

Could the authors move a target mid-episode in LIBERO and compare FLASH, SSR, and SSR without recovery? Existing checkpoints could be reused. Choose changes that full-generation π₀.₅ can handle, then measure refresh delay and task success to test whether one-step recovery adapts sufficiently. I notice that recent benchmarks such as LIBERO-Pro2.0 and LIBERO-Dynamic could help with this. They appeared very recently, so their omission is not a criticism—just a suggestion for running the experiment.

Could you include FLASH in the real-world comparisons under the same inference and execution settings? The current results do not separate SSR’s contribution from the general benefit of faster inference over π₀.₅.

2 comments
AuthorsOpenReview ↗

Response to Reviewer nRkb

We thank Reviewer nRkb for the thoughtful review and for recognizing SSR’s clear motivation, component designs, simulation gains on LIBERO-Plus, and practical benefits on latency-sensitive real-robot tasks. Below we address our contribution beyond FLASH, the real-robot evaluation setup, the operating boundaries of refresh and recovery, drafter training fairness, and mid-episode target changes in our real-world tasks.

[W1] Improving executable action yield within speculative Flow-VLA inference.

As work such as EAGLE (Li et al., 2024) and EAGLE-2 has shown in LLM speculative decoding, establishing the draft–verify paradigm leaves a central efficiency question open: how much accepted output each round of drafting and verification actually yields. In Flow-VLAs, where continuous action chunks are generated jointly under reusable visual-language conditioning, cheap speculative rounds alone do not guarantee low cost per executed action—short accepted prefixes and post-rejection fallback to 10-step flow integration (AE10) can erode the savings (in our same-backend FLASH baseline, full AE10 still supplies 56.2% of executed actions). Rather than combining isolated add-ons, SSR unifies state reuse around executable action yield per unit of complete inference cost (Eqs. 2 and 18): retained deep semantics strengthen lightweight drafting, predicted action discrepancies govern how far cached context can be reused, and rejected proposals initialize short fresh-conditioned integration when refresh is required.

Under the same torch.compile backend on standard LIBERO, SSR cuts the full-AE10 share of executed actions from 56.2% to 24.8% and reduces mean model time from 3.52 to 3.05 ms/action relative to FLASH, while nudging mean success from 95.85% to 96.65% (Table 1). Because most standard LIBERO suites are already near the 96.9% full-policy ceiling, headroom for higher success is naturally small, whereas the same-backend efficiency gain is clear.

On LIBERO-Plus, the quality benefit becomes much more pronounced: four-suite mean success rises by +5.77 points (75.22% → 80.99%) at 4.53 versus 4.38 ms/action (Table 2). Moreover, this is not uniformly a trade of extra compute for accuracy—on Long-Plus, SSR boosts success from 64.9% to 76.5% while slightly lowering model time from 3.79 to 3.74 ms/action (Table 13). Where stronger perturbations cause more recovery attempts to be rejected in favor of full generation, SSR incurs a modest time overhead in exchange for substantial robustness gains.

[W1 & Q4] Real-robot setup and practical observations with FLASH.

First, we note that both full-generation π₀.₅ and SSR in Table 5 run under identical synchronous execution on the same RTX 4090 GPU, and the baseline full π₀.₅ policy already uses Triton inference (Appendix C). Thus, the real-world performance gains do not stem from backend differences.

Prior to formal benchmarking, we also tested FLASH on the same physical setup using the same fine-tuned π₀.₅ teachers. Because synchronous execution holds the arm stationary during each inference call, smooth task progress depends heavily on the length of verified prefixes returned per call. In practice, FLASH’s memoryless shallow drafter frequently yielded very short verified prefixes, introducing repeated motion pauses that struggled to complete our fast-moving and time-limited tasks reliably; we therefore focused the formal 20-trial-per-condition evaluation on full π₀.₅ and SSR.

[Q3] Closed-loop evidence under mid-episode target changes.

Thank you for raising the question of adapting to target changes mid-episode. Notably, our real-world evaluation was specifically designed around intra-episode target dynamics rather than static initial perturbations alone:

  • Abrupt mid-episode target switches (whack-a-mole): Within each continuous 30-second episode, illuminated targets switch across spatial locations repeatedly (20 appearances per game at high speed and 15 at low speed, across 20 games each). Every newly lit button acts as an abrupt mid-episode target shift requiring timely visual refresh and trajectory redirection, where SSR improves pooled hit rates over full π₀.₅ from 19.5% to 25.8% (high speed) and from 55.7% to 71.3% (low speed; Table 5).
  • Continuous intra-episode target motion (conveyor grasping): As the target moves continuously during execution, SSR achieves 65.0% and 20.0% success at medium and high belt speeds versus 35.0% and 0.0% for full π₀.₅.

Together with the project videos, these results demonstrate closed-loop responsiveness when targets shift or move mid-execution.

AuthorsOpenReview ↗

Response to Reviewer nRkb

[W2 & Q2] Operating boundaries of Context Risk and reliability of one-step recovery.

  • 1) Decision boundaries and execution filtering of Context Risk. Beyond Figure 5(b) on Goal-Plus, we evaluated paired fresh/stale full-policy outputs under identical robot state and flow noise (Eq. 5) across 795 held-out closed-loop states (7,950 horizon positions) on standard LIBERO Long.

    At the deployed threshold, Context Risk maintains a low FPR of 11.0% across the horizon (548/4,974) and 11.3% for immediate refresh at h=1h = 1 (57/505), with a horizon-wide FNR of 35.4% (1,053/2,976). Tightening the immediate-refresh cutoff to 0.8λ0.8\lambda lowers first-step FNR to 11.7% (34/290).

    Crucially, downstream execution further limits how these misses affect control. Prefix truncation (LAL^{A}, LCL^{C}) and refresh/fallback routing filter missed above-threshold positions before they reach execution:

    Execution-level measureResult
    Missed above-threshold positions blocked before execution68.2% (718/1,053)
    Unblocked misses as a share of executed cached-draft actions in the evaluated states10.9% (335/3,067)

    The misses that reach execution are mild near-boundary cases: their normalized discrepancy r∗/λr^{\ast}/\lambda has a median of 1.12 and a 90th percentile of 1.30. Moreover, 99.4% (333/335) are followed by a fresh VLM refresh; the median wait among these cases is 6 executed actions.

  • 2) One-step recovery and the displacement gate. Requiring a context refresh does not imply that a rejected proposal has lost all useful approach geometry and temporal structure. SSR uses that structure to initialize a one-step update under fresh conditioning and checks displacement before adoption (Eq. 10): local refinements are executed cheaply, whereas larger shifts exceeding ρ\rho fall back to full 10-step AE10 generation reusing the same fresh KV without a second prefill.

    Preserving valid action structure is also desirable for control continuity. As highlighted in RTC (Black et al., 2025c), mode switches and discontinuities across consecutive chunks degrade physical motion; retaining compatible trajectory structure while incorporating fresh observations is therefore well motivated, without requiring pointwise identity to an independently sampled full trajectory. At the same time, bounding displacement is essential: in a controlled bound sweep on Goal-Plus holding all other settings fixed, relaxing ρ\rho from 0.15 to 0.30 substantially increases recovery adoption yet lowers success from 1887/2591 to 1723/2591 (-6.3 percentage points), confirming the practical role of the gate.

[W3 & Q1] Training fairness and the benefit of semantic memory.

FLASH and SSR use the same frozen π₀.₅ teacher, train/validation splits, training settings, and checkpoint-selection criterion. As specified in Appendix A.3 (Eqs. 15–17), each current frame receives one teacher action target per epoch, paired with one historical cache; historical resampling does not increase Action supervision. Risk gradients update only Risk-specific parameters, providing no auxiliary gradient to the shared block or Action drafter.

We distinguish inference-time memory removal (Figure 4b; Appendix B.6) from comparison with an independently trained memoryless baseline. Our comparison with independently trained FLASH, under matched routing without Risk or recovery, shows:

  • Draft-only control on Goal (Figure 4b): At memory refresh intervals of 10/20/40 actions, SSR achieves 88.6%/87.2%/89.2% success versus 57.0% for FLASH.
  • Four-suite evaluation with fixed PF2 refresh (Table 3): Semantic memory improves mean success from 95.85% to 96.15%, reduces the full-generated action share from 56.2% to 49.9%, and lowers model time from 3.52 to 3.31 ms/action.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.