acceptodds
Under review as a conference paper at ICLR 2027

Why Plan From Scratch? Accelerating Diffusion VLAs Via Trajectory Recycling

Abstract

Diffusion-based Vision-Language-Action (dVLA) models are a powerful paradigm for continuous robotic control, but their real-world deployment is bottlenecked by slow inference driven by iterative diffusion processes and massive multimodal backbones. Current acceleration methods naively discard unexecuted tail-end actions during dynamic re-planning. This squanders prior computational effort and repeatedly invokes the heavy dVLA backbone, bounding achievable latency reduction. We propose WPFS (Why Plan From Scratch?), a framework that accelerates dVLAs by repurposing discarded action chunks as structural priors. WPFS introduces a specialized Corrector that refines these priors into valid trajectories using only a fraction of the original denoising steps. We instantiate two variants: WPFS-Large, which leverages the base model for refinement, and WPFS-Small, which utilizes a distilled model to bypass the massive perceptual backbone entirely. To ensure reliability, an uncertainty-aware self-verification mechanism quantifies decision disagreement in parallel, dynamically triggering a full-scale fallback in complex scenarios without brittle threshold tuning. Evaluated across state-of-the-art dVLAs on simulation benchmarks (LIBERO, SimplerEnv) and real-world robots, WPFS achieves extreme latency reductions while maintaining or improving task success rates and motion smoothness. Code is available at https://anonymous.4open.science/r/WPFS-FFDD.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Diffusion Policies, Vision-Language-Action Models, Inference Acceleration, Action Chunking, Trajectory Refinement, Adaptive Computation, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wzMyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes WPFS, an inference acceleration framework for Diffusion VLA based on historical action trajectory reuse, aimed at reducing the computational overhead caused by repetitive planning. The method reuses unexecuted action chunks from previous iterations to generate new action trajectories via few-step denoising refinement, dynamically adjusts the execution horizon based on candidate trajectory consistency, and falls back to full planning when necessary. Furthermore, the authors introduce a lightweight Corrector that bypasses the large VLM backbone through knowledge distillation, further cutting inference costs. Extensive experiments conducted on pi0, pi0.5, and DB-OFT models across the LIBERO and SimplerEnv simulation benchmarks, as well as real-robot tasks, demonstrate that the proposed method achieves substantial inference speedups while maintaining or improving task success rates, with notable performance gains in dynamic real-world grasping tasks.

Critical strengths

The proposed method demonstrates a certain degree of novelty and practical value. By reusing historical Action Chunks and combining few-step denoising refinement with a dynamic execution mechanism based on candidate trajectory consistency, the method reduces the frequency of full VLA replanning. Furthermore, by distilling a lightweight Corrector, it avoids repeatedly invoking the large VLM backbone during trajectory refinement, further reducing inference overhead. Overall, the proposed method achieves a reasonable balance between inference efficiency and control performance.

Critical weaknesses
  1. Strong Dependence on Historical Action Chunks. WPFS relies on reusing unexecuted action trajectories from previous planning cycles. Although the authors investigate different execution horizons in Appendix B.1, the experiments do not directly evaluate how reduced Action Chunk overlap affects trajectory refinement. When the execution horizon approaches the Action Chunk length, the remaining historical trajectory may provide less useful information for initialization. The effectiveness of trajectory recycling under such settings remains unclear.

  2. Limited Coverage of Real-World Robotic Experiments and Real-Time Control Evaluation. Although the authors conduct real-world robotic experiments, the evaluation is limited to a single robotic platform and primarily considers static grasping and dynamic target grasping at two constant velocities. More challenging dynamic scenarios, such as sudden changes in target direction, temporary occlusions during target interception, or unexpected environmental disturbances, are not evaluated. Therefore, the robustness of WPFS in more complex dynamic environments remains insufficiently validated.

  3. Insufficient Isolation of the Contribution of Self-Verification. In Appendix B.2, removing Self-Verification also changes the fallback behavior and lineage constraints. Therefore, the performance differences cannot be attributed solely to the verification mechanism. A more controlled comparison under identical replanning constraints would help clarify the independent contribution of Self-Verification.

Questions for authors
  1. Regarding the Impact of Action Chunk Overlap: Although Appendix B.1 investigates different execution horizons, the impact of reduced Action Chunk overlap on trajectory recycling remains unclear. Could the authors further evaluate trajectory refinement performance, fallback frequency, and overall speedup under low-overlap settings to validate the applicability of the method across different execution strategies?

  2. Regarding Real-Time Performance on Physical Robots: Although WPFS significantly reduces the average inference time per action, triggering full VLA replanning may still introduce control delays. Could the authors further report the fallback frequency, full replanning latency, and actual control frequency during real-world robotic experiments to clarify how these delays affect real-time control and task execution performance?

  3. Regarding the Reliability of Self-Verification: The authors acknowledge that candidate trajectory agreement does not necessarily guarantee correctness. Could the authors further quantify cases where candidate trajectories agree but produce incorrect actions, and analyze how such cases affect execution length selection and the fallback mechanism?

9 comments
AuthorsOpenReview ↗

Response to Reviewer wzMy

We thank the reviewer for a careful review. We ran every requested evaluation. The revised PDF has been uploaded: all changes are in blue, and the new results are in Table 2 and Appendices B.2, B.5, B.6 and E (Tables R1 and R3–R6).

Weakness 2 / Question 2: Harder dynamics and real-time control (Table 2, Table R3, App. E).

  • The three scenarios the reviewer suggested (50 trials each): WPFS-Small raises success from 22% to 96% under a sudden direction change and from 24% to 80% under a temporary occlusion (p<10−7p<10^{-7} for both), and from 68% to 76% when a resting cube is pulled away.
  • Full-plan latency: a full plan is the same model call in every method (median 93–94 ms); WPFS makes plans 4.5× rarer (4.5 → 1.0 per second).
  • Fallback frequency: 0.9–3.1% of WPFS-Small's corrections and at most 3.2% of WPFS-Large's, costing ≤1.1% and ≤1.4% of control time.
  • Control frequency (synchronous): 33.1 Hz for WPFS-Small (stalled 17% of the time) against 22.6 Hz for the base policy (44%); median observation-to-command time 24 vs 93 ms (77 vs 121 ms asynchronously).
  • Task execution: an asynchronous executor cuts time stalled to ≤3% and helps the base policy (5 cm/s: 28% → 50%; 6 cm/s: 0% → 10%), but WPFS-Small still leads by 48 and 42 points (p≤2×10−5p\le2\times10^{-5}). WPFS-Large, whose observations are then as old as the base policy's, also leads (86% vs 50%; 44% vs 10%), suggesting that realigning the previous chunk matters beyond timing.

Weakness 1 / Question 1: Low chunk overlap (Table R4, App. B.5).

  • π0.5\pi_{0.5}, low overlap by default: its baseline executes 5 of 10 actions, and 63–71% of corrections start from an exhausted tail (the held last command). WPFS keeps the baseline's success on every suite at 1.83× (Large) and 3.91× (Small), with fallbacks on 14–25% of corrections. Over a 10-action chunk, the next plan stays close to the held command, so the held command serves as a prior on its own.
  • Controlled sweep on π0\pi_0, keeping only the first H′H' actions of each chunk: over π0\pi_0's 50-action chunk, the recycled tail matters. At matched guards, removing it doubles fallbacks (13.2% → 26.8%) and lowers WPFS-Large's speedup from 0.97× to 0.78×, while success goes from 96.0% to 93.4%. Both rows with ℓmax⁡=10\ell_{\max}=10 stay near or below 1× because a teacher correction (0.69 of a plan) then executes no more actions than a plan; a student correction costs 0.16 of a plan.
  • Fallbacks are most frequent where the prior has gone stale (10% → 34% of corrections on Spatial).

Weakness 3: Agreement test under identical replanning constraints (Table R1, App. B.2). With Λ\Lambda and its forced re-plans kept identical, the test adds 6.4 points at the same cap (89.2% → 95.6%). On Spatial, WPFS-Large is 6–23% faster than fixed schedules within one episode of its success, and on Long no fixed schedule reaches WPFS's success with either corrector. For WPFS-Small, a tuned fixed schedule reaches a similar operating point on Spatial, so its speedup stems mainly from recycling and distillation; the test removes the need to tune the segment per suite.

Question 3: Drafts that agree but are wrong (Tables R5–R6, App. B.6). Individual actions have no ground-truth label, so our proxy for an incorrect action is a draft that leaves the policy's own plan distribution: at every correction we draw 8 fresh plans, hold one out as an honest sample, and call a draft off-distribution when its first 5 actions are farther from the other 7 plans than for 95% of honest samples.

  • How often: accepted drafts of WPFS-Large are off-distribution about as often as the honest plan in the same corrections (3.6% vs 3.6% on Spatial; 2.6% vs 2.3% on Long), those of WPFS-Small 6.5% vs 3.7%; rejected drafts 15–25%.
  • Execution length and fallback: when an off-distribution draft is accepted, the test executes a median of 8–13 actions instead of 21–25, the next correction is rejected 1.8–2.2× as often, and the next rejection comes after 22–25 steps instead of 60–70.
  • Shared wrong prior: when the tail is replaced in 25% of corrections by another task's plan, one correction brings it from 2.98 to 0.97 times an honest plan's distance, accepted drafts are not significantly more often off-distribution (4.0% vs 3.3%), and success is unchanged (95.4%, against 95.6% when all priors are clean). As the reviewer suspects, drafts sharing a wrong prior can agree; here they agree on a repaired action.

The review found no substantive concerns about soundness, and its three weaknesses asked for evaluations that are now in the paper. If these address the concerns behind the rating, we would be grateful if the reviewer would consider raising it. We would welcome any further questions.

(Edited 13 Nov to match the revised PDF.)

AuthorsOpenReview ↗

Follow-up: additions to the paper since our response

Dear Reviewer wzMy,

A brief follow-up, as the discussion closes on 18 November. Since we posted our response on 12 November, the PDF (last updated 15 November) has gained two further baselines on all three backbones, RTI-DP and STEP (Table 1; Tables R10–R11), and experiments on deliberately wrong priors, sudden goal changes and draft selection (Tables R6–R9). In Table 1, both WPFS variants are more accurate than RTI-DP and STEP on every backbone.

The results in our response above are unchanged, and each evaluation the review asked for is in the current version: low chunk overlap (Table R4), harder dynamics and real-time control (Table 2, Table R3), the agreement test under identical replanning constraints (Table R1), and drafts that agree but are wrong (Tables R5–R6).

We would be grateful to know whether these address your concerns, or what is still missing. We will reply promptly and can run a further analysis in the time that remains.

Thank you again for your time.

Thank you for the detailed response and additional experiments. However, after carefully considering the rebuttal and the revised manuscript, my overall assessment remains unchanged. Therefore, I will maintain my original rating.

Reviewer_fdxoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WPFS reduces full diffusion VLA inference through historical trajectory reuse, observation-conditioned lightweight correction, and agreement-based scheduling. Experiments on LIBERO, SimplerEnv, and real robots demonstrate acceleration benefits. The approach has practical deployment value, but the limitations of historical priors, decision-making among multimodal candidates, and improvements over prior work require clarification.

Critical strengths
  1. Important problem and clear motivation. Exploiting trajectory redundancy across consecutive control cycles to reduce repeated computation is well motivated for real-time robot control.

  2. Complete system design. The corrector, adaptive execution, and full replanning form an integrated framework, with efficiency gains demonstrated across multiple backbones and public benchmarks.

Critical weaknesses
  1. The usefulness of historical priors is insufficiently established. Conditioning correction on current observations does not necessarily overcome the constraints imposed by an incorrect historical trajectory. When the search center is far from a high-quality solution, the prior may restrict exploration or cause negative transfer. The theory and experiments do not sufficiently establish when correction can transition between modes or when reuse should be abandoned. Moreover, candidates sharing the same incorrect prior may exhibit low disagreement and fail to trigger replanning.

  2. Valid action multimodality is not adequately addressed. Candidate disagreement may reflect multiple valid strategies rather than unreliable predictions. The method executes the accepted prefix of the first candidate without task-value-based selection. Consequently, the agreement mechanism may reject valid multimodal predictions, while there is no evidence that the first candidate offers higher quality. Global optimality is unnecessary, but the rationale for this selection rule should be established.

  3. Novelty and component attribution remain insufficiently supported. Published methods such as RTI-DP, STEP (https://proceedings.mlr.press/v306/li26aa.html, https://ieeexplore.ieee.org/document/11247391/) and Shallow-π: Knowledge Distillation for Flow-Based VLA (https://2026.ieee-iros.org/program/awards/) already explore trajectory reuse and warm-starting, yet direct comparisons are missing. T

Questions for authors
  1. When the historical trajectory is far from a valid current solution, under what conditions is its prior beneficial or harmful? Could experiments with deliberately incorrect historical trajectories or sudden goal changes compare reuse against generation from scratch?

  2. How does the method distinguish disagreement arising from multiple valid modes from disagreement caused by failed correction? Why execute the first candidate? How would random selection, candidate scoring, or within-mode agreement checking affect performance?

  3. Could you provide fair comparisons with RTI-DP, STEP, and Shallow Pi? If the comparison is not appropriate, please justify.

7 comments
AuthorsOpenReview ↗

Response to Reviewer fdxo (Part 1/2)

We thank the reviewer for a careful review. We ran the experiments the review proposes (deliberately wrong priors, sudden goal changes, random selection, candidate scoring and within-mode agreement) and added RTI-DP as a baseline in Table 1. The revised PDF marks all changes in blue; the new results are in Table 1 (RTI-DP) and Tables R6–R10 (Appendices B.4, B.6, B.7, B.8 and D). Part 2 answers Weakness 3.

Weakness 1 / Question 1: When does the prior help or hurt?

  • Theory. By Proposition 1, the one-step correction is the current plan distribution reweighted by a Gaussian kernel of bandwidth rτ0r_{\tau_0} around the prior, so the prior only reweights actions the current observation supports. Corollary 3 gives the two failure modes: as rτ0→0r_{\tau_0}\to0 a wrong prior carries over, and as rτ0→∞r_{\tau_0}\to\infty valid modes are averaged. We re-enter at τ0=0.5\tau_0=0.5 (rτ0=1r_{\tau_0}=1).
  • Where the prior helps (Table R4b). On π0\pi_0, removing the recycled tail at matched guards doubles fallbacks (13.2% → 26.8%) and lowers success from 96.0% to 93.4% and the speedup from 0.97× to 0.78×.
  • Deliberately wrong priors (Table R6; π0\pi_0 Spatial, both correctors, 500 episodes per run). In 25% of corrections, the tail is replaced by another task's plan or shifted in pose by twice the plan-to-plan spread. One correction brings it from 3–5× an honest plan's distance back to the level of a clean prior (0.93–0.97 for WPFS-Large, 1.15–1.22 for WPFS-Small). On their first 5 actions, executed drafts are not significantly more often off-distribution than with clean priors (p≥0.11p\ge0.11), and success stays within 0.6 points of the clean run (95.6%). As the reviewer suspects, drafts sharing a wrong prior agree (rejections change by at most 1.6 points), but on the repaired action.
  • Where the prior hurts: too shallow a re-entry (Table R7). At τ0=0.3\tau_0=0.3, the correction cannot leave a wrong prior (score 2.90 → 1.45), and success falls to 90.6%, or 85.2% with wrong priors (p<10−7p<10^{-7}): the small-bandwidth failure of Corollary 3. In STEP's terms, the previous chunk is temporally but not spatially consistent: starting the solver from it, as RTI-DP does, fails on π0\pi_0 (0.2–4.4%), while re-noising it to τ0=0.5\tau_0=0.5 recovers 94.2% without a trained predictor (Table R10).
  • Sudden goal changes (Table R8; π0\pi_0, LIBERO-Goal, three bowl destinations, 150 episodes per cell). The instruction switches at step TT. Without a signal, WPFS matches the baseline, which plans from scratch every 10 actions (p≥0.2p\ge0.2), so reusing the old goal's prior does not lower success. With one fresh plan at the switch for every method, WPFS matches or exceeds the baseline (48.0% vs 34.0% for WPFS-Small at T=75T=75). On the robot, WPFS-Small raises success from 22% to 96% under a sudden direction change and from 24% to 80% under occlusion (Table 2).
  • Reuse is abandoned on a rejection (the drafts split within the first ℓmin⁡\ell_{\min} actions) and after Λ=50\Lambda=50 steps; a new instruction can also trigger a fresh plan, as in the signalled runs.

Weakness 2 / Question 2: Multimodality and the first draft (Table R9, App. B.8).

  • The two kinds of disagreement. The test does not distinguish them: any split beyond the policy's own spread ends the prefix. It does remove the drafts most likely to be wrong (rejected drafts are off-distribution in 15–25% of cases, accepted ones in 3–7%; Table R5a), and rejecting a valid split costs speed, not success, since a fresh plan of the policy replaces it.
  • Why the first draft. The KK drafts share the prior and observation and differ only in noise, so they are exchangeable and executing the first is random selection. One draft is one output of the corrector, which lies within the spread of fresh plans (App. A.4, B.6), whereas averaging could blend modes. Section 3.3 now states this.
  • Selection rules (π0\pi_0 Spatial, same initial states). A random draft, the per-position median, the medoid (a value-free candidate score) or the draft closest to the prior change success by at most 1.4 points (p≥0.39p\ge0.39), at the same speed.
  • Within-mode agreement (the test applied within the largest group that agrees on the first 5 actions, if it holds at least half of the drafts). Rejections fall from 18.5% to 2.5% (WPFS-Large) and from 18.7% to 3.3% (WPFS-Small), and success is 1.2 and 0.8 points lower (p≥0.47p\ge0.47). The reviewer's suggestion thus gives a faster variant (1.61× → 1.79×; 2.86× → 3.39×). We keep the stricter default, which defers to a fresh plan whenever the drafts split, and report the faster variant.
AuthorsOpenReview ↗

Response to Reviewer fdxo (Part 2/2)

(Edited 15 Nov: when this response was posted, STEP had not yet been run. We have since run it on all three backbones and added it to Table 1; see our follow-up. The STEP paragraph below is our earlier positioning from its published numbers.)

Weakness 3 / Question 3: RTI-DP, STEP and Shallow-π. The Related Work section (Section 5) now cites and positions all three. After studying the three papers, we found RTI-DP the closest counterpart to WPFS's recycling, since it also reuses the previous chunk of a pre-trained policy, and added it as a baseline to Table 1 on all three backbones. We first positioned STEP and Shallow-π from their papers, for the reasons below.

RTI-DP in Table 1 uses the best setting of the configurations we swept (Table R10, App. D). Our port re-noises the chunk to τ′=0.5\tau'=0.5 before 5 solver steps, which makes it stronger than the original on π0\pi_0; one draw, no test. Success (mean over suites or tasks) and speedup, as in Table 1:

π0\pi_0π0.5\pi_{0.5}DB-OFT
Baseline94.1%97.5%67.9%
RTI-DP92.3%, 1.18×96.6%, 1.20×69.8%, 1.41× (best of 4 start steps)
WPFS-Large94.9%, 1.52×97.9%, 1.83×74.6%, 1.43×
WPFS-Small95.0%, 2.82×97.8%, 3.91×74.0%, 2.05×

(π0\pi_0 WPFS-Small is 2.82× in the revised Table 1, corrected from 2.95×; see our response to Reviewer Yu7f.) RTI-DP shortens only the solver and runs the backbone at every call, so its speedup stays near 1.2× on π0\pi_0 and π0.5\pi_{0.5}; on DB-OFT, deeper warm starts lose 10–33 points (Table R10).

STEP (positioned from its published numbers; our runs in the follow-up confirm both points). Its published results show no accuracy gain over the full policy, and on π0\pi_0 even a single solver step would bound it at the speed of WPFS-Large.

  • Accuracy. Its reported gains are over other few-step samplers (BRIDGER, DDIM). Against the full policy, in its own π0.5\pi_{0.5} experiment (its App. F.5, Table 18), one step plus STEP averages 96.3% over Spatial, Object, Goal and Long, against 97.2% for the 10-step policy; on its real-world tasks, it matches the full policy with 4 steps and reaches 87.5% with 2. Both WPFS variants match or raise success on every backbone (table above).
  • Speed. STEP shortens the solver with a predictor trained for each policy, and the backbone still runs at every call. Its 105.7× (4 steps, real world) is measured against a 100-step policy whose cost is almost all denoising. On π0\pi_0, the solver is 38% of a plan (Figure 1), so one step instead of ten leaves about 66% of its cost: at most 1.52× before the predictor's own cost, the speed of WPFS-Large and about half that of WPFS-Small (2.82×). On π0.5\pi_{0.5}, where STEP's own experiment reports 96.3%, halving the solver (RTI-DP) gives 1.20×, so a single step would give about 1.4×, against 1.83× (97.9%) and 3.91× (97.8%) for WPFS.
  • Complementarity. STEP's predictor could supply WPFS's prior, and the agreement test would decide how much of the result to execute.

Shallow-π produces a faster backbone for WPFS to run on, not a direct alternative to it. It distils a flow-based VLA from 18 to 6 layers (over 2× faster per call), whereas WPFS reduces how often the full model is called and wraps whichever backbone it is given. WPFS already runs on backbones of two families, and two of our three backbones, π0\pi_0 and π0.5\pi_{0.5}, belong to the same π family as Shallow-π. Assuming 2× per plan and our student unchanged, WPFS-Small on top of Shallow-π would reach about 2.2× over Shallow-π, or 4.5× over π0\pi_0 (an estimate). We found no released distilled checkpoints, and distilling each backbone we have not been able to complete yet.

What distinguishes WPFS from all three:

  1. re-noising the clean tail to a level bounded by Corollary 3 (without it, warm-starting fails on π0\pi_0);
  2. an agreement test that sets the execution horizon, so the full model runs less often, whereas RTI-DP and STEP keep a fixed cadence;
  3. a distilled corrector that removes the language backbone from corrections, which Shallow-π keeps in every call.

The revision isolates each component: recycling alone buys accuracy but not speed (Table 4); with the lineage limit kept, the agreement test adds 6.4 points at the same cap (Table R1); the dispersion term adds 1.4 points on average (Table R2).

The review marked its soundness concerns as minor or fixable. The experiments it proposed for Questions 1 and 2 are now in the paper, RTI-DP and STEP are new baselines in Table 1, and we explain above why we did not add Shallow-π. If these address the concerns behind the rating, we would be grateful if the reviewer would consider raising it. We would welcome any further questions.

AuthorsOpenReview ↗

Follow-up to Reviewer fdxo: STEP on all three backbones

We have now run STEP, the learned counterpart of RTI-DP, on all three backbones. On every backbone, both WPFS variants are more accurate than STEP, and on π0\pi_0 and π0.5\pi_{0.5} they are also faster, as our earlier response anticipated. The results are now in Table 1, with every setting we ran in Appendix D (Table R11).

Setting.

  • Port. STEP's released code covers Diffusion Policy only, so we ported it with its published predictor (two cross-attention blocks, width 128; ≈1.1M parameters with our inputs) and training (MSE, AdamW 10−410^{-4}, batch 64, 100k iterations). It is trained on (cached chunk → next plan) pairs from baseline rollouts on training states only: 25,530 on π0\pi_0, 49,890 on π0.5\pi_{0.5} and 13,248 on DB-OFT.
  • Inference, as in STEP's Algorithm 1, with one draw at the baseline's cadence. We use STEP's own settings, one solver step (τ′=0.1\tau'=0.1) for flow-matching VLAs, as in its π0.5\pi_{0.5} experiment, and two DDIM steps for diffusion policies, as in its main experiments. We add one deeper level: two steps on π0\pi_0 and π0.5\pi_{0.5} (τ′=0.2\tau'=0.2) and three on DB-OFT, which matches WPFS-Small's speed. STEP's literal warm start ignores the flow models' (1−τ)(1-\tau) scaling and collapses to 32.0% on π0\pi_0; we use (1−τ′)A^+σtε(1-\tau')\hat A+\sigma_t\varepsilon.
  • Evaluation: the initial states and cost model of Table 1, charging the predictor's 0.55 ms to every call.

Results (success %, mean and speedup; baselines and WPFS from Table 1).

π0\pi_0SpatialObjectGoalLongMeanSpeedup
Baseline96.098.495.686.294.11.00×
STEP, 1 step91.297.887.081.489.41.44×
STEP, 2 steps93.498.889.884.891.71.36×
WPFS-Large95.699.494.689.894.91.52×
WPFS-Small95.699.095.689.895.02.82×
π0.5\pi_{0.5}SpatialObjectGoalLongMeanSpeedup
Baseline98.698.898.494.097.51.00×
STEP, 1 step98.097.697.291.496.11.44×
STEP, 2 steps98.298.697.692.896.81.37×
WPFS-Large98.899.298.894.697.91.83×
WPFS-Small99.099.098.894.497.83.91×
DB-OFTCarrotSpoonStackEggplantMeanSpeedup
Baseline60.085.831.794.267.91.00×
STEP, 3 steps57.581.727.594.265.22.08×
STEP, 2 steps54.279.226.791.762.92.47×
WPFS-Large82.588.330.896.774.61.43×
WPFS-Small79.290.031.795.074.02.05×

Why the accuracies differ (our reading). STEP does not exceed the baseline on any backbone (−4.7 and −2.4 points on π0\pi_0, −1.4 and −0.7 on π0.5\pi_{0.5}, −5.0 and −2.7 on DB-OFT), while both WPFS variants keep or raise success. Three differences are the likely reasons:

  • It trusts every warm start. Each predicted chunk is executed with one draw, no test and no fallback, so any error in the prediction goes uncaught. WPFS executes only the prefix on which its drafts agree, and replans when they split.
  • It re-enters shallowly. At τ′=0.1\tau'=0.1–0.20.2 the solver stays close to its starting point, the small-bandwidth regime of Corollary 3; Table R7 shows that even τ0=0.3\tau_0=0.3 lets wrong priors through. WPFS re-enters at τ0=0.5\tau_0=0.5.
  • It never replans after an episode's first chunk, so errors can accumulate; WPFS bounds every lineage with Λ\Lambda.

Why STEP is faster than WPFS-Large on DB-OFT. DB-OFT's solver is a large share of a call, so cutting DDIM steps saves most of a plan, and STEP skips the safeguards that cost WPFS-Large speed: WPFS-Large replans at least every 15 steps (Λ\Lambda on DB-OFT) and after each rejected correction, and its drafts run the full backbone. These safeguards are what keep accuracy: at 2.08–2.47×, STEP loses 2.7–5.0 points to the baseline, whereas WPFS-Large gains 6.7 at 1.43×. WPFS-Small, whose corrections skip the backbone, matches STEP's speed (2.05× against 2.08×) with 8.8 more points (74.0% against 65.2%).

On π0\pi_0 and π0.5\pi_{0.5}, the backbone dominates each call, so STEP stays at 1.36–1.44×, below WPFS-Large (1.52× and 1.83×) and about half and a third of WPFS-Small's (2.82× and 3.91×), in line with our earlier bound.

In summary, WPFS is more accurate than STEP on all three backbones and faster on π0\pi_0 and π0.5\pi_{0.5}; on DB-OFT, WPFS-Small matches STEP's speed with 8.8 more points. On π0.5\pi_{0.5}, our port reaches 96.1% against the 96.3% STEP reports (its Table 18).

We hope that these direct comparisons with RTI-DP and STEP on all three backbones resolve the reviewer's concern about novelty. If our responses now address the concerns behind the rating, we would be grateful if the reviewer would consider updating it. We would welcome any further questions.

Reviewer_Yu7fOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WPFS, a framework for accelerating diffusion-based vision-language-action models by recycling the unexecuted tail of a previous action chunk. The recycled trajectory is re-noised and refined using the latest observation, while agreement among multiple corrected drafts determines the execution horizon and whether full replanning is necessary. The framework includes a training-free variant using the original policy and a distilled variant with an approximately 28M-parameter corrector that bypasses the language backbone. Evaluations on three policies across LIBERO and SimplerEnv report up to 3.91x normalized inference speedup with comparable or improved success rates. Physical experiments further demonstrate benefits for moving-target interception.

Overall, I lean toward acceptance because the approach offers a useful efficiency improvement supported by a reasonably broad empirical evaluation.

Critical strengths

1.The framework offers a practical combination of trajectory reuse, adaptive execution, and lightweight correction. Although warm-starting from previous actions is an established idea, WPFS goes beyond reducing denoising steps by allowing the small corrector to incorporate current visual observations without repeatedly running the language backbone. The similar task performance of WPFS-Large and WPFS-Small supports the usefulness of this design. 2.The evaluation provides meaningful evidence across different policy architectures and settings. Table 1 reports favorable speed–success trade-offs for all three evaluated policies, and the physical experiments show substantial improvements over the synchronous baseline on moving-target tasks. The fixed-horizon comparisons and paired statistical tests in Appendix B strengthen the evaluation beyond a comparison with a single default execution schedule. The explicit separation of student-training and evaluation initializations is also a positive aspect of the experimental protocol.

Critical weaknesses

1.The contributions of the agreement test and dispersion-aware distillation are not fully isolated. In Table 4, removing the agreement test also removes the lineage limit and associated full replanning, so the comparison measures more than the effect of adaptive scheduling. In addition, the similar dispersion statistics of the teacher and student do not establish the benefit of the dispersion regularizer without a matched student trained with λ = 0. The appendix acknowledges both limitations. These issues leave some uncertainty about which components are necessary, although they do not invalidate the reported performance of the complete framework. 2.The real-time conclusions have a narrower evidential basis than the computational-efficiency results. The reported latency per action is amortized GPU computation rather than observation-to-command response time, and full-plan fallbacks can still interrupt execution. Moreover, the physical evaluation uses synchronous execution on one platform and a single manipulation task under static and constant-velocity conditions. These experiments demonstrate useful gains in the tested setting, but do not establish the same advantage over asynchronous execution or under more varied dynamics. A more precise statement of this scope would improve the paper.

Questions for authors

1.Can you clarify the timing scope of the reported correction/plan latency ratios? Figure 1 reports 12.0 ms for image encoding within a 57.8 ms π₀ plan, while the student is described as re-running the policy’s vision encoder at each correction. Under identical profiling conditions, this alone would imply a ratio of approximately 0.208, above the stated WPFS-Small range of 0.028–0.15. Do implementation differences or measurement boundaries explain this discrepancy? Please specify whether current-image encoding, verification, and rejected corrections are included in the reported costs. 2.Do you have results for a fixed-segment correction schedule retaining the same lineage limit and periodic full replanning? Such a comparison would clarify how much of the benefit comes specifically from the agreement-based execution schedule. 3.Do you have a matched λ = 0 student comparison? Its success rate, draft dispersion, and rejection frequency would help determine whether the dispersion regularizer materially contributes to the reported results.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Yu7f

We thank the reviewer for a careful review. The recomputed latency ratio exposed an inconsistency, now corrected, and we ran both requested controls. The revised PDF is uploaded, with all changes in blue; new Tables R1–R3 are in Appendices B.2, B.3 and E.

Weakness 1 / Question 2: Agreement test with the lineage limit kept (Table R1, App. B.2). We ran fixed-segment schedules that keep Λ\Lambda and its periodic full re-planning (π0\pi_0; Spatial and Long; both correctors; 500 episodes per cell). With Λ\Lambda held fixed, the agreement test adds 6.4 points at the same cap (89.2% → 95.6%). On Spatial, WPFS-Large is 6–23% faster than fixed schedules within one episode of its success. On Long, no fixed schedule reaches WPFS's success with either corrector. WPFS-Small's corrections cost 0.16 of a plan, and a tuned fixed schedule reaches a similar operating point on Spatial, so the student's speedup stems mainly from recycling and distillation; the agreement test avoids tuning the segment per suite. We keep Λ\Lambda because it guards against perceptual drift and adds 1.8 points on Long for WPFS-Small (App. B.2).

Weakness 1 / Question 3: Matched λ=0\lambda=0 student (Table R2, App. B.3). Same data, architecture, seed, optimizer schedule and checkpoint rule; only λ\lambda changes. Without the regularizer, success drops on all four suites, by 1.4 points on average (95.0% → 93.6%, below the unmodified policy's 94.1%). Draft dispersion rises from 1.03× to 1.14× the teacher's, and rejections from 18.3% to 19.3%. The deviation is toward over-dispersion, which the certificate handles conservatively; Section 3.4 now covers both directions.

Question 1: Latency ratio (Fig. 1, Table 1, App. C). Two factors explain the gap. (i) On LIBERO, the plan encodes three image slots (the two cameras and openpi's masked placeholder), the input format of the released openpi checkpoints, kept unmodified for the base policy. The student encodes only the two cameras, since the placeholder carries no information (7.8 ms rather than 12.0 ms). The paper did not state this; Figure 1's caption and Appendix C now do. (ii) The π0\pi_0 student latency in Table 1 came from a one-off benchmark of an early prototype that encoded both cameras in one batched call. The final student, like the released openpi code, encodes each camera separately, but we did not re-time this entry. Re-timed in the same session as the plan, it gives γ=0.160\gamma=0.160 and a speedup of 2.82× (previously 2.95×). Had the plan also encoded only the two cameras, the speedup would be 2.76× (App. C). Both remain above the two single-stage ceilings and every baseline (≤ 1.70×). Success rates and the π0.5\pi_{0.5} and DB-OFT entries are unchanged. As requested, Appendix C now states what the costs include:

  • current-image encoding: included;
  • the KK drafts: generated inside the timed call; the CPU agreement test (47 µs, ≈0.1% of the cost) is not priced;
  • a rejected correction: charged with the full plan it triggers.

We acknowledge this oversight, which the reviewer rightly identified: it overstated the π0\pi_0 WPFS-Small speedup by about 5% in Table 1 and Figure 1. We re-checked the other numbers against the code and logs; the only remaining differences reflect measurement scope (Figure 1's per-stage profile vs end-to-end timing, 57.8 vs 58.8 ms; three- vs six-task robot averages), now stated in Appendices C and E. The corrected timing can be verified with the corrector code in the anonymous repository.

Weakness 2: Real-time scope (Table 2, Table R3, App. E). As suggested, we measured response time, fallbacks and control rate in both execution modes, and added three dynamic tasks.

  • WPFS-Small's median observation-to-command time is 24 ms synchronously and 77 ms asynchronously (base policy: 93 and 121 ms). Its fallbacks do interrupt execution (a rejected correction, then a full plan: ≈116 ms each), but cost ≤ 1.1% of control time.
  • An asynchronous executor cuts time stalled to ≤ 3% and helps the base policy (5 cm/s: 28% → 50%; 6 cm/s: 0% → 10%), but WPFS-Small stays +48 and +42 points ahead (98% and 52%; Fisher p≤2×10−5p \le 2\times10^{-5}).
  • More varied dynamics (Table 2, App. E; 50 trials each): success rises from 22% to 96% under a sudden direction change and from 24% to 80% under temporary occlusion (p<10−7p<10^{-7} for both), and from 68% to 76% when the target is pulled away.
  • The abstract and introduction no longer claim extreme latency reductions or real-time control; the conclusion states the scope (one platform, one pick-and-place task under six target behaviors, a simple asynchronous executor).

We note, respectfully, that the review's summary leans toward acceptance while the rating is 2 (Weak rejection). If the revisions above address the concerns behind this rating, we would be grateful if the reviewer would consider updating it. We welcome further questions.

(Edited 13 Nov to match the revised PDF.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.