- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces SnapFlow, a self-distillation method for VLA models. Its core idea is to combine standard flow-matching objective with two-step Euler endpoints objective generated by the model itself, alone with a zero-initialized destination-time projection. In simulation experiments, the method successfully distills pi0 and pi05 from 10-step to one-step action generation, achieving more than 3 times inference speedup without sacrificing task success rates. The paper also presents several toy real-robot deployment demonstrations.
Critical strengths
- The proposed SnapFlow maintains high success rates on the LIBERO benchmark.
Critical weaknesses
Main:
- There are many ways to distilling flow-matching models and flow0matching VLAs into on-step model, as the authors say in the introduction and related work sections. However, the paper does not provide quantitative comparisons and analysis with these methods, leaving many of its claims insufficiently supported.
- The experimental gains appear minor. In simulation, naive single-step generation already performs very similarly to the teacher, and SnapFlow offers only small additional gains over both. Naive single-step generation also appears to perform reasonably well. The real-robot experiments provide only a few toy video demonstrations, without quantitative results. It seems the paper should evaluate the method on more challenging simulation benchmarks to better demonstrate its effectiveness.
- The performance of 2-, 3-, 4-, and 5-step generation is reported only in terms of offline action quality, without corresponding online success rates. To my knowledge, many real-robot and simulation evaluations use 4 or 5 denoising steps by default to balance inference speed and task performance [1][2][3]. The authors should establish whether simply SFT on the official LIBERO dataset with a 4-step configuration could achieve comparable or even better performance.
- It will be better to report end-to-end throughput or task completion speed in both simulation and real-world experiments. Robotic tasks involve more than model inference. The time spent interacting with the environment also matters. Based on a rough visual assessment, SnapFlow may improve actual throughput by only 30–40% over the teacher, which is substantially less pronounced than the reported 3 times model inference speedup.
Minor:
- The legend in Figure 2(a) contains four items, but only two of them are visible in the plot.
[1] Chen, Kang, et al. "πrl: Online rl fine-tuning for flow-based vision-language-action models." arXiv preprint arXiv:2510.25889 8 (2025). [2] Black, Kevin, Manuel Galliker, and Sergey Levine. "Real-time execution of action chunking flow policies." Advances in Neural Information Processing Systems 38 (2026): 33383-33407. [3] Wang, Chenghua, et al. "PhyAI: Real-Time Physical AI at the Edge, Scalable Rollouts in the Cloud." arXiv preprint arXiv:2608.03682 (2026).
Questions for authors
- The experimental results refer only to LIBERO, but LIBERO comprises multiple benchmark suites, such as LIBERO-goal. Which suites, tasks, or combinations of tasks were used in the evaluation?
- How does the distillation data organized? How sensitive is SnapFlow to the distribution, quality, and quantity of the distillation data?
4 comments
Response to Reviewer XUEU (2/2)
W3. Closed-loop results at 2 to 5 steps; would 4-step SFT suffice?
Teacher at 1/2/3/4/5/10 NFE, paired, seed 1000, 400 episodes (the 2-NFE run lacks LIBERO-Spatial): 377, 278 of 300, 374, 385, 379 and 373 successes. Four steps is the best single-seed number and the other step counts are within 12 episodes of it, inside the ±4 pp single-seed CI, and over three seeds 10 steps and 1 step differ by −0.6 pp. Flow-matching SFT does not depend on the inference step count, so "SFT with a 4-step configuration" is the existing fine-tuned teacher run at 4 NFE, which is in the table. On LIBERO it performs the same as 1 NFE at the higher latency reported in Table 8 of the submission (Appendix H of the revision), so 4-step inference is a reasonable default there but not a better one. On PushT the teacher reaches 12.5%, 22.5% and 30.0% at 1, 2 and 10 NFE (n = 200, paired; 2 vs 10 steps −7.5 pp, p = 0.09), so there the step count does matter; 3 to 5 steps were not run on PushT. The teacher sweep is in Appendix O of the revision and the PushT numbers in Appendix D.4.
W4. End-to-end throughput and task-completion time.
Agreed: the 3.3× figure is model-inference latency only, and the episode-level effect is much smaller because simulation and environment stepping dominate. We did not complete a per-episode wall-clock measurement within the discussion period, so the revision makes no claim about throughput or task-completion time: it reports 3.3× as batch-one model latency, adds that naive one-step decoding gives the same speedup, so the case for distillation cannot rest on speed, and states that no real-robot timing was measured under a controlled protocol (Sections 4.5 and 5). The supplementary side-by-side videos use a timer derived from policy steps × the measured per-chunk latency, as the supplementary README states, not wall-clock time; they therefore overstate the episode-level speedup, and we no longer cite them as evidence of it.
Minor. Figure 2(a) legend.
Thank you. Two of the four series were hidden: the SmolVLA markers coincided with the π0.5 markers after per-model normalization, and the π0.5 SnapFlow marker lay below the axis range. The revised Figure 2(a) offsets the SmolVLA markers and extends the axis so that all four series are visible; panel (b) now shows the paired numbers of Table 1.
Summary of changes for this review: paired multi-seed results replace the unpaired numbers; same-budget flow-matching control and teacher at 2 to 5 NFE added; LIBERO-Plus and PushT added, with a headroom table for every base policy and benchmark; the suites, the distillation data and the budget are stated explicitly; the episode-time claim is removed; the real-robot material is qualitative only; Figure 2(a) is fixed.
Response to authors
Thank you for your reply and clarification. Some issues (such as the detailed settings of the LIBERO evaluation, the configuration of distillation data, etc.) have been resolved. I still have the following two main concerns:
(1) Many numerical results in the paper remain unconvincing due to the issue of teacher's NFE. Given that NFE=4 is already a reasonable default setting for the teacher, the reported 3.3x speedup is still calculated relative to a 10-step teacher. This result lacks justification. Should the paper instead report the speedup relative to an NFE=4 teacher?
(2) The experiments in the paper are still not sufficiently convincing. For instance, comparisons with several sota one-step flow/diffusion models (such as Shortcut Model, MeanFlow, Consistency Policy, etc.) are missing. Furthermore, while it is reasonable for less performance improvement over the teacher as a VLA acceleration work, the improvement compared to a naive 1-step approach remains marginal. Additionally, since the LIBERO benchmark itself is relatively simple and success rates across all methods are quite close, SnapFlow has not been evaluated on more challenging benchmarks. Therefore, I still have concerns about the real effectiveness of SnapFlow.
Response to Reviewer XUEU (3/2)
Thank you for the follow-up, and for confirming that the questions on the evaluation suites and the distillation data are resolved.
(1) Speedup relative to a 4-step teacher. We agree that 10 NFE is not the only reasonable reference, and that 3.3× overstates the gain against a teacher run at a practical step count. Table 13 of the revision (A800, batch 1) times the pretrained π0.5 solver at 1, 2, 3, 5 and 10 NFE: 81.0, 103.3, 124.4, 166.9 and 274.0 ms. The one-step student runs at 83 ms, so against the teacher at 3 and 5 NFE the speedup is 1.5× and 2.0×. We did not time 4 NFE, but each extra step costs about 21 ms, which puts a 4-step teacher near 145 ms and the speedup against it at about 1.75×. The VLM prefix (about 60 ms) is paid at every step count, which is why the speedup is far below the reduction in action-expert calls. On LIBERO the 4-step teacher succeeds on 385/400 episodes against 377/400 at one step (one evaluation seed; the 95% interval of a paired difference at this size is about ±4 pp), so one step costs nothing measurable there relative to four. We will report the speedup against every timed step count, give the 4-step comparison in the main text, and no longer quote 3.3× without stating its reference.
(2) One-step baselines and harder benchmarks. We have not run Shortcut Models, MeanFlow or Consistency Policy on the same backbone, they cannot be completed before the discussion closes, and we make no claim of an advantage over them.
On harder settings, the revision goes beyond standard LIBERO (Appendices D.3 and D.4). On LIBERO-Plus (400 paired perturbed variants over five perturbation dimensions) the teacher, its naïve one-step sample and the student all drop by about 29 points, to 263/400 each, so distribution shift alone does not separate them. PushT is the setting where multi-step sampling matters: the same teacher loses 17.5 pp when sampled in one step instead of ten (30.0% vs 12.5%, p < 0.001). There the gain over naïve one-step is not marginal: the two one-step students reach 25.5% and 24.0%, +13.0 pp and +11.5 pp over naïve one-step (p = 0.001 and 0.002, 200 paired episodes), and neither is significantly below the 10-step teacher (p = 0.35 and 0.22). As we said in the general response, continued flow-matching training also recovers part of the gap (21.0%), and at this sample size the students cannot be separated from it. An equal-budget comparison with that control and with the one-step baselines you list is what would settle the question, and we agree it is the main open point.
Thank you again for reading the revision closely and for the specific suggestions.
Response to Reviewer XUEU (1/2)
We first answer the two questions, then the weaknesses. The general response gives the full paired tables.
Q1. Which LIBERO suites and tasks were used?
All four standard suites, LIBERO-Spatial, LIBERO-Object, LIBERO-Goal and LIBERO-Long (libero_10), each with its 10 tasks and 10 episodes per task (400 episodes per method and evaluation seed). Table 1 reports the per-suite and average success rates (columns Spatial, Object, Goal and Long-10, the last being libero_10, as the caption now states). Appendix C lists every one of the 40 tasks for each method in the paired π0.5 runs, and Appendix D.2 gives the π0 counts for every evaluation seed and suite.
Q2. How is the distillation data organized, and how sensitive is SnapFlow to it?
SnapFlow uses no extra data. It fine-tunes on the same LIBERO demonstration set that the teacher was fine-tuned on (the LeRobot release of LIBERO, all four suites, 1,693 episodes, 273k frames). In the revised recipe every training example is one observation, its instruction and a noised version of the demonstrated action chunk; the target is the endpoint that two Euler steps of the frozen pretrained model reach from that noised input, so the demonstrated action enters the loss only through the noised input and no action label is regressed directly (the submitted recipe mixed such rows with ordinary flow-matching rows at α = 0.5). Training runs 30k micro-batches of 4, accumulated over 8 into 3,750 updates of effective batch 32, i.e. 120k examples or about 0.44 passes over the data. We have not measured sensitivity to the quantity or quality of the distillation data and say so in Section 4.1.
W1. No quantitative comparison with other distillation methods.
Same checkpoint, data and budget, paired closed-loop evaluation, one-step inference, public π0 LIBERO checkpoint, 3 evaluation seeds (n = 1200): naive one-step without any training 1127/1200 (93.9%); flow-matching fine-tune with the same budget 1101 (91.8%); submitted SnapFlow recipe after two implementation fixes 1084 (90.3%); submitted recipe with frozen-teacher target 1079 (89.9%); revised SnapFlow recipe, as defined in the general response and not the method as submitted, 1125 (93.75%); 10-step teacher 1134 (94.5%). On π0.5 (aggregate): frozen-teacher target 97.25, EMA target 98.00, 10-step online target 97.00, naive one-step 98.25. The revised recipe is the only fine-tuned student indistinguishable from the teacher (p = 0.41), and no method, ours included, is distinguishable from naive one-step inference (p = 0.92 for the revised recipe). A Shortcut Models baseline with unrestricted step sizes was not completed in time and is not reported. On LIBERO these comparisons cannot separate any one-step method from naive one-step inference, because the teacher itself gains nothing from its extra steps; the revision states this plainly.
W2. The gains are minor; harder benchmarks.
On LIBERO the gain over the teacher is zero, and the revision says so. The paired 10-step-versus-1-step difference of the teacher itself, which measures how much a one-step student could recover from the teacher's sampling, is −0.6 pp on π0 (p = 0.55, n = 1200) and 0.0 pp on π0.5 (n = 400). On LIBERO-Plus, a fixed paired set of 400 perturbed variants across five perturbation dimensions, the teacher, its naive one-step sample and the revised one-step student all succeed on 263/400 (65.8%; p = 1.0 for every pair; Appendix D.3), so distribution shift does not open a gap either. On PushT (lerobot/pusht, the LIBERO π0 weights fine-tuned for 30k steps; 200 paired episodes, one training and one evaluation seed; Appendix D.4), where the demonstrations are multimodal, the same teacher loses 17.5 pp at one step (30.0% vs 12.5%, p < 0.001). The submitted recipe with frozen-teacher target (batch 8) recovers 13.0 pp of it (25.5%; p = 0.001 vs naive one-step; −4.5 pp vs the teacher, p = 0.35) and the revised recipe 11.5 pp (24.0%; p = 0.002 vs naive; −6.0 pp vs the teacher, p = 0.22), but a control that only continues flow-matching training recovers 8.5 pp (21.0%), and the three are statistically indistinguishable (p = 0.31 and 0.55 against the control); the revised recipe also used half the samples and an eighth of the optimizer steps of the control, so its number is not an equal-budget comparison. We therefore reposition the contribution as recovering 10-step accuracy at one step with a frozen VLM and no deployment change, state explicitly that on LIBERO the teacher's own one-step sample already does this, and claim no advantage over continued flow-matching training where the headroom is positive. The real-robot material is presented as a qualitative check only.