acceptodds
Under review as a conference paper at ICLR 2027

Outcome or Process? Rethinking Supervision for Flow-Matching VLA Policies

Abstract

Flow-matching vision-language-action (VLA) policies generate actions through a solver trajectory, repeatedly evaluating a conditional velocity field at its intermediate states. This raises a fundamental question for policy distillation: should a student imitate a sampled teacher action, or the field that generates it? Distilling the teacher’s final action chunk yields conditional flow-matching targets, but a single action chunk does not uniquely specify the teacher’s local field or its intermediate computation. We study this choice with Path-wise Velocity Distillation (PVD), which regresses the frozen teacher’s velocity at detached states along the student’s own solver trajectory. Under matched teacher–student checkpoints and training exposure, PVD outperforms DAgger with action-chunk labels by 9.9 and 10.1 percentage points on ManiSkill at four- and eight-step deployment, 12.0 points on Meta-World, and 0.180 subtasks per chain on CALVIN. A one-seed ManiSkill ablation attributes more of the gain to direct teacher-velocity targets than to placing them on the student’s solver trajectory. The method fails when trained and deployed at two solver steps. Under this matched setup, velocity labels on the student’s own solver states yield better students than action-chunk labels.

Primary area:
applications to robotics, autonomy, planning
Keywords:
flow matching, vision-language-action models, policy distillation, DAgger, imitation learning, teacher-student distillation, robot learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6ijXOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how to distill flow-matching VLA policy from frozen teacher. Instead of supervising the student with the final action chunk of teacher, PVD method directly matches the velocity field at the intermediate states visited by the solver trajectory of student. Across various benchmarks, PVD consistently outperforms an action-chunk DAgger baseline under the matched training setup.

Critical strengths

None.

Critical weaknesses

The core idea is very close to existing flow-distillation methods like π\pi-flow, i.e., directly regressing the student onto the teacher's velocity field at the student's solver states. I am not convinced that moving this idea into VLA setting is enough to support a full paper.

More importantly, the paper feels oversold. The "Outcome or Process?" framing suggest the general conclusion, but the actual evidence is much narrower. PVD is quite sensitive to the solver setup. It works well when training and deployment are matched, but degrade sharply when the solver grid changes. And at two steps, it even falls below the base model while DAgger remains strong. To me, this is closer to a project-level empirical comparison rather than a comprehensive rethinking paper.

Questions for authors

See above weaknesses.

Reviewer_bfhZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether flow-matching VLA student policies should be trained by imitating final action-chunk endpoints (DAgger) or by matching the teacher's intermediate generative velocity field directly. The authors propose PVD, which regresses a frozen teacher’s velocity vector at detached intermediate states along the student's own ODE solver trajectory, thereby eliminating the need to run sequential teacher solver rollouts during training. Evaluated with matched teacher–student model architectures, PVD improves task success over standard DAgger and matches the teacher's multi-subtask completion length on CALVIN. However, the empirical ablations show that the majority of performance gains stem from direct velocity supervision rather than trajectory placement, that the method suffers a severe collapse when trained and deployed at two solver steps where DAgger excels, and that training compute overhead increases over DAgger despite teacher query batching.

Critical strengths
  1. PVD consistently outperforms DAgger across 3 manipulation benchmarks. Significant improvement while matching frozen teacher performance on CALVIN.
  2. supervising student along its own denoising trajectory is a good idea, since it gets around the bottleneck of running a sequential teacher ODE solve and enables batching.
Critical weaknesses
  1. trajectory-aligned supervision is positioned as a core novelty of the work, however Table 3 shows (and authors also note) that the primary performance benefit comes from velocity supervision itself instead of action chunks. Moreover, when noth models are trained at their optimal hyperparam setting (4 solver steps), the difference is negligible and within the reported cross-node evaluation variance.
  2. In low latency regime (2 solver steps) PVD shows a significantly worse perforamnce than DAgger, however it appears that the test is misleading: PVD was trained against a 2 step teacher that also collapsed while DAgger was trained against a functional 4-step teacher. It seems that this experiment was considered hastily and needs to be done properly to derive correct inferences.
Questions for authors
  1. Why did you not evaluate PVD at 2 solver steps using velocity labels queried from an 8-step or 4-step teacher evaluated at the student's 2-step states, and does this asymmetry explain the entire failure mode?
  2. Given that interpolant sampling is standard and avoids dependency on the student's specific solver path, what evidence shows that placing queries on the student's solver trajectory provides a generalizable benefit beyond the single-seed 4-step ManiSkill result?
  3. Table 3 highlights that PVD reduces GPU-hours relative to interpolant controls, yet Section 4.3 notes that PVD requires more GPU-hours than standard DAgger across the three benchmarks. What specific component of the PVD forward/backward pass drives this overhead?
Reviewer_2v2VOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper compares two ways of transferring an RL-tuned flow-matching VLA teacher into a student initialised from the corresponding supervised checkpoint. The DAgger-style baseline learns from teacher-generated action chunks through conditional flow matching. Path-wise velocity distillation (PVD) matches teacher velocities at intermediate states of the student's numerical solver. The teacher and student have the same architecture and size. Experiments on ManiSkill, Meta-World, and CALVIN favour PVD under the main protocol, but are sensitive to solver settings. Ablations investigate supervision density, target source, and query location. The contribution is primarily a controlled empirical comparison. The analysis discusses the fields targeted by the two objectives and bounds endpoint error in terms of velocity-matching residuals.

Critical strengths
  • Meaningful, consistent gains in the main experiments. PVD improves the primary metric in every reported main seed, with average gains of approximately 10 percentage points on ManiSkill, 12 points on Meta-World, and 0.18 completed subtasks on CALVIN. The main ManiSkill comparison uses three training seeds. Additional ManiSkill evaluations support these gains beyond a single reset distribution. This provides credible evidence that the supervision choice matters within the tested protocol.
  • Useful controls and transparent reporting of limitations. The main comparison shares architecture, initialisation, optimiser, data budget, and teacher network evaluations per chunk. Ablations and offline controls help investigate why the methods differ. The authors also report solver-dependent failures and total GPU-hours, showing that batching teacher queries does not make PVD cheaper overall. This transparency strengthens the credibility of the empirical contribution.
Critical weaknesses
  • The advance over prior work and its practical significance are insufficiently established. The authors attribute the velocity-imitation objective to π-Flow (Chen et al., 2026) and also cite Shallow-π (Jeon et al., 2026), whose ablations include teacher-trajectory imitation when distilling a flow-based VLA into a smaller student. The main difference from the latter seems to be the student size and therefore the setup. What remains unclear to me is the specific new insight this study provides. The paper should explain more clearly how its comparison differs from these acknowledged precedents, what researchers learn beyond the existing velocity-distillation results and why it is impactful.

    The theoretical contribution is also unclear to me. For Proposition 1, Appendix D itself describes the argument as standard error propagation for one-step solvers and cites (Hairer et al., 1993). Related work such as "Error Bounds for Flow Matching Methods" (Benton, Deligiannidis and Doucet) seems relevant. I have not verified whether its assumptions and conclusions cover the precise discrete result here. I would therefore ask the authors to clarify what, if anything, Proposition 1 adds beyond applying established stability arguments, and how it informs the experimental conclusions.

    I am also not convinced that the empirical contribution is sufficient in its current form, particularly without real-robot validation. Using equally sized teacher and student models is a valid control, yet the study establishes neither compression nor an inference advantage over the teacher, and does not measure savings relative to the original RL training. Since the teacher is already available and uses the same inputs, the benefit of training this student instead of deploying the teacher is unclear. The main procedure still requires collecting fresh student-rollout chunks (80k in the paper); only the smaller 20k frozen-rollout control in Section 4.3 avoids online collection, and it still relies on Base rollouts in the environment. What is the benefit of using this method compared to simply retraining the teacher?

    Together, in my opinion, these limitations do not sufficiently demonstrate the significance of the comparison for acceptance.

  • The experimental scope and unexplained protocol choices limit the conclusions. As already mentioned, there are no real-robot experiments. Given that the main contribution is empirical, I consider the lack of evidence under physical sensing and execution conditions a major limitation. The simulation results remain useful, but do not establish that the supervision advantage persists on hardware. The solver sweep further narrows the result: the seed-0 ManiSkill advantage falls from approximately 12 points with eight-step training and deployment to 2.2 points with four-step training and deployment, and becomes -39.7 points at two steps (Section 4.4). The number of seeds also varies, so it is hard to follow which results hold with what confidence.

    Section 3.3 proposes, but does not evaluate, a multiple-teacher-sample estimate of the conditional-mean target in Equation (4). The eight-point control reuses one teacher action chunk and addresses a different question. Multiple samples could reduce target variance on Meta-World and CALVIN and provide a stronger action-supervision comparison. While it is a different approach, batching several teacher samples, for example 8 or 16, may be feasible on suitable hardware. Reporting the additional time and memory would let readers judge the trade-offs.

    Important choices also lack justification. ManiSkill reuses one fixed rollout-noise vector (evenly spaced values in [-1, 1], Table 7) across chunks and training seeds, whereas Meta-World and CALVIN use Gaussian draws, and all benchmarks use Gaussian evaluation noise. The paper specifies this discrepancy without motivating it. Table 7 also gives short predicted action horizons without explaining their rationale or checkpoint history. The paper should establish whether these horizons were inherited from the released checkpoints or changed, and how any change was validated.

    As already mentioned, the exposition compounds these difficulties and makes the paper hard to follow despite careful reading. A few examples: 1. I find the stop gradient notation unnecessary: the procedure can be explained directly as collecting student solver states, treating them as fixed inputs, and fitting student velocities to the frozen teacher's outputs. 2. The paper already states that the minimiser in Equation (4) is a conditional mean; the surrounding derivation could be shortened to that familiar interpretation. 3. The DAgger mixing parameter β is used (β = 0) but never defined, and the same symbol is reused for the AdamW betas in Table 7. 4. The discussion should explicitly state that PVD needs a queryable, compatible teacher velocity field, whereas action supervision can use any teacher supplying suitable action chunks.

Questions for authors
  • What new insight does this controlled comparison add relative to (Chen et al., 2026) and (Jeon et al., 2026), and what practical benefit motivates deploying the same-size student instead of the available teacher?
  • Why is rollout noise fixed (and deterministic) only on ManiSkill, with Gaussian noise used at evaluation?
  • Were the reported action horizons (Table 7) already used to train the released Base and teacher checkpoints? If changed, when and how were they validated?
  • Can the multiple-sample control from Section 3.3 be evaluated, together with its memory and compute costs?
  • Do the results hold on real hardware?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.