acceptodds
Withdrawn from ICLR 2027

FReD-VLA: A Vision-Language-Action Model with Flow Matching-Initialized Diffusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Deploying vision-language-action (VLA) models for bimanual manipulation in real-world environments requires robots to generate accurate actions with low latency to respond to changing surroundings. State-of-the-art methods usually adopt diffusion- or flow matching-based approaches for continuous action generation. Despite promising results, balancing accuracy and latency still remains challenging. Diffusion models can generate accurate actions, but repeated denoising increases computation and latency. One-step flow matching is efficient, but its actions tend to be inaccurate for fine-grained manipulation. In this paper, we propose a Vision-language-Action model with Flow Matching initialized Diffusion (FReD-VLA), which combines fast flow-based generation with diffusion-based correction. Given visual observations, language instructions, and robot states, a flow matching model first generates a coarse action sequence. The coarse action sequence is then refined by a diffusion model through Action Distribution Calibrator, enabling refinement from a well-initialized intermediate diffusion state rather than the fully-corrupted beginning state. Empirically, on Maniskill PickCube-v1, our model achieves a mean minimum end-effector-to-cube distance of 10.8 mm, with reduction than the 43.7 mm measured for pure Diffusion. In offline evaluation, one diffusion step reduces the joint mean absolute error of a one-step flow proposal from to . This is a 47.3% reduction. With one flow evaluation and one diffusion evaluation, FReD-VLA achieves an action-generation latency of 37.0 ms, excluding shared multimodal encoding. This provides a speedup over five-step diffusion with comparable action accuracy. Real-robot tests further demonstrate dynamic bimanual assembly after adaptation with about 20–30 human demonstrations per behavior.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Bimanual Manipulation, Flow Matching, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xLpoOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FReD-VLA, a hybrid vision-language-action policy that first generates an action proposal with flow matching and then applies diffusion-based refinement through an Action Distribution Calibrator. The main motivation is to combine the low latency of flow-based generation with the accuracy of diffusion policies. The paper reports offline action-prediction results, a small ManiSkill PickCube-v1 study, solver and calibration ablations, and a qualitative dynamic bimanual assembly demonstration on a real robot.

Critical strengths
  1. The paper targets an important problem in robot control: reducing action-generation latency without sacrificing fine-grained accuracy. The ADC mechanism is clearly described and provides a simple way to transform a flow proposal into an input compatible with a diffusion denoiser.

  2. The offline evaluation includes latency measurements, Flow-NFE sweeps, and ADC-fraction ablations. Under the reported setup, the hybrid method obtains lower Joint MAE than one-step Flow and similar latency to the selected multi-step Flow baseline.

  3. The manuscript is generally well organized, and the authors make a meaningful effort to state the assumptions and limitations of the theoretical analysis. The inclusion of closed-loop simulation and a real-robot demonstration is also useful as an initial feasibility check.

Critical weaknesses
  1. The central motivation is not convincingly established. Flow matching can itself use multiple integration steps, so the paper needs to demonstrate why a diffusion refiner is preferable to simply using a stronger or multi-step Flow policy. The reported comparison with Euler-8 is not sufficient to isolate this question. In particular, the hybrid method combines two separately trained models and may benefit from additional model capacity, whereas the Flow baselines do not appear to receive a comparable parameter or training-budget allocation. The results therefore do not clearly show that diffusion refinement is intrinsically better than additional Flow computation.

  2. From the reported numbers, Flow Euler-1 and Euler-8 imply approximately 4.6–4.7 ms per Flow evaluation, whereas Pure Diffusion-2 and Pure Diffusion-5 imply an incremental cost of approximately 32.6 ms per diffusion evaluation. Thus, a diffusion call is roughly seven times more expensive than a Flow call. This is difficult to reconcile with the statement that Pure Diffusion, Pure Flow, and FReD-VLA share the same DiT multimodal backbone. If the same backbone is executed on the same action chunk with the same conditioning, the per-call costs should be broadly comparable; the difference should not normally be an order of magnitude. The discrepancy may indicate that the compared methods use different action-generation modules, different parameter counts, different sequence lengths, different caching or conditioning paths, or non-equivalent timing procedures, but the paper does not specify which explanation applies.

  3. The reported 37.0 ms latency for FReD-VLA is numerically close to one Flow call plus one diffusion call. Therefore, its speedup over five-step diffusion is primarily obtained by replacing four expensive diffusion calls with one such call and one much cheaper Flow call. This does not establish that Flow matching is intrinsically faster at the same number of steps, nor does it demonstrate that diffusion refinement is preferable to an equivalently budgeted multi-step Flow policy. The paper should provide synchronized per-module timing, exact architectures and parameter counts, FLOPs, action-chunk dimensions, precision and batching settings, feature-caching details, and end-to-end latency including multimodal encoding.

  4. In the ManiSkill experiment, only nine fixed rollouts are used, and the Flow-conditioned variant achieves the same success rate as Pure Flow. The reported 10.8 mm distance result appears to correspond specifically to the Flow-conditioned Step50 variant, rather than unambiguously to the default FReD-VLA configuration. This makes the main task-level improvement claim difficult to evaluate.

  5. No independent trial counts, success rates, failure cases, or controlled comparisons against Flow-only and Diffusion-only policies are reported. As a result, the evidence does not support broad claims about practical robustness or dynamic bimanual manipulation performance.

  6. The theoretical analysis assumes a conditional Gaussian action distribution, an ideal denoiser, deterministic first-order sampling, and simultaneous diagonalization of the action covariance and temporal-difference operators. The appendix correctly acknowledges these restrictions, but the resulting bounds do not explain the behavior of the learned multimodal policy and do not establish that diffusion refinement should outperform additional Flow steps in the actual setting.

  7. Only nine trajectory anchors with multiple stochastic samples are used in the offline evaluation. These samples assess stochastic variation but do not provide broad task or state coverage. In addition, the fairness of external baselines is difficult to assess because their training data, adaptation procedures, checkpoints, parameter counts, and computational budgets are not specified in sufficient detail.

Questions for authors
  1. The main question is why diffusion refinement is needed at all when the Flow policy can be evaluated for additional integration steps. Can the authors provide a matched comparison against Flow policies with comparable parameter counts, training budgets, and wall-clock cost, including optimized multi-step solvers or a larger Flow model?

  2. How can the authors reconcile the approximately seven-fold difference between the per-evaluation Flow and diffusion latency with the claim that the methods share the same DiT multimodal backbone? Which modules are shared, which modules are separate, and how do their parameter counts, sequence lengths, FLOPs, caching behavior, and timing boundaries compare?

  3. How many independent real-robot trials were performed for each behavior? Please provide success rates, failure categories, and comparisons with Flow-only and Diffusion-only policies under the same demonstrations and execution layer.

Reviewer_wtaNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FReD-VLA, which aims to improve the trade-off between action prediction accuracy and generation latency in vision-language-action (VLA) models. It first generates an action proposal using a conditional Flow Matching model, then employs an Action Distribution Calibrator (ADC) to convert the proposal into an intermediate state at a specified noise level following the forward noising schedule of a diffusion model, and finally refines it with a small number of diffusion steps. The authors conduct experiments on offline action prediction, ManiSkill PickCube-v1, and a dual-arm assembly task, where FReD-VLA attains a lower Joint MAE than conventional flow matching baselines, validating the effectiveness of the proposed approach.

Critical strengths
  1. The paper addresses a practically important problem: the trade-off between manipulation precision and action-generation latency. The strategy of generating a fast initial action proposal followed by local refinement is aligns with this objective.
  2. The offline experiments show localized gains. With matched conditions and comparable latency, FReD-VLA attains a lower joint MAE than simply increasing the number of flow integration steps.
Critical weaknesses
  1. Limited evidence of closed-loop benefits: The offline accuracy–latency gains are not yet supported by convincing evidence of improved closed-loop performance. On ManiSkill PickCube-v1, Flow-conditioned Step50 and Pure Flow achieve the same success rate (3/9), so no success-rate benefit from the diffusion is demonstrated. The real-robot section showcases a dynamic dual-arm assembly task, but the test details are not reported. The evaluation covers only one simulated task and one real-world assembly scenario, but its generalizability across different tasks has not yet been verified.
  2. The data split and coverage of the offline evaluation are unclear. The main offline evaluation uses only 9 trajectory anchors, each sampled 10 times. The author does not adequately describe where the anchors come from, how they were selected, how they relate to the training demonstrations. It cannot be determined whether the observed error reduction generalizes to unseen trajectories and scenes.
  3. The reported results demonstrate gains only relative to the authors' own Flow checkpoint, with no comparison against mainstream flow-matching VLAs. Tab.1 includes SmolVLA, which attains a lower MAE (0.1817°). If the proposed method were transferred to stronger, more widely used baselines — e.g., π0.5, or the more accurate SmolVLA would consistent gains still be observed?
  4. The method requires separately training or adapting two action-generation models, Flow and Diffusion (as I understand it, training two different policies and then combining them), and some simulation configurations further require adapting the diffusion refiner after the Flow model is frozen. Compared with an end-to-end policy, this incurs additional training overhead.
Questions for authors

As noted above, My main concerns are the following:

  1. Is there further experimental evidence that FReD-VLA actually improves closed-loop performance? What ultimately matters is accuracy and generalization in real closed-loop deployment, not fitting accuracy on open-loop trajectories.
  2. Offline evaluation's clarification: How were the nine trajectory anchors selected, and what is their relationship to the training and validation demonstrations?
  3. Comparison with representative flow-matching VLAs and similar work. Is there evidence that FReD-VLA provides better results than closely related methods?
  4. Quantify how much additional training requirement and cost the method incurs relative to an end-to-end VLA policy. Furthermore, would it be possible to optimize the two modules jointly, rather than training two separate policies and combining them?

The problem studied here is interesting, but the current method and experiments do not yet substantiate the claims made in the paper. If the authors can address the concerns raised above, this would directly affect my overall assessment.

Reviewer_7HFwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

FReD-VLA combines a flow-generated action proposal with short diffusion refinement. An Action Distribution Calibrator applies the diffusion forward-noising kernel to initialize refinement at an intermediate timestep. The paper provides an idealized analysis and evaluates offline action prediction, a small ManiSkill experiment, and a real-robot assembly demonstration. The clearest reported benefit is an offline accuracy–latency trade-off.

Critical strengths

The offline internal comparison reuses the standalone Flow and diffusion checkpoints, helping isolate the effect of combining them. At similar generation latency, the hybrid improves joint MAE over eight-step Flow. The solver and noise-level ablations help explain the computation trade-off. The theoretical analysis also states its Gaussian and ideal-denoiser assumptions explicitly.

Critical weaknesses
  1. The methodological contribution beyond existing proposal-and-refinement approaches is limited. ADC applies the standard diffusion forward-noising kernel to a proposal, while informed initialization and coarse-to-fine refinement already appear in CCDF and HybridFlow, both cited by the paper. The manuscript does not adequately establish what diffusion refinement adds over these alternatives under a comparable inference budget. The idealized analysis provides an explanation but does not establish this distinction.
  2. The empirical evidence is too limited to support the claimed practical benefit. Appendix B.1 evaluates only nine trajectory anchors with repeated sampling seeds. In simulation, the hybrid and Pure Flow both succeed in 3/9 rollouts and have the same mean minimum approach distance. The headline 75.3% improvement is therefore a distance reduction relative to diffusion, not evidence that refinement improves task success over Flow. Real-robot results lack a quantitative success-rate comparison and use an insufficiently described residual execution layer. The reported 37 ms also excludes multimodal encoding; against the stronger two-step diffusion baseline, the generation speedup is about 1.46× with slightly worse MAE. Relevant references: HybridFlow; Come-Closer-Diffuse-Faster.
Questions for authors
  1. How were the nine trajectory anchors selected? Were their source trajectories excluded from training and from selection of the ADC fraction?
  2. What does the expert-derived residual execution layer do, and which parts of the reported real-robot behavior depend on it?
  3. Are quantitative real-robot comparisons or additional closed-loop results already available that isolate the benefit of diffusion refinement over Pure Flow? Please distinguish these from the offline MAE results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.