- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces Latent Action Guided Flow Matching (LAFM), which replaces the fixed isotropic Gaussian source in robotic flow-matching policies with latent-action-conditioned Gaussian priors to better capture multimodal and heteroscedastic action distributions. A pretrained latent action model discretizes visual dynamics into motion primitives, and the predicted latent action selects the corresponding learnable prior, with a proposition showing that mode-aligned priors can reduce expected transport distance. Experiments on real-world Franka manipulation and LIBERO show substantial gains over standard flow matching, including +23.4 points in real-world success rate and +10.4 points on LIBERO-90.
Critical strengths
- This paper presents a clear and well-motivated idea by replacing the fixed Gaussian source with latent-action-conditioned priors, providing a simple yet intuitive way to better align the source with multimodal and heteroscedastic robotic action distributions.
- LAFM demonstrates strong empirical performance, substantially improving over standard flow matching on LIBERO-90 and real-world manipulation tasks while remaining competitive with much larger pretrained VLA models.
- This paper includes useful ablations and qualitative analyses, including latent-action conditioning, learned prior visualization, and transport trajectory analysis, which help support the effectiveness of structured source distributions beyond auxiliary latent-action supervision.
Critical weaknesses
- The novelty positioning against recent structured-source, source-reparameterization, and learned-coupling methods is incomplete. Recent works (CSFM [1], COT-FM [2], CAR-Flow [3], VCT [4]) have already explored modifying the source distribution or source-target coupling in flow-based generative models. These works are highly relevant to LAFM's core motivation, yet the current comparison mainly focuses on standard FM and FM+LAC.
- Proposition 1 relies on the assumption where is the mean of the target action mode and is the mean of the learned source prior. However, although source and target are represented in the same Euclidean ambient space, there is no reason that the learned source mean should naturally lie close to the corresponding target-mode mean. This is a very strong but potentially problematic assumption.
- The respective contributions of the learned mean and variance are not disentangled. LAFM attributes its benefit both to mode alignment and to modeling heteroscedasticity, but the current experiments jointly learn mean and variance. Mean-only and variance-only ablations would clarify whether the improvement mainly comes from relocating the source modes or from modeling mode-dependent uncertainty.
- Standard LIBERO appears close to saturation and may not sufficiently stress the proposed method [5]. A more challenging robustness-oriented benchmark such as LIBERO-PRO [6] would better test whether latent-action priors capture transferable motion structure rather than only improving in-distribution fitting.
[1] CSFM: https://arxiv.org/abs/2602.05951 (NeurIPS)
[2] COT-FM: https://arxiv.org/abs/2603.13395 (CVPR 2026)
[3] CAR-Flow: https://arxiv.org/abs/2509.19300 (NeurIPS 2025)
[4] VCT: https://arxiv.org/abs/2502.18197 (ICML 2025)
[5] https://arxiv.org/abs/2606.04233
[6] https://arxiv.org/abs/2510.03827
Questions for authors
- Please discuss CSFM [1], COT-FM [2], CAR-Flow [3], and VCT [4] in the related work section. While CSFM can be considered concurrent work, COT-FM is a closely related prior method that also introduces structured source distributions for flow matching and evaluates on LIBERO. A direct comparison with COT-FM is therefore important for clarifying the empirical advantage and novelty of LAFM.
- Could the authors empirically verify whether the two assumptions in Proposition 1 are satisfied by the learned priors, i.e., and ? This would help assess whether the proposition is applicable to the actual learned model.
- For the remaining weaknesses, I encourage the authors to provide additional experiments and analyses addressing the concerns above. If the revised evidence is sufficiently comprehensive and resolves these issues, I would be willing to reconsider my current score upward.
4 comments
Response to Reviewer p3JE (Part 2)
2. Assumptions of Proposition 1
We agree with the reviewer that nothing guarantees a priori that a learned source mean lies close to the mean of its action mode. We have made two changes.
The proposition no longer assumes exact alignment. The revised Proposition 1 gives the reduction in expected transport distance as the sum of a mean-alignment contribution and a variance-reduction contribution, and holds whenever this sum is non-negative. The mean-alignment term is positive as soon as the prior mean is closer to the mode mean than the origin is. Appendix K gives the full decomposition, including the case where the selected mode differs from the action mode.
Both conditions are verified on the learned priors (Section 3.3), on a held-out validation set of LIBERO-90:
- Mean alignment: for 95.1% of time steps, the selected prior mean is closer to the mean of the corresponding action mode than the origin, .
- Source variance: the average variance per dimension is , well below the value of 1 of the isotropic source.
The alignment is therefore an outcome of training rather than an assumption. It is also what the objective favors: the prior parameters receive gradients from the flow matching loss, whose regression target is , so that shorter transport is easier to fit, while the KL regularizer prevents the priors from collapsing. Its effect on the learned flow is measurable: the kinetic energy drops by 67% (Section 3.3).
3. Contributions of the learned mean and variance
We agree that this ablation is needed. We are training two variants on LIBERO-90 with the same architecture and configuration as LAFM:
- mean-only: learned , with fixed;
- variance-only: learned , with fixed.
We will post the results as soon as the runs finish.
4. Benchmark saturation
We agree that the four standard LIBERO suites are close to saturation, and say so in Section 3.1.2. For this reason the paper also reports LIBERO-90 and, in Appendix A, two less saturated benchmarks:
| Benchmark | FM | LAFM | Gain |
|---|---|---|---|
| LIBERO (Spatial, Object, Goal, Long) | 95.4 ± 0.3 | 96.5 ± 0.3 | +1.1 |
| LIBERO-90 | 82.6 ± 0.2 | 93.0 ± 0.2 | +10.4 |
| RoboTwin 2.0 | 53.7 ± 0.3 | 66.0 ± 0.2 | +12.3 |
| LIBERO-Plus | 50.4 ± 0.2 | 65.5 ± 0.1 | +15.1 |
LIBERO-Plus is a robustness benchmark of the same family as LIBERO-PRO. It perturbs the four LIBERO suites along seven dimensions (object layout, camera viewpoint, robot initial state, language instructions, lighting, background, and sensor noise), for 10,030 task instances unseen during training. The same LAM, trained on a different robot (Fractal), is used for all benchmarks. LIBERO-PRO perturbs objects, initial states, instructions, and environments, which LIBERO-Plus also covers.
Summary of revisions
- added Section 4.2, discussing CSFM, COT-FM, CAR-Flow, and VCT;
- revised Proposition 1 so that it no longer assumes exact mean alignment, and verified its two conditions on the learned priors (Section 3.3);
- added the FM + LP control (single learned source) to Table 3;
- added discussion and comparison with other methods (BRIDGER, Cocos, A2A).
Still running, to be posted during the discussion period: the mean-only and variance-only ablations. We hope these changes address the reviewer's concerns, and we are glad to provide any further analysis.
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts in clarifying the relationship with prior work, revising Proposition 1, and providing results on more challenging benchmarks. The revised theoretical analysis and robustness evaluations address several of my major concerns. However, I believe the mean-only and variance-only ablations are important for understanding the respective contributions of mode alignment and heteroscedasticity. I look forward to the results of these ablations before making a final decision on my score.
We thank the reviewer for the follow-up. The mean-only and variance-only ablations are now complete and have been added to the paper (Appendix H). Both variants keep the latent action model, classifier, architecture, and training configuration of LAFM, and differ only in which prior parameter is learned. Results on LIBERO-90, over 3 seeds:
| Policy | Learned mean | Learned variance | Success rate (%) |
|---|---|---|---|
| FM | – | – | 82.6 ± 0.2 |
| FM + LAC | – | – | 85.4 ± 0.1 |
| LAFM, mean only | ✓ | x | 87.7 ± 0.5 |
| LAFM, variance only | x | ✓ | 88.3 ± 0.2 |
| LAFM | ✓ | ✓ | 93.0 ± 0.2 |
Each contribution identified by Proposition 1 is useful on its own: learning either the means (mode alignment) or the variances (heteroscedasticity) improves over FM + LAC, which shares the same latent action classifier, and the two variants are within seed variability of each other. However, neither recovers the performance of LAFM. The two parameters are therefore complementary: a prior is most useful when it is both centered on its motion mode and scaled to its spread.
We have also added the controlled comparison with BRIDGeR and A2A reimplemented in our architecture (Appendix G) and added both methods, using their original implementation, as baselines on Section 3.1.2.
We hope these results address the reviewer's remaining concern.
Response to Reviewer p3JE (Part 1)
We thank the reviewer for the detailed review and for the precise list of related works and requested analyses. We address each point below. In short: the four works are now discussed in the paper; the conditions of Proposition 1 are verified empirically; the mean-only and variance-only ablations are running; and results on two less saturated benchmarks are in Appendix A.
1. Positioning against structured-source and coupling methods
We thank the reviewer for these references. All four are now discussed in a new subsection of the related work (Section 4.2). We also compare against robotics methods with condition-dependent sources (BRIDGER, Cocos, A2A) in our response to Reviewer mX8F, including a direct comparison with Cocos on LIBERO.
CSFM, CAR-Flow, VCT.
COT-FM is the closest prior method: it also assigns Gaussian sources to clusters of the target distribution. The two methods differ in how the sources are obtained and in what they are for.
On LIBERO, COT-FM is applied to the pretrained FLOWER policy (~964M parameters) and reported on the Spatial and Long suites:
These numbers show the difference in purpose. COT-FM recovers with one function evaluation what FLOWER reaches with four. LAFM improves the policy itself over its FM baseline. With roughly 9 times fewer parameters and no pretraining, LAFM is 1.1 points above COT-FM on Spatial and 2.2 points below on Long. As we show on Appendix I, both FM and LAFM are quite robust to the number of function evaluations, keeping its success rate with one step in LIBERO-90 (Figure 9). We see the two methods as complementary, one proposing an adaptation of trained policies to reduce NFE and another improving training efficency.
A close controlled comparison is given in Appendix F, where the prior library is indexed by clusters computed in action space (K-means over action chunks) instead of latent actions, with the same architecture and training. This reaches 89.5% on LIBERO-90, against 82.6% for FM and 93.0% for LAFM. Clustering the targets helps, as COT-FM argues, and latent actions give a better partition because the mode must be predicted from the observation alone.
Since CSFM, CAR-Flow, and VCT are evaluated on image generation, an empirical comparison would require reimplementing them for robot policies. The closest control is FM + LP (Table 3, added in the revision), which learns a single Gaussian source jointly with the policy. It reaches 84.3% on LIBERO-90, and its training collapses without the KL regularizer, consistent with the degenerate solutions described by CAR-Flow.