Flowing With Purpose: Latent Action Guided Flow Matching Policies For Robotic Manipulation

Flow matching has recently become a new standard for behavior cloning in robotic manipulation. However, state-of-the-art flow matching policies suffer from a systematic structural mismatch: they rely on a globally fixed isotropic source distribution despite the strongly fragmented and heteroscedastic structure of robotic action spaces. This agnostic initialization forces the model to learn highly entangled vector fields, bottlenecking training efficiency and limiting overall policy performance. To address this limitation, we introduce Latent Action Guided Flow Matching (LAFM), a novel framework that replaces the monolithic Gaussian with an adaptive library of learned prior distributions. By grounding these distributions using a latent action model, LAFM maps current observations to discrete motion primitives, selecting a specialized base distribution that provides an informed, structurally aligned initialization for the denoising process. This dynamic adaptivity naturally accommodates heteroscedasticity in human demonstrations and makes transport trajectories shorter and less entangled, reducing the number of training steps required to reach the final error of standard flow matching by 32.5%. Empirically, LAFM substantially outperforms standard flow matching formulations, increasing task success rates by 23.4 percentage points in real-world robotic deployments, 10.4 points on LIBERO-90, and 12 to 15 points on the more challenging LIBERO-Plus and RoboTwin 2.0 benchmarks. Code and model weights will be publicly released upon acceptance of the publication.