RAP: Recursive Action-Preview Flow Matching for Fast and Robust Visuomotor Control

Recent diffusion and flow-matching methods enable few-step action generation by initializing from temporally grounded sources derived from historical actions or previous action forecasts. However, source construction can differ between offline training and deployment: training sources rely on expert or ground-truth actions, whereas deployment sources come from the policy's own executed or predicted actions. This train–deployment source mismatch can make the learned source-to-action mapping unreliable as deployment sources deviate from the training source distribution, degrading policy performance and robustness. Adding Gaussian noise to temporal sources can improve robustness, but requires manual tuning of the noise scale. We propose Recursive Action-Preview Flow Matching (RAP) to align source construction between training and deployment. RAP carries forward the unexecuted part of the previous action forecast as an action preview to construct the source for the next action-generation cycle, while the latest observation guides its refinement. During training, RAP recursively constructs sources from detached policy-generated action previews, matching the source formation used at deployment without additional Gaussian noise injection. Across 8 simulated tasks from 4 benchmarks and 2 real-world tasks, RAP achieves strong performance with only 1 or 3 inference steps. Under large initial-pose perturbations, RAP improves success by 22–50 percentage points over noise-free baselines while remaining competitive with methods using Gaussian source perturbations. RAP combines fast action generation with robustness to both initial-pose changes and visual uncertainty.