RedFlow: Redirect Failure into Action-level Corrections for Flow-matching VLA Policy

Reinforcement learning (RL) can improve Vision-Language-Action (VLA) policies through deployment experience, but reward- and preference-based RL primarily identifies desirable behaviors without specifying how to correct failed actions, leaving failure trajectories underutilized and limiting sample efficiency. Can such corrections be derived from fixed rollouts? Our key insight is that rollouts with different outcomes may contain action chunks executed in similar states, allowing higher-quality chunks to serve as locally supported corrective targets. Building on this insight, we introduce **RedFlow**, an offline post-training method for flow-matching VLA policies. *Execution-Context Matching* groups chunks using a compact representation of estimated task progress and the robot's proprioceptive state. *Quality-Guided Action Redirection* assigns signed quality scores to chunks and aggregates higher-quality chunks into corrective targets, reinforcing high-quality chunks, suppressing low-quality chunks, and redirecting correctable chunks toward their targets. RedFlow requires neither human intervention nor online data collection during post-training. Across four LIBERO suites, RedFlow improves the average success rate from 56.2% to 68.2%, outperforming the strongest evaluated offline baseline, AWR (62.3%), by 5.9 percentage points. Across three real-robot tasks, it improves the average success rate from 56.7% to 74.7%. On LIBERO-Spatial, RedFlow achieves a success rate of 75.8% with 1,536 fixed rollouts, while the evaluated online methods require 8.7–16 times as many fresh post-training rollouts to reach the same threshold.