Coherence-Guided Factorized Flow Policy for Efficient Humanoid Whole-Body Control

Flow-matching policies can represent complex, multimodal action distributions, providing an expressive policy model for humanoid whole-body control including locomotion, balance, and physical interaction. However, existing methods commonly optimize all joints within a single flow network, where gradients from heterogeneous body parts can interfere, obscuring reliable improvement directions and constraining effective policy updates. To address this problem, we propose Factorized Flow Policy Optimization (FFPO), which decomposes the whole-body flow policy into functional body agents and improves them sequentially while retaining parallel action generation during execution. Within this sequential process, likelihood-ratio prefixes account for preceding agents’ policy updates when evaluating each body update. And body-wise gradient coherence, estimated using a gradient-SNR proxy, guides the allocation of a fixed optimization-epoch budget. FFPO then synchronously rescales the resulting candidate parameter updates according to their aggregate log-ratio magnitude, allowing reliable local improvements to accumulate under a controlled joint policy update. To implement these updates efficiently, FFPO uses a first-order flow-ratio estimator for the prefixes and a differentiable flow-matching proxy for current-agent gradients. Our theoretical analysis characterizes the policy-update cost of uncertain cross-body interactions and establishes conditions under which body-wise factorization yields greater expected improvement. Across 16 simulated locomotion, balance, and loco-manipulation tasks, FFPO improves mean normalized reward AUC by 14.4% and mean normalized final reward by 14.1% over the strongest evaluated baseline for each metric. Deployment on the Unitree G1 further demonstrates coordinated whole-body motion on hardware.