BFMTrack: Behavior Foundation Models for Physics-based Motion Tracking

Behavior Foundation Models (BFMs) offer a promising path toward universal physics-based control by learning a latent space of physically plausible behaviors from reward-free interactions, regularized toward large-scale motion data. Each latent specifies a stationary policy, and existing BFM inference procedures map a single task, such as goal-reaching or a state-based reward, to a single latent. Time-varying objectives such as motion tracking instead require a sequence of latents. Current BFM tracking heuristics obtain this sequence by averaging goal embeddings over a look-ahead window, which fails to capture the nuances of highly dynamic motions. We propose BFMTrack, a test-time optimization method that refines a sequence of latents with simulator feedback while keeping the pre-trained BFM frozen. BFMTrack substantially improves tracking accuracy over zero-shot BFM inference and surpasses a general motion tracker trained with a dedicated tracking reward. Under external disturbances, BFMTrack also outperforms a specialized tracker trained on a single motion, while preserving the BFM’s natural recovery behavior. We further demonstrate BFMTrack on sparse references and deploy it on a real humanoid robot.