acceptodds
Under review as a conference paper at ICLR 2027

BFMTrack: Behavior Foundation Models for Physics-based Motion Tracking

Abstract

Behavior Foundation Models (BFMs) offer a promising path toward universal physics-based control by learning a latent space of physically plausible behaviors from reward-free interactions, regularized toward large-scale motion data. Each latent specifies a stationary policy, and existing BFM inference procedures map a single task, such as goal-reaching or a state-based reward, to a single latent. Time-varying objectives such as motion tracking instead require a sequence of latents. Current BFM tracking heuristics obtain this sequence by averaging goal embeddings over a look-ahead window, which fails to capture the nuances of highly dynamic motions. We propose BFMTrack, a test-time optimization method that refines a sequence of latents with simulator feedback while keeping the pre-trained BFM frozen. BFMTrack substantially improves tracking accuracy over zero-shot BFM inference and surpasses a general motion tracker trained with a dedicated tracking reward. Under external disturbances, BFMTrack also outperforms a specialized tracker trained on a single motion, while preserving the BFM’s natural recovery behavior. We further demonstrate BFMTrack on sparse references and deploy it on a real humanoid robot.

Primary area:
reinforcement learning
Keywords:
Behavior Foundation Models, Motion Tracking, Humanoid, Deep Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GQoyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

BFMTrack keeps the pretrained BFM frozen and optimizes a latent sequence for each reference using parallel simulator rollouts and policy gradients. Temporally correlated noise encourages smooth, coherent motions. This improves tracking over zero-shot BFM inference, reducing MPJPE, and also supports sparse keyframe tracking.

Critical strengths
  1. Improves motion-tracking performance by optimizing latent sequences at test time, without fine-tuning the pretrained BFM.
  2. Demonstrates strong performance on a real humanoid robot.
Critical weaknesses

BFM-Zero provides a reusable skill space for downstream tasks, with reward inference as a core capability. Motion tracking is a special case that maximizes discounted rewards for matching reference states. BFM-Zero also shows that the pretrained space can contain better behaviors than zero-shot inference finds.

If the goal is to find better latents, the evaluation should extend beyond tracking to tasks such as reward inference, and include deeper latent-space analysis and comparisons with RL- or MPC-based methods. If the focus is time-varying tasks, the authors should explain why methods such as ReLA and LoLA cannot be extended to optimize latent sequences.The paper also does not clarify whether the tracking gap comes from zero-shot inference itself or from the choice of tracking reward—for example, whether a delta-based reward is less suitable than the reward used by PHC. Without addressing these questions, I am not yet convinced that the current contribution is sufficiently broad or substantial.

Questions for authors
  1. GPC shows strong fall recovery, so why doesn’t the generalist tracker pretrained with GPC achieve similar recovery performance?
  2. The ER baseline’s EMD seems much higher than the value reported in BFM-Zero. Is this only due to a change in the motion dataset?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer GQoy (1/3)

We thank the reviewer for the review. All section, table and figure numbers refer to the revised PDF.

W1. Relation to BFM-Zero. We agree that prior work has shown that the latent space of a BFM contains better behaviors than zero-shot inference finds. We do not claim this observation and credit Sikchi et al. and BFM-Zero for it (Sec. 2, Sec. 5.1). We would like to push back on one point: motion tracking is not a special case of reward inference. Each latent specifies a stationary policy, and reward inference maps a single state-based reward to a single latent. An ideal tracking reward favors being at the reference pose gtg_t at time tt. This reward is non-stationary, and since the BFM's state excludes time and the reference, it has no projection onto the reward family of the BFM (Sec. 3). Neither a single latent nor the value of a stationary policy captures the objective. ER instead solves a stationary surrogate of the tracking problem at every step. With L = 1, its policy maximizes the discounted goal reward of the next pose. It reaches and holds that pose, with no incentive to arrive on time or to anticipate what follows. Consistent with this, L = 1 is not the most accurate choice: averaging over the next L = 5 poses lowers the MPJPE from 5.10 cm to 4.72 cm (App. G.1, Fig. 10). In both cases, the latents are inferred from the reference alone and are never corrected by the motion they produce. Dedicated trackers make the problem stationary by conditioning the policy on the reference. A BFM cannot do the same and keep its reward inference for stationary rewards, because these come without a reference and every policy would then require one as input. BFMTrack instead keeps the BFM frozen and optimizes the latent sequence against the tracking return measured in simulation (Sec. 4). Each latent is credited with the tracking reward from its own step onward, so it is optimized to reach its pose on time and to anticipate the next ones.

Prior work on adapting latents covers three of four cases:

Nominal dynamicsDynamics shift
Single latent, stationary rewardLoLA / ReLA (Sikchi et al.), BFM-Zero (CEM, selected reward tasks)BFM-Zero (one pose, added payload)
Latent sequence, motion trackingthis paperBFM-Zero (one motion, altered friction)

BFM-Zero describes its experiments as "a preliminary investigation of test-time adaptation" and calls for "a more thorough understanding of fast adaptation". It evaluates its update of a latent sequence on one leaping motion under altered friction. We study the remaining case on 990 held-out clips and compare with the prior updates in W2a. The introduction now states this delimitation.

AuthorsOpenReview ↗

Reply to Reviewer GQoy (2/3)

W2a. Scope, and extending ReLA and LoLA. Our focus is the second case the reviewer describes: objectives that vary over time, with motion tracking as the task. Within this scope, the paper compares against two RL-trained trackers (a generalist trained with the same reward and a per-motion specialist) and against the sampling-based update that BFM-Zero applies to a latent sequence "in the spirit of DIAL-MPC" (Sec. 5.3, App. F). On the full test split, this update reduces the MPJPE of ER (L = 5) from 4.7 cm to 3.4 cm, and BFMTrack reduces it to 2.5 cm at the same rollout budget (Table 16).

ReLA and LoLA both build on the BFM's value ψ(s,a,z)⊤zr\psi(s,a,z)^\top z_r, the return of the stationary policy πz\pi_z for a fixed task latent zrz_r. Neither part carries over to tracking: a time-varying reward has no fixed zrz_r, and the state-action value of a latent sequence would have to depend on time and on all remaining reference poses. ReLA relies on this quantity directly, so it is not applicable in our setting. LoLA has a purely Monte Carlo version that skips the state-action bootstrap at the end. As a new baseline we apply this Monte Carlo estimator to the whole latent sequence at once, crediting every latent with the episode return. On the hardest subset, it does give some gains over ER latents, but performs worse than BFMTrack, even when using pink noise (App. G.7, Table 10):

Update of the latent sequenceEMDMPJPEMMPJPE
None (ER with L = 5)3.1111.4 cm84.8 cm
Monte Carlo estimator of LoLA2.7710.0 cm78.7 cm
Monte Carlo estimator of LoLA, with pink noise2.468.6 cm75.2 cm
BFMTrack2.076.8 cm63.4 cm

We are happy to run this comparison on the full test split for the final version.

W2b. Zero-shot inference vs. reward gap. Both contribute, and each of the two rewards suits one of the two methods. The ideal per-step tracking reward is a delta function at the reference pose. Zero-shot inference treats this reward as stationary (W1), so the latent it finds is the one that reaches and holds the pose, which is not necessarily the one that tracks best. Within this limit, the delta reward suits inference, because it has the closed form z = B(g) (Sec. 3), which ER averages over the next L poses. The PHC reward has no closed form. Inferring from it by least squares over 500,000 states from rollouts of the frozen BFM is less accurate than ER (EMD 5.27 vs 3.11 on the hardest subset, App. G.5). We attribute this to insufficient coverage near the reference: for a reference pose, the best of these states reaches a PHC reward of 0.54 on average on the hardest subset (maximum 1) and 0.75 on the full test split. For the optimization, it is the reverse. A rollout almost surely never hits the reference pose exactly, so the delta reward gives no signal. Its projection onto the rewards that the BFM represents, the goal-reaching reward, can be optimized but is less accurate than the PHC reward, which suggests that the projection loses information. A plausible reason is that the finite-dimensional embedding favors smooth rewards (Sec. 5.3). Sikchi et al. find the same gap for stationary rewards (§4.3 of their paper). The table gives the four cases on the hardest subset (Table 8, Fig. 16):

LatentsRewardEMDMPJPEMMPJPE
Zero-shot, ER (L = 5)Goal-reaching (delta)3.1111.4 cm84.8 cm
Zero-shot, reward inferencePHC5.2723.9 cm119.8 cm
BFMTrackGoal-reaching (projected delta)2.578.6 cm73.4 cm
BFMTrackPHC2.076.8 cm63.4 cm

Of the gap between ER and BFMTrack, 0.54 EMD comes from optimizing the latents instead of inferring them (row 3 against row 1) and 0.50 from the reward (row 4 against row 3). The full test split shows the same for the two rewards (Fig. 6, Table 15). We ran the inference baseline on the hardest subset and are happy to add the full test split for the final version.

AuthorsOpenReview ↗

Reply to Reviewer GQoy (3/3)

Q1. Fall recovery of GPC. GPC reports fall recovery for its generative controller, which samples skill tokens from the character's state, without a reference, and is free to choose a get-up skill after a fall (GPC §8.3, Fig. 6, Table 9). Our generalist reproduces GPC's tracking controller, the stage that follows a reference and that GPC evaluates on tracking accuracy only. In the revised paper, we train it with fallen-state initialization and a 2 s termination grace period (Sec. 5, App. F). It then recovers much better: its MPJPE increase under strong pushes falls from 15.1 cm to 5.6 cm (BFMTrack: 2.4 cm), and it gets up again after the box impact of Fig. 5 (see also the new clips in the supplementary material). App. G.4 compares the generalist with and without these additions.

Q2. EMD of the ER baseline. BFM-Zero computes its EMD on the 29 joint angles of the Unitree G1, whereas ours is computed on stacked positions and orientations of points along the SMPL humanoid, as done in Tirinzoni et al. (2025). App. B now also states this difference. Closest to BFM-Zero's EMD for us is computing it over the 69 SMPL joint angles. Under this EMD, our ER baseline scores 0.85 and BFMTrack 0.62 (Table 6).

Reviewer_GQoyOpenReview ↗

Response to rebuttal

I’m glad to see the additional comparisons. Although the scope is narrow, I think the problem formulation makes sense, particularly its focus on finding an optimal time-varying latent sequence. I will consider raising my score.

Reviewer_7v4WOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BFMTrack improves motion tracking by optimizing latent-command sequences for a frozen behavior foundation model. It uses simulator feedback, a policy-gradient update, and temporally correlated exploration, with tests in simulation and qualitative demonstrations on a robot.

Critical strengths
  1. A practical strength is that the method improves an existing BFM without retraining its policy.

  2. The optimizer comparison makes the reported advantage more convincing by holding initialization, reward, and rollout budget fixed.

Critical weaknesses
  1. The main contributions are not presented clearly enough. The introduction explains the task and performance gains, but does not clearly distinguish new methodological ideas from extensions of existing latent adaptation, or from the empirical findings. This makes the specific advance over LoLA and BFM-Zero difficult to assess.

  2. Motion quality between sparse targets is not fully assessed. Section 5.4 acknowledges incoherent gaits, but position errors measure only the retained target frames, while EMD ignores temporal order. These metrics can miss unnatural or incoherent movement between targets.

Questions for authors
  1. Could the authors state the main contributions more explicitly in the introduction, clarify the specific methodological advances over LoLA and BFM-Zero, and identify the experiments supporting these contributions?

  2. Could the authors quantify motion continuity and gait quality between sparse target poses as reference sparsity increases? This would help clarify when the frozen BFM retains natural motion despite the gait failures acknowledged in Section 5.4.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer 7v4W (1/2)

We thank the reviewer for the review. Both points led to additions in the revision. All section, table and figure numbers refer to the revised PDF.

W1 / Q1. Contributions and advances over LoLA and BFM-Zero. The introduction now ends with an explicit contribution list, and each item names the section with its evidence:

  1. Method (Sec. 4). A per-timestep policy gradient with temporally correlated exploration over the latent sequence of a frozen BFM.
  2. Tracking accuracy (Sec. 5.1). On three hours of held-out motions, BFMTrack improves on zero-shot BFM inference with ER (MPJPE 4.72 cm to 2.51 cm) and outperforms a generalist tracker trained in the same experimental setting (3.92 cm).
  3. Robustness (Sec. 5.2). Under external disturbances, BFMTrack outperforms a specialist tracker trained on a single motion and retains strong robustness.
  4. Ablations and applications (Sec. 5.3 and 5.4). We ablate the tracking reward, the exploration noise, the gradient estimator, and the optimizer, and demonstrate BFMTrack on sparse references and a real humanoid robot.

Relation to prior adaptation. BFMTrack builds on prior work that adapts the latents of a frozen BFM (Sec. 2). This work covers three of four cases:

Nominal dynamicsDynamics shift
Single latent, stationary rewardLoLA / ReLA (Sikchi et al.), BFM-Zero (CEM, selected reward tasks)BFM-Zero (one pose, added payload)
Latent sequence, motion trackingthis paperBFM-Zero (one motion, altered friction)

Sikchi et al. show that zero-shot inference is suboptimal for stationary rewards and adapt a single latent. BFM-Zero also adapts a single latent on selected reward tasks, and it compensates a dynamics shift in two examples, one of them a latent sequence with a sampling-based update. Whether adapting a latent sequence improves tracking when the dynamics are unchanged has not been studied.

Methodological advance. The table compares the updates of the latent sequence on the hardest subset, at the same rollout budget, warm start and reward (EMD, lower is better, Table 10):

Update of the latent sequenceEMD
None (ER with L = 5)3.11
Monte Carlo estimator of LoLA (episode return, white noise)2.77
Monte Carlo estimator of LoLA with pink noise2.46
Sampling-based update used by BFM-Zero, tuned2.48
BFMTrack (per-latent discounted return-to-go, pink noise)2.07

Relative to LoLA, the new elements are the per-latent discounted return-to-go and the temporally correlated exploration. Its Monte Carlo estimator, applied to a latent sequence, credits every latent with the episode return (App. F, App. G.7). Its critic bootstrap does not carry over, because it needs one fixed task latent, which a time-varying reward does not have. Relative to BFM-Zero, BFMTrack replaces the sampling-based update with a policy gradient (App. G.7). On the full test split, the sampling-based update reaches an EMD of 1.09, against 0.81 for BFMTrack (Fig. 7). The comparison with LoLA uses the hardest subset, and we are happy to run it on the full test split for the final version of the paper. The evidence is also broader. BFM-Zero showcases the adaptation of a latent sequence on a single motion under altered ground friction and describes it as "a preliminary investigation of test-time adaptation". We evaluate on 990 held-out clips under nominal dynamics, under external disturbances and with sparse references, and compare against a generalist and a specialist tracker.

AuthorsOpenReview ↗

Reply to Reviewer 7v4W (2/2)

W2 / Q2. Motion quality between sparse targets. Measuring motion quality is hard, especially with sparse references. Many motions pass through the retained frames, and the dense reference is only one of them, which BFMTrack need not choose. We add two metrics for the motion between the retained frames and also report the MPJPE over all frames (Sec. 5.4, Fig. 8).

Gait quality. A direct measure of gait quality is not available, so we use FID as a proxy (App. B). It compares the feature distributions of 1 s motion windows, about one gait cycle, from the policy and the reference. This comparison can judge motion where the rollout departs from the reference, as between sparse reference frames. The FID increases gradually as the reference becomes sparser. Down to 1.5 Hz, it stays below that of the generalist tracker with dense references, and at 1.0 Hz it is on par with it, which suggests that motion quality is broadly retained.

Motion continuity. We compare the jerk at the retained frames with that of the reference at the same frames. It peaks at 2.3 times that of the reference at 3.0 Hz and does not increase further as the reference becomes sparser.

As motion quality is hard to measure quantitatively, we encourage the reviewer to review the qualitative evidence presented in the supplementary videos.

Thank you for the response. I recognize that this work has practical engineering value. However, I will keep my original rating of Weak Rejection. The newly provided experimental analyses are somewhat limited in their support. Moreover, my second question regarding motion continuity and gait quality under sparse references has not yet been fully addressed. Besides, the points 2, 3 and 4 in the contribution list serve as empirical validations under different evaluation settings rather than separate methodological contributions. Presenting them as distinct advances tends to overstate the overall novelty of this work. My main concerns remain.

Dear Reviewer, 

Thank you for taking the time to go over our rebuttal and response. We are happy to hear that our work has practical engineering value!

To address your comment about the contribution list, we revised it such that it has only a single bullet for the extensive evaluations of BFMs around motion tracking in our work. The details about the experimental evaluation are summarized in a dedicated paragraph before the contributions. The updated list in the paper now reads as follows:

  • We propose BFMTrack, which optimizes the latent sequence of a frozen BFM with a per-timestep policy gradient and temporally correlated exploration (Sec. 4).
  • We provide thorough experimental evaluations of BFMTrack and existing BFM solutions in the motion tracking setting, including ablations of our design choices.
  • We demonstrate that BFMTrack improves tracking on sparse references and can be deployed on a real humanoid robot (Sec. 5.4, Fig. 1 bottom)

With regard to gait quality and motion continuity, we would like to re-emphasize that comparisons between sparse and dense reference are hard to quantify meaningfully. Nonetheless, to give a quantitative indication of the differences between gait patterns over different sparsities, we added an evaluation that considers the amount of same-foot steps (the cycle from the current left foot touch down to the next) relative to that same number when tracking a dense reference. We found that as the reference becomes sparser, both BFMTrack and its baselines match their dense counterparts more closely. This is likely because at increased sparsity the policies are less often forced to employ corrective steps to match the frames they should track. Please see Fig. 19 in Appendix I at the very end of the revised paper. We encourage the reviewer to review Timestamps 3:19 - 4:10 in the main supplemental video again to qualitatively assess the quality of the motions under sparse references.

Reviewer_RbBDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

BFMTrack improves physics-based motion tracking by optimizing a time-indexed sequence of latent inputs to a frozen Behavior Foundation Model. Starting from embeddings of reference states, it uses simulator rollouts, a tracking reward, and a REINFORCE-style update with temporally correlated exploration. Experiments cover 990 held-out AMASS clips, disturbance robustness, sparse references, and qualitative deployment on a humanoid robot. The paper provides convincing evidence that zero-shot latent inference leaves substantial tracking performance unused. My main concerns are the incremental methodological contribution, the incomplete justification of the optimization update, and whether the comparisons isolate the proposed method’s advantages.

Critical strengths
  1. A clear and useful empirical result. On the full test set, BFMTrack reduces MPJPE from 4.72 cm for zero-shot inference and 4.05 cm for the generalist tracker to 2.51 cm. Because the zero-shot baseline shares the same frozen BFM, this comparison cleanly demonstrates the value of latent sequence optimization.

  2. Substantial experimental coverage. Evaluation on 990 clips is stronger than a demonstration centered on selected motions. The appendix provides numerical results, implementation details, optimization-budget analysis, and ablations of the reward, noise spectrum, and optimizer.

  3. Promising robustness results. On the difficult subset under strong pushes, BFMTrack achieves 9.2 cm MPJPE, compared with 29.0 cm for the generalist and 33.5 cm for the specialist. This supports the practical value of retaining a broadly trained feedback controller while adapting its latent inputs.

  4. Good reporting of limitations. The authors acknowledge offline computation, limited evaluation across BFM families, and degraded motion coherence with sparse references. They also report that the specialist remains substantially more accurate without disturbances

Critical weaknesses
  1. The novelty over prior latent adaptation is relatively narrow. Frozen-BFM latent optimization already appears in Sikchi et al., while BFM-Zero already describes sampling-based optimization over latent sequences. The contribution therefore rests on the particular sequence optimizer and the breadth of its evaluation. The paper should state this distinction more precisely. Its strongest contribution is currently empirical rather than conceptual.

  2. The update requires a more careful mathematical account. Equation (6) uses marginal Gaussian scores and return-to-go, while the sampled perturbations are temporally correlated. For correlated Gaussian sequences, the joint score generally involves the inverse temporal covariance. Moreover, the usual justification for dropping past rewards from a score-function estimator relies on independence conditions that correlated perturbations do not satisfy. The authors acknowledge biased updates in Section 4, which is helpful, but do not characterize the resulting direction. It is therefore unclear how closely this algorithm corresponds to REINFORCE for the stated objective. Equation (4) also specifies an undiscounted objective, whereas the update uses discounted returns. These issues do not invalidate the empirical results, but they weaken the algorithmic explanation.

  3. Robustness comparisons confound the learning approach with recovery exposure. The BFM explicitly receives fallen-state initialization during pre-training. The trackers instead initialize from reference states and terminate training episodes when tracking error becomes large. These choices plausibly explain much of the robustness gap. The results establish an advantage over the implemented baselines, but do not isolate whether that advantage requires BFM pre-training. A tracker trained with comparable fallen-state exposure, recovery opportunities, or disturbance augmentation would be a more informative control.

  4. The optimizer comparison needs stronger tuning evidence. The DIAL-MPC-inspired baseline receives an equal rollout budget, but uses a different initial noise scale and an annealing schedule whose sensitivity is not reported. Its update is also modified substantially for this setting. The result supports superiority over this particular implementation; it does not yet establish a general advantage of policy-gradient latent optimization over sampling-based alternatives.

  5. The evidence supports narrower claims about naturalness and robustness preservation. Freezing the policy preserves its parameters, but arbitrary switching between its latent-conditioned behaviors does not guarantee natural or robust trajectories. EMD measures similarity between pose distributions and cannot establish temporal coherence. The sparse-reference failure involving repeated steps with the same foot illustrates this distinction. BFMTrack also degrades more than ER under strong pushes: MPJPE increases by 2.4 cm versus 1.0 cm, although its absolute error remains lower. “Retains strong robustness” is better supported than an unqualified claim of preserving robustness.

Questions for authors
  1. Can you derive the expected update under correlated noise and explain its relationship to the objective in Equation (4)? How does it compare with a joint-score estimator?
  2. How much of the robustness gap remains when trackers receive comparable recovery-state exposure?
  3. How sensitive is the DIAL-MPC-inspired baseline to noise scale, temperature, and annealing? Can you report a tuned comparison?
3 comments
AuthorsOpenReview ↗

Reply to Reviewer RbBD (1/3)

We thank the reviewer for the detailed and constructive review. We answer each weakness together with the related question. All section, table and figure numbers refer to the revised PDF.

W1. Novelty over prior latent adaptation. We agree with the reviewer's reading. BFMTrack builds on prior work that adapts the latents of a frozen BFM (Sec. 2), and the revision states the distinction more precisely (Sec. 1, related work in Sec. 2). Prior work covers three of four cases:

Nominal dynamicsDynamics shift
Single latent, stationary rewardLoLA / ReLA (Sikchi et al.), BFM-Zero (CEM, selected reward tasks)BFM-Zero (one pose, added payload)
Latent sequence, motion trackingthis paperBFM-Zero (one motion, altered friction)

Sikchi et al. show that zero-shot inference is suboptimal for stationary rewards and adapt a single latent. BFM-Zero also adapts a single latent on selected reward tasks, and it compensates a dynamics shift in two examples, one of them a latent sequence with a sampling-based update. BFM-Zero itself describes this as "a preliminary investigation of test-time adaptation" and calls for "a more thorough understanding of fast adaptation". We take up that direction for tracking. Whether adapting a latent sequence improves tracking when the dynamics are unchanged has not been studied. Our contribution is (i) a policy gradient over a time-indexed latent sequence with per-latent returns and temporally correlated exploration, and (ii) the evidence: on 990 held-out clips under nominal dynamics, it reduces the MPJPE of the zero-shot baseline (ER with L = 5) from 4.72 cm to 2.51 cm, is more accurate than a generalist tracker trained with the same reward, and retains strong robustness under pushes.

W2 / Q1. Mathematical account of the update. We agree with the reviewer's analysis. Sec. 4 now states that the update is not an unbiased gradient estimate; App. C.3 derives its expected direction, and App. G.6 compares it with the joint-score estimator (Table 9). The full derivation is in App. C.3, and we summarize it here in three steps from the objective in Eq. 4 to the update in Eq. 6.

For one coordinate of the latent sequence, let z=μ+σϵ\mathbf{z} = \boldsymbol{\mu} + \sigma\boldsymbol{\epsilon} with ϵ∼N(0,Σ)\boldsymbol{\epsilon} \sim \mathcal{N}(0, \Sigma), where Σ\Sigma is the temporal covariance of the noise, and let JσJ_\sigma be the expected return of Eq. 4 under this noise. The statements below hold in expectation over the noise and for a plain gradient step, without Adam and the projection onto the hypersphere.

(1) Full return. With the full return, the unbiased estimator uses the joint score Σ−1ϵ/σ\Sigma^{-1}\boldsymbol{\epsilon}/\sigma, as the reviewer notes. BFMTrack keeps the marginal score ϵt/σ\epsilon_t/\sigma of each latent, as colored-noise PPO (Hollenstein et al., 2024) and gSDE (Raffin et al., 2022) do for temporally correlated action noise. Its expected update is then Σ ∇Jσ\Sigma\,\nabla J_\sigma. Since Σ\Sigma is positive definite, this is an ascent direction on the same objective, and for pink noise it favors temporally smooth changes of the latent sequence.

(2) Return-to-go. Dropping the rewards before step tt requires that they are independent of ϵt\epsilon_t. The reviewer is right that this does not hold under correlated noise. As such, our update need not be an ascent direction.

(3) Discount. Eq. 4 is undiscounted, whereas Eq. 6 weights the reward at step jj by γj−t\gamma^{j-t} with γ=0.97\gamma = 0.97. Schulman et al. (2016) treat γ\gamma as "a variance reduction parameter in an undiscounted problem", "at the cost of introducing bias", and Sec. 4 now states that the discount is not part of the objective.

We chose our version of the update because of its precedent in the RL literature and because it yields the best performance. The table gives the EMD on the hardest subset when the score and the return are exchanged, with all other settings unchanged (Table 9):

Discounted return-to-goFull return
Marginal score2.07 (BFMTrack)2.46
Joint score2.702.85

The joint score with the full return is the unbiased estimator, and it is the least accurate. We attribute this to its variance: pink noise explores the high-frequency directions of the latent sequence only weakly, and the joint score rescales every direction by the inverse of its noise variance.

AuthorsOpenReview ↗

Reply to Reviewer RbBD (2/3)

W3 / Q2. Robustness and recovery exposure. We agree that this was a confound and ablated it in both directions.

Tracker with recovery exposure. The generalist of the paper is now trained with recovery-state exposure comparable to that of the BFM, in all experiments (Sec. 5, training details in App. F). 10% of its episodes start from fallen states, generated as in BFM pre-training, and the termination grace period is 2 s, so that it has time to recover. Architecture, data and reward are unchanged. Its accuracy without disturbances remains similar (MPJPE of 3.9 instead of 4.1 cm on the test split, Table 12). App. G.4 also shows that these additions make the generalist harder to train, and that the grace period needs a schedule.

BFM without fall initialization. We pre-trained a second BFM without fallen-state initialization and with one latent per episode instead of a new latent every 3 s. With ER, it is 0.3 cm less accurate on the test split (App. G.4).

The table gives the MPJPE on the hardest subset, with the increase over the undisturbed runs in parentheses (Fig. 14, Table 14):

UndisturbedStrong pushesSmall pushes
Generalist of the submission13.8 cm29.0 cm (+15.1)19.3 cm (+5.5)
Generalist with recovery exposure12.2 cm17.8 cm (+5.6)13.5 cm (+1.4)
BFMTrack6.8 cm9.2 cm (+2.4)8.2 cm (+1.3)
BFMTrack, BFM without fall initialization7.8 cm10.3 cm (+2.5)9.2 cm (+1.4)

BFMTrack achieves the lowest error in both regimes (Fig. 4). Recovery exposure closes much of the gap for the tracker, at the cost of much slower training. In contrast, the BFM's robustness does not depend on fall initialization. We attribute it to BFM pre-training, during which the policy is exposed to a diverse state distribution across many behaviors (Tirinzoni et al., 2025). Sec. 5.2 now states both results.

The specialist follows DeepMimic (Peng et al., 2018), which reports reference-state initialization and early termination as key components for dynamic skills (their Table 5). Since the grace period already makes the training of the generalist much slower, we did not apply the same changes to the per-clip specialist.

AuthorsOpenReview ↗

Reply to Reviewer RbBD (3/3)

W4 / Q3. Optimizer comparison. We had tuned the sampling-based baseline on the hardest subset before submission. It is now visualized in Fig. 18, and the best configuration does not change. We are happy to extend this tuning to the full test split for the final version.

Modifications. The baseline is our implementation of the trajectory adaptation of BFM-Zero, which its authors describe as "using a dual-loop annealing schedule in the spirit of DIAL-MPC". BFM-Zero reports the annealing parameters, but not the temperature, the noise scale or the weighting of the samples, and its public code does not include the adaptation. We therefore use the reported annealing parameters at the rollout budget of BFMTrack and tune the rest. The baseline is not DIAL-MPC itself, an online controller for raw actions that re-plans at every control step. This version would require a much larger computational budget because of the re-planning, and we suspect that this is also not what BFM-Zero does. DIAL-MPC gives each sampled trajectory one weight. We weight each latent by its own discounted return-to-go, as in BFMTrack, because a clip of more than 1,000 steps otherwise yields one weight per sampled sequence instead of one per latent. At the configuration of Table 5, one weight per sequence reaches an EMD of 2.85 on the hardest subset, against 2.48 with per-latent weights, so this choice favors the baseline. App. F and Table 10 now show this.

Scope. We agree that one baseline does not establish a general advantage of policy gradients over sampling-based optimization, and we do not claim one. Our claim is the one the reviewer states: at the same rollout budget, warm start and reward, BFMTrack is more accurate than this update, also after tuning it.

W5. Claims about naturalness and robustness. We agree with these points and answer them in turn.

(1) A frozen policy does not guarantee natural or robust trajectories. We agree. Motion quality can only be judged by considering a longer sequence at once. We add FID as a metric (App. B). It compares the feature distributions of 1 s motion windows from the policy and the reference. This comparison can judge motion where the rollout departs from the reference, as after a push or between sparse reference frames. On the hardest subset (Fig. 4, Table 7):

UndisturbedStrong pushesSmall pushes
ER (L = 5)8.9517.369.16
Generalist13.0131.3914.74
Specialist2.7582.8249.80
BFMTrack5.8015.466.45

Under both push regimes, BFMTrack stays closest to the reference motions, for each of the 256 push schedules, also closer than ER, which uses the same BFM. Measuring motion naturalness and quality with metrics alone is hard, so we encourage the reviewer to also consider the qualitative evidence we present in the attached supplementary videos.

(2) EMD does not enforce temporal order. We agree. However, in our results, EMD agrees closely with a frame-wise distance and with DTW over the same features, which respect temporal order (Table 6). On the full test split, frame-wise / EMD / DTW are 1.59 / 1.52 / 1.51 for ER and 0.82 / 0.81 / 0.81 for BFMTrack, and across clips EMD and DTW correlate with r > 0.99. This is likely the case because we compute EMD on rich features (body positions and orientations).

(3) BFMTrack degrades more than ER under strong pushes. We agree, and Sec. 5.2 now states it (2.4 vs 1.0 cm). Over the optimization, BFMTrack becomes more accurate both with and without pushes, and the difference between the two grows (App. G.3, Fig. 12). We attribute this to the highly dynamic motions of the hardest subset: the closer the character follows them, the less margin it has to absorb a push. Even under strong pushes, BFMTrack remains more accurate than ER without pushes (9.2 vs 11.4 cm).

(4) Same-foot steps with sparse references. These occur between widely spaced retained frames, at about one retained frame per second, where no reference specifies the gait (walking example in the supplementary video). This is a limitation of the sparse-reference application (Sec. 5.4) and does not concern tracking with dense references. Even in sparse reference tracking (Fig. 8), down to 1.5 Hz, the FID stays below that of the generalist tracker with dense references, which suggests that motion quality is broadly retained.

Claims. We agree that the claims should be narrower and adopt the reviewer's wording. The abstract, introduction, and conclusion now say that BFMTrack "retains strong robustness" (marked in blue).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reviews. We are glad that they found the paper clear and easy to follow (RbBD, 7v4W, GQoy) and see practical value in improving a pre-trained BFM without retraining its policy (7v4W, GQoy). They also point to the clear empirical result on 990 held-out clips and the robustness under pushes (RbBD), the controlled optimizer comparison (7v4W), and the demonstration on a real humanoid robot (GQoy).

We uploaded a revised paper with all changes in blue. We first address the two concerns that reviewers share and then summarize our answers and changes to the remaining points. Individual more detailed answers are provided as direct answers to the reviewers. Since motion quality is hard to judge from metrics alone, we invite the reviewers and the area chair to also consider the qualitative evidence we provide in the supplementary video: tracking with and without box impacts, and the sparse references and real robot applications. Two new videos show the retrained generalist (more in 2.).

1. Contribution relative to LoLA, ReLA and BFM-Zero (RbBD-W1, 7v4W-W1/Q1, GQoy-W1/W2). BFMTrack builds on prior work that adapts the latents of a frozen BFM, and the revised Sec. 1 and Sec. 2 now state the differences explicitly: LoLA and ReLA adapt a single latent for a stationary reward. The tracking reward is non-stationary and has no projection onto the reward family of the BFM (Sec. 3), so motion tracking requires a sequence of latents. BFM-Zero adapts a latent sequence with a sampling-based update on one motion under altered ground friction, whereas we evaluate on 990 held-out clips under nominal dynamics. BFMTrack optimizes the sequence with a policy gradient. Unlike LoLA, which credits its single latent with the return of the whole rollout, BFMTrack credits each latent with the tracking reward from its own step onward and explores with temporally correlated noise (Sec. 4). The revision compares to both prior updates (App. G.7, Table 10). On the hardest subset, both prior updates improve EMD but stay less accurate than BFMTrack:

Update of the latent sequenceEMD
None (ER with L = 5)3.11
Monte Carlo estimator of LoLA2.77
Sampling-based update used by BFM-Zero, tuned2.48
BFMTrack2.07

2. Recovery exposure in the comparison under disturbances (RbBD-W3/Q2, GQoy-Q1). To remove this confound, we retrained the generalist tracker with the fallen-state initialization of the BFM and a termination grace period of 2 seconds to improve recovery behavior. It is much more robust, but the conclusions hold: BFMTrack remains more accurate than the generalist without disturbances and has the lowest error of all methods under pushes. Conversely, a BFM pre-trained without fallen-state initialization shows almost the same error increase under pushes – it learns recovery either way. The specialist is unchanged, because DeepMimic reports reference-state initialization and early termination as important for learning dynamic motions. The updated results are in Fig. 3 to Fig. 5, and the details in App. F and App. G.4. The supplementary material includes two extra videos: one shows a side-by-side comparison of the old and new Generalist Tracker, the other all clips in the Presentation video that we will change in the final revision of this paper.

Summary of the remaining points

  • Update (RbBD-W2/Q1): Sec. 4, App. C.3, App. G.6. The update is not an unbiased gradient estimate (derivation in App. C.3) but beats ablations in practice (Table 9).
  • Sampling-based baseline (RbBD-W4/Q3): Fig. 18. shows sweep over baseline hyper-parameters. Main results unchanged.
  • Robustness claims (RbBD-W5): We adopt the "retains strong robustness" phrasing as BFMTrack MPJPE suffers more under pushes than ER.
  • EMD and temporal order (RbBD-W5, 7v4W-W2): App. G.2. A frame-wise distance and DTW agree with EMD, so EMD does not hide timing errors in our results.
  • Motion quality under pushes (RbBD-W5): Fig. 4, App. G.3. In the FID of motion windows, BFMTrack stays closest to the reference motions under both push regimes, also closer than ER with the same BFM.
  • Motion between sparse frames (7v4W-W2/Q2): Sec. 5.4, Fig. 8. In the sparse setting, down to 1.5 Hz, the FID of motion windows stays below that of the generalist tracker with dense references. The jerk at the retained frames is about twice that of the reference but does not grow as reference becomes sparser.
  • Inference or reward (GQoy-W2): Sec. 5.3, App. G.5. Zero-shot reward inference with the PHC reward is less accurate than ER latents, and BFMTrack improves on ER with either reward.
  • EMD of BFM-Zero (GQoy-Q2): App. B. BFM-Zero computes its EMD over joint angles. Over the SMPL joint angles, ER scores 0.85 and BFMTrack 0.62.

Some of the new ablations use the hardest subset of the test split (30 clips), and we are happy to run them on the full test split for the final revision of this paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.