- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Text2BFM, which generates text-conditioned motion in the policy-latent space of a frozen Behavioral Foundation Model (BFM) instead of directly generating poses. A text-aligned variational bottleneck compresses policy-latent sequences, and a conditional flow-matching model generates them from text. The method is evaluated on HumanML3D, KIT-ML, and long compositional prompts.
Critical strengths
S1. The paper presents a clear formulation for generating motion in an executable pretrained policy-latent space rather than directly in pose space.
S2. The compositional experiments show strong improvements in stage ordering and transition smoothness over the evaluated kinematic baselines.
Critical weaknesses
Most of these weaknesses are already acknowledged in the limitations section, but most of them are not sufficiently justified despite being critical to the paper’s main claims.
W1. As acknowledged in the limitations section, the paper does not compare kinematic generators tracked by the same frozen BFM or physics-based text-conditioned controllers under a shared executor. In addition, the Meta Motivo executor was pretrained on motions overlapping with the evaluation sets. Since the official Meta Motivo repository provides reproducible training code with training-from-scratch support, retraining the BFM while excluding overlapping evaluation motions appears feasible and would provide a cleaner evaluation, but this experiment is not included.
W2. The compositional evaluation lacks sufficient directly relevant baselines. Composition-specific methods are omitted even though long compositional motion generation is one of the paper’s main claims. The paper acknowledges this limitation, but does not sufficiently justify the omission of these directly relevant baselines.
Questions for authors
-
Could the authors provide stronger comparisons using the same BFM executor and evaluate a BFM trained without motions overlapping with the evaluation set?
-
Could the authors provide qualitative videos for the main ablations to better show the effect of each component?
3 comments
Thank you for the substantial additional experiments, particularly the BFM retraining and controlled comparisons. While these address several of my original concerns, Motion-Agent still achieves substantially better stage ordering and smoother transitions under the same BFM executor. Comparisons with physics-based text-conditioned controllers also remain missing. Thus, the advantages of direct policy-latent generation for long compositional motion remain insufficiently demonstrated. I therefore maintain my original rating.
Thank you for reading the new results.
Motion-Agent in composition. On the prompts of Table 4, Motion-Agent orders stages better than Text2BFM-Compose and has smoother transitions, also under the same BFM executor. Our composition result is stated only relative to the kinematic composition methods Kimodo and FlowMDM. Two differences remain relevant. Motion-Agent does not accept clause durations, while Text2BFM-Compose follows the requested duration of every clause. The prompts of Table 4 also come from the HumanML3D distribution on which Motion-Agent is trained. On prompts outside it (videos/comparison_grid), Text2BFM performs the requested skills and Motion-Agent falls back to common motions.
Physics-based text-conditioned controllers. These methods train their own controllers for their own simulators and humanoids, so they cannot be run on our BFM without retraining. We can relate our results to their published HumanML3D numbers. CLoSD reports an R-Precision Top-3 of 0.689 and an FID of 1.798, against 0.746 for real motion in its evaluation, and MoConVQ reaches 0.614 and 3.279 in the same table. The kinematic planner of CLoSD alone reaches 0.777 and 0.283, so the loss after physical execution appears there as well. Text2BFM reaches 0.876 and 1.172, against 0.797 for real motion. The evaluation setups differ in simulator, humanoid and feature conversion, so this comparison is indirect. We will add it to the paper.
We thank the reviewer for the careful reading. All changes in the revised PDF are marked in blue. In this response "the BFM" is the public Meta Motivo S-1 checkpoint unless we say otherwise.
W1, Q1. Comparison under the same BFM executor. This is done for five kinematic generators (last block of Table 3, Sec. 4.3).
The table shows where text alignment is lost. When real test motions are executed by the BFM, their R-Precision drops from 0.797 to 0.725. MoMask ends at the same level after execution (0.73). The drop is therefore caused by the step in which the BFM tracks a given motion, and a better kinematic plan cannot remove it. Text2BFM has no tracking step, because it generates the policy latents directly, and it reaches 0.876.
W1, Q1. A BFM trained without overlapping motions. The reviewer is right that the Meta Motivo training code is public, and we ran this experiment (new row in Table 2, details in Appendix I). We retrained the BFM on a list without the source sequences of the HumanML3D test motions (1296 of 6973 sequences removed), in both the S and the M-1 configuration, and retrained the bottleneck and the generator on the new latents (Table 6).
In both configurations R-Precision changes by at most 0.015, and FID and MM-Dist change by less than 0.02. R-Precision and MM-Dist stay better than those of every baseline (0.807 and 2.958 for MoMask). The new Table 12 supports this result. It evaluates all baselines on the 1420 test motions that the public BFM has not seen, and Text2BFM (M-1 checkpoint) keeps the best R-Precision there (0.830 against 0.777 for MoMask). For KIT-ML such a list cannot be built at present, because its motion ids have no published mapping to AMASS files. The revision notes this explicitly for the KIT-ML results.
W2. Composition baselines. We added FlowMDM, Motion-Agent and TEACH, and we evaluated them and Kimodo at N=4 (new Table 4, Appendix D). Text2BFM-Compose (Order Accuracy 0.671 / 0.509) is above Kimodo (0.45 / 0.17) and FlowMDM (0.53 / 0.40). The LLM-based Motion-Agent achieves higher Order Accuracy (0.91 / 0.88), also when it is executed by the same BFM (0.83 / 0.65). The abstract and Sec. 4.4 now state the result relative to the kinematic composition baselines. TEACH is trained on BABEL, so our prompts are outside its training domain (0.07 / 0.01, Appendix D).
Q2. Videos for the ablations. We added videos for the ablations to the supplementary archive (folder videos/comparison_grid, one video per prompt sequence) and describe them in Appendix J. Each video shows the same multi-clause prompt in six panels: the full model, the model without the semantic loss (no_sem), the model without temporal compression (no_compress), our generator without the BFM (vae_gen), and Motion-Agent as a kinematic output and tracked by the BFM. The prompts are out of distribution for HumanML3D, and on them Text2BFM follows the text visibly better than Motion-Agent. Without the semantic loss or without the compression, the model holds one-leg balances but does not reach the lotus pose or the standing split. Without the BFM it performs a handstand but stays upright for the breakdance and yoga clauses. Only the full model follows every clause in these sequences.
The folder videos/comparison_gravity adds videos with changed environment parameters, here Moon and Mars gravity. The generated program is executed by the BFM in the changed environment, so the resulting motion is physically consistent with that environment. The same mechanism applies to other physical parameters, such as wind and loads on the limbs. A kinematic generator alone produces the same trajectory for every environment. For comparison, the videos also show Motion-Agent tracked by the BFM under the same gravity.