acceptodds
Withdrawn from ICLR 2027

Can Text-to-3D Human Motion Models Really Understand Physical Actions?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-to-3D human motion generation aims to translate natural-language intentions into 3D human motions, serving as an important bridge between language understanding and human behavior generation, with broad potential for digital humans, computer animation, and human–computer interaction. Existing text-to-3D human motion generation methods primarily rely on large-scale motion data to learn motion representations and generative distributions, while improving generation quality and motion naturalness through generative modeling, motion priors, or post-training optimization. However, these approaches fail to establish effective supervision for the underlying physical laws and latent physical distributions of human motion during training. As a result, the generated trajectories can still deviate substantially from the physical distribution governing real human motion. To address this problem, we propose MoPhys (Motion Physics Modeling), a time-frequency modeling and physics-expert collaborative framework for physically consistent text-to-3D human motion generation. Specifically, MoPhys employs multi-scale time-frequency interactions to capture rich motion variations across different temporal scales, and introduces physics-inspired experts constructed from body observables to adaptively refine generated motions. MoPhys further transforms the structural and temporal information in real motions into direct supervision for the generative model, guiding it to learn the physical constraints embedded in the training data and thereby producing motions that better conform to the physical distribution of real human motion. Extensive experiments demonstrate that MoPhys consistently improves the physical plausibility of generated motions across diverse data distributions and multiple human motion generation paradigms, validating both the effectiveness and generalizability of the proposed framework.

Primary area:
generative models
Keywords:
Text-to-Motion Generation, 3D Human Motion Generation, Time-Frequency Modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rsbmOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that text-to-3D human motion models can produce semantically correct motions without fully capturing the kinematic regularities of real human motion. It proposes MoPhys, a plug-in post-training framework with two components. Multi-scale Time-Frequency Modeling (MTFM) applies a two-level temporal Haar decomposition to frame-level backbone features, performs text-conditioned interaction across frequency bands, and writes the reconstructed update back through a residual branch. Physics-Inspired Expert Collaboration (PEC) extracts root-relative joint positions, bone vectors, velocities, and accelerations from generated motions and feeds them to seven expert networks. A router combines the expert outputs to predict bounded corrections to non-root joint positions. MoPhys is evaluated on HumanML3D, KIT-ML, and CMP across several generation paradigms and backbones. In addition to standard text-to-motion metrics, the paper introduces Kinematic Fréchet Distance (KFD) and Power-Spectrum Wasserstein Distance (PSWD). The reported results show consistent improvements in KFD, PSWD, and Matching across the main comparison settings.

Critical strengths

S1. The framework is evaluated broadly, and the ablation design includes sensible structural controls.

MoPhys is applied across several generation paradigms and datasets, which is valuable for a plug-in method. Tab. 2 also includes parameter-matched generic MLP experts and a flat temporal-attention alternative, providing sensible controls for added capacity and attention. The statistical significance of the resulting differences, however, is unclear due to the lack of uncertainty estimates.

Critical weaknesses

W1. The reliability of the main quantitative results cannot be established (Tab. 1, Fig. 3).

  • The Base MDM row differs substantially from the original MDM results on HumanML3D. Its FID (0.5366) is close to the published 0.544±0.0440.544 \pm 0.044, whereas its R@3 (0.7086) is substantially higher than the published 0.611±0.0070.611 \pm 0.007. The paper does not explain whether this difference comes from re-training, a different checkpoint, or a different evaluation protocol. This is not an isolated case: comparisons with the published results of several other baselines reveal similarly large discrepancies. In some cases, such as MotionGPT + PPO and KIT-ML MLD + SoPo, the reported post-trained baseline performs substantially worse than the corresponding backbone reported in the original literature, and MoPhys does not fully recover this gap. Without a clear explanation of how these baselines were obtained and evaluated, it is difficult to determine whether the reported improvements reflect the proposed method or differences in the underlying baseline setup.

  • The comparison protocol is also unclear. Fig. 3 shows that multiple post-training baselines were run for several backbones, whereas Tab. 1 reports only one comparison method per backbone. The paper does not explain how these baselines were selected. In addition, no repeated-run variance or confidence intervals are reported. Together, these issues make the main quantitative evidence difficult to assess reliably.

W2. The claimed improvement in “physical consistency” is not directly demonstrated.

  • PEC is trained on joint positions, bone vectors, velocities, and accelerations, while KFD and PSWD are computed from closely related velocity, acceleration, and spectral statistics. Improvements on these metrics are therefore closely aligned with the quantities optimized during training and are not independent evidence of broader physical plausibility.

  • More importantly, the artifacts used to motivate the paper—foot sliding, floating, and penetration—are never directly evaluated, and no actual before/after qualitative results are provided; Fig. 1 is explicitly a conceptual illustration. The method also models kinematics rather than forces, contacts, or dynamics. The current evidence therefore supports improved kinematic statistics more directly than the stronger claim of physical consistency.

Questions for authors
  • Were all results in Tab. 1 obtained using the same official evaluator and protocol? How were the baseline checkpoints obtained? Please explain why Base MDM, MotionGPT + PPO, and KIT-ML MLD + SoPo differ substantially from the published results of the same backbones. Repeated-run means and confidence intervals would also be important.

  • Why are only selected post-training baselines reported in Tab. 1 when Fig. 3 shows that additional methods were run for the same backbones? Please clarify the selection rule and whether MoPhys replaces these baselines or is applied on top of them.

  • Is the original backbone frozen during MoPhys training, as suggested by Fig. 2, or does it continue to receive updates from its native objective as stated in Sec. 3.2? If the backbone is updated, an equal-extra-training baseline without MoPhys would help isolate the contribution of the proposed modules.

  • Can the authors provide independent physical-plausibility evaluations, such as foot-skating/contact error, penetration, floating, or physics-based tracking? Actual before/after qualitative results would also help establish whether lower KFD and PSWD correspond to visibly more plausible motion.

Reviewer_37XrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MoPhys, a text-driven 3D human motion generation optimization framework that integrates Multi-scale Time-Frequency Modeling (MTFM) and Physics-Inspired Expert Collaboration (PEC) to improve the kinematic plausibility and temporal consistency of generated motions. Extensive experiments on HumanML3D, KIT-ML, and CMP across various generation models demonstrate that the proposed method effectively improves kinematic distributions, spectral statistics, and text-motion alignment.

Critical strengths
  1. This paper is clearly structured,
  2. MoPhys adopts a modular architecture that decouples multi-scale time-frequency modeling from physics-inspired motion refinement, enabling flexible integration with different motion generation paradigms while preserving the original backbone architecture, thereby offering good generality and extensibility.
Critical weaknesses
  1. Could the authors report physical plausibility metrics independent of the training objective, such as foot skating and ground penetration, along with the corresponding motion sequences, to better support the claim of physical consistency?

  2. How does MoPhys ensure that physical correction neither compromises the semantics of the original motion nor introduces new skeletal artifacts?

  3. Can the proposed method maintain physical plausibility under extreme motions, such as fast running or jumping jacks?

  4. Prior motion generation work (e.g., DeepPhase, MoFTSS) typically uses a high/low-frequency split. What motivates dividing the spectrum into three bands, and is the intermediate band empirically effective?

Questions for authors

See Weaknesses.

Reviewer_wzp4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies physical consistency in text-to-3D human motion generation. It proposes MoPhys, which combines multi-scale time-frequency modeling and physics-inspired expert collaboration to refine generated motions. Experiments are conducted across different motion generation backbones and datasets.

Critical strengths
  1. The paper is generally well-written and easy to follow.

  2. The reviewer appreciates the relatively extensive experiments across different backbones and datasets.

Critical weaknesses
  1. Based on the reviewer’s understanding, it is still somewhat unclear how well the reported improvements reflect actual physical plausibility. The paper mainly evaluates physical improvement using kinematic and spectral distribution metrics such as KFD and PSWD. The reviewer did not find direct evaluation of physical violations such as foot sliding, floating, penetration, or contact. This makes the physical improvement somewhat difficult to interpret.

  2. KFD and PSWD are important metrics for supporting the main conclusion of the paper. However, the reviewer did not find much validation showing how well these metrics correspond to actual physical quality or human perception. This concern is somewhat more noticeable since the improvements on conventional motion-generation metrics are not always consistent across all settings.

  3. If the reviewer understands correctly, the CMP experiment trains and evaluates the method under the CMP setting rather than directly transferring a model trained on HumanML3D or KIT-ML to CMP. Therefore, this experiment seems to mainly show that the proposed method can also be applied to a different motion distribution. It is somewhat unclear whether this is sufficient to support the stronger claim of cross-distribution generalization.

Questions for authors

Please see the weaknesses above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.