acceptodds
Under review as a conference paper at ICLR 2027

PHYFLOW: PHYSICS-AWARE RECTIFIED FLOW WITH FINE-GRAINED SEMANTIC ALIGNMENT FOR TEXT- TO-MOTION GENERATION

Abstract

Text-to-motion generation strives to synthesize human motion sequences that are semantically matched natural-language descriptions while remaining realistic, diverse, and physically plausible. Recent latent diffusion approaches, such as ARDT, attained competitive results by generating motion within a compressed latent space and refining realism through adversarial training and temporal semantic contrastive learning. However, there are still some limitations in their semantic conditioning as it relies on a single pooled CLIP sentence embedding, which discards the phrase-level structure of the prompt before it reaches the generator. Additionally, the alignment objective operates at sequence granularity, matching one sentence to one motion, and therefore supervises which actions occur but not when they occur. Moreover, diffusion-based sampling requires a large number of denoising steps at inference. In addition, generated motions frequently exhibit kinematic artifacts, including foot sliding, bone-length drift, infeasible joint configurations, and temporal jitter. We propose PhysFlow, a semantic- and physicsaware rectified-flow framework for text-to-motion generation. PhysFlow retains the two-stage latent generation paradigm while redesigning the text-conditioned generator. In the first stage, a VAE motion sequences into a compact continuous latent representation and reconstructs them to motion space. In the next stage, latent diffusion is replaced by a Rectified Flow Transformer that learns a direct velocity field from the noise distribution to the motion latent distribution, permitting inference with substantially fewer sampling steps. Semantic conditioning is provided by a frozen Qwen3 text encoder, which yields token-level contextual representations in place of a single global embedding. An Action Parsing Module further extracts action phrases, body-part cues, directional modifiers, and temporal relations from the input text; the resulting structured semantic tokens are injected into the generator through cross-attention. To strengthen temporal grounding, we introduce Fine-Grained Semantic Alignment, which aligns action-level semantic tokens with latent motion frames rather than aligning a complete sentence with a complete sequence. Beyond semantic modeling, PhysFlow incorporates physical and distributional regularization. A Physics Guidance Module imposes differentiable penalties on foot-contact inconsistency, bone-length deviation, joint-limit violation, and temporal non-smoothness, thereby constraining generated motions toward kinematic stability. To narrow the distribution gap without reinstating adversarial training in the generator stage, we adopt non-adversarial Distribution Matching objectives comprising moment and maximum mean discrepancy regularization at both the latent level and the evaluator-feature level. The complete training objective combines rectified-flow matching, fine-grained semantic alignment, physics guidance, and distribution matching, and thus avoids the optimization instability associated with adversarial latent diffusion training. We evaluate PhysFlow on the HumanML3D and KIT-ML benchmarks. On HumanML3D, PhysFlow improves semantic retrieval relative to ARDT, attaining Top-1/Top-2/Top-3 R-precision of 0.567/0.756/0.842 and an MM-Dist of 2.802. On KIT-ML, it improves motion distribution quality, reducing FID from 0.334 to 0.278 while maintaining a diversity score close to that of real motion data. These results indicate that token-level language representation and action-level temporal alignment account for the gains in semantic fidelity, whereas physics guidance and distribution matching account for the improvement in distributional quality.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-motion, latent generation, physics-aware rectified, Rectified Flow, Semantic Alignment, Physics Guidance, Human Motion Synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5zHxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PhyFlow, a text-to-motion generation framework that integrates structured text conditioning, fine-grained semantic alignment, Rectified Flow, and physical constraints, aiming to improve semantic consistency, physical plausibility, and sampling efficiency.

Critical strengths

1.By introducing action-level semantic representations and a temporal alignment mechanism, the paper explicitly models the correspondence between individual actions in the text and motion segments, offering a new perspective on fine-grained motion control.

2.The method achieves notable improvements in semantic alignment metrics on HumanML3D, demonstrating its potential for better matching between generated motions and text descriptions.

Critical weaknesses
  1. The abstract is overly long and redundant, and the figures appear noticeably AI-generated.

  2. The physical constraints described in the paper are loss functions commonly used in motion generation; directly transferring them does not establish sufficient novelty.

  3. Built upon ARDT, the proposed method improves R-Precision, i.e., text–motion consistency, but at the cost of a considerable FID deterioration. The method fails to strike an effective balance between the two, so its contribution is not convincingly validated.

  4. While the proposed generation approach is effective, the paper lacks an efficiency comparison.

  5. The paper lacks sufficient standalone ablations and temporal alignment evaluation, making it difficult to verify the actual contribution of FSAM to fine-grained semantic control.

Questions for authors

see Weaknesses.

Reviewer_1WP8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PhyFlow, a semantic- and physics-aware rectified-flow framework for controllable text-to-motion generation. It uses a two-stage latent generation architecture and introduces structured text conditioning with action parsing and action-level alignment. Experiments report improvements in semantic alignment and motion generation quality over ARDT.

Critical strengths
  1. The method is simple and easy to flow.

  2. The method achieves high performance in some metrics, such as R-Precision and MM-Dist, on HumanML3D.

Critical weaknesses
  1. Unclear research focus and limited novelty. The paper attempts to address several different challenges, including fine-grained motion control, sampling efficiency, and physical consistency. However, these objectives are not closely connected, and the proposed solutions mainly combine existing techniques rather than introducing technically novel approaches to any of these challenges.

  2. Action-level modeling does not necessarily enable fine-grained motion control. They are conceptually different. Moreover, action-level semantic alignment and compositional motion generation have already been explored in prior works.

  3. The contribution of physics guidance is overstated. Although physics-aware generation is highlighted in the title and throughout the paper, the proposed physics guidance primarily relies on a foot contact loss, which is a commonly used regularization technique in human motion generation. This does not constitute a substantial technical contribution.

  4. The KIT-ML results reported in Tables 2, 3, and 4 are substantially inconsistent. For example, the full model achieves an FID of 0.128 in Table 2, compared with 0.2808 and 0.278 in Tables 3 and 4. These inconsistency highly affects the reliability of this paper.

  5. The action-level alignment is constructed using heuristic Gaussian windows. It is unclear whether it accurately captures actions with different durations or complex temporal relationships.

Questions for authors

Please see the weaknesses.

Reviewer_ysoyOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a latent flow-matching based text2motion generation model. It proposes action parsing, action-to-frame alignment and foot skating regularization loss to improve the performance. Quantitative experiments show that it achieves better performance against the baselines.

Critical strengths
  1. The challenging on compositional semantics, physical artifacts are important.
Critical weaknesses
  1. The reported results are inconsistent. Table 2 reports KIT-ML FID 0.128, whereas Section 4.3 and Appendix Table 4 report 0.278. The ablation full model achieves 0.2808. Retrieval results vary. These discrepancies substantially affect the credit of this work.
  2. The physics contribution has not been demonstrated. The central claim is physics awareness, while the design is a straightforward, commonly used foot sliding regularization loss, which has been used even from the first motion diffusion model paper. Further, there's no evaluation on the improvement of physical fidelity.
  3. No video comparisons were found in the supplementary material. Static film strip cannot establish smoothness, foot contact, or transition quality. It further make the claim advanced performance unconvincing.
Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.