acceptodds
Withdrawn from ICLR 2027

Timo: aming Multmodal Diffusion Transformer for Human tion Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Most existing human motion generation (HMG) methods use cross-attention modules to inject text semantics, but ignore the importance of bidirectional modeling between motion and text tokens, which limits text comprehension. A straightforward idea is introducing multimodal diffusion transformers (MMDiT), which have shown effective joint text–visual modeling in vision generation, into HMG. However, we find that articulated motion is temporally coherent but weakly correlated across joints, in which directly applying an MMDiT with flow matching produces poorly coordinated and jerky motion. In this work, we propose Timo, a novel kinematics-aware MMDiT framework tailored for HMG. Timo combines fully shared multimodal attention for bidirectional text–motion modeling with flow matching, geometric and rotational-kinematics supervision that compares actual rotations and their changes over time, and a two-stage curriculum progressing from broad motion learning to detailed caption alignment. Further, we construct a benchmark of held-out clips from six public datasets spanning diverse actions, assessing six complementary dimensions under a common evaluator and scoring protocol. Our model substantially outperforms state-of-the-art methods in both quantitative and qualitative evaluations. Remarkably, Timo surpasses Kimodo on five of six dimensions, achieving a % relative improvement in the average benchmark score. Anonymous project page: https://timoiclr2027.github.io. Anonymous demo page: https://timo.playground.wang.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
human motion generation, multimodal diffusion transformer, flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rnQGOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Timo, which applies the MMDiT architecture with flow matching to text-to-motion generation. Text and motion tokens are processed jointly by a fully shared transformer. Since directly applying MMDiT leads to jerky motion, the authors add rotational kinematics and geometric losses, and train the model with a two-stage coarse-to-fine curriculum. On a six-dataset benchmark built by the authors, Timo outperforms existing methods such as Kimodo and HY-Motion.

Critical strengths
  1. The rotational kinematics loss is simple and effective, and it clearly improves motion smoothness as shown in Table 2.
  2. The paper is well-written and easy to follow.
Critical weaknesses
  1. Limited technical contribution and novelty. The core components, namely MMDiT-style joint text–motion attention and flow matching, have already been explored in prior text-to-motion methods such as HY-Motion and MotionHiFlow [1], so the paper offers limited new insight to the field. Moreover, the abstract's first sentence claims that cross-attention limits text comprehension and that bidirectional attention addresses this. However, Table 2 shows that DDIM+RoPE and MMDiT+flow achieve the same Text-Motion Match score (80.8), indicating that bidirectional attention brings no measurable gain in text alignment. Instead, the improvement in Text-Motion Match comes mainly from mirroring and the body-rotation loss.

  2. The main comparison in Table 1 is unconvincing. The evaluation protocol is constructed by the authors, and several important details are missing.

    • The test clips are held out from the same six corpora used in Timo's second training stage, so Timo is evaluated in distribution, whereas the compared methods were trained on different datasets with different motion representations. The data split is also created by the authors. Notably, the DDIM baseline in Table 2, trained only on these public corpora, already outperforms all released methods in Table 1 by a large margin (e.g., Naturalness 75.6 vs. at most 44.2), suggesting that most of the reported gain comes from data alignment rather than from the proposed method.
    • The paper does not explain how the evaluator is trained or which data it is trained on.
    • The paper does not explain how the baselines, which use different motion representations, are converted and evaluated.
    • All results come from a single generation seed, with no confidence intervals over multiple runs.
    • No experiments are conducted on commonly used benchmarks such as HumanML3D and KIT-ML under the standard protocol.
  3. The ablation studies are heavily confounded, leaving the individual effect of each contribution unclear.

    • Table 2 presents cumulative ablations without leave-one-out experiments. The first step simultaneously changes the architecture and the generation process, and possibly also the prediction target and the sampler.
    • L_geo, the geometric loss with weights as high as 500/500/1000, is never ablated, despite being presented as part of the contributions.
    • Table 3 compares the full model against a single-stage MMDiT+flow model, so the comparison is confounded by differences in data and training curriculum.
    • In Table 4, the two-stage configuration simultaneously introduces in-house data and increases the number of training steps.
  4. Insufficient evaluation of physical plausibility. The authors claim that the generated motions are coordinated and temporally coherent, but do not report standard physical metrics such as foot skating ratio, floating, and ground penetration. Moreover, the proposed method performs considerably worse than Kimodo on limb interpenetration (4.93 vs. 1.42, Table S4).

[1]: Heng Li, Xiaotong Lin, Ling-An Zeng, et al. MotionHiFlow: Text-to-Motion via Hierarchical Flow Matching. In CVPR, 2026.

Questions for authors

See Weaknesses

Reviewer_cXSjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Timo, a framework for text-conditioned human motion generation. It combines a fully shared multimodal Transformer for bidirectional text–motion interaction with flow matching over explicit SMPL parameters. Training incorporates reconstruction, geometric, and rotational-kinematics losses, alongside a two-stage curriculum progressing from broadly labelled motion data to precisely captioned clips. The authors evaluate Timo on an author-constructed benchmark containing 40,025 held-out clips from six public datasets and report substantial improvements over released systems. The results suggest a competitive complete system. However, the evidence does not yet sufficiently establish the individual contributions of the central architectural choices, and the validity of the evaluation protocol requires further substantiation.

Critical strengths

1.The rotational-kinematics supervision has a clear technical motivation.

2.The ablations provide meaningful evidence that rotational supervision improves the proposed system.

3.Several raw metrics support the competitiveness of the complete system.

Critical weaknesses

1.The central claim that bidirectional text–motion interaction improves semantic understanding is not directly established. The paper motivates its architecture through the limitations of one-way text conditioning, but it does not compare one-way and bidirectional interaction while holding the remaining design choices fixed. The first transition in Table 2 changes both the architecture and the generative formulation. Consequently, it cannot isolate the effects of bidirectionality, parameter sharing, or flow matching. Notably, Text-Motion Match remains 80.8 after this replacement, providing no direct evidence of improved semantic alignment at this step. It leaves a central question unresolved: what does fully shared bidirectional processing contribute beyond conventional cross-attention or joint attention with modality-specific parameters? Controlled architectural comparisons and targeted tests of directional, temporal, and compositional instructions are needed to substantiate this contribution.

2.The proposed explanation connecting weak inter-joint correlation, transfer failure, and improved coordination remains insufficiently supported. The current evidence most strongly supports the effectiveness of rotational temporal supervision; the broader explanation of why direct MMDiT transfer fails remains tentative.

Questions for authors

Refer to Critical Weaknesses.

Reviewer_Dp4iOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This manuscript proposed a method called Timo for coherent human motion generation. It concatenates motion token and text token together and feed them into a MMDiT with shared multimodal attention for motion generation. To make the generated motion temporally consistent, the manuscript adds temporal structure supervision in model training.

Critical strengths

NA

Critical weaknesses
  1. I think the manuscript would benefit from substantial improvements in writing and organization. Several passages read as loosely connected statements, making the motivation, technical development, and experimental conclusions difficult to follow. A thorough revision would help present the work with the clarity, coherence, and precision expected of a rigorous academic paper.

  2. The novelty of the proposed bidirectional text–motion MMDiT is not sufficiently established. The interaction is implemented by concatenating motion and text tokens and processing them through shared self-attention blocks, which is a natural and established approach to multimodal fusion. While this is a reasonable design choice, the manuscript should explain more clearly what distinguishes it from existing multimodal Transformer architectures. A controlled comparison between unidirectional and bidirectional interaction, with other factors held constant, would also help establish its specific contribution.

  3. The temporal supervision strategy should be positioned more carefully relative to prior work. In particular, KineST [1] already computes angular increments through the logarithmic map of relative rotations between consecutive frames, closely resembling the angular-velocity construction in Eq. (5). This related work does not appear to be cited or discussed. Direct comparisons with established temporal supervision losses would further strengthen the novelty claim.

[1] Shuting Zhao, Zeyu Xiao, and Xinrong Chen. “KineST: A Kinematics-guided Spatiotemporal State Space Model for Human Motion Tracking from Sparse Signals.” arXiv:2512.16791, 2025 (published in AAAI 2026).

Questions for authors

Please check Critical Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.