- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper presents HuMoGPT, a unified model for human-motion understanding and generation. The system combines: (1) MotionRxiv, a curated corpus of approximately 1.8M motion clips with unified representations and additional synthesized supervision; (2) SeMoT, a semantic motion tokenizer aligned with language for understanding; (3) GeMoT, a continuous latent autoencoder for motion generation; and (4) a pretrained multimodal language-model backbone equipped with flow matching and modality/task/domain-routed LoRA adapters. The model supports captioning, retrieval, recognition, question answering, dialogue, text-to-motion generation, completion, in-betweening, editing, and interaction generation.
The paper reports strong results on HumanML3D retrieval, captioning, and text-to-motion generation, along with additional evaluations on recognition, grounded dialogue, completion, in-betweening, and editing. However, because HuMoGPT is trained on a substantially larger and more heterogeneous dataset than most compared methods, the current experiments do not clearly establish whether the reported gains arise from the proposed modeling choices or primarily from data scale and supervision. The paper itself acknowledges this confound for the SeMoT retrieval comparison.
Critical strengths
1. Large-scale motion-data aggregation and supervision.
The construction of MotionRxiv is potentially valuable. The authors aggregate heterogeneous motion sources into a common representation, construct leakage-aware splits, and introduce large-scale grounded QA/dialogue supervision. The appendix also provides useful discussion of representation conversion, normalization, filtering, and grounding.
2. Broad system capability.
HuMoGPT attempts to support an unusually broad range of motion understanding and synthesis tasks within one framework. The system achieves strong reported performance on several major benchmarks, particularly motion-text retrieval, captioning, and text-to-motion generation.
Critical weaknesses
1. Unclear architectural novelty and inadequate positioning against the closest prior work.
The architectural contribution appears limited relative to recent unified motion-language models. HuMoGPT combines a pretrained language backbone, continuous motion representations, separate understanding/generation pathways, continuous generative modeling, and staged multimodal training. These ideas substantially overlap with the design space of closely related unified models such as MotionGPT3, UniMotion, LLaMo, yet the manuscript does not clearly articulate the conceptual advance over such work. At the same time, the related-work section spends substantial space on systems such as MoChat and ChatPose that are substantially less comparable because they do not provide the same unified motion understanding-and-generation functionality.
This makes it difficult to identify the main technical contribution of the paper beyond system integration and scaling.
2. The experiments do not disentangle architectural improvements from massive increases in training data and supervision.
HuMoGPT and SeMoT are trained using a large heterogeneous corpus, whereas many baselines use substantially smaller benchmark-specific datasets. The authors explicitly acknowledge that the SeMoT comparison is not a controlled architectural comparison. Yet the same confound affects much of the paper. Crucially, the paper lacks controlled experiments using the same backbone, training data, and compute to establish the value of its major modeling choices. In particular, there is no convincing ablation showing that using separate SeMoT and GeMoT representations is superior to a shared motion representation. The routed-LoRA ablation is also reported primarily for text-to-motion rather than demonstrating consistent benefit across the claimed task taxonomy. This problem extends to the dataset contribution. Although the appendix explains parts of the conversion pipeline, the dataset is insufficiently characterized for something presented as a primary contribution: detailed source/task distributions, caption provenance and quality, annotation generation for sources without native text, source imbalance, duplicate statistics, and the effect of individual data sources on downstream performance are not systematically analyzed.
References:
- Li et al., LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokens, CVPR 2026;
- Zhu et al., MotionGPT3: Human Motion as a Second Modality, ICLR 2026;
- Wang et al., UniMotion: A Unified Framework for Motion-Text-Vision Understanding and Generation, ECCV 2026.
Questions for authors
The work represents substantial engineering effort and data curation, but I do not find the manuscript sufficient for acceptance, at least in its current form. The closest prior unified motion-model literature is inadequately addressed, leaving the architectural novelty unclear. More importantly, the central empirical results are strongly confounded by differences in data scale and supervision, while the key proposed components are not validated through controlled experiments. Several advertised modalities and tasks also receive substantially less evaluation than the breadth of the claims suggests.
My questions for authors:
- Can the authors provide controlled experiments in which data, backbone, and training budget are fixed, while varying the proposed architectural choices? In particular, I would like to see:
- a single shared motion representation vs. SeMoT + GeMoT;
- shared adaptation vs. routed adaptation across multiple understanding and generation tasks;
- training on benchmark-scale data vs. MotionRxiv-scale data.
Without such comparisons, I cannot determine whether the improvement is architectural or primarily due to data scaling. These are true interesting stories that worth dicussing.
-
Since MotionRxiv is presented as a central contribution, can the authors provide a substantially more detailed characterization of its composition and supervision? In particular, how are text annotations obtained for every source, including video/motion datasets without native captions, what fraction of captions are human/generated/VLM-generated, and how are these sources weighted during training? Also, will the dataset be open-sourced? This could be a great contribution to the community.
-
What measurable contribution does the video modality make? The architecture explicitly includes video-conditioned motion generation, but I could not find a corresponding controlled quantitative study and discussion showing whether video improves generation or other multimodal capabilities.
-
Can the authors provide a stronger evaluation of motion editing that separately measures instruction satisfaction and preservation of attributes that should remain unchanged? In the supplementary examples, some edits appear achievable through simple temporal transformations, while timing edits are visibly more difficult. The paper itself reports a timing-edit example for which the prediction is worse than copying the source. How do the authors distinguish genuine semantic editing from simple speed/trajectory transformations?
-
Regarding the suppementary video, is the system running in real-time? Or is the video processed and edited? For motion editing demonstration, it seems that only the frequency is faster but the root trajectory stays the same. This leads to serious drifting and unnatural motion.