- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper argues that the motion tokenizer rather than the generator is the key bottleneck in discrete-token human motion generation. To prove this claim, the authors propose to jointly scale the training motion distribution and the tokenizer capacity: they introduce a synthetic motion pipeline that generates new poses via crossover and mutation operations on a hierarchical bone representation, filters them with a pose prior, and builds sequences by interpolation, recovering root trajectories with a lightweight Transformer. The augmented tokenizer is trained on a mixture of real HumanML3D and synthetic motions with an enlarged codebook, and existing autoregressive generators are re-tokenized and fine-tuned. Experiments report reduced reconstruction FID and consistent gains across downstream backbones.
Critical strengths
[Originality]
- Reframing the problem around the tokenizer as the bottleneck is a fresh perspective. I like this way of thinking about the problem.
Critical weaknesses
n/a
Questions for authors
[Ablation]
- The ablation shows an interesting pattern: scaling the codebook alone or the synthetic data alone each hurts the quality, while the two together help. I'm curious if the authors have thoughts on why this contradictory pattern happens.
Other than this minor question, I don't have other concerns regarding this submission. It's an interesting reading and I enjoy how the authors formulate the problem. I thus lean towards accept.
Overall
We thank all reviewers for their careful and constructive feedback. We are encouraged that Reviewers v9Fy and KsJz found our tokenizer-bottleneck framing well motivated, and that Reviewers scq5 and KsJz recognized the value of our reusable approach. We are particularly grateful to Reviewer KsJz for the encouraging assessment of our method.
To address the central concern shared by Reviewers scq5 and v9Fy—the decline in text alignment accompanying FID improvements—we introduce a simple supplementary RL-controlled motion correction stage. Our T2M-GPT experiments show improved text-alignment metrics while keeping FID changes within the predefined tolerance (see our response to Reviewer v9Fy, W1). We also address the remaining comments point by point below. We will incorporate these new results and discussions into the final version of the paper.
Q1 Ablation analysis of data scaling and codebook expansion
Thank you for drawing attention to this observation. We believe it reflects how well the training motion distribution matches the tokenizer's representational capacity. When only the codebook is enlarged, the ablation study in T2M-GPT also reports a similar non-monotonic trend: among 256, 512, and 1024 codes, 512 yields the best reconstruction FID. This indicates that increasing the number of codes does not necessarily improve performance under the original data setting. Our interpretation is that, with a limited motion distribution, the additional capacity may be difficult to fully utilize.
With synthetic motion data expansion alone, the richer motion patterns may exceed the effective capacity of a fixed 512-code representation. Joint scaling increases both motion coverage and representational capacity, potentially alleviating these two forms of mismatch. The controlled comparisons in Table 2 of our paper support the complementarity of synthetic motion data expansion and codebook capacity expansion under the current experimental setting.