- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MoTween is a two-stage VQ text-to-motion model. Stage 1 encodes six parts and three groups (whole/upper/lower body) with separate encoders, refines part latents by cross-attention to the group latents, and quantizes them with part-specific codebooks. Stage 2 predicts from text a per-frame event class for each part (idle or ±x/±y/±z dominant velocity), and changes in this class define part-wise keyframes. A Keyframe Transformer generates latent anchors at those frames autoregressively, the gaps are linearly interpolated, and an In-Betweening Transformer (temporal, then part-wise attention) predicts the part tokens. Training event labels come from a rule-based procedure on GT motion. Results on HumanML3D and KIT-ML are slightly above MoGenTS; the appendix adds predictor accuracy, oracle keyframes, tokenizer comparisons, length-stratified results and slot-swap editing.
Critical strengths
Keyframe-first decoding is a sensible alternative to uniform AR or masked decoding in token-based T2M, and the paper tests it from several sides. With the same tokenizer, full-sequence AR gives FID 0.069 and random-mask completion 0.042, vs. 0.031 for MoTween (Tables 9 and 2). The advantage holds on long sequences with all methods re-run in one pipeline (Table 13), and the oracle-keyframe and weaker-predictor experiments (Tables 7-8) show how upstream errors propagate. The paper is clearly written and inference is fast (0.24 s).
Critical weaknesses
-
One-to-many generation is not addressed. As described, the pipeline is deterministic given the text: the event predictor outputs one class per frame, the Keyframe Transformer is a deterministic recursion (Eq. 3), and the token head takes "the index with the highest probability." The same prompt would then always yield the same motion, and MModality, a standard column on both benchmarks, is missing from Table 1. Relatedly, 95.8% frame-wise event accuracy and 82.8% keyframe F1 (±5 frames) from text alone are surprisingly high, because text does not determine when a hand or foot changes direction (the step phase during walking, for example). These numbers need a majority-class baseline and the class distribution, and the paper should state whether they are on the test split and whether GT length is given.
-
The gain hinges on hand-tuned pseudo-labels, and the SOTA claim is overstated. Event thresholds are set per "action category" and body part (App. B.2), but neither dataset has action categories and the paper does not say how they are assigned. The dependence is large: with one global setting, FID goes from 0.031 to 0.072 and Top-1 from 0.537 to 0.520 (Table 4), worse than the paper's own random-masking variant (0.042 / 0.523, Table 2). So keyframe guidance beats masked completion only with this tuning. On the benchmarks, gains over MoGenTS are within the reported intervals (HumanML3D FID 0.031±.002 vs. 0.033±.001; KIT Top-1 0.448±.005 vs. 0.445±.006), and published SALAD has clearly higher R-Precision (Top-1 0.581, Top-3 0.857; App. Table 12). The abstract and the "best results across all evaluation metrics" claim in Sec. 4.2.1 should be revised.
Questions for authors
- Is anything sampled at inference (events, anchors, tokens)? Please report MModality and show several samples for one prompt. This is the point most likely to change my score.
- How are action categories obtained for HumanML3D and KIT-ML? What does a majority-class predictor score on event accuracy and keyframe F1? What is the average number of keyframes per part relative to sequence length?
- In Hierarchical Cross-Attention, does attention run over time? If each part attends to one group latent per frame, the softmax has a single key and the module reduces to adding a projected group latent. Likewise, does Slot Attention take CLIP token features or the pooled 512-d sentence embedding? With a single input vector, every slot receives the same update.
- How is motion length set at inference, how do frame-level keyframes map to tokens if the encoder downsamples in time, and does Stage-2 training use GT or predicted keyframes?
- In slot-swap editing, both the cross-slot self-attention in the event predictor and the part-wise attention in the In-Betweening Transformer let a swapped slot affect other parts. Do unedited parts actually stay fixed? Joint error on unedited parts relative to the base motion would show this.
According to the comparison with MotionHiFlow, the proposed is slightly inferior to the SOTA methods. Meanwhile, the authors should consider whether the shape and surface is necessary for this paper. I believe Figure 1-4 should be further polished.