- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper aims to improve diffusion-based text-to-motion generation by jointly denoising the distances between specific joints together with the output motion. The joint connections of interest are defined based on the skeleton template to include relationships between five body parts. The paper reports quantitative results on the text-to-motion benchmark on the HumanML3D and SnapMoGen datasets. When combined with diffusion backbones, the proposed method improves the baseline performance. However, the results are worse than token-based text-to-motion methods.
Critical strengths
The paper proposed a simple method to improve the performance of diffusion-based text-to-motion models. Because it only requires recomputing relative joint distances from the dataset, the method can be used as a drop-in replacement for the motion representation.
Critical weaknesses
- There is no study on the effect of the proposed methods on controllability, even though the paper positions itself as a method that addresses relative joint relationships for specific motions such as kicking or clapping.
- Only the experiments on standard text-to-motion are reported.
- As the experiments only focus on the text-to-motion task without any additional control, it is unclear how significant the proposed method is, given that the final quantitative performance is still worse than token-based motion generation. Conventionally, it might be easier to control the motion generation with a diffusion-based method than with token-based ones. However, that aspect is not studied in this paper, which raises the question of its general practicality.
Overall, the proposed method's contribution feels incremental, only adding an additional distance vector to the model output. The paper lacks a comprehensive study on the controllability of the model after the method is introduced. From the experiments, it is unclear whether predicting this additional distance vector results in the prediction being better preservation of relative joint relations, as raised in Figure 1(a).
Questions for authors
- Because the reported metrics do not measure relative joint distances, does the proposed method affect the final qualitative results compared with other baselines?
- Does it also affect controllability during generation?
- Is there any qualitative comparison to the baselines?
3 comments
Response to Reviewer jzWC (2/2)
[W3] On recent backbones, AMDD reaches FID better than SOTA discrete token-based methods
We thank the reviewer for raising the question of practical competitiveness against discrete-token methods. When paired with two recent continuous backbones, gate-free AMDD brings quantitative performance to the level of the strongest published discrete-token methods (General Response, Table G1).
-
On HumanML3D, AMDD lowers the T2M FID of UniMoFlow from 0.047 to 0.029 (and MARDM FID from 0.047 to 0.023) while increasing R@1 to 0.579, matching or surpassing ScaleMoGen (0.030 FID, 0.577 R@1).
-
On SnapMoGen, AMDD lowers the FID of UniMoFlow from 15.88 to 14.80 and that of CMDM (uMT5-XXL) from 15.22 to 14.67, both improving upon the best published discrete-token FID of MoMask++ (15.06).
Within each pair, the backbone and its AMDD version share the exact same data, text encoder, training recipe, and sampler. Together with the training-free relation control demonstrated in [W1, W2, Q2], these results highlight the practical advantage of combining continuous diffusion backbones with an explicit distance stream.
[Q1, Q3] Joint-distance measurements and qualitative comparisons support the relational benefit
We thank the reviewer for asking whether predicting inter-joint distances directly improves relational fidelity in the generated motions. Beyond FID, we evaluate the inter-joint relations motivating Figure 1(a) directly on the generated joints.
Contact and relation dynamics on HumanML3D. On HumanML3D, AMDD () significantly narrows the Hand contact-distribution gap to real motion from 3.7 to 1.8 cm, with no statistically significant change detected in the No-contact or Head gaps. In addition, we evaluate relation dynamics by comparing 1-frame and 4-frame distance changes against real motion across all 174 non-bone pairs (including unmodeled pairs), and AMDD reduces the gap by 53% and 51% relative to Base. Together, these measurements establish clear improvements in the corresponding relational distributions.
| MDM (BERT-Dec) | Hand ↓ | No contact ↓ | Head ↓ |
|---|---|---|---|
| Base | 3.7 | 2.1 | 2.1 |
| AMDD, | 1.8* | 1.7 | 2.3 |
Contact: cluster-weighted Wasserstein-1 gap (in cm) between generated and real distributions of per-clip minimum joint distances, with each model evaluated at its T2M-selected guidance setting. * indicates a statistically significant reduction relative to Base.
Contact on SnapMoGen. Averaged across three backbones (MDM BERT-Dec, UniMoFlow, and CMDM uMT5-XXL), gate-free AMDD increases contact recall on prompts requiring contact by 4.4 to 7.7 percentage points (pp) across three contact categories, with no significant loss in contact precision:
| Contact type | Recall change | Precision change |
|---|---|---|
| Hand–hand | +7.5 pp* | −1.2 pp |
| Hand–head | +4.4 pp* | −0.5 pp |
| Foot–foot | +7.7 pp* | −1.0 pp |
*pp denotes percentage points. Precision is the fraction of clips with a generated contact whose reference clip also contains that contact. * indicates a 95% CI excluding zero; no significant precision change is detected.
Qualitative evidence. Figure 4 compares gated AMDD with Base, MDM, MoMask, and SALAD on HumanML3D, and Appendix K adds examples on both datasets (including MoMask++ on SnapMoGen) alongside the relevant distance curves. Our anonymous page provides side-by-side Base and AMDD videos for MDM, CMDM, and UniMoFlow. We will connect these qualitative examples more explicitly to the relational metrics in the revised paper.
[Contribution] One-way coupling makes the distance state useful for both generation and control
Our core contribution is a jointly denoised relational state that the motion stream can follow throughout generation. Table 3(c) compares alternative ways of incorporating the same 64 rule-selected distances into MDM (BERT-Dec), evaluated at under a common training recipe:
| Variant | FID, T2M / MARDM ↓ |
|---|---|
| Base | 0.214 / 0.300 |
| Auxiliary distance loss only | 0.240 / 0.325 |
| Concatenation in one stream | 0.327 / 0.419 |
| Bidirectional coupling | 0.303 / 0.293 |
| Motion → distance | 0.228 / 0.266 |
| Distance → motion, no gate (AMDD) | 0.182 / 0.231 |
Among these variants, only one-way distance-to-motion coupling improves FID under both evaluators. In this design, the motion stream reads the relational state, while the distance stream denoises without reading noisy motion features. The exact same reading mechanism can utilize distances either generated from text or supplied as control targets, directly connecting the generation improvement to the control interface above.
We thank the reviewer again for motivating this additional evaluation of practical controllability and relational fidelity.
Follow-up on the Additional Controllability Experiments
Thank you again for your time and valuable comments. As the discussion period nears its close, we would like to briefly follow up on our response, particularly the additional controllability experiments motivated by your suggestions.
Reusing AMDD's existing generation checkpoint without control-specific training, our distance interface follows amplitude, frequency, count, and arm-pose distance targets more closely than an adapted OmniControl baseline under the same guidance setting. Side-by-side video comparisons are available on our anonymous page.
We would greatly appreciate your feedback on whether these results address your concerns regarding controllability and practical value, or whether any points remain unresolved.
Response to Reviewer jzWC (1/2)
We thank the reviewer for highlighting controllability as a test of AMDD's practical value. Your suggestion prompted us to use the distance state learned for generation as a direct control interface. We compare it against OmniControl adapted to the same distance targets, evaluating both methods with and without its spatial gradient guidance. As in the General Response, AMDD denotes the gate-free model, and gated AMDD denotes the submitted configuration.
[W1, W2, Q2] AMDD supports relation control without retraining, with optional gradient guidance
Control interface. AMDD ordinarily predicts inter-joint distances jointly with motion from text. For control, we replace selected distance channels at each sampling step with target curves noised to that timestep, allowing the motion stream to read these states through its existing one-way attention. This reuses the generation model directly without control-specific training. Below, we evaluate both direct distance control and its combination with spatial gradient guidance.
Comparison. On HumanML3D with MDM (BERT-Dec), we compare AMDD against OmniControl (Xie et al., ICLR 2024), which combines a trainable control branch with spatial gradient guidance (G). We replace OmniControl's control signal—the 3D positions of controlled joints—with the joint-pair distance targets that AMDD receives. Compared with joint-position trajectories, distance targets express relations between moving joints directly and are invariant to global position and heading, which also makes it easier to specify movement amplitude, count, and frequency. We evaluate both models with and without G under the same gradient schedule, and all four methods receive the exact same action prompts, target curves, and controlled pairs.
Kick slope measures peak foot-height change per meter of target foot–foot distance (four levels, 0.6–1.5 m). For reference, the slope across real kick events is approximately 0.8–1.0.
Reusing the generative state for control. Without control-specific training or sampling gradients, AMDD achieves a kick-height response close to the real-data reference range and reduces mean pose-distance error to roughly one quarter of that of OmniControl. These results support the practical value of reusing the jointly generated distance state as a control interface. User-specified targets enter through the existing distance-to-motion reading path, allowing generation and control to share the same trained model.
Accuracy–geometry trade-off under gradient guidance. For pose targets, spatial guidance tightens distance tracking but increases bone-length deviations in both models. Gradient-free AMDD already keeps 98% of the evaluated pairwise distances within 20 cm of their targets, with lower bone-length errors than either guided variant. When both methods use the same guidance, AMDD achieves higher exact-count accuracy and lower pose-distance error than OmniControl, matches its rate accuracy, and has lower bone-length errors in the reported task groups. These results support gradient guidance as an optional extension for tighter tracking, while showing that distance accuracy and skeletal consistency must be assessed separately.
By contrast, Specifying the targets in words to the same AMDD model gives a 0.04 kick slope, 16% exact counts, 52 / 45 / 0% rates within ±15%, and 27 cm mean pose-distance error (45% within 20 cm)
We will add these comparisons and protocols to the experiments in Section 4. Control demonstrations and side-by-side comparisons with OmniControl are available on our anonymous page.