acceptodds
Under review as a conference paper at ICLR 2027

Inter-Joint Relations Matter: Asymmetric Motion–Distance Denoising for Text-to-Motion Generation

Abstract

In text-to-motion generation, many described actions hinge on changing relations between body parts, such as the hands meeting at the right moment during clapping. However, most generators leave these relations implicit in joint states or motion codes, using joint-derived distances, if at all, only as losses or sampling guidance, so the denoiser has no explicit relational state to follow. We propose Asymmetric Motion–Distance Denoising (AMDD), which jointly denoises motion and sparse, continuous inter-joint distances. Its two-stream denoiser couples them one way: shielded from noisy motion, the distance stream gives the motion stream a stable relational reference at every denoising step. A geometric consistency objective encourages the predicted motion to realize the predicted distances, and a text-conditioned gate scales the distance stream's updates. AMDD lowers FID by 43–84% on HumanML3D across three diffusion backbones and by 12% on SnapMoGen, and its distance stream reduces inter-joint distance error about twice as much as position error.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Motion Generation, Text-to-Motion Generation, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jzWCOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to improve diffusion-based text-to-motion generation by jointly denoising the distances between specific joints together with the output motion. The joint connections of interest are defined based on the skeleton template to include relationships between five body parts. The paper reports quantitative results on the text-to-motion benchmark on the HumanML3D and SnapMoGen datasets. When combined with diffusion backbones, the proposed method improves the baseline performance. However, the results are worse than token-based text-to-motion methods.

Critical strengths

The paper proposed a simple method to improve the performance of diffusion-based text-to-motion models. Because it only requires recomputing relative joint distances from the dataset, the method can be used as a drop-in replacement for the motion representation.

Critical weaknesses
  • There is no study on the effect of the proposed methods on controllability, even though the paper positions itself as a method that addresses relative joint relationships for specific motions such as kicking or clapping.
  • Only the experiments on standard text-to-motion are reported.
  • As the experiments only focus on the text-to-motion task without any additional control, it is unclear how significant the proposed method is, given that the final quantitative performance is still worse than token-based motion generation. Conventionally, it might be easier to control the motion generation with a diffusion-based method than with token-based ones. However, that aspect is not studied in this paper, which raises the question of its general practicality.

Overall, the proposed method's contribution feels incremental, only adding an additional distance vector to the model output. The paper lacks a comprehensive study on the controllability of the model after the method is introduced. From the experiments, it is unclear whether predicting this additional distance vector results in the prediction being better preservation of relative joint relations, as raised in Figure 1(a).

Questions for authors
  • Because the reported metrics do not measure relative joint distances, does the proposed method affect the final qualitative results compared with other baselines?
  • Does it also affect controllability during generation?
  • Is there any qualitative comparison to the baselines?
3 comments
AuthorsOpenReview ↗

Response to Reviewer jzWC (1/2)

We thank the reviewer for highlighting controllability as a test of AMDD's practical value. Your suggestion prompted us to use the distance state learned for generation as a direct control interface. We compare it against OmniControl adapted to the same distance targets, evaluating both methods with and without its spatial gradient guidance. As in the General Response, AMDD denotes the gate-free model, and gated AMDD denotes the submitted configuration.

[W1, W2, Q2] AMDD supports relation control without retraining, with optional gradient guidance

Control interface. AMDD ordinarily predicts inter-joint distances jointly with motion from text. For control, we replace selected distance channels at each sampling step with target curves noised to that timestep, allowing the motion stream to read these states through its existing one-way attention. This reuses the generation model directly without control-specific training. Below, we evaluate both direct distance control and its combination with spatial gradient guidance.

Comparison. On HumanML3D with MDM (BERT-Dec), we compare AMDD against OmniControl (Xie et al., ICLR 2024), which combines a trainable control branch with spatial gradient guidance (G). We replace OmniControl's control signal—the 3D positions of controlled joints—with the joint-pair distance targets that AMDD receives. Compared with joint-position trajectories, distance targets express relations between moving joints directly and are invariant to global position and heading, which also makes it easier to specify movement amplitude, count, and frequency. We evaluate both models with and without G under the same gradient schedule, and all four methods receive the exact same action prompts, target curves, and controlled pairs.

Task and metricOmniControl (w/o G)OmniControl (w/ G)AMDD (w/o G)AMDD (w/ G)
Control-specific branch training / sampling gradientsYes / NoYes / YesNo / NoNo / Yes
Kick-height response slope ↑0.240.470.810.77
Exact detected step count, N=2,4,6,8N=2,4,6,8 ↑74%92%91%100%
Walking / jogging / punching rate, within ±15% ↑98 / 95 / 60%100 / 100 / 100%98 / 95 / 100%100 / 100 / 100%
Pose targets: mean distance error (cm) ↓20.50.55.40.1
Pose targets: distances within 20 cm of target ↑58%99%98%100%
Pose targets: median of per-frame maximum bone-length error (cm) ↓1.96.72.54.2
Bone-length error, count / pose / kick (cm) ↓0.18 / 0.61 / 0.330.37 / 2.31 / 0.650.17 / 0.75 / 0.320.25 / 1.53 / 0.53

Kick slope measures peak foot-height change per meter of target foot–foot distance (four levels, 0.6–1.5 m). For reference, the slope across real kick events is approximately 0.8–1.0.

Reusing the generative state for control. Without control-specific training or sampling gradients, AMDD achieves a kick-height response close to the real-data reference range and reduces mean pose-distance error to roughly one quarter of that of OmniControl. These results support the practical value of reusing the jointly generated distance state as a control interface. User-specified targets enter through the existing distance-to-motion reading path, allowing generation and control to share the same trained model.

Accuracy–geometry trade-off under gradient guidance. For pose targets, spatial guidance tightens distance tracking but increases bone-length deviations in both models. Gradient-free AMDD already keeps 98% of the evaluated pairwise distances within 20 cm of their targets, with lower bone-length errors than either guided variant. When both methods use the same guidance, AMDD achieves higher exact-count accuracy and lower pose-distance error than OmniControl, matches its rate accuracy, and has lower bone-length errors in the reported task groups. These results support gradient guidance as an optional extension for tighter tracking, while showing that distance accuracy and skeletal consistency must be assessed separately.

By contrast, Specifying the targets in words to the same AMDD model gives a 0.04 kick slope, 16% exact counts, 52 / 45 / 0% rates within ±15%, and 27 cm mean pose-distance error (45% within 20 cm)

We will add these comparisons and protocols to the experiments in Section 4. Control demonstrations and side-by-side comparisons with OmniControl are available on our anonymous page.

AuthorsOpenReview ↗

Response to Reviewer jzWC (2/2)

[W3] On recent backbones, AMDD reaches FID better than SOTA discrete token-based methods

We thank the reviewer for raising the question of practical competitiveness against discrete-token methods. When paired with two recent continuous backbones, gate-free AMDD brings quantitative performance to the level of the strongest published discrete-token methods (General Response, Table G1).

  • On HumanML3D, AMDD lowers the T2M FID of UniMoFlow from 0.047 to 0.029 (and MARDM FID from 0.047 to 0.023) while increasing R@1 to 0.579, matching or surpassing ScaleMoGen (0.030 FID, 0.577 R@1).

  • On SnapMoGen, AMDD lowers the FID of UniMoFlow from 15.88 to 14.80 and that of CMDM (uMT5-XXL) from 15.22 to 14.67, both improving upon the best published discrete-token FID of MoMask++ (15.06).

Within each pair, the backbone and its AMDD version share the exact same data, text encoder, training recipe, and sampler. Together with the training-free relation control demonstrated in [W1, W2, Q2], these results highlight the practical advantage of combining continuous diffusion backbones with an explicit distance stream.

[Q1, Q3] Joint-distance measurements and qualitative comparisons support the relational benefit

We thank the reviewer for asking whether predicting inter-joint distances directly improves relational fidelity in the generated motions. Beyond FID, we evaluate the inter-joint relations motivating Figure 1(a) directly on the generated joints.

Contact and relation dynamics on HumanML3D. On HumanML3D, AMDD (K=64K=64) significantly narrows the Hand contact-distribution gap to real motion from 3.7 to 1.8 cm, with no statistically significant change detected in the No-contact or Head gaps. In addition, we evaluate relation dynamics by comparing 1-frame and 4-frame distance changes against real motion across all 174 non-bone pairs (including unmodeled pairs), and AMDD reduces the gap by 53% and 51% relative to Base. Together, these measurements establish clear improvements in the corresponding relational distributions.

MDM (BERT-Dec)Hand ↓No contact ↓Head ↓
Base3.72.12.1
AMDD, K=64K=641.8*1.72.3

Contact: cluster-weighted Wasserstein-1 gap (in cm) between generated and real distributions of per-clip minimum joint distances, with each model evaluated at its T2M-selected guidance setting. * indicates a statistically significant reduction relative to Base.

Contact on SnapMoGen. Averaged across three backbones (MDM BERT-Dec, UniMoFlow, and CMDM uMT5-XXL), gate-free AMDD increases contact recall on prompts requiring contact by 4.4 to 7.7 percentage points (pp) across three contact categories, with no significant loss in contact precision:

Contact typeRecall changePrecision change
Hand–hand+7.5 pp*−1.2 pp
Hand–head+4.4 pp*−0.5 pp
Foot–foot+7.7 pp*−1.0 pp

*pp denotes percentage points. Precision is the fraction of clips with a generated contact whose reference clip also contains that contact. * indicates a 95% CI excluding zero; no significant precision change is detected.

Qualitative evidence. Figure 4 compares gated AMDD with Base, MDM, MoMask, and SALAD on HumanML3D, and Appendix K adds examples on both datasets (including MoMask++ on SnapMoGen) alongside the relevant distance curves. Our anonymous page provides side-by-side Base and AMDD videos for MDM, CMDM, and UniMoFlow. We will connect these qualitative examples more explicitly to the relational metrics in the revised paper.

[Contribution] One-way coupling makes the distance state useful for both generation and control

Our core contribution is a jointly denoised relational state that the motion stream can follow throughout generation. Table 3(c) compares alternative ways of incorporating the same 64 rule-selected distances into MDM (BERT-Dec), evaluated at s=2.5s=2.5 under a common training recipe:

VariantFID, T2M / MARDM ↓
Base0.214 / 0.300
Auxiliary distance loss only0.240 / 0.325
Concatenation in one stream0.327 / 0.419
Bidirectional coupling0.303 / 0.293
Motion → distance0.228 / 0.266
Distance → motion, no gate (AMDD)0.182 / 0.231

Among these variants, only one-way distance-to-motion coupling improves FID under both evaluators. In this design, the motion stream reads the relational state, while the distance stream denoises without reading noisy motion features. The exact same reading mechanism can utilize distances either generated from text or supplied as control targets, directly connecting the generation improvement to the control interface above.

We thank the reviewer again for motivating this additional evaluation of practical controllability and relational fidelity.

AuthorsOpenReview ↗

Follow-up on the Additional Controllability Experiments

Thank you again for your time and valuable comments. As the discussion period nears its close, we would like to briefly follow up on our response, particularly the additional controllability experiments motivated by your suggestions.

Reusing AMDD's existing generation checkpoint without control-specific training, our distance interface follows amplitude, frequency, count, and arm-pose distance targets more closely than an adapted OmniControl baseline under the same guidance setting. Side-by-side video comparisons are available on our anonymous page.

We would greatly appreciate your feedback on whether these results address your concerns regarding controllability and practical value, or whether any points remain unresolved.

Reviewer_czuBOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Asymmetric Motion–Distance Denoising (AMDD), which explicitly models inter-joint spatial relations in text-to-motion generation by jointly denoising human motion and sparse inter-joint distances through an asymmetric dual-stream architecture, a geometric consistency objective, and a text-conditioned gating mechanism. Experiments demonstrate that AMDD substantially reduces FID on HumanML3D and SnapMoGen and improves the generation quality of certain inter-joint relations, although its gains in text-motion semantic alignment remain limited.

Critical strengths
  1. This paper is well-written, clearly structured, and easy to follow.
  2. The paper proposes an asymmetric dual-stream motion–distance denoising framework with geometric consistency constraints and text-conditioned gating. Its effectiveness is supported by comprehensive experiments, including extensive comparisons and ablation studies.
Critical weaknesses
  1. The gate’s FID gain largely coincides with an overall contraction of generated motion. Could the authors compare against a text-independent (fixed or averaged) gate to verify that text-conditioning contributes independently, beyond global contraction?
  2. In Table 4, more joint pairs degrade performance, but also raise the relative weight of the distance loss. Could the authors compare different K with fixed loss weights to tell whether the drop comes from representational redundancy or loss reweighting?
  3. Appendix B.1 shows that adding the distance stream to the original MDM (CLIP-Enc) hurts performance, which improves only after modifying text attention. Beyond attention dilution, how sensitive is AMDD to the text-injection mechanism, and does the asymmetric dual-stream design transfer to other backbones?
Questions for authors

See Weaknesses.

2 comments
AuthorsOpenReview ↗

Response to Reviewer czuB (1/2)

We thank the reviewer for finding the paper well written and clearly structured, and for recognizing our comprehensive comparisons and ablation studies. Your suggestions on testing text-independent gates, disentangling the number of joint pairs KK from loss reweighting, and analyzing sensitivity to text-injection mechanisms across backbones help us better substantiate our claims. Below, we provide additional experimental evidence and explain the corresponding revisions. As noted in the General Response, AMDD denotes the gate-free model, and gated AMDD denotes the submitted model.

[W1] The text conditioning of the gate contributes beyond global contraction

We thank the reviewer for suggesting fixed and averaged gate controls. We evaluate both controls on HumanML3D using MDM (BERT-Dec) with K=64K=64, s=2.5s=2.5, and 500k training steps across 5 evaluations on the full test set. Results are reported below.

GateFID (T2M / MARDM) ↓
None (gate-free AMDD)0.161 / 0.169
Fixed: one learned vector per layer, shared across all texts (trained from scratch)0.137 / 0.173
Averaged: trained gate replaced by its per-channel mean over training texts (inference only)0.769 / 0.879
Text-conditioned (gated AMDD)0.069 / 0.084

Compared with gate-free AMDD, the retrained fixed gate recovers only about a quarter of the T2M FID reduction (0.137 vs. 0.069) and none under the MARDM evaluator (0.173 vs. 0.169), while replacing the trained gate with its training-text mean at inference severely degrades FID to 0.769 / 0.879. Thus, the model relies on text-specific gate modulation beyond a shared global contraction. This refines our statement in Section 4.3, which described the gate's modulation as largely text-independent: the shared component alone accounts for at most a quarter of the T2M gain.

Nevertheless, the gate has clear limitations: it provides no FID benefit without classifier-free guidance (Table G.2), does not improve inter-joint relational accuracy, and—when each model is evaluated at its optimal guidance scale—yields a substantial extra FID reduction only on backbones conditioned on a single pooled text token (see W3). We will therefore present gate-free AMDD as our core method and reposition the gate as an optional module primarily for pooled-token backbones, revising Section 4.3 accordingly.

[W2] The performance drop at large KK is not caused by loss reweighting

We thank the reviewer for pointing out this confounding factor in Table 4. To isolate the effect of KK from loss reweighting, we sweep KK while holding the relative weights of the motion and distance denoising losses constant—averaging each stream's loss over its own channels and fixing their weights in Lsimple\mathcal{L}_{\mathrm{simple}} to the K=64K=64 ratio (51% motion / 49% distance) for MDM (BERT-Dec) on HumanML3D, and to Lmotion+0.5Ldist\mathcal{L}_{\mathrm{motion}} + 0.5\mathcal{L}_{\mathrm{dist}} for UniMoFlow on SnapMoGen. Both sweeps use the variance-ranked pair family. Results are reported below.

KKHumanML3D, MDM (BERT-Dec): FID T2M / MARDM (s=2.5s=2.5) ↓SnapMoGen, UniMoFlow: FID (s=4.5s=4.5) ↓
320.132 / 0.15615.16
640.152 / 0.15214.99
1280.227 / 0.30615.37
210 (dense set)0.365 / 0.49317.95

Even with the relative loss weights held fixed, FID still degrades beyond K=64K=64 on both datasets, confirming that the drop is not caused by placing a larger weight on the distance loss. Instead, the degradation points to representational redundancy: on the training sets of both benchmarks, the distances of pairs 65–128 are nearly linear functions of the first 64 pairs, with a median R2R^2 of 0.98. We will include this fixed-weight sweep and redundancy analysis in Section 4.3.

AuthorsOpenReview ↗

Response to Reviewer czuB (2/2)

[W3] Apart from text dilution, only the optional gate depends on text injection, and AMDD transfers across all five backbones

We thank the reviewer for raising this question. Below, we show that gate-free AMDD is sensitive to text injection only when attention dilution occurs, and that both the distance stream and one-way coupling transfer consistently across five backbones.

Sensitivity to text injection. Our tested backbones inject text either as a token sequence via cross-attention (MDM BERT-Dec, UniMoFlow, and CMDM) or as a single pooled token (MDM CLIP-Enc and RAM). Gate-free AMDD lowers FID under both mechanisms (second table below). The only case where performance degrades is the original MDM (CLIP-Enc) architecture in Appendix B.1, where a single pooled text token is prepended once to the input sequence and shares global self-attention with both the motion tokens and the added distance tokens. To test whether attention dilution causes this degradation, we keep the original prepended architecture unchanged and simply repeat the prepended text token 32 times:

MDM (CLIP-Enc), original text injection (s=2.5s=2.5, 500k)FID (T2M / MARDM) ↓
Backbone alone (1 prepended token)0.321 / 0.619
+ AMDD (1 prepended token)0.675 / 0.890
+ AMDD (text token prepended 32 times)0.285 / 0.367

*Appendix B.1 reports the best checkpoint of the second row, 0.633 / 0.838 at 300k.

Two findings confirm that the degradation stems from attention dilution.

First, with the distance stream unchanged, repeating the prepended text token 32 times eliminates the degradation and lowers FID below the backbone without adding any trainable parameters, showing that the issue lies in the text token's share of attention rather than model capacity.

Second, an inference-only probe on the trained models (Section 3 of our anonymous page, https://anonymous.4open.science/w/AMDD-supps-F523/#textpath) measures how strongly swapping the caption alters the motion features. Under the original single-token injection, adding the distance stream weakens the caption's influence by 39%, whereas repeating the token 32 times largely restores it. Under per-stream text attention—where each stream attends to the pooled text token in its own layer (Appendix B.1, used by our MDM CLIP-Enc and RAM backbones)—this weakening is reduced to 19%, improving FID further to 0.220 / 0.285 (table below). We will add the repeated-token experiment and sensitivity probe to Appendix B.1.

Transfer across backbones and coupling direction. In the table below, the first three FID columns use a uniform guidance scale (s=2.5s=2.5) at a fixed checkpoint on HumanML3D, while the final column reports FID(gated AMDD)−FID(AMDD)\text{FID}(\text{gated AMDD}) - \text{FID}(\text{AMDD}) when each model is evaluated at its own optimal guidance scale (negative values indicate that the gate reduces FID).

BackboneText inputBackbone alone+ AMDD+ gated AMDDGated − gate-free, best scale
MDM (BERT-Dec)Token sequence0.214 / 0.3080.161 / 0.1690.069 / 0.084−0.016 / −0.001
UniMoFlowToken sequence0.186 / 0.1670.136 / 0.1200.151 / 0.118+0.011 / +0.008
CMDM (uMT5-XXL)Token sequence0.075 / 0.0640.046 / 0.0400.048 / 0.039+0.002 / +0.001
MDM (CLIP-Enc)Pooled token, per-stream text attention0.266 / 0.3430.220 / 0.2850.055 / 0.052−0.093 / −0.138
RAMPooled token, per-stream text attention0.099 / 0.0910.093 / 0.0760.054 / 0.043−0.035 / −0.032

At s=2.5s=2.5, gate-free AMDD consistently lowers FID across all five backbones. In contrast, at each model's optimal guidance scale, the gate provides a substantial extra FID reduction only on the two pooled-token backbones (last column). Because the gate reads the caption and modulates the distance stream's updates, it provides an additional pathway for text conditioning. Our sensitivity probe confirms that this pathway is most critical when text enters as a single pooled token: swapping the caption only inside the gate produces 67%–86% of the total caption-induced feature change on the two pooled-token backbones, versus 11%–48% on the token-sequence backbones. Finally, our one-way coupling design also transfers to the new backbones: on SnapMoGen, replacing one-way with bidirectional coupling increases FID on both UniMoFlow and CMDM, mirroring the MDM (BERT-Dec) results in Table 3(c).

We thank the reviewer again for the constructive questions, which help us clarify the respective roles of the distance stream and the gate across different architectures.

Reviewer_cpZ6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AMDD, which generates sparse continuous inter-joint distances jointly with motion in a diffusion model. Separate streams interact through attention that permits motion to read distance features but prevents the reverse flow. A geometric consistency loss connects predicted motion to predicted distances, and a text-conditioned gate modulates the distance stream. Experiments on HumanML3D and SnapMoGen report lower FID across the tested backbones. Additional analyses examine alternative couplings and auxiliary representations, denoising errors, contact behavior, and the interaction between the gate and classifier-free guidance (CFG). The evidence supports a useful distance-stream contribution, while the final gate introduces a tradeoff between distributional scores and some text-dependent relations.

Critical strengths

S1. The distance stream has evidence beyond the final FID

In Table 3(a) (p. 6), adding the distance stream with the joint denoising objective already reduces T2M FID from 0.214 to 0.188, before the additional losses and gate. The gate-free K=64 model also reduces the hand-contact distribution gap from 4.1 to 1.8 cm, significantly under the cluster-based comparison (Table 5, p. 8; Table J.2 and discussion, p. 37). These results support the central idea independently of the final gate. Figure 3 (p. 8) gives complementary denoising evidence, appropriately scoped in Appendix I (p. 35) to one-step predictions on noised real clips.

S2. The controls and failure analyses make the contribution assessable

The paper compares loss-only supervision, concatenation, both coupling directions, bidirectional coupling and larger single-stream models (Table 3, p. 6), as well as alternative auxiliary representations (Table 4, p. 8). It also reports the gate's contact failures and its dependence on CFG (Appendix G, pp. 30–31). This transparency helps identify which component is useful and prevents the overall FID result from being mistaken for uniform improvement in text adherence.

Critical weaknesses

W1. The final gate improves FID while degrading some intended relations

The final AMDD increases the no-contact distribution gap from 1.3 to 4.3 cm (Table 5, p. 8), and its hand-contact gap of 2.3 cm is not significantly better than Base under the preferred cluster weighting (Appendix J, p. 37). Head-contact success falls from 87.5% to 78.0% at the reported 550k checkpoint (Table J.3, p. 38). Appendix G (pp. 30–31) links these effects to a broadly shared contraction amplified by CFG; Table G.2 (p. 30) shows that the gate's FID advantage is not established without guidance. The authors acknowledge these results, but they materially limit an interpretation of the final gate as more accurate text-specific relational modeling.

W2. The uniqueness of the distance representation is not equally established across datasets

On SnapMoGen, the position-stream control reaches FID 19.01 versus 19.10 for the corresponding K=32 distance stream, although distance TOP-1 is somewhat higher (Table H.1, p. 33). In the same configuration, removing the geometric consistency loss or using only the joint denoising loss does not worsen FID (Table H.2, p. 33). Thus, the convincing HumanML3D attribution does not fully establish that distances and the complete objective are uniquely responsible across datasets. This matters because final AMDD uses 61.0M rather than 26.2M parameters and takes 1.83 rather than 0.65 seconds per clip at batch size 1 (Table C.1, p. 22).

Overall assessment

3 / 4. Weak acceptance. The explicit distance state is a useful contribution supported by controls and contact evidence even without the final gate. My recommendation is near the acceptance boundary because the final gate does not improve all relations, and the distance-specific advantage on the second dataset is less clear. Answers that distinguish the core stream from the gate and appropriately narrow the claims would strengthen this recommendation.

Questions for authors

Q1. Can a learned text-independent gate, or a gate receiving shuffled text, reproduce the FID reduction? Appendix G (pp. 30–31) already provides useful guidance switches; a control separating text-specific adaptation from shared distribution calibration would materially affect the interpretation of the gate.

Q2. Which reported model or sampling setting best preserves relational correctness while retaining the FID gain? Please compare the gate-free, read-side and final write-side variants using the same Hand, Head and No-contact distribution metrics and matched training steps (Tables G.1–G.3, pp. 30–31; Tables J.2–J.3, pp. 37–38). Existing results or a narrower statement of the final model's benefit could resolve this concern.

Q3. How should the distance-specific claim be interpreted in light of the position-stream control and loss ablations on SnapMoGen (Tables H.1–H.2, p. 33)? A matched comparison over training seeds, with the same selection rule and loss weighting, would help distinguish a distance-specific advantage from the benefit of an auxiliary geometric stream.

3 comments
AuthorsOpenReview ↗

Response to Reviewer cpZ6 (1/3)

We thank the reviewer for recognizing that the distance stream is supported by evidence beyond overall FID, and that our control experiments and failure analyses allow the contribution of each component to be clearly assessed. Your suggestions on separating the core stream from the gate, comparing the gate variants under the same relational metrics, and clarifying the scope of the distance-specific claim help us narrow and better substantiate our claims. Below, we provide additional evidence and explain the corresponding revisions. As noted in the General Response, AMDD denotes the gate-free model, and gated AMDD denotes the submitted model.

Following your recommendation, we will present gate-free AMDD as our core contribution and the gate as an optional component. Across all three backbones conditioned on a text token sequence (MDM with BERT-Dec, UniMoFlow, and CMDM), gate-free AMDD consistently lowers FID while preserving or improving inter-joint relations, significantly narrowing the Hand contact gap without widening the remaining contact gaps (Q2, Q3). Meanwhile, the gate provides a substantial extra FID gain primarily on backbones conditioned on a single pooled text token, such as MDM (CLIP-Enc) and RAM (see our response to Reviewer czuB, W3).

[W1, Q1] Neither a fixed gate nor a shuffled-text gate reproduces the FID gain of the text-conditioned gate

We thank the reviewer for suggesting these controls, which we evaluate on HumanML3D using MDM (BERT-Dec) with K=64K=64, s=2.5s=2.5, and 500k steps across 5 repetitions on the full test set:

GateFID (T2M / MARDM) ↓R@1 (T2M / MARDM) ↑
None (gate-free AMDD)0.161 / 0.1690.548 / 0.540
Fixed: one learned vector per layer, shared by all texts (trained from scratch)0.137 / 0.1730.542 / 0.538
Shuffled text: the gate reads another sample's text in training and sampling (trained from scratch)0.142 / 0.1770.545 / 0.547
Text-conditioned (gated AMDD)0.069 / 0.0840.540 / 0.541
Gated AMDD, gate given another sample's text at inference only0.809 / 0.9280.249 / 0.252

Compared with gate-free AMDD, the fixed gate recovers only about a quarter of the T2M FID reduction (0.161 → 0.137, versus 0.069 with gated AMDD), and the shuffled-text gate recovers about a fifth (0.142). Neither control recovers any MARDM FID gain, confirming that the gate's benefit goes beyond a shared distributional calibration. Consistently, feeding another sample's text to the trained gate at inference degrades FID to 0.809 / 0.928. This refines our discussion in Section 4.3, which described the gate's modulation as largely text-independent: the shared component alone accounts for at most a quarter of the T2M FID gain.

At the same time, we agree with W1 that the gate does not improve inter-joint relational accuracy, and we will remove this claim in the revised paper. Moreover, on MDM (BERT-Dec), the gate's FID advantage largely diminishes once each model is evaluated at its own optimal guidance scale (0.069 / 0.084 vs. 0.085 / 0.085 for gate-free AMDD; see Q2). Instead, the gate is most beneficial on backbones conditioned on a single pooled text token, where it provides an additional pathway for text conditioning and yields a pronounced FID reduction (detailed in our response to Reviewer czuB, W3). We will revise Section 4.3 and Appendix G accordingly.

AuthorsOpenReview ↗

Response to Reviewer cpZ6 (2/3)

[W1, Q2] Gate-free AMDD with K=64K = 64 preserves relational correctness while retaining the FID gain

We thank the reviewer for requesting a matched comparison across model variants. We evaluate Base, gate-free AMDD, the read-side gate, and the write-side gate at a matched training step of 500k on HumanML3D, reporting each model at its optimal guidance scale alongside the Hand, No-contact, and Head distribution gaps. Results are reported below.

Model (all at 500k steps)Hand ↓No contact ↓Head ↓FID (T2M / MARDM) ↓
Base3.72.12.10.102 / 0.195
Gate-free AMDD, K=64K=641.8*1.72.30.085 / 0.085
Gate-free AMDD, K=32K=322.82.74.3†0.049 / 0.113
Read-side gate (Ne=1N_e=1), K=32K=321.2*2.93.70.058 / 0.052
Read-side gate (Ne=1N_e=1), K=64K=641.7*1.82.60.101 / 0.129
Write-side gate (Ne=4N_e=4), K=32K=322.56.0†3.50.113 / 0.133
Write-side gate (Ne=4N_e=4), K=64K=64 (gated AMDD)2.44.5†2.40.069 / 0.084

Contact gaps to real motion in cm, as in Table 5. All AMDD variants use the variance family. * / †: significantly smaller / larger than Base (p<0.05p < 0.05).

Among all compared configurations, gate-free AMDD with K=64K=64 best preserves relational correctness while retaining the FID reduction. Compared with Base, it substantially lowers FID (from 0.102 / 0.195 to 0.085 / 0.085) and significantly reduces the Hand gap (from 3.7 to 1.8 cm), while keeping the No-contact and Head gaps close to Base. The gated variants fail to maintain this balance: at K=32K=32, the read-side gate achieves the smallest Hand gap (1.2 cm) but enlarges the No-contact and Head gaps (2.9 and 3.7 cm vs. 2.1 cm); at K=64K=64, its contact gaps match those of gate-free AMDD, but its FID is worse (0.101 / 0.129). Meanwhile, the write-side gate significantly enlarges the No-contact gap at both K=32K=32 and K=64K=64. We will add this matched comparison to Section 4.4 and attribute our relational accuracy claims solely to gate-free AMDD.

AuthorsOpenReview ↗

Response to Reviewer cpZ6 (3/3)

[W2, Q3] Distances outperform positions in most settings, and the geometric losses improve contact accuracy

We thank the reviewer for asking how far the distance-specific claim extends across datasets and backbones. Our new experiments show that the improvement of gate-free AMDD has two components: adding a second geometric stream is broadly helpful, and parameterizing it as inter-joint distances yields a further gain in most settings. In addition, the two geometric losses primarily improve contact accuracy rather than global FID. We will frame our claims in this precise form.

Representation control. Table H.1 previously compared a 32-channel distance stream (K=32K=32) with a 72-channel position stream on SnapMoGen. To match the channel count and denoising-loss weight more closely, we now compare a K=64K=64 distance stream (rule family) with the 72-channel position stream on MDM (BERT-Dec), where the auxiliary streams take similar shares of Lsimple\mathcal{L}_{\text{simple}} (30.2% vs. 32.7%). Both models share the exact same architecture, loss coefficients, training recipe, and checkpoint-selection rule, and are evaluated at their respective best guidance scales. Across three random seeds, the distance stream achieves a lower FID for every seed:

SnapMoGen FID ↓ (300k, as in Table H.1)Seed 10Seed 20Seed 30Mean
+ position stream (72 channels)18.6018.9519.1118.89
+ distance stream (K=64K=64)18.0618.4318.3918.29

We further extend this control to UniMoFlow and CMDM across both SnapMoGen and HumanML3D. In addition to FID, we report relation dynamics, which measures the distributional gap between generated and real motions in 1-frame and 4-frame distance changes across non-bone joint pairs. The final column reports the relative difference in this gap between the distance and position streams (negative values favor distances, and * indicates a 95% CI excluding zero).

DatasetBackboneBackbone alone+ position stream+ distance stream (AMDD)Relation dynamics, distance vs. position (1-frame / 4-frame)
SnapMoGenMDM (BERT-Dec), 3-seed mean19.9218.8918.29+16%* / +17%*
SnapMoGenUniMoFlow15.8815.5214.80−16%* / −13%*
SnapMoGenCMDM (uMT5-XXL)15.2215.0014.67−7%* / −11%*
HumanML3DMDM (BERT-Dec)0.102 / 0.1950.067 / 0.1580.085 / 0.085−38%* / −34%*
HumanML3DUniMoFlow0.047 / 0.0470.034 / 0.0240.029 / 0.023+1% / +9%
HumanML3DCMDM (uMT5-XXL)0.048 / 0.0350.051 / 0.0400.037 / 0.031−7%* / −7%*

On HumanML3D, FID is reported as T2M / MARDM. For MDM (BERT-Dec), the backbone alone is Base, and its SnapMoGen row is evaluated at 300k as in Appendix H.

The position stream itself improves FID over the backbone alone in 5 of 6 settings, confirming that an auxiliary geometric stream is beneficial in general. Parameterizing this stream as inter-joint distances brings a consistent further improvement: the distance stream achieves a lower FID than the position stream in 8 of 9 comparisons and significantly improves relation dynamics in 4 of 6 settings.

Role of geometric losses. Removing the two geometric losses (Lgeo\mathcal{L}_{\text{geo}} and Laux\mathcal{L}_{\text{aux}}) has little impact on global FID once guidance is tuned: training with Lsimple\mathcal{L}_{\text{simple}} alone yields a similar FID to the full objective on HumanML3D (0.092 / 0.098 vs. 0.087 / 0.095 when evaluated together at the full model's best guidance scale) and is no worse on SnapMoGen (K=32K=32, s=2.5s=2.5: 18.66 vs. 19.10 in Table H.2). However, both losses are critical for contact accuracy. Training with Lsimple\mathcal{L}_{\mathrm{simple}} alone increases the Hand contact gap from 1.8 to 2.7 cm (no longer significantly below Base) and widens the Head gap from 2.3 to 3.6 cm (significantly worse than the full objective). We will clarify in Section 4.3 that the geometric losses primarily enforce fine-grained contact accuracy.

Computational cost. Gate-free AMDD uses 51.5M parameters and takes 1.19 s per clip at batch size 1, which is lower than the submitted gated model (61.0M, 1.83 s) though higher than Base (26.2M, 0.65 s). As shown in Table 3(b), simply widening or deepening Base to 51.5M parameters worsens FID, confirming that the gain does not stem from extra capacity alone. We will report these parameter and runtime comparisons directly in the main text.

We thank the reviewer again for the constructive suggestions, which help us separate the core contribution from the gate and formulate our distance-specific claims more precisely.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers and AC

We thank the reviewers for their careful reading and constructive suggestions. We are encouraged that they find the paper well written and clearly structured (Reviewer czuB), the method simple and usable as a drop-in addition (Reviewer jzWC), and the controls and failure analyses transparent enough to show which component is useful (Reviewer cpZ6).

Our central contribution is an explicit relational state that is denoised jointly with motion through one-way distance-to-motion attention. The additional results across backbones, relational metrics, and control tasks support this design independently of the gate. We will therefore present gate-free AMDD as the core method and the text gate as an optional component whose FID benefit is related to sampling setting. Throughout our responses, AMDD denotes this gate-free model, and Gated AMDD denotes the submitted model (labeled AMDD in the paper's tables).

Below, our responses to Reviewers cpZ6 and czuB report new control experiments on the gate, a position stream in place of the distance stream, fixed loss weights across KK, and text-injection mechanisms. Our response to Reviewer jzWC presents relation-level generation control and qualitative comparisons. New videos and one diagnostic figure are provided on an anonymous page (https://anonymous.4open.science/w/AMDD-supps-F523/), while the paper PDF remains unchanged.

[G1] AMDD improves two recent backbones on both datasets

We evaluate AMDD on two recent continuous backbones, UniMoFlow [1] and CMDM [2], where CMDM is retrained with a uMT5-XXL text encoder (see CMDM setting below). Within each pair, the backbone and its AMDD version share the exact same training data, text encoder, training recipe, and sampler. We also list representative discrete-token methods from Tables 1 and 2 for reference. Results on SnapMoGen and HumanML3D are reported below.

MethodSnapMoGen FID ↓SnapMoGen TOP-1 ↑HumanML3D FID (T2M / MARDM) ↓HumanML3D R@1 (T2M) ↑
MoMask17.400.7770.045 / 0.1160.521
MoMask++15.060.8020.072 / —0.528
ScaleMoGen16.350.8070.030 / —0.577
UniMoFlow15.880.8420.047 / 0.0470.571
UniMoFlow + AMDD14.800.8550.029 / 0.0230.579
CMDM (uMT5-XXL)15.220.8190.048 / 0.0350.565
CMDM (uMT5-XXL) + AMDD14.670.8330.037 / 0.0310.567

Guidance scale. Each backbone and its AMDD counterpart are evaluated over the same grid of guidance scales and reported at their respective best scales, with AMDD permitting separate scales for the motion and distance streams. Tables that specify a single scale apply it uniformly to all rows (s=2.5s=2.5 is the default scale in the paper's ablations).

CMDM setting. Both CMDM rows use a uMT5-XXL text encoder, whereas the published CMDM uses DistilBERT. In our re-run, the published DistilBERT configuration reaches a SnapMoGen FID of 15.37, compared with the reported 14.45. We will document both settings and these reproduction results in the revised paper.

[1] UniMoFlow: Grounding Instruction-Driven 3D Human Motion Editing in Generation. arXiv 2026.

[2] Causal Motion Diffusion Models for Autoregressive Motion Generation. CVPR 2026.

AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

We sincerely appreciate the time and effort of the AC and the reviewers. We have provided detailed, point-by-point responses to every weakness and question.

Per-reviewer Outcomes

We appreciate that Reviewer #cpZ6 recognized the distance stream as “a useful contribution”. Reviewer #czuB highlighted the “comprehensive experiments” and clear presentation. Reviewer #jzWC described the approach as a “simple method” for improving diffusion-based generation.

The reviewers also requested further clarification and additional experiments, which we summarize below.

Reviewer #czuB:

The reviewer focused on text-dependent gate effects, loss weighting, and backbone transfer. We provided gate controls, experiments with fixed loss weights, and text-injection analyses. The reviewer confirmed that the rebuttal “solve my problem”. Their rating remains 3 (Weak acceptance).

Reviewer #cpZ6:

The reviewer focused on relational fidelity and the distance-specific contribution. We separated the core stream from the optional gate, narrowed the claims, and provided matched model and representation comparisons. The original review stated: “Answers that distinguish the core stream from the gate and appropriately narrow the claims would strengthen this recommendation.” (score = 3, Weak acceptance).

Reviewer #jzWC:

The reviewer focused on controllability, preservation of joint relations, and practical competitiveness. We provided control experiments using the same generative model without control-specific training, comparisons with adapted OmniControl, direct relation measurements, and qualitative examples. Results on UniMoFlow and CMDM also show competitive or better performance against leading discrete-token methods. (score = 2, Weak rejection).

Current status: We are grateful that Reviewer #czuB confirmed that our responses resolved their concerns and maintained 3 (Weak acceptance). We sincerely look forward to hearing from Reviewers #cpZ6 and #jzWC when they have an opportunity to review our rebuttal.

We hope this summary demonstrates our efforts to thoroughly address the reviewer feedback. We sincerely thank the AC and all reviewers for helping improve our work and will incorporate these results and clarifications into the final version.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.