acceptodds
Withdrawn from ICLR 2027

MotionSpaceFlow: Representation-Aware Flow Matching in Direct Motion Space

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in diffusion and flow models have substantially improved text-driven human motion generation. Yet most methods generate in low-dimensional, temporally downsampled latent spaces learned primarily for reconstruction, a bottleneck that can limit generation quality and preclude direct manipulation of individual frames and joints. We introduce MotionSpaceFlow (MSFlow), a representation-aware flow-matching framework that predicts clean motion directly in continuous motion space without a learned encoder or decoder. To account for the anisotropic structure of direct motion representations, we propose representation-aware noise scaling and show how the initial Gaussian source scale governs the covariance of intermediate probability-path marginals. We further introduce a Representation-Aware Multimodal Diffusion Transformer (RA-MMDiT), which jointly updates token-level language and full-resolution motion features through joint attention while adapting temporal information flow to the motion representation: causal attention for incremental features defined by frame-to-frame changes, and bidirectional attention for global features such as absolute joint coordinates. Across different datasets and motion representations, MSFlow achieves state-of-the-art text-to-motion performance. Its global representation variant additionally enables zero-shot, inference-time control over any joint or frame through projection sampling without control-conditioned training, delivering leading motion quality with exact constraint satisfaction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Motion Generation, Flow Matching, Zero-shot controllable generation

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_wSyyOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

MSFlow is a T2M generator that performs flow-matching directly in motion space, without VAE. It predicts the clean motion (x-prediction), uses a larger Gaussian source scale, and adopts an MMDiT backbone (RA-MMDiT) with a causal attention mask for incremental 263D features and a bidirectional mask for absolute XYZ coordinates. For XYZ, it applies ProjFlow-style projection for zero-shot spatial control. It reports strong results on HumanML3D, SnapMoGen and spatial-control benchmarks.

Critical strengths

Simple and efficient. A compact model (68M) achieves competitive quality at low inference cost, making direct motion-space generation a practical baseline.

Critical weaknesses
  1. This paper overclaims its contributions with little novelty. Most components presented as contributions are already established in the community: 1.1 Direct motion-space flow matching without VAE was already done by HY-Motion 1.0, and even earlier by raw-space diffusion models such as MDM. 1.2 Clean-endpoint x-prediction follows JiT, and clean-sample prediction has long been used in motion diffusion (e.g., MDM). 1.3 The core of RA-MMDiT is choosing a causal or bidirectional attention mask according to the motion representation. This is a hyper-parameter choice between two standard masks, both already used in prior work (e.g., causal masking in CMDM, text-to-motion masking in HY-Motion), rather than a new architecture. 1.4 The zero-shot control sampler explicitly adopts ProjFlow's projection.

  2. The empirical claims are not convincingly supported. Under the standard HumanML3D evaluator, MSFlow does not even surpass MoMask in FID (0.057 vs. 0.045, App. Table 7).

Questions for authors
  1. Beyond HY-Motion's direct motion-space MMDiT, what is genuinely new in MSFlow?
  2. Under the standard HumanML3D evaluator, MSFlow does not surpass MoMask in FID. Why is the MARDM 67D evaluator used as the primary protocol?
4 comments

We thank Reviewer wSyy for noting that MSFlow is "simple and efficient" and that "a compact model (68M) achieves competitive quality at low inference cost."

Below, we address the two central concerns raised in your review:

  1. How MSFlow differs methodologically and empirically from HY-Motion 1.0 and MDM, and
  2. Why the 67D evaluator is used as the primary protocol, and how MSFlow compares to MoMask across all benchmarks.

1. Methodological & Empirical Differentiation Beyond HY-Motion 1.0 and MDM (Weakness 1 & Question 1)

"Beyond HY-Motion's direct motion-space MMDiT, what is genuinely new in MSFlow?"

A. Why Operating in Direct Motion Space Does Not Preclude Novelty

Operating directly in motion space does not, by itself, diminish MSFlow's novelty, just as MLD's introduction of latent motion diffusion does not diminish the contributions of subsequent methods such as MotionLCM, MARDM, SALAD, ACMDM, and CMDM. In both paradigms, the scientific contribution lies in identifying and resolving the fundamental bottlenecks that prevented the paradigm from achieving high fidelity. Prior to MSFlow, direct motion-space generators lagged far behind latent-space models in distributional fidelity; MSFlow is the first direct-space framework to surpass state-of-the-art latent methods.

B. Head-to-Head Comparison: Our Open 68.8M Model Outperforms the Closed 1.0B HY-Motion 1.0 Model by 9.3× in FID (Table 8)

Comparing HY-Motion 1.0 and MSFlow directly highlights the difference:

  • HY-Motion 1.0 is a 1.0-Billion-parameter (1.0B) industrial model with no open-source training code and no public training data.
  • MSFlow is a compact 68.8M-parameter model (nearly 15× smaller), trained from scratch on public benchmarks with full training code provided.

In Table 8, we evaluated MSFlow directly against HY-Motion 1.0 and its post-trained follow-up UMO on the official 272D HumanML3D benchmark:

Direct-Space Model (272D Benchmark, Table 8)Model Scale & OpennessFID ↓R-Precision Top-1 ↑R-Precision Top-2 ↑R-Precision Top-3 ↑MM-Dist ↓
MDM (ICLR 2023)Open23.4540.5230.6920.76417.423
HY-Motion 1.0 (2025)1.0B params, Closed training code/data61.0350.6670.8180.87617.530
UMO-Expert (2026, HY-Motion + Post-training)460M backbone + post-training17.0400.7630.8890.93115.490
UMO-Unified (2026, HY-Motion + Post-training)460M backbone + post-training9.4600.7740.8920.93315.220
MotionSpaceFlow (Ours, Single Stage)68.8M params (15× smaller), Open code/data6.540 ± 0.1200.792 ± 0.0030.919 ± 0.0020.955 ± 0.00213.864 ± 0.014

Despite being 15× larger (1.0B vs. 68.8M) and trained on proprietary data, HY-Motion 1.0 obtains an FID of 61.03561.035, whereas our 68.8M MSFlow achieves 6.5406.540 (a 9.3× reduction in FID), alongside a +12.5 percentage-point improvement in Top-1 R-Precision (0.667→0.7920.667 \to 0.792), outperforming even the 460M fine-tuned UMO-Unified (9.4609.460 FID) across every single metric. These results demonstrate that MSFlow represents a substantial advance in direct motion-space generation, delivering higher motion quality and text alignment with a considerably smaller model.

C. Controlled Fixed-Backbone Comparison Against MDM's x0x_0-Prediction Objective

To isolate the contribution of our training recipe from that of the architecture, we trained MDM's exact x0x_0-predicting DDPM objective on our identical 68.8M RA-MMDiT network for 500 epochs:

8/8 method × representation × evaluator rows are complete. FLOW rows report the paper results (Tables 1 & 7); MDM rows report the new 500-epoch evaluations on the identical 68.8M network.

RepresentationEvaluator protocolTraining recipeSamplerFID ↓Top-3 ↑
Incremental 263D67DFLOW (Ours, Paper Table 1)Heun 50 points0.046 ± 0.0040.853 ± 0.003
Incremental 263D67DMDM (New, Same 68.8M Net)DDPM 1000 steps0.291 ± 0.0190.840 ± 0.007
Incremental 263Dstandard263FLOW (Ours, Paper Table 7)Heun 50 points0.057 ± 0.0030.863 ± 0.003
Incremental 263Dstandard263MDM (New, Same 68.8M Net)DDPM 1000 steps0.245 ± 0.0050.846 ± 0.001
Global XYZ67DFLOW (Ours, Paper Table 1)Heun 50 points0.038 ± 0.0040.849 ± 0.003
Global XYZ67DMDM (New, Same 68.8M Net)DDPM 1000 steps0.122 ± 0.0050.807 ± 0.005
Global XYZstandard263FLOW (Ours, Paper Table 7)Heun 50 points0.050 ± 0.0040.858 ± 0.002
Global XYZstandard263MDM (New, Same 68.8M Net)DDPM 1000 steps0.167 ± 0.0040.822 ± 0.003

On the exact same 68.8M network, replacing the MSFlow formulation with MDM's x0x_0-DDPM objective degrades FID by 6.3× on 67D (0.2910.291 vs. 0.0460.046) and 4.3× on Standard 263D (0.2450.245 vs. 0.0570.057) for 263D motion, and by 3.2×–3.3× for XYZ motion (0.1220.122 vs. 0.0380.038 on 67D; 0.1670.167 vs. 0.0500.050 on Standard 263D) while requiring 20× more sampling steps (1,000 vs. 50).

Why MSFlow succeeds where MDM and HY-Motion lag:

  1. Velocity-Weighted Endpoint Regression (Eq. 5–6): Converting x^1\hat{\mathbf{x}} _ 1 into an ODE velocity field weights endpoint errors by dt−2=max⁡(1−t,σmin⁡)−2d _ t^{-2} = \max(1-t, \sigma _ {\min})^{-2}, concentrating supervision on fine kinematic structure near t→1t \to 1 rather than weighting all noise levels equally.

  2. Representation-Aware Source Scaling (s=5s=5): Expanding the source noise scale mitigates the extreme spectral anisotropy of uncompressed motion spaces (Proposition 2, Table 13), cutting FID by 58.6% on 263D (0.111→0.0460.111 \to 0.046) and 73.6% on XYZ (0.144→0.0380.144 \to 0.038) compared to unit-scale flow matching (s=1s=1, Table 3).

  3. Representation-Aware Multimodal Routing & Cross-Modal Leak Prevention: Viewing RA-MMDiT as a simple toggle between two masks overlooks the structural interaction demonstrated by our 8-model factorial study (see table in Response to qJT2): applying a causal motion mask inside a standard bidirectional MMDiT (Causal + Full text feedback) creates a 2-hop future-to-past causal leak through the text tokens, degrading 263D FID from 0.0460.046 to 0.4510.451 (nearly 10× worse). Conversely, applying causal masking to Global XYZ degrades FID from 0.0380.038 to 1.5631.563 (41× worse). Co-designing temporal routing, cross-modal feedback blocking, and source-noise scaling for each representation is what enables MSFlow to outperform HY-Motion 1.0 by 9.3× and fixed-backbone MDM by 6.3×.

2. Why the 67D Evaluator Measures Actual Motion Quality, Whereas MoMask Overfits to 196 Discarded Channels in 263D (Weakness 2 & Question 2)

"Under the standard HumanML3D evaluator, MSFlow does not surpass MoMask in FID (0.057 vs. 0.045, App. Table 7). Why is the MARDM 67D evaluator used as the primary protocol?"

  1. In HumanML3D, Only the 67D Features Construct the Actual 3D Motion, and the Remaining 196 Channels Are Discarded: As established by MARDM (CVPR 2025), ACMDM (arXiv 2025), and CMDM (CVPR 2026), when the HumanML3D pipeline reconstructs and renders an actual 3D skeletal motion from a 263D feature vector, only the first 67 dimensions (4 root trajectory parameters + 21×3=6321 \times 3 = 63 root-relative 3D joint positions) are used to recover the 3D human motion. The remaining 196 dimensions (redundant local velocities, IK-approximated 6D rotations, and binary foot-contact labels) are completely discarded during motion recovery and do not directly determine the rendered 3D motion.
    • Because the standard 263D evaluator takes 259 channels as input, 192 of its 259 input channels (74%) are redundant features that are thrown away when producing the actual 3D motion.
    • In contrast, the 67D evaluator operates exclusively on the exact 67 kinematic features that define the rendered 3D motion, providing a direct measure of true motion quality across both 263D and 66D XYZ representations.
  2. MoMask's Degradation from 263D FID (0.0450.045) to 67D FID (0.1160.116) Demonstrates Overfitting to the 196 Discarded Channels: Comparing MoMask and MSFlow across all three HumanML3D evaluation protocols exposes this effect clearly:
Method67D FID (Actual 3D Motion) ↓67D Top-1 / Top-3 ↑Std-263D FID ↓Std-263D Top-1 / Top-3 ↑Std-263D MM-Dist ↓272D FID ↓272D Top-3 ↑
MoMask (Discrete VQ-VAE)0.116 (3.1× worse)0.490 / 0.7860.0450.521 / 0.8072.95812.2320.846
CMDM (Continuous VAE)0.0780.563 / 0.8490.0680.588 / 0.8602.620——
FloodDiffusion (Continuous VAE)——0.0570.523 / 0.8102.887——
MSFlow (263D, Ours)0.0460.571 / 0.8530.0570.591 / 0.8632.5656.5400.955
MSFlow (XYZ, Ours)0.0380.566 / 0.8490.0500.581 / 0.8582.607——
  • Why does MoMask achieve 0.0450.045 FID on 263D but degrade to 0.1160.116 FID on the actual 67D motion? Because MoMask's discrete VQ-VAE codebook is trained to reconstruct all 263 channels, where the 196 unused channels outnumber the 67 actual motion channels by nearly 3-to-1. MoMask overfits to the statistics of those 196 discarded channels, while its actual 3D joint positions suffer from VQ quantization errors, resulting in a real 67D motion FID of 0.1160.116, which is 3.1× worse than MSFlow XYZ (0.0380.038) and 2.5× worse than MSFlow 263D (0.0460.046).
  • Even under the Standard 263D Evaluator (Table 7), MSFlow achieves superior overall performance:
    • MSFlow (XYZ) achieves 0.0500.050 FID, outperforming every continuous-latent, diffusion, and flow-matching model in the literature (MLD V2: 0.0780.078, SALAD: 0.0760.076, MARDM: 0.1160.116, CMDM: 0.0680.068, FloodDiffusion: 0.0570.057, MDM: 0.4890.489, ReMoDiffuse: 0.1030.103), despite generating only 66D joint coordinates without training on the 196 redundant channels.
    • On Text-Motion Alignment (Standard 263D), MSFlow substantially outperforms MoMask: MSFlow (263D) surpasses MoMask by +7.0 percentage points in Top-1 R-Precision (0.5910.591 vs. 0.5210.521), +5.6 percentage points in Top-3 (0.8630.863 vs. 0.8070.807), and achieves the #1 best MM-Dist (2.5652.565 vs. MoMask's 2.9582.958) across all 16 methods in Table 7.
    • Under the 272D Evaluator (Table 8), where all 272 channels are directly utilized for SMPL animation, MSFlow outperforms MoMask by nearly 2× in FID (6.5406.540 vs. 12.23212.232) and +10.9 percentage points in Top-3 R-Precision (0.9550.955 vs. 0.8460.846).

Summary for Reviewer wSyy: MSFlow outperforms the 1.0B HY-Motion 1.0 model by 9.3× in FID with a 15× smaller open model, outperforms fixed-backbone MDM by 3.2×–6.3× in FID, and outperforms ACMDM's paper raw velocity-prediction model by 3.2× in unconditional FID (0.0380.038 vs. 0.1210.121) as well as our source-scaled clean-prediction ACMDM raw-flow variant by 2.3× in unconditional FID (0.0380.038 vs. 0.0860.086) and by 1.62× in controlled FID (0.10610.1061 vs. 0.17220.1722) under the identical shared ProjFlow sampler. Meanwhile, MoMask's narrow margin on 263D FID (0.0450.045 vs. 0.0500.050) is an artifact of overfitting to 196 redundant channels that HumanML3D discards during motion recovery. When evaluated on the actual 67D 3D motion, MSFlow outperforms MoMask by 3.1× in FID (0.0380.038 vs. 0.1160.116) while also surpassing MoMask by +7.0% in 263D Top-1 R-Precision. We respectfully ask you to re-evaluate our work based on this evidence and raise your rating.

Reviewer_k446OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MotionSpaceFlow (MSFlow), a text-to-motion generation framework that applies flow matching directly to continuous motion representations without relying on a learned motion autoencoder. The method supports both the standard 263D HumanML3D representation and global XYZ joint coordinates. The proposed framework combines clean-motion endpoint prediction, Gaussian source-noise scaling, and a Representation-Aware Multimodal Diffusion Transformer (RA-MMDiT). The authors adopt causal attention for incremental motion representations and bidirectional attention for global joint coordinates. The framework also supports training-free spatial control through inference-time projection sampling. Experiments on HumanML3D and SnapMoGen demonstrate competitive or state-of-the-art performance across several evaluation metrics.

Critical strengths
  1. Strong empirical performance on established benchmarks. The proposed method achieves impressive results on HumanML3D, particularly in terms of FID. The global XYZ variant achieves an FID of 0.038 under the 67D evaluation protocol, substantially improving over several existing methods. The authors also report competitive results under the standard 263D evaluation protocol and strong text-motion alignment on SnapMoGen.

  2. Relatively comprehensive experimental analysis. The authors provide ablations on prediction targets, Gaussian source scales, attention configurations, model capacity, and text-motion conditioning. The additional experiments on different motion representations and evaluation protocols strengthen the empirical evaluation.

  3. Practical flexibility for spatially controlled motion generation. The direct XYZ formulation supports joint-level and frame-level constraints without additional control-conditioned training. Although the projection mechanism is largely inherited from previous work, the demonstrated combination of generation quality and controllability is practically useful.

Critical weaknesses
  1. Limited methodological novelty compared with existing direct motion-space generation methods. For example, MDM (ICLR 2023) directly applies diffusion to continuous motion representations and predicts clean motion samples. ReMoDiffuse (ICCV 2023) and GMD (ICCV 2023) further demonstrate diffusion-based motion generation and control without requiring a separately learned motion VAE. More recently, ACMDM investigates direct generation in raw global XYZ coordinates, including flow-matching formulations, while ProjFlow (CVPR 2026) demonstrates training-free spatial control using a flow-matching motion prior. Although the authors acknowledge several of these studies, the distinction between the proposed method and existing direct-space approaches remains insufficiently established.

  2. The contribution of representation-aware noise scaling is not sufficiently distinguished from conventional noise-scale selection. The covariance analysis demonstrates that increasing isotropic Gaussian source variance improves the condition number of intermediate probability-path marginals. While mathematically correct, this is a general property of adding isotropic covariance rather than a result specific to human motion representations. It therefore remains unclear whether the reported improvements primarily result from better covariance conditioning, a modified signal-to-noise schedule, or conventional hyperparameter tuning.

  3. The independent novelty of inference-time spatial control is limited. The proposed projection sampler follows ProjFlow and adopts its kinematics-aware metric and pseudo-observation construction. Furthermore, exact constraint satisfaction is ensured by the final hard projection. The improved controllable-generation results are encouraging, but it remains unclear how much of the improvement comes from the proposed generative backbone rather than the previously established projection mechanism.

Questions for authors

Q1. Novelty relative to existing raw-motion generators. Given that MDM, ReMoDiffuse, and ACMDM have already investigated generation directly in continuous motion representations, could the authors clarify the specific technical challenges that remain unresolved by these methods and are uniquely addressed by MSFlow? In particular, a controlled comparison with an existing raw-motion flow-matching baseline, such as the raw-flow variant of ACMDM, would help establish the benefits of the proposed framework independently of the choice to remove the autoencoder.

Q2. Interpretation and selection of Gaussian source scale. The analysis shows that increasing the source scale improves intermediate covariance conditioning. However, it also changes the signal-to-noise ratio along the probability path. Could the authors clarify which of these effects is primarily responsible for the observed FID improvements? A more detailed source-scale sensitivity analysis, or a comparison with an alternative noise schedule, would help distinguish the proposed representation-aware interpretation from standard noise hyperparameter tuning.

Q3. Conditional diversity and evaluation stability. The standard 263D evaluation reports a substantially lower multimodality score for MSFlow than for several competing models, despite improvements in FID and R-Precision. Could the authors discuss this issue?

6 comments

We thank Reviewer k446 for recognizing our "impressive results on HumanML3D, particularly in terms of FID (0.0380.038)", our "comprehensive experimental analysis", and the "practical flexibility for spatially controlled motion generation." Below, we address each of your technical questions with controlled baseline comparisons and formal analysis.

1. Methodological Differentiation & Controlled Comparisons Against Raw-Space Baselines (Weakness 1 & Q1)

"Given that MDM, ReMoDiffuse, and ACMDM have already investigated generation directly in continuous motion representations, could the authors clarify the specific technical challenges that remain unresolved by these methods and are uniquely addressed by MSFlow? In particular, a controlled comparison with an existing raw-motion flow-matching baseline, such as the raw-flow variant of ACMDM, would help..."

What technical challenges remained unresolved in prior direct-space generators? Prior work operating directly in motion space does not, by itself, negate our novelty, just as MLD's use of latent diffusion does not diminish the novelty of subsequent methods such as SALAD, MARDM, ACMDM, and CMDM. The central scientific challenge in direct motion space is overcoming the severe fidelity and alignment gap relative to latent-space models. Earlier raw-space diffusion models such as MDM (ICLR 2023) and ReMoDiffuse (ICCV 2023) avoid a VAE, but suffer from poor distributional fidelity (Table 1: MDM FID is 0.5180.518, ReMoDiffuse is 0.8830.883, whereas MSFlow achieves 0.0380.038, a >13×>13\times reduction in FID). Similarly, ACMDM's raw-space flow variant underperformed its latent After-Patching autoencoder because naive direct-space flow matching fails to address severe spatiotemporal covariance anisotropy (Aspec>1800A _ {\text{spec}} > 1800, Table 13) and representation-specific temporal routing.

To demonstrate that MSFlow's performance stems from our representation-aware flow formulation rather than merely removing the autoencoder or increasing backbone size, we conducted two controlled experiments:

Experiment 1A: Controlled Comparison Across ACMDM Paper Raw, Source-Scaled Clean-Prediction ACMDM Raw-Flow, and MSFlow XYZ

In the original ACMDM paper (Table 1 of Meng et al., 2025), the authors already trained a direct-XYZ velocity-prediction (v\mathbf{v}) flow-matching model without an autoencoder (ACMDM-S-PS22 w/o AE, s=1s=1), which ProjFlow subsequently adopted as its base flow-matching prior. To evaluate whether applying our source-scaled clean-prediction flow formulation (s=5s=5, uniform tt, x^1→v\hat{\mathbf{x}}_1 \to \mathbf{v}) also improves the ACMDM raw-flow architecture, and how both compare to MSFlow (XYZ), we trained a 500-epoch (191,500-update) ACMDM raw-flow XYZ variant with our source-scaled clean-prediction recipe and evaluated all models under the 67D evaluation protocol:

Direct-XYZ Flow Model (500 Epochs, No Autoencoder)Flow Objective & Source ScaleFID ↓R-Precision Top-1 ↑R-Precision Top-2 ↑R-Precision Top-3 ↑MM-Dist ↓
ACMDM Paper Raw (ACMDM Table 1; used in ProjFlow)Direct Velocity (v\mathbf{v}), s=1s = 10.121 ± 0.0060.502 ± 0.0020.692 ± 0.0030.789 ± 0.0033.304 ± 0.008
ACMDM Raw-Flow (Our Source-Scaled Clean-Prediction Recipe)Clean-Prediction (x^1→v\hat{\mathbf{x}}_1 \to \mathbf{v}), s=5s = 5, Uniform tt0.086 ± 0.0080.518 ± 0.0080.715 ± 0.0050.808 ± 0.0013.207 ± 0.012
MSFlow XYZ (Ours, Paper Table 1)Clean-Prediction (x^1→v\hat{\mathbf{x}}_1 \to \mathbf{v}), s=5s = 5, Uniform tt0.038 ± 0.0040.566 ± 0.0040.759 ± 0.0030.849 ± 0.0032.916 ± 0.008

What this three-way comparison demonstrates:

  1. Our source-scaled clean-prediction recipe directly improves the ACMDM raw-flow backbone: Replacing ACMDM's original raw velocity-prediction recipe (0.1210.121 FID, 0.5020.502 Top-1, 0.7890.789 Top-3) with our source-scaled clean-prediction formulation (s=5s=5, uniform tt, x^1→v\hat{\mathbf{x}}_1 \to \mathbf{v}) reduces ACMDM's raw-space FID by 28.9% (0.121→0.0860.121 \to 0.086) and improves Top-3 R-Precision from 0.7890.789 to 0.8080.808 (and also improves over unit-scale s=1s=1 clean-prediction ACMDM raw-flow at 0.094±0.0070.094 \pm 0.007 FID and 0.766±0.0020.766 \pm 0.002 Top-3).
  2. MSFlow XYZ substantially outperforms both ACMDM raw-space models: MSFlow XYZ reduces FID by 3.2× compared to the ACMDM paper's raw velocity-prediction model (0.121→0.0380.121 \to 0.038) and by 2.3× compared to the source-scaled clean-prediction ACMDM raw-flow variant (0.086→0.0380.086 \to 0.038), while improving Top-1 R-Precision by +6.4 / +4.8 percentage points (0.502→0.518→0.5660.502 \to 0.518 \to 0.566) and Top-3 by +6.0 / +4.1 percentage points (0.789→0.808→0.8490.789 \to 0.808 \to 0.849).

Experiment 1B: Fixed-Backbone Comparison Against MDM's x0x_0-DDPM Recipe

To isolate the contribution of our training recipe from that of the architecture, we trained MDM's exact clean-motion (x0x_0) DDPM objective and 1,000-step cosine schedule on our identical 68.8M RA-MMDiT backbone (matching parameter count, text encoder, normalization, optimizer, and the 500-epoch / 191,500-update budget):

RepresentationEvaluator ProtocolIdentical 68.8M Network + Generative RecipeSamplerFID ↓R-Precision Top-3 ↑
Incremental 263D67DMSFlow Recipe (Ours, Paper Table 1)Heun (50 steps)0.046 ± 0.0040.853 ± 0.003
Incremental 263D67DMDM x0x_0-DDPM Recipe (New)DDPM (1000 steps)0.291 ± 0.0190.840 ± 0.007
Incremental 263DStandard 263DMSFlow Recipe (Ours, Paper Table 7)Heun (50 steps)0.057 ± 0.0030.863 ± 0.003
Incremental 263DStandard 263DMDM x0x_0-DDPM Recipe (New)DDPM (1000 steps)0.245 ± 0.0050.846 ± 0.001
Global XYZ67DMSFlow Recipe (Ours, Paper Table 1)Heun (50 steps)0.038 ± 0.0040.849 ± 0.003
Global XYZ67DMDM x0x_0-DDPM Recipe (New)DDPM (1000 steps)0.122 ± 0.0050.807 ± 0.005
Global XYZStandard 263DMSFlow Recipe (Ours, Paper Table 7)Heun (50 steps)0.050 ± 0.0040.858 ± 0.002
Global XYZStandard 263DMDM x0x_0-DDPM Recipe (New)DDPM (1000 steps)0.167 ± 0.0040.822 ± 0.003

With the 68.8M backbone held strictly fixed, MSFlow outperforms MDM's x0x_0-diffusion recipe by 6.3× in FID on 67D (0.0460.046 vs. 0.2910.291) and 4.3× on Standard 263D (0.0570.057 vs. 0.2450.245) for 263D motion, and by 3.2×–3.3× for XYZ motion (0.0380.038 vs. 0.1220.122 on 67D; 0.0500.050 vs. 0.1670.167 on Standard 263D) while requiring 20× fewer sampling steps (50 vs. 1,000).

2. Source-Scale Conditioning vs. SNR Scheduling (Weakness 2 & Q2)

"The analysis shows that increasing the source scale improves intermediate covariance conditioning. However, it also changes the signal-to-noise ratio along the probability path. Could the authors clarify which of these effects is primarily responsible for the observed FID improvements? A more detailed source-scale sensitivity analysis, or a comparison with an alternative noise schedule, would help..."

Your mathematical observation is exact: scaling the isotropic Gaussian source by ss simultaneously alters fixed-tt covariance conditioning and shifts the signal-to-noise ratio (SNR) schedule along t∈[0,1]t \in [0, 1], which is also equivalent to an alternative noise schedule. We formalize their exact relationship below.

Exact Algebraic Reparameterization and Finite-Network Mechanisms

For clean motion x\mathbf{x} and unit Gaussian noise ϵ∼N(0,I)\boldsymbol{\epsilon} \sim \mathcal{N}(\mathbf{0}, \mathbf{I}), define the amplitude factor A(t)=t+s(1−t)A(t) = t + s(1-t) and the warped time coordinate u(t)=tt+s(1−t)∈[0,1]u(t) = \frac{t}{t + s(1-t)} \in [0, 1]. The scaled-source probability path satisfies the exact identity:

xt(s)=tx+s(1−t)ϵ=A(t)[u(t)x+(1−u(t))ϵ]=A(t)xu(t)(1).\mathbf{x} _ t^{(s)} = t\mathbf{x} + s(1-t)\boldsymbol{\epsilon} = A(t)\Big[u(t)\mathbf{x} + \big(1 - u(t)\big)\boldsymbol{\epsilon}\Big] = A(t)\mathbf{x} _ {u(t)}^{(1)}.

We verified this identity numerically across 6 scales (s∈{1,2,3,5,7.5,10}s \in \lbrace 1, 2, 3, 5, 7.5, 10 \rbrace) and 1,000 time steps against empirical HumanML3D training covariances (maximum relative condition-number discrepancy 6.40×10−156.40 \times 10^{-15}). This yields two insights:

  1. Duality of Fixed-tt Conditioning and SNR Warping: Because scalar multiplication by A(t)A(t) preserves the condition number, κt(s)\kappa _ t(s) of the scaled-source path equals κu(t)(1)\kappa _ {u(t)}(1) of the unit-source path at matched u(t)u(t). Thus, Proposition 2(ii) (improved conditioning at fixed tt) and Proposition 2(i) (delayed signal emergence / SNR shift from tt to u(t)u(t), inducing density ps(u)=s[1+(s−1)u]2p _ s(u) = \frac{s}{[1+(s-1)u]^2} under t∼U(0,1)t \sim \mathcal{U}(0,1)) are dual characterizations of the exact same path geometry.
  2. Why s>1s > 1 Is Not Reducible to Time Warping Alone in a Neural Network: Even at matched uu, training an unpreconditioned network fθ(xt,t,c)f _ \theta(\mathbf{x} _ t, t, \mathbf{c}) on the ss-scaled path differs from a time-warped unit-source schedule in three concrete ways:
    • Input Amplitude Scaling (A(t)A(t)): Network inputs are scaled by A(t)=t+s(1−t)A(t) = t + s(1-t), directly modulating input variance across layers.
    • Terminal Velocity Loss Weighting: Converting x^1\hat{\mathbf{x}} _ 1 to velocity divides endpoint error by dt2=max⁡(1−t,σmin⁡)2d _ t^2 = \max(1-t, \sigma _ {\min})^2. Under the t↔ut \leftrightarrow u Jacobian, the effective weight becomes smax⁡(1−u,σmin⁡[1+(s−1)u])2\frac{s}{\max(1-u, \sigma _ {\min}[1+(s-1)u])^2}, altering the terminal high-SNR supervision window near t→1t \to 1.
    • ODE Solver Discretization: Uniform Heun steps in t∈[0,1]t \in [0, 1] map to non-uniform steps in uu that automatically concentrate solver evaluations near t→1t \to 1, where fine kinematic details are resolved.

Source-Scale Sensitivity Analysis (s∈{1,3,5}s \in \lbrace 1, 3, 5 \rbrace)

We evaluated 500-epoch causal 263D models across s∈{1,3,5}s \in \lbrace 1, 3, 5 \rbrace under identical training configurations:

Source Scale (ss)Midpoint Conditioning log⁡10κ0.5(s)\log _ {10}\kappa _ {0.5}(s) ↓FID ↓ (67D Protocol)R-Precision Top-1 ↑R-Precision Top-3 ↑MM-Dist ↓
s=1s = 1 (Paper Table 6 & 15)3.4100.111 ± 0.0050.580 ± 0.0040.861 ± 0.0022.851 ± 0.007
s=3s = 3 (New)2.4560.0440.5670.8562.876
s=5s = 5 (Paper Table 1, 6 & 15)2.0170.046 ± 0.0040.571 ± 0.0050.853 ± 0.0032.890 ± 0.010

Increasing ss from 11 to 33 or 55 reduces FID by nearly 60% (0.111→0.044/0.0460.111 \to 0.044 / 0.046), with a broad performance plateau between s=3s=3 and s=5s=5 demonstrating that the improvement does not rely on brittle hyperparameter tuning. We will incorporate both the A(t)/u(t)A(t)/u(t) derivation and this sensitivity table into Section 3.3.

Reviewer_NsLvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper provides a framework, MotionSpaceFlow, which aims to perform text-driven human action generation in full-resolution motion space without using a learned motion encoder or decoder. It uses causal attention on the 263D representation in HumanML3D and bidirectional attention on absolute XYZ coordinates, combined with endpoint projection. This method is evaluated on two datasets using quantitative metrics and a multi-component ablation study, the results of which show its competitive performance in motion distribution quality and text matching.

Critical strengths
  1. Tables 3 and 6 show that the 263D and XYZ representations have different preferences for temporal attention, and that the noise scale and prediction target can substantially affect the results. These findings have practical value, as they highlight the importance of choosing generation mechanisms according to the motion representation. The authors also reasonably acknowledge that causal attention does not have a universal advantage.

  2. Table 4 shows that this method improves motion FID and text matching compared with the baseline. In Table 1, the XYZ variant achieves better FID, while the 263D variant shows stronger text-matching performance. I believe the improvement in controlled generation quality remains practically valuable.

Critical weaknesses
  1. The effect of the temporal attention mechanism should be validated. It is unclear how much of the gain comes from the match between attention and motion representations. A fair comparison should change only the attention between motion frames and be reported in the main results table.

  2. A concern about the generation efficiency trade-off. The paper already shows competitive generation speed in the 196-frame setting, but this is not sufficient to demonstrate efficiency for much longer sequences. I believe reporting generation times and peak VRAM usage for various sequence lengths will provide much stronger evidence.

Questions for authors
  1. Is it possible to provide a controlled experiment to distinguish the roles of temporal context and cross-modal interaction? I believe this will provide more evidence for temporal modelling by conducting the comparison under the current evaluation protocol.
  2. Can you provide more experiments to demonstrate the method’s efficiency for much longer sequences of different lengths?
2 comments

We thank Reviewer NsLv for highlighting the practical significance of our representation-dependent design findings and our improvements in both text-to-motion and spatially controlled generation. Below, we present the controlled 8-model attention ablation and the long-sequence efficiency benchmarks you requested.

1. Controlled 2×2×22 \times 2 \times 2 Attention Ablation Under the Main Evaluation Protocol (Weakness 1 & Question 1)

"Is it possible to provide a controlled experiment to distinguish the roles of temporal context and cross-modal interaction? I believe this will provide more evidence for temporal modelling by conducting the comparison under the current evaluation protocol."

Yes. We completed the full 2×2×22 \times 2 \times 2 factorial comparison across Representation (Incremental 263D vs. Global XYZ) ×\times Motion-to-Motion Attention (Causal vs. Bidirectional) ×\times Text-Query Feedback (Blocked vs. Full), with all eight models trained for 500 epochs and evaluated under the primary 67D evaluation protocol:

RepresentationMotion AttentionText-Query FeedbackFID ↓R-Precision Top-1 ↑R-Precision Top-3 ↑MM-Dist ↓
Incremental 263D (Paper)Causal (Ours)Blocked0.046 ± 0.0040.571 ± 0.0050.853 ± 0.0032.890 ± 0.010
Incremental 263D (New)BidirectionalBlocked0.081 ± 0.0070.561 ± 0.0040.848 ± 0.0032.966 ± 0.020
Incremental 263D (New)CausalFull0.451 ± 0.0190.531 ± 0.0060.826 ± 0.0023.095 ± 0.018
Incremental 263D (Paper)BidirectionalFull0.067 ± 0.0020.558 ± 0.0040.846 ± 0.0022.973 ± 0.005
Global XYZ (Paper)CausalBlocked1.563 ± 0.0250.506 ± 0.0040.800 ± 0.0023.243 ± 0.008
Global XYZ (New)BidirectionalBlocked0.074 ± 0.0140.564 ± 0.0060.851 ± 0.0032.949 ± 0.015
Global XYZ (New)CausalFull1.565 ± 0.0400.517 ± 0.0050.806 ± 0.0053.227 ± 0.031
Global XYZ (Paper)Bidirectional (Ours)Full0.038 ± 0.0040.566 ± 0.0040.849 ± 0.0032.916 ± 0.008

Key Conclusions:

  1. Unconfounded Comparison of Motion Temporal Attention (Blocked Rows): When text queries are restricted to text tokens (Blocked), varying only the motion-to-motion attention mask confirms our core hypothesis across all metrics: Incremental 263D achieves nearly 2× lower FID with Causal attention (FID: Causal 0.0460.046 vs. Bidirectional 0.0810.081), whereas Global XYZ achieves 21× lower FID with Bidirectional attention (FID: Bidirectional 0.0740.074 vs. Causal 1.5631.563).
  2. Interaction Between Temporal Causality and Cross-Modal Feedback (Full Rows): Allowing text queries to attend to motion tokens (Full) inside a nominally Causal 263D model introduces a 2-layer future-to-past information leak (future motion frame j>ij > i updates text tokens in Layer ℓ\ell, which are read by past motion frame ii in Layer ℓ+1\ell+1). Eliminating this causal leak (Causal + Blocked) improves 263D FID by nearly 10× (0.451→0.0460.451 \to 0.046). Conversely, for Global XYZ, where global trajectory smoothing is beneficial, bidirectional cross-modal context (Bidirectional + Full) further improves FID to 0.0380.038.

As requested, we will integrate this complete 8-row controlled ablation directly into the main results section.

2. Generation Latency and Peak VRAM Across Extended Sequence Lengths (Weakness 2 & Question 2)

"Can you provide more experiments to demonstrate the method’s efficiency for much longer sequences of different lengths?"

We extended our A100-SXM4-80GB inference benchmark (Table 10) from 196 frames (9.8s) to 392, 784, and 1,024 frames (51.2s of continuous 20fps motion, 5.2×5.2\times longer) using batch size 1, CFG w=3.0w=3.0, and 50-step Heun sampling. We report the median of 5 synchronized runs both with cached text features (matching Table 10) and with live DistilBERT text encoding included on every generation call:

Sequence Length (Frames)Motion Duration (@20fps)Latency w/ Cached Text (s) ↓Latency w/ Live Text Encoder (s) ↓Peak Allocated VRAM (Cached / Live Text) ↓Peak Reserved VRAM (Cached / Live Text) ↓
196 (1×, Paper Table 10)9.8 s1.742 s1.857 s313 MB / 568 MB340 MB / 610 MB
392 (2×)19.6 s2.160 s2.199 s353 MB / 608 MB404 MB / 654 MB
784 (4×)39.2 s2.805 s2.847 s434 MB / 694 MB512 MB / 762 MB
1,024 (5.2×)51.2 s3.228 s3.284 s485 MB / 740 MB588 MB / 858 MB

Key Takeaways:

  • Sub-linear wall-clock scaling: Increasing sequence length by 5.2× (from 196 to 1,024 frames) increases generation time by only 1.85× (1.742s→3.228s1.742\text{s} \to 3.228\text{s}). Notably, MSFlow generates 1,024 frames faster (3.228s) than FloodDiffusion (3.537s), MotionStreamer (9.023s), and MARDM (9.579s) generate 196 frames (Table 10).
  • Compact memory footprint: Even at 1,024 frames with the live text encoder resident in memory, peak allocated VRAM is only 740 MB (<0.75 GB<0.75\text{ GB}), enabling deployment on commodity GPUs.

Summary for Reviewer NsLv: Both requested experiments (the controlled 8-model attention comparison under the main protocol and the long-horizon latency/VRAM scaling benchmarks up to 1,024 frames) have been completed with strong results. We hope these findings address your concerns in full and support raising your rating to acceptance.

Reviewer_qJT2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MotionSpaceFlow, a framework for text-conditioned human motion generation directly in motion space. It combines clean-endpoint prediction, source-noise scaling, and a multimodal transformer whose attention mask depends on the motion representation. The incremental HumanML3D representation uses causal attention, whereas absolute XYZ coordinates use bidirectional attention. The XYZ model additionally supports inference-time spatial control through endpoint projection. Experiments cover HumanML3D under multiple evaluation protocols, SnapMoGen, component ablations, and spatial-control tasks.

Critical strengths
  1. The framework combines clean-motion prediction with representation-aware noise scaling and temporal attention, removing dependence on a learned motion autoencoder while preserving full temporal resolution.

  2. Experiments cover multiple motion representations and evaluation protocols on HumanML3D, together with SnapMoGen. The accompanying ablations provide useful evidence about prediction targets, source-noise scales, and architectural choices.

  3. The XYZ variant supports frame- and joint-level constraints without control-conditioned training. It improves motion quality and text alignment over ProjFlow under the reported protocol while maintaining exact coordinate satisfaction.

Critical weaknesses

The trade-offs underlying the quality improvements are insufficiently characterized. Although Table 10 provides an inference-efficiency comparison, training costs and convergence behavior relative to latent-space methods remain unclear. The reported training time of approximately 450 minutes for MSFLOW alone does not establish this comparison. In addition, Table 7 shows substantially lower conditional multimodality for the 263D model than CMDM (0.890 versus 1.785). Together, these gaps limit the assessment of whether the generation-quality gains offer a favorable trade-off at comparable training budgets and conditional diversity.

The attention ablation confounds temporal context with cross-modal feedback. In Section 3.4, switching from causal to bidirectional attention changes both the temporal context available to motion queries and whether text queries can attend to motion. Consequently, Tables 3 and 6 do not isolate the effect of temporal attention. This weakens the evidence for the representation-dependent temporal modeling principle, a central contribution of the paper, while leaving the reported performance of the complete configurations intact.

Questions for authors
  1. Could the authors compare the training costs of MSFLOW with those of latent-space methods to clarify the computational trade-offs of removing the VAE?

  2. Could you evaluate bidirectional motion attention while keeping text queries restricted to text tokens? This would help distinguish the contribution of temporal motion context from motion-to-text feedback.

  3. What explains the reduced conditional multimodality in Table 7? Could a small classifier-free guidance sweep report FID, R-Precision, and multimodality together, and establish whether the quality advantage persists at comparable conditional diversity?

3 comments

We thank Reviewer qJT2 for recognizing the strengths of our autoencoder-free full-resolution formulation, our multi-protocol experimental evaluation, and our zero-shot spatial control capabilities. We have conducted all three experimental studies you requested.

1. Full 2×2×22 \times 2 \times 2 Attention Factorial: Separating Motion Attention from Text Feedback (Weakness 2 & Question 2)

"Could you evaluate bidirectional motion attention while keeping text queries restricted to text tokens? This would help distinguish the contribution of temporal motion context from motion-to-text feedback."

Thank you for catching this important detail. In Section 3.4 (Tables 3 and 6), switching from the Causal configuration to the Bidirectional configuration changed two factors at the same time:

  1. Motion-to-Motion Attention: Whether a motion frame can only attend to past frames (Causal) or to both past and future frames (Bidirectional).
  2. Text-Query Feedback: Whether text queries are restricted to text tokens (Blocked, used in our submitted Causal models) or also attend to motion frames (Full, used in our submitted Bidirectional models).

To separate these two effects completely, we trained the remaining factorial configurations for the full 500-epoch (191,500-update) budget under identical settings, completing all 8 combinations under the 67D evaluation protocol (CFG w=3.0w=3.0, 50-step Heun solver):

All eight configurations were trained for 500 epochs (191,500 updates). Results for settings covered in the submission are taken from Table 6, and results for the newly tested settings come from the corresponding 500-epoch evaluations.

RepresentationMotion attentionText-query feedbackEpochsFID ↓R-Precision Top-3 ↑
Incremental 263D (Paper)CausalBlocked5000.046 ± 0.0040.853 ± 0.003
Incremental 263D (New)BidirectionalBlocked5000.081 ± 0.0070.848 ± 0.003
Incremental 263D (New)CausalFull5000.451 ± 0.0190.826 ± 0.002
Incremental 263D (Paper)BidirectionalFull5000.067 ± 0.0020.846 ± 0.002
Global XYZ (Paper)CausalBlocked5001.563 ± 0.0250.800 ± 0.002
Global XYZ (New)BidirectionalBlocked5000.074 ± 0.0140.851 ± 0.003
Global XYZ (New)CausalFull5001.565 ± 0.0400.806 ± 0.005
Global XYZ (Paper)BidirectionalFull5000.038 ± 0.0040.849 ± 0.003

What these 8 models demonstrate:

  1. When Text-Query Feedback is kept fixed (Blocked), our main claim holds:
    • For Incremental 263D, attending only to the past (Causal) cuts FID error nearly in half compared to attending also to the future (Bidirectional): 0.0460.046 vs. 0.0810.081, while improving Top-3 R-Precision (0.8530.853 vs. 0.8480.848).
    • For Global XYZ, attending both backward and forward (Bidirectional) is 21× better in FID than attending only to the past (Causal): 0.0740.074 vs. 1.5631.563, and improves Top-3 R-Precision from 0.8000.800 to 0.8510.851.
  2. Why blocking text feedback is required for Causal 263D, while full feedback helps Bidirectional XYZ:
    • Consider a multi-layer network where motion attention is Causal, but text feedback is Full. In Layer 1, text tokens read future motion frame 100. In Layer 2, past motion frame 10 reads those updated text tokens. This means that future motion leaks backward into past motion, which ruins the causal model, making 263D FID nearly 10× worse (0.046→0.4510.046 \to 0.451). Blocking text feedback is therefore essential to keep a causal model truly causal.
    • Conversely, for Global XYZ, where two-sided trajectory smoothing across all frames helps maintain global coordinate consistency, allowing text tokens to aggregate motion context (Bidirectional + Full) further improves FID from 0.0740.074 down to 0.0380.038.

The two settings we chose in our paper (Causal + Blocked for 263D at 0.0460.046, and Bidirectional + Full for XYZ at 0.0380.038) are the #1 best models out of all 8 combinations. We will include this complete 8-row factorial table in the main manuscript.

2. End-to-End Training Cost Comparison Against Latent-Space Methods (Weakness 1 & Question 1)

"Could the authors compare the training costs of MSFLOW with those of latent-space methods to clarify the computational trade-offs of removing the VAE?"

In Appendix C, we reported an end-to-end wall-clock training time of approximately 6.74 hours for 500 epochs of MSFlow on a single NVIDIA A100-SXM4-80GB GPU. To establish a fair comparison against five representative latent-space baselines (CMDM, MARDM-SiT-XL, SALAD, MotionStreamer, and FloodDiffusion), we profiled the native optimizer updates of all six methods on the same A100 GPU and extrapolated to each method's official training budget across both Stage-1 Codec/VAE pretraining and Stage-2 Generator training:

MethodGenerative ParadigmStage 1: Codec / VAE (Updates / Hours)Stage 2: Generator (Updates / Hours)Total Core Training (GPU-Hours)
MotionSpaceFlow (Ours)Direct Motion (1 Stage)None / 0.00 h191,500 / 6.74 h6.74 h
CMDMContinuous Latent (2 Stages)38,250 / 1.37 h*191,500 / 3.72 h5.09 h*
MARDM-SiT-XLContinuous Latent (2 Stages)360,850 / 2.13 h191,500 / 23.99 h26.12 h (3.9× slower)
SALADContinuous Latent (2 Stages)360,850 / 27.17 h191,500 / 5.50 h32.67 h (4.8× slower)
FloodDiffusion (300k VAE budget)Continuous Latent (2 Stages)300,000 / 43.46 h300,000 / 20.69 h64.15 h (9.5× slower)
MotionStreamerContinuous Latent (2 Stages)2,000,999 / 20.75 h100,001 / 69.16 h89.91 h (13.3× slower)
FloodDiffusion (default 3M VAE budget)Continuous Latent (2 Stages)3,000,000 / 434.61 h300,000 / 20.69 h455.29 h (67.5× slower)

*Note: CMDM's 1.37h Stage-1 cost excludes the prerequisite pretraining of its 6-part Part-TMR motion-language alignment network.

Key Takeaways: While a single full-resolution generator update in MSFlow (106.4 ms106.4\text{ ms}) is naturally more compute-intensive than a 4×4\times-temporally compressed latent update in CMDM (64.0 ms64.0\text{ ms}) or SALAD (78.2 ms78.2\text{ ms}), eliminating Stage-1 VAE pretraining yields large savings in total training cost. Latent methods incur substantial overhead in Stage-1 autoencoder training (e.g., 27.17h for SALAD; 43.46h–434.61h for FloodDiffusion, plus 3.25h for latent pre-tokenization). MSFlow completes core training in 6.74 GPU-hours, making it 3.9× to 67.5× faster than MARDM, SALAD, FloodDiffusion, and MotionStreamer while removing two-stage training complexity and reconstruction bottlenecks.

3. Motion Variety (Multimodality) & Guidance Strength Sweep (Weakness 1 & Question 3)

"What explains the reduced conditional multimodality in Table 7? Could a small classifier-free guidance sweep report FID, R-Precision, and multimodality together, and establish whether the quality advantage persists at comparable conditional diversity?"

Multimodality measures variation across motions generated from the same text prompt, without directly assessing whether those motions remain aligned with the prompt. It should therefore be interpreted alongside text-matching accuracy and FID:

  • Naturally, generating random motions regardless of the text prompt achieves the highest Multimodality score. That is why in Tables 1 and 7, the models with the lowest text-matching accuracy and worst FID (such as MDM, with 0.5180.518 / 0.4890.489 FID and 0.4400.440 / 0.4550.455 Top-1 R-Precision) report the highest Multimodality scores, because their generated motions drift randomly instead of adhering to the prompt.
  • Conversely, when a model enforces strict prompt adherence (as in our Causal 263D model at CFG=3.0\text{CFG} = 3.0, which achieves the #1 highest text-matching accuracy across all 16 methods in Table 7: Top-1 0.5910.591, Top-3 0.8630.863, MM-Dist 2.5652.565), the generated motions consistently execute the exact action described by the prompt, naturally reducing variation across repeats.
  • Furthermore, our MSFlow (XYZ) variant already achieves a Multimodality of 1.518 on 67D (Table 1) and 1.287 on Standard 263D (Table 7), surpassing MoMask (1.2631.263 / 1.2411.241) and MMM (1.2261.226) while delivering 0.0380.038 / 0.0500.050 FID.

As requested, we evaluated our Causal 263D model across guidance strengths from CFG=0.0\text{CFG} = 0.0 (ignoring the text prompt completely) to CFG=3.0\text{CFG} = 3.0 under the Standard 263D evaluator, jointly measuring FID, R-Precision Top-3, MM-Dist, and Multimodality:

Guidance Weight (CFG ww)FID ↓R-Precision Top-3 ↑MM-Dist ↓Multimodality ↑
0.00 (Unconditional / Random)5.6430.1105.6055.900
0.502.326 ± 0.0750.684 ± 0.0033.597 ± 0.0152.537 ± 0.227
0.651.285 ± 0.0390.763 ± 0.0073.230 ± 0.0281.736 ± 0.104
1.000.489 ± 0.0170.823 ± 0.0052.827 ± 0.0211.213 ± 0.098
1.500.217 ± 0.0070.847 ± 0.0062.667 ± 0.0110.985 ± 0.101
2.000.122 ± 0.0030.859 ± 0.0052.606 ± 0.0150.897 ± 0.093
2.500.078 ± 0.0030.863 ± 0.0022.580 ± 0.0170.850 ± 0.088
3.00 (Paper, Table 7)0.057 ± 0.0030.863 ± 0.0032.565 ± 0.0070.890 ± 0.048

At CFG=0.00\text{CFG} = 0.00, as shown on the top row, where the model generates random motions regardless of the prompt, it achieves a multimodality score of 5.900 (and 2.537 at CFG=0.50\text{CFG} = 0.50), demonstrating that the learned distribution possesses high intrinsic diversity without mode collapse. Increasing CFG smoothly trades off unguided dispersion for tighter prompt alignment and better FID. Moreover, when applications require both higher conditional Multimodality and state-of-the-art fidelity at w=3.0w=3.0, MSFlow (XYZ) simultaneously delivers 0.0380.038 / 0.0500.050 FID and 1.5181.518 / 1.2871.287 Multimodality. We will include this complete table in the revised manuscript.

Summary for Reviewer qJT2: We have completed all three requested experiments: the 8-model attention ablation, the 6-method training time comparison, and the 7-point CFG sweep. These results directly address your concerns, and we respectfully ask you to reconsider your score in light of this additional evidence supporting the effectiveness of our MSFlow.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Official Comment to Area Chairs and All Reviewers

We thank the Area Chairs and Reviewers for their constructive assessment of our work. Across the reviews, there is broad consensus on our core empirical strengths: MotionSpaceFlow (MSFlow) achieves state-of-the-art distributional fidelity and text-motion alignment across multiple benchmarks (qJT2, NsLv, k446), supports exact, training-free joint- and frame-level spatial control (qJT2, NsLv, k446), and operates as a compact, single-stage 68.8M-parameter model with low inference latency (wSyy).

The reservations raised in the initial reviews center on two primary questions:

  1. Experimental Isolation of Mechanisms and Trade-Offs: Do our design choices remain validated when individual factors (temporal attention vs. cross-modal feedback, training budgets, long-horizon scaling, and guidance-diversity trade-offs) are strictly isolated? (qJT2, NsLv, k446)
  2. Methodological Novelty in Direct Motion Space: What distinguishes MSFlow from prior direct motion-space approaches (e.g., MDM, HY-Motion 1.0, ACMDM) and inference-time projection methods (ProjFlow)? (k446, wSyy)

To address every question with concrete evidence, we conducted five sets of controlled experiments during the rebuttal period, training 10 full 500-epoch models from scratch and executing over 60 full test-set evaluations:

  1. Why Prior Direct-Space Models Lagged Behind Latent Paradigms, and Why MSFlow Succeeds (Addresses k446, wSyy): Prior work operating in raw motion space (e.g., MDM, HY-Motion) does not, by itself, negate our novelty, just as MLD's use of latent diffusion does not diminish the contributions of later methods such as SALAD, MARDM, ACMDM, and CMDM. What matters is solving the bottlenecks that held raw-motion generation back. For example, on the 272D benchmark (Table 8), HY-Motion 1.0, a 1.0B-parameter model with neither publicly available training code nor training data, achieves an FID of 61.035, compared with 6.540 for our 68.8M-parameter MSFlow. MSFlow achieves an approximately 9.3× lower FID and also outperforms HY-Motion’s post-trained variant. To isolate the contribution of our training recipe from that of the architecture, we trained MDM's exact x0x_0-DDPM objective on our identical 68.8M RA-MMDiT network for 500 epochs. Using MDM's recipe on our network makes FID 6.3× worse on the 67D test (0.2910.291 vs. 0.0460.046) and 4.3× worse on the standard 263D test (0.2450.245 vs. 0.0570.057) for 263D motion, and 3.2×–3.3× worse for XYZ motion (0.1220.122 vs. 0.0380.038 on 67D; 0.1670.167 vs. 0.0500.050 on standard 263D).
  2. Full 2×2×22 \times 2 \times 2 Attention Factorial (8 Models ×\times 500 Epochs; Addresses qJT2, NsLv): Reviewers qJT2 and NsLv noted that our attention comparison changed two factors at once: (a) whether motion frames attend to future motion frames, and (b) whether text tokens attend to motion frames. We completed all 8 possible combinations. When text tokens are blocked from reading motion frames (Blocked), incremental 263D motion works best with causal attention (FID: Causal 0.0460.046 vs. Bidirectional 0.0810.081), whereas global XYZ coordinates work best with bidirectional attention (FID: Bidirectional 0.0740.074 vs. Causal 1.5631.563, a 21× improvement). Critically, if we let text tokens attend to motion frames during "Causal" 263D training, future motion frames indirectly provide information into past motion frames through the text. This leakage severely degrades performance, making FID nearly 10× worse (0.046→0.4510.046 \to 0.451). Conversely, bidirectional XYZ benefits from global cross-modal aggregation, reaching 0.0380.038 FID. Thus, the two settings chosen in our paper (0.0460.046 for 263D and 0.0380.038 for XYZ) are the exact #1 best settings out of all 8 combinations.
  3. Six-Method Audited Training Cost & Long-Horizon Inference Scaling (Addresses qJT2, NsLv): By eliminating Stage-1 VAE pretraining, MSFlow requires only 6.74 GPU-hours of core training on a single A100 GPU, making it 3.9× to 67.5× faster to train than MARDM (26.12h), SALAD (32.67h), FloodDiffusion (64.15h–455.29h), and MotionStreamer (89.91h). At inference, MSFlow generates 196 frames in 1.742s and scales smoothly to 1,024 frames (51.2s of motion) in 3.228s using only 740 MB of peak VRAM.
  1. Controlled Raw-Space & Spatial Control Comparisons Against ACMDM and ProjFlow (Addresses k446, wSyy): The original ACMDM paper (Table 1) already trained a raw-space velocity-prediction (v\mathbf{v}) flow-matching model without an autoencoder (which ProjFlow also adopted as its base prior), achieving an unconditional 67D FID of 0.1210.121 (Top-1 0.5020.502, Top-3 0.7890.789). Training the ACMDM raw-flow XYZ backbone with our source-scaled clean-prediction formulation (s=5s=5, uniform tt, x^1→v\hat{\mathbf{x}}_1 \to \mathbf{v}) improves its unconditional FID to 0.0860.086 (Top-1 0.5180.518, Top-3 0.8080.808), while MSFlow XYZ achieves 0.0380.038 FID (Top-1 0.5660.566, Top-3 0.8490.849), representing a 3.2× reduction in FID over ACMDM's original raw velocity-prediction model and a 2.3× reduction over the source-scaled clean-prediction ACMDM raw-flow model. Under the identical shared ProjFlow projection implementation on dense pelvis control, MSFlow XYZ achieves 0.10610.1061 FID compared with 0.17220.1722 for the source-scaled clean-prediction ACMDM raw-flow model (38.4% lower FID, alongside lower foot skating: 0.06150.0615 vs. 0.08270.0827), while endpoint/hard projection alone gives 1.98781.9878 vs. 2.57132.5713. Across the full 5-density OmniControl benchmark (Table 4), MSFlow improves over ProjFlow (which uses the original velocity-prediction ACMDM prior) from 0.107→0.0680.107 \to 0.068 FID on Pelvis and 0.097→0.0610.097 \to 0.061 FID across All Joints.
  2. Exact Equivalence of Source Scaling and Alternative Noise/SNR Scheduling & Guidance–Diversity Analysis (Addresses qJT2, k446): We prove via an exact algebraic identity that scaling the source distribution by ss is mathematically equivalent to applying an alternative noise/SNR schedule u(t)=tt+s(1−t)u(t) = \frac{t}{t + s(1-t)} to a unit-source path (up to state amplitude A(t)=t+s(1−t)A(t) = t + s(1-t)), and that intermediate covariance conditioning κt(s)\kappa _ t(s) and SNRs(t)\text{SNR} _ s(t) are a strict one-to-one function of each other rather than competing effects. We complement this derivation with a sensitivity study across s∈{1,3,5}s \in \lbrace 1, 3, 5 \rbrace (∼60\sim 60% FID reduction over s=1s=1) and an 8-point CFG sweep (CFG∈[0.0,3.0]\text{CFG} \in [0.0, 3.0]) characterizing the trade-off between prompt adherence and conditional variation.

We have addressed all the questions raised in the reviews with clear, fair experiments. We respectfully ask the reviewers to reconsider their scores in light of these results.

Dear reviewers,

Thank you for your reviews and constructive feedback.

The authors have submitted their rebuttal. Please take a moment to review it and consider whether your concerns have been adequately addressed. It would be helpful if you could briefly indicate:

  • Whether the rebuttal addresses your main concerns;

  • Whether any significant issues remain unresolved;

  • Whether your overall recommendation has changed.

If appropriate, please also update your review and recommendation in the system.

Thank you for your time and valuable contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.