acceptodds
Under review as a conference paper at ICLR 2027

MoGeFi: Learning Geometric Feedback for Few-Step Human Motion Generation

14 views

Abstract

Interactive human motion generation calls for natural, text-aligned movement, accurate geometric control, and low latency. Precise control often relies on costly test-time refinement, while fast generators can leave substantial joint errors. We present MoGeFi, a framework that combines one- and few-step conditional motion generation with learned geometric feedback. The backbone learns finite-interval latent updates conditioned on text and optional joint-position constraints, without relying on a pretrained diffusion teacher. For controlled synthesis, proposal-based geometric feedback corrects the backbone's latent velocity using joint geometry estimated from the current state and refreshed at each sampling step. These corrections are learned through geometric and distributional supervision of the model's own short rollouts, enabling forward-only control without per-sample optimization. Extensive experiments on HumanML3D demonstrate that MoGeFi combines high motion quality, accurate geometric control, and interactive latency across multiple motion generation tasks. In sparse joint control, it achieves comparable FID and roughly halves trajectory and location error rates compared with MaskControl-Fast, while generating each sequence in approximately 61 ms. Ablations show that explicit geometric feedback and per-step proposal refresh improve constraint satisfaction. We also execute generated motions on a physical humanoid using an existing motion tracker. Our project page is available at https://anonymous.4open.science/w/MoGeFi/.

Primary area:
generative models
Keywords:
Human motion generation, Controllable generation, Few-step generation, Geometric feedback, Flow matching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fAeBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MoGeFi, a few-step model for text-to-motion generation and joint-position control on HumanML3D. The backbone is an average-velocity DiT trained in the frozen MotionLCM-V2 VAE latent space, with joint observations provided as conditioning tokens. The main contribution is proposal-based geometric feedback: at each sampling step, a frozen observer predicts a clean-motion proposal, joint residuals to the target are computed in world and heading-aligned frames, and a controller uses these residuals to predict a correction to the latent velocity. The proposal is refreshed at every step. Training combines a flow loss, interpolant and rollout geometric losses, and a Fréchet loss in the feature space of the T2M evaluator. The paper reports FID 0.018 for one-step text-to-motion, 8.41 cm mean error for sparse control at about 0.061 s, and 2.54 cm for initial-motion control. The ablations study the effects of geometric feedback, proposal refresh, distributional supervision, and observer calibration.

Critical strengths

S1. The paper provides strong evidence that explicit learned feedback improves fast forward-only control.

Under the same task and supervision, the output-supervised backbone reaches 16.28 cm mean error, while adding geometric feedback reduces it to 8.41 cm. Reusing the initial proposal instead of refreshing it increases the error to 25.67 cm. These controlled ablations support the main claim that state-dependent geometric feedback provides a real gain beyond output supervision alone. The resulting 8.41 cm error at about 0.06 s also shows that MoGeFi improves control accuracy while retaining real-time inference.

Critical weaknesses

W1. The current comparisons do not fully isolate the advantage of learned feedback over optimization under a similar inference budget.

  • A natural baseline is missing: the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of the learned Δu\Delta u. Since MoGeFi already computes a clean proposal and its joint residuals during inference, this would directly test whether learned feedback is more effective than a small amount of test-time optimization.

  • The large gain over MotionLCM-V2 ControlNet should also be interpreted cautiously, since the V2 authors already report that ControlNet alone performs poorly on sparse joint constraints (V2 Tab. 4). The cleaner evidence for the effect of feedback is the internal comparison from 16.28 cm to 8.41 cm (Tab. 4(a)).

  • MaskControl also supports different optimization budgets (MaskControl Sec. A.6), but the paper does not compare against a setting tuned to a latency closer to MoGeFi. In addition, parameter counts and memory usage for the backbone, observer, and feedback modules are not reported, which makes the overall cost of the feedback design harder to judge beyond latency alone.

W2. The main motion-quality metrics are closely related to the feature space used by the training objective.

  • The Fréchet loss LFDL_{\mathrm{FD}} matches feature statistics in the same T2M evaluator space used for FID, R-Precision, and MM-Dist. This is not data leakage, but it gives MoGeFi direct supervision in the representation used by its main quality metrics, while the reported baseline numbers are not controlled for the same objective.

  • The paper's own ablation shows that adding LFDL_{\mathrm{FD}} improves FID from 0.178 to 0.083, while mean joint error changes only from 11.01 cm to 10.28 cm (Tab. 4(b)). This suggests that part of the FID improvement may come from directly matching statistics in the same feature space used for evaluation.

  • For text-to-motion, the training schedule includes a distributional adaptation stage with λFD=1\lambda_{\mathrm{FD}}=1 and λflow=0.01\lambda_{\mathrm{flow}}=0.01 (Tab. S4), while the paper does not report text-to-motion results before this adaptation. The final model achieves the best FID in Tab. 1, but its R@1 and MM-Dist are worse than MotionLCM-V2 for all reported sampling budgets.

Questions for authors
  • Can you report (a) the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of Δu\Delta u, and (b) a MaskControl setting with fewer optimization iterations at a latency closer to MoGeFi? If learned feedback remains more accurate under a similar inference budget, this concern would be substantially reduced.

  • Can you report (a) text-to-motion results before FD adaptation, (b) the quality of the default model trained with T2TM features under an independent evaluator such as TMR, and, if possible, (c) MotionLCM-V2 fine-tuned with the same LFDL_{\mathrm{FD}}? If the quality advantage remains under these settings, W2 would be substantially reduced.

  • Please report parameter counts and memory usage for the backbone, observer, and feedback modules, together with the training budget of the initial generator stage.

  • The proposal descriptors are held fixed in the JVP but recomputed during sampling (Eq. S5). Is the flow objective an exact identity for the full feedback field, or a local approximation? Relatedly, why does control become worse from two to four steps (Tab. S8: 8.41 to 9.76 cm) even though four-step rollouts are explicitly used during training (Tab. S3)? Does repeated feedback accumulate correction error, or is the controller sensitive to shorter sampling intervals?

11 comments

W1 / Q1: Learned Feedback and Test-Time Optimization

W1. The current comparisons do not fully isolate the advantage of learned feedback over optimization under a similar inference budget.

Question Q1

Can you report (a) the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of Δu\Delta u, and (b) a MaskControl setting with fewer optimization iterations at a latency closer to MoGeFi? If learned feedback remains more accurate under a similar inference budget, this concern would be substantially reduced.

W1(a): Replacing Learned Feedback with Gradient Corrections

A natural baseline is missing: the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of the learned Δu\Delta u. Since MoGeFi already computes a clean proposal and its joint residuals during inference, this would directly test whether learned feedback is more effective than a small amount of test-time optimization.

Thank you for suggesting this meaningful comparison. We retain the same backbone and observer and replace learned feedback with gradient corrections based on the proposal joint error. This directly compares how the two approaches use the current geometric information for control. For two-step generation, we apply 1, 2, or 5 gradient corrections at each generation stage.

Response Table 1. Control accuracy and inference time when gradient corrections replace feedback

MethodOptimizations per stageTotal optimizationsAITS↓Mean joint error↓
Feedback disabled, no gradient correction0028.58 ms35.30 cm
Gradient corrections replacing feedback1285.93 ms24.66 cm
Gradient corrections replacing feedback24144.33 ms20.04 cm
Gradient corrections replacing feedback510256.59 ms15.16 cm
MoGeFi: learned feedback0061.2 ms8.41 cm

A small number of gradient corrections improves control, but MoGeFi achieves substantially lower joint error in less inference time without test-time gradient optimization. Even with 5 gradient corrections at each generation stage, the mean error is 15.16 cm, compared with 8.41 cm for MoGeFi. This shows that learned geometric feedback can directly provide effective control corrections during few-step generation, making better use of a low-latency inference budget.

W1(b): MotionLCM-V2 and the Internal Feedback Comparison

The large gain over MotionLCM-V2 ControlNet should also be interpreted cautiously, since the V2 authors already report that ControlNet alone performs poorly on sparse joint constraints (V2 Tab. 4). The cleaner evidence for the effect of feedback is the internal comparison from 16.28 cm to 8.41 cm (Tab. 4(a)).

Thank you for pointing this out. The MotionLCM-V2 authors observe that ControlNet alone achieves lower accuracy on sparse joint control than on initial-motion control. Sparse joint control is more challenging, a difference also reflected in our results: MoGeFi obtains mean errors of 8.41 cm and 2.54 cm on sparse joint and initial-motion control, respectively.

MotionLCM-V2 is our direct baseline. MoGeFi retains its pretrained VAE, text encoder, and motion representation, so this comparison evaluates the complete method against an existing fast control framework. The contribution of feedback is established by the internal comparison: under the same task and supervision objectives, adding feedback reduces the mean error from 16.28 cm to 8.41 cm. The new gradient-replacement experiment with the same backbone and observer further shows that learned corrections use proposal geometry more effectively than a small number of gradient iterations.

We will clarify the respective roles of these comparisons in the revised paper to avoid misunderstanding.

W1(c): Optimization Budgets and Model Cost

MaskControl also supports different optimization budgets (MaskControl Sec. A.6), but the paper does not compare against a setting tuned to a latency closer to MoGeFi. In addition, parameter counts and memory usage for the backbone, observer, and feedback modules are not reported, which makes the overall cost of the feedback design harder to judge beyond latency alone.

Question Q3

Please report parameter counts and memory usage for the backbone, observer, and feedback modules, together with the training budget of the initial generator stage.

Thank you for this suggestion. Such a comparison is important for understanding control accuracy under low-latency constraints. We therefore evaluate MaskControl at different optimization budgets and report both control metrics and AITS.

To further examine MoGeFi under a comparable inference budget using gradient optimization similar to MaskControl, we also apply terminal gradient optimization to the output of the complete model. After generation, we compute the geometric error between the controlled joints and their targets through the frozen motion decoder and iteratively update the generated continuous latent.

Response Table 2. Generation quality, control accuracy, and inference time across optimization budgets

Traj. and Loc. are failure rates (%) at the 50 cm threshold. The optimization counts refer to terminal gradient updates after generation. Bold AITS, FID, and mean-error values indicate the better of the two methods at the same optimization count.

Opt.MethodAITS ms↓FID↓Avg. cm↓Traj.↓Loc.↓
0MoGeFi (paper model)610.06338.412.250.59
1MaskControl2230.067749.5347.9729.64
1MoGeFi + Gradient840.06827.932.220.54
2MaskControl2420.068147.2646.1128.15
2MoGeFi + Gradient1000.06827.561.990.49
5MaskControl3120.068941.4340.8124.31
5MoGeFi + Gradient1540.06866.611.640.39
10MaskControl4440.069234.0534.2319.57
10MoGeFi + Gradient2580.06905.481.220.25
20MaskControl7810.067224.0025.0613.01
20MoGeFi + Gradient4750.06944.130.820.17
50MaskControl1,6100.078910.1810.504.44
50MoGeFi + Gradient1,1430.06822.650.350.043
100MaskControl-Fast2,6200.09323.222.720.84
100MoGeFi + Gradient2,2280.06621.960.190.012

Control is evaluated by sampling a single joint and observation density for each sequence; the resulting metrics may differ slightly from those reported in Table 2 of the paper. AITS is measured on an RTX 4090.

MoGeFi has a clear control-accuracy advantage at low latency. The default MoGeFi achieves 8.41 cm mean error in 61 ms. MaskControl takes 1.61 s for 50 optimizations and still has a mean error of 10.18 cm. Thus, MoGeFi achieves lower error in approximately 1/26 of the inference time.

MoGeFi with gradient optimization also retains a clear advantage at similar latency. At approximately 0.25 s, MoGeFi + Gradient achieves 5.48 cm mean error, compared with 47.26 cm for MaskControl. At approximately 0.45 s, the errors are 4.13 cm and 34.05 cm, respectively.

With more inference computation, MoGeFi with 50 terminal optimizations achieves 2.65 cm in 1.14 s, outperforming MaskControl-Fast's 3.22 cm / 2.62 s. With 100 optimizations, MoGeFi further reduces the error to 1.96 cm with an AITS of 2.23 s, while also achieving lower FID and 50 cm failure rates.

These results show that MoGeFi provides low-latency control through learned feedback during few-step generation, while terminal optimization can further improve control accuracy when more inference computation is available.

W1(c): Optimization Budgets and Model Cost (Continue)

Model Size and Training Budget

Thank you for the suggestion. We have described the architecture of each module in the supplementary material: the generative backbone is a 12-layer DiT with a hidden dimension of 768 and 12 attention heads. To present the scale of each component more directly, we additionally report the inference parameter counts below. The joint-observation encoder encodes observations, while the geometric context encoder and feedback controller construct and apply feedback from the observer's motion estimate.

Response Table 3. Inference parameter counts

ModuleParameters
DiT backbone132.98 M
Joint-observation encoder12.32 M
Geometric context encoder1.73 M
Feedback controller0.88 M
Sentence-T5 text encoder335.73 M
Motion VAE decoder10.21 M
Total inference parameters493.85 M

The backbone has 132.98 M parameters. The observer and candidate share the backbone and joint-observation encoder parameters. Dynamic feedback adds only 2.61 M parameters for the geometric context encoder and feedback controller, approximately 1.97% of the backbone parameter count.

On an RTX 3090, inference with FP32, batch size 1, 196 frames, and two generation steps has a peak allocated GPU memory of 1.86 GiB.

The initial generator training budgets for the individual tasks are:

Response Table 4. Initial generator training budgets

TaskTraining stepsLearning rate
Text-to-motion36k10−410^{-4}
Temporal six-joint control32k10−410^{-4}
Fixed-pelvis control32k10−410^{-4}
Random-joint control32k10−410^{-4}

The training steps and learning rates for subsequent distributional adaptation, feedback learning, and feedback calibration are already reported in Table S3 of the paper. We will add the parameter counts, memory usage, and initial training budgets above to the revised paper.


Reviewer_jdfSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper generates 3D human motion from text in one or two steps. When the user fixes joint positions at chosen frames, a small learned module corrects each step toward these targets without per-sample optimisation.

  • Input: A sentence, a motion length and optional joint targets, given as 3D positions with an observation mask over frames and joints.
  • Output: A motion in the HumanML3D representation, decoded from the MotionLCM-V2 latent space.
  • Method: A Transformer backbone learns average latent velocities over time intervals with the mean-flow objective, conditioned on text and joint-observation tokens. For control, a frozen copy of the generator predicts the finished motion from the current state at each step. The model compares this proposal with the targets. A learned controller turns the comparison, the current state and the intended update into a velocity correction. Training runs the actual few-step sampler and adds a joint-position loss and a Fréchet distance loss on features of the HumanML3D evaluator.
  • Delta: The backbone objective comes from mean flows and the distribution loss from the representation Fréchet loss. The new part is the refreshed, learned geometric feedback, trained through short rollouts.
  • Experimental validation: All experiments use HumanML3D with its standard evaluator. For text-to-motion, the one-step model reaches FID 0.018 at 23 ms. For sparse joint control, the two-step model reaches FID 0.081, 2.38% trajectory errors and 8.41 cm mean joint error in 61 ms. MaskControl-Fast needs 2.7 s for FID 0.079, 4.81% and 5.63 cm. Ablations isolate the feedback, proposal refresh and Fréchet loss, and a Unitree G1 robot replays generated motions with an existing tracker.
Critical strengths
  • The learned, refreshed feedback gives large and well-isolated gains. In Table 4, output supervision alone reduces the mean joint error from 34.89 to 16.28 cm. The feedback module reduces it further to 8.41 cm. Reusing the first proposal instead of refreshing it raises the error to 25.67 cm.
  • The method is fast, and the latency comparison is fair. The controlled sampler runs in 61 ms, against 2.7 s for MaskControl-Fast. All methods are timed end to end on one RTX 4090 at batch size one, with text encoding included (Section S2.5). Initial-motion control reaches 2.54 cm in 71 ms (Table 3).
  • The appendix analyses the trade-offs thoroughly. It varies sampling steps, observer computation, guidance strength, training schedule and the feature space of the Fréchet loss (Tables S6 to S12).
Critical weaknesses
  • Fast controllable generation without optimisation already exists. DartControl [2] (ICLR 2025) adds spatial control to real-time text-driven generation, and TLControl [3] (ECCV 2024) refines a fast prediction with short optimisation. ARDY [1] (SIGGRAPH 2026) uses the MaskControl protocol on HumanML3D and reaches 4.15 cm mean joint error in 0.15 s without optimisation. MoGeFi reaches 8.41 cm, yet the paper neither cites nor compares with any of these works. All three appeared at peer-reviewed venues before the contemporaneous-work cutoff, so a comparison is expected.
  • Each task needs its own model. The paper trains four backbones from random initialisation: text-to-motion, initial-motion control, sparse control over six joints, and pelvis-only control (Sections 3.1, S2.1 and S2.3). Each has its own weights and guidance setting, so one deployed model cannot switch between these tasks. ARDY [1], in contrast, handles text, keyframes, root paths and sparse joints with one model through a general constraint mask.
  • All results use HumanML3D, although a larger benchmark with its own evaluator exists. BONES-SEED [6], released in March 2026, provides about 288 hours of motion capture with up to six text descriptions per motion. HumanML3D has about 30 hours. The Kimodo benchmark [5] on BONES-SEED tests text-to-motion, full-body keyframes, end-effector targets and root paths, with an independently trained text-motion retrieval model. Moving to its skeleton takes work, but the dataset was public six months before the deadline.
  • The headline quality results use a metric that the model is trained to minimise. The Fréchet loss uses the motion encoder of the HumanML3D evaluator, which also computes FID, R-Precision and matching distance (Eq. 16, Sections S1.5 and S2.3). The text-to-motion model has a dedicated stage that minimises this distance, with a flow-loss weight of only 0.01 (Table S4). Its FID of 0.018 is far below all baselines, but its R@1 (0.515) and matching distance (2.919) are worse than MotionLCM-V2. Training the loss on TMR features also lowers FID for control (Table S6), but no result uses an independent feature space for evaluation.
  • The abstract overstates the control accuracy. The abstract and the caption of Table 2 report halved trajectory and location error rates against MaskControl-Fast. In the same table, the mean joint error is 49% higher (8.41 against 5.63 cm), and foot skating is 62% higher (0.0901 against 0.0555). OmniControl (4.04 cm) and MaskControl-Accurate (0.72 cm) are also more accurate. More sampling steps or stronger guidance increase the error further (Table S8, Section S4.2).

Additional Comments

  • The OpenReview abstract shows raw LaTeX. The latency reads "61,ms", and the project link appears as a literal \url{} command. The PDF abstract is correct, so only the submission metadata needs a fix.
  • Streaming generation is not discussed. ActionPlan [4] (ECCV 2026) streams text-driven motion at about 40 ms per four-frame token and supports in-betweening. The interactive-latency framing should discuss this line of work.
  • Eq. 11 uses the query features as values. The equation sets V=ZkqV = \mathbf{Z}^q_k, but the text says that the geometric context supplies keys and values. Please state which one the model uses.
  • Two protocols report the same mean error. Table 2 averages six per-joint runs and Table 4 samples random joints, yet both give 8.41 cm. Please confirm both values.
  • The output-supervised backbone has an unexplained FID. Section S2.2 says that it uses the Fréchet loss, but its FID is 0.148 against 0.063 for the full model (Table 4).
  • MotionFlow is missing from the control tables. It appears in Table 1 and has a control encoder, so it is a natural fast baseline for Tables 2 and 3.
  • The robot experiment shows only that motions are trackable. The robot uses the external SONIC tracker, so it does not test the control accuracy of the method.
  • Training cost and code release are not stated. Please give the training hardware and time, and state whether code and models will be released.

References (not cited in paper)

[1] Kaifeng Zhao, Mathis Petrovich, Haotian Zhang, Tingwu Wang, Siyu Tang, Davis Rempe. "ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Human Motion Generation." ACM Transactions on Graphics (SIGGRAPH) 2026. arXiv:2607.08741.

[2] Kaifeng Zhao, Gen Li, Siyu Tang. "DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Control." ICLR 2025. arXiv:2410.05260.

[3] Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu. "TLControl: Trajectory and Language Control for Human Motion Synthesis." ECCV 2024. arXiv:2311.17135.

[4] Eric Nazarenus, Chuqiao Li, Yannan He, Xianghui Xie, Jan Eric Lenssen, Gerard Pons-Moll. "ActionPlan: Future-Aware Streaming Motion Synthesis via Frame-Level Action Planning." ECCV 2026. arXiv:2603.13500.

[5] Davis Rempe, Mathis Petrovich, Ye Yuan, et al. "Kimodo: Scaling Controllable Human Motion Generation." arXiv 2026. arXiv:2603.15546.

[6] Bones Studio. "BONES-SEED." Motion capture dataset with text annotations, released at NVIDIA GTC, March 2026.

Questions for authors
  • Explain what MoGeFi adds beyond DartControl [2], TLControl [3] and ARDY [1].
  • Compare with [2] and [3] on the Table 2 protocol, where their code allows.
  • Report ARDY [1] under the exact Table 2 protocol. Its code is public, and its data preparation differs slightly.
  • Evaluate Tables 1 and 2 in an independent feature space, such as TMR features, for all methods.
  • Report the text-to-motion model without the distributional adaptation stage.
  • Report Table 2 for a control model trained without the Fréchet loss.
  • Compare with a latency-matched test-time baseline, such as one or two gradient steps on the proposal error.
13 comments

W1 / Q1–Q3

W1. Fast controllable generation without optimisation already exists. DartControl [2] (ICLR 2025) adds spatial control to real-time text-driven generation, and TLControl [3] (ECCV 2024) refines a fast prediction with short optimisation. ARDY [1] (SIGGRAPH 2026) uses the MaskControl protocol on HumanML3D and reaches 4.15 cm mean joint error in 0.15 s without optimisation. MoGeFi reaches 8.41 cm, yet the paper neither cites nor compares with any of these works. All three appeared at peer-reviewed venues before the contemporaneous-work cutoff, so a comparison is expected.

Q1. Explain what MoGeFi adds beyond DartControl [2], TLControl [3] and ARDY [1].

Q2. Compare with [2] and [3] on the Table 2 protocol, where their code allows.

Q3. Report ARDY [1] under the exact Table 2 protocol. Its code is public, and its data preparation differs slightly.

The observation that “fast controllable generation without optimisation already exists” does not contradict the contribution claimed in our paper. The introduction already acknowledges this on page 1, lines 048–049:

“One- and few-step latent models already support real-time generation from text and joint observations.”

The problem we investigate is then stated on page 2, lines 054–055:

“Achieving high-quality motion generation and accurate joint control at interactive latency therefore remains challenging.”

Our goal is to achieve high-quality, text-consistent motion generation and accurate geometric control simultaneously under a very low inference budget, rather than to claim the first fast controllable generator.

To address this problem, MoGeFi learns finite-interval latent updates from motion data, conditioned jointly on text and spatiotemporal joint observations. This yields one- and few-step generators without diffusion-teacher distillation. For controlled generation, we introduce proposal-based geometric feedback, which uses the geometric relationship between the current full-motion estimate and the target to learn a correction to the generation velocity. We train the feedback using geometric and distributional supervision on the model's own short rollouts. This approach supports both high-quality one-step text-to-motion generation and two-step geometric control without test-time optimisation. The design is described in lines 057–083 and its contributions are summarised in lines 091–099 of the introduction.

DartControl, TLControl and ARDY study related motion generation problems, but differ in their generation methods, control strategies and computation over a complete motion sequence.

Response Table A: Generation and control mechanisms and computational cost

MethodGeneration and control mechanismGeneration computation for 196 framesTest-time optimisation
DartControlAutoregressive latent diffusion; the spatial-control path considered here uses generation-noise optimisation25 segments with 10 DDIM steps each: 250 denoising steps without optimisationThe official quantitative configuration performs 300 noise updates; including the final output, this amounts to 75,250 denoising steps
TLControlMTT predicts a motion representation in a forward pass, followed by L-BFGS optimisation of a continuous latent to satisfy trajectory constraints1 forward predictionAn average of 49.8 loss and gradient evaluations in our evaluation
ARDYAutoregressive diffusion with explicit root and latent body representations; constraints are incorporated into root/body denoisingApproximately five 40-frame windows with 10 steps each: 50 diffusion steps, each containing root and body prediction stagesNo-optimisation control configuration
MoGeFiFinite-interval generation of the complete motion, with a learned velocity correction based on the current proposal's geometry2 full-sequence generation updates + 2 observer predictions0 gradient-optimisation steps

DartControl refers to the noise-optimisation control path of its public HumanML3D model. Its total number of denoising steps is 25 × 10 × (300 + 1) = 75,250.

A shared research goal does not imply the same technical contribution. The original ablations provide direct evidence for our geometric feedback design: page 9, lines 458–461, reports that adding feedback under the same supervision reduces mean error from 16.28 cm to 8.41 cm; lines 471–475 report that, with fixed model weights, refreshing the proposal at each step reduces error from 25.67 cm to 8.41 cm compared with reusing the initial proposal. The additional comparisons below evaluate the complete framework in terms of generation quality, control accuracy and inference efficiency.

W1 / Q1–Q3 (continue)

1. DartControl

We evaluated DartControl using its public HumanML3D weights and official fixed-standing-pose initialisation.

Response Table B: HumanML3D text-to-motion quality and inference time

MethodAITS (s) ↓T2TM FID ↓R@1 ↑MM Dist ↓Diversity →
DartControl, no optimisation1.01051.310 ± 0.0260.4243.6009.860
MoGeFi, 1 step0.022620.0180.5152.9199.405
MoGeFi, 2 steps0.027910.0210.5302.8629.507

The real-data reference value for Diversity is 9.503.

MoGeFi improves generation quality, text matching and inference efficiency simultaneously. Compared with DartControl's 1.0105 s, one- and two-step generation take 0.02262 s and 0.02791 s, respectively, corresponding to 44.7× and 36.2× speedups. Their FIDs are 0.018 and 0.021, substantially lower than 1.310, while R@1 and MM Dist also improve.

This result directly demonstrates the practical advance over DartControl: MoGeFi improves motion distribution quality and text consistency while substantially reducing generation time. The advantage reflects the complete few-step conditional generation approach, rather than merely adding a geometric correction to generated motions.

2. TLControl

We evaluated TLControl using the protocol in Table 2 of our paper.

Response Table C: HumanML3D sparse single-joint control

MethodAITS (s) ↓FID ↓R@3 ↑Traj. @50 cm ↓Loc. @50 cm ↓Mean error ↓
TLControl1.1362.8160.6301.09%0.72%5.50 cm
MoGeFi, 2 steps0.06120.0810.8092.38%0.60%8.41 cm

TLControl achieves lower mean position error, while its generation quality and inference speed are substantially worse than MoGeFi's. MoGeFi's FID is 0.081 versus 2.816 for TLControl; R@3 is 0.809 versus 0.630, and inference time is 0.0612 s versus 1.136 s.

This is the central problem addressed by our paper: accurately following joint targets must be combined with motion quality, text consistency and interactive speed. MoGeFi achieves a strong balance of these capabilities with two generation steps and no test-time optimisation. Thus, evaluating its progress over existing control methods requires considering generation quality, control accuracy and computational cost together, rather than mean joint error alone.

3. ARDY

ARDY's public interface requires additional control inputs, and the HumanML3D-specific weights used in its paper are not available; consequently, we cannot conduct a complete evaluation under the same protocol as Table 2.

Specifically, when a global position target is provided for a non-root joint, ARDY's public interface also requires the root's planar position at the same frame. The _fill_global_position_constraints function in the official ardy_motionrep.py raises an error if this condition is missing:

“For constraining global positions, root 2D should also be constrained.”

In contrast, Table 2 of our paper provides only the controlled joint's position; the model generates the remaining joints and root motion. Adding a root target changes the control input—it is not merely a minor difference in data preparation.

For the pelvis-conditioned setting that can be run, we evaluated the public Rigplay weights on HumanML3D and remeasured inference time using the same 1,000 samples as in our paper:

Response Table D: HumanML3D pelvis evaluation of ARDY's public weights

ModelFID ↓Mean joint error ↓AITS (s) ↓
ARDY, Rigplay weights7.6193.17 cm0.3518

Timing was measured on one RTX 4090 with batch size 1.

This result uses the public Rigplay weights, whereas the 4.15 cm cited in the review comes from ARDY's HumanML3D-specific model. With the corresponding weights unavailable and different input conditions for non-root tasks, the reported 4.15 cm and our 8.41 cm cannot be treated as a comparison under the same protocol.

These comparisons answer the question of MoGeFi's advances over prior work from different angles: the few-step conditional generation backbone substantially improves text-to-motion quality and efficiency, while learned geometric feedback enables effective spatial control with two generation steps. We will add citations, method discussions and experimental comparisons for these works in the revised paper.

W2: Support for Different Control Conditions with a Single Model

W2. Each task needs its own model. The paper trains four backbones from random initialisation: text-to-motion, initial-motion control, sparse control over six joints, and pelvis-only control (Sections 3.1, S2.1 and S2.3). Each has its own weights and guidance setting, so one deployed model cannot switch between these tasks. ARDY [1], in contrast, handles text, keyframes, root paths and sparse joints with one model through a general constraint mask.

We train separate task-specific models to align with the experimental settings of our direct baselines, MotionLCM and MotionLCM-V2; this does not mean that different joint targets require separate models. The same random-joint model already supports independent control of the pelvis, both feet, the head and both wrists, across different observation densities. These results are reported in Table 2 and Supplementary Table S14.

1. The same weights support different joints, sparse position targets and dense trajectories

Table S14 on page 24, lines 1246–1247, states:

“Models are trained for control of all six joints and evaluated on one joint at a time.”

Pelvis-only control is performed directly by this random-joint model, achieving a mean error of 7.16 cm without changing weights or retraining. The same model also controls both feet, the head and both wrists; the 8.41 cm in Table 2 is the arithmetic mean of the results for the six joints.

MoGeFi uses a unified spatiotemporal joint mask in Eq. (1), lines 143–147, to specify target positions. Sparse control includes 1, 2 or 5 observed frames and densities of 25% or 100%: a small number of pelvis position targets corresponds to root-position keypoints, while a complete pelvis trajectory corresponds to root trajectory following; for other joints, these conditions correspond to sparse position targets or dense joint trajectories. These forms of control are selected by changing the target joint, observation frames and target coordinates, using the same weights and guidance setting. In the absence of joint observations, the algorithm performs text-conditioned generation with M=0\mathbf M=0.

2. The same model directly responds to edited spatial targets

Following reviewer RP1e's Q3 on target editing, we further evaluated the paper's original random-joint model with modified spatial targets. We fixed the text, initial noise and control frames, then shifted the left or right wrist target upward or outward by 5, 10 or 20 cm. All settings use the same model weights; only the input target changes, with no retraining.

Response Table E: Displacement response to different joints, directions and edit magnitudes using the same model

Target jointEdit directionTarget shift: 5 cmTarget shift: 10 cmTarget shift: 20 cm
Left wristUpward4.068.2817.40
Left wristOutward4.338.6417.21
Right wristUpward4.068.2317.23
Right wristOutward4.338.6417.24

Values are the generated motion's displacement along the target-edit direction relative to the unedited output, in cm.

The generated motions achieve 81.1%–87.0% of the requested displacement. This further demonstrates the same model's ability to adjust the direction and magnitude of different joint movements in response to the input target. Together with the paper's six-joint and density results, joint selection, observation times and target positions can all be changed at inference without training separate models for these control forms.

3. Comparing with ARDY requires considering the actual control inputs

MoGeFi also uses a unified spatiotemporal mask to express control conditions. For wrist, foot and head control, we provide only the target position of the selected joint, while the model generates the root and remaining body motion. When ARDY's public interface receives a global position constraint for a non-root joint, it additionally requires the root's planar position at the same frame. Thus, comparisons of control flexibility must also consider the control information actually provided to the model.

Reviewer_RP1eOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MoGeFi combines a few-step motion generator with learned geometric feedback for text-conditioned and spatially controlled human motion synthesis. A frozen observer estimates a clean-motion proposal, whose joint-position residuals guide corrections to the generator’s latent updates. Training combines flow, geometric, and distributional objectives. Experiments on HumanML3D evaluate text-to-motion generation, sparse joint control, and initial-motion control, alongside qualitative humanoid execution.

Critical strengths
  1. A substantial improvement in the speed–control trade-off. MoGeFi generates sparse-control sequences in approximately 61 ms, compared with 2.70 s for MaskControl-Fast, while achieving similar FID and fewer target deviations above 50 cm. The timing protocol includes text encoding, observer evaluations, and decoding, making the efficiency result practically meaningful.

  2. Useful ablations support the feedback mechanism. Under matched loss terms, adding feedback reduces mean joint error from 16.28 to 8.41 cm. The fixed-versus-refreshed proposal experiments further demonstrate that updated geometric context contributes to constraint satisfaction, rather than the gains arising solely from additional output supervision.

Critical weaknesses
  1. The strongest motion-quality claim needs independent validation. The distributional training objective directly optimizes feature statistics from the same T2TM evaluator used to report FID and other benchmark metrics. This does not invalidate the results, but it makes the exceptionally low text-to-motion FID of 0.018 insufficient evidence of broadly improved perceptual quality. The TMR supervision experiment in Table S6 is useful, but it changes the training representation rather than independently evaluating the final default model. Evaluation using an unused representation or a blinded perceptual study would strengthen the central quality claim.

  2. The evidence for precise control is mixed. In sparse joint control, MoGeFi reduces error rates measured at a relatively permissive 50 cm threshold, but its mean position error is higher than MaskControl-Fast’s: 8.41 versus 5.63 cm. Foot skating is also worse: 0.0901 versus 0.0555. Thus, the results demonstrate fast generation with fewer large deviations, rather than uniformly better geometric accuracy or motion realism. Evaluation at tighter tolerances and under simultaneous multi-joint constraints beyond the prefix setting is needed to establish the broader control claim.

Questions for authors
  1. Does the final default model retain its motion-quality advantage when assessed with a feature encoder not used in training, or through blinded human evaluation?

  2. How do the control methods compare at 5, 10, and 20 cm thresholds? What explains MoGeFi’s lower large-error rates but higher average error and foot skating relative to MaskControl-Fast?

  3. Does the feedback mechanism remain effective for simultaneous sparse constraints on multiple joints and targets that depart from the reference-motion distribution?

5 comments

W1 / Q1: Independent Validation of Motion Quality

Reviewer — Weakness W1

The strongest motion-quality claim needs independent validation. The distributional training objective directly optimizes feature statistics from the same T2TM evaluator used to report FID and other benchmark metrics. This does not invalidate the results, but it makes the exceptionally low text-to-motion FID of 0.018 insufficient evidence of broadly improved perceptual quality. The TMR supervision experiment in Table S6 is useful, but it changes the training representation rather than independently evaluating the final default model. Evaluation using an unused representation or a blinded perceptual study would strengthen the central quality claim.

Reviewer — Question Q1

Does the final default model retain its motion-quality advantage when assessed with a feature encoder not used in training, or through blinded human evaluation?

The motion-quality improvement from FD supervision is not confined to the feature space used for training. The paper already includes control experiments in which FD training uses TMR features and evaluation uses T2TM. We additionally conducted the same experiment for text-to-motion generation and summarize the results for the different training features below.

Table A. T2TM FID (↓) under different FD supervision features

TaskStepsT2TM supervisionTMR supervision
Text-to-motion10.01810.0356
Text-to-motion20.02050.0403
Text-to-motion40.02390.0411
Pelvis control20.10020.0971
Pelvis control40.09470.0908
Initial-motion control20.07530.0952
Initial-motion control40.06060.0633
Sparse joint control20.0830.077
Sparse joint control40.05450.0618

With TMR features used for FD training, MoGeFi still achieves strong quality under T2TM evaluation: its one-step T2M FID is 0.0356, better than MoMask's 0.045. For pelvis control, TMR supervision even yields lower T2TM FID at both two and four steps. These results show that the quality improvement from FD supervision is not confined to the feature space used for training.

To fully address the concern about the quality of the paper's default model, we further evaluated that model and the publicly available baselines using a TMR encoder that was not used for its FD training.

Table B. Independent TMR feature-space evaluation on T2M (↓)

MethodStepsRaw TMR FD↓Normalized TMR FD↓AITS (s)↓
MDM1000159.80150.096895.9452
MLD5064.70530.038840.1415
StableMoFusion1055.23360.032920.1560
MoMask1037.79130.022840.1068
MotionLCM4117.29280.071000.0306
MotionLCM-V2143.43980.026030.0227
MotionLCM-V2243.98750.026210.0256
MotionLCM-V2444.64040.026520.0330
MoGeFi138.28560.023000.0226
MoGeFi231.33260.018800.0279
MoGeFi428.83310.017280.0384

Bold TMR FD values are better than all evaluated baselines; the bold AITS is the fastest in its column. AITS values are taken directly from Table 1 of the paper.

The paper's default model achieves the lowest TMR FD among the evaluated methods with both two and four steps. Compared with MoMask, normalized TMR FD decreases by 17.7% and 24.3%, respectively, and raw TMR FD by 17.1% and 23.7%. MoGeFi's motion-quality advantage therefore also holds in an independent feature space not used for training.


W2 / Q2

Weakness W2

The evidence for precise control is mixed. In sparse joint control, MoGeFi reduces error rates measured at a relatively permissive 50 cm threshold, but its mean position error is higher than MaskControl-Fast’s: 8.41 versus 5.63 cm. Foot skating is also worse: 0.0901 versus 0.0555. Thus, the results demonstrate fast generation with fewer large deviations, rather than uniformly better geometric accuracy or motion realism. Evaluation at tighter tolerances and under simultaneous multi-joint constraints beyond the prefix setting is needed to establish the broader control claim.

Question Q2

How do the control methods compare at 5, 10, and 20 cm thresholds? What explains MoGeFi’s lower large-error rates but higher average error and foot skating relative to MaskControl-Fast?

MoGeFi aims to achieve both high-quality motion generation and accurate geometric control under a low-latency one- or few-step generation budget, which is particularly important for real-time interaction. With so few generation updates, the model cannot rely on prolonged iterations to gradually correct joint errors, making it challenging to achieve both motion quality and control accuracy. We therefore designed geometric feedback to use the discrepancy between the estimated motion and the targets to directly adjust generation updates.

In the paper's sparse joint control comparison, MoGeFi has lower failure rates at 50 cm, while MaskControl-Fast achieves lower mean error and higher accuracy at tighter thresholds. MaskControl-Fast's accuracy comes from 100 iterations of test-time gradient optimization: these iterations progressively reduce residual control errors, but also increase inference cost.

Control Accuracy at Different Inference Budgets

As reviewer fAeB pointed out, MaskControl supports different optimization budgets (MaskControl Sec. A.6), but the paper did not compare a setting with latency closer to MoGeFi. Such a comparison is important for understanding control accuracy under low-latency constraints. We therefore evaluated MaskControl at different optimization budgets and report both control metrics and AITS.

To further examine MoGeFi under a comparable inference budget using gradient optimization similar to MaskControl's, we also added terminal gradient optimization to the outputs of the full model. After generation, we use the frozen motion decoder to compute the geometric error between controlled joints and their targets, then iteratively update the generated continuous latent.

Table C. Motion quality, control accuracy, and inference time under different optimization budgets

Traj. and Loc. report failure rates (%) at thresholds of 5, 10, 20, and 50 cm, respectively. Optimization counts denote terminal gradient updates after generation. Bold AITS, FID, and mean-error values identify the better method at the same optimization count.

Opt.MethodAITS ms↓FID↓Avg. cm↓Traj. @5/10/20/50↓Loc. @5/10/20/50↓
0MoGeFi (paper model)610.06338.4178.13 / 54.10 / 23.66 / 2.2555.44 / 27.58 / 7.78 / 0.59
1MaskControl2230.067749.5394.25 / 86.58 / 74.73 / 47.9786.78 / 73.72 / 56.56 / 29.64
1MoGeFi + Gradient840.06827.9375.91 / 51.98 / 22.02 / 2.2252.54 / 25.40 / 6.91 / 0.54
2MaskControl2420.068147.2692.77 / 84.84 / 72.72 / 46.1184.76 / 71.42 / 54.37 / 28.15
2MoGeFi + Gradient1000.06827.5673.47 / 49.74 / 20.61 / 1.9950.00 / 23.66 / 6.22 / 0.49
5MaskControl3120.068941.4388.43 / 79.85 / 66.99 / 40.8178.86 / 65.11 / 48.33 / 24.31
5MoGeFi + Gradient1540.06866.6168.00 / 44.15 / 17.64 / 1.6443.62 / 19.45 / 4.98 / 0.39
10MaskControl4440.069234.0582.71 / 73.49 / 59.54 / 34.2371.06 / 56.72 / 40.51 / 19.57
10MoGeFi + Gradient2580.06905.4860.67 / 37.04 / 14.03 / 1.2235.63 / 14.51 / 3.37 / 0.25
20MaskControl7810.067224.0073.80 / 62.82 / 48.32 / 25.0658.95 / 44.38 / 29.66 / 13.01
20MoGeFi + Gradient4750.06944.1350.55 / 29.00 / 9.84 / 0.8225.83 / 9.23 / 1.92 / 0.17
50MaskControl1,6100.078910.1853.72 / 40.56 / 26.86 / 10.5034.27 / 22.30 / 12.79 / 4.44
50MoGeFi + Gradient1,1430.06822.6537.36 / 19.12 / 6.10 / 0.3514.26 / 4.11 / 0.79 / 0.043
100MaskControl-Fast2,6200.09323.2231.23 / 18.89 / 9.93 / 2.7213.21 / 6.84 / 3.20 / 0.84
100MoGeFi + Gradient2,2280.06621.9629.65 / 13.84 / 3.87 / 0.199.11 / 2.40 / 0.42 / 0.012

Control metrics use mixed evaluation with a randomly sampled single joint and observation density; they may differ slightly from the results in Table 2 of the paper. AITS is measured on RTX 4090 GPUs.

W2 / Q2 (Continue)

MoGeFi achieves a clear control-accuracy advantage at low latency. Without test-time optimization, MoGeFi achieves 8.41 cm mean error in approximately 0.061 s. After 50 optimization iterations, MaskControl has a mean error of 10.18 cm and an AITS of 1.61 s. Thus, MoGeFi achieves lower mean error using approximately 1/26 of the inference time.

With terminal optimization, MoGeFi also outperforms MaskControl-Fast at a comparable inference budget. After 50 updates, MoGeFi reaches 2.65 cm mean error, better than the 3.22 cm of MaskControl-Fast with 100 updates. With 100 updates, MoGeFi further reduces mean error to 1.96 cm and achieves lower failure rates at all four thresholds of 5, 10, 20, and 50 cm. Its FID is 0.0662 and AITS is approximately 2.23 s, compared with 0.0932 and 2.62 s for MaskControl-Fast.

These results show that MoGeFi achieves low-latency control through learned feedback in few-step generation, while terminal optimization can further improve control accuracy when additional inference computation is available.

Foot Skating

In response to the reviewer's concern about skating, we re-examined the evaluation implementations and found a difference in the temporal extent used for the skating calculation: MaskControl's official implementation computes skating over the padded 196 frames, whereas MoGeFi follows MotionLCM's evaluation and uses the motion's true valid length.

We recomputed the results using a unified 196-frame length:

MethodReported skating↓Unified skating↓
MoGeFi0.09010.0673
MaskControl-Fast0.05550.0555

With the same length, the skating gap decreases from approximately 0.0346 to 0.0118, a reduction of approximately 66%. We will unify the skating evaluation protocol in the revision, update the corresponding results, and clearly state the protocol used.


Comments on the paper

1 comment

Dear Reviewers,

A friendly reminder to check the authors’ rebuttals and engage in the discussion, as the deadline is approaching. Authors still have time to address any further questions or concerns you may have, so your timely feedback would be greatly appreciated.

Thank you for your time and effort!

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.