- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes MoGeFi, a few-step model for text-to-motion generation and joint-position control on HumanML3D. The backbone is an average-velocity DiT trained in the frozen MotionLCM-V2 VAE latent space, with joint observations provided as conditioning tokens. The main contribution is proposal-based geometric feedback: at each sampling step, a frozen observer predicts a clean-motion proposal, joint residuals to the target are computed in world and heading-aligned frames, and a controller uses these residuals to predict a correction to the latent velocity. The proposal is refreshed at every step. Training combines a flow loss, interpolant and rollout geometric losses, and a Fréchet loss in the feature space of the T2M evaluator. The paper reports FID 0.018 for one-step text-to-motion, 8.41 cm mean error for sparse control at about 0.061 s, and 2.54 cm for initial-motion control. The ablations study the effects of geometric feedback, proposal refresh, distributional supervision, and observer calibration.
Critical strengths
S1. The paper provides strong evidence that explicit learned feedback improves fast forward-only control.
Under the same task and supervision, the output-supervised backbone reaches 16.28 cm mean error, while adding geometric feedback reduces it to 8.41 cm. Reusing the initial proposal instead of refreshing it increases the error to 25.67 cm. These controlled ablations support the main claim that state-dependent geometric feedback provides a real gain beyond output supervision alone. The resulting 8.41 cm error at about 0.06 s also shows that MoGeFi improves control accuracy while retaining real-time inference.
Critical weaknesses
W1. The current comparisons do not fully isolate the advantage of learned feedback over optimization under a similar inference budget.
-
A natural baseline is missing: the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of the learned . Since MoGeFi already computes a clean proposal and its joint residuals during inference, this would directly test whether learned feedback is more effective than a small amount of test-time optimization.
-
The large gain over MotionLCM-V2 ControlNet should also be interpreted cautiously, since the V2 authors already report that ControlNet alone performs poorly on sparse joint constraints (V2 Tab. 4). The cleaner evidence for the effect of feedback is the internal comparison from 16.28 cm to 8.41 cm (Tab. 4(a)).
-
MaskControl also supports different optimization budgets (MaskControl Sec. A.6), but the paper does not compare against a setting tuned to a latency closer to MoGeFi. In addition, parameter counts and memory usage for the backbone, observer, and feedback modules are not reported, which makes the overall cost of the feedback design harder to judge beyond latency alone.
W2. The main motion-quality metrics are closely related to the feature space used by the training objective.
-
The Fréchet loss matches feature statistics in the same T2M evaluator space used for FID, R-Precision, and MM-Dist. This is not data leakage, but it gives MoGeFi direct supervision in the representation used by its main quality metrics, while the reported baseline numbers are not controlled for the same objective.
-
The paper's own ablation shows that adding improves FID from 0.178 to 0.083, while mean joint error changes only from 11.01 cm to 10.28 cm (Tab. 4(b)). This suggests that part of the FID improvement may come from directly matching statistics in the same feature space used for evaluation.
-
For text-to-motion, the training schedule includes a distributional adaptation stage with and (Tab. S4), while the paper does not report text-to-motion results before this adaptation. The final model achieves the best FID in Tab. 1, but its R@1 and MM-Dist are worse than MotionLCM-V2 for all reported sampling budgets.
Questions for authors
-
Can you report (a) the same backbone and observer with one or two gradient-based correction steps from the proposal joint loss instead of , and (b) a MaskControl setting with fewer optimization iterations at a latency closer to MoGeFi? If learned feedback remains more accurate under a similar inference budget, this concern would be substantially reduced.
-
Can you report (a) text-to-motion results before FD adaptation, (b) the quality of the default model trained with T2TM features under an independent evaluator such as TMR, and, if possible, (c) MotionLCM-V2 fine-tuned with the same ? If the quality advantage remains under these settings, W2 would be substantially reduced.
-
Please report parameter counts and memory usage for the backbone, observer, and feedback modules, together with the training budget of the initial generator stage.
-
The proposal descriptors are held fixed in the JVP but recomputed during sampling (Eq. S5). Is the flow objective an exact identity for the full feedback field, or a local approximation? Relatedly, why does control become worse from two to four steps (Tab. S8: 8.41 to 9.76 cm) even though four-step rollouts are explicitly used during training (Tab. S3)? Does repeated feedback accumulate correction error, or is the controller sensitive to shorter sampling intervals?
11 comments
W1(c): Optimization Budgets and Model Cost
MaskControl also supports different optimization budgets (MaskControl Sec. A.6), but the paper does not compare against a setting tuned to a latency closer to MoGeFi. In addition, parameter counts and memory usage for the backbone, observer, and feedback modules are not reported, which makes the overall cost of the feedback design harder to judge beyond latency alone.
Question Q3
Please report parameter counts and memory usage for the backbone, observer, and feedback modules, together with the training budget of the initial generator stage.
Thank you for this suggestion. Such a comparison is important for understanding control accuracy under low-latency constraints. We therefore evaluate MaskControl at different optimization budgets and report both control metrics and AITS.
To further examine MoGeFi under a comparable inference budget using gradient optimization similar to MaskControl, we also apply terminal gradient optimization to the output of the complete model. After generation, we compute the geometric error between the controlled joints and their targets through the frozen motion decoder and iteratively update the generated continuous latent.
Response Table 2. Generation quality, control accuracy, and inference time across optimization budgets
Traj. and Loc. are failure rates (%) at the 50 cm threshold. The optimization counts refer to terminal gradient updates after generation. Bold AITS, FID, and mean-error values indicate the better of the two methods at the same optimization count.
| Opt. | Method | AITS ms↓ | FID↓ | Avg. cm↓ | Traj.↓ | Loc.↓ |
|---|---|---|---|---|---|---|
| 0 | MoGeFi (paper model) | 61 | 0.0633 | 8.41 | 2.25 | 0.59 |
| 1 | MaskControl | 223 | 0.0677 | 49.53 | 47.97 | 29.64 |
| 1 | MoGeFi + Gradient | 84 | 0.0682 | 7.93 | 2.22 | 0.54 |
| 2 | MaskControl | 242 | 0.0681 | 47.26 | 46.11 | 28.15 |
| 2 | MoGeFi + Gradient | 100 | 0.0682 | 7.56 | 1.99 | 0.49 |
| 5 | MaskControl | 312 | 0.0689 | 41.43 | 40.81 | 24.31 |
| 5 | MoGeFi + Gradient | 154 | 0.0686 | 6.61 | 1.64 | 0.39 |
| 10 | MaskControl | 444 | 0.0692 | 34.05 | 34.23 | 19.57 |
| 10 | MoGeFi + Gradient | 258 | 0.0690 | 5.48 | 1.22 | 0.25 |
| 20 | MaskControl | 781 | 0.0672 | 24.00 | 25.06 | 13.01 |
| 20 | MoGeFi + Gradient | 475 | 0.0694 | 4.13 | 0.82 | 0.17 |
| 50 | MaskControl | 1,610 | 0.0789 | 10.18 | 10.50 | 4.44 |
| 50 | MoGeFi + Gradient | 1,143 | 0.0682 | 2.65 | 0.35 | 0.043 |
| 100 | MaskControl-Fast | 2,620 | 0.0932 | 3.22 | 2.72 | 0.84 |
| 100 | MoGeFi + Gradient | 2,228 | 0.0662 | 1.96 | 0.19 | 0.012 |
Control is evaluated by sampling a single joint and observation density for each sequence; the resulting metrics may differ slightly from those reported in Table 2 of the paper. AITS is measured on an RTX 4090.
MoGeFi has a clear control-accuracy advantage at low latency. The default MoGeFi achieves 8.41 cm mean error in 61 ms. MaskControl takes 1.61 s for 50 optimizations and still has a mean error of 10.18 cm. Thus, MoGeFi achieves lower error in approximately 1/26 of the inference time.
MoGeFi with gradient optimization also retains a clear advantage at similar latency. At approximately 0.25 s, MoGeFi + Gradient achieves 5.48 cm mean error, compared with 47.26 cm for MaskControl. At approximately 0.45 s, the errors are 4.13 cm and 34.05 cm, respectively.
With more inference computation, MoGeFi with 50 terminal optimizations achieves 2.65 cm in 1.14 s, outperforming MaskControl-Fast's 3.22 cm / 2.62 s. With 100 optimizations, MoGeFi further reduces the error to 1.96 cm with an AITS of 2.23 s, while also achieving lower FID and 50 cm failure rates.
These results show that MoGeFi provides low-latency control through learned feedback during few-step generation, while terminal optimization can further improve control accuracy when more inference computation is available.
W1(c): Optimization Budgets and Model Cost (Continue)
Model Size and Training Budget
Thank you for the suggestion. We have described the architecture of each module in the supplementary material: the generative backbone is a 12-layer DiT with a hidden dimension of 768 and 12 attention heads. To present the scale of each component more directly, we additionally report the inference parameter counts below. The joint-observation encoder encodes observations, while the geometric context encoder and feedback controller construct and apply feedback from the observer's motion estimate.
Response Table 3. Inference parameter counts
| Module | Parameters |
|---|---|
| DiT backbone | 132.98 M |
| Joint-observation encoder | 12.32 M |
| Geometric context encoder | 1.73 M |
| Feedback controller | 0.88 M |
| Sentence-T5 text encoder | 335.73 M |
| Motion VAE decoder | 10.21 M |
| Total inference parameters | 493.85 M |
The backbone has 132.98 M parameters. The observer and candidate share the backbone and joint-observation encoder parameters. Dynamic feedback adds only 2.61 M parameters for the geometric context encoder and feedback controller, approximately 1.97% of the backbone parameter count.
On an RTX 3090, inference with FP32, batch size 1, 196 frames, and two generation steps has a peak allocated GPU memory of 1.86 GiB.
The initial generator training budgets for the individual tasks are:
Response Table 4. Initial generator training budgets
| Task | Training steps | Learning rate |
|---|---|---|
| Text-to-motion | 36k | |
| Temporal six-joint control | 32k | |
| Fixed-pelvis control | 32k | |
| Random-joint control | 32k |
The training steps and learning rates for subsequent distributional adaptation, feedback learning, and feedback calibration are already reported in Table S3 of the paper. We will add the parameter counts, memory usage, and initial training budgets above to the revised paper.
W1 / Q1: Learned Feedback and Test-Time Optimization
Question Q1
W1(a): Replacing Learned Feedback with Gradient Corrections
Thank you for suggesting this meaningful comparison. We retain the same backbone and observer and replace learned feedback with gradient corrections based on the proposal joint error. This directly compares how the two approaches use the current geometric information for control. For two-step generation, we apply 1, 2, or 5 gradient corrections at each generation stage.
Response Table 1. Control accuracy and inference time when gradient corrections replace feedback
A small number of gradient corrections improves control, but MoGeFi achieves substantially lower joint error in less inference time without test-time gradient optimization. Even with 5 gradient corrections at each generation stage, the mean error is 15.16 cm, compared with 8.41 cm for MoGeFi. This shows that learned geometric feedback can directly provide effective control corrections during few-step generation, making better use of a low-latency inference budget.
W1(b): MotionLCM-V2 and the Internal Feedback Comparison
Thank you for pointing this out. The MotionLCM-V2 authors observe that ControlNet alone achieves lower accuracy on sparse joint control than on initial-motion control. Sparse joint control is more challenging, a difference also reflected in our results: MoGeFi obtains mean errors of 8.41 cm and 2.54 cm on sparse joint and initial-motion control, respectively.
MotionLCM-V2 is our direct baseline. MoGeFi retains its pretrained VAE, text encoder, and motion representation, so this comparison evaluates the complete method against an existing fast control framework. The contribution of feedback is established by the internal comparison: under the same task and supervision objectives, adding feedback reduces the mean error from 16.28 cm to 8.41 cm. The new gradient-replacement experiment with the same backbone and observer further shows that learned corrections use proposal geometry more effectively than a small number of gradient iterations.
We will clarify the respective roles of these comparisons in the revised paper to avoid misunderstanding.