- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ImpactFlow modifies flow-matching training to penalize prediction errors according to their effect on the final generated output. It estimates this sensitivity by differentiating through a frozen teacher’s remaining flow, decoder, and motion features. Random probes make the additional loss practical, and inference remains unchanged.
Experiments on HumanML3D evaluate continuation from a pretrained model. The paper compares against endpoint-based controls, studies the sensitivity estimates, and reports results under both matched-update and matched-time budgets.
Critical strengths
-
The method addresses a meaningful limitation of the standard FM objective. Equal-sized velocity errors can have different downstream effects. Accounting for the remaining flow gives a concrete way to distinguish these directions, while leaving inference unchanged. This is the main methodological contribution in my assessment.
-
The controls provide convincing evidence that the transported direction matters. The endpoint-matched comparison controls several scalar factors that could otherwise explain the improvement. ImpactFlow improves FID in nine of ten seeds, with a mean relative improvement of 13.23%. Together with the sensitivity and direction-shuffling experiments, this supports the proposed mechanism beyond a simple increase in auxiliary supervision.
Critical weaknesses
-
The practical benefit under a fixed compute budget remains uncertain. Each update requires additional teacher and sensitivity computation. In the matched-time experiment, the mean FID improvement is 10.87%, but its 95% confidence interval is [−1.86%, 23.60%]. The matched-update results support the value of the training signal, but the evidence that it pays for its computational cost is less conclusive. This limits the practical significance of the reported gains.
-
The evaluation does not establish whether the FID gains translate into broader motion-quality improvements. The method explicitly uses joint positions, velocities, bone lengths, and contact-related features, yet the reported generation results do not adequately cover physical quality, text alignment, or diversity. These measurements matter because improving FID alone does not rule out tradeoffs in other aspects of generation. The restriction to one dataset, model, and short continuation setting further limits the scope of the evidence.
Questions for authors
-
Clarification on compute efficiency: How sensitive is the matched-time result to the chosen continuation budget? If existing measurements cover other budgets, do they show a consistent advantage after including all geometry-computation costs?
-
Clarification on generation quality: Do you have text-alignment, diversity, or physical-quality measurements for the same checkpoints? These would materially affect my assessment of whether the FID improvement represents a broader quality gain.
3 comments
Official Comment by Authors
W2/Q2 — Motion quality on the same checkpoints and evaluation breadth
Thank you for emphasizing that FID should be considered together with text alignment, diversity, and physical motion quality. We measured these metrics on the same 32 pairs of HumanML3D checkpoints used for the main FID comparison.
| Method | R-Precision @1 ↑ | MM-Dist ↓ | Diversity | Multimodality | Foot-skating rate ↓ |
|---|---|---|---|---|---|
| Standard FM | 0.4462 | 3.330 | 9.472 | 1.977 | 7.24% |
| ImpactFlow | 0.4447 | 3.325 | 9.462 | 2.037 | 7.51% |
| Holm-adjusted p | 0.28988 | 0.59409 | 0.59409 | 6.52 × 10⁻⁹ | 0.00024 |
The table reports text alignment, overall diversity, multimodality, and foot skating on the same checkpoints as the primary FID evaluation. ImpactFlow also increases Multimodality from 1.977 to 2.037. This provides a direct, metric-by-metric account of how the methods compare beyond FID.
We further broadened the evaluation to KIT-ML and two additional HumanML3D backbone sizes:
| Study | Setting | Standard FM FID ↓ | ImpactFlow FID ↓ |
|---|---|---|---|
| KIT-ML | 16 models, 2,000 pretraining epochs, matched continuation updates, NFE 8 | 2.64103 | 2.60573 |
| HumanML3D | 16.8M backbone, single-seed evaluation, NFE 8 | 0.225 | 0.200 |
| HumanML3D | 130M backbone, single-seed evaluation, NFE 8 | 0.157 | 0.133 |
On KIT-ML, the mean modelwise relative FID reduction is 1.32%; the 95% CI for the absolute Standard-minus-ImpactFlow reduction is [0.00294, 0.06767], with exact paired p = 0.02304. These additions extend the evaluation across datasets, model sizes, training budgets, and complementary motion-quality measures.
Thank you again for emphasizing the importance of evaluating compute cost and motion quality together. The expanded equal-time study and the additional quality metrics provide a more complete account of ImpactFlow's behavior under the requested settings. We hope this revised evidence and presentation address your questions.
Follow-up on Additional Efficiency and Motion Quality Results
Dear Reviewer wW8Z,
Thank you again for your thoughtful review and for emphasizing the importance of evaluating computational efficiency and motion quality together. We particularly appreciate your suggestions regarding the equal-time setting, continuation-budget sensitivity, and complementary motion-quality metrics.
Following your suggestions, we expanded the equal-time HumanML3D study to 32 independently pretrained models, including all teacher rollout and sensitivity computation costs. The expanded study shows a 10.03% mean relative FID reduction under the same continuation-time budget. We also evaluated three continuation budgets and reported R-Precision, MM-Dist, Diversity, Multimodality, and foot-skating measurements on the same checkpoints used for the main FID comparison. In addition, we added KIT-ML experiments and evaluations with different backbone sizes.
We hope these additional experiments provide a clearer and more comprehensive picture of ImpactFlow's practical effectiveness and motion-generation quality.
As the discussion deadline is approaching, we wanted to kindly ask whether you have had a chance to review our updated responses. We would greatly appreciate any further thoughts on whether the additional evidence addresses your questions.
Thank you again for your time, thoughtful feedback, and consideration.
Official Comment by Authors
We thank you for the careful assessment of both the computational cost and the motion quality of ImpactFlow. Your questions motivated us to strengthen the end-to-end equal-time evaluation, examine multiple continuation budgets, and report text-alignment, diversity, multimodality, and foot-skating metrics on the same checkpoints as the main FID comparison. We also broadened the evaluation to KIT-ML and additional backbone sizes. We respond to each point below.
W1/Q1 — Practical benefit and sensitivity to continuation budget
We appreciate the focus on end-to-end compute, since the practical value of ImpactFlow should be assessed with geometry estimation included. We therefore expanded the equal-time HumanML3D comparison to 32 paired models. Both methods receive 1,919 seconds on four A100 GPUs; the ImpactFlow budget includes teacher rollout, sensitivity computation, student updates, startup, and finalization. Both methods use the same Standard-FM-selected midpoint NFE 8 / CFG 3 deployment.
Across the 32 paired models, the absolute FID reduction is 0.01100 (95% CI [0.00215, 0.01985], exact two-sided p = 0.01707), and the mean modelwise relative reduction is 10.03% (95% CI [1.18%, 18.88%], p = 0.02809). Each model is evaluated four times, with the independently trained model as the statistical unit.
We also examined three continuation budgets using the same eight prespecified parent models. Each budget uses independently trained continuations, and geometry computation is included in the time cap.
ImpactFlow obtains lower mean FID at all three continuation budgets. This budget study complements the larger equal-time result and provides a direct view of how the method behaves as the available continuation time changes.