- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper asks whether flow-map distillation gets easier by only changing the time parameterization of a frozen teacher ODE. It estimates a normal-acceleration profile from teacher trajectories, builds a shared clock whose density follows the square root of this profile, and plugs the reparameterized field into Lagrangian flow-map distillation. Under the stated assumptions the clock equalizes the normal-acceleration statistic. On a 2D checkerboard, CIFAR-10 and CelebA-64, one and twostep FID to target improve over the identity clock, while results at more steps mostly get worse.
Critical strengths
- The clock keeps the teacher's paths fixed, is built offline and adds no network evaluations at sampling time, so it could be dropped into existing pipelines.
- The internal comparison shows gains at one and two steps. With matched architecture, budget and hyperparameters, one-step FID at 400k goes from 8.57 to 7.44 on CIFAR-10 and from 6.19 to 5.58 on CelebA-64. This shows the time parameterization matters, but not why.
Critical weaknesses
- first of all, the geometric signal is not separated from simpler factors. In the image experiments the objective can be rewritten as standard LMD with resampled time pairs, a residual weight and a transformed time input. This does not rule out geometry, but there is no identity-clock student trained on the same time-pair distribution, no normal versus full acceleration comparison, and no direct link between the profile and distillation error. So I think the key evidence for the main design choice is missing.
- further, the coverage and statistics are too limited to show practical value. Only CIFAR-10 and CelebA-64 are used, with two internal variants, no external few-step baselines, no image samples and no larger or latent-space settings. Everything is a single run, which matters most for the small two-step gains of 5.71 → 5.31 on CIFAR and 4.35 → 4.25 on CelebA at 400k. Results at 4, 8 and 20 steps mostly get worse, with one exception at 20 steps on CIFAR. I can't place the method's performance or the benefit of the geometric signal, and this is the main reason for my rejection.
Questions for authors
Please refer to major weakness. If the identity-clock student keeps physical time as input but samples time pairs from your clock's distribution, how much of the one-step gain remains? Under the same budget, what does the normal-acceleration clock gain over a full-acceleration clock, and is there any measurement linking per-time distillation error to the hardness profile? Could you report the number, means and variances of the preliminary repetitions, together with image samples? Why do results at more steps mostly degrade, and why do the CelebA two-step FID-to-target and the later teacher-relative metrics disagree? Do you have results for an identity-clock student using your sampling grid?
3 comments
Thank you for the additional experiments. The sampling ablation addresses my concern that the gains might come from sampling high-curvature regions more often. I also appreciate the theoretical motivation for normal acceleration, though I would still like to see how it compares with full acceleration under the same training budget. The image experiments remain limited to CIFAR-10 and CelebA-64, so it is also difficult to judge how well the gains generalize. Given the stronger evidence in the rebuttal, I am raising my score from 1 to 2.
Thank you for taking the time to consider our rebuttal and for updating your assessment.
Regarding generalization, we have now completed additional experiments on class-conditional CIFAR-10, using a different pretrained flow-matching teacher (C2OT). The reparameterized student consistently improves target FID and KID across all evaluated inference budgets and training checkpoints. These results are included in the latest manuscript revision (Appendix C.4).
We acknowledge that a direct empirical comparison against a full-acceleration-based clock remains an important direction for further investigation. Our new Appendix D provides theoretical motivation for normal acceleration, but we do not claim that it establishes empirical superiority over alternative geometric diagnostics.
Thank you again for your constructive feedback and consideration.
We thank the reviewer for their detailed feedback. We have substantially revised the manuscript, particularly to address the concerns about experimental controls and the evidence supporting our geometric design choice.
1. Separating reparameterization from timestep sampling. We now include the requested identity-clock baseline trained with the same curvature-biased physical-time distribution, alongside a full 2x2 ablation (Section 5.2.3, Table 1). Importantly, curvature-biased sampling alone worsens performance, whereas reparameterization improves results under both sampling distributions. Our revised main experiments also use identical physical-time training distributions and inference grids, eliminating these sampling differences as explanations for the observed gains.
2. Stronger and more consistent results. With the revised protocol, the reparameterized student improves target FID at every evaluated inference budget (1, 2, 4, 8, and 20 NFEs) on both datasets, resolving the previously reported multi-step degradation relative to the identity baseline. Full FID/KID results across training checkpoints are provided in Appendix C. We also explicitly distinguish target fidelity from teacher-relative fidelity: improvements in the former do not necessarily imply improvements in the latter.
3. Theoretical motivation for normal acceleration. We have strengthened Section 4.1 and added Appendix D. We establish a local connection between normal acceleration and directional variation in finite-time average velocities, prove an optimality property of the equalizing clock, and analyze why exact full-acceleration equalization is more difficult and may not admit a smooth solution. While these results support our design choice, they do not establish a direct guarantee on neural-network distillation error.
4. Additional evidence. We have added qualitative CIFAR-10 and CelebA-64 samples (Appendix E, Figures 8–10) and measured the negligible offline construction overhead. We agree that additional training seeds, external baselines, and larger-scale evaluations would further strengthen the empirical evidence. These remain limitations of the current study.
We hope the new controlled experiments and theoretical analysis address the reviewer's principal concerns about soundness and clarify the contribution of time reparameterization beyond timestep sampling.