- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Defect-Guided Time-Warp Flow Distillation (DG-TWFD), which focuses on trajectory-based diffusion distillation. DG-TWFD jointly learns a continuous-time student flow map and a monotone time warp that reallocates temporal resolution toward regions with high compositional difficulty. By explicitly minimizing the semigroup defect, which measures the discrepancy between a direct transition and its multi-step composition, DG-TWFD aims to improve the robustness of diffusion generation across different inference step counts. The authors evaluate their method on CIFAR-10 and ImageNet-64.
Critical strengths
- The paper is well-written and easy to follow.
- The paper presents an interesting attempt to introduce adaptive timestep scheduling based on the compositional difficulty observed during training.
Critical weaknesses
- Limited evaluation on large-scale generation tasks. The experimental evaluation primarily focuses on CIFAR-10 and ImageNet-64. More comprehensive experiments on higher-resolution datasets, such as ImageNet-256, or text-to-image generation tasks would help demonstrate the scalability and generalizability of the proposed method.
- Insufficient ablation of the time-warping mechanism. The proposed method combines defect-aware training-time sampling with an adaptive inference schedule. However, the individual contributions of these two components are not sufficiently disentangled, making it difficult to determine which component is primarily responsible for the performance improvements.
Questions for authors
- What are the individual contributions of training-time sampling and inference-time scheduling? Could the authors provide an ablation study that separates these two components to clarify their respective contributions to the performance gains?
- Can the authors provide comparisons under matched computational budgets? Since DG-TWFD introduces additional operations for semigroup defect estimation and adaptive time warping, it would be helpful to compare its performance with existing distillation methods under matched training computational budgets.
2 comments
Thank you for your reply!
I thank the author for their detailed reply. I apologize for overlooking the ImageNet-256 experiments in Appendix F.2.. However, these results do not sufficiently address my concern for two reasons: first the FID is not evaluated at its standard protocol, and the result is not promising enough to demonstrate whether the method is practical.
I agree that not every suggested experiment is necessary, but I find that only experiment at small scale datasets like CIFAR10 and Imagenet64 does not provides much insight for demonstrating the effectiveness of the method.
Moreover, even if we are talking about small scale dataset experiments, many baselines in literature are neglected by the paper. Please see the references below.
Therefore, I am lowering my score.
[1] Xu, Yilun, Weili Nie, and Arash Vahdat. "One-step diffusion models with $ f $-divergence distribution matching." arXiv preprint arXiv:2502.15681 (2025).
[2] Geng, Zhengyang, et al. "Mean flows for one-step generative modeling." Advances in Neural Information Processing Systems 38 (2026): 75460-75482.
[3] Geng, Zhengyang, et al. "Consistency models made easy." International Conference on Learning Representations. Vol. 2025. 2025.
[4] Zhou, Mingyuan, et al. "Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation." Forty-first International Conference on Machine Learning. 2024.
[5] Zhou, Mingyuan, et al. "Adversarial score identity distillation: Rapidly surpassing the teacher in one step." arXiv preprint arXiv:2410.14919 (2024).
Response to Reviewer 8oNa
We thank the reviewer for the constructive comments. We answer each point below.
W1 (large-scale evaluation).
The ImageNet-256 experiment is in Appendix F.2 (latent space): the warp lowers the defect by 6–12% at every checkpoint, which shows that the method scales to 256×256. Higher resolutions and text-to-image use the same latent formulation and would not add further insight (see also our reply to TGbi, W3).
W2 (ablation of the two components).
Both components contribute, and they are coupled by design: one set of bin masses defines the training sampling and the inference grid (Eqs. 12–14). Fig. 4 isolates the warp (4-step FID from 2.79 to 2.62 on CIFAR-10 and from 2.78 to 2.58 on ImageNet64), and Table 4 shows that replacing the learned clock with fixed schedules degrades FID at every step count.
Q1 (individual contributions).
We separate the two components with two controlled comparisons. First, Fig. 4 adds them one at a time on top of arbitrary-time distillation, with the number of updates and all other settings fixed:
Turning on the warp, which sets both the training-time sampling and the inference grid, lowers the 4-step FID by a further 0.17 and 0.20. Second, Table 9 isolates inference-time scheduling: on one checkpoint, it changes only the inference grid. No fixed shift is best at every budget, and the learned grid gives the best 2-step FID and the lowest worst-case FID (3.24 vs. 3.29–3.56). Since one set of bin masses drives both components by design (Eqs. 12–14), Table 4 evaluates the warp as a whole and shows that replacing it with fixed schedules degrades FID at every step count.
Q2 (matched compute).
The extra cost is small: the defect adds student forward passes, and the warp update is an EMA over a few bins. The three runs in Fig. 4 share the same number of updates, so the gains of Ldef and the warp are measured at matched budgets. Against other distillation methods, DG-TWFD trains on a single A100 GPU, while CTM, for example, reports four V100 GPUs for CIFAR-10 and eight A100 GPUs for ImageNet64 and also trains a discriminator, so our budget is of the same order as the baselines we compare with.