- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Reinterprets flow-matching training as a denoising loss with a timestep weight that emphasizes low noise, and frames noise level as curriculum difficulty (Theorem 1: optimal loss of a linear denoiser decreases in t). CurFM samples t from a logit-normal and lowers μ by 0.05 whenever one-step reconstruction FID stagnates for two 20-epoch checks. On CIFAR-10, CelebA-64 and ImageNet-128 (UNet, v- and ε-prediction) it reports faster convergence and FID 2.19 / 1.79 vs. uniform, logit-normal, Laplace, ANT, DTDC; plus small SR and T2I studies.
Critical strengths
-
One-line change to training, no inference change, 2.6–6.2% overhead.
-
Observation 1 (low-noise training transfers to high noise; high-noise-only training is unstable) is a useful small finding.
Critical weaknesses
-
Theory is near-tautological and does not justify a curriculum. Theorem 1 says irreducible MMSE grows with noise, which holds for any optimal denoiser on a Gaussian channel without the linear assumption. Higher irreducible error ≠ harder to learn, and the theorem only orders difficulty; it does not show easy-to-hard helps. DTDC defines difficulty by convergence and gets the opposite order; the paper claims to refute it with a different definition. Calling the static FM weight a “curriculum” is a conceptual slip.
-
The adaptive trigger is never isolated from a fixed decreasing schedule, and all evidence is single-seed within FID noise. Manual staged schedule (Fig. 3d) 2.22 vs. CurFM 2.19; Table 4 schedules 2.19/2.24/2.32—inside typical CIFAR-10 FID-50K variance (Fig. 3d’s non-monotone μ=0.9 shows the same). No pre-defined linear-μ control, though that is the stated advantage. CurFM itself has per-dataset μ₀, σ, μ̄, T̄, δ, interval. Fig. 4 has no table of final baseline FIDs and no error bars.
Questions for authors
-
Add a fixed linear-μ schedule control (no FID signal, matched start/end) and multi-seed mean ± std for Fig. 3d, Fig. 4, Table 4.
-
Table of final FIDs for Fig. 4 / Fig. 8.
-
Sample count and split for the one-step FID check; does using FID as both signal and metric bias results?
5 comments
Rebuttal (Part 2/4)
Response to W2, Q1 & Q2: Adaptive scheduling, convergence efficiency, and statistical reliability.
We sincerely thank the reviewer for these constructive suggestions. We would like to clarify that the primary advantage of CurFM lies not only in final generation quality, but also in accelerating training convergence while avoiding expensive manual curriculum design.
(1) Regarding predefined curriculum schedules
We emphasize that identifying an effective manual schedule, as illustrated in Figure 3, requires extensive grid search of timestep distributions across multiple training stages. Specifically, each stage involves training several models with different values of to identify the preferred distribution before proceeding to the next stage. This process incurs substantial computational costs and is impractical for general deployment. In contrast, CurFM automatically determines curriculum transitions based on the model's learning progress, without requiring repeated searches for stage-wise optimal sampling distributions. Moreover, the main scheduler hyperparameters, , , and an interval=20, have been systematically examined through the sensitivity analyses in Figures 5 and 9. These settings are shared across all experiments rather than independently tuned for each dataset. Regarding the initial sampling distribution, we adopt for all datasets following the implementation in the official Flow Matching repository, along with for CIFAR-10. For CelebA and ImageNet, we perform a simple linear sweep of over a small number of epochs to identify suitable initialization values.
To summarise, the only dataset-specific tuning in our method involves selecting the initial sampling mean value, while the adaptive curriculum mechanism and its hyperparameters remain consistent across datasets. We additionally evaluate CurFM on CIFAR-10 with different initial sampling distributions (), comparing each configuration against its standard FM counterpart with the same fixed sampling mean. We observe that CurFM consistently outperforms the corresponding static FM baseline across all tested initializations and training checkpoints, demonstrating that its benefits are not restricted to a particular choice of . We acknowledge that the initial distribution will influence absolute performance, and should be carefully selected for both standard FM and CurFM.
| FID-50K | Epoch=200 | Epoch=400 | Epoch=600 | Epoch=800 | Epoch=1000 |
|---|---|---|---|---|---|
| FM (static, ) | 3.74 | 2.78 | 2.66 | 2.56 | 2.23 |
| CurFM () | 3.57 | 2.59 | 2.29 | 2.23 | 2.19 |
| FM (static, ) | 5.19 | 3.19 | 2.75 | 2.54 | 2.40 |
| CurFM () | 4.31 | 2.96 | 2.46 | 2.41 | 2.28 |
| FM (static, ) | 6.53 | 3.91 | 3.23 | 2.92 | 2.82 |
| CurFM () | 5.49 | 3.80 | 2.93 | 2.80 | 2.70 |
To further isolate the contribution of adaptive scheduling, following your suggestion, we have additionally introduced a predefined linear- baseline that shares the same initial and final sampling distributions as CurFM but interpolates between them linearly, without using the FID-based scheduler. The results, incorporated into the ablation study in Figure 6 and summarized below, show that CurFM consistently outperforms linear decay across all evaluated checkpoints on both CIFAR-10 and CelebA, achieving final FID scores of 2.19 vs. 2.24 and 1.80 vs. 1.92, respectively. These findings demonstrate that CurFM's benefits extend beyond a predefined easy-to-hard progression, highlighting the effectiveness of its adaptive, performance-driven scheduling mechanism.
| CIFAR-10 (FID-50K) | Epoch=200 | Epoch=400 | Epoch=600 | Epoch=800 | Epoch=1000 |
|---|---|---|---|---|---|
| linear-decay | 3.75 | 2.71 | 2.32 | 2.30 | 2.24 |
| CurFM | 3.57 | 2.59 | 2.29 | 2.23 | 2.19 |
| CelebA (FID-50K) | Epoch=100 | Epoch=300 | Epoch=500 | Epoch=700 |
|---|---|---|---|---|
| linear-decay | 2.80 | 2.12 | 1.93 | 1.92 |
| CurFM | 2.72 | 2.01 | 1.81 | 1.80 |
Rebuttal (3/4)
(Continued from the previous response)
(2) Regarding statistical reliability
We acknowledge the importance of accounting for FID variance and conduct additional multi-seed experiments to strengthen our evaluation. Specifically, we repeat each experiment with three independent training seeds and report the mean and standard deviation. We reproduce Figure 3(d) and Table 4 with multiple seeds. For Figure 4, due to computational constraints, we additionally evaluate two representative settings, Figures 4(a) and 4(c), covering -prediction on CIFAR-10 and CelebA with representative baselines. The multi-seed results for Figures 3(d) and Figure 4 are consistent with our original observations: CurFM consistently achieves faster convergence and comparable or superior final performance.
For Table 4, we clarify that its purpose is not to establish the superiority of the linear schedule over other update functions, but to evaluate CurFM's robustness to different scheduling behaviors. Since curriculum transitions are primarily determined adaptively by the model's learning progress, the different update variants achieve comparable performance, supporting the robustness of the proposed framework.
| Multi-seed run for Figure 3(d) | update- | |||
|---|---|---|---|---|
| FID-50K (mean std) | 2.28 0.04 | 2.42 0.08 | 2.32 0.07 | 2.22 0.04 |
For comparison, CurFM achieves a mean FID of across three training seeds, as reported in the multi-seed results for Figure 4(a), compared with for the manually updated schedule.
| Multi-seed run for Figure 4(a)<br />CIFAR-10 (v-prediction) | Epoch=200 | Epoch=400 | Epoch=600 | Epoch=800 | Epoch=1000 |
|---|---|---|---|---|---|
| 3.77$\pm$0.04 | 2.75$\pm$0.03 | 2.68$\pm$0.05 | 2.51$\pm$0.04 | 2.26$\pm$0.06 | |
| ANT | 6.60 0.13 | 4.19 0.16 | 3.20 0.09 | 2.88 0.06 | 2.81 0.04 |
| DTDC | 7.35 0.22 | 5.19 0.17 | 4.72 0.10 | 4.10 0.23 | 4.00 0.17 |
| CurFM | 3.59 0.03 | 2.59 0.01 | 2.25 0.05 | 2.21 0.03 | 2.18 0.04 |
| Multi-seed run for Figure 4(c)<br />CelebA (v-prediction) | Epoch=100 | Epoch=300 | Epoch=500 | Epoch=700 |
|---|---|---|---|---|
| 2.77 0.06 | 2.11 0.08 | 1.83 0.05 | 1.97 0.03 | |
| ANT | 3.88 0.19 | 2.45 0.13 | 2.16 0.08 | 2.08 0.06 |
| DTDC | 3.66 0.25 | 3.15 0.14 | 2.50 0.07 | 2.21 0.09 |
| CurFM | 2.69 0.05 | 2.02 0.04 | 1.79 0.03 | 1.78 0.05 |
| Multi-seed run for Table 4 | Epoch=200 | Epoch=400 | Epoch=600 | Epoch=800 | Epoch=1000 |
|---|---|---|---|---|---|
| Geometric | 3.54$\pm$0.04 | 2.68$\pm$0.03 | 2.32$\pm$0.02 | 2.22$\pm$0.05 | 2.24$\pm$0.03 |
| Root | 3.67$\pm$0.07 | 2.68$\pm$0.05 | 2.34$\pm$0.04 | 2.26$\pm$0.02 | 2.28$\pm$0.05 |
| Linear | 3.59$\pm$0.03 | 2.59$\pm$0.01 | 2.25$\pm$0.05 | 2.21$\pm$0.03 | 2.18$\pm$0.04 |
More importantly, our contribution concerns not only final FID, but also training convergence speed. As shown in Figure 4, CurFM maintains consistently lower FID throughout training and converges substantially earlier than competing methods. This sustained advantage cannot be fully captured by comparing final FID values alone, which is why we use convergence curves as the primary form of presentation. Following the reviewer's suggestion, we additionally report the final FID values in Table 8-10, Appendix C.9 of the revised manuscript, and summarize them below for convenience. Overall, CurFM achieves faster convergence together with competitive or superior final performance, while introducing only negligible overhead and minimal modifications to existing training pipelines.
| Final FID table For Figure 4 (a) (b)<br />CIFAR10-FID 50K | Uniform | Laplace | ANT | DTDC | CurFM | |||
|---|---|---|---|---|---|---|---|---|
| -prediction | 3.10 | 6.74 | 2.23 | 2.96 | 4.24 | 2.82 | 4.02 | 2.19 |
| -prediction | 2.42 | 3.38 | 2.45 | 2.55 | 3.38 | 2.42 | 3.28 | 2.34 |
| Final FID table For Figure 4 (c) (d)<br />CelebA-FID 50K | Uniform | Laplace | ANT | DTDC | CurFM | |||
|---|---|---|---|---|---|---|---|---|
| -prediction | 2.02 | 1.96 | 2.47 | 2.10 | 2.01 | 2.03 | 2.23 | 1.80 |
| -prediction | 2.25 | 2.11 | 4.46 | 2.20 | 2.45 | 2.38 | 2.88 | 1.99 |
| Final FID table For Figure 8<br />ImageNet-FID 50K | Uniform | Laplace | ANT | DTDC | CurFM | |
|---|---|---|---|---|---|---|
| -prediction | 10.23 | 10.48 | 11.48 | 11.51 | 12.61 | 9.24 |
| -prediction | 18.60 | 17.00 | 18.30 | 17.30 | 18.40 | 14.00 |
Thanks again for your valuable comment!
Rebuttal (Part 1/4)
We sincerely thank the reviewers for their time and effort in reviewing our paper. We have carefully addressed each comment below and revised the corresponding sections of the manuscript, with all changes highlighted in blue. We would be happy to provide further clarification or answer any additional questions.
Response to W1: Theoretical motivation and justification of curriculum learning.
We sincerely thank the reviewer for these comments. We would like to clarify the theoretical motivation and scope of our analysis.
First, our definition of denoising difficulty is not newly proposed or heuristic, but follows the established ideal difficulty formulation in prior curriculum learning theory work (see Definition 1 in Weinshall et al., ICML 2018 [1]), which defines difficulty through the loss under the optimal hypothesis. Importantly, this prior work also theoretically establishes the convergence benefits of prioritizing easier samples under this difficulty formulation. Our theoretical analysis thus builds upon this foundation by establishing that noise level induces a consistent ordering under this difficulty definition. We acknowledge that our theoretical results do not prove samples with higher noise are inherently harder to learn. Rather, it provides a principled connection between noise levels and an established difficulty definition from prior curriculum learning works.
Second, the effectiveness of the resulting easy-to-hard curriculum is further supported empirically. Figure 3 demonstrates that the preferred noise distribution evolves throughout training and that a stage-wise easy-to-hard schedule outperforms the examined static alternatives. Furthermore, Figure 6 shows that reversing the curriculum direction (hard-to-easy) leads to inferior performance, while Figure 4 demonstrates faster overall convergence compared with other baselines.
Third, regarding DTDC, the seemingly conflicting orderings arise from distinct difficulty definitions. DTDC characterizes difficulty through empirical timestep-specific convergence behavior, whereas CurFM follows the established ideal-difficulty formulation. These criteria can induce opposite orderings without being mathematically contradictory. Specifically, faster convergence on individual timestep tasks does not establish that prioritizing those tasks will yield faster convergence of the overall generative model. And more importantly, the resulting curriculum direction is coupled with each method's scheduling mechanism: DTDC uses loss-based pacing over timestep clusters, whereas CurFM adaptively shifts the sampling distribution according to generation-level progress. Consequently, the preferred ordering need not be universal across the two frameworks. Indeed, our experiments show that CurFM achieves faster overall convergence than DTDC and other baselines despite adopting the opposite ordering. We have further clarified the core differences between DTDC and CurFM in Appendix B.2.
Thanks again for your valuble comments!
References:
[1] Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks, ICML 2018