acceptodds
Under review as a conference paper at ICLR 2027

Training Flow-based Generative Models with Adaptive Curriculum Sampler

Abstract

The training of state-of-the-art flow-based generative models can be viewed as learning to denoise latents across a continuum of timesteps, or noise levels. Despite their empirical success, the role of timestep sampling in training and the relationship between flow-based generative models and conventional denoising methods remain insufficiently understood in existing literature. To answer this question, in this paper we present a principled view that interprets noise level as a proxy for sample difficulty in curriculum learning from a denoising perspective, thereby providing a natural basis for organizing samples with different noise levels into an easy-to-hard curricula. We show both theoretically and empirically that noise level provides a consistent measure for this curriculum-level difficulty, establishing it as a foundation for guiding the sampling distribution. This perspective further suggests that the timestep sampling should adapt to the model’s evolving denoising ability during training. However, existing methods typically rely on entirely fixed or weakly adaptive strategies, limiting the generation performance. To address this issue, we further propose CurFM}, a Curriculum-based adaptive sampler for flow-based generative models that progressively shifts the timestep distribution toward more challenging noise levels according to the model’s current denoising capability, without relying on unstable loss signals. CurFM benefits in minimal modifications to existing flow-based training pipelines without changes during inference, while achieving faster convergence as well as competitive or superior performance under both v- and -prediction frameworks.

Primary area:
generative models
Keywords:
Flow Matching, Generative Models, Timestep Sampling, Curriculum Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AC2NOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Reinterprets flow-matching training as a denoising loss with a timestep weight that emphasizes low noise, and frames noise level as curriculum difficulty (Theorem 1: optimal loss of a linear denoiser decreases in t). CurFM samples t from a logit-normal and lowers μ by 0.05 whenever one-step reconstruction FID stagnates for two 20-epoch checks. On CIFAR-10, CelebA-64 and ImageNet-128 (UNet, v- and ε-prediction) it reports faster convergence and FID 2.19 / 1.79 vs. uniform, logit-normal, Laplace, ANT, DTDC; plus small SR and T2I studies.

Critical strengths
  1. One-line change to training, no inference change, 2.6–6.2% overhead.

  2. Observation 1 (low-noise training transfers to high noise; high-noise-only training is unstable) is a useful small finding.

Critical weaknesses
  1. Theory is near-tautological and does not justify a curriculum. Theorem 1 says irreducible MMSE grows with noise, which holds for any optimal denoiser on a Gaussian channel without the linear assumption. Higher irreducible error ≠ harder to learn, and the theorem only orders difficulty; it does not show easy-to-hard helps. DTDC defines difficulty by convergence and gets the opposite order; the paper claims to refute it with a different definition. Calling the static FM weight a “curriculum” is a conceptual slip.

  2. The adaptive trigger is never isolated from a fixed decreasing schedule, and all evidence is single-seed within FID noise. Manual staged schedule (Fig. 3d) 2.22 vs. CurFM 2.19; Table 4 schedules 2.19/2.24/2.32—inside typical CIFAR-10 FID-50K variance (Fig. 3d’s non-monotone μ=0.9 shows the same). No pre-defined linear-μ control, though that is the stated advantage. CurFM itself has per-dataset μ₀, σ, μ̄, T̄, δ, interval. Fig. 4 has no table of final baseline FIDs and no error bars.

Questions for authors
  1. Add a fixed linear-μ schedule control (no FID signal, matched start/end) and multi-seed mean ± std for Fig. 3d, Fig. 4, Table 4.

  2. Table of final FIDs for Fig. 4 / Fig. 8.

  3. Sample count and split for the one-step FID check; does using FID as both signal and metric bias results?

5 comments
AuthorsOpenReview ↗

Rebuttal (Part 1/4)

We sincerely thank the reviewers for their time and effort in reviewing our paper. We have carefully addressed each comment below and revised the corresponding sections of the manuscript, with all changes highlighted in blue. We would be happy to provide further clarification or answer any additional questions.

Response to W1: Theoretical motivation and justification of curriculum learning.

We sincerely thank the reviewer for these comments. We would like to clarify the theoretical motivation and scope of our analysis.

First, our definition of denoising difficulty is not newly proposed or heuristic, but follows the established ideal difficulty formulation in prior curriculum learning theory work (see Definition 1 in Weinshall et al., ICML 2018 [1]), which defines difficulty through the loss under the optimal hypothesis. Importantly, this prior work also theoretically establishes the convergence benefits of prioritizing easier samples under this difficulty formulation. Our theoretical analysis thus builds upon this foundation by establishing that noise level induces a consistent ordering under this difficulty definition. We acknowledge that our theoretical results do not prove samples with higher noise are inherently harder to learn. Rather, it provides a principled connection between noise levels and an established difficulty definition from prior curriculum learning works.

Second, the effectiveness of the resulting easy-to-hard curriculum is further supported empirically. Figure 3 demonstrates that the preferred noise distribution evolves throughout training and that a stage-wise easy-to-hard schedule outperforms the examined static alternatives. Furthermore, Figure 6 shows that reversing the curriculum direction (hard-to-easy) leads to inferior performance, while Figure 4 demonstrates faster overall convergence compared with other baselines.

Third, regarding DTDC, the seemingly conflicting orderings arise from distinct difficulty definitions. DTDC characterizes difficulty through empirical timestep-specific convergence behavior, whereas CurFM follows the established ideal-difficulty formulation. These criteria can induce opposite orderings without being mathematically contradictory. Specifically, faster convergence on individual timestep tasks does not establish that prioritizing those tasks will yield faster convergence of the overall generative model. And more importantly, the resulting curriculum direction is coupled with each method's scheduling mechanism: DTDC uses loss-based pacing over timestep clusters, whereas CurFM adaptively shifts the sampling distribution according to generation-level progress. Consequently, the preferred ordering need not be universal across the two frameworks. Indeed, our experiments show that CurFM achieves faster overall convergence than DTDC and other baselines despite adopting the opposite ordering. We have further clarified the core differences between DTDC and CurFM in Appendix B.2.

Thanks again for your valuble comments!

References:

[1] Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks, ICML 2018

AuthorsOpenReview ↗

Rebuttal (Part 2/4)

Response to W2, Q1 & Q2: Adaptive scheduling, convergence efficiency, and statistical reliability.

We sincerely thank the reviewer for these constructive suggestions. We would like to clarify that the primary advantage of CurFM lies not only in final generation quality, but also in accelerating training convergence while avoiding expensive manual curriculum design.

(1) Regarding predefined curriculum schedules

We emphasize that identifying an effective manual schedule, as illustrated in Figure 3, requires extensive grid search of timestep distributions across multiple training stages. Specifically, each stage involves training several models with different values of μ\mu to identify the preferred distribution before proceeding to the next stage. This process incurs substantial computational costs and is impractical for general deployment. In contrast, CurFM automatically determines curriculum transitions based on the model's learning progress, without requiring repeated searches for stage-wise optimal sampling distributions. Moreover, the main scheduler hyperparameters, μˉ=0.05\bar{\mu}=0.05, Tˉ=2\bar{T}=2, and an interval=20, have been systematically examined through the sensitivity analyses in Figures 5 and 9. These settings are shared across all experiments rather than independently tuned for each dataset. Regarding the initial sampling distribution, we adopt σ=1.2\sigma=1.2 for all datasets following the implementation in the official Flow Matching repository, along with μ0=1.2\mu_0=1.2 for CIFAR-10. For CelebA and ImageNet, we perform a simple linear sweep of μ0\mu_0 over a small number of epochs to identify suitable initialization values.

To summarise, the only dataset-specific tuning in our method involves selecting the initial sampling mean value, while the adaptive curriculum mechanism and its hyperparameters remain consistent across datasets. We additionally evaluate CurFM on CIFAR-10 with different initial sampling distributions (μ0\mu_0), comparing each configuration against its standard FM counterpart with the same fixed sampling mean. We observe that CurFM consistently outperforms the corresponding static FM baseline across all tested initializations and training checkpoints, demonstrating that its benefits are not restricted to a particular choice of μ0\mu_0. We acknowledge that the initial distribution will influence absolute performance, and should be carefully selected for both standard FM and CurFM.

FID-50KEpoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
FM (static, μ=1.2\mu=1.2)3.742.782.662.562.23
CurFM (μ0=1.2\mu_0=1.2)3.572.592.292.232.19
FM (static, μ=0.6\mu=0.6)5.193.192.752.542.40
CurFM (μ0=0.6\mu_0=0.6)4.312.962.462.412.28
FM (static, μ=0.2\mu=0.2)6.533.913.232.922.82
CurFM (μ0=0.2\mu_0=0.2)5.493.802.932.802.70

To further isolate the contribution of adaptive scheduling, following your suggestion, we have additionally introduced a predefined linear-μ\mu baseline that shares the same initial and final sampling distributions as CurFM but interpolates between them linearly, without using the FID-based scheduler. The results, incorporated into the ablation study in Figure 6 and summarized below, show that CurFM consistently outperforms linear decay across all evaluated checkpoints on both CIFAR-10 and CelebA, achieving final FID scores of 2.19 vs. 2.24 and 1.80 vs. 1.92, respectively. These findings demonstrate that CurFM's benefits extend beyond a predefined easy-to-hard progression, highlighting the effectiveness of its adaptive, performance-driven scheduling mechanism.

CIFAR-10 (FID-50K)Epoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
linear-decay3.752.712.322.302.24
CurFM3.572.592.292.232.19
CelebA (FID-50K)Epoch=100Epoch=300Epoch=500Epoch=700
linear-decay2.802.121.931.92
CurFM2.722.011.811.80
AuthorsOpenReview ↗

Rebuttal (3/4)

(Continued from the previous response)

(2) Regarding statistical reliability

We acknowledge the importance of accounting for FID variance and conduct additional multi-seed experiments to strengthen our evaluation. Specifically, we repeat each experiment with three independent training seeds and report the mean and standard deviation. We reproduce Figure 3(d) and Table 4 with multiple seeds. For Figure 4, due to computational constraints, we additionally evaluate two representative settings, Figures 4(a) and 4(c), covering vv-prediction on CIFAR-10 and CelebA with representative baselines. The multi-seed results for Figures 3(d) and Figure 4 are consistent with our original observations: CurFM consistently achieves faster convergence and comparable or superior final performance.

For Table 4, we clarify that its purpose is not to establish the superiority of the linear schedule over other update functions, but to evaluate CurFM's robustness to different scheduling behaviors. Since curriculum transitions are primarily determined adaptively by the model's learning progress, the different update variants achieve comparable performance, supporting the robustness of the proposed framework.

Multi-seed run for Figure 3(d)μ=1.1\mu=1.1μ=0.9\mu=0.9μ=0.8\mu=0.8update-μ\mu
FID-50K (mean ±\pm std)2.28 ±\pm 0.042.42 ±\pm 0.082.32 ±\pm 0.072.22 ±\pm 0.04

For comparison, CurFM achieves a mean FID of 2.182.18 across three training seeds, as reported in the multi-seed results for Figure 4(a), compared with 2.222.22 for the manually updated schedule.

Multi-seed run for Figure 4(a)<br />CIFAR-10 (v-prediction)Epoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
μ=1.2\mu=1.23.77$\pm$0.042.75$\pm$0.032.68$\pm$0.052.51$\pm$0.042.26$\pm$0.06
ANT6.60 ±\pm 0.134.19 ±\pm 0.163.20 ±\pm 0.092.88 ±\pm 0.062.81 ±\pm 0.04
DTDC7.35 ±\pm 0.225.19 ±\pm 0.174.72 ±\pm 0.104.10 ±\pm 0.234.00 ±\pm 0.17
CurFM3.59 ±\pm 0.032.59 ±\pm 0.012.25 ±\pm 0.052.21 ±\pm 0.032.18 ±\pm 0.04
Multi-seed run for Figure 4(c)<br />CelebA (v-prediction)Epoch=100Epoch=300Epoch=500Epoch=700
μ=0.2\mu=0.22.77 ±\pm 0.062.11 ±\pm 0.081.83 ±\pm 0.051.97 ±\pm 0.03
ANT3.88 ±\pm 0.192.45 ±\pm 0.132.16 ±\pm 0.082.08 ±\pm 0.06
DTDC3.66 ±\pm 0.253.15 ±\pm 0.142.50 ±\pm 0.072.21 ±\pm 0.09
CurFM2.69 ±\pm 0.052.02 ±\pm 0.041.79 ±\pm 0.031.78 ±\pm 0.05
Multi-seed run for Table 4Epoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
Geometric3.54$\pm$0.042.68$\pm$0.032.32$\pm$0.022.22$\pm$0.052.24$\pm$0.03
Root3.67$\pm$0.072.68$\pm$0.052.34$\pm$0.042.26$\pm$0.022.28$\pm$0.05
Linear3.59$\pm$0.032.59$\pm$0.012.25$\pm$0.052.21$\pm$0.032.18$\pm$0.04

More importantly, our contribution concerns not only final FID, but also training convergence speed. As shown in Figure 4, CurFM maintains consistently lower FID throughout training and converges substantially earlier than competing methods. This sustained advantage cannot be fully captured by comparing final FID values alone, which is why we use convergence curves as the primary form of presentation. Following the reviewer's suggestion, we additionally report the final FID values in Table 8-10, Appendix C.9 of the revised manuscript, and summarize them below for convenience. Overall, CurFM achieves faster convergence together with competitive or superior final performance, while introducing only negligible overhead and minimal modifications to existing training pipelines.

Final FID table For Figure 4 (a) (b)<br />CIFAR10-FID 50Kμ=0.0\mu=0.0μ=−1.0\mu=-1.0μ=1.2\mu=1.2UniformLaplaceANTDTDCCurFM
vv-prediction3.106.742.232.964.242.824.022.19
ϵ\epsilon-prediction2.423.382.452.553.382.423.282.34
Final FID table For Figure 4 (c) (d)<br />CelebA-FID 50Kμ=0.0\mu=0.0μ=0.2\mu=0.2μ=1.2\mu=1.2UniformLaplaceANTDTDCCurFM
vv-prediction2.021.962.472.102.012.032.231.80
ϵ\epsilon-prediction2.252.114.462.202.452.382.881.99
Final FID table For Figure 8<br />ImageNet-FID 50Kμ=1.2\mu=1.2UniformLaplaceANTDTDCCurFM
vv-prediction10.2310.4811.4811.5112.619.24
ϵ\epsilon-prediction18.6017.0018.3017.3018.4014.00

Thanks again for your valuable comment!

Reviewer_tLKiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents CurFM, a method to train flow models via a curriculum. The papers is based on the intuition that low-noise samples provide more stable optimization signals, while high-noise levels are usually more challengning. They offer a principled view and theoretical justification that noise level directly is a good proxy for denoising difficulty. The authors propose a method to progressively adapt the timestep distribution during training, with FID score based denoising capability checks, and show faster convergence with improved stability on multiple small-scale datasets.

Critical strengths

The paper nicely motivates their approach with noise being a good proxy for denoising difficulty, but then also show evidence that the optimal timestep distribution changes during training. The experiment in figure 3 nicely shows that the best logit-normal mean changing towards more noise over training time, and I would argue that this observation is the papers main conceptual contribution, even though this might be confounded by factors like learning-rate decay, starting μ\mu, or resolution. This is main reason I see value in the paper, as it challenges fixed timestep samplers.

Critical weaknesses

W1: Missing connection to resolution Prior works have shown that the timestep distribution is dependent on the resolution of the data [1,2]. The same timestep results in an effectively different signal-to-noise ratio (SNR) at different resolutions, so it destroys less information the higher the resolution. JiT itself also shifts their timestep distribution towards noisier states for that reason (and so does Flux). Even though this is a major factor for timestep distribution selection, CurFM does not include this factor and mainly attributes the gains to "evolving denoising ability", neglecting the simpler explanation of just finding the right resolution-dependent sampler.

W2: Generalization of method The paper argues that a a fixed "schedule must be manually tuned for each model and dataset" (L332), which would require extensive hyper-parameter search, and presents CurFM as the general fix. However, it seems like μ\mu differs per dataset, as stated in L808 (1.2 for CIFAR-10, 0.2 for CelebA, ...). Further, the method introduces additional hyperparameters, such as the patience interval Tˉ\bar{T} and FID-score delta, where the latter especially being questionable, as FID for text-to-image (1k) and the main experiment FID differ in sample size and thus scale, so the delta can actually make a difference. It seems like these parameters are found empirically via a grid-search, and thus dont give any principled heuristics how to select them. Further, if we think about scaling this method, I wonder how the overhead of computing FID every N steps actually impacts model throughput.

W3: Loss fluctuations One key motivation for the papers adaptive sampler is, that the loss is unstable and highly stochastic. The paper does not discuss or analyse where this variability actually comes from. Taking the FM loss, we form the expectation over noise levels t∼p(t)t \sim p(t), and the per t loss magnitudes differ substantially (as theorem 1 of the paper shows for the linear case). The fluctuations therefore might mainly reflect which noise levels are sampled per batch, not the model's progress itself. For CurFM this matters even more, as shifting μ\mu to higher noise actually increases the pooled loss, so the raw loss is not directly comparable across curriculum stages. Further, the proposed FID-based signal itself seems to be noisy, as it is also computed over a mixture of timesteps. Depending on the FID sample size, this also might be super noisy.

References

  • [1] Hoogeboom, E., Heek, J., & Salimans, T. (2023, July). simple diffusion: End-to-end diffusion for high resolution images. In International Conference on Machine Learning (pp. 13213-13232). PMLR.
  • [2] Chen, T. (2023). On the importance of noise scheduling for diffusion models. arXiv preprint arXiv:2301.10972.
Questions for authors
  • Regarding W1: It would help to run static, resolution-shifted baselines at each resolution (SD3-style shift and a simple diffusion shift), as well as a μ\mu sweep on ImageNet-128 that includes μ<=0\mu <= 0. This would help to explain whether the gains stem from the proposed curriculum or are rather a good search of a timestep sampling strategy.
  • How would one determine the final hyper-parameters of the model? As the method seems to be quite sensitive to the specific choice, it would be helpful to see the generalization (e.g. show how CurFM performs with the same μ0\mu_0 on each dataset).
  • The text-to-image experiments use FID-1K (L1024). FID is strongly biased at small sample sizes, and the bias depends on the model, so rankings can flip. Did you consider using more samples or an unbiased metric such as KID?
  • Table 6: The batch size for the text-to-image experiment is 16 and performance is evaluated at 9k steps (approximately 1/4 of a single epoch), which for diffusion usually is far from being converged. How do image generations look after 9k steps? Furthermore, how is the baseline trained? Is it using the same Flow Matching formulation and which Logit-Normal timestep sampler (parameters)? Also, how is FID-1K obtained with an evaluation set of 500 text-image pairs?
  • It would also be interesting to see a visualization of qualitative samples per noise scale and for each training interval (e.g. how do images change with each adaptive step).
  • I further would appreciate a baseline with fixed, predefined decay of μ\mu. This would definitely strengthen the papers claim about the adaptive curriculum.
10 comments
AuthorsOpenReview ↗

Rebuttal (Part 1/4)

We sincerely thank the reviewers for their time and effort in reviewing our paper. We have carefully addressed each comment below and revised the corresponding sections of the manuscript, with all changes highlighted in blue. We would be happy to provide further clarification or answer any additional questions.

Response to W1 and Q1: Resolution-dependent timestep sampling.

We sincerely thank the reviewer for highlighting the importance of resolution-dependent timestep sampling. We agree that different resolutions may favor different noise distributions. However, resolution-dependent sampling and CurFM's training-dependent adaptation address complementary factors.

First, under our flow-matching formulation, SD3-style resolution shifts and Simple Diffusion-style SNR scaling, when applied to a logit-normal sampler, are mathematically equivalent to static adjustments of its mean μ\mu. Specifically, using μ=1.2\mu=1.2 at 32×3232\times32 as the reference, the corresponding resolution-shifted means are μ=1.2\mu=1.2, μ≈0.507\mu\approx0.507, and μ≈−0.186\mu\approx-0.186 for CIFAR-10, CelebA, and ImageNet-128, respectively. These configurations are either directly included or closely approximated by our extended static-μ\mu experiments below, providing empirical coverage of resolution-dependent sampling strategies.

Second, Figure 3 directly demonstrates the benefits of curriculum adaptation under fixed dataset and resolution. The preferred sampling mean, obtained from grid search, progressively changes from 1.1 to 0.9 and 0.8 across successive training stages. Moreover, the staged schedule outperforms static samplers with these values throughout training. Since resolution remains unchanged, these findings cannot be explained solely by resolution-dependent sampler selection.

Furthermore, following the reviewer's suggestion, we have extended the static-μ\mu sweep to all three datasets, including μ≤0\mu\leq0 on ImageNet-128, while keeping all other settings unchanged, using vv-prediction. The additional results are presented in Figure 12 of Appendix C.7, with numerical values provided below for convenience.

As shown, CurFM consistently achieves faster convergence and superior final FID across all three datasets compared with the evaluated static samplers. These results further demonstrate that CurFM's improvements cannot be explained solely by selecting a favorable static timestep distribution, highlighting the benefits of adapting the sampling distribution throughout training.

We thank the reviewer again for this valuable suggestion, which has helped strengthen our empirical evaluation.

CIFAR-10 (FID-50K)Epoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
μ=1.4\mu=1.44.933.122.742.512.41
μ=1.2\mu=1.23.742.782.662.562.23
μ=1.0\mu=1.04.452.892.502.332.31
μ=0.8\mu=0.84.652.992.592.442.31
μ=0.6\mu=0.65.193.192.752.542.40
μ=0.4\mu=0.45.933.542.972.682.59
μ=0.2\mu=0.26.533.913.232.922.82
CurFM3.572.592.292.232.19
CelebA (FID-50K)Epoch=100Epoch=300Epoch=500Epoch=700
μ=0.8\mu=0.83.042.232.082.06
μ=0.6\mu=0.62.822.151.961.99
μ=0.4\mu=0.42.862.121.891.95
μ=0.2\mu=0.22.722.041.841.96
μ=0.0\mu=0.02.802.161.952.02
CurFM2.722.011.811.80
ImageNet (FID-50K)Epoch=100Epoch=200Epoch=300Epoch=400Epoch=500
μ=1.2\mu=1.232.5822.0719.3916.8310.23
μ=1.0\mu=1.030.1519.2119.9517.3011.70
μ=0.8\mu=0.833.1220.8818.5617.7114.34
μ=0.6\mu=0.635.7724.0221.7820.6416.59
μ=0.0\mu=0.043.1026.7921.3422.1217.80
μ=−0.2\mu=-0.240.1229.0724.4724.6819.30
CurFM20.1217.5311.3610.089.24
AuthorsOpenReview ↗

Rebuttal (Part 2/4)

Response to W2 and Q2: Hyperparameter selection, generalization, and computational overhead.

We sincerely thank the reviewer for raising these concerns. We clarify that the scheduler of CurFM aims to eliminate expensive manual curriculum design, rather than all hyperparameter tuning.

(1) Regarding hyperparameter selection

The main scheduler hyperparameters, μˉ=0.05\bar{\mu}=0.05, Tˉ=2\bar{T}=2, and interval=20 are shared across all datasets, with sensitivity analyses provided in Figures 5 and 9. We adopt σ=1.2\sigma=1.2 for all three datasets following the official Flow Matching implementation as well as μ0=1.2\mu_0=1.2 for CIFAR-10. For ImageNet and CelebA, we select the initial μ0\mu_0 through a lightweight sweep over a few training epochs to get a good starting point. Importantly, unlike manually designed curricula, which require repeated training and parameter sweeps at successive stages, and is therefore impractical for deployment, CurFM automatically determines transition points within a single end-to-end training. Furthermore, following the reviewer's suggestion, we investigate CurFM's robustness to different initial sampling distributions using vv-prediction on CIFAR-10. The additional results are presented in Table 7 of Appendix C.8 in the revised manuscript and reproduced below for convenience. We evaluate three initial sampling means, μ0∈{1.2,0.6,0.2}\mu_0\in\{1.2,0.6,0.2\}, including μ0=0.2\mu_0=0.2, which is also used to initialize CurFM on CelebA. We observe that CurFM consistently outperforms the corresponding static FM baseline across all tested initializations and training checkpoints, demonstrating that its benefits are not restricted to a particular choice of μ0\mu_0. We acknowledge that the initial distribution will influence absolute performance, and should be carefully selected for both standard FM and CurFM. Nevertheless, we show that adaptive curriculum scheduling in CurFM provides consistent improvements regardless of the starting point.

FID-50KEpoch=200Epoch=400Epoch=600Epoch=800Epoch=1000
FM (static, μ=1.2\mu=1.2)3.742.782.662.562.23
CurFM (μ0=1.2\mu_0=1.2)3.572.592.292.232.19
FM (static, μ=0.6\mu=0.6)5.193.192.752.542.40
CurFM (μ0=0.6\mu_0=0.6)4.312.962.462.412.28
FM (static, μ=0.2\mu=0.2)6.533.913.232.922.82
CurFM (μ0=0.2\mu_0=0.2)5.493.802.932.802.70

(2) Regarding the FID threshold

We would like to clarify that δ=0.01\delta=0.01 serves merely as a small positive tolerance for detecting performance stagnation, rather than a parameter requiring extensive tuning. Importantly, CurFM monitors changes in FID within each curriculum stage under a fixed evaluation protocol, rather than comparing absolute FID values across different datasets or sample sizes. We use the same value for δ\delta across all experiments without dataset-specific adjustment. We apologize for the potential confusion and have revised the corresponding part in Section 4.2.

(3) Regarding computational overhead and scalability

CurFM performs FID evaluation only every 20 epochs using efficient one-step reconstruction, with the cost amortized over training iterations. As shown in Table 1, the overhead is only 6.2%, 2.6%, and 2.9% on CIFAR-10, CelebA, and ImageNet, respectively. Notably, the overhead remains below 3% on the more computationally demanding ImageNet benchmark, demonstrating that CurFM maintains low computational overhead at increased training scale.

Overall, the scheduler in CurFM provides an efficient alternative to manually tuning curriculum schedules while maintaining largely consistent hyperparameter settings across datasets. We thank the reviewer again for the valuable comment.

AuthorsOpenReview ↗

Rebuttal (Part 3/4)

Response to W3: Loss fluctuations and reliability of the curriculum signal.

We sincerely thank the reviewer for this insightful observation. First, we clarify that Figure 1 reports epoch-averaged training loss under a fixed timestep distribution, rather than individual mini-batch losses. Although stochastic timestep sampling may contribute to variability, its influence is substantially reduced through epoch-level averaging. More importantly, our central observation is the misalignment between training-loss convergence and generation quality: the training loss continues to decrease and fluctuate even after FID has largely converged.

Second, we agree that raw training losses are not directly comparable across curriculum stages with different timestep distributions. CurFM avoids this issue by evaluating performance stagnation within each stage, where the sampling distribution remains fixed, and resetting the monitoring process after each curriculum transition (Equations 11–14).

Finally, regarding FID reliability, we clarify that although one-step reconstruction FID is evaluated over multiple noise levels, the timestep distribution remains fixed within each curriculum stage. Therefore, successive FID evaluations measure generation quality under the same noise distribution, ensuring comparability when assessing performance stagnation. While finite-sample variability may exist, sampling from a mixture of timesteps does not inherently imply an unstable evaluation signal. For the main experiment, we use FID-50K as the curriculum signal, which is standard practice, while for the supplement text-to-image experiment in Table 6, we use FID-1K mainly due to computational efficiency. Furthermore, our ablation in Figure 6 (variant loss-based) empirically supports the effectiveness of FID-based scheduling over loss-based scheduling.

Response to Q3: FID-1K in text-to-image experiment.

We sincerely thank the reviewer for this insightful suggestion. We adopt FID-1K instead of FID-50K in the supplementary text-to-image experiments primarily for computational efficiency, as FID-1K is used solely as a curriculum scheduling signal rather than the final evaluation metric. We acknowledge that FID estimates with limited samples may exhibit potential statistical bias. Nevertheless, Table 6 demonstrates that CurFM, even when guided by FID-1K, consistently outperforms standard FM under both CLIP-Text and CLIP-Image metrics, which are independent of the curriculum scheduling criterion. We have revised Appendix C.5 to further clarify this point. Thanks again for your valuable suggestion.

Response to Q4: Text-to-image generation experiment.

We sincerely thank the reviewer for these insightful questions. We clarify that the text-to-image experiment in Appendix C.5 serves as a lightweight proof of concept to investigate CurFM's generalizability to more challenging conditional generation tasks, complementing our comprehensive experiments on CIFAR-10, CelebA, and ImageNet-128, which demonstrate consistently faster convergence and superior or comparable final FID. Due to computational constraints, the text-to-image models are not trained to full convergence. Nevertheless, Table 6 shows that CurFM consistently outperforms standard FM in both CLIP-Text and CLIP-Image scores across all evaluated training stages. To further assess general generation quality and convergence for the text-to-image experiment, we additionally report FID-50K at different training steps, using real-image reference statistics from the training set. As shown below, FID decreases consistently for both methods, while CurFM achieves substantially lower FID at every checkpoint (e.g., 39.0 vs. 52.4 at 9K steps), demonstrating faster convergence and consistent improvements in generation quality even during early training. We have revised Appendix C.5 accordingly.

FID-50Kstep=3000step=6000step=9000
Baseline(FM)71.360.352.4
CurFM62.948.739.0

Regarding implementation details, the baseline employs the same standard Flow Matching formulation as CurFM, with a fixed Logit-Normal timestep distribution (μ=1.2\mu=1.2, σ=1.2\sigma=1.2). This distribution also serves as the initial timestep distribution for CurFM, consistent with our main experiments. The FID-1K used for curriculum scheduling is computed from 1,000 one-step reconstructed samples, using real-image reference statistics obtained from subsets of the training set. This evaluation is independent of the 500 text-image pairs reserved for CLIP-based evaluation. We have revised the corresponding section in Appendix C.5 to avoid potential confusions. Thanks again for your valuable suggestion!

Reviewer_P78cOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CurFM, a curriculum-based adaptive timestep sampling strategy for training flow-based generative models. The authors reinterpret flow-matching objectives from a denoising perspective and argue that noise levels can serve as a proxy for the difficulty, where higher noise levels correspond to more challenging denoising tasks. They further provide a theoretical analysis under a linear denoising setting to support this interpretation. Building on this perspective, CurFM progressively shifts the timestep sampling distribution from easier (low-noise) to harder (high-noise) regions as training proceeds. Specifically, the method adaptively adjusts the mean of a logit-normal timestep distribution based on the stagnation of one-step reconstruction FID, rather than relying on potentially unstable training losses. Experiments on CIFAR-10, CelebA, and ImageNet demonstrate faster convergence and competitive or improved generation quality compared to existing timestep sampling methods under both velocity and noise prediction parameterizations.

Critical strengths

1. Simple and Practical Adaptive Sampling Strategy

The proposed method is simple and easy to integrate into existing flow-matching training pipelines. CurFM only modifies the timestep sampling distribution without requiring changes to the model architecture or inference procedure. Moreover, the adaptive scheduling mechanism introduces relatively small computational overhead, making it a practical approach for improving training efficiency.

2. Consistent Empirical Improvements Across Different Settings

The paper provides extensive experimental evaluations across multiple datasets, including CIFAR-10, CelebA, and ImageNet, under both velocity and noise prediction parameterizations. CurFM demonstrates faster convergence and competitive or improved generation quality compared to existing timestep sampling strategies. The additional experiments on conditional generation further support the applicability of the proposed approach (even if there are similar works).

Critical weaknesses

1. Additional Discussion on Related Work

The paper would benefit from a more comprehensive discussion of prior studies on timestep-dependent training in diffusion models. For example, P2 weighting [1] investigates the perceptual importance of different noise levels, while ANT [2] analyzes negative transfer and task conflicts between denoising tasks across timesteps. More recently, SpeeD [3] explores the varying contributions of timesteps to training efficiency and proposes asymmetric timestep sampling and weighting strategies. Since these studies are closely related to the motivation behind CurFM, discussing how the proposed difficulty-based curriculum relates to their findings would help better contextualize the contribution of this work. And, some parts of observations are largely overlapped with these papers.

2. Insufficient Differentiation from Prior Curriculum Learning Methods

DTDC has already investigated the varying difficulties of denoising tasks across timesteps and proposed an easy-to-hard curriculum to improve generation quality and training convergence. Importantly, DTDC also introduces an adaptive pacing strategy that determines curriculum transitions based on training-loss stagnation. Therefore, the central idea of adapting curriculum progression according to the model's learning progress is not entirely new. The main distinction of CurFM appears to lie in its use of one-step reconstruction FID as the transition criterion and its continuous adjustment of the timestep sampling distribution. However, the paper does not sufficiently establish how much these specific design choices contribute beyond the existing adaptive curriculum framework.

Furthermore, the two methods appear to adopt different interpretations of denoising difficulty. DTDC considers higher-noise denoising tasks relatively easier based on convergence behavior and distributional analysis, whereas CurFM considers higher-noise samples more difficult based on clean-image reconstruction error. Although these differences may arise from their respective prediction objectives and difficulty definitions, the paper does not adequately reconcile these seemingly conflicting observations. In particular, the characterization of DTDC's difficulty ordering in Appendix B.2 would benefit from clarification, as the timestep conventions differ between the two formulations.

Given the substantial conceptual overlap, a more careful comparison with DTDC, particularly regarding the definition of difficulty, curriculum progression, and adaptive transition criteria, is necessary to establish the novelty and theoretical motivation of CurFM.

References

[1] "Perception Prioritized Training of Diffusion Models," CVPR, 2022.

[2] "Addressing Negative Transfer in Diffusion Models," NeurIPS, 2023.

[3] "A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training," CVPR, 2025.

[4] "Denoising Task Difficulty-based Curriculum for Training Diffusion Models," ICLR, 2025.

Questions for authors

Q1. Relationship to DTDC and the Definition of Denoising Difficulty

DTDC (ICLR 2025) also proposes an easy-to-hard curriculum with adaptive pacing based on training-loss stagnation. However, DTDC and CurFM appear to adopt different interpretations of denoising difficulty, particularly regarding whether high-noise timesteps correspond to easier or harder tasks.

Could the authors clarify how these seemingly conflicting difficulty orderings can be reconciled, considering differences in prediction targets, timestep conventions, and difficulty definitions? Furthermore, since both methods employ adaptive curriculum scheduling, what is the key advantage of CurFM's FID-based transition criterion over the loss-based adaptive pacing used in DTDC?

4 comments
AuthorsOpenReview ↗

Rebuttal (Part 1/2)

We sincerely thank the reviewers for their time and effort in reviewing our paper. We have carefully addressed each comment below and revised the corresponding sections of the manuscript, with all changes highlighted in blue. We would be happy to provide further clarification or answer any additional questions.

Response to W1: Additional discussion of related timestep-dependent training methods.

We sincerely thank the reviewer for suggesting these relevant studies. We agree that P2, ANT, and SpeeD provide important insights into the heterogeneous roles of timesteps in diffusion training, and we have expanded our related-work discussion (Section 2 in the main paper) to better acknowledge their contributions.

Specifically, P2 investigates the perceptual importance of different noise levels and introduces SNR-based loss weighting to prioritize perceptually meaningful denoising tasks. ANT studies diffusion training from a multi-task learning perspective, revealing negative transfer between timestep-specific tasks and mitigating task conflicts through clustering. SpeeD analyzes the varying contributions of timesteps through process increments and proposes asymmetric sampling and weighting to improve training efficiency.

These studies share our broader observation that timesteps should not necessarily receive uniform training emphasis. However, they investigate different properties of timestep-specific tasks and address different aspects of training. P2 focuses on perceptual importance, ANT on task interference, and SpeeD on training efficiency. In contrast, CurFM investigates timestep-dependent denoising difficulty from a curriculum learning perspective and, more importantly, how the allocation of training emphasis should evolve with the model's learning progress.

Unlike the prescribed timestep weighting or sampling strategies in these works, CurFM adaptively updates the timestep distribution based on generation-level performance. Our stage-wise experiments in Figure 3 and Figure 11 further show that the preferred timestep distribution changes throughout training, motivating training-dependent curriculum adaptation rather than a fixed sampling strategy. This observation constitutes a key distinction from these existing approaches.

We have revised the related-work section to better clarify these connections and more precisely distinguish our contribution from prior timestep-dependent training studies.

AuthorsOpenReview ↗

Rebuttal (Part 2/2)

Response to W2 and Q1: Relationship to DTDC, difficulty ordering, and adaptive curriculum scheduling.

We sincerely thank the reviewer for highlighting the relationship between CurFM and DTDC. We acknowledge that both methods employ dynamic timestep sampling over diffusion/flow timesteps. However, they differ in their theoretical motivation, curriculum construction, and transition criterion. We clarify these distinctions below.

(1) Reconciling the opposite difficulty orderings.

First, we would like to clarify that CurFM's difficulty definition is not introduced specifically to contradict DTDC. Instead, it follows the ideal difficulty formulation established in prior Curriculum Learning theory work (see Definition 1 in Weinshall et al., ICML 2018 [1]), which theoretically studies the convergence benefits of easy-to-hard learning under this specific difficulty definition. Built upon this, our theoretical analysis establishes the relationship between noise level and this difficulty measure under the considered denoising formulation, identifying higher-noise samples as more difficult.

In contrast, DTDC characterizes difficulty through empirical convergence behavior and relative entropy between consecutive timestep distributions, identifying higher-noise denoising tasks as relatively easier. These two definitions characterize different properties of denoising tasks and can therefore yield opposite orderings. Specifically, faster convergence on individual timestep tasks does not establish that prioritizing those tasks will yield faster convergence of the overall generative model. And more importantly, the resulting curriculum direction is coupled with each method's scheduling mechanism: DTDC uses loss-based pacing over timestep clusters, whereas CurFM adaptively shifts the sampling distribution according to generation-level progress. Consequently, the preferred ordering need not be universal across the two frameworks. Indeed, our experiments show that CurFM achieves faster overall convergence than DTDC and other baselines despite adopting the opposite ordering.

We also acknowledge the difference in timestep conventions: smaller timesteps correspond to higher noise in CurFM but lower noise in DTDC. We have corrected and clarified the description in Appendix B.2.

(2) Distinction in curriculum construction.

While both methods adopt adaptive easy-to-hard scheduling, DTDC partitions timesteps into discrete clusters and progressively incorporates additional clusters during training. CurFM instead continuously shifts a logit-normal timestep distribution toward higher-noise regions while retaining sampling coverage across the noise spectrum. Consequently, CurFM does not require discrete task partitioning and provides a smooth adjustment of training emphasis.

(3) Advantage of FID-based adaptive transitions.

The primary motivation for our FID-based criterion is the potential mismatch between training-loss convergence and generation quality. As illustrated in Figure 1, FID can reach convergence while the training loss continues to decrease and fluctuate, making loss stagnation an imperfect indicator of generation-level progress. CurFM therefore evaluates one-step reconstruction FID to determine when the model is ready to advance its curriculum.

Importantly, we have investigated these design choices in the ablation study of Figure 6. Specifically, we consider two variants: (1) loss-based, which replaces our FID-based transition criterion with training-loss stagnation while retaining the proposed curriculum direction; and (2) hard-to-easy, which reverses the curriculum direction, matching DTDC's high-noise-to-low-noise progression while retaining our adaptive transition mechanism. Both variants consistently underperform CurFM, providing empirical evidence for the effectiveness of our FID-based transition criterion and curriculum direction, respectively. These results demonstrate that CurFM's improvements arise not merely from adopting an adaptive curriculum framework, but also from its specific curriculum direction and transition criterion.

We have revised Appendix B.2 to provide a more thorough comparison with DTDC. We thank the reviewer again for the valuable comments and would be happy to address any further questions.

References

[1] Curriculum Learning by Transfer Learning: Theory and Experiments with Deep Networks, ICML 2018

Thanks the authors for the thoughtful rebuttal and manuscript revisions.

The expanded discussion of related timestep-dependent methods (P2, ANT, SpeeD), the correction of timestep conventions in Appendix B.2, and the explanation of how CurFM's difficulty formulation and scheduling mechanism differ from DTDC have largely addressed our main concerns.

I would like to increase my rating since I don't find the issue. However, I will decide this after the other reviewers respond.

We sincerely thank the reviewer for the prompt response, positive feedback, and encouraging comments. We are glad that our rebuttal and manuscript revisions have addressed your main concerns. We greatly appreciate your careful consideration and would be happy to provide any further clarification if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.