acceptodds
Under review as a conference paper at ICLR 2027

Timestep Weighting Through the Lens of Flow Matching Dynamics

Abstract

Flow matching learns generative models by predicting velocity fields along a predefined path between a prior and a data distribution. The design of the training objective, in particular the weighting across timesteps, is known to have a strong impact on performance, yet further insight into the underlying generation dynamics is needed to guide its design. In this work, we study marginal flow dynamics through the propagation of velocity prediction errors to the generated endpoint. For Gaussian mixture targets, we derive a decomposition of the velocity Jacobian into a within-component term and a term governed by uncertainty over component assignments, and use this structure to approximate timestep-dependent error amplification. We complement the analysis with perturbation probes that measure endpoint sensitivity along model prediction-error directions. Our analysis and observations suggest a transition from an earlier regime in which component ambiguity contributes to error amplification to a later regime in which this contribution diminishes and within-component dynamics dominate. This transition motivates allocating training emphasis asymmetrically across the generation process. Building on logit-normal weighting, we introduce a parameterized refinement that reduces the relative emphasis on the later regime while retaining endpoint suppression. Experiments demonstrate improved generation quality over the logit-normal baseline across multiple image datasets and all tested model sizes.

Primary area:
generative models
Keywords:
Generative Models, Flow Matching, Path-Dependent Weigthing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZeDfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper models the data distribution and model dynamics using a GMM to analyze how velocity errors at different timesteps affect the final generated images. Based on this analysis, the authors propose a time-decaying weighting scheme to guide FM training and demonstrate improvements across different models and datasets.

Critical strengths

The paper provides a GMM-based perspective for analyzing flow dynamics and investigating how early-stage velocity prediction errors affect the final generated images. The timestep weighting problem studied in this paper is meaningful and plays an important role in Flow Matching training. Moreover, the proposed time-decaying weighting strategy achieves consistent performance improvements across different model scales and datasets.

Critical weaknesses

1. Concerns about the consistency between GMM reference dynamics and the actual model dynamics.

The paper primarily studies endpoint sensitivity using the reference velocity field derived from a GMM. Even in Figure 2, perturbations are propagated using the GMM reference velocity rather than the trained SiT model. The authors do not provide sufficient evidence that these two dynamics exhibit consistent error propagation behavior. In particular, the model's own generation process may correct or attenuate early-stage perturbations. Why not directly use the trained model for subsequent integration to analyze its actual endpoint sensitivity?

2. Insufficient connection between endpoint sensitivity and generation quality.

Previous studies have shown that the high-noise stage mainly determines the global structure, while the low-noise stage primarily refines details. Therefore, it is somewhat intuitive that early-stage perturbations may have a larger impact on the final generated image. However, a larger endpoint deviation does not necessarily imply worse generation quality. For example, a perturbation may simply shift the generation result toward another valid data mode. The paper lacks direct evidence showing how endpoint sensitivity relates to final generation quality and why assigning larger training weights to earlier timesteps is justified.

3. Missing comparison with a simple shifted logit-normal baseline.

Directly adjusting the mean of the logit-normal distribution can also increase the relative training emphasis on earlier timesteps, which is a simple and commonly used approach. However, the paper does not include this baseline. Consequently, it is difficult to determine whether the proposed sensitivity-guided attenuation provides advantages over a straightforward adjustment of the timestep distribution. The necessity of the specific weighting design therefore remains unclear.

Questions for authors

Q1. Generalization of GMM-based sensitivity analysis.

Have the authors considered directly measuring endpoint sensitivity using the learned SiT velocity field instead of the GMM reference dynamics? Would the early-stage error amplification observed in Figure 2 still hold when perturbations are propagated through the trained model itself?

Q2. Connection between endpoint sensitivity and generation quality.

How does the measured endpoint sensitivity relate to actual generation quality? Since a larger endpoint deviation does not necessarily imply a lower-quality sample, could the authors provide further evidence that reducing errors at high-sensitivity timesteps leads to better generation quality?

Q3. Comparison with shifted logit-normal weighting.

Could the authors compare the proposed weighting strategy with a simple shifted logit-normal distribution, where the mean parameter is adjusted to emphasize earlier timesteps? This would help clarify whether the proposed sensitivity-guided attenuation provides benefits beyond a straightforward adjustment of the timestep distribution.

2 comments

We sincerely thank you for your thoughtful and constructive feedback, and for recognizing the significance and clarity of our work. Below, we provide point-by-point responses to your questions and concerns.

  1. W1, Q1: Generalization of GMM-based sensitivity analysis.

    Response. As described in Section 3.4, we use the same GMM for the dynamics analysis and the perturbation probes. Each data point is the mean of an isotropic Gaussian component, whose variance is set to the average posterior variance of the VAE outputs. The component variance is about 10−410^{-4}, while the latent data are on a scale of about 11. This motivates using narrow Gaussian components to approximate the target distribution. To further assess how well the GMM approximates the target distribution, we trained two SiT-B models for 250K steps, one on ImageNet latent codes and the other on samples from our GMM approximation. The model trained on the GMM achieved an FID of 35.7935.79, compared with 28.2128.21 for the model trained on ImageNet latents. The FIDs are comparable in scale, suggesting that the GMM retains structure in the target distribution that is relevant to generation.

    Velocity prediction errors can accumulate during generation and affect the endpoint, making it harder to isolate the effect of a single local perturbation. In the original probe, we therefore use the GMM reference velocity field to propagate the perturbation to the endpoint and measure its effect under the reference dynamics. To test whether the sensitivity pattern persists under the learned dynamics, we propagated perturbations to the endpoint using the trained SiT velocity field. Figure S3 shows that sensitivity drops near the GMM-selected onset a=0.6a=0.6 and approaches the within-component reference at later timesteps. This supports qualitative agreement between the learned and GMM reference dynamics. We report this experiment in the revised appendix.

  2. W2, Q2: Connection between endpoint sensitivity and generation quality.

    Response. We thank the reviewer for this insightful question. The training objective in flow matching is to approximate a reference flow field. Its connection to generation quality rests on the premise that a learned flow closer to the reference flow produces endpoints closer to the reference endpoints and is expected to improve overall generation quality. This motivates our focus on endpoint sensitivity and on reducing endpoint displacement.

    Under the flow matching ODE, there is no theoretical guarantee that a displaced endpoint will land in another valid data mode. We therefore cannot assume this in general when analyzing the flow matching model. There are more evidences illustrate reducing the endpoint displacement improves the generation quality. First, Tables 1, 4, and 5 show that the sensitivity-guided refinement improves generation quality across weighting schemes and datasets. Second, Figure S2 shows that FID is sensitive to the choice of attenuation onset aa, which we determine from the endpoint sensitivity analysis. Finally, Figure S4 shows that the refinement reduces endpoint displacement, as expected from its design. Together with the generation results, this provides empirical support for the connection between endpoint sensitivity and generation quality.

  3. W3, Q3: Comparison with shifted logit-normal weighting.

    Response: We adopt the standard logit-normal setting that Esser et al. (2024) selected through a hyperparameter sweep. Under matched training and evaluation conditions, we also evaluate several earlier-shifted logit-normal schedules with and without refinement. The table below summarizes the results. The best earlier-shifted setting (ES-LN) yields FID and IS values comparable to those of the standard logit-normal setting (LN). The refinement improves performance with both schedules under matched conditions. Therefore, tuning the shift alone falls short of these gains, demonstrating the necessity of the refinement design.

    WeightingFID ↓IS ↑
    ES-LN22.7429.53
    LN23.0431.75
    ES-LN + Refinement (ours)21.6831.21
    LN + Refinement (ours)21.7632.22

References

  • Esser et al. (2024). Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv:2403.03206.
Reviewer_ZeDfOpenReview ↗

Response to Authors

Thank you for the response. Figure S3 addresses my concern about the GMM dynamics, and the shifted logit-normal comparison answers W3.

However, W2 remains unresolved. The paper's logic relies on larger endpoint displacement implying worse generation quality, but this link is not established. The FID gains show that the weighting works, not why it works. Figure S4 shows only a small difference, and the very large early sensitivity in Figure S3 may simply reflect switching to another valid mode. Without direct evidence connecting early-stage errors to generation quality, the motivation for the weighting does not hold. I will therefore maintain my rating.

Reviewer_wWWtOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies marginal flow dynamics through the propagation of velocity prediction errors to the generated endpoint. Specifically, the paper analyzes the dynamics of the reference marginal flow under a Gaussian mixture model, identifying two factors that govern time step-dependent endpoint sensitivity. Furthermore, the paper presents the probing endpoint sensitivity implementation along model prediction-error directions. Based on the probing analysis, the paper proposes an asymmetric refinement of existing timestep weightings. Finally, the paper experiments on several datasets to show the effectiveness of the proposed method.

Critical strengths
  1. The paper is organized with direct motives.

  2. The perspective of using endpoint sensitivity to guide the timestep weight refinement is novel.

Critical weaknesses
  1. The clarity and coherence of the presentation needs further improvement to increase the informativeness of the paper. Section 3 gives a closed-form endpoint sensitivity analysis, but it only applies to Gaussian mixture models. One of the main understanding of this section is that “the temporal transition of component ambiguity depends on the spread of the target distribution”, but this is not directly or clearly applied in the sensitivity guided weighting refinement. The reason of choosing the exact formula choice of Equation (15) is relatively ambiguous and heuristic, with only using one parameter aa obtained from Section 4, reducing the credibility of the proposed method. It would be more informative and coherent if we can obtain an overall integrated modeling (maybe using mixture gaussian model) of Equation (15) using more original information from the target distribution like “the spread”.

  2. For experiments, I would suggest moving Section 5.1 out of Section 5 and make it a new section. For all the tables, it would be beneficial to add the confidence intervals for the numerical results.

Questions for authors
  1. Why do the paper choose this specific function form of Equation (15)? Will the experimental result essentially change if the function form of Equation (15) change? I believe this is an important issue needs to be clarified.

  2. Is it feasible to present some theoretical analysis for the effect of the sensitivity guided weighting refinement? An analysis based on GMM model would already be beneficial if the general analysis is relatively hard to establish.

  3. Does the propose method also improves the performance of other timestep weightings beyond the logit-normal weighting?

2 comments

We sincerely thank you for your thoughtful and constructive feedback, and for recognizing the significance and clarity of our work. Below, we provide point-by-point responses to your questions and concerns.

  1. W1: The coherence between the dynamic analysis and the refinement design

    Response. The target spread enters Eq. (15) through the attenuation onset aa. Figure 1(c) shows that increasing the spread of the GMM component centers shifts the decay of the component-ambiguity factor in Eq. (12) to earlier timesteps. We set aa to the first evaluated timestep at which this factor falls below 1+10−41+10^{-4}. Eq. (15) then shifts a larger share of the training weight toward the interval before aa, where component ambiguity contributes to endpoint-error amplification, while preserving the relative weights within that interval.

    The same rule gives a=0.4a=0.4, 0.50.5, and 0.60.6 for CIFAR-10, Food-101, and ImageNet, respectively. Table 5 evaluates the same three candidate onsets on each dataset, and the onset selected by the GMM analysis gives the lowest observed FID in each case. This supports using the ambiguity transition to determine where attenuation begins. Figure S2 shows that FID is sensitive to the attenuation onset and that the refinement improves FID across the tested values of bb at the selected onset. These results further demonstrate the importance of using the dynamics analysis to guide the refinement.

    More broadly, the dynamics analysis identifies two dataset-dependent factors that influence endpoint displacement. We use the decomposition and perturbation probes to assess their contributions. These approaches help explain how the structure of the target distribution shapes the downstream effects of prediction errors and provide a reusable tool for analyzing flow matching models. This perspective sheds light on both the interpretation of flow dynamics and the design of training objectives. As one application, we use the point at which component ambiguity contributes little to error amplification to decide when to start downweighting later timesteps. The resulting refinement yields consistent improvements across existing weighting schemes and multiple datasets.

  2. W2: For experiments, I would suggest moving Section 5.1 out of Section 5 and make it a new section. For all the tables, it would be beneficial to add the confidence intervals for the numerical results.

    Response. In the revision, we move Section 5.1 into a standalone section and add confidence intervals for the numerical results.

  3. Q1: Why do the paper choose this specific function form of Equation (15)? Will the experimental result essentially change if the function form of Equation (15) change? I believe this is an important issue needs to be clarified.

    Response. We thank the reviewer for pointing out that the choice of attenuation profile needs further explanation. To assess the choice of decay function in Eq. (15), we evaluate a range of attenuation profiles that differ in when and how sharply the weight falls after the onset aa. The sweep covers profiles ranging from an abrupt piecewise drop just after aa to a sharp drop near the endpoint. It also includes linear decay and other intermediate profiles. The tested profiles yield FIDs from 33.9133.91 to 37.9037.90, compared with 38.4538.45 for the unrefined baseline under the same settings. We adopt the profile with the lowest FID. The results show that the refinement guided by component ambiguity is generally effective across the tested profiles. We provide the candidate profiles, evaluation settings, and full results in the revised appendix.

  4. Q2: Is it feasible to present some theoretical analysis for the effect of the sensitivity guided weighting refinement? An analysis based on GMM model would already be beneficial if the general analysis is relatively hard to establish.

    Response. To examine the effect of the refinement on error propagation, we compare two SiT-B checkpoints trained for 150K steps, one with refinement and one without. We measure endpoint displacement using the same probing procedure described in Section 4. The refinement reduces the normalized RMS endpoint displacement, averaged uniformly over the evaluated time bins, by 11.91%11.91\%. The reduction in endpoint displacement is consistent with the rationale for the refinement. We provide the comparison figure and measurement details in the revised appendix.

  5. Q3: Does the propose method also improves the performance of other timestep weightings beyond the logit-normal weighting?

    Response. We also apply the refinement to constant and CosMap weighting. Table 4 shows that refinement improves FID with both base weightings. Its benefit therefore extends beyond the logit-normal base.

Thanks the authors for their responses. However, I believe the choice of the specific function form and the generalization and the credibility of the sensitivity analysis are still ambiguous. Therefore, I would like to keep my score.

Reviewer_4UsEOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how velocity prediction errors at different timesteps influence the final samples of flow-matching models. Based on this observation, the authors identify stronger early-stage endpoint sensitivity and introduce an asymmetric refinement of timestep weighting, improving image generation.

Critical strengths
  • The GMM-based Jacobian decomposition provides an interpretable distinction between within-component dynamics and component ambiguity.
  • The weighting refinement is straightforward to implement and does not require architectural modifications or additional training-time Jacobian computation.
Critical weaknesses
  • Timestep reweighting and early-navigation versus late-refinement dynamics have been extensively studied. The proposed adjustment is incremental and lacks novelty.
  • ImageNet FID reductions are consistent but relatively small, with approximately 4–6% relative improvements over logit-normal weighting.
  • The paper does not compare against adequately tuned or earlier-shifted logit-normal weighting, leaving the refinement's necessity unclear.
  • Sensitivity analysis guides onset a, but attenuation strength b=0.3 is selected empirically without a reported quantitative performance sweep.
  • On ImageNet, varying a changes FID by 3.80, exceeding the reported 1.53 improvement over the logit-normal baseline, raising robustness concerns.
Questions for authors

See Weaknesses and below:

  • How does the method compare with mean-shifted logit-normal, piecewise weighting, and simple early-timestep upweighting under matched training conditions?
  • Can the authors report FID across multiple b values and provide an a x b performance grid?
2 comments

We sincerely thank you for your thoughtful and constructive feedback.

  1. Response to W1: Our analysis quantifies how local velocity prediction errors at different timesteps propagate to the generated endpoint, and identifies how uncertainty over component assignments contributes to error amplification along the remaining flow. In previous works, Liu et al. (2026) characterize early navigation and late refinement through oracle velocities, and Ren et al. (2026) use the timing of coarse structure and fine detail formation to design frequency-aware conditioning. Our analysis addresses a distinct question by quantifying how velocity prediction errors affect the generated endpoint. The decomposition and perturbation probes provide a reusable tool for analyzing flow matching models and help explain how the structure of the target distribution shapes the downstream effects of prediction errors. This perspective sheds light on both the interpretation of flow dynamics and the design of training objectives.

    Timestep reweighting in flow matching models remains underexplored as well. As discussed in Section 1, most weighting schemes, such as logit-normal weighting and CSFlow, are based on empirical design choices. In addition, loss weighting schemes proposed for diffusion models, such as Min-SNR and EDM, have fallen short of expectations when applied to flow matching models. The principles underlying the design of timestep weighting in flow matching therefore merit further investigation.

  2. Response to W2: Figure 4 and Table 1 in the paper are to show that the refinement improves FID at most training checkpoints and across model sizes. To further assess the improvement, we extend SiT-XL training to 700K steps. At that point, the refined model achieves an FID of 6.87 and the logit-normal baseline achieves 7.41. The gap continues to widen, reaching 0.540.54. The baseline requires more than 200K additional steps to reach a comparable FID, corresponding to a 28%28\% convergence speedup.

  3. Response to W3: We adopt the standard logit-normal setting that Esser et al. (2024) selected through a hyperparameter sweep. Under matched training and evaluation conditions, we also evaluate several earlier-shifted logit-normal schedules with and without refinement. The table below summarizes the results. The best earlier-shifted setting (ES-LN) yields FID and IS values comparable to those of the standard logit-normal setting (LN). The refinement improves performance with both schedules under matched conditions. Therefore, tuning the shift alone falls short of these gains, demonstrating the necessity of the refinement design. |Weighting|FID ↓| IS ↑| |:-|-:|-:| | ES-LN|22.74|29.53| | LN|23.04 |31.75| | ES-LN + Refinement|21.68|31.21| | LN + Refinement|21.76|32.22|

  4. Response to W4, Q2: We swept aa and bb on ImageNet with SiT-B. At the selected onset a=0.6a=0.6, all nine tested bb values improved on the unrefined baseline of 38.45 (b=1)(b=1), with FIDs of 33.91–38.21. The paper’s choice, b=0.3b=0.3, performed best. Most settings with a≠0.6a\ne0.6 performed worse than the baseline. See the revised appendix for the full grid.

  5. Response to W5: We interpret the sensitivity to aa in light of the strong connection between component ambiguity and training performance. It shows that FID varies with aa in a consistent pattern across datasets. Table 5 shows that this computed onset achieves the lowest FID among the tested values on all three datasets. The agreement across datasets, together with the sensitivity to aa, supports our interpretation that aligning attenuation with the component ambiguity transition matters for generation quality. The a×ba \times b grid in the Figure S2 also shows that, with aa fixed at the computed onset, refinement consistently improves FID across the tested values of bb. This demonstrates robustness of the refinement design at the computed onset.

  6. Response to Q1: The earlier-shifted logit-normal weighting is reported in response to W3. To assess the choice of decay function in Eq. (15), we evaluated a range of attenuation profiles that differ in when and how sharply the weight falls after the onset aa. The sweep ranged from an abrupt piecewise drop just after aa to a sharp drop near the endpoint, with decay equivalent to simple early-timestep upweighting and many other intermediate profiles in between. The piecewise profile achieved an FID of 34.88, and the profile in Eq. (15) achieved the lowest FID of 33.91. The unrefined baseline yielded an FID of 38.45 under the same settings. We provide the full results in the revised appendix.

References

  • Liu et al. (2026). From Navigation to Refinement: Revealing the Two-Stage Nature of Flow-based Diffusion Models through Oracle Velocity.
  • Ren et al. (2026). Frequency-Aware Flow Matching for High-Quality Image Generation.
  • Esser et al. (2024). Scaling Rectified Flow Transformers for High-Resolution Image Synthesis.

Thank you for the rebuttal. I appreciate the additional experiments on shifted logit-normal weighting and hyperparameter sweeps, which address my biggest concerns. However, I still find the methodological novelty limited. I will raise my score from 1 to 2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.