acceptodds
Under review as a conference paper at ICLR 2027

Velocity Scaling in Flow Matching

Abstract

Scaling a learned flow-matching velocity field by a gain was recently shown to greatly improve generation quality. Prior work argued that velocity fields trained with mean-squared error (MSE) systematically underestimate velocity magnitude and that scaling corrects this error. We show that MSE training does not create a velocity-magnitude deficit. We find instead that velocity scaling reduces *population time lag*: sampled states at model time $t$ resemble training states from an earlier time. Velocity scaling and moving model time back are two ways to address this population time lag. Across architectures and model sizes, measuring population time lag and using it to select a gain greatly improves generation quality, reducing FID from 28.0 to 12.2 on ImageNet-256 at NFE 25 without guidance.

Primary area:
generative models
Keywords:
flow matching, exposure bias, image generation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Xw7QOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies an empirical observation in Flow Matching: scaling the predicted velocity during sampling often improves generation quality. Previous work attributed this improvement to MSE training underestimating the velocity magnitude, but the authors show both theoretically and empirically that MSE does not necessarily create such a velocity deficit. Instead, they identify state–time mismatch as a key mechanism, where sampled states at a given model time resemble training states from an earlier time. They introduce a probe to measure this mismatch and show that both shifting the model time backward and scaling the velocity can reduce it.

Critical strengths
  1. The paper provides a solid theoretical analysis showing that MSE training does not cause a systematic velocity deficit, and identifies state–time mismatch as a new explanation for why velocity scaling improves generation.
  2. The proposed explanation is supported by careful and well-designed experiments. The authors validate the time probe using along-path and perpendicular perturbations, show that model-time correction improves FID without scaling the velocity, and use frozen-field and radial controls to isolate the role of the changed-state response. The results are also tested across multiple SiT sizes, REPA-SiT, an ADM U-Net, CIFAR-10, different NFEs, and different solvers. These experiments make the proposed explanation much more convincing than a correlation-only argument.
Critical weaknesses

None.

Questions for authors
  1. Can velocity scaling and model-time correction be applied jointly? For example, after applying a lag-based velocity gain, one could measure the remaining time lag and adjust the model-time input accordingly. Have the authors tested whether this combined correction provides additional improvement over either method alone?
  2. Although the paper provides extensive experiments on earlier flow-matching image models such as SiT, do the authors expect state–time mismatch to also exist in more recent and stronger flow-matching models, for both image and video generation? If so, would velocity scaling still be effective in these settings? A discussion of the authors’ view would be sufficient, while any preliminary experimental evidence, if available, would be even more helpful.
Reviewer_5iRdOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies why slightly increasing the output velocity of an already trained model can substantially improve generation. The prior Velocity Deficit work attributes this benefit to MSE-induced magnitude contraction causing insufficient progress during generation, and proposes methods such as SSC. The present paper challenges this theoretical attribution: the population MSE velocity field can transport the distribution correctly, and the tested network has a near-unit MSE-optimal calibration gain on genuine training-path states.

The paper connects the benefits of scaling to population time lag: states produced by coarse sampling can lag behind the time supplied to the model. It examines two ways to address this mismatch: increasing the velocity to advance the state, as already done by SSC, or adjusting the model’s time input to better match the current state. Experiments across models and sampling settings, together with time-correction and trajectory controls, provide support for this explanation.

The paper also proposes selecting the gain using measured lag rather than FID. This improves generation in the tested ImageNet settings, although it generally falls short of gains selected directly through an FID search. The authors acknowledge that lag correction explains only part of the benefit of velocity scaling.

Critical strengths

The paper clarifies an important theoretical issue. A velocity-norm gap caused by conditional averaging does not directly imply insufficient distributional transport. The population transport argument and Gaussian example distinguish a norm gap relative to training targets from a deficit in the velocity needed for correct generation. This is useful for interpreting velocity scaling, even though questions about actual generated states remain open.

The paper provides a systematic analysis of sampling behavior and a useful calibration method. The authors measure population time lag, examine model-time correction, velocity scaling, and related controls, and compare different models, sampling budgets, solvers, and guidance settings. Selecting the gain through measured lag improves generation in the tested ImageNet settings without an FID search for that gain. These results provide useful empirical evidence about when scaling helps and how much scaling to apply.

Critical weaknesses

The evidence does not fully convince me of the broad claim that velocity scaling does not correct a deficit during actual sampling. I accept the theoretical clarification that a conditional-mean norm gap does not imply a failure of transport by the population MSE field. I also recognize that Section 4 explicitly restricts its near-unit scalar calibration result to genuine training-path states. However, generated states may follow a different distribution, so this result does not directly rule out local magnitude underestimation or insufficient progress on those states. The prior Velocity Deficit work reports that increasing movement along the model velocity improves intermediate and endpoint distributional statistics. Those observations do not establish their cause, but the present paper has not ruled out insufficient progress as a contributing explanation either. If insufficient progress can produce temporal lag, observing population time lag does not distinguish these explanations. The broader “no velocity deficit” framing in the paper should therefore match the scope of the theoretical and empirical evidence.

The interventions support time lag as a useful explanation, but do not clearly separate it from related magnitude and trajectory effects. Changing the time input can change both the magnitude and direction of the predicted velocity, so the effectiveness of time correction does not independently rule out a magnitude issue. The frozen-field and radial controls establish the importance of subsequent model evaluations and the direction of the additional displacement, respectively. However, these findings are also compatible with an explanation involving additional progress followed by updated predictions at the new states. I recognize that the authors acknowledge that lag explains only part of the benefit; I am not asking it to explain all gains. My concern is which specific mechanistic conclusions these controls establish and which alternatives remain open.

Questions for authors
  1. Does “no velocity deficit” refer specifically to rejecting the claim that conditional averaging necessarily causes insufficient transport, or does it also exclude magnitude underestimation or insufficient progress on generated states? How does the calibration in Eq. (5), evaluated on genuine training-path states, support the latter conclusion? If the latter remains unresolved, should the abstract, contributions, and conclusion be narrowed accordingly?

  2. Is there evidence separating the benefit of better time conditioning from the accompanying changes in velocity magnitude and direction? Similarly, which findings from the frozen-field and radial controls distinguish lag correction from other progress or trajectory effects? If these effects cannot currently be separated, please clarify which mechanistic conclusions are supported and which remain hypotheses.

  3. How should we interpret the CIFAR-10 result where measured lag decreases but FID worsens? Could the selected gain be evaluated directly, with uncertainty estimates, to verify the interpolated result and clarify when lag is a reliable criterion for gain selection?

Although this paper did not fully convince me, and I have some concerns regarding the scope of the conclusions and the causal explanations. I find it to be an interesting and valuable piece of research. Therefore, I am currently leaning toward a "weak accept," though I would like to engage in further discussion with the authors.

Reviewer_ZVvCOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper revisits the effectiveness of velocity scaling in flow matching. Contrary to the previously proposed velocity-deficit explanation, the authors argue that MSE training does not inherently underestimate the velocity magnitude. They then introduce population time lag as an alternative explanation and propose a probe-based approach to measure this lag and calibrate the velocity gain. The paper provides extensive empirical analyses across different architectures, sampling solvers, and generation settings.

Critical strengths
  1. Valuable theoretical analysis. I appreciate the analysis in Section 4, which challenges the previous velocity-deficit explanation. The authors provide a proof showing that the population-optimal flow-matching velocity does not inherently suffer from the proposed magnitude deficit. This is a meaningful clarification of the underlying mechanism.

  2. Extensive empirical investigation. The paper presents a comprehensive set of experiments across different model architectures, model sizes, sampling budgets, and solvers.

Critical weaknesses
  1. Poor presentation and unclear organization.

My primary concern is the overall presentation of the paper. The manuscript lacks a dedicated section that systematically introduces the proposed methodology. Instead, methodological details, theoretical discussions, and experimental findings are scattered throughout different sections, particularly Sections 5–7. This makes it unnecessarily difficult to understand the overall framework and follow the logical development of the paper. Furthermore, several important experimental results and figures are placed in the appendix, even though they are essential to understanding and evaluating the main claims.

I strongly encourage the authors to reorganize the manuscript, clearly separate the methodology from the empirical analysis, and move the most important supporting figures into the main text.

  1. Limited scalability and uncertain reliability of the probe-based calibration.

The proposed calibration procedure relies on several indirect estimation steps. First, an additional time-prediction probe must be trained on genuine interpolation states. The probe is subsequently applied to generated trajectories to estimate population time lag, which is then used to select the velocity gain.

In particular, the probe is trained exclusively on genuine interpolation states but evaluated on generated states that may differ substantially from the training distribution. It is unclear whether the probe can reliably distinguish actual temporal misalignment from other distributional discrepancies. Moreover, population time lag is measured using a single learned scalar statistic, which may not adequately characterize the discrepancy between the generated and target distributions.

The practical applicability of this procedure is also unclear for more general conditional generation tasks, such as text-to-image generation, where training a reliable condition-aware time estimator may be considerably more challenging. The need for configuration-dependent gain calibration further complicates its deployment.

  1. Incomplete explanation of why velocity scaling improves generation quality.

While I appreciate the theoretical analysis in Section 4, I believe the paper still falls short of providing a convincing mechanistic explanation for the effectiveness of velocity scaling. The population time lag hypothesis offers an interesting perspective, and the first-order argument in Section 5.3 provides some intuition for why scaling can advance lagging states. However, this analysis does not establish a quantitative relationship between lag correction and improvements in generation quality.

More importantly, the experimental results reveal a substantial discrepancy between the lag-minimizing gain and the FID-optimal gain. For example, in Appendix Table 8, the FID-optimal gains can increase the measured population time lag relative to unscaled sampling, particularly at higher NFEs, while still significantly improving FID. This observation raises a fundamental question about how much of the benefit can actually be attributed to correcting population time lag.

Questions for authors
  1. How robust is the estimated population time lag to the choice of probe architecture and training distribution? Can the authors validate the proposed lag measurements using an independent distributional metric rather than another learned time estimator?

  2. How do the authors reconcile the population time lag hypothesis with the observation that FID-optimal gains can substantially increase the measured lag?

  3. How practical is the proposed gain calibration in more general conditional generation settings, particularly text-conditioned generation?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank you for the constructive discussion

We would like to sincerely thank all three reviewers for their thoughtful reviews, constructive engagement, and positive final assessments. Your willingness to engage in discussion made the review process particularly enjoyable and productive.

We thank ZVvC for suggesting independent validation of the population time lag measurements. This motivated the new spectral checks, which significantly strengthen the evidence presented in the paper.

We thank 5iRd for initiating an interesting discussion about model errors at intermediate sampling states, their relationship to population time lag, and the distinction from the proposed velocity deficit. We particularly enjoyed thinking through these questions, and the discussion helped us articulate these distinctions more clearly.

We thank Xw7Q for suggesting the combined velocity-scaling and model-time correction experiments, as well as investigating newer generative models. These experiments yielded interesting additional findings.

We look forward to incorporating these experiments and insights into the final manuscript.

Thank you again for making this a constructive and engaging discussion. We're happy to answer any further questions or clarify anything else.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.