- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
Presents an interpretation of the MeanFlow objective and formulation in the language of Temporal Difference (TD) learning and propose a Jacobian formulated probe of vector field sensitivity, or ‘stiffness’, with respect to light input and output variation. Authors explain why this factor is hidden from the MeanFlow learning objective, and explore how curricula designed to alter stiffness impact downstream FID scores. They identify high stiffness (or low sensitivity) with more reliable large-gap predictions.
Critical strengths
Strengths:
(1) Interpreting the MeanFlow objective in the language of Temporal Difference learning appears to be a genuinely novel approach.
(2) The resolution of the Jacobian without full formulation appears to be novel (as far as I am aware) and seems like an insightful approach to studying the dynamics of flow fields.
(3) Characterizing new measures of desirable or undesirable latent behavior in flow fields is a valuable goal, and measures of field sensitivity are intriguing.
Critical weaknesses
Weaknesses (see Questions for elaboration):
(1) Swathes of the main body text are readily identifiable as LLM generated. While the AI use statement does disclose this use for “readability”, it in fact has the opposite effect. This weakness obscures the manuscripts content and contribution.
(2) It seems that the stiffness is trends similarly to the inverse of the learning rate. This likeness is not mentioned or explored, yet appears to be a genuine source of confounding.
(3) Sensitivities to noise scales and solver budgets are alluded to but are not explored or discussed in the main body.
(4) The main body does not clearly communicate what the dosing/treatment method actually is, or what its expected experimental results would be.
(5) The results presented are mixed, and it is unclear what robust connections if any has to training dynamics or outcomes.
(6) The results are strictly confined to MeanFlows, but the probe formulation may not be strictly limited to this setting.
Questions for authors
Questions:
(1) It’s not clear from the main body what exactly is. What is , and how are readers meant to know? The first reference to the term appears to be line 301-302. It is referred to in many places throughout the text, so it seems important, but it is not clearly defined. What is ?
(2) Sensitivity to the solver budget is mentioned in line 258. How sensitive is to this budget? Similarly, how sensitive is to the step size?
(3) Finding 1 (lines 295-307) claim that “sensitivity accelerates after learning rate warmup” and peaks before FID and notes a similar result on ImageNet without the warmup period. However, in both of these cases the learning rate is dynamic, and the observed stiffness uptrend is consistent with the inverse of a decreasing learning rate. Is just measuring the inverse learning rate or some rescaling of it? Intuitively the learning could be viewed as iteratively refining the field, in which case the we should expect the stiffness to be connected with it (ie, the model isn't impacted as much by weight perturbations smaller than the LR, ). Again here, the size becomes important to document (see question 2). If not, why not? A connected question could be to be rephrase the above around the sensitivity of the integrator method, rather than the LR.
(4) Finding 1 and elsewhere claims that increases as generation FID increases. This appears confounded by the lr note in question 3, but more importantly, it’s not actually clear if this claim is true. As the abstract points out, large gap supervision decreased FID performance on ImageNet. Extracting this signal more cleanly would illuminate the connections of with training outcomes.
(5) The dosing/treatment processes are not clearly described in the main body, but the setting is fairly clear - transitioning from standard CFM training to MeanFlow training and tracking the magnitude and the stiffness as training continues. The dosing and supervision procedures in section 5 are not clearly described. However, their effects and the authors’ conclusions seem in line with a general intuition: supervising large gaps helps, you cannot remove small gaps altogether, and delayed treatment can destabilize established networks. What is the treatment, and how precisely is it done?
(6) Should readers expect different results for Rectified Linear Flows or other flow-family methods? The probe approach may not be strictly limited to MeanFlow conceptually. This may be best left for future work.
3 comments
Response to Authors
Clarity: My primary weakness remains unaddressed.
The main body text after revision is not much easier on the eyes, and appears to most likely be largely the output of an LLM. The persistent lack of clarity obfuscates the methods, contributions, and experiments.
Relation of with the learning rate: I am unconvinced by the authors reasoning.
Just because the learning rate does not enter the calculation of does not mean that it is isolated. The learning rate determines the structure and resolution of the loss landscape learned by . However, with Table 2 and some inline text clarifying constant learning rates in training, I agree that it is unlikely at play in their experiments. I would be surprised if future examinations revealed that is independent of learning rate magnitude and schedule.
Contribution: The stated contributions are muddled.
The stated three primary contributions in the introduction appear to be approximate restatements of the same core, that “ is a neat way to measure flows”. I agree that it could be, if it measured anything other than itself, but the authors do not provide convincing evidence that ultimately measures anything other than itself. There are no robust connections with training dynamics, impacts on resolution at inference time, or effects on evaluations revealed. In short, it is simply not clear why matters.
At this time, I do not feel that my score has changed.
Contribution. This is an analysis paper. It reads MeanFlow training as a TD bootstrap whose loop gain plays the part of the discount in stability (Table 1, Section 3.2), and it measures this gain on trained models (Algorithm 1). As with in TD, a gain below one guarantees that the linearized bootstrap contracts, even with a delayed, averaged or periodically refreshed target (Appendix B). MeanFlow's gain is far beyond this bound, reaching about at large gaps (Section 4), yet training still succeeds. is the quantity in this stability condition, not a measure of itself. Its growth comes from bootstrapped large-gap training (Table 2), and once the target lags, the large gaps fail first and one-step FID rises to between 171 and 251, against 12 to 34 online (Section 5, Table 10). The paper never claims that improves FID, ranks models or yields a better training recipe.
Learning rate. We agree that , like any property of trained weights, depends on the whole recipe, and the paper does not claim otherwise. Its claim is that the rise of is not a rescaling of the learning rate. The rise continues after the rate becomes constant, and a tenfold lower rate does not raise (Section 4, Appendix C).
Clarity and LLM use. The revision rewrote the main text, defines and the control where they first appear (Section 1) and moves the probe into the main text (Algorithm 1). As ICLR 2027 requires, the AI use statement discloses our use of LLMs in writing, and we checked every sentence ourselves. The comment names no sentence that is unclear, and the reviewer is welcome to name one.
We thank the reviewer for the detailed questions and apologize for the poor readability of the submission. We uploaded a revised manuscript, and the section, table and figure numbers below refer to it.
What is h (Question 1).
h=t−r is the time gap between the two times of a MeanFlow pair, and h=1 is one-step generation. g is largest at large gaps (Tables 3, 6 and 7). The revision defines h in Sections 1 and 3.1.
Sensitivity to the solver budget and other settings (Question 2, Weakness 3).
This is now in the main text (Section 3.3, Table 5). The solver budget and the probe batch change the absolute value of g, so every value is a lower bound and we quote medians over three batches. The ratios of medians that our claims rest on change by less than a factor 1.5. The probe has no step size, because it uses exact Jacobian-vector products (Algorithm 1).
Is g the inverse learning rate (Question 3, Weakness 2).
No. g is the largest change in dtduθ, the derivative term of the target, per unit change in the output uθ. It is computed on a frozen checkpoint, and the learning rate does not enter it. As an analogy, g is to MeanFlow's bootstrap what sharpness, the largest Hessian eigenvalue, is to gradient descent. Both are properties of the current weights. Gradient descent is guaranteed stable when sharpness is below 2/η, which contains the learning rate. The bootstrap is guaranteed to contract when (t−r)g<1, which contains only the time gap. In our runs the learning rate never decreases, while g rises (Table 2, Appendix C).
Does g rise as FID improves, and how is g connected to training outcomes (Question 4, Weakness 5).
We claim no relation between g and the outcome of training as measured by FID. g rises while FID improves within a run (Table 2), and α-Flow and Decoupled MeanFlow (DMF), which generate better than MeanFlow, are not less stiff (Table 3). g is not a quality metric. We claim two links to training dynamics. First, g grows because of bootstrapped large-gap training (Table 2). Second, the large gaps, where the gain (t−r)g is largest, are the first to break when the target contains an error (Section 5, Fig. 2). Although g does not track FID, it sets the gain (t−r)g of a bootstrapped target. Many one-step models are trained with such a target, as in TD learning, and the probe can be applied to them (Question 6).
What is the treatment (Question 5, Weakness 4).
g is not a quality metric, so it does not need a treatment. We focus on the measurement of g and on its test in Section 5.
Other flow families (Question 6, Weakness 6).
The probe is not limited to MeanFlow, and the way to measure g is our first contribution (Algorithm 1). It runs on a frozen checkpoint without any training and can be applied to any model whose target contains the network's own output or derivative, such as consistency, shortcut and flow-map models (Section 6).