- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies conditional flow-matching models (CFMs) in regression-type settings and identifies a variance inflation phenomenon: when the learned velocity field underfits its dependence on the conditioning variable, the generated samples have larger conditional variance than the true conditional distribution. The authors attribute this to a smoothing bias in vector-field learning, support it with synthetic experiments (early stopping, explicit roughness penalty, kernel ridge regression) and an offline RL benchmark, and provide an idealized theoretical model based on an equivalent-kernel approximation to regularized flow matching, which yields a bias–variance inequality (Theorem 1). Finally, they show empirically that predictive sets built directly from CFM samples are approximately calibrated in the underfitting regime on policy evaluation, cell perturbation and behavior cloning tasks.
I find the phenomenon itself clearly presented and the synthetic experiments well designed. However, in my opinion the contribution is overstated: the effect is neither specific to flow matching nor surprising from a probabilistic modeling perspective, the theoretical model relies on explicit regularization rather than on the underfitting/early-stopping regime the paper actually argues about, and the experimental section lacks numerical results, comparisons to other probabilistic methods and implementation details. The practical use-case (coverage without conformal correction) is only verified empirically and trades off predictive accuracy of the mean.
Critical strengths
I like the clear identification and illustration of the variance inflation effect, in particular the controlled experiments in Figure 2 and Appendix C that vary three different smoothing mechanisms and show the same qualitative behavior. The idealized model in Section 5 is simple and gives a clean interpretation of the inflated variance as a local mixture of true conditionals, and the correlation between squared bias and excess variance in Figure 18 is a nice empirical check of the theory.
Critical weaknesses
1. Motivation and contribution
The main claim of the paper is, in my view, not specific to conditional flow models and not particularly surprising. Any probabilistic model minimizes some divergence or metric between its predictive distribution and the true conditional distribution. If the model class is constrained, e.g. by regularization or limited capacity, one expects it to learn a projection onto that class, which naturally absorbs the unresolved (epistemic) information into the predictive uncertainty. I verified that the same qualitative effect appears for a simple MLP trained with a Gaussian NLL under the same smoothing regimes, so the selling point is really about smoothing of probabilistic models in general rather than about flow matching. The paper should either sharpen what is specific to CFMs or reframe the contribution accordingly.
Related to this, the theoretical model uses an explicit regularization term in (5.1) and a smoothing bias, whereas the paper argues that the same holds for early stopping and the underfitting regime. In practice, one is interested in the optimal point just before overfitting (selected by validation loss), and it is unclear whether early stopping at this point induces a bias strong enough for the effect to be noticeable. Figure 1 with proper early stopping would likely look quite different, and with a proper training protocol one would not stop as early as in the shown examples. The paper also claims that “any” inductive bias producing smoothing has the same effect (l. 83ff.), but only three mechanisms are tested.
Finally, the practical motivation is not clear to me. The proposed use-case sits between underfitting (inflated variance, badly fitted mean) and overfitting (loss of coverage). Reliable coverage is obtained at the cost of predictive accuracy of the mean function, whereas in practice one wants a good mean function together with reliable uncertainty. The coverage analysis is purely empirical, which the authors acknowledge, and the decomposition (A.15) essentially already contains the variance inflation statement, so Theorem 1 adds comparatively little beyond the local-mixture representation.
2. Baselines and experiments
The experimental section is in its current form not sufficient to assess the method as an uncertainty quantification approach. There are no numerical results beyond the coverage and width curves, e.g. no accuracy or other metrics and no comparison to other probabilistic methods (apart from DER and HGR at a single checkpoint). Several strong claims rely on visual inspection of figures, e.g. l. 169 (“establish that it is driven by smoothing”), l. 223 (“reveals that the phenomenon is not tied to a single estimator”) and l. 229 (“as can be observed from the figures”), and should be backed by quantitative evidence.
Further, no code is provided, and the synthetic experiments lack a description of the actual implementation, i.e. the MLP architecture and the functions used in Figures 1, 2 and 15–17. Several benchmarks are cited only via preprints.
3. Minor points and style
- l. 38: "numerous other scientific estimation and time-series problems" is backed by a single reference, I would expect more citations here.
- Definition 3.1: should take three arguments; the notation mixes , and throughout the paper. Also, the domain/codomain in the definition appear to be swapped ( vs. ).
- Eq. (5.1): the space is never defined.
- l. 324: the whole motivation of Assumption 5.1 is moved to the appendix; at least 1–2 sentences should remain in the main text.
- l. 681: missing reference, indicated by a question mark.
- Assumption A.1: and are not defined before use.
- Why is "apparent aleatoric variance" used in the title and throughout? It should be clear that the model's predictive uncertainty is always influenced by the estimation error, so the terminology seems somewhat misleading. This might be worth a discussion with respect to literature on aleatoric/epistemic uncertainty in general.
Questions for authors
- In Figures 5–7 and 11–14, how is the end of the underfitting regime (the point where train and validation losses diverge) selected? Is this done qualitatively, or is there a specific mechanism or heuristic behind the shaded region?
- Can the authors comment on what is specific to conditional flow models in the variance inflation phenomenon, as opposed to any probabilistic model under a smoothing bias, for example, a proper scoring rule based network (e.g. a Gaussian NLL network), a BNN or a diffusion model?
- Does the effect persist with proper early stopping based on validation loss, i.e. at the training point one would actually use in practice, and how large is the resulting inflation in that case?