- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the end-to-end convergence of flow matching with deep ReLU networks. It analyzes the convergence of gradient descent and combines optimization, approximation, generalization, and discretization errors to derive Wasserstein-2 bounds between the generated and target distributions.
Critical strengths
The paper addresses a theoretical problem by explicitly incorporating gradient-descent optimization error into the end-to-end convergence analysis of flow matching.
Critical weaknesses
-
Limited novelty and significance. The optimal velocity field obtained by minimizing the population L2 objective is already known to preserve the prescribed marginal distributions in flow matching. Moreover, existing studies have established generation error bounds given an approximate velocity field. Therefore, once convergence to the optimal velocity field is guaranteed, the resulting distributional convergence is largely expected. It is unclear what fundamentally new theoretical insights this work provides beyond combining existing neural network optimization theory with established flow matching error analysis.
-
Insufficient justification of the end-to-end convergence rate. The convergence rate claimed in Theorem 5.1 does not appear to follow directly from the preceding error bounds. In particular, Lemma 2.3 provides a Lipschitz bound of order ()), while Equation (16) contains an exponential amplification factor (). Under the proposed choice (), this factor is not uniformly bounded and may overwhelm the polynomial decay of the estimation error. Moreover, substituting the Lipschitz bound into Equation (15) does not yield a vanishing error bound. Therefore, additional assumptions or a more rigorous argument seem necessary to establish the claimed Wasserstein convergence rate.
Questions for authors
- Theorem 3.5 establishes convergence for the empirical loss defined using fixed time discretization points in Eq. (4), whereas Section 4 considers a different Monte Carlo-based empirical objective in Eq. (10). How is convergence for the former sufficient to control the optimization error of the latter? Is an additional approximation or discretization error bound required?
- The network defined in Eq. (5) does not explicitly include bias terms, despite the claim that biases are absorbed. In particular, at (), this architecture is positively homogeneous in (x), whereas the optimal flow-matching velocity is generally affine: (). How does the proposed architecture satisfy the universal approximation guarantee in Theorem 3.1?
- The learning-rate and iteration-complexity conditions in Theorem 3.5 appear different from those in Theorem E.45, which additionally involves the minimum input separation and time quadrature weights. Could the authors clarify how the convergence guarantee stated in the main text follows from the appendix?
- The analysis relies on full-batch projected gradient descent, fixed input and output layers, and sufficiently overparameterized ReLU networks. To what extent do these guarantees provide meaningful insights into practical flow-matching models trained with mini-batch optimization and more expressive architectures?
- Theorem 3.5 establishes geometric convergence of the empirical training loss, but the experiments mainly examine the final Wasserstein error. Could the authors provide direct empirical verification of the claimed convergence rate, particularly under different network widths and training configurations?