acceptodds
Under review as a conference paper at ICLR 2027

Flow-Driven Optimization Methods for Image Inverse Problems

Abstract

Flow matching is a recent state-of-the-art framework for generative modeling. A variety of optimization methods that integrate flow matching have been developed for image inverse problems. In this paper, a framework of gradient-related optimization methods is proposed that incorporates the flow matching model for image inverse problems. We introduce a novel regularization term that takes the form of a distance-based function. We prove that the negative gradient of this regularization term corresponds to the velocity field of the flow matching model. Based on this theoretical result, any gradient-related optimization method can be applied to solve optimization problems involving this new regularization term, provided that the data consistency term is differentiable. We propose flow-driven two-step gradient descent (TSGD), a flow-driven fast version of TSGD, flow-driven BCD, and flow-driven ADMM algorithms. Numerically, we evaluate these gradient-related methods on four inverse problems across three datasets. Numerical results demonstrate that FlowDPS, FlowBCD, and FlowADMM achieve superior performance, with FlowBCD and FlowADMM standing out in particular.

Primary area:
optimization
Keywords:
Gradient-related optimization, Flow matching, optimization method, image inverse problems

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_t5uFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework of gradient-related optimization methods to solve image inverse problem based on pre-trained flow matching models. Specifically, the authors introduce a novel regularization term through a distance function and formulated the related optimization problem. The authors demonstrate that any gradient-based methods can be applied to solve the optimization problem and provide several algorithm examples including FlowDPS, FlowTSGD, FlowFTSGD, FlowBCD, and FlowADMM. These algorithms are evaluated on four image inverse problem tasks and three datasets, with FlowDPS, FlowBCD, and FlowADMM performing well.

Critical strengths
  1. Interpreting the pre-trained flow matching model as the gradient of an explicit regularization term is meaningful and works as effective tools to involve any gradient-based methods to solve inverse problems.

  2. The main experiments are reasonable, covering four kinds of inverse problems (super-resolution via bicubic and avgpool downsamplings, Gaussian and Motion deblurring) under three datasets (DIV2K, FFHQ, AFHQ) and four common image metrics (PSNR, SSIM, LPIPS, FID).

Critical weaknesses
  1. Although the paper lists existing relevant methods in Related Works, it does not clearly separate its contributions into a new regularizer in the field of solving inverse problems or an optimization interpretation of an existing flow prior. I could not get the point about the paper's contribution or improvement especially on what existing methods could not do.

  2. The comparison to baselines and ablation study are insufficient. The paper only compared two methods (FlowChef and FlowDPS). Some other related works like Flower [1] should be also added as baselines. Also, the ablation study for hyperparameter ρ0∈{35,40,45}\rho_0 \in \{ 35, 40, 45 \} shows the trend that smaller ρ0=35\rho_0 = 35 leads to better performance in most metrics (PSNR, SSIM, FID). It remains unclear whether a much smaller ρ0<35\rho_0 < 35 would leads to better performace.

[1] Pourya et al. "Flower: A flow-matching solver for inverse problems", ICLR 2026.

Questions for authors

I list my questions below and if I misunderstood any part of the method, please correct me.

  1. I'm confused about the definition of the velocity field vθv_\theta in the manuscript: vθv_\theta seems to have opposite signs in Theorem 3.1 and Definition 4.1. In Theorem 3.1 and Appendix A, the training loss leads to the velocity vθ≈−a˙tu−b˙tϵv _\theta \approx - \dot{a} _t u - \dot{b} _t \epsilon. However, in Definition 4.1 which is based on the proposition 1 in FlowDPS, the Tweedie's formula used the common velocity field vθ≈a˙tu+b˙tϵv _\theta \approx \dot{a} _t u + \dot{b} _t \epsilon, which leads to contradiction.

  2. When deriving FlowFTSGD, the paper seems unifying two methods (FlowDPS and PnP-Flow), but it remains unclear that under what exact values of τt\tau_t, FlowFTSGD would correspond to these two methods.

  3. The inequality used in Line 632 to prove Theorem 3.1 is Jensen's Inequality, yes? It would be better to have clarification after Line 645.

  4. There seems a typo in Line 695 (Equation (4.4), Line 4): Eqn. (4.4) is a single-line optimization problem and contains no "Line 4". Please correct it.

  5. There seems a missing related work: Webber et al. "Solving Inverse Problems with Flow-based Models via Model Predictive Control", ICML 2026.

  6. The paper describes four optimization methods with the iterative scheme but no pseudo-code algorithms. Adding some pseudo-code algorithms could improve clarity and reproducibility.

5 comments
AuthorsOpenReview ↗

Response to Reviewer t5uF

[W1] We aim to establish a connection between generative models and optimization and understand flow matching-based imaging inverse problems from an optimization perspective.

In classical optimization methods for imaging inverse problems, we consider min⁡uGt(u)=λtF(u)+Rt(u),\min_u G_t(u)=\lambda_t F(u)+R_t(u), where F(u)F(u) is the data-fidelity objective and Rt(u)R_t(u) is the regularization term.

When solving it with gradient-based methods, we need ∇uGt(u)=λt∇uF(u)+∇uRt(u).\nabla_u G_t(u)=\lambda_t\nabla_u F(u)+\nabla_u R_t(u). In Section 3, we set Rt(u)R_t(u) as a distance-based function show that ∇uRt(u)=−vθ,\nabla_u R_t(u)=-v_\theta, where vθv_\theta is the pretrained velocity field in the unconditional flow matching model.

Now, both ∇uGt(u)\nabla_u G_t(u) and ∇uRt(u)\nabla_u R_t(u) are determined and we can apply gradient-based optimization methods directly.

[W2] To ensure good LPIPS performance, we set ρ0=40\rho_0=40 in this paper. Numerical results for ρ0=25,30\rho_0=25,30 show no performance improvement.

Methodρ0\rho_0PSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$Methodρ0\rho_0PSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
FlowBCD2527.030.72019.120.197FlowADMM2527.020.72420.040.203
FlowBCD3027.050.72418.710.192FlowADMM3027.060.72619.100.195
FlowBCD3527.010.72518.530.186FlowADMM3527.030.72718.550.186
FlowBCD4026.880.72219.520.182FlowADMM4026.880.72319.420.181
FlowBCD4526.470.70623.030.190FlowADMM4526.470.70823.040.190

[Q1] We revised Theorem 3.1 as follows:

Let MM be a data manifold. For any ϵ∼N(0,I)\epsilon\sim N(0,I) and u∈Mu\in M, define z=atu+btϵ\mathcal z=a_t u+b_t\epsilon, where at,bt≥0a_t,b_t\ge0. Let vθ(z,t)v_\theta(z,t) be the pre-trained neural network to estimate the velocity field in the flow matching model (see the overview in Appendix A). Set ℓt(z)=μtfM(ctz,σt)−ηt2∣∣z∣∣2\ell_t(z)=\mu_t f_\mathcal M(c_t z,\sigma_t)-\frac{\eta_t}{2}||z||^2. When the neural network is optimal trained--that is, when ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt=0\int^1_0\mathbb E_\epsilon\mathbb E_u||\dot a_t u+\dot b_t\epsilon-v_\theta(a_t u+b_t\epsilon,t)||^2dt=0, it follows that ∇zℓt(z)=−vθ(z,t),\nabla_z\ell_t(z)=-v_\theta(z,t), where ct=1at,σt=btat,μt=atbt(a˙tbt−atb˙t),ηt=μtat2+b˙tbt.c_t=\frac{1}{a_t},\quad \sigma_t=\frac{b_t}{a_t},\quad \mu_t=\frac{a_t}{b_t}(\dot a_tb_t-a_t\dot b_t),\quad \eta_t=\frac{\mu_t}{a^2_t}+\frac{\dot b_t}{b_t}. In practice, ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt>0\int^1_0\mathbb E_\epsilon\mathbb E_u||\dot a_t u+\dot b_t\epsilon-v_\theta(a_t u+b_t\epsilon,t)||^2dt>0 is a small constant after training, vθ(zt,t)v_\theta(z_t,t) approximates the negative gradients of the function ℓt(z)\ell_t(z) with respect to zz.

[Q2] τt=bt\tau_t=b_t for the FlowDPS, while PnP-Flow corresponds to the FlowTSGD.

[Q3-5] Yes, we used Jensen's Inequality to prove Theorem 3.1, which is added in revision. For Q4, we modify to Equation (4.3), Line 3. For Q5, we add this reference to related works. Thank you for the comments.

[Q6] Pseudocode of FlowTSGD

Input: at,bt,ζt,t∈[0,1]a_t,b_t,\zeta_t,t\in[0,1], predicted velocity vθ(z,t)v_\theta(z,t), num,

t=0t=0, δ=1num\delta=\frac{1}{\mathrm{num}},

Sample z∼N(0,I)z\sim N(0,I),

For(t<1t<1)

u←z−bt∇zℓt(z)=z+btvθ(z,t)u\leftarrow z-b_t\nabla_z\ell_t(z)=\mathcal z+b_t v_\theta(z,t),

u←u−ζt∇uF(u)u\leftarrow\mathcal u-\zeta_t\nabla_{u} F(u),

t←t+δt\leftarrow t+\delta,

z←Qt(u,z)z\leftarrow Q_{t}(u,z),

Output: zz.

Pseudocode of FlowFTSGD

Input: at,bt,ζt,τt,t∈[0,1]a_t,b_t,\zeta_t,\tau_t,t\in[0,1], predicted velocity vθ(z,t)v_\theta(z,t), num,

t=0t=0, δ=1num\delta=\frac{1}{\mathrm{num}},

Sample z∼N(0,I)z\sim N(0,I),

For(t<1t<1)

u←z−bt∇zℓt(z)=z+btvθ(z,t)u\leftarrow z-b_t\nabla_z\ell_t(z)=z+b_t v_\theta(z,t),

v←u−ζt∇uF(u)v\leftarrow u-\zeta_t\nabla_u F(u),

u←τtu+(1−τt)vu\leftarrow\tau_t u+(1-\tau_t)v,

t←t+δt\leftarrow t+\delta,

z←Qt(u,z)z\leftarrow Q_{t}(u,z),

Output: zz.

Pseudocode of FlowBCD

Input: at,bt,ζt,τt,ρt,λt,t∈[0,1]a_t,b_t,\zeta_t,\tau_t,\rho_t,\lambda_t,t\in[0,1], predicted velocity vθ(z,t)v_\theta(z,t), num,

t=0t=0, δ=1num\delta=\frac{1}{\mathrm{num}},

Sample z∼N(0,I)z\sim N(0,I), w∼N(0,I)w\sim N(0,I),

For(t<1t<1)

u←Pt(z)u\leftarrow P_t(z),

v←Pt(w)v\leftarrow P_t(w),

u←z+btvθ(z,t)−ρt(u−v)u\leftarrow z+b_tv_\theta(z,t)-\rho_t(u-v),

s←v−ζt[λt∇vF(v)+ρt(v−u)]s\leftarrow v-\zeta_t[\lambda_t\nabla_v F(v)+\rho_t(v-u)],

v←τtv+(1−τt)sv\leftarrow\tau_tv+(1-\tau_t)s,

t←t+δt\leftarrow t+\delta,

z←Qt(u,z)z\leftarrow Q_t(u,z),

w←Qt(v,w)w\leftarrow Q_t(v,w).

Output: ww.

Pseudocode of FlowADMM

Input: at,bt,ζt,τt,ρt,λt,t∈[0,1]a_t,b_t,\zeta_t,\tau_t,\rho_t,\lambda_t,t\in[0,1], predicted velocity vθ(z,t)v_\theta(z,t), num,

t=0t=0, δ=1num\delta=\frac{1}{\mathrm{num}},

Sample z∼N(0,I)z\sim N(0,I), w∼N(0,I)w\sim N(0,I),

Λ=0\Lambda=0,

For(t<1t<1)

u←Pt(z)u\leftarrow P_t(z),

v←Pt(w)v\leftarrow P_t(w),

u←z+btvθ(z,t)−ρt(u−v)+Λu\leftarrow z+b_tv_\theta(z,t)-\rho_t(u-v)+\Lambda,

s←v−ζt[λt∇vF(vt)+ρt(v−u)]−ζtΛs\leftarrow v-\zeta_t\left[\lambda_t\nabla_v F(v_t)+\rho_t(v-u)\right]-\zeta_t\Lambda,

v←τtv+(1−τt)sv\leftarrow\tau_tv+(1-\tau_t)s,

Λ←Λ+ρt(v−u)\Lambda\leftarrow\Lambda+\rho_t(v-u),

t←t+δt\leftarrow t+\delta,

z←Qt(u,z)z\leftarrow Q_t(u,z),

w←Qt(v,w)w\leftarrow Q_t(v,w).

Output: ww.

I thank the authors for the detailed rebuttal. While some of my concerns have been addressed, several issues remain:

  1. The response to W1 reiterates the goal of connecting generative models with optimization and interpreting the velocity field as a regularization gradient. However, it still fails to clarify the paper's novelty/improvement on what existing methods could not do and show what limitations it addresses.

  2. W2 is also only partially addressed, since my original questions concerned two parts: insufficient baseline comparisons and ablation study. The response provides the ablation study results of hyperparameter ρ0\rho_0 but misses the additional comparisons with closely related methods such as Flower [1].

  3. The revised version of Theorem 3.1 seems reasonable, but without an updated proof, its correctness could not be verified. Could the authors provide the complete revised derivation?

  4. As for Q3-Q5, I could not locate the revisions and please let me know if I may have missed them in the current version.

As for Q6, I thank the authors for providing these pseudo-code algorithms which improves the paper's readability. Overall, I appreciate the authors’ clarifications and additional results, but the concerns above remain unresolved. Therefore, I maintain my original rating.

[1] Pourya et al. "Flower: A flow-matching solver for inverse problems", ICLR 2026.

[Q1] Our key insight is that the velocity field of a flow matching model can be interpreted as the gradient of an explicit regularization term Rt(u)R_t(u), i.e., the generative prior enters the optimization as a well-defined regularizer rather than as sampling-time guidance.

  • It lets us embed a flow matching prior into classical optimization algorithms (BCD and ADMM, giving FlowBCD and FlowADMM), which come with established convergence analysis — something guidance-based solvers lack.

  • Because the prior is an explicit term in the objective, the regularization strength λt\lambda_t has a clear interpretation and can be set by standard parameter-selection principles rather than those guidance schedules.

  • Most researchers in the image processing community are more familiar with optimization than with probability and stochastic processes, so our paper serves as a bridge between generative modeling and this community.

[Q2] The following presents experimental comparisons on Flower, where the number of sampling steps is set to 28 for fairness, matching the other methods.

Avgpool ×12\times12 SR

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ16.020.32099.420.320
FFHQ15.930.363195.960.345
DIV2K14.790.230160.980.407

Bicubic ×12\times12 SR

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ15.890.36399.900.382
FFHQ15.730.423191.500.347
DIV2K14.950.271159.820.403

Gaussian Deblurring

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ16.310.27859.080.205
FFHQ15.900.320141.710.188
DIV2K15.460.238117.760.223

Motion Deblurring

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ19.700.43722.57‾\underline{22.57}0.156
FFHQ18.890.42962.020.145
DIV2K18.330.40153.910.142

[Q3] We have updated the revision, thank you for your time and efforts reviewing our paper.

Reviewer_95A4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a method for solving linear image inverse problems for flow matching models. The authors introduce a distance based regulation term and show that its negative gradient equals flow matching velocity. As a result, we can leverage classical optimization methods such as block coordinate descent (BCD) and alternating direction method of multipliers (ADMM) to use a pretrained flow matching model to solve linear inverse problems. The resulting methods have been evaluated on tasks such as super resolution, gaussian debarring, motion deblurring, on datasets such as AFHQ, FFHQ, and DIV2K.

Critical strengths
  1. This paper has a well explained introduction and related work.
  2. This work bridges inverse problems with classical optimization which allows us to leverage considerable knowledge from optimization literature to solve inverse problems with pretrained flow matching models.
Critical weaknesses
  1. Overclaiming in the abstract and introduction, as the paper states that “We propose flow driven two step gradient descent (FlowTSGD), a flow-driven fast version of TSGD (FlowFTSGD)…”, but later on in the results section (Line 349-351), the paper states: “As FlowChef and FlowDPS are exact counterparts of FlowTSGD and FlowFTSGD, respectively, we adopt the numerical results of FlowChef and FlowDPS in their place.” FlowTSGD and FlowFTSGD shouldn’t be listed as contributions early in the paper.
  2. Missing baselines/ablations:
    1. Other relevant baselines that were also included in FlowDPS are missing e.g. Resample, LatentDAPS, Pnp-Flow etc.
    2. FlowBCD and FlowADMM use twice as many NFEs as FlowDPS and yet, FlowDPS beats them on FID on SR on AFHQ and DIV2K, and on 3 metrics on FFHQ. Further, an ablation with comparable number of NFEs with FlowDPS is missing.
  3. Inconsistent/incorrect notations:
    1. Missing integral dudu in Line 174.
    2. In Line 622, fM(z)f_\mathcal{M} (z) is notation is inconsistent with the definition.
    3. There is inconsistent use of σt\sigma_t. Line 376 states that σt\sigma_t is a fixed parameter in unconditional flow model, while Theorem 3.1 defines σt=btat\sigma_t = \frac{b_t}{a_t}.
    4. Undefined λt\lambda_t and τt\tau_t in the updates for FlowBCD and FlowADMM.
  4. Incorrect/inconsistent math:
    1. Tweedie’s formula in Definition 4.1 uses vθ=a˙tu+b˙tϵv_\theta = \dot{a}_t u + \dot{b}_t \epsilon but in the previous section, Theorem 3.1 trains the flow matching model to be vθ=−a˙tu−b˙tϵv_\theta = -\dot{a}_t u - \dot{b}_t \epsilon (i.e. negative velocity field.).
    2. In Line 220-221, btatlt(z)\frac{b_t}{a_t}l_t(z) would not be equal to μtatbt\frac{\mu_t a_t}{b_t}. This factor btat\frac{b_t}{a_t} is also missing from the second term. In addition, it is unclear how division by 0 is handled at the boundaries here.
    3. Equation 4.6 with Augmented Lagrangian looks incorrect. The right way to optimize this would be to minimize with respect to the primal variables (u,v)(u, v) and maximize with respect to the Lagrange multiplier Λ\Lambda. Also, if Eq. 4.6 has minimization, it is unclear why Eq. 289 and Eq. 290 perform an ascent instead of descent. In addition, in the objective, the regularizer Rt(u)R_t(u) acts on uu and F(v)F(v) acts on vv. However, the methods for FlowADMM and FlowBCD maintain another sampling trajectory based on wtw_t which does not follow from the minimizing equation and imposes a second flow prior while doubling NFEs. In addition, is the final output u1u_1 or v1v_1?
    4. In Appendix B, in proof of Theorem 3.1, the steps of the proof needs to be written more clearly. Currently, I think there are some constants and factors that disappear through the steps proof (e.g. I’m not sure where did the btb_t in the denominator disappear. Similarly, p(z∣t)p(z|t) vanishes.) Also, dz=btddϵdz = b_t^d d \epsilon, therefore, it is unclear why btdb_t^d is present in Line 641.
    5. Proof of Theorem 3.1 shows that 0≤g1(θ)≤g2(θ)0 \leq g_1(\theta) \leq g_2(\theta). However, in this case, g2(θ)g_2(\theta) which is conditional flow matching loss, cannot reach 0. Please see proof of Theorem 2 in Lipman et al. 2022 [1]. There is a conditional variance term that is strictly positive because noisy images from different clean images can map at the exact same noisy point. Therefore the reasoning towards the end of this proof of theorem 3.1 is flawed.

[1] Lipman, Yaron, et al. "Flow matching for generative modeling." arXiv preprint arXiv:2210.02747 (2022).



Due to inconsistent math above, I’m very much confused about the actual method that is implemented and why it works.

Questions for authors
  1. What is the λt\lambda_t used in practice (appears in Eq. 4.4, 4.5, 4.6)? Similarly what is τt\tau_t in Line 276 and 289? In addition, what is Λ0\Lambda_0 initialized to?
  2. (Minor) Please consider revisiting the citation for Tweedie’s (line 206) to ”Robbins, Herbert E. "An empirical Bayes approach to statistics." Breakthroughs in Statistics: Foundations and basic theory. New York, NY: Springer New York, 1992. 388-394.”
  3. In Section 4, all the equations are written assuming flow matching model is in pixel space but later on, the paper switches to use SD3 which would require a VAE decoder. Please consider updating the expression of the forward operator to include a decoder.
  4. Why does FlowADMM have almost similar numbers as FlowBCD? This would imply that ADMM dual variable Λ\Lambda doesn’t add much value.
4 comments

[W1] We updated as follows:

We propose flow-driven two-step gradient descent (FlowTSGD), a flow-driven fast version of TSGD (FlowFTSGD), flow-driven BCD (FlowBCD), and flow-driven ADMM (FlowADMM) algorithms from an optimization perspective, which establish a connection between generative models and optimization.

[W2] [Experiments on Flower] The number of sampling steps is set to 28 for fairness, matching the other methods.

Avgpool ×12\times12 SR

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ16.020.32099.420.320
FFHQ15.930.363195.960.345
DIV2K14.790.230160.980.407

Bicubic ×12\times12 SR

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ15.890.36399.900.382
FFHQ15.730.423191.500.347
DIV2K14.950.271159.820.403

Gaussian Deblurring

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ16.310.27859.080.205
FFHQ15.900.320141.710.188
DIV2K15.460.238117.760.223

Motion Deblurring

DatasetPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
AFHQ19.700.43722.57‾\underline{22.57}0.156
FFHQ18.890.42962.020.145
DIV2K18.330.40153.910.142

[compare number of NFEs] It is unfair to compare the number of NFEs alone.

We provided a computational complexity in Subsection 5.3 and Appendix C.1. Although both FlowBCD and FlowADMM invoke the NN twice per step, Table 4 shows that their mean runtimes exceed that of FlowDPS by only 11.4%11.4\% and 12.6%12.6\%, respectively. This additional cost stems primarily from the second forward evaluation of the flow model, while the decoding and gradient computations for data consistency are not doubled. NFEs counts alone do not reflect the full sampling cost.

[W3] We add dudu, update fM(ctz,σt)f_M(c_tz,\sigma_t) in Line 622.

For btb_t, we updated: btb_t is a determined parameter by unconditional FM.

The balanced parameter λt=1\lambda_t=1 in solving optimization models (4.5) and (4.6). btb_t is used to determine ρt=ρ0bt\rho_t=\rho_0b_t.

[W4] [thoerem 3.1]: We updated Theorem 3.1 as follows:

Let MM be a data manifold. For any ϵ∼N(0,I)\epsilon\sim N(0,I) and u∈Mu\in M, define z=atu+btϵz=a_t u+b_t\epsilon, where at,bt≥0a_t,b_t\ge0. Let vθ(z,t)v_\theta(z,t) be the pre-trained neural network to estimate the velocity field in the flow matching model (see the overview in Appendix A). Set ℓt(z)=μtfM(ctz,σt)−ηt2∣∣z∣∣2\ell_t(z)=\mu_t f_M(c_t z,\sigma_t)-\frac{\eta_t}{2}||z||^2. When the neural network is optimal trained--that is, when ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt=0\int^1_0\mathbb E_\epsilon\mathbb E_u||\dot a_t u+\dot b_t\epsilon-v_\theta(a_t u+b_t\epsilon,t)||^2dt=0, it follows that ∇zℓt(z)=−vθ(z,t),\nabla_z\ell_t(z)=-v_\theta(z,t), where ct=1at, σt=btat, μt=atbt(a˙tbt−atb˙t), ηt=μtat2+b˙tbt.c_t=\frac{1}{a_t},\ \sigma_t=\frac{b_t}{a_t},\ \mu_t=\frac{a_t}{b_t}(\dot a_tb_t-a_t\dot b_t),\ \eta_t=\frac{\mu_t}{a^2_t}+\frac{\dot b_t}{b_t}. In practice, ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt>0\int^1_0\mathbb E_\epsilon\mathbb E_u||\dot a_tu+\dot b_t\epsilon-v_\theta(a_tu+b_t\epsilon,t)||^2dt>0 is a small constant after training, vθ(zt,t)v_\theta(z_t,t) approximates the negative gradients of the function ℓt(z)\ell_t(z) with respect to zz.

[Lines 220--221]

Rt(u):=btatℓt(z)+12(1−at)ξt∣∣u−z∣∣2=μtbtatfM0,t(ctz,σt)−ηtbt2at∣∣z∣∣2+12(1−at)ξt∣∣u−z∣∣2.R_t(u):=\frac{b_t}{a_t}\ell_t(z)+\frac{1}{2(1-a_t)\xi_t}||u-z||^2=\frac{\mu_tb_t}{a_t} f_{M_{0,t}}(c_tz,\sigma_t)-\frac{\eta_tb_t}{2a_t}||z||^2+\frac{1}{2(1-a_t)\xi_t}||u-z||^2.

[Divide 0] As almost all generative models (DMs or FMs), we set a0a_0 to be a small positive number for a0=0a_0=0.

[Dual variable]

max⁡Λmin⁡u,v∈M0,t Rt(u)+λtF(v)+⟨Λ,v−u⟩+ρt∣∣v−u∣∣2/2.\max_{\Lambda}\min_{u,v\in M_{0,t}}\ R_t(u)+\lambda_t F(v)+\langle\Lambda,v-u\rangle+\rho_t||v-u||^2/2.

[Final output:] For each tt, we solve the optimization problem for FlowBCD: min⁡u,v∈M0,t Rt(u)+λtF(v)+ρt∣∣u−v∣∣2/2.\min_{u,v\in M_{0,t}}\ R_t(u)+\lambda_t F(v)+\rho_t||u-v||^2/2.

As tt goes from 0 to 1, we solve a sequence of optimization problems. Both ztz_t and wtw_t are intermediate variables. When solving the tt-th optimization problem, we initialize based on the (t−Δt)(t-\Delta t)-th output. Let vtv_t be the (t−Δt)(t-\Delta t)-th output; then the initialization of the tt-th iteration is ut=Pt∘Qt(ut,zt−Δt).u_t=P_{t}\circ Q_t(u_t,z_{t-\Delta t}). Since we need zt−Δtz_{t-\Delta t} to obtain utu_t, we introduce ztz_t as an intermediate vector. The same applies to wtw_t and to the corresponding variables in FlowADMM. The final output is w1=v1w_1=v_1.

[Appendix B] We use the Bayes' formula p(u∣z,t)p(z∣t)=p(u,z∣t)=p(z∣u,t)p(u∣t)=p(z∣u,t)p(u)p(u|z,t)p(z|t)=p(u,z|t)=p(z|u,t)p(u|t)=p(z|u,t)p(u). We also corrected to dz=btdϵdz=b^d_t\epsilon.

[g2(θ)g_2(\theta) not reach 0:] The equality is in the ideal case. In the theorem, we stated the approximation result:

In practice, ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt>0\int^1_0\mathbb E_\epsilon\mathbb E_u||\dot a_tu+\dot b_t\epsilon-v_\theta(a_tu+b_t\epsilon,t)||^2dt>0 is a small constant after training, vθ(zt,t)v_\theta(z_t,t) approximates the negative gradients of the function ℓt(z)\ell_t(z) with respect to zz.

[Q1-3] Q1: λt=1\lambda_t=1, τt=bt\tau_t=b_t, Λ0=0\Lambda_0=0. We add the reference for Q2 and the latent space to Subsection 3.4 for Q3.

[Q4] Quantitative comparisons show that both FlowBCD and FlowADMM achieve better performance. Classical optimization methods for image inverse problems typically report only PSNR or SSIM; here, we additionally adopt FID and LPIPS for evaluation, and all results are reported based on these four metrics.

Thank you for revising the paper and applying fixes such as correcting the Lagrangian formulation as well as the sign of vθv _ \theta. However, some concerns still remain:

  • I respectfully don’t agree that g2(θ)=0g_2(\theta) = 0 holds "in the ideal case", and Lines 707–708 of the revised manuscript still argue: "When g2(θ)=0g_2(\theta) = 0, we have g1(θ)=0g_1(\theta) = 0". This is false even in the ideal, infinite-capacity case. As shown in Lipman et al. (2022), the Conditional Flow Matching loss g2(θ)g_2(\theta) decomposes into the marginal FM loss plus an irreducible conditional variance term Var(a˙tu+b˙tϵ∣z)>0\text{Var}(\dot{a} _ t u + \dot{b} _ t \epsilon \mid z) > 0. Because distinct pairs (u,ϵ)(u, \epsilon) pass through the same noisy point zz at time t∈(0,1)t \in (0, 1), min⁡θg2(θ)>0\min _ \theta g_2(\theta) > 0 strictly. Therefore, bounding 0≤g1(θ)≤g2(θ)0 \le g_1(\theta) \le g_2(\theta) via Jensen's inequality cannot prove that minimizing g2(θ)g_2(\theta) drives g1(θ)g_1(\theta) to 00.

  • In the response above, wtw_t is explained as an initialization vt=Pt(wt)v_t = \mathcal{P} _ t(w_t) with wt=Qt(vt−Δt,wt−Δt)w_t = \mathcal{Q} _ t(v _ {t-\Delta t}, w _ {t-\Delta t}). However, applying the re-noising and Tweedie denoising operators Pt∘Qt\mathcal{P} _ t \circ \mathcal{Q} _ t to vv applies the generative flow prior a second time to vv at every step. This does not follow from optimizing Lagrangian formulation and remains an ad-hoc two-trajectory heuristic rather than a standard BCD/ADMM. NOte that In classical variable splitting (min⁡u,vRt(u)+λtF(v) s.t. u=v\min _ {u,v} R_t(u) + \lambda_t F(v) \text{ s.t. } u = v), the auxiliary variable vv is introduced specifically to decouple the generative prior Rt(u)R_t(u) from the data-fidelity term F(v)F(v). Only uu is regularized by RtR_t, while vv should be updated via ∇vF(v)\nabla_v F(v) and the quadratic penalty.

  • Wall-clock similarity due to VAE decoding overhead does not change the fact that FlowBCD/ADMM get twice as many evaluations of the generative prior (56 vs 28 NFEs). Without running FlowDPS for 56 steps or ablating a 1-NFE variant of FlowBCD without the second trajectory wtw_t, it is impossible to tell whether the PSNR/SSIM gains come from variable splitting or simply from doubling the flow model evaluations .

we are grateful to Reviewer 95A4 for their helpful comments.

[Q1] We delete this statement and update as follows:

Let M\mathcal M be a data manifold. For any ϵ∼N(0,I)\epsilon\sim\mathcal N(0,I) and u∈M\mathcal u\in\mathcal M, define z=atu+btϵ\mathcal z=a_t\mathcal u+b_t\epsilon, where at,bt≥0a_t,b_t\ge0. Let vθ(z,t)v_\theta(\mathcal z,t) be the pre-trained neural network to estimate the velocity field in the flow matching model (see the overview in Appendix A). Set ℓt(z)=μtfM(ctz,σt)−ηt2∣∣z∣∣2\ell_t(\mathcal z)=\mu_t f_\mathcal M(c_t \mathcal z,\sigma_t)-\frac{\eta_t}{2}||\mathcal z||^2. When ct=1at,σt=btat, μt=atbt(a˙tbt−atb˙t), ηt=μtat2+b˙tbt,c_t=\frac{1}{a_t},\quad \sigma_t=\frac{b_t}{a_t},\ \mu_t=\frac{a_t}{b_t}(\dot a_tb_t-a_t\dot b_t),\ \eta_t=\frac{\mu_t}{a^2_t}+\frac{\dot b_t}{b_t}, we have ∇zℓt(z)≈−vθ(z,t),\nabla_\mathcal z\ell_t(\mathcal z)\approx-v_\theta(\mathcal z,t), if ∫01EϵEu∣∣a˙tu+b˙tϵ−vθ(atu+btϵ,t)∣∣2dt>0\int^1_0\mathbb E_\epsilon\mathbb E_\mathcal u||\dot a_t\mathcal u+\dot b_t\epsilon-v_\theta(a_t\mathcal u+b_t\epsilon,t)||^2dt>0 is a small constant after training. That is, vθ(z,t)v_\theta(\mathcal z,t) approximates the negative gradients of the function ℓt(z)\ell_t(\mathcal z) with respect to z\mathcal z.

[Q2] We thank the reviewer for this comment.

In the optimization models (both (4.5) and (4.6)), we require v∈M0,tv\in\mathcal M_{0,t}. That is why we introduce the operators of Pt+Δ(⋅)P_{t+\Delta}(\cdot) and Qt+Δ(⋅,⋅)Q_{t+\Delta}(\cdot,\cdot), which is understood as the projection to M0,t\mathcal M_{0,t}.

We have tried updating v\mathcal v (the vector in the data fidelity term) using the classical optimization approach before, but the numerical results were not good. In the classical optimization, the variable (vector) in the regularization term is updated continuously, whereas the vector may jump from one location to another on the data manifold when we use a regularization term related to flow matching. We conjecture that this discontinuity in the update renders the update of the vector in the data fidelity term ineffective.

We have added a brief discussion of this point to the revised manuscript (Lines 307--312, page 6), and we hope this clarifies why we adopted the proposed scheme instead.

[Q3] We indeed need two forward computations of the neural networks per NFE—once for the vector u\mathcal u and once for the vector v\mathcal v. In addition, there are the computations of the encoder and decoder, since we work in the latent space, as well as the gradient descent for the fidelity term in the imaging inverse problems. Therefore, the count of NFEs alone is not sufficient to characterize the computational complexity.

Our main contribution is to make any gradient-based methods applicable to flow matching models in general—the methods we list are just popular examples, not an exhaustive set.

Reviewer_uWegOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a framework for using flow matching models to solve image inverse problems. The main idea is to connect the velocity field of a pretrained flow model to the gradient of a regularization function. Based on this connection, the authors develop several optimization methods, including FlowTSGD, FlowFTSGD, FlowBCD, and FlowADMM. The paper tests these methods on four image inverse problems across three datasets. The authors report strong results, especially for FlowBCD and FlowADMM.

Critical strengths
  1. A useful connection between generative models and optimization. The paper tries to give a mathematical explanation for using flow matching models as image priors. If correct, this could help connect generative models with classical optimization methods.

  2. Several optimization methods are studied. The authors do not focus on only one algorithm. They develop different optimization methods and compare their performance on several inverse problems. This makes the proposed framework potentially useful beyond a single application.

Critical weaknesses

-The claimed acceleration is not clearly demonstrated. The paper does not report several important experimental settings, including the exact data-fidelity objective, optimization parameters, and how the pretrained flow model is used. It is also unclear whether the experiments use conditional, unconditional, or guided flow predictions. The paper claims that FlowFTSGD provides acceleration, but it does not clearly show how this improves over the original method beyond changing the step size. A direct comparison using the same computational budget would help support this claim.

Questions for authors
  1. Can the authors provide the full experimental settings, including the data-fidelity objective, optimization parameters, scheduler, and number of updates per step?
  2. Can the authors compare FlowFTSGD with FlowTSGD under the same computational budget and explain whether the acceleration step provides benefits beyond changing the step size?
1 comment

[Q1] We thank the reviewer for the comment, we are sorry for any confusion.

There is no training. As you stated in strength, we aim to establish a connection between generative models and optimization and understand flow matching-based imaging inverse problem from an optimization perspective.

In classical optimization methods for imaging inverse problems, we consider min⁡uGt(u)=λtF(u)+Rt(u),\min_u G_t(u)=\lambda_t F(u)+R_t(u), where F(u)F(u) is the data-fidelity objective and Rt(u)R_t(u) is the regularization term.

When solving it with gradient-based methods, we need ∇uGt(u)=λt∇uF(u)+∇uRt(u).\nabla_u G_t(u)=\lambda_t\nabla_u F(u)+\nabla_u R_t(u). In Section 3, we set Rt(u)R_t(u) as a distance-based function show that ∇uRt(u)=−vθ,\nabla_u R_t(u)=-v_\theta, where vθv_\theta is the pretrained velocity field in the unconditional flow matching model.

Now, both ∇uGt(u)\nabla_u G_t(u) and ∇uRt(u)\nabla_u R_t(u) are determined and we can apply gradient-based optimization methods directly.

Setups for FlowBCD and FlowADMM: In solving the classical optimization model, we using the following setups for FlowBCD and FlowADMM, which is deferred in Appendix C.1 due to space limitations.

We set λt=1\lambda_t=1, ζt=20\zeta_t = 20, Λ0=0\Lambda_0=0 and the schedule is given by ρt=ρ0bt\rho_t = \rho_0 b_t with ρ0=40\rho_0 = 40, where btb_t represents the scheduler noise level.

[Q2] The following is the ablation study on τt\tau_t in FlowFTSGD for bicubic ×12 super-resolution on the AFHQ dataset.

τt\tau_tPSNR$\uparrow$SSIM$\uparrow$FID$\downarrow$LPIPS$\downarrow$
τt=1\tau_t = 125.200.73926.650.216
τt=bt\tau_t = b_t26.070.66215.710.188
τt=1−bt\tau_t = 1 - b_t23.570.67132.840.257

The FlowTSGD corresponds to τt=1\tau_t=1. Numerical results show that the introduction of τt\tau_t with an appropriate choice can improve performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.