acceptodds
Under review as a conference paper at ICLR 2027

Vector–Jacobian Endpoint Geometry for Inverse Problems: Exploiting the Local Structure of Flow Predictions

Abstract

Pretrained flow-matching models provide generative priors for inverse problems, but treating their clean predictions only as point estimates leaves local structure unused during measurement correction. We introduce Vector–Jacobian Endpoint Geometry (\VEG), a clean-image correction that uses both measurements and local flow structure to determine where to move, how far to move, and whether to accept the move. We call the local variation among clean endpoints compatible with the current flow state endpoint geometry. uses a clean-prediction vector–Jacobian product for direction, norm matching for proposal scale, and safeguarded line search for step selection. Our analysis characterizes how measurement evidence should shift the predicted clean image among these plausible endpoints, and shows that this shift is shaped by their local variation. Experiments on nonlinear and linear inverse problems show improved reconstruction quality over ordinary clean-prediction correction with modest additional local computation, while controlled interventions demonstrate that endpoint geometry can guide image changes that the measurement alone cannot determine.

Primary area:
generative models
Keywords:
inverse problems, flow matching, generative priors, image restoration, test-time conditioning, Jacobian methods

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_A36sOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on solving inverse problems with flow matching priors. In particular, the authors propose VEG, which replaces the data-gradient step on the clean image with a step preconditioned by the Jacobian of the denoiser (computed with a vector–Jacobian product), and then re-noises to keep running the solver. This is motivated by a first-order analysis showing that the measurement shifts the conditional mean along a covariance-shaped direction, and that for the linear flow-matching path this covariance is proportional to the Jacobian of the denoiser. Experiments on AFHQ-Cat, CelebA and ImageNet (JiT) cover nonlinear blur, HDR and several linear inverse problems.

Critical strengths
  1. The paper connects the clean-image correction to a clear statistical quantity: the first-order change of the conditional mean is a covariance-shaped direction, which for the linear path can be computed with one VJP of the denoiser. Even though the identities are known, putting them together gives a simple and cheap method.

  2. The experiments are extensive (several datasets, operators and pretrained models) and include negative results, although some comparisons are missing (see below). In particular, the comparison with ordinary gradient corrections is convincing: one VEG correction gets higher PSNR than ten ordinary corrections (+0.58 dB) while using 0.64× the runtime, so the gain comes from the direction and not just from more optimization

Critical weaknesses
  1. I believe the main weakness of the paper is the limited novelty. The gradient in eq. (15) is exactly DPS in the flow matching language (also explored in OT-ODE [2]), and Propositions 1 and 2 are well-known results (e.g., Peng et al. 2024 and Boys et al. 2024, both cited in the paper, and [3]). Furthermore, the algorithm is essentially the same splitting scheme as PnP-Flow, DiffPIR [4] and FLOWER, as the side-by-side comparison shows (DtD_t is the denoiser, ε,z∼N(0,I)\varepsilon, z \sim \mathcal{N}(0, I)):

    PnP-Flow (Martin et al., 2025), at each tkt_k:

    1. Data step on the clean image: zk=xk−γk∇ℓy(xk)z_k = x_k - \gamma_k \nabla \ell_y(x_k).
    2. Re-noise: z~k=tkzk+(1−tk) ε\tilde{z}_k = t_k z_k + (1 - t_k)\,\varepsilon.
    3. Denoise: xk+1=Dtk(z~k)x_{k+1} = D_{t_k}(\tilde{z}_k).

    VEG, at each tnt_n:

    1. Data step on the clean image: c~n=cn+αndn\tilde{c}_n = c_n + \alpha_n d_n, with dn=−∥gn∥JD(xn)⊤gn∥JD(xn)⊤gn∥d_n = -\|g_n\| \frac{J_D(x_n)^\top g_n}{\|J_D(x_n)^\top g_n\|}, gn=∇ℓy(cn)g_n = \nabla \ell_y(c_n), and αn\alpha_n from an Armijo line search on UnU_n.
    2. Re-noise: xn+1=tn+1c~n+(1−tn+1)zx_{n+1} = t_{n+1} \tilde{c}_n + (1 - t_{n+1}) z.
    3. Denoise: cn+1=Dtn+1(xn+1)c_{n+1} = D_{t_{n+1}}(x_{n+1}).

    Up to re-indexing the loop, the only differences are the direction of the data step (JD⊤gJ_D^\top g instead of gg, i.e., a gradient step preconditioned by the denoiser Jacobian) and the step-size rule (line search instead of a fixed schedule). The norm matching is not the contribution either, since Table 7 shows that the raw and theory-scaled VJPs perform as well or better. So I think the method is better described as "PnP-Flow/FLOWER with a covariance-preconditioned data step", which is a much narrower claim than "endpoint geometry". From this view, the natural missing comparison is the exact solve in the covariance metric, arg⁡min⁡cℓy(c)+12∥c−cn∥C−12\arg\min_c \ell_y(c) + \frac{1}{2}\|c - c_n\|^2_{C^{-1}}, which for linear hh is the MMPS [3] update cn−CH⊤(HCH⊤+Γ)−1(Hcn−y)c_n - C H^\top (H C H^\top + \Gamma)^{-1}(H c_n - y); similarly, if CC is the right geometry, I'd expect the proximal term in UnU_n to use C−1C^{-1} and not be isotropic. Also, the norm matching throws away the scaling (1−t)2/t(1-t)^2/t of Proposition 2, so the step size is set by the line search and not by the theory (without backtracking the method diverges, Table 7).

    Regarding the writing, the paper spends a lot of time on results and definitions that are not fundamental to the key message, and I found it hard to read. For instance, "endpoint geometry" is just Cov(X1∣xt)\mathrm{Cov}(X_1 | x_t) with a new name, and the algorithm is only given in the appendix (Algorithm 1), although it would directly show what changes with respect to PnP-Flow/FLOWER.

  2. The second main weakness is the experimental part. First, Table 1 doesn't include PnP-Flow, which is as efficient as VEG; its gradient step only needs ∇ℓy\nabla \ell_y, and the authors already adapted DAPS, FLOWER and D-Flow to the nonlinear setting. Second, the results don't show a clear improvement over the baselines: on the nonlinear benchmark (Table 1), VEG doesn't get the best PSNR or LPIPS in any of the four dataset–task groups (e.g., D-Flow is 3.4 dB better on CelebA HDR), and on JiT (Table 16) FLOWER gets higher PSNR in 4 out of 5 tasks. The linear results (Table 2) are more favorable, but use only 10 images per dataset and task. Since the paper is framed as a quality–compute trade-off, the comparison should be against methods with a similar runtime (see Clarification 1), and not mostly against much slower ones. Finally, Table 3 is hard to understand from the main text: the angle-matched random direction is only defined in Appendix B.4, and mixing development and held-out "cohorts" makes the rows hard to compare.

Overall, I am not convinced by the numerical experiments, and I believe the contribution is narrow, which leans me towards rejection.

Questions for authors
  1. The gradient in (6) is w.r.t. ctc_t? or xtx_t?

Clarifications (answers could affect my assessment)

  1. How does VEG compare with FlowDPS [1] or OT-ODE [2] on the main AFHQ-Cat/CelebA benchmark? FlowDPS is only compared on JiT (Appendix E), and since the paper is framed as a quality–compute trade-off, I think methods with a similar runtime are the relevant comparison.

  2. Can you compare against MMPS/TMPD-style [3] updates that actually solve with the Jacobian (e.g., a few CG steps using VJPs)? This would tell us whether the first-order approximation is the right trade-off.

  3. Did you run JAPS on nonlinear blur and HDR? Both papers use these tasks, so a direct comparison seems feasible.

  4. In Section 4.1 you state that, to first order, a noisy-state update (as in DPS) changes the clean image by −JDJD⊤g-J_D J_D^\top g (≈ −C2gˉ-C^2 \bar{g}), while the clean-image correction gives −JD⊤g-J_D^\top g (≈ −Cgˉ-C \bar{g}). This is stated without a derivation or discussion. Furthermore, can you explain intuitively why −Cgˉ-C \bar{g} should be preferred over −C2gˉ-C^2 \bar{g}?

    Also, the noisy-state baseline in Table 3 / Appendix B.5 changes the step size, the line search and the continuation at the same time, so the gain can't be attributed to where this update is applied. Can you provide an experiment where only the placement changes?

[1] Kim, J., Kim, B. S., & Ye, J. C. (2025, October). Flowdps: Flow-driven posterior sampling for inverse problems. ICCV 2025

[2] Pokle, Ashwini, Matthew J. Muckley, Ricky TQ Chen, and Brian Karrer. "Training-free linear image inverses via flows." TMLR

[3] Rozet, François, Gérôme Andry, François Lanusse, and Gilles Louppe. "Learning diffusion priors from observations by expectation maximization." NeurIPS 2024

[4] Zhu, Yuanzhi, Kai Zhang, Jingyun Liang, Jiezhang Cao, Bihan Wen, Radu Timofte, and Luc Van Gool. "Denoising diffusion models for plug-and-play image restoration." CVPR 2023

8 comments

Dear Reviewer A36s,

Thank you for your detailed review. We have expanded baseline comparisons, clarified the theoretical interpretation, and improved the presentation of controlled experiments. We address your questions first, followed by the broader concerns.

Q1: Gradient Definition

The gradient in Eq. (6) is taken with respect to the predicted clean image ctc_t, not the flow state xtx_t. Specifically, gt=∇ctℓy(ct)g_t=\nabla_{c_t}\ell_y(c_t), whereas Eq. (15) computes JDt(xt)⊤gt=∇xtℓy(Dt(xt))J_{D_t}(x_t)^\top g_t=\nabla_{x_t}\ell_y(D_t(x_t)) by the chain rule. We clarified this in Sections 2.2 and 4.1.

Clarification 1: Comparison with FlowDPS and Similar-Compute Methods

VEG achieves higher reconstruction quality than FlowDPS and PnP-Flow on the main nonlinear benchmark, including a substantial improvement over PnP-Flow at comparable runtime. We evaluated FlowDPS [1], PnP-Flow [2], and JAPS [3] under the shared pretrained OT-flow prior on 120 held-out cases, averaged equally across four dataset–task groups.

MethodPSNR ↑LPIPS ↓Time (s) ↓
VEG26.0390.159017.11
FlowDPS-NL [1]24.1530.22476.05
PnP-Flow-NL [2]23.4460.246420.06
JAPS (adapt.) [3]26.3660.112245.73
D-Flow-NL27.7260.0840333.74

VEG improves mean PSNR by 1.886 dB over FlowDPS-NL and 2.593 dB over PnP-Flow-NL, with lower LPIPS in both comparisons. FlowDPS-NL is faster, while PnP-Flow-NL has comparable runtime. JAPS and D-Flow achieve higher fidelity but require 2.67× and 19.5× the mean runtime, respectively. These results support a favorable quality–compute trade-off rather than uniform superiority. Tables 1–2 report the full results, with adaptations in Appendix D.1.

Clarification 2: Comparison with MMPS/TMPD-Style Jacobian Solves

The benefit of exploiting the flow-derived covariance structure depends on how it is used, rather than simply on the number of Jacobian computations. We examine this through iterative corrections motivated by MMPS [4]. We first evaluated Conjugate Gradient (CG) with one and three iterations using a clean-prediction Jacobian covariance proxy. On 60 paired linear cases, VEG achieves 29.699 dB PSNR with 99 predictor VJPs, compared with 29.488 dB using 198 VJPs and 29.195 dB using 396 VJPs for CG-1 and CG-3. However, the learned Jacobian does not guarantee a symmetric positive-definite system, limiting standard CG. We report details in Appendix B.8 (Table 13).

To address this limitation, we further compare against GMRES [5], which handles nonsymmetric linear systems. MMPS likewise replaces CG with GMRES in its diffusion benchmark because learned denoiser Jacobians produce nonsymmetric, non-definite systems. We adapt this measurement-space formulation to the frozen flow predictor. We construct a measurement-space system using the clean-prediction Jacobian as a covariance proxy, combined with the local measurement Jacobian and observation-noise covariance. Let ct=Dt(xt)c_t=D_t(x_t), Jt=JDt(xt)J_t=J_{D_t}(x_t), Ht=Jh(ct)H_t=J_h(c_t), χt=(1−t)2/t\chi_t=(1-t)^2/t, and Γ=(σy2+10−6)I\Gamma=(\sigma_y^2+10^{-6})I. We define

M_t(u)&=\chi_tJ_t^\top u,\\\\ S_t(q)&=\Gamma q+H_tM_t(H_t^\top q),\\\\ q_t^{(k)}&=\operatorname{GMRES}_k(S_t,y-h(c_t);0),\\\\ \delta_t^{(k)}&=M_t(H_t^\top q_t^{(k)}),\quad k\in\{1,3\}. \end{aligned}$$ **GMRES solves the measurement-space system approximately using the observation residual, then maps the solution to a clean-image correction through $H_t^\top$ and $M_t$.** Additional iterations can refine its direction through further Jacobian products. We match VEG's proposal norm and retain its descent test, Armijo search, and noise-mixing continuation. The results cover **120 paired nonlinear cases**, with runtime measured on the same RTX 5080 GPU. | Method | PSNR ↑ | LPIPS ↓ | Time (s) ↓ | Predictor VJPs ↓ | |---|---:|---:|---:|---:| | **VEG** | 26.037 | 0.16050 | **15.51** | **99** | | GMRES-1 | 26.009 | 0.15983 | 26.83 | 297 | | GMRES-3 | **27.396** | **0.12685** | 38.65 | 495 | GMRES-1 produces a correction collinear with the original VJP, up to sign, without resolved quality improvement. **GMRES-3 improves PSNR by 1.359 dB and reduces LPIPS by 0.03365**, but requires **2.49× runtime and five times the predictor VJPs**. **These results suggest that effective use of the Jacobian-based covariance proxy can improve reconstruction, whereas more Jacobian computations alone do not guarantee improvement.** VEG remains a lower-compute alternative. Neither comparison constitutes an exact posterior-covariance solve or a reproduction of native MMPS/TMPD. We added these results and details in Appendix B.9 (Table 14).

Clarification 3: JAPS on Nonlinear Blur and HDR

We directly evaluated JAPS [3] on both nonlinear blur and HDR saturation using the same frozen OT-flow prior, images, and observations as VEG. As shown in Clarification 1, JAPS achieves slightly higher mean PSNR and lower LPIPS but requires 2.67× the mean runtime. On nonlinear blur, VEG achieves comparable PSNR at substantially lower computational cost. The comparison retains JAPS's sampling procedure while adapting its denoiser interface to the shared flow predictor. Detailed results are reported in Table 1, with implementation details in Appendix D.1 and a methodological comparison in Appendix B.7 (Table 12).

Clarification 4: DPS Relation and Correction Placement

VEG and DPS use proportional VJP guidance directions under the shared flow formulation, but they are not identical correction algorithms. Under affine re-noising, VEG's clean-image correction induces a proportional next-state VJP update, while finite-step scaling and acceptance can produce different corrections and trajectories. The previously discussed JJ⊤gJJ^\top g term instead describes same-time predictor reevaluation:

=-\alpha JJ^\top g+o(\alpha).$$ **This is not the actual DPS next-state update.** We therefore removed Proposition 3 and revised the DPS interpretation in Section 4.1. **Direct clean-image correction substantially reduces computation, with no statistically resolved PSNR difference in the matched placement comparison.** To isolate correction placement, we compare direct correction $c_n+\alpha d_n$ with state reevaluation $D_{t_n}(x_n+\alpha d_n)$, keeping the direction construction, local objective, Armijo policy, fallback, and noise-mixing continuation with shared Gaussian noise draws fixed. | Correction placement | PSNR ↑ | LPIPS ↓ | Time (s) ↓ | |---|---:|---:|---:| | Direct clean correction | 25.379 | 0.1627 | **15.83** | | Same-time state reevaluation | 25.450 | **0.1560** | 62.97 | On 40 development cases, the PSNR difference is unresolved, while state reevaluation achieves lower LPIPS at **3.98× runtime** because each line-search candidate requires predictor reevaluation. Thus, the advantage of direct correction is computational efficiency rather than intrinsically better reconstruction quality. We clarified this comparison in Section 6.3 and report the controlled results in Appendix B.5 (Table 10). **The same-sampler DPS comparison further shows that proportional VJP directions may not yield identical reconstruction performance.** VEG improves PSNR by **2.362 dB over DPS and 1.552 dB over DPS+Armijo**, with lower LPIPS and measurement NRMSE. These gains are consistent with differences in correction scaling and acceptance rather than a distinct VJP direction. We added these comparisons to Table 5 and detailed the baseline implementations in Appendix D.1. # W1: Novelty and Meaning of Endpoint Geometry **Our contribution is to characterize how measurements change the conditional mean of clean endpoints in a flow model and validate the resulting correction direction, rather than to introduce a new VJP or splitting procedure.** Endpoint geometry refers to the local variation of clean endpoints conditioned on the flow state, $q_t(x_1\mid x_t)$, characterized by its covariance $C_t$. Incorporating measurement $y$ reweights this distribution toward $p(x_1\mid x_t,y)$. Section 3 characterizes the resulting first-order mean change as $m_t'(0)=-\operatorname{Cov}_{q_t}(X_1,\ell_y(X_1))\approx-C_t\bar g_t$, which is proportional to the clean-prediction VJP under the ideal independent-Gaussian flow. While the underlying statistical and Jacobian identities are known, **we connect them to measurement-guided endpoint correction and investigate the reconstruction information carried by this direction.** **Tables 3 and 4 examine whether the flow-derived direction provides useful information beyond ordinary measurement descent and how that information affects reconstruction.** Table 3 tests whether its benefit comes from the direction itself, rather than correction magnitude, alignment with ordinary descent, or additional refinement. Table 4 investigates whether the direction can guide reconstruction in measurement-insensitive regions, showing improvements in missing or saturated regions where the measurement gradient vanishes, even when alternative corrections yield identical measurement predictions. **We substantially revised Section 6.3** to clarify the motivation, controlled conditions, and interpretation of both experiments, with further details in Appendices B and C.

Cont.

W1: Norm Matching, Theoretical Scale, and Proximal Metric

The theoretical covariance interpretation motivates the correction direction, while norm matching and line search control its finite-step implementation. The factor (1−t)2/t(1-t)^2/t relates the ideal conditional covariance to the clean-prediction Jacobian but does not prescribe the optimal step size for a learned predictor. Raw, theory-scaled, and norm-matched VJPs achieve similar PSNR with backtracking, whereas removing backtracking can cause divergence. Thus, norm matching is a practical safeguard rather than an independent source of reconstruction gains.

Likewise, the isotropic proximal penalty controls the correction magnitude rather than approximating the covariance-weighted metric Ct−1C_t^{-1}. We clarified these roles and the independent-Gaussian assumptions in Sections 3.3–4.2 and Appendix A, with the scaling ablation reported in Appendix B.4 (Table 8).

Writing and Presentation Improvements

We substantially revised the manuscript to clarify the central idea and improve the presentation of the experimental evidence. Specifically, we clarified endpoint geometry in Sections 2–3, moved Algorithm 1 into Section 4.2, and substantially revised Section 6.3 to explain the motivation and controlled conditions of Tables 3–4, including the angle-matched random control and the distinction between development and held-out evaluations.

[1] Kim, J., Kim, B. S., and Ye, J. C. FlowDPS: Flow-Driven Posterior Sampling for Inverse Problems. ICCV, 2025.

[2] Martin, S., Gagneux, A., Hagemann, P., and Steidl, G. PnP-Flow: Plug-and-Play Image Restoration with Flow Matching. ICLR, 2025.

[3] Hen, L., Tirer, T., Giryes, R., and Abu-Hussein, S. Jacobian-Aware Posterior Sampling for Inverse Problems. TMLR, 2026.

[4] Rozet, F., Andry, G., Lanusse, F., and Louppe, G. Learning Diffusion Priors from Observations by Expectation Maximization. NeurIPS, 2024.

[5] Saad, Y., and Schultz, M. H. GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems. SIAM Journal on Scientific and Statistical Computing, 1986.

Thanks,

Authors

Reviewer_NBWWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes VEG, a training-free method for image inverse problems using pretrained flow models. VEG computes a vector–Jacobian product through the clean-image predictor and directly corrects the predicted clean image, with norm matching, descent checks, and line search controlling the update. The theoretical analysis connects measurement-induced changes in the conditional endpoint mean to conditional covariance and the predictor’s Jacobian, providing a statistical interpretation of the correction direction.

Critical strengths

1.Coherent and informative theoretical analysis.

The authors first characterize the conditional endpoint mean’s response to measurement-likelihood tilting, then obtain a covariance-weighted measurement gradient through local linearization. Using an existing Jacobian–covariance relationship, they motivate applying the VJP correction in clean-image space. The paper distinguishes exact identities, local approximations, and ideal-model assumptions. This analysis contributes useful understanding beyond a purely algorithmic combination.

2.Careful controlled experiments and mechanism analysis.

The paper includes controls for ordinary-gradient refinement depth, restarted line searches, and angle-matched random directions. The 400-case extension supports persistent PSNR improvements over ordinary refinement. Interventions that retain, remove, or reverse measurement-indifferent update components also provide evidence that the model’s local structure contributes useful reconstruction information.

Critical weaknesses

1.The connection between the theoretical assumptions and the primary experimental models remains insufficiently explained.

The Jacobian–covariance relationship assumes independence between the Gaussian source and clean images, whereas Appendix D.1 states that the primary experiments use OT-coupled checkpoints. Although the paper discusses neural approximation errors, this difference in training coupling is a distinct issue requiring clarification. It remains unclear whether the direction computed from these checkpoints approximates conditional covariance or should be interpreted as an empirically useful correction motivated by an ideal model. This limits how directly the primary experiments validate the covariance interpretation.

2.Limited empirical significance relative to existing solvers.

The controlled experiments support the effectiveness of the proposed direction over ordinary clean-prediction correction. However, the benefits over established solvers remain substantially task dependent. In Table 2, FLOWER is consistently faster and achieves better reconstruction quality on AFHQ Gaussian deblurring, while the PSNR improvement on CelebA inpainting is small despite substantially higher runtime. Although the super-resolution gains are notable, given the incremental nature of the method, I find the demonstrated practical benefits insufficient to establish a sufficiently significant contribution.

Questions for authors

1.How should the independence assumption in Proposition 2 be reconciled with the OT-coupled checkpoints used in the primary experiments? Which aspects of the covariance interpretation remain valid for these checkpoints, and which should be treated only as motivation under an ideal model?

2.The paper already compares VEG and JAPS at the construction level. Could the authors provide an empirical comparison or an ablation under comparable settings to clarify the practical benefits of these construction differences?

4 comments

Dear Reviewer NBWW,

Thank you for recognizing the theoretical analysis and controlled mechanism experiments. We have clarified the theoretical assumptions and strengthened the empirical comparisons to address your concerns.

W1 + Q1: Independence Assumption and OT-Coupled Checkpoints

The independent-Gaussian assumption is required for the Jacobian–covariance identity, but not for characterizing how measurements change the conditional endpoint mean. Proposition 1 gives the exact first-order change under likelihood reweighting without requiring independent flow coupling. Its local approximation yields the covariance-shaped direction −Ctgˉt-C_t\bar g_t, while Proposition 2 relates this direction to the clean-prediction VJP only under the ideal independent-Gaussian flow path. For OT-coupled neural predictors, the VJP serves as a theoretically motivated surrogate rather than an exact conditional covariance estimate. We clarified this distinction in Section 3 and Appendix A.

However, the reconstruction benefit of the VJP direction persists under OT-coupled flows. We compare pretrained flow checkpoints trained with optimal-transport (OT) coupling against separately pretrained checkpoints using independent noise–image pairing. Within each coupling, we evaluate VEG relative to ordinary measurement-gradient correction, where KK denotes the maximum number of corrections per flow interval. All comparisons use the same 120 nonlinear cases.

CouplingOrdinary controlΔPSNR (dB) ↑ [95% CI]ΔLPIPS ↓
OTK = 1+1.069 [0.828, 1.334]−0.03042
OTK = 10+0.576 [0.405, 0.749]+0.00012
IndependentK = 1+1.971 [1.361, 2.545]−0.04946
IndependentK = 10+0.858 [0.301, 1.383]−0.01715

Although the gains are larger under independent coupling, VEG retains its reconstruction advantage under OT coupling, supporting the practical usefulness of the flow-derived direction beyond the ideal theoretical assumptions. These results are reported in Appendix D.5 (Table 20).

Cont.

W2 + Q2: Practical Significance and Direct JAPS Comparison

As requested, we included a direct empirical comparison with JAPS [3] under the same frozen OT-flow prior, images, and observations. The following results from Tables 1–2 compare both methods across nonlinear and linear tasks. Each entry reports PSNR (dB) ↑ / LPIPS ↓ / runtime (s) ↓.

Nonlinear reconstruction (30 held-out cases per dataset–task group)

DatasetMethodNonlinear blurHDR saturation
AFHQ-CatVEG26.254 / 0.33897 / 21.8126.452 / 0.12034 / 9.10
JAPS (adapt.)26.245 / 0.14734 / 80.7226.783 / 0.16614 / 7.55
CelebAVEG27.734 / 0.10978 / 26.6123.715 / 0.06694 / 10.92
JAPS (adapt.)27.597 / 0.07353 / 85.6324.840 / 0.06179 / 9.01

Linear reconstruction (10 paired cases per dataset and task)

DatasetMethodGaussian deblur4× SRBox inpainting
AFHQ-CatVEG28.683 / 0.32556 / 9.0228.340 / 0.20883 / 8.7327.233 / 0.05934 / 8.76
JAPS (adapt.)27.517 / 0.13707 / 6.9420.627 / 0.39287 / 6.9026.182 / 0.04972 / 6.90
CelebAVEG35.064 / 0.03522 / 10.8927.362 / 0.09019 / 10.4731.876 / 0.01761 / 10.70
JAPS (adapt.)34.855 / 0.01704 / 9.0314.524 / 0.34836 / 9.0032.541 / 0.01414 / 8.98

The comparison reveals task-dependent quality–compute trade-offs. On nonlinear blur, VEG achieves comparable PSNR to JAPS at substantially lower runtime, while JAPS achieves lower LPIPS. Across all nonlinear cases, JAPS provides higher mean fidelity but requires 2.67× the average runtime. On linear tasks, VEG achieves higher PSNR on both super-resolution settings without retuning, while JAPS is faster and the remaining quality comparisons are mixed.

The broader comparisons further demonstrate VEG's practical benefits. On nonlinear tasks, VEG achieves 1.886 dB and 2.593 dB higher mean PSNR than FlowDPS-NL and PnP-Flow-NL, respectively, with lower LPIPS. FlowDPS-NL is faster, while PnP-Flow-NL has comparable runtime. D-Flow-NL achieves higher fidelity but requires approximately 19.5× the mean runtime. These results support a favorable quality–compute trade-off rather than uniform superiority. Beyond JAPS, we added FlowDPS [1] and PnP-Flow [2] in Tables 1–2, with adaptation details in Appendix D.1 and the methodological comparison with JAPS in Appendix B.7.

[1] Kim, J., Kim, B. S., and Ye, J. C. FlowDPS: Flow-Driven Posterior Sampling for Inverse Problems. ICCV, 2025.

[2] Martin, S., Gagneux, A., Hagemann, P., and Steidl, G. PnP-Flow: Plug-and-Play Image Restoration with Flow Matching. ICLR, 2025.

[3] Hen, L., Tirer, T., Giryes, R., and Abu-Hussein, S. Jacobian-Aware Posterior Sampling for Inverse Problems. TMLR, 2026.

Thanks,

Authors

Thank you for the detailed response and for adding the direct comparison with JAPS. I find the geometric motivation behind VEG interesting, and the gains over JAPS on 4× super-resolution are substantial. On AFHQ Gaussian deblurring, however, FLOWER achieves better reconstruction quality in less time. On CelebA inpainting, VEG achieves only a small PSNR gain at a considerably higher computational cost. My main concern is how consistently VEG can deliver meaningful improvements across tasks when both reconstruction quality and runtime are taken into account.

Dear Reviewer NBWW,

Thank you for clarifying your remaining concern.

Our expanded evaluation demonstrates a meaningful quality–runtime tradeoff across multiple tasks. We updated Tables 1,2 to 100 images per dataset and task, resulting in some changes to the reported averages while preserving the main findings. VEG achieves the highest PSNR in four of six linear settings. On 4× SR, it improves PSNR over FLOWER by 3.06 dB on AFHQ-Cat and 4.83 dB on CelebA, at approximately 3.4× and 2.6× runtime, respectively. On nonlinear tasks, VEG achieves higher PSNR and lower LPIPS than the faster FlowDPS across all four settings, while the higher-fidelity D-Flow requires approximately 21× VEG's mean runtime.

Our new analysis further identifies when these gains are particularly valuable. VEG achieves the highest PSNR among the evaluated published methods at 8× SR and at both 10% and 25% block compressed sensing on both datasets. As explained in our General Comment, VEG is especially promising when limited observations retain informative relationships with unconstrained image components.

These results, presented in Tables 1,2 and 5, Section 6.4, and Appendix C.6, demonstrate substantial practical benefits while acknowledging task-dependent quality–runtime tradeoffs.

Reviewer_fLdSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces VEG, a training-free a method to apply the likelihood update for inverse problems using the vector jacobian product of the clean-prediction. Similar to PnP-Flow it applies the correction to the clean prediction and renoises to continue sampling. Other in this family correct the prediction with the plain measurement gradient, which treats the prediction as a point estimate. Usually tge correction is the gradient step, ∇l y(ct). VEG instead backpropagates that gradient through the model’s “clean-prediction map”, JDt(xt)^T ∇ly(ct). It then rescales the result to the same norm and only accepts it after a descent test and a backtracking line search. That costs one extra backward pass (VJP) per step.

Critical strengths
  1. Clean theory and explanation of why the correction along J_{Dt}(xt)^T g_t makes sense and how it relates to DPS.
  2. Relatively cheap and can be plugged in easily: It adds one backward pass per step, with no retraining and no backpropagation through the trajectory.
  3. Fair control experiments/ablations. Direction is decoupled from the matched norm, matched optimization steps. Especially the keep vs remove or reverse experiment is interesting and shows that the measurement independent directions also benefit from the VEG update.
Critical weaknesses
  1. Gains in PSNR compared to ordinary gradient baseline, but no numbers for LPIPS. Against established methods the results are mixed.
  2. Limited novelty. The direction is the DPS gradient as acknowledged in 4.1 and the re-noising is also done in FLOWER. Also re-noising is linear in the corrected clean image so the VEG update can be also phrased as a DPS-style update of the next noisy state. The only difference is the step size. I think correcting the clean prediction is should be the same as DPS with a scaled DPS gradient. An experiment with DPS using the same re-noising sampler with DPS standard step rule would be helpful. And another version with the proposed line search would be helpful as well.
Questions for authors
  1. As mentioned above: An experiment with DPS using the same re-noising sampler with 1. DPS standard step rule and 2. with the line search would be helpful.
  2. Table 3 is very insightful, but it is missing LPIPS results.
4 comments

Dear Reviewer fLdS,

Thank you for your constructive review. We have strengthened the paper with matched-sampler DPS comparisons, perceptual-quality results, and additional evaluations against established methods. We provide point-by-point responses below.

W2 + Q1: Contribution Relative to DPS and Same-Sampler Comparison

Our central contribution is to characterize how measurements change the conditional mean of clean endpoints in a flow model, rather than to introduce a new VJP operator. While DPS also uses the VJP for measurement guidance, Section 3 derives the first-order change in this conditional mean under likelihood reweighting and shows that it is approximately covariance-shaped, with a direction proportional to the clean-prediction VJP under the ideal independent-Gaussian flow path. This interpretation explains how local flow structure can guide reconstruction beyond the ordinary measurement gradient.

The experiments confirm that the flow-derived direction provides useful information beyond measurement fitting. Table 3 shows that it improves reconstruction over the ordinary gradient under otherwise identical conditions. More importantly, Table 4 shows that it guides image changes in missing or saturated regions, where the measurements cannot distinguish different corrections. These changes improve reconstruction despite identical measurement predictions at the same state, as the clean-prediction Jacobian couples image components, allowing the learned prior to guide corrections where the measurement gradient vanishes. We substantially revised Section 6.3 to clarify the purpose and interpretation of both experiments, with details in Tables 3–4 and Appendices B–C.

Sharing the VJP does not imply identical reconstruction performance, as the finite-step correction procedures can differ. Under affine re-noising, VEG's clean-image VJP correction can indeed be expressed as a rescaled next-state VJP update. We therefore compare VEG against standard DPS and DPS+Armijo using the same pretrained flow and affine re-noising sampler, with both DPS variants using the same initial guidance scale. Across 120 held-out nonlinear cases, the results are:

MethodPSNR (dB) ↑LPIPS ↓Measurement NRMSE ↓
VEG26.0390.15900.0875
DPS (adapt.)23.6770.22520.1135
DPS (adapt.) + Armijo24.4870.21670.1044

VEG improves PSNR by 2.362 dB over standard DPS and 1.552 dB over DPS+Armijo, with lower LPIPS and measurement NRMSE in both comparisons. These gains reflect differences in the overall correction procedures, rather than the use of a different VJP direction. DPS uses residual-norm guidance, while DPS+Armijo adds backtracking but retains DPS's guidance scaling and omits VEG's ordinary-gradient fallback. Thus, the same VJP can lead to different accepted corrections and trajectories. We clarify this distinction in Section 4.1 and report the comparisons in Table 5 (Section 6.3), with details in Appendix D.1.

Cont.

W1 + Q2: Perceptual Quality and Comparisons with Existing Methods

VEG improves both PSNR and perceptual quality over matched ordinary correction, although its perceptual advantage is not uniform across correction depths. With one correction, VEG improves PSNR by 1.069 dB and reduces LPIPS by 0.03042, with both paired 95% confidence intervals excluding zero. Against ten ordinary corrections, VEG retains a 0.576 dB PSNR advantage at 0.642× runtime, while the LPIPS difference is statistically unresolved. The larger 400-case evaluation also confirms PSNR gains of 0.929 dB and 0.474 dB over two and ten ordinary corrections, respectively. As requested, these results are reported in Table 3 and Appendix D.3.

Compared with published solvers, VEG offers a favorable quality–compute trade-off rather than uniform superiority. On the nonlinear benchmark, VEG achieves 1.886 dB and 2.593 dB higher mean PSNR than FlowDPS-NL and PnP-Flow-NL, respectively, with lower LPIPS. FlowDPS-NL is faster, while PnP-Flow-NL has comparable runtime. Conversely, JAPS (adapt.) and D-Flow-NL achieve higher mean PSNR, but require 2.67× and 19.5× the mean runtime, respectively. On linear tasks, VEG transfers without retuning and is particularly effective for super-resolution. We additionally included FlowDPS [1], PnP-Flow [2], and JAPS [3] in the nonlinear benchmark, and FlowDPS and JAPS in the linear benchmark. Tables 1–2 report their PSNR, SSIM, LPIPS, and runtime, with adaptation details in Appendix D.1.

[1] Kim, J., Kim, B. S., and Ye, J. C. FlowDPS: Flow-Driven Posterior Sampling for Inverse Problems. ICCV, 2025.

[2] Martin, S., Gagneux, A., Hagemann, P., and Steidl, G. PnP-Flow: Plug-and-Play Image Restoration with Flow Matching. ICLR, 2025.

[3] Hen, L., Tirer, T., Giryes, R., and Abu-Hussein, S. Jacobian-Aware Posterior Sampling for Inverse Problems. TMLR, 2026.

If you have any question, please feel free to let us know.

Thanks,

Authors

Thank you for the additional experiments. They show that with affine renoising, VEG is a rescaled DPS update of the next state (which is now stated in Sec. 4.1), and that correcting the clean image instead of the state does not improve quality (Table 10). So the only real difference to DPS is the step-size rule. I think this makes the method contribution too narrow. The theory in Section 3 explains the direction, which DPS shares, so it does not explain the gain over DPS. The paper does not explain why a well-tuned DPS should not reach similar results.

Dear Reviewer fLdS,

Thank you for clarifying your concern. We agree that VEG shares the VJP direction with DPS and that affine re-noising enables an equivalent rescaled state update. As discussed in our previous response, differences in finite-step scaling and acceptance contribute to the observed performance differences.

We have therefore strengthened the paper by characterizing what useful reconstruction information this shared direction provides and when it matters. The VJP uses learned relationships between image components to guide changes that measurements alone cannot determine. Our controls show that simply interpolating observed-pixel corrections does not reproduce this benefit.

Our key finding is that VEG is especially promising when limited observations retain informative relationships with unconstrained image components. The newly updated super-resolution analysis shows that joint correction becomes more valuable as observations become sparser, while block compressed sensing extends this finding beyond pixel subsampling.

As detailed in our General Comment, this characterization provides insight into the shared VJP direction beyond VEG's particular step-selection procedure. The analysis is presented in Sections 6.3–6.4 and Appendices C.5–C.6.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response Regarding Clarified Contributions and New Evidence

Dear Reviewers,

Thank you for your constructive feedback. We have strengthened and clarified the paper's contribution by investigating why VEG performs particularly well in super-resolution (SR), when its correction is useful, and whether the benefit extends to other inverse problems.

Motivated by our strong 4× SR results, we extended the evaluation to 2× and 8× SR. VEG achieves the highest PSNR and lowest LPIPS among the evaluated published methods at all three scales on both datasets, with particularly large gains at 8×.

We investigated why VEG excels under sparse observations. VEG uses the pretrained flow's local structure to translate measurement residuals into coordinated corrections of observed and unobserved image content. Our controls show that correcting observed pixels alone can also improve unobserved content through later flow predictions, but this becomes less effective as observations become sparser. Our findings suggest that VEG is especially promising when limited observations retain informative relationships with unconstrained image components.

We further tested this principle beyond SR using block compressed sensing (CS). Here, measurements combine multiple pixels, leaving ambiguities in their values rather than simply missing pixels. VEG uses the learned relationships to guide corrections of these ambiguous components, achieving the highest PSNR among the evaluated published methods at both 10% and 25% measurement rates.

TaskAFHQ-Cat: VEG / best baselineCelebA: VEG / best baseline
2× SR32.13 / 31.6933.53 / 32.54
4× SR (100 images)27.93 / 24.8727.52 / 24.63
8× SR23.73 / 16.8121.94 / 16.27
Block CS, 10%28.98 / 27.9130.28 / 28.62
Block CS, 25%31.89 / 31.2133.84 / 32.97

Finally, controls that preserve measurement predictions or replace the learned correction with equally sized fixed interpolation further confirm the importance of the selected direction.

These findings are incorporated into Sections 6.3–6.4, Tables 2 and 5, and Appendices C.5–C.6.

Thanks,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.