- Rating
- 3
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors propose FlowSGS, a split Gibbs sampling (SGS) method for posterior sampling in imaging inverse problems with a flow-model prior. FlowSGS generalizes the PnP-DM method (developed for diffusion-model priors) to all stochastic interpolant (SI) models, including flow matching, which improves efficiency. The authors provide theoretical results about how the prior step in SGS is equivalent to solving the SI reverse-time SDE, how PnP-DM is a special case of FlowSGS, and convergence guarantees, although these theoretical results are largely direct extensions of results already established by PnP-DM. The authors validate on several linear inverse problems and one nonlinear inverse problem, showing improved image quality and computational efficiency over existing state-of-the-art SI-based methods for posterior sampling.
Critical strengths
The method itself is extremely sound and could be already practically useful. It is clear that an SGS method that supports any SI prior ranging from a flow-matching prior to a diffusion-model prior would be useful for a variety of inverse problems, and this, to the best of my knowledge, is the first paper to layout a framework for doing so and provide sound validation.
Critical weaknesses
There’s nothing surprising or novel about this paper. The method and theoretical results are obvious (at least in hindsight) extensions of PnP-DM.
Questions for authors
For the result about warmup invariance (Corollary 3), it is confusing what the tradeoff is between efficiency and image quality when using one-step flow models for acceleration. The manuscript states: “We show that this does not affect convergence: any nonpathlogical prior step sampler may be used during warm-up, including one-step flow-map models, and the convergence rate and approximation floor of the subsequent stationary stage are unaffected.” But there must be some tradeoff. Perhaps the answer is in the bound in Equation 9, but it would be nice to include some concrete statements about the tradeoff to aid users in deciding how many warmup prior steps to take.
This is more of a clarification question, but in Figure 4, it is confusing why Mode 2 would even show up in the posterior since it so clearly doesn’t make sense under the prior. Can you provide some intuitive explanation for why this mode appears?
3 comments
Thank you to the authors for responding to the reviewer comments and concerns. They have adequately addressed my questions, including clarifying the differences from PnP-DM and the warmup tradeoff. I'll be happy to raise my score since I believe this is a practically useful paper.
Thank you for your positive feedback and helpful suggestions. The discussions of the warm-up tradeoff and the second mode in Fourier phase retrieval that you suggested add important context, and have significantly strengthened our manuscript.
Thank you for the positive assessment and for describing the method as extremely sound and practically useful.
W: Extension of PnP-DM. We agree the SGS template comes from PnP-DM (Corollary 1), but the extension is not automatic. Please see G1 in the global response.
Q1: Warm-up tradeoff (Corollary 3). You are right that there is one, and it sits in the initialization term of Equation 9, C⋅KL(μw∥πρmin)/(Ksmin(ρmin,δ)2): a cheaper warm-up leaves a larger KL and needs more stationary iterations Ks for the same bound, while the rate and floor are unchanged. With defaults, MFM warm-up costs 0.38 dB at fixed Ks for a 9.6× speedup (Tables 6 and 10). New Figure 15 varies the number of warm-up iterations that use MFM on motion deblurring: as more iterations use MFM, the total NFE drops from 778 to 100, while LPIPS (0.164 to 0.151) stays within the error bars and PSNR within 0.15 dB. This is in accordance with Corollary 3, which states that the warm-up does not affect the final convergence. As already discussed after Corollary 3, the warm-up affects only the bound, leaving the convergence rate and error floor unaffected, and we now also point to Figure 15 there.
As noted in Section 4.3, these guarantees do not apply to nonlinear problems, which explains the MFM failure in FPR, where the non-convex likelihood pushes warm-up iterates off the data manifold and the one-step map fails.
Q2: Why Mode 2 in Figure 4? We agree that rotated images are not seen when we train the prior, but we would like to point out that the rotated mode exists for the following reason, in addition to the symmetry of the Fourier phase retrieval forward model. Rotated faces follow the same natural image statistics as upright faces, and have only a slightly lower prior log-likelihood. To verify this, we measure the prior log-likelihood of face images and their rotated counterparts in Figure 11 (3.395 vs. 3.384 nats per dimension, compared to −0.871 for pure noise). Although not seen during training, the rotated face gets a much larger prior log-likelihood than pure noise, which is not a natural image.
Thank you again for the supportive review.