- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Flow-based generative methods usually find a mapping from a simple prior (often a Gaussian distribution) to the data distribution. This paper proposes to reuse pretrained flow models to define probability-preserving stochastic dynamics, allowing the mapping of points sampled from or near the learned data distribution. To do so, they propose two schemes: 1) dMALA, which uses denoising and Metropolis-adjusted Langevin dynamics, and 2) a predictor–corrector scheme consisting of a prediction step, i.e., noising via the bridge (interpolation) used by the underlying flow model, followed by a correction step: running the pretrained flow from the noised version back to a data sample, either via numerical integration of the ODE or using a flow map model. This allows pretrained models to be used to refine corrupted data samples or explore the data manifold from predefined states.
They establish invariance properties for the idealized kernels and provide a Wasserstein perturbation bound to study approximation errors. The theoretical development seems partially supported by experiments on synthetic distributions, Flowers-102, and ImageNet demonstrating promising results
Critical strengths
- The problem seems well motivated, and the paper is well written, making the theory easy to follow.
- The theoretical results appear correct under the stated assumptions.
- The experiments cover several settings and demonstrate promising results for sample refinement.
Critical weaknesses
- It is hard to judge the novelty and advantages of the method. I think it would help to move the related work section to the main paper to better motivate and contextualize the problem. The experiment section would benefit from comparing with related methods, such as SDEdit and Restart Sampling (which is currently not cited). This can be discussed, but as things stand, it is not clear to me how the proposed method differs from or improves upon these existing methods (although I agree the problems aren't exactly the same). Is the main contribution the probability-preservation interpretation, or does the proposed method also offer practical advantages in terms of sample quality, diversity, or computational cost? A more direct comparison would help clarify this point.
- Although mentioned in passing in Appendix C.4.3, it isn't clear to me that the theory holds in the ImageNet case, nor that the algorithm instantiated in this setting corresponds to the method developed theoretically in Section 2. Given the effort and emphasis placed on the theory, this disconnect seems like a potential issue. Is there a way to verify the assumptions, even empirically, in this setting? Or to justify the instantiated method and its acceptance rule?
Questions for authors
- Could you clarify the main differences between the problem at hand and related methods and what is the advantage of the proposed method ?
- Could you touch on the difference between the method employed in the ImageNet case and the one described in Section 2. Does the theory hold in this case? Can it be shown empirically?
- How do you explain the deterioration of FID over longer dMALA chains, intuitively, in the limit where we evaluate FID on many samples, shouldn't we roughly preserve FID if the data distribution is conserved ?