- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the reliability of importance weights when diffusion/flow (ODE-based) models are used as few-step proposals for self-normalized importance sampling (SNIS), such as in Boltzmann generators. The paper separates three sources of error:
- O1: discretization error in the log weight
- O2: error between the likelihood returned by the implementation and the exact likelihood of the deployed K-step map.
- O3: the resulting bias in SNIS estimates.
Experiments include synthetic GMMs, iDEM samplers, MeanFlow, FALCON.
Critical strengths
- The problem is important. Few-step flow/diffusion models are attractive for sampling, but their likelihoods usually no reliable for importance sampling.
- The authors split the question into O1/O2/O3 parts. It makes clear that discretization error, likelihood error, and SNIS bias are analyze seperately.
- The result for O1 is clean. The Richardson-based certificate is reference-free, and Theorem 2 gives explicit conditions for soundness and asymptotic tightness. Proposition 3 gives a useful explanation of the limitation of ESS.
Critical weaknesses
-
The strongest theory is not for the error that actually causes SNIS bias. The theorem which is theoretically proved, is mainly for O1. As the paper notes, if the exact density of the discrete K-step map is used, SNIS remains consistent. O2 is the error that can actually bias SNIS. However, O2 is handled mainly through the heuristic diagnostic, without a guarantee. Lemma 4 gives a bias bound if per-sample O2 error bounds are already known, but the paper does not provide a practical way to obtain such bounds. This makes the paper feel like two separate contributions. The experiments also blur this distinction. I also have no idea to estimate/improve the bounds of existing models, e.g., Falcon, Meanflow.
-
For FALCON, the goal is not simply to approximate the continuous ODE with fewer steps. The model learns large, invertible jumps along the flow so that the discrete map itself has a valid likelihood. From this view, closeness to the continuous-flow limit (O1) is not the main question. More relevant questions are: a). How much invertibility error does the learned map have? b). How does this error affect the likelihood and SNIS bias? c). How close is the few-step proposal to the target distribution? These seem closer to the actual failure modes of few-step learned maps. In the FALCON experiment, the paper instead introduces an artificial log-determinant truncation. This tests the proposed O2 diagnostic, but it does not test the main approximation error of FALCON itself, such as the reported cycle residual.
-
The related-work discussion should include several papers: (a) F2D2 (Ai et al., 2025), which jointly distills trajectories and cumulative divergence for few-step sampling and likelihood evaluation. (b) Flow Map Matching (Boffi, Albergo & Vanden-Eijnden, 2024), which is closely related to the flow-map models considered here. (c) Free-form Flows (Draxler et al., 2024), which studies likelihood estimation for approximately invertible learned maps. Authors should introduce the connection and difference with these work.
-
The error analysis leads to little practical algorithmic improvement. The main correction is to subtract the signed from the log weight. However, this only applies to the first-order log-determinant approximation and works only when the series converges. (This setting is also somewhat artificial.) The analysis also does not show how to train few-step maps with smaller error. The practical contribution would be stronger with a theory-guided improvement, such as: (a) an extrapolated or bias-corrected weight; (2) improvement of Falcon training objective.
[1] Ai X, He Y K, Gu A, et al. Joint distillation for fast likelihood evaluation and sampling in flow-based models[C]//International Conference on Learning Representations. 2026, 2026: 44940-44977. [2] Boffi N M, Albergo M S, Vanden-Eijnden E. Flow map matching with stochastic interpolants: A mathematical framework for consistency models[J]. arXiv preprint arXiv:2406.07507, 2024. [3] Draxler F, Sorrenson P, Zimmermann L, et al. Free-form flows: Make any architecture a normalizing flow[C]//International Conference on Artificial Intelligence and Statistics. PMLR, 2024: 2197-2205.
Questions for authors
- Proposition 3(i): a constant likelihood shift does not always “bias nothing.” This is true for self-normalized estimates, but not for unnormalized IS estimates such as partition functions or free-energy differences.
The same as Weaknesses.