- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes GLSM, which retains the standard Gaussian forward diffusion and introduces Lévy jumps with a prescribed measure only in the generative dynamics. Building on the Lévy score of Huang et al. and the identity , it derives:
- a marginal-preserving reverse-time Lévy–Itô SDE (Thm 3.1);
- a denoising objective (DLSM) that learns from standard DSM pairs, with closed-form targets for compound-Poisson and -stable jumps (Thm 3.2);
- an annealed pure-jump Lévy Langevin sampler (Thm 3.3, Alg. 1).
Experiments cover two 2D targets, FID versus NFE on CIFAR-10, CelebA-64 and CIFAR-10-LT, and long-tail metrics on CIFAR-10-LT. All image experiments use compound-Poisson jumps with Gaussian amplitudes.
Critical strengths
S1. The core idea is simple and practical. The Gaussian prior, the DSM data pipeline and standard architectures are reused unchanged; only the regression target and the sampling update differ.
S2. The learning objective is technically well executed. The closed-form conditional targets for compound-Poisson and α-stable jumps, and the Bregman-divergence treatment of the infinite-variance stable target (Appendix B.3), are clean and reusable.
S3. The 2D experiments (Fig. 2, Table 1) show clear gains in heavy-tail and low-mass-mode recovery: sliced W1 is reduced by 9.4× and tail MSLE by roughly 38× on the Cauchy target.
S4. On images, GLSM consistently outperforms the stochastic baselines in the 128–256 NFE range on all three datasets, and the ablations over ladder resolution, updates per level and are reasonably extensive.
Critical weaknesses
W1. (Gap between the headline theorem and the algorithm actually used.) The experimental sampler (Alg. 1) does not use the jump-diffusion reverse SDE of Thm 3.1; it only anneals the pure-jump corrector. Moreover, Figs. 6–7 and Sec. D.3.3 show that GLSM works only in the regime of a fine ladder with a single update per level (, ), whereas few levels with many updates fail for both methods (FID 200–500 for ). Hence the interpretation of the LLC as a corrector that relaxes toward (Thm 3.3) does not describe how the method operates in practice. In this regime the sampler effectively acts as a reverse-time integrator, yet the theorem relevant to that view is never used.
Questions for authors
Q1. Could the authors report FID for the PF-ODE (Heun/RK45), DDIM and DPM-Solver++ on the same NCSN++ checkpoint at 32–256 NFE? Fig. 4 compares only stochastic samplers (Table 9 already includes DDIM and DPM-Solver(++) for coverage), so the fixed-budget quality claim cannot be assessed.
Q2. The compound-Poisson increment on images (Eqs. 21–22) is a Poisson scale mixture of Gaussians with the same mean covariance as SMLD's noise, and the Eq. (18) target is a radially reweighted score . Could the authors isolate the source of the gain at fixed , e.g. , via (a) Lévy-score drift with Gaussian noise, (b) Stein-score drift with compound-Poisson noise, and (c) removing the soft clipping?
4 comments
Response to Reviewer 9Uw4 — Part 2/2
(Q1) Comparison with deterministic diffusion solvers. We thank the reviewer for suggesting stronger deterministic baselines. We have evaluated DDIM, Heun PF-ODE, and DPM-Solver++ using the same frozen NCSN++ VE checkpoint as our main CIFAR-10 experiment. The results are summarized below.
| FID | 32 | 64 | 128 | 256 |
|---|---|---|---|---|
| DDIM / Euler PF-ODE | 10.34 | 8.83 | 8.29 | 8.11 |
| DPM-Solver++ (2M) | 8.29 | 8.04 | 7.96 | 7.97 |
| Heun PF-ODE | 8.14 | 7.97 | 7.93 | 7.94 |
| GLSM (ours) | 81.8 | 34.1 | 10.07 | 6.47 |
These deterministic solvers are specifically designed to accelerate probability-flow ODE sampling through higher-order numerical integration. Their superior performance at very low NFEs is therefore expected. Nevertheless, at 256 NFEs, GLSM achieves an FID of , outperforming Heun () and DPM-Solver++ ().
Our main contribution for this paper is a new stochastic generative framework based on Lévy dynamics, rather than a high-order numerical solver, thus our primary comparisons focus on stochastic samplers. Developing more efficient numerical integration schemes for Lévy dynamics, including both the marginal-preserving reverse SDE in Theorem 3.1 and the annealed LLC in Algorithm 1, is a separate research direction that we are currently pursuing.
(Q2) Ablation of Lévy scores, jump noise, and clipping. Theoretically, to preserve the desired invariant distribution , the drift and noise generator must be matched to satisfy the stationary Fokker-Planck equation: Using a mismatched drift-noise pair generally fails to preserve the target distribution . Numerically, we apply soft clipping to stabilize the drift updates, consistent with the use of clipping in Lévy-based generative models such as LIM/DLPM [6,7].
To further distinguish the contributions of the Lévy score and jump noise, we performed the requested ablations on CIFAR-10 at NFEs ranging from 128 to 2048, using the same trained scores and sampling hyperparameters.
| Configuration | 128 | 256 | 512 | 1024 | 2048 |
|---|---|---|---|---|---|
| Lévy drift + CP noise (ours) | 10.07 | 6.47 | 5.07 | 4.53 | 4.44 |
| Lévy drift + Gaussian noise (a) | 9.47 | 5.24 | 5.27 | 4.76 | 4.55 |
| Stein drift + CP noise (b) | 328.4 | 309.1 | 161.3 | 139.0 | 127.7 |
| GLSM without soft clipping (c) | 370.2 | 365.2 | 330.4 | 7.18 | 4.52 |
The results show that (b) performs substantially worse, while (c) confirms the importance of soft clipping for numerical stability at low NFEs. Interestingly, (a) achieves slightly better FID at 128/256 NFEs, but GLSM outperforms it at 512–2048 NFEs. The similar performance at low NFEs may be attributed to two factors: (1) errors introduced by score learning and numerical clipping; and (2) our compound-Poisson increments are Poisson mixtures of Gaussians with the same covariance as Brownian increments, making similar finite-step performance plausible under our sampling parameters.
However, the finite-step FID comparison alone cannot establish whether (a) preserves the correct target distribution. To further verify the importance of matching the drift and noise, we conducted an experiment on an analytically tractable 2D Gaussian mixture using exact Stein and Lévy scores. Starting from , we fix the corrector time and refine the timestep. At , we obtain:
| GLSM (ours) | (a) | (b) | |
|---|---|---|---|
| Sliced | 0.0051 | 0.0347 | 0.0554 |
(a) and (b) retain substantially larger errors under timestep refinement, which confirms that they fail to preserve the target distribution even with exact scores.
Moreover, the computational overhead of compound-Poisson noise is negligible. On an NVIDIA H200 GPU with batch size 1024 at 128 NFEs, GLSM takes 20.87 ms per image, compared with 20.72 ms for (a), an overhead of only about 0.7%. Thus, there is little computational incentive to sacrifice the correct invariant distribution by replacing the matched Lévy noise with Gaussian noise.
[6] Yoon et al., Score-based Generative Models with Lévy Processes, NeurIPS 2023.
[7] Shariatian et al., Denoising Lévy Probabilistic Models, ICLR 2025.
Thank you for the detailed responses and additional experiment results. The clarification between Theorem 3.1 and Algorithm 1 make me deeply understand the paper as well. Most of my concerns are resolved in this discussion period. Can the authors add the deterministic-solver comparison from the rebuttal to the paper and focus the overall contribution of the paper on stochastic reverse process?
We thank the reviewer for the encouraging feedback and constructive suggestions. We have added the deterministic-solver comparisons (DDIM, Heun PF-ODE, and DPM-Solver++) to Appendix D.3 of the revised manuscript.
We also agree that the paper should be positioned primarily as a framework for stochastic generative dynamics, rather than as an attempt to develop faster numerical solvers for existing probability-flow ODEs. Our main contribution is to incorporate Lévy jumps into generation while retaining the standard Gaussian forward process. Specifically, Theorem 3.1 establishes marginal-preserving Lévy jump-diffusion reverse dynamics, while Algorithm 1 provides a distinct stochastic sampling construction based on annealed Lévy Langevin correctors. We have further clarified this distinction and emphasized the stochastic perspective in the revised manuscript.
We are glad that our responses have addressed most of your concerns. We hope these additional revisions also address your remaining suggestions, and would be grateful if you would consider revisiting your rating.
Response to Reviewer 9Uw4 — Part 1/2
We thank the reviewer for recognizing the theoretical and practical contributions of GLSM. We address the concerns below.
(W1) Theorem 3.1, Algorithm 1, and the annealing schedule. We would like to clarify that Theorem 3.1 and Algorithm 1 describe two distinct sampling dynamics, rather than a continuous-time SDE and its numerical discretization. This is directly analogous to the distinction between reverse-time SDE sampling [1] and annealed Langevin sampling [2] in conventional Brownian score-based models.
Theorem 3.1 constructs a reverse Lévy SDE whose time marginals exactly match the Gaussian marginal path pT−s: dYs=[−f(Ys−,T−s)+2g2(T−s)+h2(s)S[pT−s](Ys−)+SνT−s[pT−s](Ys−)]ds+h(s)dWs+dJs.
Writing ti=T−si, its Euler discretization is Ysi+1=Ysi+Δsi[−f(Ysi,ti)+2g2(ti)+h2(si)S[pti](Ysi)+Sνti[pti](Ysi)]+h(si)Δsiξi+ΔJi.
Algorithm 1 (LLC) instead follows the annealed Langevin sampling paradigm. At each noise level ti, we consider the continuous-time LLC targeting the fixed distribution pti: dYτ(i)=Sνti[pti](Yτ−(i))dτ+dLτνti.
Its Euler updates are Yi,m+1=Yi,m+ηiSνti[pti](Yi,m)+ΔLi,mνti,Yi+1,0=Yi,Mi.
Our image experiments use Mi=1, giving Yi+1=Yi+ηiSνti[pti](Yi)+ΔLi.
Thus, even when Mi=1, LLC remains an annealed Langevin sampler, not a discretization of the reverse SDE. In particular, the LLC update does not contain the original forward drift or the Stein-score term required by Theorem 3.1. We have clarified this distinction in Section 3.1 (lines 247-253) and Section 3.3 (lines 308-339).
The annealing procedure can be understood as tracking a sequence of gradually changing target distributions. Several works have established non-asymptotic guarantees for Brownian annealed Langevin sampling [3-5]. These results show that accurate final sampling can be achieved without full equilibration at every intermediate target under suitable conditions.
We also conducted empirical ablation studies of the annealing parameters L and M (Appendix D.3.1-D.3.3, Table 7 and Figures 6-7). Fixing L=32 and increasing M from 1 to 32 improves CIFAR-10 validation FID from 92.7 to 20.6, compared with 97.1 for SMLD at M=32. Even with only L=16 levels and M=128 updates per level, GLSM achieves an FID of 23.9, compared with 89.9 for SMLD.
When fixing the total budget LM at 128 or 256 NFEs, allocating more evaluations to distinct noise levels yields better results, with M=1 achieving the lowest FID among the tested configurations (Figures 6-7). Therefore, M=1 is a numerical choice motivated by efficiency under a fixed computational budget, rather than a requirement of the LLC construction.
[1] Song, Y., et al. (2021). Score-Based Generative Modeling through Stochastic Differential Equations. ICLR.
[2] Song, Y., and Ermon, S. (2019). Generative Modeling by Estimating Gradients of the Data Distribution. NeurIPS.
[3] Guo et al., Provable Benefit of Annealed Langevin Monte Carlo for Non-Log-Concave Sampling, ICLR 2025.
[4] Cordero-Encinar et al., Non-asymptotic Analysis of Diffusion Annealed Langevin Monte Carlo for Generative Modelling, 2025.
[5] Nitanda et al., Slowly Annealed Langevin Dynamics: Theory and Applications to Training-Free Guided Generation, NeurIPS 2026.