- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Paper Summary
This paper generalizes DLPM by letting the stable tail index vary across timesteps: small near the data end, annealed to at the terminal end. Generation reverses this schedule — starting near-Gaussian like DDPM, ending with heavy-tailed , whose large jumps help cover rare modes in long-tailed data. Since stable increments with different tail indices are not additive-closed, the authors condition on the full scale path and introduce a signal-correction term plus a variance recursion, keeping the forward marginal Gaussian and training cheap. They provide complete training and sampling algorithms, and an effective spectral index characterizing the noise's tail behavior per frequency — heavy-tailed at low frequencies (), near-Gaussian at high frequencies ().
Critical strengths
Strengths
-
Sound and rigorous theory within the established paradigm. The methodology follows the DDPM/DLPM line of work. And all derivations and proofs (forward marginals, reverse kernels, distribution characterization) are clear and, to my checking, correct.
-
A faithful and complete generalization of DLPM. The paper generalizes DLPM end-to-end, from the forward process design (signal-correction term and variance recursion accommodating time-varying tail indices) to the reverse sampling procedure (arbitrary-jump conditional-Gaussian kernels with respacing support). When the tail index is constant, the framework reduces exactly to DLPM (Prop. 8), making the generalization conservative and verifiable.
-
Clear organization and presentation. The paper is well-structured and logically coherent: the problem statement is precise (four criteria for a valid construction, each addressed in turn), the writing is fluent, and the extensive appendix (proofs, algorithms, ablations, sensitivity analyses) makes the work easy to verify and reproduce.
Critical weaknesses
Weaknesses
-
Convergence of the underlying framework remains open. The only sampling guarantee for fixed- DLPM (Cherkaoui et al., 2026, Thm. 2.2) assumes uniform approximation of the reverse object by an idealized TV-risk minimizer, whereas practical epsilon-RMSE training yields only -type control — and the guarantee does not extend to the time-varying construction here. We encourage the community to prioritize these foundational convergence questions.
-
A theory–practice gap in conditioning. The reverse kernel is exact only given the realized scale path, yet the network sees only and learns a marginalized predictor; how this predictor interacts with the path-conditioned kernel coefficients at generation is not analyzed.
-
Unanalyzed plug-in approximation. Sampling substitutes and resampled scales into a kernel that is exact only for the true and scale path; the resulting error is neither bounded nor empirically isolated.
-
Theory does not cover the implemented object. Thm. 1/2 and Prop. 9 assume untruncated stable scales, but the implementation clips , after which increments are no longer stable; results are sensitive to the threshold and no truncated theory is given.
Questions for authors
Questions For Authors
-
Convergence. Can any distribution-level sampling guarantee (even under strong assumptions) be established for the time-varying tail-index construction? If not, which step of the Cherkaoui et al. (2026) framework breaks down most fundamentally — the non-Markovianity after marginalizing , the mixed-power residuals, or the scale-augmented epsilon target?
-
Conditioning gap. The network is trained marginally over scale paths (input only ), while the reverse kernel's coefficients are computed from the realized path. Can you quantify — theoretically or empirically — how much sampling error comes from this mismatch, e.g., by comparing against an oracle that also receives the true scales?
-
Plug-in error. How sensitive is generation quality to the accuracy of ? For instance, does an oracle sampler using the true (on synthetic data where it is available) close a significant fraction of the gap to the target distribution?
-
Clipping. Do truncated analogues of Thm. 1/2 and Prop. 9 exist (e.g., tail behavior governed by up to the clip level)? How do the main results vary with the clipping threshold, and is needed uniformly across all experiments?
8 comments
Q2. Conditioning gap. The network is trained marginally over scale paths (input only ), while the reverse kernel's coefficients are computed from the realized path. Can you quantify — theoretically or empirically — how much sampling error comes from this mismatch, e.g., by comparing against an oracle that also receives the true scales?
We can quantify the effect of omitted scale information on endpoint prediction and on an individual reverse update. In the two balanced synthetic mixtures below, the comparison between the marginal and scale-conditioned conditional-mean references shows no consistent generation-quality advantage from observing the scales.
Algorithm 2 in Appendix E samples a complete scale path before the reverse loop and retains it for the reverse coefficients. The network receives only . Its endpoint estimate is the quantity already defined in Eq. (11):
\frac{X_t-\bar\sigma_t\epsilon_\theta(X_t,t)} {\bar\gamma_t}.$$ At a fixed time $t$, define the two conditional-mean references under the true forward joint distribution: $$m=\mathbb E[X_0\mid X_t],\qquad m_A=\mathbb E[X_0\mid X_t,A].$$ The first uses the same information as the network; the second additionally observes the realized scale path. For data with finite second moments and an endpoint predictor with finite squared error, conditional expectation gives $$\mathbb E\\|\widehat X_{0,\theta}-m_A\\|^2 = \mathbb E\\|\widehat X_{0,\theta}-m\\|^2+ \mathbb E\\|m-m_A\\|^2.$$ The cross term vanishes because $\mathbb E[m_A\mid X_t]=m$ and $\widehat X_{0,\theta}$ depends only on $(X_t,t)$. In particular, the second term remains even if the endpoint predictor equals $m$. It also equals the reduction in optimal endpoint MSE obtained by observing the scales, $\mathbb E\\|X_0-m\\|^2-\mathbb E\\|X_0-m_A\\|^2$. For a retained pair $s<t$, let $a=a_{s,t}$ and $b=b_{s,t}$ be the coefficients in Eq. (14), and let $\nu=\nu_{s,t}>0$ be the variance in Eq. (15). At a fixed state and realized scale path, replacing the actual endpoint estimate by $m_A$ changes the plug-in Gaussian reverse kernel by $$D_{\mathrm{KL}}\left( \mathcal N(aX_t+b\widehat X_{0,\theta},\nu I) \\,\Vert\\, \mathcal N(aX_t+bm_A,\nu I) \right)= \frac{b^2}{2\nu}\\|\widehat X_{0,\theta}-m_A\\|^2.$$ Setting $\widehat X_{0,\theta}=m$ in this expression isolates the local change due to omitted scale information. The factor $b^2/(2\nu)$ depends on the realized path, so the unweighted squared-error decomposition above does not automatically decompose the expected KL. Empirically, we compared the learned endpoint estimate with the analytic $m_A$ on a known two-dimensional Gaussian mixture. We also approximated $m$ by randomized quasi-Monte Carlo integration over the prior scale-path law with posterior likelihood weights. All predictors use the same plug-in Gaussian update, including the same conditional variance. The diagnostic uses A-DLPM checkpoints trained on the two-dimensional mixtures across five training seeds, with five generation seeds per checkpoint and 5,000 samples per generation. The conditional-mean references have no trained parameters and use the same five generation seeds. Predictors share initialization, scale paths, and Gaussian innovations within each generation seed. Both mixtures consist of equally weighted isotropic Gaussian components ($\sigma=0.1$). Entries are mean sliced-W1; lower is better. | Scale cap | Separation | Original predictor | Refined predictor | Marginal reference $m$ | Scale-conditioned reference $m_A$ | | --- | --- | --- | --- | --- | --- | | Untruncated | $4\sigma$ | .01758 | .01101 | .01955 | .02041 | | Untruncated | $16\sigma$ | .03251 | .01518 | .01999 | .01931 | | $C=20$ | $4\sigma$ | .01166 | .00634 | .01068 | .01216 | | $C=20$ | $16\sigma$ | .02042 | .01258 | .01345 | .01525 | The refined predictor achieves lower mean sliced-W1 than both conditional-mean references across all four settings. Comparing the two conditional-mean references, $m_A$ gives a lower mean sliced-W1 only in the untruncated $16\sigma$ setting, whereas $m$ gives lower values in the other three. Thus, providing true scale information yields no consistent full-generation advantage under the fixed plug-in update. We also observed a similar phenomenon in our early experimental design: feeding the scale $A_t$ to the network as an additional input leads to a clear degradation in performance, which is also reported by Appendix G of DLPM paper [1]. [1] Dario Shariatian, Umut Simsekli, and Alain Durmus. Denoising Lévy Probabilistic Models. *ICLR, 2025*. https://arxiv.org/abs/2407.18609Q3. Plug-in error. How sensitive is generation quality to the accuracy of ? For instance, does an oracle sampler using the true (on synthetic data where it is available) close a significant fraction of the gap to the target distribution?
Reducing endpoint prediction error improves individual reverse updates, and a posterior oracle substantially lowers mean error in full generation on the known Gaussian mixture.
At a fixed state and scale path, let and be the coefficients in Eq. (14), and be the variance in Eq. (15). Proposition 7 gives the conditional Gaussian update in Eq. (12). Replacing the true endpoint by the network prediction changes the reverse mean by , giving
\mathcal N(aX_t+bX_0,\nu I) \\,\Vert\\, \mathcal N(aX_t+b\widehat X_{0,\theta},\nu I) \right) =\frac{b^2}{2\nu}\\|\widehat X_{0,\theta}-X_0\\|^2.$$ Consequently, this KL divergence of the reverse update exhibits a quadratic dependence on the endpoint prediction error. Eq. (11) also gives $\widehat X_{0,\theta}-X_0=(\bar\sigma_t/\bar\gamma_t)(\epsilon_t-\epsilon_\theta)$, linking this sensitivity directly to epsilon prediction. We tested sensitivity using $X_0+\lambda(\widehat X_{0,\theta}-X_0)$, with $\lambda\in\\{0,1/2,1\\}$, while holding the forward state, scale path, and Gaussian innovation fixed. Across four mixture settings and $t\in\\{25,50,75,99\\}$, halving the endpoint error reduced the mean sliced-W1 (relative to samples from the true endpoint) by 36.9%–49.9%. Each comparison averages three repetitions. This measures the effect on samples after one reverse step; the quadratic formula above concerns conditional KL. We use positive-variance steps, since supplying the true endpoint at the final deterministic step simply returns that data point. For full generation, since the ground-truth endpoint $X_0$ is unknown, the oracle instead draws $X_0\sim p(X_0\mid X_t,A)$ at each step and applies the conditional Gaussian update, implementing the exact reverse conditional. It shares terminal initialization, scale paths, and Gaussian innovations with the learned sampler, with a separate stream for posterior draws. We compare the original predictor with a refined predictor trained with additional updates (15,000 for untruncated scales, 5,000 for $C=20$) selected via independent validation samples. Both predictors receive $(X_t,t)$ and use the same epsilon-RMSE objective. The results below report the balanced mixture settings. Each learned configuration uses five paired training checkpoints and five generation seeds per checkpoint, with 5,000 samples per generation. We average over generation seeds, then checkpoints. The oracle uses the same five generation seeds and has no trained parameters. Entries are mean sliced-W1 to a common target reference; relative reductions use unrounded means. | Scale cap | Separation | Original predictor | Refined predictor | Posterior oracle | Relative reduction from refined predictor | | --- | --- | --- | --- | --- | --- | | Untruncated | $4\sigma$ | .01758 | .01101 | .00454 | 58.7% | | Untruncated | $16\sigma$ | .03251 | .01518 | .00806 | 46.9% | | $C=20$ | $4\sigma$ | .01166 | .00634 | .00341 | 46.1% | | $C=20$ | $16\sigma$ | .02042 | .01258 | .00627 | 50.1% | The refined predictor achieves lower mean sliced-W1 than the original predictor in all four settings. For the capped process, its endpoint MSE, averaged over all 99 positive forward times, is also lower. The posterior oracle further reduces mean sliced-W1 by 46%–59% relative to the refined predictor. This comparison reflects the combined benefit of conditioning on the realized scale path and sampling from the conditional posterior.
We thank the reviewer for their thoughtful feedback and for recognizing the strengths of our work. Below, we address the reviewer's concerns in detail.
Q1. Convergence. Can any distribution-level sampling guarantee (even under strong assumptions) be established for the time-varying tail-index construction? If not, which step of the Cherkaoui et al. (2026) framework breaks down most fundamentally — the non-Markovianity after marginalizing X0, the mixed-power residuals, or the scale-augmented epsilon target?
The main obstacle to transferring this framework is controlling the local TV errors of the implemented reverse transitions. In particular, the deterministic final denoising step creates a maximal local TV discrepancy, regardless of the prediction accuracy.Our epsilon-RMSE objective trains the network to predict a point estimate, whereas the analysis in Cherkaoui et al. [1] (Theorem 2.2 and Appendix C.2). controls the TV distance between reverse transition distributions.
Cherkaoui et al. derive their sampling bound by decomposing the total-variation (TV) error into terminal-initialization and reverse-transition errors, then controlling the latter through a TV-based learning objective. Their decomposition applies to Markov path laws. In A-DLPM, the forward transition in Eq. (7) explicitly depends on X0, so the process is generally non-Markovian after marginalizing X0. Nevertheless, a corresponding decomposition remains valid for the final marginal distribution. Following the notation of [1], with a denoting the realized scale path and α the tail-index schedule, we have
= \int k_{t-1\mid t,a}^{(\alpha)}(B\mid x) \, k_{t\mid a}^{(\alpha)}(\mathrm dx), \qquad \forall B\subset\mathbb{R}^d\text{ Borel}$$ Here $k_{s\mid t,a}^{(\alpha)}(x)=\mathcal L(X_s\mid X_t=x,A=a)$ is the exact reverse probability law. Let $\overleftarrow q_{s\mid t,a}^{\theta}(\cdot\mid x)$ be its implemented counterpart: the Gaussian in Proposition 7 with $X_0$ replaced by $\widehat X_0(x,t)$. For retained times $0=t_0<\cdots<t_N$, let $\widehat k_{t_i\mid a}^{\theta}$ denote the sampler marginal. Eq. (8) of [1] gives the following TV error decomposition: $$\begin{aligned} \mathrm{TV}\\!\left(\widehat k_{0\mid a}^{\theta},k_{0\mid a}^{(\alpha)}\right) &\leq \mathrm{TV}\\!\left(\widehat k_{t_N\mid a}^{\theta},k_{t_N\mid a}^{(\alpha)}\right)\\\\ &\quad+\sum_{i=1}^{N}\int \mathrm{TV}\\!\left( k_{t_{i-1}\mid t_i,a}^{(\alpha)}( \cdot \mid x), \overleftarrow q_{t_{i-1}\mid t_i,a}^{\theta}( \cdot \mid x) \right)k_{t_i\mid a}^{(\alpha)}(\mathrm dx). \end{aligned}$$ The first term is the terminal-initialization error; each transition error is averaged over the true conditional forward marginal. Averaging the right-hand side over the common scale-path law also bounds the unconditional output TV error, by convexity of TV. The exact reverse kernel averages over the posterior of $X_0$, whereas the implemented kernel uses a point estimate. At the final step, $t=t_1$ to $0$, they become $$\begin{aligned} k_{0\mid t,a}^{(\alpha)}(x) &=\mathcal L(X_0\mid X_t=x,A=a)\\\\ \overleftarrow q_{0\mid t,a}^{\theta}(x) &=\delta_{\widehat X_0(x,t)} \end{aligned}$$ Suppose the data distribution has a density and the forward conditional noise variance is positive. The posterior of $X_0$ then assigns zero probability to every singleton. Taking $B=\\{\widehat X_0(x,t)\\}$, the implemented kernel assigns probability one to $B$, while the exact kernel assigns probability zero. Hence, $$\mathrm{TV}\\!\left( k_{0\mid t,a}^{(\alpha)}( \cdot \mid x), \overleftarrow q_{0\mid t,a}^{\theta}(\cdot \mid x) \right)=1.$$ The final-step term alone therefore makes the upper bound at least one, regardless of prediction accuracy. Since TV is at most one, this bound is uninformative for the implemented sampler. Regarding the three possible obstacles, non-Markovianity prevents direct use of the complete-path decomposition but does not invalidate marginal-level error propagation. The mixed-power residuals require new estimates of the terminal-initialization error, since the fixed-index stable noise law in [1] is replaced by the mixed-power characteristic exponent in Eq. (18). The more fundamental difficulty lies in the learned reverse transitions. The epsilon-RMSE objective controls prediction error rather than the TV distance between conditional distributions. In particular, the deterministic final update makes the corresponding local TV error equal to one under a non-atomic data distribution, regardless of prediction accuracy. A nontrivial distribution-level guarantee therefore requires a different way to control the actual sampler's output distribution, together with a new initialization analysis for the mixed-power residuals.