- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies sampling corrections for pretrained diffusion models, with a particular focus on errors in minority-mode representation. It replaces the KL energy underlying standard Langevin correction with Rényi divergence in the Wasserstein gradient-flow framework. This yields a nonlinear, density-dependent mobility that changes both drift and diffusion, with ordinary Langevin correction recovered at order one. The practical sampler uses pretrained scores and auxiliary density-ratio estimates; it avoids retraining the diffusion network but still requires reference data and ratio estimation.
The theoretical analysis derives fixed-noise and moving-target dissipation identities and examines how estimated scores and density ratios affect them. Validation includes imbalanced Gaussian mixtures, Stacked-MNIST, and a mixture of human and cat faces, using ancestral sampling and Langevin correction as baselines. Reported improvements concern mode-frequency TV, confidence-filtered TV, rare-class frequency, and class-specific FID. These results establish a useful mechanism, although the incremental practical benefit remains insufficiently validated.
Critical strengths
-
A coherent extension with a genuine change in sampling dynamics. Although Rényi divergence itself is established, the method changes density-dependent drift and noise rather than merely renaming an objective. Recovering Langevin correction at order one provides a meaningful reference point. The dissipation analysis and discussion of estimator errors help explain both the intended behavior and its limitations. This is the paper’s main conceptual contribution.
-
Controlled evidence of minority-mode correction. The experiments extend beyond face generation and include both underrepresentation and overrepresentation in Gaussian mixtures. The order and correction-budget sweeps, together with comparisons against Langevin, provide evidence that the proposed mechanism can affect mode frequencies. In the Faces experiment, the mean cat frequency increases from to per 500 images against a target of , recovering about % of the ancestral deficit (Tables 4 and 11). This is a substantive result even though the FID gains are modest.
Critical weaknesses
-
The incremental benefit over closely related correction methods is not adequately established. The experimental baselines are ancestral sampling and Langevin correction. The paper discusses Discriminator Guidance (Kim et al., 2023), which also uses an auxiliary discriminator to correct a pretrained diffusion model, but provides no direct comparison under comparable reference-data access and computational budgets. The final paragraph of the sampling-related discussion in Section 2 also needs a more precise distinction: target preservation alone does not resolve the comparison with methods intended to correct the model distribution toward the data distribution. The existing results establish an extension of Langevin, but do not sufficiently establish the practical significance of that extension. This matters because the natural-image evidence is confined to one constructed human–cat mixture, rather than demonstrating a broader range of sampling errors.
-
The empirical evidence does not yet establish a reliable and worthwhile improvement in distributional fidelity. For Faces, cat FID decreases from for matched Langevin at to for Rényi correction, a relative reduction of about % (Tables 4 and 11). Appendix E.3 explicitly leaves unspecified whether the reported variability is a standard deviation or standard error and gives neither the generated nor reference sample counts used for FID. Without paired seed differences and a clearly controlled class-specific FID protocol, these means do not establish the reliability of the increment; they also do not establish that the increment is merely noise.
The other experiments reveal important limits to a broad quality claim. Mode-frequency TV does not capture within-mode fidelity, and Appendix E.1 reports cases where rare-mode mass improves while overall TV or mode shape does not. For Stacked-MNIST at and order , filtered TV improves by about % over Langevin, while reported generation time increases from to seconds, about % (Tables 9–10). Appendix E.2 also reports substantial mobility clipping in this setting. The paper acknowledges estimation and discretization limitations, but does not sufficiently establish which gains persist under these implementation choices. These are limitations of the central practical claim, rather than a request to test every possible dataset.
Questions for authors
-
Evaluation clarification: What exactly do the values in the Faces results represent? Please specify the generated and reference sample counts for each class-specific FID and provide paired per-seed differences or confidence intervals for Rényi versus matched Langevin. Are the methods evaluated with equal numbers of cat samples, or otherwise controlled for sample-size effects? This could materially change my assessment of the reported FID gain.
-
Comparative evidence: Is there a comparison with Discriminator Guidance or another closely related correction method using the same pretrained model, reference data, and comparable total computational budget? If not, which demonstrated property establishes the added value over those approaches? A fair comparison or a precise explanation of a consequential methodological distinction could change my assessment of significance.
-
Implementation and cost clarification: What hardware, batch size, and operations are included in the reported timing? Does it include discriminator pretraining and online updates? How sensitive are the gains to mobility clipping and step caps, particularly in the Stacked-MNIST configuration where many weights are clipped? These details are needed to interpret the practical cost and the connection to the analyzed dynamics.
-
Claim scope: How should the claimed improvement in distributional fidelity be interpreted in the GMM cases where rare-mode mass moves toward its target while total TV or within-mode shape worsens? Please identify the empirical conditions under which the paper supports a broader fidelity improvement. This asks for clarification of existing evidence; additional application domains are future work.
5 comments
Response to Reviewer 7qF5
W2 and Q1. Clarification of the FID gain
The values are population standard deviations across five sampling seeds, with 500 generated images per seed and sampling setting. We set aside 2000 human reference images and 493 cat reference images for the FID calculation. We compute the class-specific FID on the generated images the classifier assigns to each class, so the number of generated images per class do vary by method and seed. These details have been added to Section 4.3 and Appendix E.3.
We have also added a paired per-seed difference. The full tables are added in Appendix E.3.
Paired comparisons against ancestral sampling
| Method | Cat count: Δ [95% CI]; raw p | Cat FID: Δ [95% CI]; raw p | Human FID: Δ [95% CI]; raw p |
|---|---|---|---|
| Boost-and-Skip (Boost 3, Skip 12) | +113.2 [+104.4, +122.0]; 3.74e-6 | -7.25 [-7.97, -6.53]; 9.75e-6 | +19.86 [+19.20, +20.52]; 1.23e-7 |
| Boost-and-Skip (Boost 1.5, Skip 12) | +45.4 [+40.5, +50.3]; 1.31e-5 | -6.36 [-7.25, -5.47]; 3.77e-5 | +9.30 [+9.05, +9.55]; 5.57e-8 |
| Minority Guidance | +20.2 [+16.8, +23.6]; 8.34e-5 | +2.10 [+1.74, +2.46]; 8.37e-5 | +13.90 [+12.75, +15.05]; 4.69e-6 |
| Discriminator Guidance | -0.2 [-1.2, +0.8]; 0.6213 | -2.45 [-2.76, -2.14]; 2.45e-5 | +0.11 [-0.17, +0.39]; 0.3322 |
| Langevin K=2 | +0.4 [-1.0, +1.8]; 0.4766 | -0.60 [-1.16, -0.04]; 0.0413 | +0.08 [-0.09, +0.25]; 0.2671 |
| Rényi α=0.7, K=2 | +4.2 [+3.2, +5.2]; 0.0004 | -1.97 [-2.93, -1.01]; 0.0048 | +0.08 [-0.06, +0.22]; 0.1951 |
Paired comparisons against matching Langevin
| Method | Cat count: Δ [95% CI]; raw p | Cat FID: Δ [95% CI]; raw p | Human FID: Δ [95% CI]; raw p |
|---|---|---|---|
| Rényi α=0.7, K=1 | +4.6 [+2.2, +7.0]; 0.0062 | -1.56 [-2.32, -0.80]; 0.0046 | -0.03 [-0.40, +0.34]; 0.8340 |
| Rényi α=0.7, K=2 | +3.8 [+2.8, +4.8]; 0.0005 | -1.37 [-2.36, -0.38]; 0.0182 | +0.00 [-0.18, +0.18]; 1.0000 |
| Rényi α=0.7, K=3 | +3.8 [+1.0, +6.6]; 0.0204 | -1.33 [-2.38, -0.28]; 0.0246 | -0.04 [-0.41, +0.33]; 0.7812 |
Q3. Cost clarification
We are unable to access to the original machine with RTX 5090 that we used for the numbers reported in the paper. We thus performed a more complete profiling run on the stacked-MNIST with similar settings (10,000 evaluated outputs per seed, 64 correction anchors, for correction, we have 1024 generated reference particles and 1024 holdout particles from data). The results for the profiling run consists of 64 (anchors) × 3 (steps) = 192 refresh calls, each taking 12 optimizer steps: 2,304 online updates per sampling run.
| Component | Seconds | Share of wall time |
|---|---|---|
| Denoiser | 92.776 | 68.7% |
| Online refresh | 29.259 | 21.7% |
| Ratio/logit gradients | 10.497 | 7.8% |
| Mobility/normalization | 0.119 | 0.1% |
| Residual correction overheads | 2.380 | 1.8% |
| Total | 135.031 | 100% |
This profiling shows that the additional overhead is mainly dominated by the online refresh. We note that while we used an online refresh in our experiments, a fixed correction policy permits an alternative implementation in which a critic, conditioned on noise level and correction stage, is trained offline on representative trajectories and frozen during sampling. This would remove online optimizer updates and the online refresh overhead.
Clipping and step caps: Clipping keeps the mobility positive, so it preserves the corrector’s stationary density. It does, however, change the mobility, and hence the dynamics relative to the flow we analyse. In the setting the reviewer highlights (, ), 29.07% of weights are clipped and 2.60% of steps are capped (Appendix E.2). Caps shorten the correction time, but the realized correction time is 0.588277 against 0.589247 for Langevin, so the comparison remains matched in correction time.
Response to Reviewer 7qF5
Q4. Scope of distributional fidelity in GMM
We agree that improved rare-mode proportions alone do not establish broader distributional fidelity. In the GMM cases, we observe an improvement in TV for smaller in the under-represented case. Across all six settings, gives the lowest mean mode-frequency TV, 3.5–10% below Langevin correction (), while TV rises for . Thus, there are consistent improvements for the rare-mode proportions and the distributional fidelity throughout these under-represented rare mode settings.
One metric that may deteriorate is that of the mean radius relative to the correct width. For the case of , we observe that this ratio rises from 1.56 () to 1.92 (), while rising less in the other two depleted settings, from 1.26 to 1.34 at and from 1.21 to 1.28 at . Our understanding is that smaller does increase the correction mobility in the under-represented mode, pushing more mass toward it, but the correction is not ideal and leaves the rare mode wider. This is actually consistent with the theory for the following considerations.
- The fixed point is the model’s. The corrector converges to the learned score’s equilibrium. The rare mode is already too wide at , and most so in the rarest setting, consistent with a score fitted to few rare samples. Added mass therefore fills the mode in the model’s wider shape rather than the target’s.
- The budget is finite. Samples transported from the common modes plausibly arrive at the rare mode’s outskirts with little time to relax inward, and some are stranded between modes on the way. This also explains why there are more unassigned samples.
Response to Reviewer 7qF5
W1 and Q2. Comparison to Discriminator guidance and other existing samplers
We thank the reviewer and agree that target preservation alone does not settle the comparison with Discriminator Guidance. We have added Discriminator Guidance, together with Minority Guidance and Boost-and-Skip, on the same pretrained DDPM checkpoints for the face mixture and Stacked-MNIST. The results and discussion have been added to the revised paper in Sec 4.3 and Appendix E.3.
Face mixtures
Discriminator Guidance changes the target by adding the gradient of an estimated log density ratio to the model score, sθ(x,t)+∇xlogρt(x)πt(x), where πt is the noised data distribution and ρt the sampling distribution at noise level t. It gives slightly better cat FID than Rényi correction (44.68 vs. 45.16, within one seed standard deviation) but leaves the cat proportion unchanged. We attribute the two effects to different noise levels. Changing class proportions requires moving samples between classes, which can only happen at high noise, where noised cats and humans nearly coincide and the cross-class guidance is weak. At low noise the classes are separated, the discriminator easily tells real from generated cats, and the guidance is strong. These low-noise corrections carry through to the final images, and hence to cat FID, while a proportion error leaves the ratio flat within each class.
Boost-and-Skip greatly increases the assigned cat count and lowers cat FID, partly because FID is biased upward for small sample sets. However, it overshoots the 5% target and substantially worsens human FID. The boost places more probability in the tails of the initial distribution, raising the chance of sampling minority modes, but its effect is hard to calibrate. After observing the overshoot at boost 3, we reduced it to 1.5, which reduced the overshoot and improved human FID at the cost of slightly worse cat FID. Boost-and-Skip is a powerful way to sample minority modes, but controlling how far it goes is difficult.
Rényi correction instead aims to reproduce the proportions of the training data. It shares Langevin's equilibrium and changes only the transport toward it, weighting the correction toward regions the sampler under-covers. It moved the cat count toward the 5% target while slightly improving cat FID. Because the equilibrium is unchanged, correction with a suitable α moves the sampler toward the stationary limit instead of pushing the minority proportion past the target, as the boost does.
Stacked MNIST
Minority Guidance and both Boost-and-Skip settings performed worse than ancestral sampling. Discriminator Guidance improves slightly on ancestral sampling and Langevin correction. Overall, Rényi correction still retains the best performance.