acceptodds
Under review as a conference paper at ICLR 2027

Rényi Sampling: Nonlinear Wasserstein Corrections for Diffusion Models

Abstract

We introduce Rényi sampling, nonlinear corrections for pretrained denoising diffusion probabilistic models that require no diffusion-network retraining or semantic class guidance. Our construction generalizes Langevin correction from the Wasserstein gradient flow of Kullback–Leibler divergence to that of Rényi divergence. The stochastic corrector combines a pretrained score with an estimated density ratio, with the Rényi order controlling local correction strength. We establish a moving-target dissipation identity showing that Rényi sampling reduces distributional mismatch throughout denoising while preserving the intended noisy-data marginals. We show that helps to rectify mode deficits, while removes excess mode representations. Relative to Langevin correction, recovers of the rare-mode deficit, and removes of excess rare mass for imbalanced Gaussian mixtures experiments. Extending our method to a face generation model trained on a cat- human data mixture, correction increases the generated cat proportion from under ancestral sampling to , while improving cat Fréchet Inception Distance from to . These results demonstrate the potential of Rényi correction to improve both distributional fidelity and generation quality during sampling. Our code is available anonymously at https://github.com/Anonymous-PaperSubmission/RenyiSamplingDDPM.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
DDPM Sampling, Rényi divergence, Wasserstein gradient flow, Langevin correction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7qF5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies sampling corrections for pretrained diffusion models, with a particular focus on errors in minority-mode representation. It replaces the KL energy underlying standard Langevin correction with Rényi divergence in the Wasserstein gradient-flow framework. This yields a nonlinear, density-dependent mobility that changes both drift and diffusion, with ordinary Langevin correction recovered at order one. The practical sampler uses pretrained scores and auxiliary density-ratio estimates; it avoids retraining the diffusion network but still requires reference data and ratio estimation.

The theoretical analysis derives fixed-noise and moving-target dissipation identities and examines how estimated scores and density ratios affect them. Validation includes imbalanced Gaussian mixtures, Stacked-MNIST, and a mixture of human and cat faces, using ancestral sampling and Langevin correction as baselines. Reported improvements concern mode-frequency TV, confidence-filtered TV, rare-class frequency, and class-specific FID. These results establish a useful mechanism, although the incremental practical benefit remains insufficiently validated.

Critical strengths
  1. A coherent extension with a genuine change in sampling dynamics. Although Rényi divergence itself is established, the method changes density-dependent drift and noise rather than merely renaming an objective. Recovering Langevin correction at order one provides a meaningful reference point. The dissipation analysis and discussion of estimator errors help explain both the intended behavior and its limitations. This is the paper’s main conceptual contribution.

  2. Controlled evidence of minority-mode correction. The experiments extend beyond face generation and include both underrepresentation and overrepresentation in Gaussian mixtures. The order and correction-budget sweeps, together with comparisons against Langevin, provide evidence that the proposed mechanism can affect mode frequencies. In the Faces experiment, the mean cat frequency increases from 19.019.0 to 23.223.2 per 500 images against a target of 2525, recovering about 7070% of the ancestral deficit (Tables 4 and 11). This is a substantive result even though the FID gains are modest.

Critical weaknesses
  1. The incremental benefit over closely related correction methods is not adequately established. The experimental baselines are ancestral sampling and Langevin correction. The paper discusses Discriminator Guidance (Kim et al., 2023), which also uses an auxiliary discriminator to correct a pretrained diffusion model, but provides no direct comparison under comparable reference-data access and computational budgets. The final paragraph of the sampling-related discussion in Section 2 also needs a more precise distinction: target preservation alone does not resolve the comparison with methods intended to correct the model distribution toward the data distribution. The existing results establish an extension of Langevin, but do not sufficiently establish the practical significance of that extension. This matters because the natural-image evidence is confined to one constructed human–cat mixture, rather than demonstrating a broader range of sampling errors.

  2. The empirical evidence does not yet establish a reliable and worthwhile improvement in distributional fidelity. For Faces, cat FID decreases from 46.53±0.9846.53 \pm 0.98 for matched Langevin at K=2K = 2 to 45.16±0.8445.16 \pm 0.84 for Rényi correction, a relative reduction of about 2.942.94% (Tables 4 and 11). Appendix E.3 explicitly leaves unspecified whether the reported variability is a standard deviation or standard error and gives neither the generated nor reference sample counts used for FID. Without paired seed differences and a clearly controlled class-specific FID protocol, these means do not establish the reliability of the increment; they also do not establish that the increment is merely noise.

    The other experiments reveal important limits to a broad quality claim. Mode-frequency TV does not capture within-mode fidelity, and Appendix E.1 reports cases where rare-mode mass improves while overall TV or mode shape does not. For Stacked-MNIST at K=3K = 3 and order 0.50.5, filtered TV improves by about 1.221.22% over Langevin, while reported generation time increases from 53.1453.14 to 70.5270.52 seconds, about 32.732.7% (Tables 9–10). Appendix E.2 also reports substantial mobility clipping in this setting. The paper acknowledges estimation and discretization limitations, but does not sufficiently establish which gains persist under these implementation choices. These are limitations of the central practical claim, rather than a request to test every possible dataset.

Questions for authors
  1. Evaluation clarification: What exactly do the ±\pm values in the Faces results represent? Please specify the generated and reference sample counts for each class-specific FID and provide paired per-seed differences or confidence intervals for Rényi versus matched Langevin. Are the methods evaluated with equal numbers of cat samples, or otherwise controlled for sample-size effects? This could materially change my assessment of the reported FID gain.

  2. Comparative evidence: Is there a comparison with Discriminator Guidance or another closely related correction method using the same pretrained model, reference data, and comparable total computational budget? If not, which demonstrated property establishes the added value over those approaches? A fair comparison or a precise explanation of a consequential methodological distinction could change my assessment of significance.

  3. Implementation and cost clarification: What hardware, batch size, and operations are included in the reported timing? Does it include discriminator pretraining and online updates? How sensitive are the gains to mobility clipping and step caps, particularly in the Stacked-MNIST configuration where many weights are clipped? These details are needed to interpret the practical cost and the connection to the analyzed dynamics.

  4. Claim scope: How should the claimed improvement in distributional fidelity be interpreted in the GMM cases where rare-mode mass moves toward its target while total TV or within-mode shape worsens? Please identify the empirical conditions under which the paper supports a broader fidelity improvement. This asks for clarification of existing evidence; additional application domains are future work.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 7qF5

W1 and Q2. Comparison to Discriminator guidance and other existing samplers

We thank the reviewer and agree that target preservation alone does not settle the comparison with Discriminator Guidance. We have added Discriminator Guidance, together with Minority Guidance and Boost-and-Skip, on the same pretrained DDPM checkpoints for the face mixture and Stacked-MNIST. The results and discussion have been added to the revised paper in Sec 4.3 and Appendix E.3.

  • Discriminator Guidance. We ran the fixed denoiser through complete unconditional sampling trajectories, using 512 generated images to train the discriminator and 128 for validation, together with 1,024 real training images balanced between humans and cats. Following Kim et al. (2023), the discriminator separates forward-noised real and generated images at every noise level, and its weights stay frozen during generation.
  • Minority Guidance. We constructed rarity labels from all 33,039 training images. Each image's rarity score is the mean VGG-LPIPS distance between the image and five Tweedie reconstructions from independent forward-noising draws at timestep t=605t=605 of the 1,000-step schedule, then discretized into 100 empirical quantile bins. A rarity-classification head is used to provide rarity-based guidance.
  • Boost-and-Skip. We used boost factors 3 and 1.5 with a skip of 12 steps.

Face mixtures

MethodAssigned cats / 500Cat FIDHuman FID
Ancestral19.0±3.419.0 \pm 3.447.13±1.2147.13\pm1.2117.37±0.5417.37\pm0.54
Minority Guidance39.2±5.439.2 \pm 5.449.23±1.4149.23\pm1.4131.27±1.3131.27\pm1.31
Boost-and-Skip, boost=3,skip=12\text{boost}=3, \text{skip}=12132.2±9.7132.2\pm9.739.88±1.0439.88\pm1.0437.23±0.8837.23\pm0.88
Boost-and-Skip, boost=1.5,skip=12\text{boost}=1.5, \text{skip}=1264.4±6.964.4\pm6.940.77±0.9640.77\pm0.9626.67±0.6426.67\pm0.64
Discriminator Guidance18.8±3.318.8\pm3.344.68±1.0344.68\pm1.0317.48±0.5917.48\pm0.59
Langevin, K=2K=219.4±3.519.4\pm3.546.53±0.9846.53\pm0.9817.45±0.5717.45\pm0.57
Rényi, α=0.7,K=2\alpha=0.7, K=223.2±3.023.2\pm3.045.16±0.8445.16\pm0.8417.45±0.4917.45\pm0.49

Discriminator Guidance changes the target by adding the gradient of an estimated log density ratio to the model score, sθ(x,t)+∇xlog⁡πt(x)ρt(x),s_ \theta(x,t)+\nabla_ x\log\frac{\pi_ t(x)}{\rho_ t(x)}, where πt\pi_ t is the noised data distribution and ρt\rho_ t the sampling distribution at noise level tt. It gives slightly better cat FID than Rényi correction (44.68 vs. 45.16, within one seed standard deviation) but leaves the cat proportion unchanged. We attribute the two effects to different noise levels. Changing class proportions requires moving samples between classes, which can only happen at high noise, where noised cats and humans nearly coincide and the cross-class guidance is weak. At low noise the classes are separated, the discriminator easily tells real from generated cats, and the guidance is strong. These low-noise corrections carry through to the final images, and hence to cat FID, while a proportion error leaves the ratio flat within each class.

Boost-and-Skip greatly increases the assigned cat count and lowers cat FID, partly because FID is biased upward for small sample sets. However, it overshoots the 5% target and substantially worsens human FID. The boost places more probability in the tails of the initial distribution, raising the chance of sampling minority modes, but its effect is hard to calibrate. After observing the overshoot at boost 3, we reduced it to 1.5, which reduced the overshoot and improved human FID at the cost of slightly worse cat FID. Boost-and-Skip is a powerful way to sample minority modes, but controlling how far it goes is difficult.

Rényi correction instead aims to reproduce the proportions of the training data. It shares Langevin's equilibrium and changes only the transport toward it, weighting the correction toward regions the sampler under-covers. It moved the cat count toward the 5% target while slightly improving cat FID. Because the equilibrium is unchanged, correction with a suitable α\alpha moves the sampler toward the stationary limit instead of pushing the minority proportion past the target, as the boost does.

Stacked MNIST

MethodFiltered TV, mean ± population SD across seeds
Ancestral0.43530±0.002320.43530 \pm 0.00232
Minority Guidance0.44526±0.004490.44526 \pm 0.00449
Boost-and-Skip (Boost 3, Skip 12)0.45384±0.006190.45384 \pm 0.00619
Boost-and-Skip (Boost 1.5, Skip 12)0.44953±0.004760.44953 \pm 0.00476
Discriminator Guidance0.42013±0.002750.42013 \pm 0.00275
Langevin, K=3K=30.42418±0.003840.42418 \pm 0.00384
Rényi, α=0.5,K=3\alpha=0.5,K=30.41900±0.001850.41900 \pm 0.00185

Minority Guidance and both Boost-and-Skip settings performed worse than ancestral sampling. Discriminator Guidance improves slightly on ancestral sampling and Langevin correction. Overall, Rényi correction still retains the best performance.

AuthorsOpenReview ↗

Response to Reviewer 7qF5

W2 and Q1. Clarification of the FID gain

The ±\pm values are population standard deviations across five sampling seeds, with 500 generated images per seed and sampling setting. We set aside 2000 human reference images and 493 cat reference images for the FID calculation. We compute the class-specific FID on the generated images the classifier assigns to each class, so the number of generated images per class do vary by method and seed. These details have been added to Section 4.3 and Appendix E.3.

We have also added a paired per-seed difference. The full tables are added in Appendix E.3.

Paired comparisons against ancestral sampling

MethodCat count: Δ [95% CI]; raw pCat FID: Δ [95% CI]; raw pHuman FID: Δ [95% CI]; raw p
Boost-and-Skip (Boost 3, Skip 12)+113.2 [+104.4, +122.0]; 3.74e-6-7.25 [-7.97, -6.53]; 9.75e-6+19.86 [+19.20, +20.52]; 1.23e-7
Boost-and-Skip (Boost 1.5, Skip 12)+45.4 [+40.5, +50.3]; 1.31e-5-6.36 [-7.25, -5.47]; 3.77e-5+9.30 [+9.05, +9.55]; 5.57e-8
Minority Guidance+20.2 [+16.8, +23.6]; 8.34e-5+2.10 [+1.74, +2.46]; 8.37e-5+13.90 [+12.75, +15.05]; 4.69e-6
Discriminator Guidance-0.2 [-1.2, +0.8]; 0.6213-2.45 [-2.76, -2.14]; 2.45e-5+0.11 [-0.17, +0.39]; 0.3322
Langevin K=2+0.4 [-1.0, +1.8]; 0.4766-0.60 [-1.16, -0.04]; 0.0413+0.08 [-0.09, +0.25]; 0.2671
Rényi α=0.7, K=2+4.2 [+3.2, +5.2]; 0.0004-1.97 [-2.93, -1.01]; 0.0048+0.08 [-0.06, +0.22]; 0.1951

Paired comparisons against matching Langevin

MethodCat count: Δ [95% CI]; raw pCat FID: Δ [95% CI]; raw pHuman FID: Δ [95% CI]; raw p
Rényi α=0.7, K=1+4.6 [+2.2, +7.0]; 0.0062-1.56 [-2.32, -0.80]; 0.0046-0.03 [-0.40, +0.34]; 0.8340
Rényi α=0.7, K=2+3.8 [+2.8, +4.8]; 0.0005-1.37 [-2.36, -0.38]; 0.0182+0.00 [-0.18, +0.18]; 1.0000
Rényi α=0.7, K=3+3.8 [+1.0, +6.6]; 0.0204-1.33 [-2.38, -0.28]; 0.0246-0.04 [-0.41, +0.33]; 0.7812

Q3. Cost clarification

We are unable to access to the original machine with RTX 5090 that we used for the numbers reported in the paper. We thus performed a more complete profiling run on the stacked-MNIST with similar settings (10,000 evaluated outputs per seed, 64 correction anchors, for correction, we have 1024 generated reference particles and 1024 holdout particles from data). The results for the K=3K=3 profiling run consists of 64 (anchors) × 3 (steps) = 192 refresh calls, each taking 12 optimizer steps: 2,304 online updates per sampling run.

ComponentSecondsShare of wall time
Denoiser92.77668.7%
Online refresh29.25921.7%
Ratio/logit gradients10.4977.8%
Mobility/normalization0.1190.1%
Residual correction overheads2.3801.8%
Total135.031100%

This profiling shows that the additional overhead is mainly dominated by the online refresh. We note that while we used an online refresh in our experiments, a fixed correction policy permits an alternative implementation in which a critic, conditioned on noise level and correction stage, is trained offline on representative trajectories and frozen during sampling. This would remove online optimizer updates and the online refresh overhead.

Clipping and step caps: Clipping keeps the mobility positive, so it preserves the corrector’s stationary density. It does, however, change the mobility, and hence the dynamics relative to the flow we analyse. In the setting the reviewer highlights (K=3K=3, α=0.5\alpha=0.5), 29.07% of weights are clipped and 2.60% of steps are capped (Appendix E.2). Caps shorten the correction time, but the realized correction time is 0.588277 against 0.589247 for Langevin, so the comparison remains matched in correction time.

AuthorsOpenReview ↗

Response to Reviewer 7qF5

Q4. Scope of distributional fidelity in GMM

We agree that improved rare-mode proportions alone do not establish broader distributional fidelity. In the GMM cases, we observe an improvement in TV for smaller α\alpha in the under-represented case. Across all six settings, α=0.7\alpha=0.7 gives the lowest mean mode-frequency TV, 3.5–10% below Langevin correction (α=1\alpha=1), while TV rises for α>1\alpha>1. Thus, there are consistent improvements for the rare-mode proportions and the distributional fidelity throughout these under-represented rare mode settings.

One metric that may deteriorate is that of the mean radius relative to the correct width. For the case of p=0.001p=0.001, we observe that this ratio rises from 1.56 (α=1\alpha=1) to 1.92 (α=0.7\alpha=0.7), while rising less in the other two depleted settings, from 1.26 to 1.34 at p=0.003p=0.003 and from 1.21 to 1.28 at p=0.005p=0.005. Our understanding is that smaller α\alpha does increase the correction mobility in the under-represented mode, pushing more mass toward it, but the correction is not ideal and leaves the rare mode wider. This is actually consistent with the theory for the following considerations.

  • The fixed point is the model’s. The corrector converges to the learned score’s equilibrium. The rare mode is already too wide at α=1\alpha=1, and most so in the rarest setting, consistent with a score fitted to few rare samples. Added mass therefore fills the mode in the model’s wider shape rather than the target’s.
  • The budget is finite. Samples transported from the common modes plausibly arrive at the rare mode’s outskirts with little time to relax inward, and some are stranded between modes on the way. This also explains why there are more unassigned samples.
Reviewer_rC6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper generalises the Langevin corrector of predictor–corrector sampling. Standard correction is the Wasserstein-2 gradient flow of KL divergence to π_t; the authors replace the energy with (1/α)·D_α(ρ‖π_t), keeping the geometry. This gives a nonlinear FPE with normalised mobility w_τ = r_τ^{α−1}/E_ρ[r_τ^{α−1}], r_τ = ρ_τ/π_t, and an equivalent SDE with modulated drift and diffusion. Mobility and its gradient are estimated from an auxiliary discriminator separating current particles from forwarded reference data, whose logit estimates log r; a constant logit offset cancels. The paper proves a fixed-noise dissipation identity, a moving-target identity showing shared probability-flow transport leaves D_α unchanged, and bounds on score-, log-ratio- and normalisation-error effects. Experiments: imbalanced Gaussian mixtures (under- and over-represented via tilted training), stacked MNIST with a 1% rare triple, and a 5% cat / 95% human face mixture.

Critical strengths

The moving-target identity is a clean, non-obvious result. Equation (22) shows that π_t and ϱ_u are transported by the same velocity field, so the common component cancels from d/du E_{α,t} and only reverse-diffusion and correction terms dissipate. Treating an evolving target correctly is a common failure point in this literature, and it is handled properly here; the α→0⁺ limit in Appendix D, recovering reverse KL with mobility π_t/ρ, gives the order parameter concrete meaning rather than leaving it a knob.

The GMM study is genuinely controlled. The signed-sensitivity analysis (κ̂ ≈ 0.020 − 1.003·log r_d(1), correlation −0.873, with a separate fit for the oversampled regime) goes beyond "α<1 helped", and the paper is careful to note that the pointwise mobility argument does not by itself fix the sign of the mode-mass response, since that depends on flux across mode boundaries.

Critical weaknesses

The methods that solve the same problem are never compared. Related work cites Sehwag et al. (2022, hardness guidance), Um et al. (2024, minority guidance) and Um et al. (2025, boost-and-skip), the last two of which require neither retraining nor class labels — exactly the setting claimed as the contribution — and none is run. The stated motivation is recovering minority modes without identities, labels or explicit guidance, but a reader cannot tell whether this works better than, worse than, or the same as existing training-free minority samplers; only that it beats plain Langevin correction, which is not a minority-generation method and is not expected to help. Since the absolute effects are small (cat frequency 3.80% → 4.64% against a 5% target; cat FID 47.13 → 45.16), the case has to be made on relative merit, and the comparator is missing.

The cost of the machinery is unaccounted for. The discriminator must be fitted against the current sampling law and refreshed as sampling proceeds (Appendix B.1), at each noise level and each of 64 correction anchors, plus a normaliser over reference particles. Reported timings (47.5 s ancestral → 53–71 s for Rényi on 10k MNIST samples) cover the correction steps only, not discriminator training. With a benefit of a few percent TV, the burden of showing the trade is favourable falls on the paper. Two smaller points compound this: α differs per experiment (0.7 for GMMs and faces, 0.5 for MNIST) with no a priori selection rule, and the MNIST filtered-TV gain over matched Langevin is 1.22%, within roughly two standard deviations at the reported ±s.

Questions for authors
  1. Can you compare against hardness-based guidance (Sehwag et al., 2022) and/or boost-and-skip (Um et al., 2025) on the face mixture and stacked MNIST? Without it the "beyond existing training-free methods" claim is unestablished.
  2. What is the total overhead of discriminator fitting, refresh and normalisation relative to sampling time, and how does discriminator accuracy degrade as α moves away from 1, given the bound in Eq. 59 scales with |α−1|?
  3. How should α be chosen without sweeping it per dataset? Section 4.1 gives a post-hoc fit; is there a calibration procedure using only log r_d(1)?
  4. In the over-represented regime the training itself is altered via tilted weights. Does that generator exhibit the same failure mode as an ordinarily trained over-representing model?
  5. The faces experiment reports variability for cat counts but not human FID (Appendix E.3). Can you report uncertainty for all FIDs and state the sample sizes?
3 comments
AuthorsOpenReview ↗

Response to Reviewer rC6f

W1 & Q1. Comparison with existing minority samplers

We thank the reviewer for pointing out this gap. We have added Minority Guidance, Boost-and-Skip and Discriminator Guidance to the Stacked-MNIST and face-mixture experiments, using the same DDPM checkpoints.

  • Minority Guidance. We constructed rarity labels from all 33,039 training images. Each image's rarity score is the mean VGG-LPIPS distance between the image and five Tweedie reconstructions from independent forward-noising draws at timestep t=605t=605 of the 1,000-step schedule. We discretized the scores into 100 empirical quantile bins, held out about 5% of each bin for validation, and trained a rarity-classification head on a frozen pretrained noisy-image feature extractor for 6,000 updates.
  • Boost-and-Skip. We used boost factors 3 and 1.5 with a skip of 12 steps.
  • Discriminator Guidance. We ran the fixed denoiser through complete unconditional sampling trajectories, using 512 generated images to train the discriminator and 128 for validation, together with 1,024 real training images balanced between humans and cats. Following Kim et al. (2023), the discriminator separates forward-noised real and generated images at every noise level, and its weights stay frozen during generation.

The face results are in Table 4 of the revised paper and the full comparator protocols and results are in Appendix E.3.

Face mixtures

MethodAssigned cats / 500Cat FIDHuman FID
Ancestral19.0±3.419.0 \pm 3.447.13±1.2147.13\pm1.2117.37±0.5417.37\pm0.54
Minority Guidance39.2±5.439.2 \pm 5.449.23±1.4149.23\pm1.4131.27±1.3131.27\pm1.31
Boost-and-Skip, boost=3,skip=12\text{boost}=3, \text{skip}=12132.2±9.7132.2\pm9.739.88±1.0439.88\pm1.0437.23±0.8837.23\pm0.88
Boost-and-Skip, boost=1.5,skip=12\text{boost}=1.5, \text{skip}=1264.4±6.964.4\pm6.940.77±0.9640.77\pm0.9626.67±0.6426.67\pm0.64
Discriminator Guidance18.8±3.318.8\pm3.344.68±1.0344.68\pm1.0317.48±0.5917.48\pm0.59
Langevin, K=2K=219.4±3.519.4\pm3.546.53±0.9846.53\pm0.9817.45±0.5717.45\pm0.57
Rényi, α=0.7,K=2\alpha=0.7, K=223.2±3.023.2\pm3.045.16±0.8445.16\pm0.8417.45±0.4917.45\pm0.49

Boost-and-Skip greatly increases the assigned cat count and lowers cat FID, partly because FID is biased upward for small sample sets. However, it overshoots the 5% target and substantially worsens human FID. The boost places more probability in the tails of the initial distribution, raising the chance of sampling minority modes, but its effect is hard to calibrate. After observing the overshoot at boost 3, we reduced it to 1.5, which reduced the overshoot and improved human FID at the cost of slightly worse cat FID. Boost-and-Skip is a powerful way to sample minority modes, but controlling how far it goes is difficult.

Rényi correction instead aims to reproduce the proportions of the training data. It shares Langevin's equilibrium and changes only the transport toward it, weighting the correction toward regions the sampler under-covers. It moved the cat count toward the 5% target while slightly improving cat FID. Because the equilibrium is unchanged, correction with a suitable α\alpha moves the sampler toward the stationary limit instead of pushing the minority proportion past the target, as the boost does.

Discriminator Guidance changes the target by adding the gradient of an estimated log density ratio to the model score,

sθ(x,t)+∇xlog⁡πt(x)ρt(x),s_ \theta(x,t)+\nabla_ x\log\frac{\pi_ t(x)}{\rho_ t(x)},

where πt\pi_ t is the noised data distribution and ρt\rho_ t the sampling distribution at noise level tt. It gives slightly better cat FID than Rényi correction (44.68 vs. 45.16, within one seed standard deviation) but leaves the cat proportion unchanged. We attribute the two effects to different noise levels. Changing class proportions requires moving samples between classes, which can only happen at high noise, where noised cats and humans nearly coincide and the cross-class guidance is weak. At low noise the classes are separated, the discriminator easily tells real from generated cats, and the guidance is strong. These low-noise corrections carry through to the final images, and hence to cat FID, while a proportion error leaves the ratio flat within each class.

Stacked MNIST

MethodFiltered TV, mean ± population SD across seeds
Ancestral0.43530±0.002320.43530 \pm 0.00232
Minority Guidance0.44526±0.004490.44526 \pm 0.00449
Boost-and-Skip (Boost 3, Skip 12)0.45384±0.006190.45384 \pm 0.00619
Boost-and-Skip (Boost 1.5, Skip 12)0.44953±0.004760.44953 \pm 0.00476
Discriminator Guidance0.42013±0.002750.42013 \pm 0.00275
Langevin, K=3K=30.42418±0.003840.42418 \pm 0.00384
Rényi, α=0.5,K=3\alpha=0.5,K=30.41900±0.001850.41900 \pm 0.00185

Minority Guidance and both Boost-and-Skip settings performed worse than ancestral sampling. Discriminator Guidance improves slightly on ancestral sampling and Langevin correction. Overall, Rényi correction still retains the best performance.

AuthorsOpenReview ↗

Response to Reviewer rC6f

W2 & Q2. Cost/overhead and discriminator accuracy

We use an RTX 4090 for a profiling run. We are unable to access to the original machine with RTX 5090 that we used for the numbers reported in the paper. The profiling run is performed on the stacked-MNIST with similar settings (10,000 evaluated outputs per seed, 64 correction anchors, for correction, we have 1024 generated reference particles and 1024 holdout particles from data). The results for the K=3K=3 profiling run consists of 64 (anchors) × 3 (steps) = 192 refresh calls, each taking 12 optimizer steps: 2,304 online updates per sampling run.

ComponentSecondsShare of wall time
Denoiser92.77668.7%
Online refresh29.25921.7%
Ratio/logit gradients10.4977.8%
Mobility/normalization0.1190.1%
Residual correction overheads2.3801.8%
Total135.031100%

This profiling shows that the additional overhead is mainly dominated by the online refresh. We note that while we used an online refresh in our experiments, a fixed correction policy permits an alternative implementation in which a critic, conditioned on noise level and correction stage, is trained offline on representative trajectories and frozen during sampling. This would remove online optimizer updates and the online refresh overhead.

In Eq. 59, we bound the mobility factor's error as:

\qquad q_\tau=2|\alpha-1|\varepsilon_\tau+\nu_\tau,$$ where $\epsilon_\tau$ is the upper bound for the log-ratio error and $\nu_\tau$ is the upper bound for the normalization error. The bounds here are worst-case bounds and it does not imply that the estimator accuracy deteriorates as order deviates further from one. ### Q3. Choosing $\alpha$ If we do have prior knowledge of $\log r_d(1)$, a general rule of thumb is to pick $\alpha<1$ (a good default value is $\alpha=0.7$) if $\log r_d(1)<0$ (i.e. underrepresentation case) and to pick $\alpha>1$ (a good default value is $\alpha=1.3$) if $\log r_d(1)>0$ (i.e. overrepresention case). In practical situations however, we do not assume that the end user has access to the training dataset, which makes post-hoc calibration and $\log r_d(1)$ inaccessible. Our recommendation is give the option for sampling multiple copies (e.g. 1 ancestral, 1 with $\alpha=0.7, K=2$, 1 with $\alpha=1.3, K=2$) from the same seed and initial noise. At the expense of longer sampling time, the user could gauge whether the Rényi corrections do bring perceptible improvements for guiding further sampling decisions. ### W2. Significance of stacked-MNIST gains We agree that the filtered TV metrics improvements for the stacked MNIST experiments are marginal. However, we also think that it is consistent with expectations since fundamentally our proposed Rényi sampling is a correction scheme with the same correction limit as Langevin sampling. ### Q4. Tilted training We employed tilted training as a controlled intervention. The exponential tilted training scheme is formulated with a tilting parameter $\gamma$: $$\ell_i(\theta)=\mathbb E_{t,\varepsilon}\big\|\varepsilon-\varepsilon_\theta\big(\sqrt{\bar\alpha_t}\,x^{(i)}+\sqrt{1-\bar\alpha_t}\,\varepsilon,\;t\big)\big\|^2$$ $$\mathcal L_\gamma(\theta)=\frac1\gamma\log\Big(\frac1N\sum_{i=1}^N e^{\gamma\,\ell_i(\theta)}\Big),\qquad \nabla_\theta\mathcal L_\gamma=\sum_{i=1}^N w_i\,\nabla_\theta\ell_i,\qquad w_i=\frac{e^{\gamma\ell_i}}{\sum_j e^{\gamma\ell_j}}.$$ Samples from the rare mode have higher loss and receive larger weight, so training behaves approximately as if on a mixture with an inflated rare-mode weight. The correction target and evaluation weights remain the original mixture. The resulting error is therefore designed to be mainly an error in mode mass, with a systematic direction. We employed this intervention to investigate the case of mode over-representation incurred by training.
AuthorsOpenReview ↗

Response to Reviewer rC6f

Q5. FID uncertainty and sample sizes

We have added the FIDs for the human samples. The uncertainty reported is the population standard deviation across 5 seed runs, where we sampled 500 images per seed. We have also included a paired per-seed T-test for the face mixture results. The full tables are added in Appendix E.3.

Paired comparisons against ancestral sampling

MethodCat count: Δ [95% CI]; raw pCat FID: Δ [95% CI]; raw pHuman FID: Δ [95% CI]; raw p
Boost-and-Skip (Boost 3, Skip 12)+113.2 [+104.4, +122.0]; 3.74e-6-7.25 [-7.97, -6.53]; 9.75e-6+19.86 [+19.20, +20.52]; 1.23e-7
Boost-and-Skip (Boost 1.5, Skip 12)+45.4 [+40.5, +50.3]; 1.31e-5-6.36 [-7.25, -5.47]; 3.77e-5+9.30 [+9.05, +9.55]; 5.57e-8
Minority Guidance+20.2 [+16.8, +23.6]; 8.34e-5+2.10 [+1.74, +2.46]; 8.37e-5+13.90 [+12.75, +15.05]; 4.69e-6
Discriminator Guidance-0.2 [-1.2, +0.8]; 0.6213-2.45 [-2.76, -2.14]; 2.45e-5+0.11 [-0.17, +0.39]; 0.3322
Langevin K=2+0.4 [-1.0, +1.8]; 0.4766-0.60 [-1.16, -0.04]; 0.0413+0.08 [-0.09, +0.25]; 0.2671
Rényi α=0.7, K=2+4.2 [+3.2, +5.2]; 0.0004-1.97 [-2.93, -1.01]; 0.0048+0.08 [-0.06, +0.22]; 0.1951

Paired comparisons against matching Langevin

MethodCat count: Δ [95% CI]; raw pCat FID: Δ [95% CI]; raw pHuman FID: Δ [95% CI]; raw p
Rényi α=0.7, K=1+4.6 [+2.2, +7.0]; 0.0062-1.56 [-2.32, -0.80]; 0.0046-0.03 [-0.40, +0.34]; 0.8340
Rényi α=0.7, K=2+3.8 [+2.8, +4.8]; 0.0005-1.37 [-2.36, -0.38]; 0.0182+0.00 [-0.18, +0.18]; 1.0000
Rényi α=0.7, K=3+3.8 [+1.0, +6.6]; 0.0204-1.33 [-2.38, -0.28]; 0.0246-0.04 [-0.41, +0.33]; 0.7812
Reviewer_GYSYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper aims to recover under-represented modes (for example, few cats among many human faces) when sampling from a pretrained diffusion model, without labels or retraining. It replaces the KL energy behind Langevin correction with a Rényi divergence of order α. Each sample's correction becomes faster or slower depending on whether its region is over- or under-covered, as judged by a discriminator between generated and real noisy samples. α = 1 recovers Langevin correction. The paper claims three contributions: 1.A new correction step that works with any pretrained diffusion model, without labels or retraining. 2.Theory showing that, under ideal conditions, the correction always reduces the gap to the target distribution. 3.Experiments on Gaussian mixtures, stacked MNIST, and a 5% cat / 95% human face dataset. α below 1 is reported to bring back missing rare modes, α above 1 to remove extra ones, and cat image quality to improve

Critical strengths

S1. The paper targets a relevant and practically important problem Recovering under-represented modes at sampling time, without labels, retraining or a classifier, addresses a real gap. Minority modes are often unknown in advance or expensive to annotate. Classifier guidance and classifier-free guidance cannot be applied without them

S2. The method is training-free and modular It needs only a pretrained diffusion model, unlabeled reference data and an auxiliary discriminator. No diffusion-network retraining or class labels are required, and it plugs into standard predictor–corrector sampling.

Critical weaknesses

W1. The method cannot change the stationary distribution, so the theory does not support the main claim With a learned score s_θ, the corrector's Fokker–Planck flux is w(∇ρ − ρ·s_θ). This is zero at the model marginal π̂_t for any positive mobility w. Every α therefore has the same stationary distribution as Langevin correction: the model's distribution, not the data distribution. α changes only the convergence rate. The discriminator does not change this. Its gradient enters the drift through ∇w, which exactly cancels the noise-induced drift of the state-dependent diffusion. Density-ratio information affects speed, not the fixed point. •Score error is out of reach. Mode-weight errors caused by imbalanced training are score error (model error). The method cannot correct them for any α. It can only reduce sampler error (discretization or initialization), which Langevin already reduces. •A 1D check confirms it. I solved the 1D Fokker–Planck equation with true rare mass 0.0757 and model rare mass 0.0475. α ∈ {0.5, 1, 1.5} all converge to 0.0475; α = 0.5 only converges faster. Initialized at the model distribution, no α changes anything. •The theory assumes it away. The dissipation theorems assume an exact score. Under that assumption Langevin also converges to π_t, so the theory gives no reason to prefer α ≠ 1. App. E.1 concedes that the weighting does not determine the sign of mode-mass changes. •Gains are unexplained. Any reported gain must be a finite-time (transient) or discretization effect. The paper analyses neither

W2. The dissipation results are generic Eq. 22 holds for any π_t-invariant corrector by the data-processing inequality, including plain Langevin, so it does not distinguish the proposed method. The paper's own rate constant (Eq. 41) is weaker at α = 0.7 (exponent 3.8) than at α = 1 (exponent 2.0).

Questions for authors

Where the missing mass comes from. For the mixtures, can you show plain sampling (K = 0) and a long Langevin run?

3 comments
AuthorsOpenReview ↗

Response to Reviewer GYSY

W1. Stationary distribution

We thank the reviewer for this observation and agree that Rényi and Langevin correction share the same stationary limit. For a conservative learned score sθ(⋅,t)=∇log⁡π^ts_ \theta(\cdot,t)=\nabla\log\hat\pi_ t, the corrector obeys ∂τρτ=∇⋅[ρτw^τ∇log⁡(ρτ/π^t)]\partial_ \tau\rho_ \tau=\nabla\cdot[\rho_ \tau \widehat{w}_ \tau \nabla\log(\rho_ \tau/\hat\pi_ t)], so π^t\hat\pi_ t has zero flux for every positive mobility, exact or estimated, even when the mobility is computed from an exact ratio to the data target. The ∇w^τ\nabla\widehat{w}_ \tau drift is the Itô correction for the state-dependent diffusion, not an additive repair of the score.

The difference from Langevin correction is therefore the path, not the fixed point. Three distributions need not coincide: the data marginal πt\pi_ t, the equilibrium π^t\hat\pi_ t of the frozen learned score, and the law ρτ\rho_ \tau that the finite-step sampler hands to the corrector. The corrector gets a small fixed budget at each noise level and is never run to stationarity. The mobility redistributes this budget across regions according to the density ratio, changing the trajectory of ρτ\rho_ \tau but not its limit π^t\hat\pi_ t.

Finite-time analysis. We agree the submission lacked one and add the following analysis to Sec 3.3 in the revised paper. Eq. 23 measures each order with its own energy Eα,t\mathcal{E}_ {\alpha,t}, so its rates are not comparable across orders. We instead measure all orders with one common energy, E1,t(ρ)=DKL(ρ∥πt)\mathcal{E}_ {1,t}(\rho)=D_ {\mathrm{KL}}(\rho\Vert\pi_ t). Along the corrector with mobility w^τ\widehat{w}_ \tau and learned score sθs_ \theta,

ddτE1,t(ρτ)=−Eρτ[w^τgτ],gτ=∥∇log⁡rτ∥2−∇log⁡rτ⋅(sθ−st).\frac{d}{d\tau}\mathcal{E}_ {1,t}(\rho_ \tau)=-\mathbb{E}_ {\rho_ \tau}[\widehat{w}_ \tau g_ \tau], \qquad g_ \tau=\Vert\nabla\log r_ \tau\Vert^2-\nabla\log r_ \tau\cdot(s_ \theta-s_ t).

For Langevin correction, w^τ≡1\widehat{w}_ \tau\equiv 1 and gτg_ \tau is its KL dissipation rate. Since Eρτ[w^τ]=1\mathbb{E}_ {\rho_ \tau}[\widehat{w}_ \tau]=1, Rényi correction spends the same total budget but redistributes it. Starting order α\alpha and Langevin from the same ρ0\rho_ 0 and running each for correction time τ\tau,

E1,t(ρτ(α))−E1,t(ρτ(1))=−τCovρ0(w^0,g0)+O(τ2),\mathcal{E}_ {1,t}(\rho^{(\alpha)}_ \tau)-\mathcal{E}_ {1,t}(\rho^{(1)}_ \tau)=-\tau \mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,g_ 0)+O(\tau^2),

Covρ0(w^0,g0)=Covρ0(w^0,∥∇log⁡r0∥2)⏟exact-score gain−Covρ0(w^0,∇log⁡r0⋅(sθ−st))⏟score-error interaction.\mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,g_ 0)=\underbrace{\mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,\Vert\nabla\log r_ 0\Vert^2)}_ {\text{exact-score gain}}-\underbrace{\mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,\nabla\log r_ 0\cdot(s_ \theta-s_ t))}_ {\text{score-error interaction}}.

This answers two of the reviewer's points. First, the orders differ even with an exact score: the first term is nonzero in general, so the orders share the limit πt\pi_ t but not the finite-budget rate. It is positive when the mobility is larger where ∥∇log⁡r0∥2\Vert\nabla\log r_ 0\Vert^2 is larger, and negative otherwise. The second term is not sign-definite and can offset or reverse the first.

In the exact-mobility idealization, we can also characterize the sign of the gain. Expanding the mobility around α=1\alpha=1 gives w=1+(α−1)(log⁡r−E1,t(ρ))+O((α−1)2)w=1+(\alpha-1)(\log r-\mathcal{E}_ {1,t}(\rho))+O((\alpha-1)^2), so

Covρ0(w0,g0)=(α−1)Covρ0(log⁡r0,g0)+O((α−1)2).\mathrm{Cov}_ {\rho_ 0}(w_ 0,g_ 0)=(\alpha-1)\mathrm{Cov}_ {\rho_ 0}(\log r_ 0,g_ 0)+O((\alpha-1)^2).

For a given sampler state, one side of α=1\alpha=1 therefore improves on Langevin to first order and the other does not: α<1\alpha<1 when the dissipation is concentrated where the sampler under-covers the data (log⁡r0\log r_ 0 below its mean), and α>1\alpha>1 when it is concentrated where the sampler over-covers it. This matches our Gaussian-mixture experiments (Section 4.1): where the rare mode is depleted, lowering α\alpha from 1 to 0.7 recovers 11–15% of the rare-mass deficit, and where it is overrepresented, raising α\alpha to 1.3 removes 25–46% of the excess.

Discretisation analysis. The same holds for the implemented KK-step Euler–Maruyama corrector (Eq. 15). One step changes the energy by E1,t(ρk+1)−E1,t(ρk)=(Eρk+1−Eρk)[log⁡rk]+DKL(ρk+1∥ρk)\mathcal{E}_ {1,t}(\rho^{k+1})-\mathcal{E}_ {1,t}(\rho^{k})=(\mathbb{E}_ {\rho^{k+1}}-\mathbb{E}_ {\rho^{k}})[\log r^k]+D_ {\mathrm{KL}}(\rho^{k+1}\Vert\rho^k). To first order in the step size, this equals the change of the continuous flow over the same time: E1,t(ρk+1)=E1,t(ρk)−ϵtEρk[w^kgk]+O(ϵt2)\mathcal{E}_ {1,t}(\rho^{k+1})=\mathcal{E}_ {1,t}(\rho^k)-\epsilon_ t\mathbb{E}_ {\rho^k}[\widehat{w}^k g^k]+O(\epsilon_ t^2). Summing KK steps with total correction time τ=Kϵt\tau=K\epsilon_ t gives the continuous-time comparison,

E1,t(ρ(α),K)−E1,t(ρ(1),K)=−τCovρ0(w^0,g0)+O(τ2),\mathcal{E}_ {1,t}(\rho^{(\alpha),K})-\mathcal{E}_ {1,t}(\rho^{(1),K})=-\tau \mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,g_ 0)+O(\tau^2),

for any number of steps, including K=1K=1. The gain over Langevin is therefore a property of the dynamics, not of the discretisation. The discretisation error enters only at second order: splitting the same τ\tau into KK steps gives O(τ2/K)O(\tau^2/K), which decreases with KK, whereas the gain does not depend on KK.

AuthorsOpenReview ↗

Response to Reviewer GYSY

W2. Generic dissipation and the rate constant

We agree with both observations; the finite-budget analysis in our response to W1 addresses them.

On Eq. 22: monotone decrease of a divergence under a πt\pi_ t-invariant evolution follows from the data-processing inequality, for Langevin correction as much as for ours. We do not offer monotonicity as evidence for the method, and will say so in the revision. Eq. 22 is useful for its explicit form: the transport shared by the sampler and the target cancels exactly, leaving the dissipation in closed form, which is the starting point for our bounds on score and ratio error. What distinguishes our method from Langevin correction is the common-energy comparison in our response to W1. Measuring every order with E1,t=DKL(ρτ∥πt)\mathcal{E}_ {1,t}=D_ {\mathrm{KL}}(\rho_ \tau\Vert\pi_ t) from the same ρ0\rho_ 0, order α\alpha and Langevin differ after correction time τ\tau by −τCovρ0(w^0,g0)+O(τ2)-\tau \mathrm{Cov}_ {\rho_ 0}(\widehat{w}_ 0,g_ 0)+O(\tau^2).

On Eq. 41: you are right that the exponent is 3.8 at α=0.7\alpha=0.7 and 2.0 at α=1\alpha=1. However, this Poincaré constant gives a worst-case bound, and the bound controls a different energy at each order, so ranking the orders by these exponents does not tell the whole story. Our finite-budget analysis makes this precise. Consider a sampler close to the target, ρ0=πt(1+δφ)\rho_ 0=\pi_ t(1+\delta\varphi) with Eπt[φ]=0\mathbb{E}_ {\pi_ t}[\varphi]=0, under the exact score and mobility. Then r0=1+δφr_ 0=1+\delta\varphi, w0=1+(α−1)δφ+O(δ2)w_ 0=1+(\alpha-1)\delta\varphi+O(\delta^2), and the comparison with Langevin becomes

E1,t(ρτ(α))−E1,t(ρτ(1))=−τ(α−1)δ3Eπt[φ∥∇φ∥2]+O(τδ4)+O(τ2).\mathcal{E}_ {1,t}(\rho^{(\alpha)}_ \tau)-\mathcal{E}_ {1,t}(\rho^{(1)}_ \tau)=-\tau(\alpha-1)\delta^3\mathbb{E}_ {\pi_ t}[\varphi\Vert\nabla\varphi\Vert^2]+O(\tau\delta^4)+O(\tau^2).

The leading term is odd in δ\delta: for a given direction φ\varphi, an order that reduces the KL divergence faster than Langevin under the perturbation +δφ+\delta\varphi reduces it more slowly, by the same amount, under −δφ-\delta\varphi. Reversing the sign exchanges under- and over-coverage of the same regions, so this again says that α<1\alpha<1 helps with deficits and α>1\alpha>1 with excesses. No α\alpha improves on Langevin uniformly over sampler states, so a the worst case bound does not adequately reflect the practical benefits of a chosen α\alpha.

Q1. Missing mass

The correction conserves probability, so mass gained by the rare mode must come from the common modes or from the unassigned region. A sample is unassigned when it lies farther from every mode centre than half the minimum centre separation, i.e. if it is stranded between modes or far from each mode center. Lowering α\alpha to 0.70.7 transports samples out of the common modes the sampler over-covers, which lose 1.81.8–3.5×10−33.5\times10^{-3} of the total mass across the six settings. Most of this mass settles in the under-covered common modes, which gain 1.01.0–3.2×10−33.2\times10^{-3}. In the three depleted settings, part of it reaches the rare mode, which gains 0.70.7–3.3×10−43.3\times10^{-4}. A common mode’s change is negatively correlated with its log⁡rk\log r_k in every setting (−0.43-0.43 to −0.83-0.83): the more a mode is over-covered, the more mass it gives up. Not every transported sample completes the journey. Unassigned mass rises by 33–7×10−57\times10^{-5}, consistent with some samples leaving a common mode but being stranded in the gap before reaching another. At p=0.003p=0.003, about 164 of every 48,000 samples leave over-covered modes: about 146 settle in under-covered common modes, 13 reach the rare mode and 3 are stranded in between.

Per your suggestion, we show ancestral sampling and longer correction budgets on the depleted GMM p=0.003p=0.003 checkpoint.

MethodRare-mode ratioRare radiusTVUnassigned ×104\times10^4
Ancestral0.374±0.0300.374\pm0.0301.259±0.0571.259\pm0.0570.02646±0.001130.02646\pm0.001130.06±0.070.06\pm0.07
α=0.7,K=1\alpha=0.7,K=10.467±0.0770.467\pm0.0771.340±0.0531.340\pm0.0530.02338±0.001370.02338\pm0.001370.79±0.360.79\pm0.36
α=1,K=1\alpha=1,K=10.372±0.0270.372\pm0.0271.264±0.0441.264\pm0.0440.02605±0.000840.02605\pm0.000840.08±0.100.08\pm0.10
α=0.7,K=16\alpha=0.7,K=160.479±0.0410.479\pm0.0411.380±0.0931.380\pm0.0930.02334±0.001200.02334\pm0.001200.58±0.270.58\pm0.27
α=1,K=16\alpha=1,K=160.375±0.0310.375\pm0.0311.258±0.0781.258\pm0.0780.0241±0.001380.0241\pm0.001380.08±0.100.08\pm0.10

We note that for the longer Langevin run at K=16K=16 correction steps v.s. the K=1K=1 base case, there is improvements in the TV metric, but little changes in the rare-mode ratio. Transport dynamics may be crucial because improving density within a mode and moving probability between modes occur on different timescales. Langevin correction can rearrange samples within well-populated modes while producing few transitions across the low-density regions separating them. This would explain why additional steps slightly improve TV without appreciably increasing rare-mode occupancy.

Thank you for the response. The shared stationary limit is now clear, and the finite-time analysis is useful. However, its advantage depends on a sign-indefinite covariance that is not evaluated, leaving the connection to the observed mode-mass gains unestablished. Under the stated step-size rule, K=16K=16 also refines a fixed correction time rather than establishes long-time Langevin behaviour. The generic dissipation results provide no comparative advantage over Langevin. I acknowledge the improvements, but my central concerns remain only partially addressed. Thus, I will maintain my previous score of 2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

To all reviewers and the AC

We thank the reviewers for their insightful reviews, careful reading and constructive feedback.

Here is a summary of the key results:

  • Finite-budget analysis (GYSY). Why Rényi correction can improve on Langevin correction even though both share the stationary distribution of the learned score.
  • Mass transport in the GMM experiments (GYSY). Where the mass gained by the rare mode comes from, with ancestral sampling and longer correction runs.
  • Comparison with existing samplers (rC6f, 7qF5). Minority Guidance, Boost-and-Skip and Discriminator Guidance on the face-mixture and Stacked-MNIST experiments.
  • Clarification of results and fidelity gains (rC6f, 7qF5). Definitions of the reported uncertainties, FID sample sizes and per-seed comparisons.
  • Computational cost (rC6f, 7qF5). More detailed profiling of sampling and correction costs.
  • Choosing α\alpha (rC6f). Practical guidance on selecting the order.

Key revisions to the manuscript:

  • Sec. 3.3: Finite-budget comparison of Rényi and Langevin correction.
  • Sec. 4.3: Comparison with existing samplers on the face mixture (Table 4), with paired per-seed comparisons and discussion.
  • Appendix E.2: Stacked-MNIST comparison with existing samplers.
  • Appendix E.3: Full protocols for the face-mixture experiments and comparators, and complete results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.