acceptodds
Under review as a conference paper at ICLR 2027

When Exact Likelihood Is Anti-Robust: Diagnosing and Fixing Diffusion Classifiers under Domain Shift

Abstract

Generative classifiers are widely believed to be more robust than their discriminative counterparts. We show that for diffusion classifiers that score classes by exact likelihood the opposite holds under domain shift, and that the failure is structural: it follows from how the likelihood weights noise levels. Decomposing the class-conditional negative log-likelihood across noise levels via the information-theoretic diffusion identity, we find that the fraction of discriminative margin retained under shift rises steadily with noise level. Low-noise terms do not merely degrade out of distribution: classifying with them alone falls below chance, while high-noise terms retain roughly half of their in-domain margin. The exact likelihood integrates uniformly in log-SNR, which under the model's schedule places about three times the uniform-timestep weight on the lowest-noise bucket, precisely where evidence is most fragile. On the same model and the same per-noise-level denoising errors, the three scores order monotonically, exact < uniform-timestep < band-restricted, on all six Rotated MNIST folds and on CIFAR-10-C noise corruptions, where the exact likelihood is at chance and the uniform-timestep score used in practice reaches 36%. The diagnosis implies a one-line fix with no retraining: restrict the evidence integral to a noise band chosen without access to the test domain. On six-fold Rotated MNIST (DomainBed protocol), band restriction lifts a joint diffusion classifier from 57.9% to 92.7% mean OOD accuracy (31% to 87% on the hardest fold). In-domain validation alone selects a biased band; shifting it upward in noise by a fixed offset, as the diagnosis predicts, matches leave-one-domain-out selection while using only single-domain data. Transferred unchanged to CIFAR-10-C, the same rule recovers noise-family accuracy from 36% to 78% and lifts the repaired classifier above a matched ERM ResNet-18 on mean corruption accuracy (82% vs. 75%), though not above one trained with AugMix. Where restriction hurts, on fog, contrast, and the natural shift CIFAR-10.1, the damage sits in the high-noise band, so the per-bucket decomposition predicts which shifts the fix helps. For the shifts we study, these results reverse the robustness narrative for exact-likelihood classification and reduce the repair to a one-line change in the evidence integral.

Primary area:
generative models
Keywords:
diffusion classifiers, generative classifiers, domain shift, out-of-distribution robustness, exact likelihood, diffusion models, domain generalization, common corruptions

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WzfUOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why likelihood-weighted diffusion classifiers fail when faced with domain shift. The authors decomposed class scores into components corresponding to different noise levels, discovering that low-noise terms can bias the classifier toward the wrong class when images undergo rotation or are subjected to noise interference. Consequently, the authors proposed restricting classification to specific noise intervals and averaging the denoising errors within those intervals—an approach that requires no model retraining. This method improved accuracy on the Rotated MNIST dataset from 57.9% to 92.7% and on the CIFAR-10-C noisy dataset from 36.3% to 77.8%, although performance declined for certain other types of shifts.

Critical strengths
  1. This paper identifies a specific failure mechanism: in the presence of domain shift, low-noise evidence can be misleading, and likelihood-based weighting amplifies this effect.
  2. Comparative experiments utilized identical models and denoising errors, enabling an isolated assessment of the impact of scoring rules; experiments involving "bucket removal" and "finer integration" further corroborate this analysis.
  3. The proposed solution is simple to implement and requires no model retraining. Additionally, the paper outlines scenarios where the method fails, thereby clarifying its scope of applicability and the limits of its benefits.
Critical weaknesses
  1. The experiments are limited to small, class-conditioned U-Net models trained on the MNIST and CIFAR datasets. It remains unclear whether the "low-noise failure" phenomenon mentioned in the paper exists in larger-scale pre-trained models or generalizes to text-to-image models or SIT weights. Consequently, the primary issue with this paper stems from the inherent limitations of the diffusion classifier itself.

  2. The analysis above suggests the existence of a baseline method superior to simply discarding unreliable "buckets." If certain buckets are systematically biased toward incorrect classes, their outputs can serve as negative references in contrastive logit extrapolation, specifically: ℓguided=ℓbad+s(ℓgood−ℓbad)\ell_{\mathrm{guided}}=\ell_{\mathrm{bad}}+s(\ell_{\mathrm{good}}-\ell_{\mathrm{bad}}). When "good buckets" and "bad buckets" correspond to the margins of the correct class in the positive and negative directions, respectively, this approach can yield further performance improvements compared to using only "good buckets"; the authors might consider verifying the feasibility of this method.

Questions for authors

No

7 comments
AuthorsOpenReview ↗

Response to Q1 of Reviewer WzfU (1/3)

Thank you for the careful and constructive review! Both of your suggestions could be tested during the discussion period. One clarification on your closing remark in Weakness 1: we agree that part of what the paper documents is a limitation of diffusion classifiers as such, and the paper does not argue otherwise (ERM is stronger on Rotated MNIST, AugMix on CIFAR-10-C). Its claim is narrower: which part of the score causes the collapse, and that this is a property of the weighting rather than of one model, which is what your Q1 asks us to test.

Q1. Does the low-noise failure exist in larger or pretrained models (text-to-image, SiT)?

Yes in pixel-space models, no in the two latent models we tested. (1) With the pretrained class-conditional CIFAR-10 EDM of Karras et al. (2022), every finding of the paper reproduces on all 97 settings. (2) With the pretrained ImageNet-64 EDM (296M parameters), the low-noise failure under noise corruptions reproduces, but the band selected on CIFAR-10 is too narrow and has to be re-selected. (3) With SiT-XL/2 (675M parameters, latent space), the failure under noise corruptions does not appear; the ordering is reversed there. (4) With Stable Diffusion v1.4 (text-to-image, latent space), the exact score and uniform-t are within sampling error and the band gives no gain. Details follow; we report the negative results as they are and will narrow the paper's claim accordingly.

Setup. We use the authors' released edm-cifar10-32x32-cond-vp checkpoint (DDPM++ backbone, 56M parameters, σ-parameterised, trained by its authors with their own log-normal σ sampling and augmentation; our paper's U-Net has 13.3M parameters and was trained by us with the cosine schedule). Nothing else in the pipeline changes: the same 16 log-SNR buckets, the same shared (t, ε) pairs, the same ITD quadrature weights, the same band-selection rules, and the same 2,000 test images per setting (the first 2,000 of the paper's fixed permutation). EDM is queried at σ(t) = exp(−α(t)/2) with ε̂ = (x_σ − D(x_σ))/σ. We evaluated all 97 settings of the paper (15 corruptions × 5 severities, the 4 hold-out corruptions × 5 severities, clean, CIFAR-10.1). The "U-Net" rows below are the paper's model scored on the identical 2,000 images, so the two denoisers are compared on the same inputs.

Score (CIFAR-10-C, N = 2,000 per setting)mean of 15 corruptionsnoise familynon-noiseclean
ERM ResNet-18 (paper, Table 3)75.054.880.095.1
EDM, exact likelihood (ITD quadrature)59.413.670.895.2
EDM, uniform-t average82.263.386.996.4
EDM, band selected on clean data (0–15; the in-domain trap of Sec. 5.3)82.263.386.996.4
EDM, band = clean-selected + 2 (2–15; the paper's locked rule)85.081.985.894.6
EDM, band selected on the 4 hold-out corruptions (3–15; = oracle)85.387.984.693.8
U-Net (paper), exact / uniform-t / band 2–15 / band 3–1556.0 / 74.1 / 81.6 / 82.510.1 / 36.2 / 77.0 / 85.367.4 / 83.5 / 82.8 / 81.991.5 / 93.8 / 91.8 / 91.0

What reproduces.

  1. The ordering exact < uniform-t < band holds for the pretrained model on the noise family (13.6 < 63.3 < 81.9) and over all 15 corruptions (59.4 < 82.2 < 85.0).
  2. Every selection rule picks the same band as for our U-Net: clean selection keeps the full band 0–15, the +2 upshift gives 2–15, and hold-out selection gives 3–15, which is also the oracle band.
  3. The per-bucket curve has the shape of Figure 1 (Figure S1(a) in the supplementary PDF, rebuttal_supplement.pdf): on the noise family the lowest-noise bucket classifies at 4.4% (below the 10% chance floor; our U-Net: 8.0%), buckets 1–2 at 21% and 42%, and accuracy peaks at 80% in buckets 5–6. At severity 5, buckets 0–2 are all below chance (4%, 6%, 13%).
  4. The paper's stated exception reproduces exactly: the only settings on which the exact score beats uniform-t are contrast at severities 2–5 and fog at 3–5 (Sec. 5.3, "and nowhere else"), the same seven of 75 settings.
  5. The boundary case reproduces: on CIFAR-10.1 the locked band (88.0) sits below uniform-t (89.9), as in Appendix E.

EDM is the stronger model throughout (uniform-t 82.2 vs 74.1 for our U-Net on the same images), and its band-restricted classifier reaches 85.3 on the 15 corruptions, matching the AugMix reference in Table 3. At 32×32 the low-noise failure is therefore a property of what the diffusion likelihood weights, not of our architecture, our training, or the ε-parameterisation.

AuthorsOpenReview ↗

Response to Q1 of Reviewer WzfU (2/3)

(Continued, part 2 of 3.)

Larger pretrained models (run during the discussion period). We scored two ImageNet models that we did not train, on the same 500 ImageNet validation images (the 10 Imagenette classes, 50 per class; 10-way classification with the 1000-class model), clean plus 14 corrupted settings (Gaussian, shot and impulse noise, fog, contrast, defocus blur, JPEG; severities 3 and 5; ImageNet-C / Tiny-ImageNet-C constants). The pipeline is unchanged: the same 16 log-SNR buckets, shared (t, ε) pairs and ITD quadrature weights. (a) ImageNet-64 EDM (Karras et al. 2022; ADM backbone, 296M parameters, pixel space). (b) SiT-XL/2 at 256×256 (Ma et al. 2024; 675M parameters, latent space, velocity prediction; the corruption is applied to the image, which is then encoded by the VAE; the buckets are mapped to the interpolant time through log-SNR).

Score (N = 500 per setting)cleannoise family (6 settings)non-noise (8 settings)all 14
ImageNet-64 EDM, exact likelihood (ITD)96.418.469.747.7
ImageNet-64 EDM, uniform-t97.028.471.252.8
ImageNet-64 EDM, band 2–15 (the paper's locked rule)96.432.566.451.9
ImageNet-64 EDM, band 3–15 (the paper's hold-out band)95.440.264.754.2
ImageNet-64 EDM, band selected on the other corruption types (7–15 to 9–15)93–9483.855.067.3
SiT-XL/2, exact likelihood (ITD)95.459.564.962.6
SiT-XL/2, uniform-t97.454.669.963.3
SiT-XL/2, band 2–1597.452.770.562.9
SiT-XL/2, band 3–1597.050.669.861.6

We report these results as they are; they support the diagnosis on the pixel-space model and they do not support it on the latent model.

ImageNet-64 EDM (pixel space): the mechanism reproduces, the numerical band does not transfer. On the noise family the ordering exact < uniform-t < band holds (18.4 < 28.4 < 32.5 < 40.2), and the four lowest-noise buckets classify at 9%, 7%, 7% and 9%, at or below the 10% chance floor, while buckets 8–11 classify at 71–79%. However, the contaminated region is wider than on CIFAR-10 (it reaches bucket 4 at severity 3 and bucket 6 at severity 5, against buckets 0–2 on CIFAR-10), so the bands locked on CIFAR-10 recover only a small part of the gap; a band whose lower edge is selected on the other corruption types (leave-one-corruption-out) reaches 83.8. We have not tested the cause; the corruption constants at 64×64 are stronger than CIFAR-10-C (Gaussian σ of 0.12 and 0.18 against 0.08 and 0.10), and the same log-SNR leaves more recoverable signal at higher resolution. The trade-off the paper reports for fog and contrast is also sharper here: those corruptions are best classified at the clean end (contrast, severity 5: exact 69.6, uniform-t 23.4, band 2–15 8.6), the exact score beats uniform-t on 6 of the 14 settings (contrast and fog at both severities, defocus blur and impulse noise at severity 5), and any band that repairs the noise family costs about 15 points on the non-noise settings. Over all 14 settings the locked band gives no net gain over uniform-t (51.9 vs 52.8).

SiT-XL/2 (latent space): the noise-family result does not reproduce. The order is reversed on all six noise settings (exact 59.5 > uniform-t 54.6 > band 52.7). After the VAE encoder, pixel noise damages the high-noise buckets most (buckets 12–15 classify at 13–26%) and the best buckets are low-noise ones (buckets 2–4, 56–58%). The exact score is still below uniform-t on clean data (95.4 vs 97.4) and on the non-noise settings (64.9 vs 69.9), because the single lowest-noise bucket, which receives 20% of the ITD weight, is the weakest low-noise bucket in every group (55% on clean data); but exact is below uniform-t on only 7 of the 14 shifted settings, the three scores are within sampling error over all 14 (62.6, 63.3, 62.9; the standard error of one setting is about 2 points), and dropping the clean end helps on no group. Our band rule should therefore not be applied to this model.

AuthorsOpenReview ↗

Response to Q1 of Reviewer WzfU (3/3)

(Continued, part 3 of 3.)

Stable Diffusion v1.4 (text-to-image, latent space): no low-noise failure and no gain from the band. We use the model zero-shot with the prompts "a photo of a [class name]" on the same 10 classes (500 images, 32 timesteps; clean data plus 8 corrupted settings: Gaussian, shot and impulse noise at severities 3 and 5, fog and contrast at severity 5). The exact score and uniform-t are within sampling error on every group (clean 95.4 vs 94.8; noise 78.0 vs 77.3; fog and contrast 84.0 vs 82.9; the standard error of one setting is about 2 points). Removing the low-noise end does not help: cutting at the two log-SNR edges of the paper gives 77.4 and 76.7 on the noise settings and 81.8 and 77.8 on fog and contrast (contrast alone: 81.6 exact, 70.2 with the wider cut). As with SiT, pixel noise damages the high-noise timesteps most after the VAE encoder: at severity 5 the timesteps with log-SNR below −4.5 classify at 15–17%, against 44–47% for log-SNR between 3.1 and 0.8. One caveat limits this test: the noise schedule of this model ends at log-SNR 4.25, so only 1 of the 32 timesteps lies above the paper's edge at 3.11, and it carries 6.2% of the ITD weight. That timestep is the weakest of the low-noise ones (33% at severity 5), but it has too little weight to separate the scores (per-setting numbers: Table S3 of the supplementary PDF). Our band rule should not be applied to this model either.

Scope. The evidence is now: two 32×32 class-conditional models on all 97 CIFAR-10-C settings (N = 2,000 each), two ImageNet models on a 10-class, 500-image subset with 14 corrupted settings each, and Stable Diffusion v1.4 on the same subset with 8 corrupted settings (single evaluation, no repeated seeds). The full 1000-way ImageNet-C protocol was beyond the compute we had during the discussion period. In the revision we will (i) restrict the claim "likelihood weighting is anti-robust under noise-like shift" to pixel-space diffusion classifiers, (ii) report the SiT and Stable Diffusion results as negative results for latent models, and (iii) state that the band has to be re-selected per model and resolution; the band numbers in the paper are specific to 32×32.

We will add this experiment as a new subsection of Sec. 5, with full per-corruption, per-severity and per-bucket tables in a new appendix, and move our own U-Net to a secondary role in the revision.

Reviewer_drZLOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper aims to investigate whether diffusion generative models function as better and more robust discriminators compared to standard classifiers. The authors find a correlation between the timestep and the classification capability of the model during evaluation. In particular, they reweight the evaluated likelihood by the timestep, and consequently the PSNR of the diffusion process, to correct predictions.

Leveraging knowledge of these noise components, the authors exploit this information to achieve gains on rotated CIFAR and MNIST datasets for both classification and out-of-distribution (OOD) tasks. Further experiments are provided on CIFAR-10-C and CIFAR-10.1 across different sources of corruption.

The paper shows minor improvements across the previously listed datasets, though it does not achieve state-of-the-art performance in all cases. In the experimental section, the authors demonstrate that excluding certain timesteps during the classification procedure can improve performance, applying a shift-dependent repair in these scenarios.

Critical strengths

The paper addresses the interesting problem of modulating classification performed via diffusion models according to the noise level across timesteps. The application of simple noise-aware reweighting and shifting improves classification performance.

The approach shows generalization across the evaluated benchmarks beyond the tuning datasets.

Also interesting is the analysis of failure cases and the awareness of the method's limitations, alongside the reporting of favorable comparisons.

Critical weaknesses

The paper lacks clarity in introducing the problem, and the discussion of prior work is limited. This lack of clarity extends to the presentation of the analysis and experiments, as well as the metric definitions and setup.

The listed findings are not well motivated. When describing them, the authors rely solely on empirical evidence without providing formal justification. In Finding 3, the authors even contradict the validity of the finding without explanation.

In Section 4, the experimental setup lacks motivation. The choice of hyperparameters, such as the chosen timestep buckets, is unclear and is justified neither relative to the noise levels nor the data. Furthermore, the baselines used for comparison are not discussed in either the experimental section or the related work section, making the manuscript difficult to follow.

Finally, the evaluation shows only marginal improvements in some cases, failing to achieve state-of-the-art performance on the chosen datasets.

Questions for authors

a. How do you motivate the choice of buckets, and how do you relate the choice of intervals to the noise scheduler of the diffusion process?

b. The datasets currently considered are relatively easy for a diffusion process to fit, even when trained with the DDPM framework. This simplicity provides limited insight into how the method would perform on larger, more complex datasets with more intertwined classes.

c. It is not clear why the authors decided to evaluate the method only on rotated data rather than on real data or realistic out-of-distribution (OOD) samples from other datasets.

d. The findings are not clearly supported, so I think that section should be reframed.

10 comments
AuthorsOpenReview ↗

Response to Reviewer drZL, part 1: choice of buckets and relation to the noise schedule (Question a, Weakness 3)

Thank you for the review. Several of your points identify places where the paper is unclear or under-motivated, and we address them with new analyses and concrete revisions.

Q-a. How do you motivate the choice of buckets, and how do the intervals relate to the noise scheduler?

You are right that the submission does not motivate this well, and one of the paper's headline numbers depends on it. We answer in three parts.

What the buckets are. The 16 buckets are equal-width intervals of the diffusion timestep t under the model's T = 1000 cosine schedule; they are an analysis grid, not a tuned hyperparameter. Each bucket corresponds to an interval of log-SNR (bucket 0: [4.4, 10.1]; Appendix A lists all 16 widths). We will move this mapping into Sec. 2 as a table of bucket / timestep range / log-SNR range / σ range.

What depends on the schedule and what does not. The likelihood-derived (ITD) weighting is uniform in log-SNR and does not depend on the schedule. The uniform-timestep score it is compared with does: "three times the weight on the lowest-noise bucket" is specific to the cosine schedule. The paper says "under the model's schedule" but does not examine this, so we did. Using the same saved bucket errors, we re-weighted them as a uniform-timestep sampler would under other schedules, and as the ITD weighting would with different log-SNR truncations.

Weighting of the same 16 bucket errorsweight on bucket 0 (%)Rotated MNIST OODCIFAR-10-C noise / all 15 (our U-Net)CIFAR-10-C noise / all 15 (pretrained EDM)
uniform-t, Stable Diffusion (scaled-linear) schedule1.374.765.8 / 80.578.2 / 85.6
uniform-t, linear-β (DDPM) schedule3.074.144.7 / 76.970.4 / 84.7
uniform-t, cosine schedule (paper)6.257.936.3 / 74.263.3 / 82.2
ITD, log-SNR truncated to [−18.4, 6]6.542.517.1 / 67.534.7 / 73.1
ITD, log-SNR truncated to [−18.4, 8]13.633.911.3 / 60.920.0 / 65.1
ITD, full range [−18.4, 10.1] (paper)19.927.29.8 / 55.913.6 / 59.4
EDM training distribution (log-normal σ)20.112.015.7 / 62.630.2 / 68.5
ITD, log-SNR truncated to [−6, 10.1]35.324.89.6 / 54.813.0 / 58.5
band 2–15 (paper's rule; LODO band on MNIST)092.777.8 / 81.881.9 / 85.0

Two conclusions. First, the size of the gap between the likelihood weighting and uniform-timestep scoring is schedule-specific, as your question anticipates: under the linear or Stable Diffusion schedule a uniform-timestep score puts only 1–3% of its weight on the lowest-noise bucket and is more robust than under the cosine schedule. We will say this explicitly and stop presenting "three times" as if it were schedule-free. Second, the direction is not schedule-specific: across twelve weightings, out-of-distribution accuracy decreases monotonically with the weight placed on the lowest-noise bucket (Spearman ρ = −0.97 on Rotated MNIST, −0.90 on the CIFAR-10-C noise family for both denoisers), while in-domain accuracy barely moves (97.5–91.4% on Rotated MNIST). No weighting that keeps the low-noise buckets reaches the band on the noise family. We also found one dependence the paper does not report: cutting the clean end of the log-SNR range improves the ITD score (27.2 → 42.5 on Rotated MNIST), although it stays far below the band; the paper's Appendix B varies only the high-noise end.

Does the number of buckets matter? From the per-timestep (M = 128) errors on Rotated MNIST we repeated band selection with 8, 16, 32 and 64 buckets, shifting the in-domain band upward by one eighth of the schedule in each case: mean OOD accuracy is 92.4, 92.5, 92.0 and 92.3. Appendix D additionally shows that the ordering survives a 128-point per-timestep quadrature.

We will add this table and the bucket-count check to the paper.

AuthorsOpenReview ↗

Response to Reviewer drZL, part 2: larger and more complex datasets (Question b) (1/2)

Q-b. The datasets are relatively easy for a diffusion process to fit; limited insight into larger, more complex datasets with more intertwined classes.

We agree that MNIST and CIFAR-10 with our own small DDPM are a narrow basis, and two other reviewers raise the same point. We ran additional studies with four pretrained models that we did not train.

(i) Pretrained EDM on CIFAR-10-C (Karras et al. 2022; DDPM++ backbone, 56M parameters, σ-parameterised, trained by its authors). Everything else in the pipeline is unchanged and both denoisers are scored on the identical 2,000 test images per setting (97 settings).

Score (CIFAR-10-C, N = 2,000 per setting)mean of 15 corruptionsnoise familynon-noiseclean
ERM ResNet-18 (paper, Table 3)75.054.880.095.1
EDM, likelihood (ITD) weighting59.413.670.895.2
EDM, uniform-t average82.263.386.996.4
EDM, band = clean-selected + 2 (2–15; the paper's locked rule)85.081.985.894.6
U-Net (paper), ITD / uniform-t / band 2–1556.0 / 74.1 / 81.610.1 / 36.2 / 77.067.4 / 83.5 / 82.891.5 / 93.8 / 91.8

The ordering, the selected bands, the below-chance lowest-noise bucket (4.4% on the noise family) and the fog/contrast exception all reproduce (Figure S1 in the supplementary PDF, rebuttal_supplement.pdf).

(ii) Two pretrained ImageNet models. We scored two ImageNet models that we did not train, on the same 500 ImageNet validation images (the 10 Imagenette classes, 50 per class; 10-way classification with the 1000-class model), clean plus 14 corrupted settings (Gaussian, shot and impulse noise, fog, contrast, defocus blur, JPEG; severities 3 and 5; ImageNet-C / Tiny-ImageNet-C constants). The pipeline is unchanged: the same 16 log-SNR buckets, shared (t, ε) pairs and ITD quadrature weights. (a) ImageNet-64 EDM (Karras et al. 2022; ADM backbone, 296M parameters, pixel space). (b) SiT-XL/2 at 256×256 (Ma et al. 2024; 675M parameters, latent space, velocity prediction; the corruption is applied to the image, which is then encoded by the VAE; the buckets are mapped to the interpolant time through log-SNR).

Score (N = 500 per setting)cleannoise family (6 settings)non-noise (8 settings)all 14
ImageNet-64 EDM, exact likelihood (ITD)96.418.469.747.7
ImageNet-64 EDM, uniform-t97.028.471.252.8
ImageNet-64 EDM, band 2–15 (the paper's locked rule)96.432.566.451.9
ImageNet-64 EDM, band 3–15 (the paper's hold-out band)95.440.264.754.2
ImageNet-64 EDM, band selected on the other corruption types (7–15 to 9–15)93–9483.855.067.3
SiT-XL/2, exact likelihood (ITD)95.459.564.962.6
SiT-XL/2, uniform-t97.454.669.963.3
SiT-XL/2, band 2–1597.452.770.562.9
SiT-XL/2, band 3–1597.050.669.861.6

We report these results as they are; they support the diagnosis on the pixel-space model and they do not support it on the latent model.

ImageNet-64 EDM (pixel space): the mechanism reproduces, the numerical band does not transfer. On the noise family the ordering exact < uniform-t < band holds (18.4 < 28.4 < 32.5 < 40.2), and the four lowest-noise buckets classify at 9%, 7%, 7% and 9%, at or below the 10% chance floor, while buckets 8–11 classify at 71–79%. However, the contaminated region is wider than on CIFAR-10 (it reaches bucket 4 at severity 3 and bucket 6 at severity 5, against buckets 0–2 on CIFAR-10), so the bands locked on CIFAR-10 recover only a small part of the gap; a band whose lower edge is selected on the other corruption types (leave-one-corruption-out) reaches 83.8. We have not tested the cause; the corruption constants at 64×64 are stronger than CIFAR-10-C (Gaussian σ of 0.12 and 0.18 against 0.08 and 0.10), and the same log-SNR leaves more recoverable signal at higher resolution. The trade-off the paper reports for fog and contrast is also sharper here: those corruptions are best classified at the clean end (contrast, severity 5: exact 69.6, uniform-t 23.4, band 2–15 8.6), the exact score beats uniform-t on 6 of the 14 settings (contrast and fog at both severities, defocus blur and impulse noise at severity 5), and any band that repairs the noise family costs about 15 points on the non-noise settings. Over all 14 settings the locked band gives no net gain over uniform-t (51.9 vs 52.8).

AuthorsOpenReview ↗

Response to Reviewer drZL, part 3: realistic out-of-distribution data (Question c)

Q-c. Why evaluate only on rotated data rather than on real data or realistic OOD samples from other datasets?

Thank you for raising this; the paper's organisation invites this reading, since the figures and the three Findings in the main text are all on Rotated MNIST. To clarify what is evaluated: rotation is applied only to MNIST, which serves as the diagnostic benchmark; CIFAR-10 is evaluated under the 15 common corruptions of CIFAR-10-C at five severities (Sec. 5.3, Tables 3 and 8–9) and on CIFAR-10.1, a re-collected natural test set with no synthetic transformation (Appendix E, Table 12). (The quantity that is re-weighted is the per-noise-level denoising error; no PSNR is involved.)

We also want to be direct about what the natural-shift result shows, because it bears on your concern. On CIFAR-10.1 the band does not help: 83.4% against 86.8% for the uniform-timestep score (three seeds; the pretrained EDM gives 88.0 vs 89.9). The per-bucket profile explains why, since CIFAR-10.1 preserves the low-noise evidence that noise-type corruptions destroy, but the practical statement is that the repair is specific to shifts that damage fine-scale evidence. We do not have a cross-dataset benchmark such as PACS or ImageNet-R, and we will not claim coverage of such shifts. In the revision we will move the CIFAR-10.1 result and the CIFAR-10-C per-bucket figure from the appendix into the main text, and state in the abstract that the benefit is confined to rotation and noise-type corruptions.

Reviewer_UZNNOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the robustness of diffusion-based generative classifiers under domain shift. The central observation is that likelihood-derived scoring can perform substantially worse out of distribution than the commonly used uniform-timestep diffusion-classifier score.

Critical strengths

The study provides a mechanistic analysis rather than only demonstrating another diffusion-classifier scoring heuristic. The per-noise-level decomposition shows where discriminative evidence fails under domain shift and connects this failure to the weighting induced by likelihood-derived scoring.

Critical weaknesses

The empirical scope is still relatively narrow for the broader anti-robustness framing. The main evidence comes from Rotated MNIST and CIFAR-10-C, with CIFAR-10.1 used as a boundary case. These analysis seems to only cover a limited range of architectures, dataset scales, and domain shifts.

Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Response to Reviewer UZNN (1/2)

Thank you for the positive assessment and for the clear statement of the main concern.

"The empirical scope is narrow: one architecture, small datasets, a limited range of shift types."

We addressed the architecture concern directly during the discussion period by repeating the full CIFAR-10-C study with a pretrained, externally trained denoiser: the class-conditional CIFAR-10 EDM of Karras et al. (2022), DDPM++ backbone, 56M parameters, σ-parameterised, trained by its authors with their own schedule and augmentation (our U-Net: 13.3M parameters, ε-parameterised, cosine schedule, trained by us). Everything else in the pipeline is unchanged, and the two denoisers are scored on the identical 2,000 test images per setting (97 settings).

Score (CIFAR-10-C, N = 2,000 per setting)mean of 15 corruptionsnoise familynon-noiseclean
ERM ResNet-18 (paper, Table 3)75.054.880.095.1
EDM, exact likelihood (ITD quadrature)59.413.670.895.2
EDM, uniform-t average82.263.386.996.4
EDM, band = clean-selected + 2 (2–15; the paper's locked rule)85.081.985.894.6
EDM, band selected on the 4 hold-out corruptions (3–15; = oracle)85.387.984.693.8
U-Net (paper), exact / uniform-t / band 2–15 / band 3–1556.0 / 74.1 / 81.6 / 82.510.1 / 36.2 / 77.0 / 85.367.4 / 83.5 / 82.8 / 81.991.5 / 93.8 / 91.8 / 91.0

The ordering exact < uniform-t < band, the selected bands, the below-chance lowest-noise bucket (4.4% on the noise family; our U-Net 8.0%), the fog/contrast exception and the CIFAR-10.1 boundary case all reproduce on the pretrained model (Figure S1 in the supplementary PDF, rebuttal_supplement.pdf), which is also the stronger classifier (uniform-t 82.2 vs 74.1 on the same images).

On dataset scale, we also ran two pretrained ImageNet models. We scored two ImageNet models that we did not train, on the same 500 ImageNet validation images (the 10 Imagenette classes, 50 per class; 10-way classification with the 1000-class model), clean plus 14 corrupted settings (Gaussian, shot and impulse noise, fog, contrast, defocus blur, JPEG; severities 3 and 5; ImageNet-C / Tiny-ImageNet-C constants). The pipeline is unchanged: the same 16 log-SNR buckets, shared (t, ε) pairs and ITD quadrature weights. (a) ImageNet-64 EDM (Karras et al. 2022; ADM backbone, 296M parameters, pixel space). (b) SiT-XL/2 at 256×256 (Ma et al. 2024; 675M parameters, latent space, velocity prediction; the corruption is applied to the image, which is then encoded by the VAE; the buckets are mapped to the interpolant time through log-SNR).

Score (N = 500 per setting)cleannoise family (6 settings)non-noise (8 settings)all 14
ImageNet-64 EDM, exact likelihood (ITD)96.418.469.747.7
ImageNet-64 EDM, uniform-t97.028.471.252.8
ImageNet-64 EDM, band 2–15 (the paper's locked rule)96.432.566.451.9
ImageNet-64 EDM, band 3–15 (the paper's hold-out band)95.440.264.754.2
ImageNet-64 EDM, band selected on the other corruption types (7–15 to 9–15)93–9483.855.067.3
SiT-XL/2, exact likelihood (ITD)95.459.564.962.6
SiT-XL/2, uniform-t97.454.669.963.3
SiT-XL/2, band 2–1597.452.770.562.9
SiT-XL/2, band 3–1597.050.669.861.6

We report these results as they are; they support the diagnosis on the pixel-space model and they do not support it on the latent model.

AuthorsOpenReview ↗

Response to Reviewer UZNN (2/2)

(Continued, part 2 of 2.)

ImageNet-64 EDM (pixel space): the mechanism reproduces, the numerical band does not transfer. On the noise family the ordering exact < uniform-t < band holds (18.4 < 28.4 < 32.5 < 40.2), and the four lowest-noise buckets classify at 9%, 7%, 7% and 9%, at or below the 10% chance floor, while buckets 8–11 classify at 71–79%. However, the contaminated region is wider than on CIFAR-10 (it reaches bucket 4 at severity 3 and bucket 6 at severity 5, against buckets 0–2 on CIFAR-10), so the bands locked on CIFAR-10 recover only a small part of the gap; a band whose lower edge is selected on the other corruption types (leave-one-corruption-out) reaches 83.8. We have not tested the cause; the corruption constants at 64×64 are stronger than CIFAR-10-C (Gaussian σ of 0.12 and 0.18 against 0.08 and 0.10), and the same log-SNR leaves more recoverable signal at higher resolution. The trade-off the paper reports for fog and contrast is also sharper here: those corruptions are best classified at the clean end (contrast, severity 5: exact 69.6, uniform-t 23.4, band 2–15 8.6), the exact score beats uniform-t on 6 of the 14 settings (contrast and fog at both severities, defocus blur and impulse noise at severity 5), and any band that repairs the noise family costs about 15 points on the non-noise settings. Over all 14 settings the locked band gives no net gain over uniform-t (51.9 vs 52.8).

SiT-XL/2 (latent space): the noise-family result does not reproduce. The order is reversed on all six noise settings (exact 59.5 > uniform-t 54.6 > band 52.7). After the VAE encoder, pixel noise damages the high-noise buckets most (buckets 12–15 classify at 13–26%) and the best buckets are low-noise ones (buckets 2–4, 56–58%). The exact score is still below uniform-t on clean data (95.4 vs 97.4) and on the non-noise settings (64.9 vs 69.9), because the single lowest-noise bucket, which receives 20% of the ITD weight, is the weakest low-noise bucket in every group (55% on clean data); but exact is below uniform-t on only 7 of the 14 shifted settings, the three scores are within sampling error over all 14 (62.6, 63.3, 62.9; the standard error of one setting is about 2 points), and dropping the clean end helps on no group. Our band rule should therefore not be applied to this model.

Stable Diffusion v1.4 (text-to-image, latent space). We also scored this model zero-shot on the same 10 classes (clean plus 8 corrupted settings). It behaves like SiT: the exact score and uniform-t are within sampling error (noise settings 78.0 vs 77.3; fog and contrast 84.0 vs 82.9), and removing the low-noise end gives no gain (77.4 on the noise settings). Its noise schedule barely reaches the low-noise range of the paper, so this is a weak test; details are in our response to Reviewer WzfU (Q1) and in Table S3 of the supplementary PDF.

On shift types: the paper covers rotation (Rotated MNIST, six folds), 19 synthetic corruptions at five severities (CIFAR-10-C, including the four hold-out corruptions), and one natural dataset shift (CIFAR-10.1, Appendix E), and it reports the shifts on which the fix does not help (fog, contrast, CIFAR-10.1) together with the per-bucket profile that explains why. The ImageNet studies use a 10-class, 500-image subset; the full 1000-way ImageNet-C protocol was beyond the compute we had during the discussion period. In the revision we will restrict the claim about noise-like shift to pixel-space models, add both ImageNet studies and the Stable Diffusion study, including the negative results on the two latent models, and state that the band must be re-selected per model and resolution.

Please let us know if there are any concerns we have not fully addressed, or any points you would like us to clarify. We would be happy to discuss them further during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.