- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes RepDS, a variant of denoised smoothing that replaces the pixel-space diffusion denoiser with denoising in representation space. A pretrained ViT (DINOv2, DINOv3 or SigLIP2, depending on the setting) serves as a frozen teacher and as the initialization of a -conditioned student, followed by a small residual refinement module. Training first regresses the teacher's clean patch and CLS features from noisy inputs, then adds cross-entropy and consistency losses through frozen probes, and finally fits a classification head. For zero-shot transfer, only the feature-recovery stage is used and the vision–language readout stays fixed. Each fixed yields a deterministic base classifier, which is certified with the standard Cohen et al. (2019) procedure. Experiments cover ImageNet and CIFAR-10 certification, certified zero-shot transfer, segmentation, video classification and ablations, with the main claim being higher ImageNet certified accuracy than diffusion-based pipelines at a much lower per-draw cost.
Critical strengths
- Strong ImageNet certified accuracy at much lower per-draw cost, with sound certification.
- In Table 1, RepDS-H+ has the highest reported certified accuracy at every radius, e.g. 64.2 vs. 54.6 at against the DINOv3-H+ DDS rerun. Its cost is 225.8 GFLOPs per draw versus 1,344.6 for that rerun.
- These are point estimates on 500 images. The margin is large at but only 1.2 points at its smallest, and the comparison is against a quoted result (DMAE-L, 27.5) rather than a rerun.
- The certificate is the unmodified Cohen et al. procedure on a deterministic base classifier, so the speedup does not compromise statistical validity. Combined with roughly 4× lower time than a same-backbone diffusion pipeline, this is the main reason I lean toward acceptance.
Critical weaknesses
-
The source of the gain is not isolated, and the "backbone-matched" baseline may not be equally adapted.
- The DDS + DINOv3-H+ rerun is described as using "a head trained on DDS outputs as the classifier"; whether the encoder is also trained is not stated. RepDS-H+ fine-tunes all Transformer blocks with AdaLN conditioning and consistency regularization. Given that FT-CADIS already fine-tunes the classifier on denoised images, the rerun may not be the strongest DDS variant.
- On ImageNet, the paper lacks both a controlled ablation that removes only the recovery loss and a competitive alternative recipe (the same backbone fine-tuned under Gaussian noise with a jointly trained head and comparable resources).
- The only "no recovery loss" evidence comes from CIFAR-10 with 100 test images (ACR 0.669 vs. 0.749). That variant still trains through probes fitted on clean teacher features, so teacher-derived supervision is not fully removed.
- The paper's own wording is appropriately cautious, but part of the gain may stem from fine-tuning a very strong foundation model under noise.
-
The efficiency positioning against non-diffusion methods is incomplete.
- The abstract scopes its speed claims to diffusion pipelines, which is correct. Against non-diffusion baselines the picture is mixed: on ImageNet, DMAE-L costs 61.6 GFLOPs versus 225.8 for RepDS-H+, and the cheaper RepDS-L falls behind DMAE-L at and .
- On CIFAR-10, rRCM-B is both faster than RepDS-S and more accurate at (39.2 vs. 38.4), yet rRCM is absent from Table 1. Figure 1 plots accuracy against throughput only at , where RepDS-H+ leads most, and without rRCM.
- Neither RepDS-H+ nor DMAE-L dominates on both accuracy and cost, and the paper does not discuss this trade-off explicitly. Training cost is also left out of the efficiency discussion.
Questions for authors
see the weakness part.
3 comments
Response to Reviewer UTLk (2/2)
W2. Accuracy–cost trade-offs with rRCM and DMAE
RepDS and non-diffusion methods occupy different accuracy–cost operating points. We have added rRCM-B to Table 1 and Figure 1, and summarize the ImageNet comparison below.
| ImageNet | N | Time (s / 1,000 draws) | GFLOPs | Cert. time per image (s) | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] |
|---|---|---|---|---|---|---|---|---|---|
| DMAE-L | 10,000 | 5.3 | 61.6 | 53 | 73.6 | 64.6 | 53.7 | 41.5 | 27.5 |
| rRCM-B | 10,000 | 1.8 | 17.7 | 18 | 62.6 | 45.2 | 33.8 | 27.0 | – |
| rRCM-B-Deep | 100,000 | 4.0 | 35.2 | 400 | 64.0 | 51.2 | 40.0 | 32.6 | – |
| RepDS-L | 10,000 | 8.4 | 83.0 | 84 | 77.4 | 67.4 | 55.4 | 35.0 | 20.2 |
| RepDS-H+ | 10,000 | 22.4 | 225.8 | 224 | 81.0 | 73.2 | 64.2 | 48.8 | 29.6 |
N is the number of certification samples per image. Certification time per image is computed from the time per 1,000 draws and N.
Non-diffusion methods such as DMAE and rRCM have low inference cost, but their certified accuracy on ImageNet has so far been limited, even with their largest published models. RepDS-H+ exceeds this level at every radius while removing the diffusion denoiser, which dominates the cost of diffusion-based pipelines. This holds even against the larger rRCM-B-Deep, which is reported only with 100,000 certification samples per image, whereas our evaluation uses 10,000; in these settings, RepDS-H+ also takes less certification time per image (224 vs. 400 seconds). We agree that RepDS-H+ has a higher per-draw cost than DMAE-L, so neither dominates on both accuracy and cost; given the accuracy gap, however, the two are difficult to compare directly as a trade-off. RepDS-L offers a lower-cost operating point: it is more accurate than DMAE-L at radii up to 1.5 and less accurate at radii 2.0 and 3.0.
| CIFAR-10 | N | Time (s / 1,000 draws) | GFLOPs | Cert. time per image (s) | CA@0 | [email protected] | [email protected] | [email protected] | [email protected] |
|---|---|---|---|---|---|---|---|---|---|
| rRCM-B | 100,000 | 0.529 | 5.69 | 52.9 | 83.6 | 73.4 | 61.4 | 48.0 | 39.2 |
| RepDS-S | 100,000 | 0.791 | 6.21 | 79.1 | 91.6 | 83.2 | 70.0 | 52.2 | 38.4 |
| RepDS-S | 10,000 | 0.791 | 6.21 | 7.9 | 90.8 | 80.6 | 68.0 | 49.0 | 37.0 |
CA@0 is the radius-zero certified accuracy at σ = 0.25.
On CIFAR-10, rRCM-B is indeed faster per draw than RepDS-S and 0.8 points more accurate at radius 1.0. At smaller radii, however, RepDS-S is substantially more accurate (by 8.6 points at radius 0.5), which makes a direct accuracy–cost trade-off comparison between the two difficult. Even with 10× fewer certification samples (N = 10,000), RepDS-S remains more accurate at radii up to 0.75 while taking about one seventh of the certification time.
We have added this accuracy–cost discussion to the revision and report the training cost of each method in Appendix C (on ImageNet, 58.6 EFLOPs for RepDS-H+ versus 396.5 for DMAE-L and 1,133.6 for rRCM-B, excluding the pretraining of the public DINOv3 weights).
Thank you for the additional experiments. Most of my concerns have been addressed. I would like to maintain my score of weak accept.
Response to Reviewer UTLk (1/2)
We sincerely appreciate your constructive feedback and the time invested in reviewing our work. We address your concerns one-by-one below and are happy to provide further clarification if needed.
W1-1. Adaptation of the backbone-matched DDS baseline
In the backbone-matched DDS baseline of Table 1, the encoder was frozen and only the classification head was trained. The head uses the same architecture and training recipe as the head of RepDS-H+.
To test a stronger version of this baseline, we additionally fine-tuned the encoder on denoised images. We ran this experiment on CIFAR-10 with DINOv2-S, where all 12 encoder blocks were fine-tuned behind the frozen one-step denoiser.
All methods use the same 500 test images and N = 10,000.
Fine-tuning the encoder improves DDS and makes it competitive with RepDS-S. RepDS-S remains 0.016 higher in ACR at each noise level while taking about half the inference time on CIFAR-10. On ImageNet, the gap is about 4× (89.0 vs. 22.4 seconds per 1,000 draws): the denoiser alone takes 1,119.8 of the 1,344.6 GFLOPs of the backbone-matched DDS baseline, compared with 225.8 GFLOPs in total for RepDS-H+.
We have clarified in the revision that only the classification head was trained for the DDS baseline in Table 1, and added the encoder fine-tuning results to Appendix C.
W1-2. Contribution of clean-feature recovery
The new CIFAR-10 control experiments support a benefit from clean-feature recovery. To remove teacher-derived supervision entirely, we jointly fine-tuned DINOv2-S and a trainable classifier under Gaussian noise using cross-entropy and symmetric-KL consistency, without recovery targets or probes fitted on teacher features. We report both 20k- and 200k-update runs; the latter matches the number of noisy views used to train RepDS.
CIFAR-10, DINOv2-S; all rows use 500 test images, N = 10,000, and identical noise draws.
RepDS exceeds the stronger direct fine-tuning run (20k updates) by 0.048 ACR at σ = 0.25 and 0.063 at σ = 0.5. Separately, within the RepDS pipeline, removing recovery from both stages reduces ACR from 0.569 to 0.534 on the expanded 500-image ablation at σ = 0.25 (Table 5). The first comparison tests a teacher-free alternative recipe; the second measures the contribution of recovery within our pipeline. We have added the teacher-free comparison to the revision (Appendix C).
We agree that part of the gain comes from the strong foundation model, and we view this as an advantage of RepDS: it brings the representation of a pretrained foundation model into randomized smoothing with a single encoder pass. However, on CIFAR-10, neither the foundation model nor noisy fine-tuning alone explains the result: without adaptation, the pretrained encoder is not robust to noise (ACR 0.116 in the table above), and direct noisy fine-tuning of the same backbone remains below RepDS (ACR 0.676 vs. 0.739 at σ = 0.5). On ImageNet, the DDS baseline in Table 1 uses the same DINOv3-H+ encoder yet reaches 54.6% at radius 1.5, compared with 64.2% for RepDS-H+.
Controlled experiments at ImageNet scale are costly: training and certifying a single RepDS-H+ configuration takes an estimated 800 GPU-hours on RTX 6000 Ada GPUs. Given the limited time and compute available during the discussion period, we conducted the controlled comparisons above on CIFAR-10, and will extend them to ImageNet in the final version.