acceptodds
Under review as a conference paper at ICLR 2027

Fast Randomized Smoothing via Representation Space Denoising

Abstract

Diffusion Denoised Smoothing (DDS) is currently a prominent approach for obtaining certified robustness, but its prohibitively high inference cost remains a major barrier to practical deployment. Specifically, each Monte Carlo draw in the smoothing process requires a large diffusion denoiser to reconstruct an image in pixel space, and a separate pretrained model to classify the denoised image. In this paper, we propose Representation-space Denoised Smoothing (**RepDS**), a framework that replaces pixel reconstruction with denoising in the feature space of a pretrained vision encoder. Given a Gaussian-corrupted image, RepDS adapts the encoder to recover representations aligned with those of the corresponding clean image, directly supervising the features used for prediction instead of relying on pixel-space fidelity as an indirect proxy. At inference, RepDS predicts from the recovered features in a single non-generative forward pass per smoothing draw, eliminating the need for both pixel-space reconstruction and an additional encoder pass over the reconstructed image. Extensive experiments show that RepDS achieves state-of-the-art certified accuracy across all reported ImageNet radii while substantially reducing inference cost. At radius 1.5, RepDS-H+ reaches 64.2% certified accuracy compared with 54.6% for our backbone-matched DDS rerun, while running approximately faster under the same timing protocol. Compared with DensePure, RepDS-H+ reduces per-draw inference time by over three orders of magnitude.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
certified robustness, randomized smoothing, denoised smoothing, adversarial robustness, representation learning, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_UTLkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes RepDS, a variant of denoised smoothing that replaces the pixel-space diffusion denoiser with denoising in representation space. A pretrained ViT (DINOv2, DINOv3 or SigLIP2, depending on the setting) serves as a frozen teacher and as the initialization of a σ\sigma-conditioned student, followed by a small residual refinement module. Training first regresses the teacher's clean patch and CLS features from noisy inputs, then adds cross-entropy and consistency losses through frozen probes, and finally fits a classification head. For zero-shot transfer, only the feature-recovery stage is used and the vision–language readout stays fixed. Each fixed σ\sigma yields a deterministic base classifier, which is certified with the standard Cohen et al. (2019) procedure. Experiments cover ImageNet and CIFAR-10 certification, certified zero-shot transfer, segmentation, video classification and ablations, with the main claim being higher ImageNet certified accuracy than diffusion-based pipelines at a much lower per-draw cost.

Critical strengths
  1. Strong ImageNet certified accuracy at much lower per-draw cost, with sound certification.
    • In Table 1, RepDS-H+ has the highest reported certified accuracy at every radius, e.g. 64.2 vs. 54.6 at ϵ=1.5\epsilon=1.5 against the DINOv3-H+ DDS rerun. Its cost is 225.8 GFLOPs per draw versus 1,344.6 for that rerun.
    • These are point estimates on 500 images. The margin is large at ϵ=1.5\epsilon=1.5 but only 1.2 points at its smallest, and the ϵ=3.0\epsilon=3.0 comparison is against a quoted result (DMAE-L, 27.5) rather than a rerun.
    • The certificate is the unmodified Cohen et al. procedure on a deterministic base classifier, so the speedup does not compromise statistical validity. Combined with roughly 4× lower time than a same-backbone diffusion pipeline, this is the main reason I lean toward acceptance.
Critical weaknesses
  1. The source of the gain is not isolated, and the "backbone-matched" baseline may not be equally adapted.

    • The DDS + DINOv3-H+ rerun is described as using "a head trained on DDS outputs as the classifier"; whether the encoder is also trained is not stated. RepDS-H+ fine-tunes all Transformer blocks with AdaLN conditioning and consistency regularization. Given that FT-CADIS already fine-tunes the classifier on denoised images, the rerun may not be the strongest DDS variant.
    • On ImageNet, the paper lacks both a controlled ablation that removes only the recovery loss and a competitive alternative recipe (the same backbone fine-tuned under Gaussian noise with a jointly trained head and comparable resources).
    • The only "no recovery loss" evidence comes from CIFAR-10 with 100 test images (ACR 0.669 vs. 0.749). That variant still trains through probes fitted on clean teacher features, so teacher-derived supervision is not fully removed.
    • The paper's own wording is appropriately cautious, but part of the gain may stem from fine-tuning a very strong foundation model under noise.
  2. The efficiency positioning against non-diffusion methods is incomplete.

    • The abstract scopes its speed claims to diffusion pipelines, which is correct. Against non-diffusion baselines the picture is mixed: on ImageNet, DMAE-L costs 61.6 GFLOPs versus 225.8 for RepDS-H+, and the cheaper RepDS-L falls behind DMAE-L at ϵ=2.0\epsilon=2.0 and 3.03.0.
    • On CIFAR-10, rRCM-B is both faster than RepDS-S and more accurate at ϵ=1.0\epsilon=1.0 (39.2 vs. 38.4), yet rRCM is absent from Table 1. Figure 1 plots accuracy against throughput only at r=1.5r=1.5, where RepDS-H+ leads most, and without rRCM.
    • Neither RepDS-H+ nor DMAE-L dominates on both accuracy and cost, and the paper does not discuss this trade-off explicitly. Training cost is also left out of the efficiency discussion.
Questions for authors

see the weakness part.

3 comments
AuthorsOpenReview ↗

Response to Reviewer UTLk (1/2)

We sincerely appreciate your constructive feedback and the time invested in reviewing our work. We address your concerns one-by-one below and are happy to provide further clarification if needed.

W1-1. Adaptation of the backbone-matched DDS baseline

In the backbone-matched DDS baseline of Table 1, the encoder was frozen and only the classification head was trained. The head uses the same architecture and training recipe as the head of RepDS-H+.

To test a stronger version of this baseline, we additionally fine-tuned the encoder on denoised images. We ran this experiment on CIFAR-10 with DINOv2-S, where all 12 encoder blocks were fine-tuned behind the frozen one-step denoiser.

MethodTime (s / 1,000 draws)ACR (σ=0.25)[email protected] (σ=0.25)ACR (σ=0.5)[email protected] (σ=0.5)[email protected] (σ=0.5)
DDS, frozen encoder1.5480.50055.40.66451.033.6
DDS, fine-tuned encoder1.5480.55364.60.72357.436.6
RepDS-S0.7910.56968.00.73958.237.0

All methods use the same 500 test images and N = 10,000.

Fine-tuning the encoder improves DDS and makes it competitive with RepDS-S. RepDS-S remains 0.016 higher in ACR at each noise level while taking about half the inference time on CIFAR-10. On ImageNet, the gap is about 4× (89.0 vs. 22.4 seconds per 1,000 draws): the denoiser alone takes 1,119.8 of the 1,344.6 GFLOPs of the backbone-matched DDS baseline, compared with 225.8 GFLOPs in total for RepDS-H+.

We have clarified in the revision that only the classification head was trained for the DDS baseline in Table 1, and added the encoder fine-tuning results to Appendix C.

W1-2. Contribution of clean-feature recovery

The new CIFAR-10 control experiments support a benefit from clean-feature recovery. To remove teacher-derived supervision entirely, we jointly fine-tuned DINOv2-S and a trainable classifier under Gaussian noise using cross-entropy and symmetric-KL consistency, without recovery targets or probes fitted on teacher features. We report both 20k- and 200k-update runs; the latter matches the number of noisy views used to train RepDS.

MethodACR (σ=0.25)[email protected] (σ=0.25)ACR (σ=0.5)[email protected] (σ=0.5)[email protected] (σ=0.5)
Pretrained DINOv2-S (no adaptation)0.15912.40.1169.04.6
Direct fine-tuning (20k updates)0.52160.20.67651.436.8
Direct fine-tuning (200k updates)0.39247.40.53241.027.4
RepDS-S0.56968.00.73958.237.0

CIFAR-10, DINOv2-S; all rows use 500 test images, N = 10,000, and identical noise draws.

RepDS exceeds the stronger direct fine-tuning run (20k updates) by 0.048 ACR at σ = 0.25 and 0.063 at σ = 0.5. Separately, within the RepDS pipeline, removing recovery from both stages reduces ACR from 0.569 to 0.534 on the expanded 500-image ablation at σ = 0.25 (Table 5). The first comparison tests a teacher-free alternative recipe; the second measures the contribution of recovery within our pipeline. We have added the teacher-free comparison to the revision (Appendix C).

We agree that part of the gain comes from the strong foundation model, and we view this as an advantage of RepDS: it brings the representation of a pretrained foundation model into randomized smoothing with a single encoder pass. However, on CIFAR-10, neither the foundation model nor noisy fine-tuning alone explains the result: without adaptation, the pretrained encoder is not robust to noise (ACR 0.116 in the table above), and direct noisy fine-tuning of the same backbone remains below RepDS (ACR 0.676 vs. 0.739 at σ = 0.5). On ImageNet, the DDS baseline in Table 1 uses the same DINOv3-H+ encoder yet reaches 54.6% at radius 1.5, compared with 64.2% for RepDS-H+.

Controlled experiments at ImageNet scale are costly: training and certifying a single RepDS-H+ configuration takes an estimated 800 GPU-hours on RTX 6000 Ada GPUs. Given the limited time and compute available during the discussion period, we conducted the controlled comparisons above on CIFAR-10, and will extend them to ImageNet in the final version.

AuthorsOpenReview ↗

Response to Reviewer UTLk (2/2)

W2. Accuracy–cost trade-offs with rRCM and DMAE

RepDS and non-diffusion methods occupy different accuracy–cost operating points. We have added rRCM-B to Table 1 and Figure 1, and summarize the ImageNet comparison below.

ImageNetNTime (s / 1,000 draws)GFLOPsCert. time per image (s)[email protected][email protected][email protected][email protected][email protected]
DMAE-L10,0005.361.65373.664.653.741.527.5
rRCM-B10,0001.817.71862.645.233.827.0–
rRCM-B-Deep100,0004.035.240064.051.240.032.6–
RepDS-L10,0008.483.08477.467.455.435.020.2
RepDS-H+10,00022.4225.822481.073.264.248.829.6

N is the number of certification samples per image. Certification time per image is computed from the time per 1,000 draws and N.

Non-diffusion methods such as DMAE and rRCM have low inference cost, but their certified accuracy on ImageNet has so far been limited, even with their largest published models. RepDS-H+ exceeds this level at every radius while removing the diffusion denoiser, which dominates the cost of diffusion-based pipelines. This holds even against the larger rRCM-B-Deep, which is reported only with 100,000 certification samples per image, whereas our evaluation uses 10,000; in these settings, RepDS-H+ also takes less certification time per image (224 vs. 400 seconds). We agree that RepDS-H+ has a higher per-draw cost than DMAE-L, so neither dominates on both accuracy and cost; given the accuracy gap, however, the two are difficult to compare directly as a trade-off. RepDS-L offers a lower-cost operating point: it is more accurate than DMAE-L at radii up to 1.5 and less accurate at radii 2.0 and 3.0.

CIFAR-10NTime (s / 1,000 draws)GFLOPsCert. time per image (s)CA@0[email protected][email protected][email protected][email protected]
rRCM-B100,0000.5295.6952.983.673.461.448.039.2
RepDS-S100,0000.7916.2179.191.683.270.052.238.4
RepDS-S10,0000.7916.217.990.880.668.049.037.0

CA@0 is the radius-zero certified accuracy at σ = 0.25.

On CIFAR-10, rRCM-B is indeed faster per draw than RepDS-S and 0.8 points more accurate at radius 1.0. At smaller radii, however, RepDS-S is substantially more accurate (by 8.6 points at radius 0.5), which makes a direct accuracy–cost trade-off comparison between the two difficult. Even with 10× fewer certification samples (N = 10,000), RepDS-S remains more accurate at radii up to 0.75 while taking about one seventh of the certification time.

We have added this accuracy–cost discussion to the revision and report the training cost of each method in Appendix C (on ImageNet, 58.6 EFLOPs for RepDS-H+ versus 396.5 for DMAE-L and 1,133.6 for rRCM-B, excluding the pretraining of the public DINOv3 weights).

Thank you for the additional experiments. Most of my concerns have been addressed. I would like to maintain my score of weak accept.

Reviewer_kCsUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Representation-space Denoised Smoothing (RepDS), which replaces the pixel-space denoising and subsequent image encoding of diffusion denoised smoothing with direct recovery of clean visual representations. A noise-conditioned student encoder and lightweight refinement blocks are trained to match patch and global features from a frozen clean-image teacher. A second stage introduces classification and prediction-consistency objectives through fixed auxiliary probes, followed by training a final classification head. The resulting classifier uses the standard randomized-smoothing certificate. Experiments cover ImageNet and CIFAR-10, zero-shot classification with SigLIP2, and transfer to semantic segmentation and video classification.

Critical strengths
  1. The method addresses a meaningful practical bottleneck with a clear and well-motivated design. Recovering the representations needed for prediction avoids reconstructing an image and encoding it again for every smoothing sample. Since certification requires thousands of such evaluations, reducing the cost of each evaluation has substantial practical value. The method also retains the standard randomized-smoothing guarantee without requiring a new certification procedure.
  2. The main results demonstrate a strong accuracy-efficiency trade-off under backbone-matched comparisons. On ImageNet, RepDS-H+ improves certified accuracy at radius 1.5 from 54.6% to 64.2% while reducing the measured time per 1,000 draws from 89.0 to 22.4 seconds. The DDS baseline uses the same DINOv3 backbone and a head trained on its denoised outputs, making this a meaningful comparison. Improvements across all reported radii on both ImageNet and CIFAR-10 against the matched DDS baselines strengthen the empirical case.
  3. The experiments support the central feature-recovery objective and demonstrate useful breadth. With the representation-training budget held fixed, removing recovery throughout training reduces ACR from 0.749 to 0.669 in Table 5. This provides evidence that clean-teacher supervision contributes beyond classification and consistency training within the proposed pipeline. The SigLIP2, segmentation, and video experiments further demonstrate that the recovered representations can support different readouts and tasks, suggesting potential beyond a single classification benchmark.
Critical weaknesses
  1. The distinction from existing representation-based approaches could be better isolated. Comparisons against the closest alternatives, or joint encoder-classifier fine-tuning with Gaussian augmentation and consistency regularization, using the same backbone and adaptation budget would clarify the specific benefit of direct clean-feature recovery beyond the demonstrated improvements over DDS.

  2. The zero-shot results compare different backbone capacities and adaptation budgets. RepDS uses SigLIP2-SO400M/16 with 200,000 ImageNet adaptation updates, whereas DDS uses a frozen SigLIP2-B/16. As acknowledged by the authors, Table 3 supports an end-to-end pipeline comparison rather than isolating the recovery mechanism. A matched-backbone comparison would strengthen this result.

  3. Evidence for some individual components remains limited. The ablations use only 100 test images without training-run variability, while several gains are small or noise-level dependent. Removing consistency also removes both cross-entropy and symmetric KL, preventing attribution to the latter alone. Separate loss ablations and larger evaluations would make these conclusions more convincing.

  4. The transfer results should be viewed as preliminary. Segmentation lacks a quantitative backbone-matched DDS comparison, and the video improvement at radius 0.5 corresponds to two additional correctly certified videos out of 101. These experiments establish feasibility, but larger, matched evaluations would better quantify the gains.

Questions for authors
  1. Can the authors compare against conventional noisy fine-tuning or the closest representation-based alternatives under matched backbones and adaptation budgets?
  2. Can Table 3 include a matched-backbone DDS baseline and direct SigLIP2 smoothing to clarify the source of the zero-shot improvements?
  3. Can the authors separate cross-entropy and symmetric KL in the ablations and report variability on larger ablation and transfer evaluation sets?
2 comments
AuthorsOpenReview ↗

Response to Reviewer kCsU

We sincerely appreciate your constructive feedback and the time invested in reviewing our work. We address your concerns one-by-one below and are happy to provide further clarification if needed.

W1/Q1. Comparison with direct noisy fine-tuning

RepDS achieves higher ACR than direct noisy fine-tuning of the same DINOv2-S backbone on CIFAR-10 at both noise levels. The baseline jointly trains the encoder and a classifier with Gaussian augmentation, cross-entropy, and symmetric-KL consistency, without teacher-feature recovery or fixed probes. We report both 20k- and 200k-update runs; the latter matches the number of noisy views used to train RepDS.

MethodACR (σ=0.25)[email protected] (σ=0.25)ACR (σ=0.5)[email protected] (σ=0.5)[email protected] (σ=0.5)
Pretrained DINOv2-S (no adaptation)0.15912.40.1169.04.6
Direct fine-tuning (20k updates)0.52160.20.67651.436.8
Direct fine-tuning (200k updates)0.39247.40.53241.027.4
RepDS-S0.56968.00.73958.237.0

CIFAR-10, DINOv2-S; all rows use 500 test images, N = 10,000, and identical noise draws.

Compared with the stronger direct baseline (20k updates), RepDS improves ACR by 0.048 at σ = 0.25 and 0.063 at σ = 0.5. The 200k-update run is weaker than the 20k-update run: its accuracy drops even at radius zero (83.2% → 64.2% at σ = 0.25), which suggests that prolonged direct noisy fine-tuning degrades the pretrained representation. RepDS, trained on the same number of noisy views, retains 90.8%. We have added this comparison and the training setup of the baseline to the revision (Appendix C).

For the closest representation-based alternative, rRCM, we have added rRCM-B to Table 1: at radius 1.5 on ImageNet, it reaches 33.8%, compared with 64.2% for RepDS-H+, with N = 10,000 for both (see also our response to Reviewer Pxqc, W1-1/Q1-1).

W2/W4/Q2. Backbone-matched zero-shot and transfer evaluation

Table 3 compares complete pipelines under inference cost: removing the diffusion denoiser lets RepDS use a larger encoder while still costing about one tenth of DDS per draw (3.7 vs. 36.2 seconds per 1,000 draws). Following your suggestion, we additionally evaluated DDS with the same SigLIP2-SO400M encoder and direct smoothing of the pretrained SO400M encoder. RepDS remains competitive with the backbone-matched DDS baseline (48.9% vs. 48.0% on average), while direct smoothing largely fails under noise (4.7% on average).

Certified accuracy at radius 0.5 (%)Direct SO400MDDS + SO400MRepDS
Flowers-1021.269.872.6
DTD2.450.247.2
GTSRB2.437.436.4
EuroSAT10.013.414.2
VOC07-C3.477.075.4
SVHN8.640.447.4
Mean4.748.048.9

For segmentation, a backbone-matched comparison on Cityscapes with DINOv3-H+ (256×512, N = 1,000; a separate protocol from Table 4) shows that RepDS improves certified pixel accuracy from 76.2% to 77.8% while being 4.3× faster per draw; the mIoU of the certified prediction map is comparable (35.2% for DDS vs. 36.2% for RepDS). We agree that the UCF101 gain corresponds to a small number of videos; we present it as initial evidence of transfer and leave larger-scale evaluation to future work. We have added the matched results to the revision (Appendix D).

W3/Q3. Expanded ablations and separate CE/KL analysis

The full configuration achieves the highest ACR among the training-component variants, and symmetric KL contributes beyond cross-entropy. To obtain these results, we expanded the main ablations (Table 5, σ = 0.25) from 100 to 500 test images, added a variant that removes only the symmetric KL term while keeping cross-entropy, and repeated Stage 2 with a second seed to measure training-run variability.

Variant (σ = 0.25, 500 test images)CA@0ACR
RepDS-S (full)90.80.569
RepDS-S (full), second seed91.20.569
w/o Stage 189.60.550
w/o Stage 2 recovery88.20.564
w/o Stage 2 consistency (CE and KL)91.20.551
w/o symmetric KL only92.00.551
w/o symmetric KL only, second seed92.20.551
w/o head refitting87.20.553
w/o recovery in both stages85.20.534

Removing only symmetric KL reduces ACR from 0.569 to 0.551, even though radius-zero accuracy increases. This is comparable to removing both cross-entropy and KL (ACR 0.551), suggesting that KL provides most of the consistency benefit in this setting. Across the two seeds, ACR changes by less than 0.001. The revision reports these results in Table 5 and Section 4.3.

Thank you for the detailed responses and additional experiments. Most of my concerns have been addressed, particularly through the direct noisy fine-tuning baseline, backbone-matched evaluations, and expanded ablations. The accuracy gains in some backbone-matched comparisons and the contributions of certain components remain modest. Nevertheless, the substantial computational savings while maintaining competitive certified accuracy make the approach practically useful. Overall, I maintain my recommendation of weak acceptance.

Reviewer_PxqcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RepDS recovers features of Gaussian-corrupted images in the representation space of a pretrained vision encoder for randomized smoothing. The frozen teacher supervises patch and global feature recovery, followed by classification and cross-noise prediction-consistency training. And the deployment classifier fitted separately. Each smoothing draw predicts directly from the recovered features, avoiding diffusion-based image reconstruction and re-encoding. The paper evaluates certified accuracy and inference cost on public datasets, and explores zero-shot classification, semantic segmentation, and video classification transfer.

Critical strengths

In Table 1, RepDS-H+ achieves 64.2% certified accuracy at radius 1.5, versus 54.6% for the DDS rerun using the same DINOv3-H+/16 backbone. Under the common timing protocol, inference time falls from 89.0 to 22.4 seconds per 1,000 draws.

Results obtained by freezing the ImageNet-trained encoders and fitting segmentation or video heads provide initial evidence of representation transfer. UCF101 also includes a DINOv3-L baseline under the same protocol.

Critical weaknesses
  1. The closest comparisons and the positioning of the contribution remain incomplete. The issue is not simply the age of the baselines:
  • ImageNet Table 1 omits the cited rRCM, which also uses representation learning to reduce certification inference cost.
  • In CIFAR-10 Table 2, rRCM-B is faster than RepDS-S (0.529 versus 0.791 seconds per 1,000 draws), so the relevant comparison is the accuracy–cost trade-off, rather than speedup over a heavy DDS pipeline alone. Teacher supervision, conditioning, and consistency training form a coherent design here, but the specific value added by their combination needs clearer explanation. These shared components alone do not establish that the method is an arbitrary assembly of prior work.
  1. The attribution of training gains and the scope of transfer claims remain limited.
    • The “no recovery loss” variant in Table 5 retains fixed teacher-feature probes and does not replace a simple baseline that jointly fine-tunes the same backbone and a trainable classifier on noisy inputs.
    • The ablations show that recovery helps within the proposed framework, but do not isolate the benefit of the multi-stage procedure over that simpler alternative.
    • The transfer results should also be interpreted under the disclosed conditions: the SigLIP2 comparison uses differently sized encoders, and UCF101 is evaluated on only 101 videos.
  • These results provide initial evidence of transfer, but do not by themselves isolate the method's contribution or establish broad generalization.
Questions for authors

Why is rRCM omitted from ImageNet Table 1? Please include available results where comparable, or explain the protocol differences, including sample counts, backbones, and timing settings. Please also briefly distinguish RepDS from Ditch the Denoiser and Feature-Space Smoothing in supervision targets, training procedures, and what is certified. For the latter two, a methodological discussion is the main request; they need not be forced into the same evaluation protocol, and re-running every method is unnecessary.

Do you have results from directly fine-tuning the student jointly with a trainable classifier under the same backbone, noise distribution, and a comparable training budget? If not, one comparison under the existing reduced CIFAR-10 certification protocol would be useful: retain the architecture, use classification and prediction-consistency objectives, and remove teacher-feature recovery and the fixed probes. Reporting certified accuracy and ACR would suffice.

2 comments
AuthorsOpenReview ↗

Response to Reviewer Pxqc (1/2)

We sincerely appreciate your constructive feedback and the time invested in reviewing our work. We address your concerns one-by-one below and are happy to provide further clarification if needed.

W1-1/Q1-1. Comparison with rRCM

RepDS and rRCM occupy different accuracy–cost operating points. We have added rRCM-B to Table 1 and Figure 1, timed under the same protocol as the other methods.

ImageNetNTime (s / 1,000 draws)GFLOPsCert. time per image (s)[email protected][email protected][email protected][email protected]
rRCM-B10,0001.817.71862.645.233.827.0
rRCM-B-Deep100,0004.035.240064.051.240.032.6
RepDS-H+10,00022.4225.822481.073.264.248.8

N is the number of certification samples per image. Certification time per image is computed from the time per 1,000 draws and N.

rRCM-B is the smaller rRCM variant. The larger rRCM-B-Deep, which is closer to RepDS in model scale, is reported only with 100,000 certification samples per image, whereas our evaluation uses 10,000. Even under this less favorable condition, RepDS-H+ achieves higher certified accuracy at every reported radius, with a shorter certification time per image (224 vs. 400 seconds).

CIFAR-10NTime (s / 1,000 draws)GFLOPsCert. time per image (s)CA@0[email protected][email protected][email protected][email protected]
rRCM-B100,0000.5295.6952.983.673.461.448.039.2
RepDS-S100,0000.7916.2179.191.683.270.052.238.4
RepDS-S10,0000.7916.217.990.880.668.049.037.0

CA@0 is the radius-zero certified accuracy at σ = 0.25.

On CIFAR-10, RepDS-S, which uses the smallest DINOv2 model, has an inference cost comparable to rRCM-B and is substantially more accurate at radii up to 0.75 (by 8.6 percentage points at radius 0.5), while rRCM-B is 0.8 points more accurate at radius 1.0. Given this gap, the two methods are difficult to compare directly on a single accuracy–cost trade-off. As a reference point, with 10× fewer certification samples (N = 10,000), RepDS-S remains more accurate at radii up to 0.75 while taking about one seventh of the certification time.

W1-2/Q1-2. Comparison with related representation-based methods

We did not include these two methods in our evaluation because their results are not comparable under our protocol. Ditch the Denoiser [1] is not a certified method: it reports only empirical accuracy under noise. Feature-Space Smoothing [2] is a certified method, but its ImageNet results are reported on a 50-class subset with frozen vision–language encoders, so they cannot be placed in Table 1.

Methodologically, Ditch the Denoiser trains with a denoised-to-noisy curriculum and frozen-teacher supervision on self-distillation output scores, whereas RepDS recovers the clean patch and CLS representations used for prediction and certifies with standard randomized smoothing.

Feature-Space Smoothing keeps the encoder frozen and trains a pixel-space denoiser and a feature mapper around it, and it certifies predictions through a bound on feature similarity. RepDS instead fine-tunes the encoder itself to recover clean representations, predicts without a pixel-space denoiser, and certifies the predicted class directly with standard randomized smoothing. We have added Feature-Space Smoothing to the related work in Section 2.

[1] Lu et al., Ditch the Denoiser: Emergence of Noise Robustness in Self-Supervised Learning from Data Curriculum, NeurIPS 2025

[2] Xia et al., Feature-Space Smoothing: Certified Robustness of Deep Representations, arXiv 2026

AuthorsOpenReview ↗

Response to Reviewer Pxqc (2/2)

W2-1/Q2. Comparison with teacher-free noisy fine-tuning

RepDS achieves higher ACR than direct noisy fine-tuning of the same DINOv2-S backbone on CIFAR-10 at both noise levels. The baseline jointly trains the encoder and a classifier with Gaussian augmentation, cross-entropy, and symmetric-KL consistency, without teacher-feature recovery or fixed probes. We report both 20k- and 200k-update runs; the latter matches the number of noisy views used to train RepDS.

MethodACR (σ=0.25)[email protected] (σ=0.25)ACR (σ=0.5)[email protected] (σ=0.5)[email protected] (σ=0.5)
Pretrained DINOv2-S (no adaptation)0.15912.40.1169.04.6
Direct fine-tuning (20k updates)0.52160.20.67651.436.8
Direct fine-tuning (200k updates)0.39247.40.53241.027.4
RepDS-S0.56968.00.73958.237.0

CIFAR-10, DINOv2-S; all rows use 500 test images, N = 10,000, and identical noise draws.

Compared with the stronger direct baseline (20k updates), RepDS improves ACR by 0.048 at σ = 0.25 and 0.063 at σ = 0.5. The 200k-update run is weaker than the 20k-update run: its accuracy drops even at radius zero (83.2% → 64.2% at σ = 0.25), which suggests that prolonged direct noisy fine-tuning degrades the pretrained representation. RepDS, trained on the same number of noisy views, retains 90.8%. We have added this comparison and the training setup of the baseline to the revision (Appendix C).

For reference, the Table 5 variant that keeps the RepDS architecture and fixed probes but removes recovery reaches an ACR of 0.534 at σ = 0.25 on the same 500 images (0.569 for RepDS-S). Conversely, keeping recovery but removing the symmetric-KL consistency term reduces ACR to 0.551. These results support the complementary roles of the two objectives: recovery aligns noisy features with clean teacher representations, while symmetric-KL consistency encourages agreement across noisy predictions. Removing either objective reduces ACR.

W2-2. Backbone-matched transfer evaluation

Table 3 compares complete pipelines under inference cost: removing the diffusion denoiser lets RepDS use a larger encoder while still costing about one tenth of DDS per draw (3.7 vs. 36.2 seconds per 1,000 draws). Following your comment, we additionally evaluated DDS with the same SigLIP2-SO400M encoder.

Certified accuracy at radius 0.5 (%)DDS + SO400MRepDS
Flowers-10269.872.6
DTD50.247.2
GTSRB37.436.4
EuroSAT13.414.2
VOC07-C77.075.4
SVHN40.447.4
Mean48.048.9

RepDS remains competitive with this backbone-matched baseline on average.

A backbone-matched comparison on Cityscapes with DINOv3-H+ (256×512, N = 1,000; a separate protocol from Table 4) shows the same trend: RepDS improves certified pixel accuracy from 76.2% to 77.8% while being 4.3× faster per draw. We agree that the UCF101 evaluation on 101 videos is small in scale; we present it as initial evidence of transfer and leave larger-scale evaluation to future work. We have added the matched results to the revision (Appendix D).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revision

Dear Area Chair and Reviewers,

We sincerely appreciate your time and effort in reviewing our paper.

Your constructive comments have been invaluable in improving our manuscript. We have incorporated them into the revised manuscript, where the additional results and revisions are marked in blue. We summarize the main changes below.

  • Comparison with rRCM and non-diffusion methods. We added rRCM-B to Table 1 and Figure 1 (33.8% vs. 64.2% for RepDS-H+ at radius 1.5 on ImageNet), together with an accuracy–cost comparison with DMAE and rRCM on ImageNet and CIFAR-10 (Pxqc W1-1, Q1-1; kCsU W1; UTLk W2).
  • Teacher-free noisy fine-tuning. With the same backbone on CIFAR-10, RepDS-S achieves higher ACR than direct noisy fine-tuning without teacher-feature recovery at both noise levels (0.739 vs. 0.676 at σ = 0.5) (Pxqc W2-1, Q2; kCsU W1, Q1; UTLk W1-2).
  • Backbone-matched DDS baseline. We clarified that this baseline keeps the encoder frozen and trains only the classification head. Fine-tuning the encoder makes DDS competitive with RepDS-S on CIFAR-10 (ACR 0.723 vs. 0.739 at σ = 0.5), at about twice the inference cost (UTLk W1-1).
  • Backbone-matched transfer. With the same SigLIP2-SO400M encoder, RepDS remains competitive with DDS in zero-shot transfer (48.9% vs. 48.0% at radius 0.5 over six domains), while direct smoothing of the pretrained encoder largely fails (4.7%). On Cityscapes, RepDS improves certified pixel accuracy over backbone-matched DDS (77.8% vs. 76.2%) with 4.3× lower per-draw latency (Pxqc W2-2; kCsU W2, W4, Q2).
  • Ablations. We expanded the main CIFAR-10 ablations (Table 5, σ = 0.25) from 100 to 500 test images. Removing only the symmetric-KL term reduces ACR from 0.569 to 0.551, and repeating Stage 2 with a second seed for the full model and this variant changes ACR by less than 0.001 (kCsU W3, Q3).
  • Training cost. Starting from public DINOv3 weights, RepDS-H+ takes an estimated 58.6 EFLOPs of training compute on ImageNet, compared with 396.5 for DMAE-L and 1,133.6 for rRCM-B (UTLk W2).
  • Related work. We added a discussion of Feature-Space Smoothing (Pxqc W1-2, Q1-2).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.