Fast Randomized Smoothing via Representation Space Denoising

Diffusion Denoised Smoothing (DDS) is currently a prominent approach for obtaining certified robustness, but its prohibitively high inference cost remains a major barrier to practical deployment. Specifically, each Monte Carlo draw in the smoothing process requires a large diffusion denoiser to reconstruct an image in pixel space, and a separate pretrained model to classify the denoised image. In this paper, we propose Representation-space Denoised Smoothing (**RepDS**), a framework that replaces pixel reconstruction with denoising in the feature space of a pretrained vision encoder. Given a Gaussian-corrupted image, RepDS adapts the encoder to recover representations aligned with those of the corresponding clean image, directly supervising the features used for prediction instead of relying on pixel-space fidelity as an indirect proxy. At inference, RepDS predicts from the recovered features in a single non-generative forward pass per smoothing draw, eliminating the need for both pixel-space reconstruction and an additional encoder pass over the reconstructed image. Extensive experiments show that RepDS achieves state-of-the-art certified accuracy across all reported ImageNet radii while substantially reducing inference cost. At radius 1.5, RepDS-H+ reaches 64.2% certified accuracy compared with 54.6% for our backbone-matched DDS rerun, while running approximately $4\times$ faster under the same timing protocol. Compared with DensePure, RepDS-H+ reduces per-draw inference time by over three orders of magnitude.