- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper extends Pixel DP from CNN to Vision Transformers, and proposes several solutions to this problem.
Critical strengths
None
Critical weaknesses
- Poor writing, for example, serveral symbol was first introduced without any explaination.
- The method/algorithm is not clear, where and how do you add laplace noise?
- No baseline is reported in the experiments.
Questions for authors
- Why extend PixelDP to ViT is an important research question?
- Could you please revise the paper with a focus on readability?
2 comments
Reviewer Dtvu,
Thank you again for emphasizing the presentation and methodological-clarity issues.
We have addressed these concerns directly in the final manuscript. The revised method section now states explicitly that noise is injected immediately after the non-overlapping patch embedding and before the first self-attention block. If the patch embedding produces , the deployed randomized representation is , and the corresponding sensitivity constraint is enforced after every optimizer step.
We also revised the notation to distinguish the inherited grouped bound, the exact stride-aware operator norm, and the spectral constraint, and we clarified the roles of the construction scale, inference-time noise scale, and certification radius.
The experimental comparisons have likewise been reorganized. The paper now includes explicit grouped-, exact-, and spectral- constraint controls, Gaussian and Laplace PixelDP configurations, a matched input randomized-smoothing baseline, multi-seed validation, a larger ViT-Small control, and matched adversarial-training controls. We therefore believe the method and the relevant baselines are now stated unambiguously.
Finally, the motivation has been sharpened. The paper does not claim that ViTs lack certified defenses. It asks a more specific question: whether PixelDP's internal randomized-representation construction can be transferred to a ViT while preserving an enforceable pixel-space sensitivity bound and a valid certificate.
We appreciate that the original presentation did not make all of these distinctions sufficiently explicit. The final revision addresses the specific readability concerns while preserving the underlying technical structure of the work.
Thank you again for the feedback, which helped us make the manuscript considerably clearer and easier to reproduce.
Submission17899 Authors
Dear Reviewer Dtvu,
Thank you for pointing out the specific presentation issues. We have made targeted clarifications to address them, particularly regarding the noise-injection location, the sensitivity notation, and the organization of experimental baselines.
Our intention was to make the construction easier to reproduce and the mathematical distinctions easier to follow, while preserving the overall structure of the paper.
We now explicitly define the randomized representation immediately after the patch embedding, distinguish the inherited grouped and exact stride-aware sensitivity norms, and clarify how the Gaussian, Laplace, and input randomized-smoothing baselines relate to our construction.
1. Explicit noise-injection procedure.
We now state the mechanism directly in Section 3.1. Noise is injected immediately after the non-overlapping patch embedding and before the first self-attention block.
If the patch embedding produces g(x), the randomized representation is g(x)=g(x)+Z, where Z is sampled independently from the calibrated Laplace or Gaussian mechanism. The remaining ViT processes this randomized representation normally. During training, the selected sensitivity constraint is enforced after every optimizer step.
We also explain why this placement is used: the patch embedding is linear and admits an explicit pixel-space sensitivity bound, whereas standard dot-product self-attention does not provide the same simple global bound on an unbounded domain.
2. Clarified mathematical notation.
We revised the notation around the patch-embedding sensitivity in Appendix A.2. In particular, we explicitly distinguish the inherited grouped bound Δˉ1,1 from the exact stride-aware induced norm Δ1,1exact. The spectral sensitivity constraint is denoted separately by Δ2,2.
We also clarify the meaning of the construction scale, inference-time noise scale, and candidate certification radius to avoid conflating these quantities.
3. More explicit experimental baselines.
We have reorganized the comparisons to make the competing constructions easier to identify.
The main comparison includes Gaussian and Laplace PixelDP configurations. The constraint-only study in Table 5 now explicitly compares inherited grouped ℓ1, exact stride-aware ℓ1, and spectral ℓ2 constraints across CIFAR-10, CIFAR-100, and SVHN and four construction scales.
At L=b=0.03, the three constraints achieve 27.91%, 69.65%, and 86.59% on CIFAR-10 in the three-seed experiment. We additionally report a larger ViT-Small control and the full exact-ℓ1 certification results.
We retain the input Gaussian randomized-smoothing comparison and matched-budget adversarial-training baselines, now with clearer explanations of what each comparison measures.
4. Expanded related work and clearer motivation.
We have added discussion of SPLITZ, latent-space randomized smoothing, Smoothed ViTs, diffusion-denoised smoothing, and prior PixelDP extensions.
The revised motivation is also more precise: we do not argue that Vision Transformers lack certified defenses. Instead, we investigate whether PixelDP's intermediate randomized-representation construction can be transferred to self-attention architectures while retaining pixel-space perturbation semantics and a valid post-processing certificate.
We have revised the Introduction, Related Work, method description, and experimental discussion accordingly.
We appreciate your emphasis on readability. These changes were intended to make the noise placement, sensitivity constraints, and baseline comparisons explicit without changing the underlying experimental methodology.
We hope these targeted changes resolve the readability and methodological concerns raised in your review. We would sincerely appreciate your reconsideration of the current evaluation in light of the revised presentation and additional experimental evidence.
Thank you again for your helpful feedback.
Yours Sincerely,
Submission 17899 Team