- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes randomized manifold smoothing, which constructs a Gaussian smoothing distribution from the local PCA of neighboring training examples. The method uses whitening to obtain certificates with the covariance fixed at a reference input and analyzes how local covariance estimation affects the certification geometry. Experiments cover image classification, semantic segmentation, concept-space classification, and image-text retrieval. The authors report improved preservation of predictive performance compared with isotropic smoothing at the evaluated noise settings, together with ablations on neighborhood size, retained dimension, and sampling budgets.
Critical strengths
- A simple and practically useful approach to geometry-adapted smoothing.
The paper addresses a meaningful limitation of isotropic smoothing: perturbations in high-dimensional spaces can substantially disrupt the structures that a predictor relies on. Estimating the smoothing covariance through neighborhood PCA provides a concrete, training-free way to incorporate local data variation. For a fixed reference covariance and a specified support subspace, whitening connects the construction to established randomized-smoothing certification procedures. I view this combination of simplicity, applicability to existing predictors, and geometric motivation as the main reason to support acceptance.
- A broad empirical study with informative ablations and diagnostics.
The evaluation extends beyond a single classification benchmark to segmentation, sparse concept representations, and retrieval. The appendix examines neighborhood size, retained dimension, Monte Carlo sample count, and evaluation-set size, and reports variation across five certification seeds. The concept-activation and prediction-collapse analyses provide additional insight into how the smoothing distributions affect model behavior. While the robustness comparisons require careful interpretation, the results provide useful evidence that local covariance-based perturbations can preserve predictive structure across several settings.
Critical weaknesses
- The scope of the certification guarantee needs a more consistent formulation.
My positive assessment rests on interpreting the method as fixed-reference smoothing within the retained PCA subspace. The manuscript explicitly states the fixed-covariance convention and gives a supported parameterization, but some subsequent statements extend beyond what this interpretation establishes.
In particular, Eq. (9) expresses the certified region through a quadratic inequality without explicitly requiring
For rank-deficient covariance, this admits unrestricted displacements in the orthogonal complement, where the smoothing argument provides no guarantee. The supported region should instead be expressed as
Theorem A.2 needs a corresponding revision: adding the support restriction alone does not establish its set inclusions when the empirical and ideal subspaces differ. This is a substantive issue in the set-level analysis, rather than merely a typographical omission.
The fixed-reference predictor also needs to be distinguished from the query-adaptive predictor. Certifying with covariance fixed at does not automatically certify , which recomputes the covariance at each query. The intended inference procedure and the scope of the guarantee should therefore be stated consistently.
I regard these as potentially addressable limitations of the current formulation. They do not negate the useful fixed-reference, subspace-restricted construction, but they do limit the broader formal claims.
- The experimental advantage should be interpreted with the different perturbation scales and geometries in mind.
Using the same scalar does not match perturbation magnitude between the methods:
Consequently, improved prediction preservation can reflect both covariance orientation and a smaller effective perturbation. The authors acknowledge that radii and volumes in the different geometries are not directly comparable, and Appendix D provides additional radius-threshold curves. Nevertheless, the reported comparisons do not fully isolate the contribution of local geometry or establish superiority under a shared semantic perturbation requirement.
Similarly, certification at examples from an attribute-defined distribution shift does not establish that the attribute change itself is covered by the certificate. The retrieval results also provide empirical robustness evidence rather than certificates against the caption corruptions. These limitations qualify the interpretation of the results, although the observed predictive stability under the proposed noise model remains useful.
Questions for authors
-
Please clarify the exact certified set in the rank-deficient case. Does the implementation restrict noise to the retained PCA subspace, or add positive variance in every ambient direction? How should Eq. (9) and Theorem A.2 be interpreted consistently with Eq. (10), Algorithm 1, and the eigenvalue floor in Appendix B? In particular, how are the set-level statements handled when the empirical and ideal support subspaces differ?
-
Is the intended deployed predictor the reference-conditioned predictor , or the query-adaptive predictor ? For the former, how is the reference input selected or retained during inference? For the latter, what additional argument establishes robustness when the covariance changes with the query?
-
Can the authors quantify the perturbation magnitudes used in the main comparisons, for example through covariance traces or noise-norm statistics? How much of the observed advantage remains when the isotropic baseline is compared at a matched perturbation magnitude, and what common robustness requirement is intended when interpreting the certified-accuracy differences?
-
In the attribute-defined OOD experiments, does the nearest-neighbor reference pool contain only the classifier-training subgroup with , or also examples with ? Do these experiments establish coverage of the attribute change itself, or certification of additional local perturbations around the shifted examples?
These questions concern the interpretation and validity of the submitted contribution, rather than unrelated future extensions. Clear answers would strengthen my confidence in the positive recommendation.
4 comments
Rebuttal Response to Reviewer DRUj (2/2)
3. OOD and scope of certificates (W2, Q4)
The nearest-neighbor pool contains only the classifier-training subgroup (), so the experiments are truly OOD: geometry around each shifted example is estimated from in-distribution data. We agree: they certify local perturbations around shifted examples, not the attribute change itself. They still answer a key question: do guarantees over semantic variations persist on data unseen in training? Even with in-distribution neighbors only, RMS keeps nontrivial certified accuracy across all six shifts (Fig. 3), whereas isotropic smoothing degrades. Fig. 4 shows that certified variations include attribute-like changes (mouth opening, smiling), though they remain local. We will rephrase Sec. 5.1 to state this scope precisely.
Summary. We thank the reviewer for their suggestions: their comments sharpen the formal scope of our guarantee (Eq. 9, Theorem A.2) while leaving the method and its results unchanged. We have incorporated the changes in our manuscript, addressing the certification region (Eq. 9 updated), the noise magnitude comparison in App. D.1, and clarified the OOD sampling pool in Sec. 5. RMS remains a simple, training-free way to certify existing predictors along data-supported directions, and the new matched-magnitude baselines show that our advantage holds beyond noise scale and comes from aligning the noise with local data geometry.
We hope these clarifications and additions will convince you to give us a positive recommendation.
Response to rebuttal
Many thanks to the authors for their detailed rebuttal. I am satisfied with how they addressed two of my main concerns.
- Scope of the certificate. The revised Eq. (9) explicitly restricts the certified region to the retained PCA subspace, and Eq. (10) clarifies that the covariance is held fixed at the reference input.
- Matched-noise comparison. The authors added the projected isotropic baseline I was hoping to see in Table 6. It matches both the PCA subspace and the expected squared perturbation norm, making the benefit of eigenvalue-dependent scaling at larger noise levels clearer.
I am therefore happy to raise my score from 2 to 3. Best of luck to the authors!
Response to Reviewer DRUj
We thank reviewer DRUj greatly for staying engaged and acknowledging our detailed rebuttal. We very much appreciate raising their score from 2 to 3 (weak accept).
If there are still weak points open that the reviewer might think render the current manuscript not a clear acceptance (rating of 4), please let us know. We would highly value your further constructive input to improve the manuscript. We are happy to address any remaining such points while the discussion period is open.
Rebuttal Response to Reviewer DRUj (1/2)
Dear Reviewer DRUj,
Thank you for the thorough and constructive review. We are glad you see simplicity, applicability to existing predictors, and geometric motivation as reasons to support acceptance, and that local covariance-based perturbations preserve predictive structure across settings. We address your questions below.
1. Scope of the certified region (W1, Q1)
Certified region. We agree: Eq. 9 must restrict z−x to span(Ux,m), the retained subspace where Eq. 11 and Algorithm 1 sample noise:
Ex(r)={x+Ux,mΛx,m1/2v: v∈Rm, ∥v∥2≤r}
Without it, the quadratic inequality defines an unbounded cylinder when (m<d). The guarantee covers perturbations within the retained subspace that satisfy the bound, but does not extend to discarded directions.
Theorem A.2. We agree that the support restriction alone does not establish the inclusions when the empirical and ideal subspaces differ. For Theorem A.2, the stated inclusions hold for the ambient quadratic sublevel sets, but when span(Ux,m)=span(Ux⋆), they do not imply literal inclusions between the support-restricted certified ellipsoids. These ellipsoids can instead be compared after aligning their subspaces, with geometric convergence following from covariance convergence. We will add the restriction to Eq. 9 and restate Theorem A.2 for the support-restricted sets.
Eigenvalue floor (App. B). Flooring only replaces each retained eigenvalue by max{λx,j,10−6} for numerical stability, consistently in sampling and certification. Theorem A.2 applies unchanged when the floor is inactive; otherwise it must account for the flooring error or compare against the floored ideal covariance.
Implementation. Noise is restricted to the retained PCA subspace; no variance is added in other ambient directions. Our main experiments retain m=min(k,d)=500 components from k=500 neighbors; we will state m explicitly in Sec. 4.
2. Matched perturbation magnitude (W2, Q3)
We agree that the same σ does not imply the same magnitude: E∥δISO∥22=σ2d, whereas E∥δRMS∥22=σ2Tx with Tx=tr(Λx,m). To isolate geometry from magnitude, we add two Gaussian baselines matching Randomized Manifold Smoothing’s (RMS) expected squared norm:
Matched ambient ISO (new) : Gaussian noise across all d ambient dimensions, with σI2=σ2tr(Λx,m)/d, matching RMS's expected squared perturbation norm.
Matched projected ISO (new): Gaussian noise restricted to the same retained PCA subspace as RMS, with α2=σ2tr(Λx,m)/m, matching the expected squared norm while assigning equal variance to all retained directions.
Obsereved Perturbation magnitude (E∥δ∥22):
Certified accuracy (%):
Results. Matching magnitude explains part of the gap to standard ISO, but RMS substantially outperforms matched ambient ISO at larger scales. Restricting noise to the PCA subspace helps further, and RMS matches projected ISO at small scales and outperforms it at larger ones through eigenvalue-dependent scaling. RMS's advantage thus comes not only from smaller magnitude or dimensionality reduction, but also from aligning perturbation variance with local data geometry. Complete formulations and results are added in Appendix D.1.