- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies quantized tensor train for adversarial robustness. It observes that (1) The energy distribution across factors in QTT differs between natural images and adversarial images. (2) Based on this it further observes that by checking the trajectory of predictions over different QTT compression levels, you have a signal that can indicate adversarial attack (e.g., a prediction changes only when very high ranks are included in the reconstruction). (3) Furthermore it brings QTT in to make a more effective randomized smoothing classifier certificate by a pipeline of Noise=>QTT Denoise=> Classifier where the QTT mitigates the negative effect of the added noise and leads to a better certificate overall.
Critical strengths
S1: The observation of energy distribution across QTT factors is interesting and intuitive (but it also mirrors the expectation from the broader literature on compression based approaches to filtering adversarial perturbations.)
S2: The rank trajectory observations are an interesting extension of the more expected energy distribution observation. (However, as an adversarial detector they were not robust to adaptive attacks).
S3: The QTT denoiser indeed improved the randomized smoothing certificate compared to basic Cohen smoothing.
Critical weaknesses
W1: The QTT energy distribution over factors is not surprising. It’s roughly the expected QTT instantiation of the general intuition in compression of adversarial examples. The rank trajectory was more novel to my knowledge, but was not robust to adversarial attacks.
W2: Paper logic & narrative. The first and second parts are essentially disconnected, despite the title and narrative claiming they are related. In the end the adversarial defense finally proposed uses QTT as a denoiser. It does not use the rank trajectory which was the novel part of the first section. So the more novel observation was not used. In the end a known tensor factorization-based compression was used with a known workflow for improving randomized smoothing classifiers by denoising.
W2: Novelty & Comparison WRT Denoising based defenses. Following above, denoising based approaches to adversarial defenses are quite well studied. However, the paper doesn’t compare to any of them. So, we don’t know if QTT is really better than other denoising based approaches? and if so why is it better? The paper only comparison Cohen 2019. It mentions Salman (2020), Carlini (2023) as following this workflow, but doesn’t compare them. Other more recent examples of the same workflow includes Li, Consistency Purification, NeurIPS 2024, etc. All of these should be directly comparable in principle, so having only a single 7 year old competitor seems not enough. Current paper also does some fine-tuning etc which those others may not do. So they should be compared in properly matched comparison. As a “non-learned” denoiser, QTT could also compare against other non-learned denoisers such as alternative tensor factorisations, wavelet, JPEG, etc.
W3: Comparisons (2). In the broader certified adversarial defense literature, there are lots of approaches – some based on smoothing and others not. But none of it is compared. From the certified robustness leaderboard (https://sokcertifiedrobustness.github.io/leaderboard/) it seems like other competitors are stronger than QTT. For example SPLITZ (https://ieeexplore.ieee.org/abstract/document/11108253) seems to be better.
W4: I didn’t understand why the paper makes a big deal out of non-expansiveness. This might be unnecessary mathiness if the non-expansiveness is not a necessary condition for obtaining the certificate.
Questions for authors
Please clarify the importance of significance non-expansiveness result?