acceptodds
Under review as a conference paper at ICLR 2027

Rank Trajectories for Adversarial Robustness via Quantized Tensor Train

Abstract

Many defenses against adversarial examples rely on an explicit or implicit model of the natural-image distribution and are evaluated primarily by reconstruction quality. We pursue a different question: whether the multiscale tensor structure of an image provides an interpretable and computable basis for analyzing adversarial robustness. We study this question through quantized tensor-train (QTT) representations, whose modes encode spatial scales and whose bond ranks measure their interactions. Tensors satisfying nontrivial QTT-rank constraints form lower-dimensional algebraic sets, and natural images empirically concentrate near this low-rank structure. An adversarial perturbation, in contrast, comes from a search constrained only by a norm budget and a misclassification objective, and neither constraint confines it to these sets. To connect this geometry to model behavior, we introduce rank trajectories, which track predictions along progressively higher-rank reconstructions from a single decomposition sweep. Clean examples typically stabilize at moderate ranks, whereas adversarial examples often emerge later. The same geometry also has a constructive use inside randomized smoothing. A training-free filter suppresses the weak singular directions that carry most of the Gaussian smoothing noise. Experiments show that it consistently improves certified accuracy over plain smoothing in the high-noise regime. Together, these results establish QTT rank as a multiscale coordinate for analyzing adversarial robustness, and improves certified robustness under strong smoothing noise.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Robustness, Tensor Network

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZFmZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies quantized tensor train for adversarial robustness. It observes that (1) The energy distribution across factors in QTT differs between natural images and adversarial images. (2) Based on this it further observes that by checking the trajectory of predictions over different QTT compression levels, you have a signal that can indicate adversarial attack (e.g., a prediction changes only when very high ranks are included in the reconstruction). (3) Furthermore it brings QTT in to make a more effective randomized smoothing classifier certificate by a pipeline of Noise=>QTT Denoise=> Classifier where the QTT mitigates the negative effect of the added noise and leads to a better certificate overall.

Critical strengths

S1: The observation of energy distribution across QTT factors is interesting and intuitive (but it also mirrors the expectation from the broader literature on compression based approaches to filtering adversarial perturbations.)

S2: The rank trajectory observations are an interesting extension of the more expected energy distribution observation. (However, as an adversarial detector they were not robust to adaptive attacks).

S3: The QTT denoiser indeed improved the randomized smoothing certificate compared to basic Cohen smoothing.

Critical weaknesses

W1: The QTT energy distribution over factors is not surprising. It’s roughly the expected QTT instantiation of the general intuition in compression of adversarial examples. The rank trajectory was more novel to my knowledge, but was not robust to adversarial attacks.

W2: Paper logic & narrative. The first and second parts are essentially disconnected, despite the title and narrative claiming they are related. In the end the adversarial defense finally proposed uses QTT as a denoiser. It does not use the rank trajectory which was the novel part of the first section. So the more novel observation was not used. In the end a known tensor factorization-based compression was used with a known workflow for improving randomized smoothing classifiers by denoising.

W2: Novelty & Comparison WRT Denoising based defenses. Following above, denoising based approaches to adversarial defenses are quite well studied. However, the paper doesn’t compare to any of them. So, we don’t know if QTT is really better than other denoising based approaches? and if so why is it better? The paper only comparison Cohen 2019. It mentions Salman (2020), Carlini (2023) as following this workflow, but doesn’t compare them. Other more recent examples of the same workflow includes Li, Consistency Purification, NeurIPS 2024, etc. All of these should be directly comparable in principle, so having only a single 7 year old competitor seems not enough. Current paper also does some fine-tuning etc which those others may not do. So they should be compared in properly matched comparison. As a “non-learned” denoiser, QTT could also compare against other non-learned denoisers such as alternative tensor factorisations, wavelet, JPEG, etc.

W3: Comparisons (2). In the broader certified adversarial defense literature, there are lots of approaches – some based on smoothing and others not. But none of it is compared. From the certified robustness leaderboard (https://sokcertifiedrobustness.github.io/leaderboard/) it seems like other competitors are stronger than QTT. For example SPLITZ (https://ieeexplore.ieee.org/abstract/document/11108253) seems to be better.

W4: I didn’t understand why the paper makes a big deal out of non-expansiveness. This might be unnecessary mathiness if the non-expansiveness is not a necessary condition for obtaining the certificate.

Questions for authors

Please clarify the importance of significance non-expansiveness result?

Reviewer_RwKrOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a defence approach against adversarial attacks on image datasets. The method relies on a novel integration of Quantized Tensor Train (QTT) representations, motivated by the intuition that QTT organises images into a hierarchy of spatial scales. The authors explicitly motivate this representation from a robustness perspective, arguing that such a multiscale decomposition may provide useful structure for analysing adversarial perturbations. To connect this representation to model behaviour, the authors introduce the notion of rank trajectories, which track model predictions across reconstructions obtained at different QTT ranks. The experimental section is detailed and reveals several interesting observations.

Critical strengths

Investigation of QTT representation for Adversarial defence is novel and worth investigating.

Critical weaknesses

Overall, the paper is well written, and I enjoyed reading it. However, I found it somewhat difficult to identify the main takeaways. The authors could more clearly articulate the central contributions, hypotheses or research questions, and then explicitly demonstrate how the experimental results support them. In its current form, the paper reads more like an exploratory study or empirical investigation than a work leading to a small number of concrete conclusions. A stronger synthesis of the findings would substantially improve the paper.

The Preliminaries section introduces notation such as DD, CC, NN, and KK rather abruptly, which makes the section difficult to follow. It would also be helpful to provide more intuition for the Tensor Train representation and QTT tensorisation, potentially with a simple illustrative example in the main paper or appendix.

The QTT tensorisation choices for the two datasets also appear somewhat arbitrary. In particular, it is unclear why D=5D=5 and D=8D=8 are selected for the respective datasets. The paper should explain these choices and ideally discuss their sensitivity. In addition, the tables should use upward/downward arrows to indicate whether each reported metric is intended to be maximised or minimised.

I also found it difficult to draw a clear conclusion from Table 2. While the Clean-Percentile combination performs well, methods such as Perturbation Forgery and LID also achieve competitive results. This raises questions about the practical benefit of introducing a QTT-based defence. The paper would benefit from a more detailed discussion highlighting where the proposed method provides a meaningful improvement over existing state-of-the-art approaches, and under which settings the QTT representation is particularly advantageous.

Table 4 is interesting, particularly the comparison involving smoothing-based robustness. However, it is unclear why Cohen et al.'s randomized smoothing is used as the primary benchmark and why other smoothing approaches are not considered. A broader comparison would strengthen the evaluation. It would also be useful to report the computational efficiency of QTT-SVT, including its runtime and additional inference cost. Finally, the practical significance of the reported improvement in ACR should be discussed more clearly: for example, what does the observed ACR gain imply in terms of certified robustness, and is the improvement large enough to justify the additional complexity introduced by QTT-SVT?

Questions for authors

See my concerns in the weakness section. It would be good if authors can address my following question which are quite fundamental:

a) What is the central hypothesis of the paper, and which experimental results provide the strongest evidence for it? and

b) What specific advantage does the proposed QTT/rank-trajectory approach provide over existing adversarial detection or defence methods?

Reviewer_rLUEOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper observes that natural images often concentrate at moderate rank, which captures complex interactions across spatial scales. Crucially, adversarial examples do not preserve this property. The paper constructs a trajectory that records the evolution of classifier predictions and confidence from coarse to fine-detailed. Adversarial effects emerge only at higher-rank reconstructions. The paper correctly notes that this property is empirical, so adaptive attacks can bypass it, limiting the effectiveness of truncation methods. The paper proposes replacing the diffusion model with a QTT-based filter in the randomized smoothing pipeline of (Carlini et al. 2023). It proves that this parameter-free filter is continuous and cannot increase the input norm.

Critical strengths

The paper avoids the trap of ignoring adaptive attacks and treating the rank property of adversarial examples as sufficient for detecting or eliminating them.

Critical weaknesses

The advantage of this filtering method is not supported by experimental evidence. The paper compares only against the original randomized smoothing method (Cohen et al., 2019) and does not consider other filtering/denoising-based methods. Comparing against (Carlini et al. 2023), whose pipeline the paper adopts, is the bare minimum, comparison against a successor such as [A] is recommended.

[A] Consistency Purification: Effective and Efficient Diffusion Purification towards Certified Robustness, NeurIPS 2024.

Questions for authors
  • Can the authors provide experimental evidence demonstrating the advantage of the proposed QTT-based filter over the original randomized smoothing pipeline?
  • Can the authors compare against at least one other filtering/denoising-based certified robustness method, such as [A]?

I would be more lenient toward acceptance if this critical weakness is addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.