- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the novel problem of ordered top-k universal adversarial perturbations (UAPs). The authors show that the ASR of ordered top-k UAPs can be decomposed into membership ASR and ordering ASR, and hypothesizes that these two components are controlled by different factors. The authors use this decomposition to evaluate and analyze different attack losses and propose a novel attack method StO by combining existing ones. StO outperfoms existing attacks in certain models under per-model ordered ASR.
Critical strengths
The paper proposes a novel decomposition of ordered-set attacks and performs extensive evaluation and analysis of different attack losses on the ordered top-k UAPs task. Based on this analysis, the authors propose a novel attack method, StO, that outperforms baselines in certain models under per-model ordered ASR.
Critical weaknesses
- The motivation for studying ordered top-K UAPs is not sufficiently clarified. Providing a concrete scenario where an attacker would need to enforce the same exact top-K ranking across different inputs and multiple models would be helpful.
- The proposed joint ASR metric is central to the analysis, but is not used in evaluating the proposed StO attack.
- The claim that “the two factors are controlled by different things”/“membership and ordering are bought separately” is not sufficiently supported by the experimental results. The results have shown that membership and ordering respond differently under different losses, but this does not establish that the two components are independently controllable or governed by different factors.
Questions for authors
None