- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies the losing of specificity in generated perturbations of amortized attackers. The authors observe that amortized attackers tend to generate pertubations that are both diverse and have good transferability. They propose a training objective consisting of teacher guidance, attack loss, and other regularization terms, and use it to study the effect of each term on the specificity and energy distribution of perturbations geneated by the learned amortized attacker. The warmup ablation demonstrates the benefits of teacher guidance in preserving specificity, but the full training recipe can still fail to maintain specificity.
Critical strengths
The authors propose a novel training objective and use the objective to study the effect of teacher guidance, attack loss, and other regularization terms on the specificity and energy distribution of generated perturbations. The objective itself is novel and an useful diagnosing tool for studying the behaviors of amortized attackers.
Critical weaknesses
-
The motivation of this work is unclear. In the Introduction, the authors observe that perturbations generated by amortized attackers are diverse and have good transferabilty, but do not explain why such diverse but generic perturbations should be considered as a failure that needs to be addressed.
-
Organizaton is poor and text is hard to follow. The Related Work section includes a paragraph discussing experimental results which is misplaced, and discusses text-base attacks which is irrelavant. Section 6 is difficult to follow, the corresponding tables are not explicitly referenced in the text (e.g., Image specificity discussion is not linked to any experimental results). Table 1 is also never explicitly referenced or discussed in the main text. Section 6 compare the proposed amortized attacker with other attack methods, but it’s unclear how this comparsion is related to the original silent loss problem. Most of the result discussions are enumerating observations without providing a clear conclusion.
Questions for authors
None