- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates how the composition order of common corruptions and adversarial attacks affects model robustness. It compares two settings, degrade-then-attack and attack-then-degrade, and reports substantial accuracy differences across multiple models and corruption types on CIFAR-10 and ImageNet. The authors further decompose the order effect into recovery and exposure terms, analyze its relationship with decision boundary geometry, and examine the influence of attacker knowledge and attack timing. The findings highlight the importance of explicitly specifying composition order when evaluating robustness under combined threats.
Critical strengths
The paper addresses an important issue in robustness evaluation by showing that composition order can substantially affect adversarial accuracy and even change model rankings. This finding has practical implications for specifying threat models and comparing robust models.
Critical weaknesses
The importance of composition order has been explored in prior studies, particularly Towards Compositional Adversarial Robustness (CVPR 2023), which demonstrated that perturbation order can significantly affect attack effectiveness. Although this paper investigates a different threat model and systematically quantifies the order effect, its novelty remains somewhat limited. Moreover, it is unclear how much of the observed accuracy gap represents a non-trivial interaction beyond the expected effects of corruption and adversarial vulnerability.
The exposure term includes errors caused by corruption itself, making it difficult to isolate the additional vulnerability introduced by adversarial attacks. Moreover, the decision boundary analysis relies mainly on correlations and uses an l2 distance estimate for l_\infty attacks, limiting the strength of the mechanistic conclusions.
The Introduction lacks a clear logical progression from the problem background and limitations of existing studies to the research motivation. It introduces too many technical details before adequately establishing the core problem, reducing readability. Moreover, the Introduction does not sufficiently acknowledge that prior works, such as Towards Compositional Adversarial Robustness and Composite Adversarial Attacks, have already investigated the effects of composition order. The distinctions from these studies and the paper's additional contributions should be clarified to better justify its novelty.
Questions for authors
Given prior work on compositional adversarial robustness and OODRobustBench, what is the main independent scientific contribution beyond systematically quantifying the effects of composition order?