- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Dynamic Competitor Discovery (DCD), which repeatedly re-ranks competing classes, progressively contracts the candidate set, and uses uniform mean-margin updates to select an attack target. A targeted attack then restarts from the clean input. Experiments demonstrate improvements across multiple attack families, models, and perturbation norms, as well as complementary coverage beyond AutoAttack.
Critical strengths
- Broad empirical effectiveness. DCD improves 49/50 standard L∞ attack–model comparisons under equal or lower gradient budgets, with consistent benefits under L2.
- Practical applicability. The procedure integrates with several existing attacks. Figure 3 also demonstrates strong performance with relatively few target attempts.
- Extensive experimental analysis. The paper includes mechanism ablations and sensitivity studies, and explicitly reports degradation under gradient masking.
Critical weaknesses
1. A central ablation contains an unresolved consistency issue
CIFAR-10 has nine incorrect classes. Under Eq. 91, setting K₀ = K_f = 9 yields K_t = 9 throughout discovery. According to Appendix C.8, this should be equivalent to fixed-width TKMM with K = 9. However, Table 12 reports:
| Configuration | Carm. | Rebu. |
|---|---|---|
K_f = 9 | 60.06% | 67.01% |
TKMM K = 9 | 60.67% | 67.62% |
Both differences are 0.61 percentage points. Since the stated ablation protocol changes only the named component, these discrepancies cannot be explained by the current description. Clarifying implementation differences or correcting the results is important for interpreting the contribution of support contraction.
2. The contribution of progressive contraction under uniform weighting is not fully isolated
On datasets with many classes, fixed-width controls only use K ∈ {2, 3, 5, 9}, so their degradation may partly reflect insufficient initial coverage. The FullSupportCE versus T-CE (τ = 1) comparison provides evidence for contraction under CE weighting. However, the default mean-margin formulation lacks a matched fixed-full-support control on these datasets. Consequently, attributing the principal gains to progressive contraction requires stronger evidence separating contraction from broad initial coverage.
Questions for authors
- Please explain why
K_f = 9and TKMMK = 9differ on CIFAR-10. Are there undocumented differences in budgets, updates, target selection, or initialization? - Please provide a fixed-full-support mean-margin control on datasets with many classes, keeping the 15-step discovery phase, 85-step downstream attack, and remaining settings identical.
- Optional, not central to my recommendation: Compare DCD against existing attacks given the same additional budget on AutoAttack survivors, to quantify its practical value as an additional evaluation component.
4 comments
Reply to Reviewer Vs31 Part 2
1. Isolating support contraction from initial coverage.
Both Standard DCD and the new control start with all eligible competitors and use identical mean-margin weighting, optimization updates, target selection, and gradient budgets (15 discovery steps and 85 targeted attack steps). The only difference is whether the active support progressively contracts. Standard DCD gradually reduces the support size, whereas retains all competitors throughout discovery.
2. Experimental evidence for progressive contraction. Standard DCD outperforms the fixed-full-support control on all eight models, reducing mean robust accuracy from 35.62% to 35.15%, an improvement of 0.47 percentage points. The benefit is particularly clear on large-class models. On ImageNet, the improvements are 0.94 and 0.50 percentage points, while ImageNet-12K shows improvements of 0.52 and 0.26 points.
These results directly isolate the contribution of progressive contraction under uniform mean-margin weighting, independent of initial candidate coverage. Together with the existing FullSupportCE versus T-CE () comparison, they show that progressive support contraction improves attack performance under both mean-margin and CE-style weighting. We have incorporated the new control results and clarified these findings in the revised manuscript.
Reply to Q3.
We thank the reviewer for this valuable suggestion. We have extended Table 3 to compare APGD-DCD9 with MALT, MM, and MultiTargeted on the same examples that remain robust to AutoAttack. Each attack is evaluated independently, allowing a direct comparison of the additional adversarial examples found by APGD-DCD9 and the competing attacks.
1. Experimental setup and computational budget. Each supplementary attack is evaluated independently on the same AutoAttack survivors. We follow each method's evaluation protocol and gradient budget reported in Table 2. For APGD-DCD9, the stated budget includes all gradients used for target discovery. Table 8 (reproduced below) reports end-to-end runtime, and shows that APGD-DCD9 achieves comparable average runtime to the multi-target baselines.
Average attack runtime per image (seconds, lower is better). Times include target discovery or ranking where applicable, including the additional gradient computations used by MALT for target ranking. Avg. is the arithmetic mean of the displayed runtimes across the eight models, rounded to three decimal places.
CIFAR-10 CIFAR-100 ImageNet ImageNet12K Method Carm. Rebu. Gowal Rice Engs. Salm. CXC ViTC Avg. APGD-T 0.517 2.883 3.488 0.473 1.307 1.051 3.017 2.791 1.941 MM+ 0.491 3.181 3.323 0.462 1.328 1.014 2.973 2.808 1.948 MALT 0.510 2.913 3.423 0.497 1.359 1.180 3.188 2.981 2.006 APGD-DCD9 0.495 2.904 3.398 0.466 1.297 1.121 3.096 2.786 1.945
Additional adversarial examples found after AutoAttack. Each attack independently evaluates the same examples that remain robust to AutoAttack. Higher is better. Total is the sum across ten models.
CIFAR-10 CIFAR-100 ImageNet ImageNet-12K Attack Carm. Rebu. Gowal Rice Engs. Salm. MIM. CXN CXC ViTC Total APGD-DCD9 5 5 8 3 5 6 1 9 20 26 88 MALT 0 1 8 1 4 6 1 6 14 21 62 MM+ 4 6 4 1 2 2 0 5 13 21 58 MultiTargetedPGD 0 1 1 0 0 3 0 4 17 25 51 +APGD-T-R 5 4 2 1 1 2 0 2 3 4 24
2. Additional adversarial examples beyond AutoAttack.
Table 3 (reproduced above) shows that APGD-DCD9 uncovers 88 additional adversarial examples, compared with 62 for MALT, 58 for MM, 51 for MultiTargetedPGD, and 24 for APGD-T-Restart. It achieves the highest or tied-highest success count on nine of the ten models. Notably, on ImageNet-12K CXN, CXC, and ViTC, DCD9 finds 9, 20, and 26 additional examples, respectively, exceeding all competing attacks.
Overall, APGD-DCD9 uncovers the most additional adversarial examples missed by AutoAttack, leading or tying on nine of the ten models. It outperforms MultiTargetedPGD, MM+, and MALT in total additional successes, at comparable gradient budgets per image (Table 2) and achieving comparable average runtime (Table 8). These results demonstrate the effectiveness of dynamic competitor discovery in revealing vulnerabilities missed by existing attacks and enabling more thorough robustness evaluation.
Thank you for the detailed response and the additional experiments. Most of my concerns have been addressed. After reading the other reviews, I decide to maintain my current rating
Thank you again for your careful review. Your feedback was essential to improving the quality of our work.
Reply to Reviewer Vs31 Part 1
Reply to W1 and Q1.
We thank the reviewer for identifying this inconsistency. The reviewer is correct that, on CIFAR-10, Kf=9 and TKMM K=9 should be equivalent under the stated ablation setting. The discrepancy resulted from an implementation error in the original TKMM experiments, which we have identified and corrected.
1. Cause of the inconsistency.
The inconsistency arose because the original TKMM implementation used a different update rule. Unlike DCD, which updates the current input xt at each iteration, TKMM computed each new input relative to the original input x0. As a result, the original comparison changed both the support size and the update rule, rather than isolating the effect of support size. We have corrected TKMM to use the same update rule as DCD, with only the support size fixed at Kt=K.
2. Corrected ablation results.
We reran all affected TKMM experiments and updated the corresponding results in the paper. The corrected results are reproduced below.
The corrected results fully resolve the reported equivalence issue. On CIFAR-10, TKMM K=9 and Kf=9 now produce identical robust accuracies of 60.06% on Carmon and 67.01% on Rebuffi. The Kf=∣U∣ variant also yields the same results because CIFAR-10 has exactly nine competitors.
3. Progressive support contraction remains beneficial after the correction. With the corrected implementation, DCD outperforms all fixed-K variants on every evaluated model. DCD progressively reduces the active support and achieves a mean robust accuracy of 35.15%, compared with 35.39--35.59% when the support size remains fixed and 35.62% when all competitors remain active. Although the improvements are smaller than originally reported, these results consistently demonstrate the benefit of progressive support contraction under the same update rule.
We have corrected the TKMM implementation and updated the affected ablation results and discussion. This discrepancy also prompted us to conduct a systematic review of all experiments. We verified that the reported results matched their actual implementations and corrected any inconsistencies. We have documented the implementation fix in the code and confirmed that the revised results are consistent with the experimental protocols described in the paper. We thank the reviewer for identifying this important issue.
Reply to W2 and Q2.
We agree that the fixed small-K baselines do not fully isolate the benefit of progressive support contraction. Unlike DCD, these variants consider only a few competitors from the beginning, so their weaker performance may also result from limited initial candidate coverage. To address this concern, we have added a matched fixed-full-support control (Kf=∣U∣) to Table 15, reproduced below.