acceptodds
Withdrawn from ICLR 2027

Can Active Rival Interactions Expose Adversarial Vulnerabilities in Deep Networks?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Adversarial attacks exploit the decision geometry of deep neural networks to expose vulnerabilities under small input perturbations. However, these attacks primarily optimize top-1 misclassification losses or manipulate ranked outputs, leaving dynamic pairwise interactions among non-source rival classes closest to overtaking the source class underexplored, which may, in turn, limit attack effectiveness. In particular, probing a direction associated with one rival class can reveal changes in another rival class’s normalized margin gradient. This reveals a potential source of attack potency: pairwise margin-field responses provide geometric information beyond individual rival gradients at the current iterate and can expose adversarial directions missed by independent classwise optimization or single-margin ascent. We therefore introduce Rival Interaction Optimization for Holistic Attack (RoHoAttack), a geometry-aware white-box attack that exploits active top-K non-source rival-class selection, reciprocal probing, residual refinement, and projected lookahead, thereby retaining the highest-margin evaluated ℓp-feasible candidate while refreshing the active rival classes after each update. Across natural-vision and medical-imaging benchmarks, RoHoAttack achieves higher attack success rates and reduces robust accuracy by up to ≈ 12.5 points over leading baselines, thereby demonstrating its attack effectiveness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Attacks, Deep Learning, Medical Image Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8waJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RoHoAttack, a white-box adversarial attack based on interactions between rival-class gradient vector fields. It constructs additional candidate search directions by taking pairwise finite-difference responses between active rival gradients, and then selects promising projected updates through lookahead evaluation. This expands the search space beyond conventional single-gradient attacks and can improve attack success. However, generating and evaluating many pairwise directions also introduces substantially higher computational complexity and runtime.

Critical strengths
  1. The reciprocal finite-difference construction is a genuine idea beyond ordinary single-gradient ascent. It captures local changes between rival margin-gradient fields.

  2. The method shows consistent gains across multiple architectures, datasets, defenses, and ordered top-K settings. The ablations also suggest that its main components contribute incrementally, although the lack of compute-matched baselines limits how strongly these gains can be interpreted.

Critical weaknesses
  1. The paper positions RoHoAttack as a “high-potency adversarial evaluator,” yet the robust-defense evaluation is conducted against PGD-20, APGD-20, and AA-20 rather than a clearly specified full-strength APGD/RobustBench-style protocol. Moreover, the main RoHoAttack iteration/restart budget T,RT,R is not explicitly reported, while its runtime in Table 1 is about 10–20× PGD and APGD, making it difficult to determine whether the reported gains come from stronger interaction-aware search or simply a substantially larger evaluation budget.

  2. The vector-field/Lie-bracket motivation appears only partially reflected in the actual algorithm. Proposition 2 gives the reciprocal cross-response residual a local Jacobian/Lie-bracket interpretation under a special case, but the implemented bounded and projected probes, residual refinement, fallback rule, and projected lookahead-margin selection are not derived from this theory. As a result, the method can largely be viewed as active rival-pair direction generation plus projected candidate search, and the paper does not establish that this allocation of additional forward/backward evaluations is more efficient than alternative strong attack optimization under matched compute.

Questions for authors
  1. How should the active-rival size KK scale with the number of classes? The paper uses K=5K=5 by default, but the sensitivity study is only conducted on ImageNet-1K. Why should the same KK be appropriate for datasets with very different numbers of rivals?

  2. Why is interaction modeling restricted to pairwise reciprocal responses? Is there any theoretical or empirical reason to expect pairwise interactions to be sufficient compared with higher-order rival interactions?

  3. Since AK(xi⋆;y)\mathcal{A}_K(x_i^\star; y) is selected according to the current logits, can RoHoAttack miss a low-logit class whose decision boundary is actually closer or easier to cross? Does dynamic rival refresh reliably recover such cases, or have alternative rival-selection criteria been considered?

  4. Residual refinement explicitly removes components aligned with the original rival-gradient subspace. Why should interaction information in this subspace be considered redundant, and could the refinement discard useful adversarial directions?

  5. How sensitive are the results to η\eta, α\alpha, ρ\rho, and ϵnum\epsilon_{\mathrm{num}}?

Reviewer_d6F3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RoHoAttack, a white-box adversarial attack that leverages gradient interactions among active competing classes. The method generates candidate directions through bidirectional gradient probing and residual processing, then selects the update that yields the best attack performance via multi-scale projection-based lookahead. The authors evaluated the approach on natural images, medical images, and various defense models; they also provide an ordered Top-K attack variant and conduct component ablation studies, reporting higher attack success rates and lower robust accuracy.

Critical strengths
  1. The approach to constructing directions for gradient interaction across categories is somewhat novel, offering a clear methodological design for exploring attack directions that might be overlooked by conventional gradient update processes.
  2. The experiments encompass a diverse range of datasets, model architectures, and defense mechanisms; furthermore, the inclusion of component ablation studies provides preliminary evidence regarding the method’s scope of applicability and the specific roles of its individual components.
Critical weaknesses
  1. The experimental setup for comparison is insufficient. The defense evaluation employs AA-20, whereas RoHoAttack incurs a significantly higher computational overhead. The absence of a comparison against the full-standard AutoAttack—with a matched computational budget—makes it difficult to discern what portion of the performance gain stems from the design of the interaction strategy versus the additional search cost.
  2. There are internal inconsistencies in the reported results. In Table 1, the L1 and L2 values ​​violate the standard relationship between these norms, suggesting that the column labels might be swapped. Furthermore, in Table 12, for the variant that retains the current point and selects the best candidate based on post-projection loss, the monotonicity rate is only 95.8%. These issues require clarification; otherwise, they undermine the credibility of the experimental conclusions.
Questions for authors
  1. When using the full, standard AutoAttack—while accounting for runtime or the budget for gradient computations—do the primary advantages still hold?
  2. Are the L1 and L2 columns in Table 1 swapped? Furthermore, why does the variant in Table 12—which retains the current point and performs selection after projection—fail to achieve a 100% monotonicity rate?
  3. What specific ordered objective function does RoHoAttack-O employ? How does the prescribed ordering of target classes influence the generation of attack directions and the selection of candidates?
  4. In the context of randomized defenses, do the various gradient probes share the same random state? Is the final attack efficacy evaluated through repeated trials using independent random samples?
Reviewer_4JhXOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes the RoHoAttacK that dynamically selects the top-K rival classes, constructs gradient cross-response residuals via reciprocal probing, and suppresses the components aligned with the original-class gradient to obtain interaction-aware attack directions. The authors further introduce a projected lookahead selection mechanism, which evaluates feasible candidate adversarial examples across different directions and step sizes and retains the update that yields the largest attack margin. Moreover, the method is also extended to the ordered top-K attack setting. Experiments on multiple datasets show that it achieves higher attack success rates than existing baselines against a range of classifiers and robust defenses.

Critical strengths
  1. RoHoAttack dynamically selects the active rival classes, probes the reciprocal gradient responses between them, and refines the resulting cross-response residuals. This design captures local variations in the loss landscape that individual class gradients fail to reflect, thereby providing additional directions for adversarial search.

  2. The proposed projected lookahead selection mechanism generates candidates along different directions and with different step sizes, and then selects the update according to the attack loss achieved after projection. This aligns the direction generation with the final attack objective, ensuring that the optimization remains consistent throughout.

  3. The experiments cover multiple datasets, and medical-imaging datasets, multiple CNN,VIT models, and various robust defenses, with comparisons against attacks such as PGD, APGD, and AutoAttack. The results show that the proposed method achieves stronger attack performance in the evaluated settings.

Critical weaknesses
  1. The paper does not sufficiently explain the role of the successive gradient normalization operations in constructing the probing direction, and the individual contribution of each normalization step to the overall performance remains unclear.

  2. The description of the ordered top-K attack variant (RoHoAttack-O) is insufficiently detailed. In particular, the paper does not specify how the ordering constraints on the pre-specified target classes are incorporated into the optimization framework.

  3. The connection between the Lie-bracket interpretation and the practical construction of the interaction residuals is not sufficiently established. Moreover, the paper does not discuss how the operations used in this construction affect the interpretability of the resulting residuals and their associated approximation error.

  4. Eq. 21 adopts a nonlinear cosine decay schedule to adjust the search step size, but the paper does not sufficiently justify this choice.

  5. Although RoHoAttack achieves stronger attack performance, it does so at the cost of low efficiency.

  6. Although the residual refinement mechanism is designed to remove components aligned with the gradients of the original competing classes, its additional contribution to attack performance is relatively limited. Furthermore, the paper does not clearly describe how candidates are selected in the corresponding ablation variants, which makes it difficult to assess the practical effectiveness of this mechanism.

  7. As K increases from 5 to 10, the attack has to consider more rival classes, which should in principle increase the computational overhead. However, Table 2 shows that the top-10 attack on ResNet runs faster than the top-5 attack. The paper does not explain this counterintuitive result.

Questions for authors
  1. Eq. 13 applies RMS-based gradient normalization, and Eq. 14 further applies RMSNorm to the probing direction. What role does each normalization step play, and how does each affect attack performance? Would other normalization methods perform comparably or better?

  2. The ordered top-K attack requires the target classes to follow a predefined ranking. Could the authors describe the ranking-based loss used in RoHoAttack-O, and explain how the ranking constraints are incorporated into both direction generation and candidate scoring? In particular, how are Eqs. 4–5 and Eq. 23 modified for the ordered variant?

  3. Eq. 17 derives the Lie-bracket relation under the assumption that the probing direction equals the normalized gradient, whereas the practical method uses a bounded mapping based on RMSNorm and tanh. How does this discrepancy affect the geometric interpretation of the interaction residuals and their approximation error? To what extent does the Lie-bracket interpretation still hold for the probing mechanism actually used?

  4. Could the authors explain why Eq. 21 adopts a nonlinear cosine decay schedule for the step size rather than simpler alternatives such as a fixed step size or linear decay, and how this choice affects attack performance?

  5. Table 2 shows that the per-sample runtime of RoHoAttack-O is generally longer than that of existing methods. Have the authors explored strategies to reduce this overhead, and if so, how do these strategies trade off runtime against attack performance?

  6. In Table 10, residual refinement improves the average ASR over four models only marginally (from 98.67% to 98.88%), and per-model results are not reported. Is this gain consistent across architectures? In addition, without projected lookahead selection, how does the “Refined residual w/o PLM” variant select candidates?

  7. Since increasing K from 5 to 10 requires considering more rival classes, why does the top-10 attack on ResNet run faster than the top-5 attack in Table 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.