acceptodds
Withdrawn from ICLR 2027

Towards Shared Flat Regions: Boosting Adversarial Transferability by Exponentially Tilting Curvature-Regularized Gradients

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Transfer-based adversarial attacks rely on accessible surrogate models to fool unseen targets, but often overfit surrogate-specific loss landscapes. Although flatness-aware attacks mitigate this issue, local flatness on a surrogate does not guarantee agreement with target models. We investigate neighborhood gradient stability as a surrogate-observable signal for constructing transferable attack directions, motivated by the hypothesis that effective perturbations exploit stable, high-loss regions shared across models. We propose Tilting Curvature Regularized Attack (TCRA), which combines response-guided finite-difference probing, parallel–orthogonal gradient decomposition, and exponential tilting. Local probes capture directional gradient changes, while tilting prioritizes candidate directions from high-loss neighborhoods. Our analysis bounds the deviation of the constructed direction and establishes sufficient conditions for target-loss ascent under model discrepancy, including momentum and projected sign updates. Experiments on ImageNet-compatible and CIFAR-10 benchmarks demonstrate improved transferability across standard, adversarially trained, and Transformer-based targets. TCRA also improves the evaluated gradient-based and input-transformation attacks without target queries or surrogate-parameter tuning.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Adversarial Transferability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ytFxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TCRA (Tilt Curvature Regularization Attack) to improve the transferability of confrontational examples. The motivation of this method is to assume that the transferable disturbance should be located in the flat area shared by the model, not just the specific flat area of the agent. TCRA combines finite difference gradient detection, parallel intersection gradient decomposition and exponential tilt to construct the curvature perception attack direction without querying the target model. Experiments on ImageNet compatibility and CIFAR-10 benchmarks show that the success rate of attacks based on standard, counter training and Transformer-based target models has improved.

Critical strengths
  1. The paper provides a meaningful perspective on adversarial transferability by distinguishing surrogate-specific flatness from potentially shared cross-model flat regions.
  2. TCRA combines finite-difference probing, gradient decomposition, and exponential weighting in a coherent attack framework without target-model queries.
  3. The method is evaluated across multiple surrogate/target models, adversarially trained models, Transformers, defenses, CIFAR-10, and several existing attack frameworks.
Critical weaknesses
  1. The connection between the proposed “shared flat region” motivation and the actual optimization is not fully established. The conceptual curvature-regularized objective in Eq. (3) is not directly optimized by TCRA, and the paper itself acknowledges that the proposed update is only motivated by, rather than being the gradient of, this objective. Moreover, the current visualizations mainly examine the surrogate loss landscape and do not directly demonstrate that the discovered regions are flat across unseen target models
  2. Several experimental protocol and reporting issues require clarification. The sensitivity analysis appears to select δ\delta and ζ\zeta based on performance over the same black-box target models used for evaluation, which may introduce target-side hyperparameter tuning. In addition, important implementation details such as the exponential tilting parameter cc are not clearly reported.
  3. There are inconsistencies in the reported results. For example, Table 4 reports an average ASR of 86.5% over six black-box models, which does not match the corresponding average implied by Table 1. Similarly, some “black-box average” results in Tables 2 and 3 appear to include the white-box source model, and the text incorrectly refers to RAP as the strongest baseline in one comparison where PGN performs better.
  4. The experimental comparisons could be stronger and more up to date. Although the related work discusses recent landscape-aware attacks such as CWA, AWT, LPAA, and PFA, these closely related methods are not included in the main experimental comparison.
  5. Some methodological claims are stronger than what is theoretically supported. In particular, orthogonal gradient components are described as more stable or transferable, although the theoretical analysis only establishes geometric properties and explicitly notes that orthogonality itself does not guarantee transferability. Likewise, exponential tilting favors high-loss samples but does not directly prefer low-curvature regions.
Questions for authors
  1. Were δ\delta, ζ\zeta, and other hyperparameters selected using the same black-box target models reported in the final evaluation, or were they fixed using a separate development setting?
  2. Could the authors clarify the discrepancy between the 86.5% average ASR reported in Table 4 and the corresponding results in Table 1, as well as whether the averages in Tables 2–3 include white-box source models?
  3. What values are used for the exponential tilting parameter cc and TCRA momentum coefficient μ\mu in the experiments?
  4. Can the authors provide direct evidence that TCRA discovers regions that are flat across both surrogate and target models, rather than only flatter regions on the surrogate?
Reviewer_HRD1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates untargeted transfer-based adversarial attacks that do not require queries to the target model, and proposes a method called TCRA. By combining neighborhood gradient probing, parallel and orthogonal decomposition, and loss-based exponential weighting, TCRA leverages local gradient stability on a surrogate model to identify update directions with higher transferability. The authors provide a theoretical analysis under specific conditions and validate the approach on ImageNet-compatible datasets as well as CIFAR-10, using standard models, adversarially trained models, and Transformers. The results demonstrate consistent performance improvements over baselines such as PGN.

Critical strengths
  1. The improvement over PGN is consistent. For instance, when using Inc-v3 as the surrogate model, the average attack success rate across six black-box targets increases from 72.80% to 78.18%, demonstrating the practical effectiveness of the proposed method.
  2. The experiments encompass a variety of surrogate models, adversarially trained models, and Transformer-based architectures. Furthermore, the inclusion of component ablation studies and results from repeated runs provides comprehensive evidence for the method's effectiveness across the range of models tested.
Critical weaknesses
  1. The benefits derived from the method’s design versus those resulting from additional computation have not been sufficiently disentangled. In the current setup, TCRA requires approximately 800 gradient computations per image, compared to about 400 for PGN. While the appendix reports only a modest increase in runtime, the lack of a comparison based on an equal gradient budget makes it unclear whether the observed advantages would persist if the sampling or iteration count for PGN were increased.
  2. The validation of the proposed contributions is insufficient. The paper’s unique design elements—response-guided probing, orthogonal correction, and loss weighting—are not adequately evaluated against other relevant, state-of-the-art methods (such as PFA, which is mentioned in the text). Consequently, it is difficult to determine the actual added value of these design choices compared to existing flatness-based attacks.
Questions for authors
  1. Given the same gradient computation budget—for instance, by increasing the number of samples for PGN to match the budget used for TCRA—does TCRA still maintain its advantage?
  2. What are the specific values ​​used for the exponentially weighted parameter cc, and how are they determined? Were the hyperparameters shown in Figure 6 selected using images or target models that are independent of those used for the final testing?
  3. How is the "Curvature Variation" in Table 4 calculated—specifically regarding the probe radius, the estimation method, and the normalization process? Which surrogate models and target models were used for the results in this table? If the authors provide a compelling explanation, I would be open to increasing my score.
Reviewer_kxHFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes TCRA, a query-free transfer attack whose conceptual goal is to find high-loss regions where the surrogate's input gradient varies little across a neighborhood, corresponding locally to low curvature rather than merely a small gradient norm. It hypothesizes that such regions support cross-model transfer. The attack combines finite-difference probing, parallel-orthogonal gradient decomposition, and loss-based exponential weighting. The analysis provides conditional target-loss ascent bounds, and experiments on ImageNet-compatible data and CIFAR-10 report improvements over selected baselines, including CNN-to-Transformer transfer.

Critical strengths

Strong performance against the reported baselines. TCRA consistently improves on PGN across the held-out targets in Table 1. For Inception-v3-to-Transformer transfer, mean ASR increases from 56.98% to 62.60%, with gains on all four targets (Table 8). The paper also reports improvements when combining the method with existing attacks. These results support the heuristic's practical usefulness within the evaluated settings.

Critical weaknesses
  • The advantage over prior flatness-based methods is empirical rather than conceptually established. The paper correctly observes that favorable surrogate geometry need not be shared by an unknown target, but TCRA retains this same limitation. Its surrogate-target gradient-closeness assumption does not distinguish it from prior methods: favorable cross-model agreement can also support their transfer. The issue is not whether this assumption holds in practice, but why TCRA should transfer better under it. Neither the surrogate-only construction nor the conditional bounds establish such an advantage. The reported gains therefore demonstrate empirical improvement, not a resolution of the fundamental limitation used to motivate the method.
  • Relevant prior work and comparisons are missing. The paper neither cites nor compares with CORTA [1], which explicitly models surrogate-target differences through parameter and representation perturbations. Other missing single-surrogate comparators include DHF [2], BFA [3], and ANDA [4]; AdaEA [5] is missing for the ensemble setting. Several landscape-aware methods discussed in related work are also not compared. These omissions leave competitiveness insufficiently established.
  • The reported results provide limited evidence of cross-architecture generality. CNN-to-Transformer transfer is evaluated only from Inception-v3 to four targets (Table 8), with no Transformer-to-CNN evaluation. Unlike Table 1, Table 8 also reports no run-to-run uncertainty. The results therefore do not establish whether the gains persist across different source architectures or in the reverse transfer direction.

References

[1] Shixin Li, Zewei Li, Xiaojing Ma, Xiaofan Bai, Pingyi Hu, Dongmei Zhang, and Bin Benjamin Zhu. Consensus-Robust Transfer Attacks via Parameter and Representation Perturbations. NeurIPS, 2025. (CORTA)

[2] Zhiyuan Wang, Zeliang Zhang, Siyuan Liang, and Xiaosen Wang. Diversifying the High-Level Features for Better Adversarial Transferability. BMVC, 2023. (DHF)

[3] Maoyuan Wang, Jinwei Wang, Bin Ma, and Xiangyang Luo. Improving the Transferability of Adversarial Examples through Black-Box Feature Attacks. Neurocomputing, 595:127863, 2024. (BFA)

[4] Zhengwei Fang, Rui Wang, Tao Huang, and Liping Jing. Strong Transferable Adversarial Attacks via Ensembled Asymptotically Normal Distribution Learning. CVPR, pp. 24841-24850, 2024. (ANDA)

[5] Bin Chen, Jiali Yin, Shukai Chen, Bohao Chen, and Ximeng Liu. An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability. ICCV, pp. 4489-4498, 2023. (AdaEA)

Questions for authors
  • Conceptual justification: Given that favorable surrogate geometry need not be shared by the target, why should TCRA transfer better than prior flatness-based methods under the same gradient-closeness assumption? Does the analysis distinguish their transferability, or is the comparative advantage established only empirically?
  • Missing comparisons: How does TCRA compare with CORTA and representative missing single-surrogate baselines such as DHF, BFA, and ANDA, and with AdaEA in the ensemble setting? Please use matched surrogate access, data, models, perturbation budgets, and success-rate definitions. Method-specific iteration counts are acceptable, with computational costs reported separately.
  • Architectural coverage: Do the CNN-to-Transformer gains persist with a CNN source other than Inception-v3, and do gains also hold for Transformer-to-CNN transfer? Please include run-to-run uncertainty for these results.
  • Minor presentation clarification: What value of cc was used, how was it selected, and how was the curvature-variation metric in Table 4 computed?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.