acceptodds
Withdrawn from ICLR 2027

Damped Sparsity Control for Generator-Based Sparse Adversarial Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Generator-based sparse adversarial attacks train encoder–decoder networks to produce transferable sparse perturbations for black-box robustness evaluation. Their perturbation density, however, is set only indirectly by a fixed sparsity coefficient whose mapping to density is unknown and shifts during training, so reaching a target density requires re-tuning; this fixed pressure also yields little transferability gain once density reduction begins. We propose Density Feedback Control (DFC), which tightens or relaxes this coefficient at the end of each epoch according to the gap between the observed hard-mask density and the target, without changing the generator architecture or loss. At a 10% target density, DFC brings the mean density of three base attacks within one percentage point of the target in a single run and improves the transfer attack success rate (ASR) of two of them on CNN and Transformer targets, including under per-image budgets, while perturbing fewer pixels. Tracing GPG's training, we observe that as DFC relaxes the coefficient, the transfer ASR recovers with the density and overtakes the fixed-coefficient baseline with fewer pixels. For EGS-TSSA, which restricts perturbations to source-model feature-importance regions, DFC yields no consistent gain; we analyze how this restriction interacts with density feedback.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Sparse adversarial attacks, Adversarial transferability, Black-box, Generator-based, Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_maZHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the author studied the practical problem in sparse adversarial attacks, where the choice of sparsity coefficient controls the balance between ASR and density. Instead of predetermining the sparsity coefficient, the authors propose Density Feedback Control (DFC), which tightens or relaxes this coefficient at the end of each epoch according to the gap between the observed hard-mask density and the target, without changing the generator architecture or loss. Experiments demonstrate that DFC can be considered a plug-in component to improve the performance of existing methods.

Critical strengths
  1. This paper studied a practical problem in sparse adversarial attacks. The choice of sparsity coefficient is important in controlling the balance between ASR and density.
  2. The proposed DFC methods can be considered as a plug-in component to improve the performance of existing methods. Experiments also demonstrate its success under most of the scenarios.
Critical weaknesses
  1. Lack of technical contribution, as the proposed DFC is essentially a constrained error-feedback update with momentum.
  2. More experiments should be conducted to demonstrate the generality of the proposed method, and the ablation study lacks a control variable.
Questions for authors
  1. Applying existing concepts of regularization-based feedback control to generator-based sparse adversarial attacks could be valuable. However, for a venue like ICLR, it is essential to provide a technical contribution. As it stands, I do not see any novel control theory, stability analysis, convergence guarantees, or control mechanisms specifically designed to enhance adversarial transferability. The author should further justify the technical contribution and novelty of this paper.

  2. In Figure 4, the authors compare c=0.5c=0.5 and c=1c=1. It can be observed that the average ASR across the ten target models remains nearly identical. In other words, completely removing the momentum term results in almost no drop in ASR. This further undermines the necessity of the specific damped controller design. I will ask the authors to provide a clear answer: what, precisely, are the unique technical contributions of the DFC approach that cannot be achieved through a standard Lagrangian multiplier update?

  3. There are two additional confounding factors in the experimental section. First, the number of warm-up epochs is inconsistent. The original method employs 3 warm-up epochs, whereas DFC uses 2. Second, the initial λ\lambda value for TSAA has been modified. The original TSAA reduction/stabilization uses 1.5×10−41.5\times10^{-4}, while the initial λ0\lambda_0 in DFC is set to 5×10−55\times10^{-5}. The authors explain that this adjustment is necessary because a larger coefficient would cause the density to drop to zero following the warm-up phase. However, this also implies that DFC and the baseline differ not only in their control strategies but also in their initial regularization strength.

  4. In the experiment, the full text only tests ρ∗=10%\rho^*=10\%. It is necessary to show the performance under different ρ\rho, because the dynamic behavior at different target densities can be entirely different.

  5. Furthermore, the author claims that all three methods can achieve a mean density within ±1 percentage point of the target. However, as shown in Table 1, the original TSAA and EGS-TSSA already meet this ±1 pp criterion. Incorporating DFC into EGS-TSSA actually causes the result to deviate further from the target. Therefore, the current experimental results only support the conclusion that DFC provides better density tracking performance for this specific target.

Reviewer_yE93OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the difficulty of achieving a prescribed perturbation density while maintaining transferability in generator-based sparse adversarial attacks trained with fixed sparsity coefficients. It proposes Density Feedback Control (DFC), which updates the sparsity coefficient at the end of each epoch using the discrepancy between the observed hard-mask density and a prescribed target, with a damped momentum update. DFC preserves the architecture and loss formulation of the base attack. Experiments on ImageNet with ResNet-50 as the source model show that DFC improves the mean transfer attack success rate (ASR) of GPG and TSAA on CNN and Transformer targets, including under per-image ℓ₀ budgets. However, DFC does not consistently improve EGS-TSSA. This paper also examines support replacement during training and the interaction between density feedback and feature-importance restrictions.

Critical strengths

1.The method addresses a practically relevant problem with minimal implementation changes. Fixed sparsity coefficients provide only indirect control over the perturbation density used in evaluation, whereas feedback based on hard-mask density directly addresses this mismatch. Separating the differentiable training loss from the measurement of actual density makes the method readily applicable to existing generators. 2.The improvements for GPG and TSAA are valuable, and the evaluation goes beyond mean density. On CNN targets, the mean transfer ASR increases from 32.99% to 42.09% for GPG and from 25.23% to 40.83% for TSAA, while mean perturbation density decreases. The per-image budget evaluation and density distributions provide useful evidence about practical constraint satisfaction. The authors also report the lack of consistent improvement when combining DFC with EGS-TSSA, clarifying the scope of applicability of the method.

Critical weaknesses

1.The contribution of density feedback itself to the performance improvements, and the rationale for selecting the default controller parameters, require further clarification. Section 4.1 and Appendix E state that all generator-based methods are trained for 15 epochs, but the baselines use three warm-up epochs whereas DFC uses two. It is also unclear whether the two share the same initial generator weights or warm-up checkpoint. Differences in the warm-up schedule may affect the subsequent optimization trajectory. I suggest an additional comparison with the same warm-up duration and a shared warm-up checkpoint, while keeping the total training budget unchanged, to better assess the effect of feedback updates. In addition, the initial sparsity coefficient λ₀ is adjusted for TSAA-DFC; its selection rationale, tuning budget, and sensitivity should be explained. 2.The applicability of the proposed method at different target densities, as well as the choice of the initial sparsity coefficient and feedback gain, warrants further validation. All main experiments use a training target density of 10%. The different thresholds in Table 3 are evaluation budgets rather than different training targets. Appendix E explains that an initial sparsity coefficient of λ₀ = 10⁻⁴ drives the density of TSAA-DFC to zero, so it is reduced to 5 × 10⁻⁵. According to Equation (5), this adjustment also reduces the feedback gain κ from 10⁻⁴ to 10⁻⁵. This indicates that the choice of the initial sparsity coefficient and feedback gain merits further analysis. I suggest experiments at additional target densities while keeping the parameter-selection rule unchanged, to assess the method’s ability to adapt to changes in the target and its practical tuning requirements.

Questions for authors

1.How much of the performance improvement for GPG and TSAA remains when the warm-up duration, generator initialization, and total training budget are identical? Please clarify whether such controlled comparisons are already available. 2.How were the default damping coefficient c = 0.5 and the initial sparsity coefficient λ₀ = 5 × 10⁻⁵ for TSAA-DFC selected? Please specify the range of values tried, the selection criteria, and whether target-model attack success rates were used in the selection. 3.Can the same parameter-selection rule reliably achieve other target densities without target-specific retuning? Please distinguish cases where the target is reached directly from those requiring a different initial coefficient or longer training.

Reviewer_LiSVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Density Feedback Control (DFC) to improve density control and transferability in generative sparse adversarial attacks. Existing generator-based methods typically rely on manually tuned sparsity coefficients, which makes it difficult to reach a target perturbation density without sacrificing attack effectiveness. DFC addresses this by comparing the average hard-mask density with a predefined target and adjusting the sparsity penalty through feedback, thereby reducing reliance on manual tuning. It also allows the perturbation density and ASR to recover when the sparsity constraint becomes overly restrictive. Experiments on ImageNet use ResNet-50 as the source model and multiple CNN and Transformer models as targets. The results show that DFC improves the transfer ASR of existing generative sparse attacks.

Critical strengths
  1. Unlike traditional sparse attacks that update perturbations through repeated queries and feedback, DFC regulates the sparsity penalty using a normalized density error, which improves both training efficiency and attack strength.

  2. Through feedback, DFC can relax overly restrictive sparsity penalties, allowing the perturbation density and ASR to recover and helping maintain attack effectiveness at lower perturbation densities.

  3. The authors conduct extensive experiments on ImageNet with both CNN and Transformer models. The results show that DFC improves the transferability of GPG and TSAA across different target architectures, supporting the effectiveness of the proposed method.

Critical weaknesses
  1. The effects of the feedback control parameters kk and cc on density control, training stability, and attack performance are not sufficiently investigated.

  2. The role of vtv_t in the feedback controller is not sufficiently clarified. Although its update equation relates it to the density error ete_t, the paper offers limited intuition on how historical and current density errors jointly influence the update of the sparsity penalty coefficient λt\lambda_t.

  3. DFC uses the average hard-mask density as its feedback signal without sufficient justification. Moreover, the paper does not compare it with alternative strategies, so it remains unclear whether average-density feedback is the most effective choice.

  4. The paper lacks a systematic evaluation of the computational efficiency of DFC, including training time, computational overhead, and hyperparameter search cost. As a result, the practical efficiency benefit of introducing feedback control remains unclear.

  5. The effectiveness of DFC under more stringent target densities (e.g., 2%-3%) is not sufficiently validated. The experiments mainly adopt a target density of 10%10\%, and the reported per-image budgets of 10%-12% do not show whether DFC remains effective when trained with lower target densities.

  6. The limited diversity of source models restricts the generalizability of the findings. The transfer experiments mainly use ResNet-50 as the source model, so the effectiveness of DFC across different source architectures is not sufficiently validated.

Questions for authors
  1. Could the authors compare average-density feedback with alternative strategies under matched sparsity budgets to clarify the advantages of the proposed design?

  2. Could the authors analyze, from a methodological perspective, whether the structured sparsity optimization in EGS-TSSA interacts unfavorably with the density feedback mechanism?

  3. Could the authors report results with additional surrogate models (e.g., Inception-V3) and different sparsity budgets to validate the generalizability of DFC across different source architectures?

  4. Could the authors clarify the role of vtv_t in the feedback controller, particularly how it incorporates historical density errors and influences the update of the sparsity penalty coefficient λt\lambda_t?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.