acceptodds
Withdrawn from ICLR 2027

Beyond Ground-Truth Suppression: Dynamic Negative Target Guidance for Transferable Skeleton Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent studies have highlighted the vulnerability of skeleton-based action recognition models to adversarial attacks. However, existing untargeted transfer objectives typically drive misclassification through objectives dominated by the ground-truth supervision signal, while the responses of non-ground-truth classes are only exploited implicitly and without explicit selection. Rethinking this from the perspective of negative-class responses, we observe that high-confidence negative classes exhibit notable cross-model consistency across different skeleton recognition models. Consequently, explicitly exploiting these consistent negative responses can provide more informative guidance for cross-model transfer optimization. Motivated by this, we propose Dynamic Negative Target Guidance (DNTG), a plug-and-play method designed to boost black-box transferability. Specifically, DNTG dynamically mines a set of Top- high-response negative targets from the surrogate model. Instead of relying on rigid targets, we construct a smoothed, soft negative distribution via temperature-scaled softmax reweighting, explicitly guiding the optimization while mitigating the risk of overfitting to isolated surrogate features. Furthermore, by periodically updating this target set during attack iterations, DNTG ensures that the guidance trajectory adaptively aligns with the continuously evolving state of the adversarial example. Extensive evaluations on the NTU RGB+D 60, NTU RGB+D 120, and HDM05 datasets demonstrate that DNTG can be seamlessly integrated into diverse attack algorithms and improves average black-box transferability across multiple representative skeleton recognition models.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
adversarial attacks, adversarial transferability, skeleton-based action recognition, black-box attacks, transfer attacks, adversarial robustness, negative target guidance

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_TFJBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the transferability of untargeted adversarial attacks against skeleton-based human action recognition models. The authors argue that existing attack methods mainly focus on suppressing the ground-truth class while insufficiently exploiting information from non-ground-truth classes. Motivated by the observed cross-model consistency of high-confidence negative classes, the paper proposes Dynamic Negative Target Guidance (DNTG), a plug-and-play framework designed to improve black-box attack transferability.

The main contribution is a negative-class guidance mechanism that dynamically selects the Top-K high-confidence negative classes, assigns soft guidance weights through temperature-scaled softmax, and periodically updates the selected targets during optimization. This guidance is incorporated into existing attack objectives without requiring model retraining.

Critical strengths
  1. A simple intervention with improvements across a reasonably broad set of tests.DNTG is tested with six attack baselines, two surrogate choices, and three benchmarks. For example, with CTR-GCN as surrogate on NTU120, DIM improves from 27.41% to 35.15% mean ASR (Table 1). Other gains are smaller, but their breadth makes the method potentially useful as an addition to existing robustness evaluations. The objective-level integration is also straightforward.

  2. An empirical motivation that is connected to the design.Figure 2 reports both initial negative-rank concentration and cross-model top-10 overlap. Tables 3 and 4 further examine whether refreshing the negative set helps and whether the selected classes change over the trajectory. These analyses provide a more informative motivation than reporting transfer scores alone, even though the causal interpretation still needs stronger controls.

Critical weaknesses

<1>Although the paper reports improvements in attack transferability, it does not sufficiently establish the added value of its proposed negative-target guidance mechanism over closely related loss formulations. When K=1, the DNTG objective reduces to a selected-negative logit margin. Moreover, with detached soft weights and a fixed negative set, its local gradient matches that of a temperature-scaled Log-Sum-Exp margin. However, the paper lacks direct comparisons with these alternatives. Table 3 also does not separately evaluate confidence-based versus random negative selection or temperature-scaled versus uniform weighting. Consequently, it remains unclear whether the improvements arise from the distinctive design of DNTG or simply from introducing an additional margin-based objective. This uncertainty directly affects the assessment of the paper's main methodological contribution.

<2>The paper does not clarify whether gradients propagate through the soft target weights in Eqs. (11)–(15), which affects the actual optimization objective. Important evaluation details, including sample-selection rules, clean model accuracies, and the precise ASR definition, are also missing. Furthermore, Tables 3–4 and Figure 4 do not clearly specify their experimental settings, and no uncertainty estimates are provided for the reported gains. This is particularly concerning for improvements that are relatively small. Although these omissions do not invalidate the results, they make it difficult to assess their reproducibility, fairness, and practical significance. As a result, the current experimental evidence is not sufficiently convincing to support acceptance.

Questions for authors

<1>Regarding gradient computation and implementation, in Eqs. (11)–(15), is a stop-gradient operation applied to the soft target weight q_t before computing gradients? Please clarify the actual gradient computation, and how, for each baseline (e.g., the momentum in MI-FGSM, the input transformation in DIM, and the specific design of TASAR), the additional objective is combined with the original loss, perturbation constraints, and update rule.

<2>Regarding data splits, sample selection, and ASR computation, please specify the split protocol for each dataset, the number of evaluation samples used, the sample-selection rule, and the exact criterion for computing the attack success rate.

<3>Regarding novelty and the independent contribution of key components, could you add loss-level controlled comparisons under matched conditions against a single highest-scoring negative class (K=1), random negative classes, Top-K uniform weighting, and a plain margin objective, in order to separate the class-selection mechanism, the soft-weighting mechanism, and a generic margin loss?

<4>Regarding the ablation configuration and hyperparameters, please clarify the settings used for Table 3, Table 4, and Figure 4, including the dataset, surrogate model, attack baseline, number of evaluation samples, and other hyperparameter values.

Reviewer_MEx3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Dynamic Negative Target Guidance (DNTG) to improve the transferability of adversarial attacks on skeleton-based action recognition models. DNTG dynamically selects high-response non-ground-truth classes and constructs a temperature-weighted logit-margin objective. Experiments across three datasets and six attack baselines show improvements in average black-box attack success rates (ASR).

Critical strengths

1.Meaningful problem. The paper addresses black-box transferability in skeleton-based action recognition from the perspective of negative-class responses.

2.Simple and flexible design. DNTG can be integrated into existing iterative attacks without modifying or retraining recognition models.

3.Broad empirical evaluation. Results across three datasets, six attack baselines, and multiple victim models show consistent improvements in average ASR (Tables 1–2).

Critical weaknesses

1.Limited methodological novelty. DNTG combines Top-K negative-class selection, temperature weighting, and logit-margin optimization. Its distinction from existing Top-K and margin-based attacks remains insufficiently established.

2.Insufficient evidence for the proposed transfer mechanism. The negative-class overlap in Fig. 2(b) may reflect shared class confusion rather than transferable attack directions. No direct analysis connects this overlap to gradient alignment or transfer success.

3.Unclear source of performance gains. The proposed loss jointly increases negative-class logits and suppresses the ground-truth logit (Eqs. 12–14). Table 3 lacks controls that isolate these effects from generic margin optimization.

4.Incomplete motivation analysis. Fig. 2(a) shows that 76.9% of successful adversarial predictions fall within the initial Top-10 negative classes. However, this may simply reflect their proximity to the original decision boundary. The experimental setting is also insufficiently specified.

5.Limited justification for dynamic refresh. Table 4 demonstrates changes in negative-class sets but does not establish their relationship with improved transferability. Sensitivity to the refresh interval (R) is not reported.

6.Insufficient experimental details. The setup underlying Table 3 is unspecified. The ASR denominator, evaluation sample counts, and statistical variability are also unclear, particularly for small improvements.

7.Limited cross-architecture evaluation. Although STTFormer appears in Fig. 2(b), the main transfer evaluations (Tables 1–2) focus on GCN-based models, limiting claims about architectural generalization.

8.Non-negligible computational overhead. Table 5 reports runtime increases from 7 to 13 seconds for MI-FGSM and from 28 to 44 seconds for SMART. Whether DNTG remains advantageous under equal runtime budgets is unclear.

Questions for authors

1.Regarding W1–W3: How does DNTG compare with Top-1 margin, uniform Top-K, random negative-class guidance, and ground-truth logit suppression under identical attack budgets?

2.Regarding W2 and W4: Can the authors clarify the settings of Fig. 2 and examine whether negative-class overlap correlates with cross-model gradient alignment or transfer ASR, beyond class-confusion priors?

3.Regarding W5: How sensitive is DNTG to the refresh interval (R), and how does periodic refresh compare with updating the negative set every iteration?

4.Regarding W6–W7: What are the exact experimental settings for Table 3 and the ASR evaluation protocol? Can transfer results on Transformer-based victim models be provided?

5.Regarding W8: Does DNTG retain its advantage over the baseline attacks under equal wall-clock time budgets?

Reviewer_WWZdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the insufficient transferability of black-box transfer attacks on skeleton-based human action recognition models by proposing a Dynamic Negative Target Guidance (DNTG) method. The proposed method selects the Top-K high-confidence non-ground-truth classes from the predictions of the surrogate model, constructs a negative target distribution using temperature-scaled Softmax, and periodically updates the negative target set to provide additional optimization guidance for existing attack methods. As a plug-and-play module, DNTG can be integrated into different skeleton-based adversarial attack methods. Experiments on multiple datasets and models demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper investigates the transferability of skeleton-based attacks from the perspective of response consistency among non-ground-truth classes, which holds certain research value.
  2. DNTG requires no modification to the surrogate model architecture and can be integrated into different methods as an additional optimization objective.
Critical weaknesses
  1. The core operations of DNTG include Top-K non-ground-truth class selection, temperature-scaled weight assignment, and periodic target updating. However, these operations essentially construct a weighted logit-margin optimization objective based on non-ground-truth classes, which is strongly related to existing multi-target attacks, Top-K logit optimization, and inter-class margin-based attacks, resulting in relatively limited methodological novelty. Moreover, the paper lacks direct comparisons with similar attacks and fails to clarify the distinctions from these methods.
  2. The paper does not clarify whether gradients are propagated through the negative target weights dynamically computed in Equations (11) to (13), nor does it analyze the impact of this design on optimization stability and the transferability of adversarial examples.
  3. Figure 2 only presents the negative class rankings of successful attacks and the class overlap across different models, which is insufficient to demonstrate that these classes can provide more transferable optimization directions. It is recommended to add a comparison between successful and failed attacks as well as an analysis of cross-model gradient similarity.
  4. Although DNTG can improve certain attack performance of existing methods, it significantly increases computational overhead.
  5. Table 3 does not specify the concrete configurations of the ablation experiments, and the paper lacks quantitative analyses of skeleton length variation, motion smoothness, and temporal continuity.
Questions for authors

It is recommended that the authors provide further clarifications and additional experimental evidence to address the concerns outlined above. In particular, they should clarify the methodological novelty and necessity of the key design choices, substantiate the relationship between negative-class guidance and adversarial transferability, and strengthen the experimental validation through computationally fair comparisons, more comprehensive ablation studies, and quantitative analyses to better support the paper's main conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.