Beyond Ground-Truth Suppression: Dynamic Negative Target Guidance for Transferable Skeleton Attacks

Recent studies have highlighted the vulnerability of skeleton-based action recognition models to adversarial attacks. However, existing untargeted transfer objectives typically drive misclassification through objectives dominated by the ground-truth supervision signal, while the responses of non-ground-truth classes are only exploited implicitly and without explicit selection. Rethinking this from the perspective of negative-class responses, we observe that high-confidence negative classes exhibit notable cross-model consistency across different skeleton recognition models. Consequently, explicitly exploiting these consistent negative responses can provide more informative guidance for cross-model transfer optimization. Motivated by this, we propose Dynamic Negative Target Guidance (DNTG), a plug-and-play method designed to boost black-box transferability. Specifically, DNTG dynamically mines a set of Top-$K$ high-response negative targets from the surrogate model. Instead of relying on rigid targets, we construct a smoothed, soft negative distribution via temperature-scaled softmax reweighting, explicitly guiding the optimization while mitigating the risk of overfitting to isolated surrogate features. Furthermore, by periodically updating this target set during attack iterations, DNTG ensures that the guidance trajectory adaptively aligns with the continuously evolving state of the adversarial example. Extensive evaluations on the NTU RGB+D 60, NTU RGB+D 120, and HDM05 datasets demonstrate that DNTG can be seamlessly integrated into diverse attack algorithms and improves average black-box transferability across multiple representative skeleton recognition models.