DASH: Learning Adversarial Distribution Shifts for Transferable Black-Box Attacks on Object Detectors

Object detectors are widely deployed in safety-critical applications. Since deployed models conceal internal information, transferable black-box attacks provide a practical robustness evaluation. Unlike image-classification attacks, detection attacks must suppress multiple objects while transferring across heterogeneous architectures. Our gradient analysis identifies two obstacles: detector gradients are poorly aligned, encouraging surrogate-specific solutions, while object gradients can conflict and interfere with joint suppression. Distribution-level generation provides a natural mechanism for this goal: by aggregating attack supervision over the training distribution and encoding it into generator parameters, it can capture reusable adversarial patterns rather than instance- and surrogate-specific optimization directions, while enabling efficient inference-time generation. Based on this insight, we propose Detector Adversarial Distribution Shift (DASH), which aggregates attack supervision over the training distribution and encodes reusable knowledge into an input-conditioned mapping from natural to adversarial images. DASH combines class-agnostic confidence suppression to coordinate multiple objects with CLIP feature disruption to reduce surrogate dependence. We instantiate this formulation with conditional GAN and Stable Diffusion generators. Across three datasets and thirteen detectors, DASH-GAN and DASH-SD achieve average ASRs of \(83.6%\) and \(80.08%\), outperforming the strongest baseline by up to \(36.14\) percentage points with generation times of \(0.02\) and \(0.75\) seconds per image. DASH-SD also averages \(28.14%\) ASR on two ODDefense-protected models, exceeding the strongest prior baseline by \(10.76\) percentage points. Thus, DASH balances black-box transferability, generation efficiency, and defense robustness.