One Detector Can Catch Them All: Unified On-Device AI Audio Detection

Recent advancements in generative AI technologies have lowered the bar to generate human-like audio ranging from voice, music, to sound effects. Today, AI audio can be used maliciously at low cost, from deepfake calls (voice), to streaming fraud (music), and even to fake emergency/disaster scenarios (sound effects). In response, we need the ability to detect AI-generated audio content. However, adoption of AI audio detectors has been sparse largely due to three limitations: 1) existing detectors are specialized for certain types of AI audio (e.g., voice-only), which do not scale when end-users must employ one detector per use case; 2) detectors are vulnerable to audio transformations, like resampling and compression, that are commonly applied before audio reaches the end-user; and 3) more robust detectors require multiple GB of VRAM that device manufacturers are reluctant to give up for AI audio detection, forcing audio data to be sent to third party cloud services. In this paper, we argue that the "ideal" design for AI audio detection should address all three limitations: a _single_, robust, on-device model, that is capable of simultaneously detecting all three kinds of AI audio (voice, music, and sound effect). Our work presents the first step toward realizing a _unified_ AI audio detector. In order to accomplish this, we conduct a large-scale measurement study by first collecting and aggregating 15 AI audio benchmarks across voice, music, and sound effect, including AI samples generated by over 230 unique AI audio generators. Our initial results show that unified AI audio detection is possible (AUROC $\ge$ 0.91), however achieving low false positive rates sacrifices detection performance (30% TPR at 0.5% FPR). To improve existing detectors, we propose a lightweight, model-agnostic, and data-free training harness called Rubato that leverages previously unused signals (codec and sampling rate) in audio metadata as additional inputs. Experiments show that Rubato improves detection rate in low false positive regimes by over 13%.