Deployment Performance Estimation for Time-Series Anomaly Detection

Time-series anomaly detectors can exhibit substantial instance-to-instance performance variation even within the same data source. Yet conventional benchmark evaluations summarize performance over labeled collections and provide little indication of how a given detector will perform on a particular deployment instance. We define deployment-time performance estimation (DPE) for time-series anomaly detection as the task of estimating a detector's instance-level performance on an unlabeled deployment instance, given its historical evaluation records. To address this challenge, we introduce TimeDPE, which constructs a compact, label-free surrogate representation from the detector's anomaly-score sequence and the raw time series to capture performance-relevant information, and learns its relationship with detector performance from historical evaluations. Across 17 data sources and 10 anomaly detectors, TimeDPE reduces the macro-averaged mean absolute error of instance-level VUS-PR estimation by 40.2% relative to the conventional practice of predicting each target with the mean performance from historical benchmark evaluations. Moreover, we show that the surrogate representation plays a key role in TimeDPE's sample efficiency, enabling accurate performance estimation from limited historical evaluations.