Uncertainty-Driven Unsupervised Detection Of Reliability-Relevant Distribution Drift

Unsupervised concept drift detection commonly relies on identifying changes in the distribution of inputs to a trained machine learning model. Yet not every distribution shift should trigger an alarm: a deployment distribution may differ from recent observations, while remaining covered by prediction-relevant variations encountered during training. Drift thus should be triggered when incoming data introduce novel predictive behavior that was insufficiently supported by the training data. We show that this distinction is naturally captured by epistemic uncertainty (EU). Focusing on Bayesian neural networks (NNs) obtained through a Laplace approximation, we de-rive upper and lower bounds on EU in a local prediction-relevant geometry induced jointly by the trained model and its training data. The bounds reveal that EU along training-covered directions is attenuated according to the information accumulated during training, whereas components orthogonal to the training-supported subspace retain substantially larger EU. Guided by this characterization, we develop an unsupervised drift detector that monitors EU rather than input-distribution change directly, and can be applied post hoc to conventionally trained NNs. Experiments across non-stationary learning settings show that the proposed approach suppresses alarms on training-covered shifts while reliably detecting novel ones.