The Unactionability of Concept Drift Detection

Change detectors are commonly used alongside fixed heuristic rules as adaptation mechanisms in online learning algorithms deployed in non-stationary environments. This work investigates whether a change detector signal contains sufficient information to determine an appropriate model adaptation strategy. We show theoretically that a detector can be unactionable: distinct post-drift distributions may produce the same detector signal while having different optimal adaptation strategies. We further derive an information-theoretic bound showing that the information available to an adaptation policy is limited by the information exposed by the detector, and that this can lead to unavoidable misadaptation when the entropy of the optimal adaptation decision exceeds detector capacity. We evaluate these results on controlled synthetic streams and study their practical implications across 24 streams from the USP DS repository. Empirical results show substantial variation in the utility of fixed detector-triggered adaptation across stream-learning architectures and datasets, motivating adaptation policies that utilise more information than a fixed detector signal alone.