ReCaP: Calibrating Change-Point Tests with No-Change Reference Sequences

Testing an ordered sequence for a distributional change is difficult when a useful statistic has an unknown null distribution. We address this problem by using historical sequences known to contain no change as a reference sample. We compute the same statistic for the test sequence and each reference sequence, then use the resulting statistics to form a randomized conformal $p$-value. If these statistics are exchangeable under the null, the $p$-value controls Type I error in finite samples. Each statistic may be computed from its own sequence, from the pooled sample of the reference sequences and the test sequence, or from that pooled sample together with optional sequences labelled as containing a change. We also establish finite-sample Type I error control under specified forms of heterogeneity across sequences. A multiscale extension detects multiple changes while retaining the same validity guarantee. Numerical studies demonstrate that our method yields empirical Type I error close to the nominal level and comparable power. We apply the method to detect activity transitions in wearable-sensor data and screen industrial screw-driving operations for anomalies.