Refine, Don't Merge: Certified Repair of Blind Conformal Detectors
Abstract
Sequential conformal detectors can control false alarms yet miss changes that preserve their score distribution. This matters for score-based monitoring, including prospective monitoring of language-model agents. We formalize these rank-fiber collisions and introduce CoFi-E, which refines inherited score groups and independently certifies each retained split's information gain. We prove information retention and a geometric rate for population greedy refinement under a weak-witness condition. Freezing the repair preserves runtime false-alarm control under conditional exchangeability. In a controlled joint collision, CoFi-E detects 84.1% of streams versus 1.4% for marginal merging. Experiments span image, time-series, and intrusion data. Across 290 matched tasks, selected partitions reduce restricted delay by 34.8% against the inherited detector and 18.8% against a fixed grid; audit-gated prefixes yield 26.7% and 24.6%, respectively. These results support certified repair of specified detector blind spots.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.