Beyond Accuracy: Conformal Risk Control for Medical Time-Series Classification

Medical time-series classifiers are typically evaluated by predictive accuracy, yet accuracy alone provides no finite-sample control of clinically relevant deployment risks. We study conformal risk control (CRC) for medical time-series classification and show that its direct application is complicated by the hierarchical structure of medical time series: medical time series have a patient–window hierarchy in which each patient contributes many correlated windows. Treating windows as exchangeable calibration units creates a nominal sample-size illusion and may control a window-weighted risk rather than the risk for a newly sampled patient. Patient-level calibration corrects this mismatch but substantially reduces the effective calibration size, yielding a small-sample regime in which post-hoc CRC can become conservative. Separately, end-to-end conformal risk training requires careful model selection to preserve the target risk constraint. We develop classification-specific CRC for binary false-negative-rate control and multiclass coverage, together with a subject-aware formulation that calibrates directly over patients. We further extend the framework to subject-level conditional value-at-risk (CVaR) to control tail risk across patients. Across 12 architectures and three EEG/ECG datasets, experimental results reveal a validity–efficiency–stability trade-off: post-hoc CRC provides finite-sample expected-risk control under exchangeability, whereas end-to-end training can improve efficiency on binary tasks but is less efficient than closed-form fine-tuning on the multiclass ADFTD task once model selection correctly enforces the coverage constraint. Reliable deployment risk control therefore requires careful choice of both the calibration unit and the model-selection criterion used during optimization.