acceptodds
Desk rejected at ICLR 2027

Detecting Care: Treatment Policy and Retrospective Validation in Clinical Risk Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Clinical risk models are trained on outcomes recorded under a treatment policy, so a model can learn to detect care rather than risk. On a published ICU benchmark, masking one clinician-set feature reduces AUROC more than masking any of six non-manipulable physiological features. In this paper, we prove that a risk score with no intrinsic predictive value that triggers effective treatment has retrospective AUROC strictly below 0.5, and we derive a closed-form solution for the deficit as a function of treatment policy and treatment effect. We also show that this deficit does not vanish as intrinsic quality increases, and as a result, retrospective AUROC is not comparable across sites with differing treatment policies. For empirical evidence, we split 116 eICU hospitals by severity-adjusted vasopressor practice and find that a clinician-set feature which improves same-practice external validation degrades by a similar amount on transfer to a different practice. Moreover, the model's calibration error is concentrated on untreated patients, which is exactly the group these models exist to identify. The failure we characterize is invisible to discrimination, calibration, and same-practice external validation and our results can be extended to any acted-upon score, clinical or otherwise.

Primary area:
causal reasoning
Keywords:
treatment policy, policy sensitivity, causal inference, distribution shift, external validation, clinical risk prediction, model evaluation, electronic health records, label bias

Related papers

Discussion (0)

Sign in to comment.