ACTIVE LEARNING WITH UTILITY-TAILORED STRICTLY PROPER SCORES

Medical active learning must value labels according to the decisions that predictions support, while preserving accurate probability estimation. We introduce Domain-Utility-Sensitive Active Learning (DUSAL), which constructs a strictly proper score from a complete action–utility table. DUSAL-Hard adds normalized Bayes value to a strictly proper base potential. Its divergence decomposes exactly into predictive divergence and normalized decision regret; we characterize this augmentation up to affine terms. Under exact Bayesian updating, acquisition separates into Bayesian Estimate of Mean Proper Scores (BEMPS) and clinical expected value of sample information (EVSI). We also derive representation invariances, a smooth utility-gap variant, and predictive and decision convergence under explicit progress assumptions. Exact finite-model experiments demonstrate decision-regret reductions through boundary-sensitive acquisition and within-region smoothing, alongside prediction–decision trade-offs. Across five public medical tasks, DUSAL makes label selection sensitive to utility information, producing measurable differences from matched BEMPS. Although additional cost gains from utility augmentation remain unconfirmed under Holm correction, DUSAL-Div achieves Holm-controlled reductions in realized-cost area under the learning curve (AULC) on three of four tasks relative to top-bselection. Comparable gains with BEMPS-Div demonstrate the practical benefit of batch diversification.