Learning from Ugly Ducklings: Hierarchical Modeling of Intra-Patient Variance for Melanoma Prediction

Medical AI shows great potential to improve the efficiency and accuracy of diagnosis from medical images. However, disease manifestations vary across individuals, such that a similar observation may be typical for one patient but concerning for another. Most medical image classifiers assume independent and identically distributed (IID) observations and therefore do not account for intrapatient variation. In melanoma detection, dermatologists account for this variation by assessing both a lesion’s appearance and whether it stands out from the patient’s other lesions, a pattern known as the ugly-duckling principle. We model this setting with a hierarchical model that estimates a patient reference from unlabeled same-patient lesions and derives how deviations from this reference modify lesion-level evidence. We introduce EIDER (Evidence In Deviations from an Estimated Reference), which estimates this reference and learns how lesion-to-reference deviations adjust lesion-level predictions. We evaluate EIDER across three dermatology settings. On total-body photography, EIDER achieves an AUROC of 0.959, compared with 0.921 for our strongest image-only baseline. Controlled analyses show that replacing a patient's reference with that of another patient decreases performance, larger reference sets are associated with greater gains over lesion-only prediction, and combining signed and squared deviations performs better than either restricted deviation form; we further evaluate EIDER in dermoscopic and longitudinal settings, and assess its applicability to EEG seizure detection.