CancerEmbed: Learning Temporal Patient Representations with Comorbidity-Guided Geometry

Identifying the primary site of cancer from longitudinal electronic health records (EHRs) is challenging due to heterogeneous clinical evidence, irregular trajectories, and nonspecific phenotypes. We introduce CancerEmbed, a representation-learning framework that formulates primary-site identification as trajectory-aware patient retrieval over structured and unstructured EHRs. CancerEmbed combines a diagnosis-relative temporal Transformer that integrates summarized clinical notes with structured context, cluster-specific sparse attention (CSA) that extracts cancer-specific evidence directly from full clinical notes, and comorbidity-guided deep metric learning (DML) that structures patient representations using phenotypic similarity beyond cancer labels. Across five cancer types, CancerEmbed demonstrates strong generalization to an independent cohort of diagnostically challenging cases. The temporal Transformer with SupCon achieves 86.8% H@5 and 61.1% P@5, improving over the same representation without DML by 28.3 and 15.1 percentage points, respectively, and reaches 64.2% classification accuracy and 45.4% macro-F1. Complementarily, CSA achieves 81.1% H@5 and 53.6% P@5 without DML using full clinical notes alone, without structured EHR inputs at inference. These results show that jointly modeling temporal, semantic, and phenotypic similarity improves generalization under distribution shift and that metric-learning gains depend on the underlying representation. More broadly, CancerEmbed provides a framework for organizing heterogeneous longitudinal clinical data into meaningful patient neighborhoods for retrieval-based reasoning.