EHRTemp: Learning Dynamic Clinical Decision Making from Longitudinal Electronic Health Records

Clinical diagnosis requires physicians to acquire evidence and revise differential diagnoses as the patient’s condition evolves over time. Existing sequential diagnostic agents often interact with a fixed patient snapshot, leaving the effects of clinical progression on observations and evidence acquisition underexplored. We formalize dynamic clinical decision making (DCDM) and introduce EHRTemp, an interactive environment for studying this setting through longitudinal electronic health record replay. Within EHRTemp, we train EHRTemp-AGENT to update its diagnostic belief through differential diagnosis and select between laboratory test requests and final diagnosis. We initialize the agent with differential-diagnosis supervision and optimize its clinical decisions through reinforcement learning with By-Turn Process-Advantage Estimation (BT-PAE), which aligns process supervision with the clinical stage of each decision. On EHRSHOT, our 8B agent achieves the highest Sample F1, Micro F1, and diagnostic precision among the evaluated models, and remains competitive on an independent MIMIC-IV cohort.