- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Authors develop a world model that represents patient EHR using patient conditions, measurements, and medications. They create an algorithm to create patient future trajectories conditions on treatment. They also develop classfier for several conditions like sepsis, MIs, etc.
The main conribution is a new archtecture for medication conditions generations of future trajectories.
They use MIMIC-IV with patient disjoint train, val, and test sets. They evaluate future measurement accuracy, outcome prediction, similarity between generated and observed representations, and preservation of demographic associations. Results are mixed.
Generating patient trajectories is not itself new.
Changing medication inputs changes the generated trajectories, but the clinical correctness of those changes is not validated
Critical strengths
An interesting model achieving competative performance with other investiagtoed models capable of generating conditioned patient future trajectories
Critical weaknesses
The paper does not isolate whether generating future trajectory improves outcome prediction over directly using the same observed patient history.
Outcome definitions are doubioys considering that MIMIC-IV was used. Authors predict long term wheres MIMIC-IV is a reconst of ED and ICU followed by hospital admission. MIMIC is much better fro short-term inpatient forcasting (like sepsis).
Questions for authors
I am not really clear how medications inputs are selected. Are subsequent medication inputs the treatments actually administered later, or a plan known at prediction time? Where do the reported forecast windows begin relative to the two observed initialization periods, and do all baselines receive equivalent information?
You predict trajecroty based on latent representation and then predict outcomes based on trajecotry. What is vaue added of tarjecotry. What not to predict everything using latent info?
It needs to be explained how long term disease pancratatic cancer is established. In this dataset if patient have no ED or hospitalisatio that does not mean that it is negative for given outcome (pancreating cancer)
Figure 5 is not explaned in suffcient detail, what are the trajecotries what do they represnt. when medications for those patients where administered?
3 comments
I appreciate the authors’ efforts to address my comments on such short notice. I find their responses and the corresponding revisions satisfactory.
I would, however, like to highlight one omission that I overlooked in my original review and subsequently raised in my comment to the Area Chair. ETHOS is particularly relevant prior work: it was trained and evaluated on MIMIC-IV, has a publicly available implementation, and generates patient-specific future trajectories conditioned on longitudinal medical histories. It should therefore be cited and discussed directly, including how EHR-WM’s explicit separation of patient states and clinical actions differs from this earlier approach.
Reference: Renc et al. “Zero shot health trajectory prediction using transformer.” npj Digital Medicine 7, 256 (2024). https://doi.org/10.1038/s41746-024-01235-0.
Response S43h
Thank you again for your constructive feedback and for sharing the relevant literature. We have provided a detailed response to your original comments in the Senior AC thread above, including an explanation of the corresponding revisions to the manuscript. We hope these revisions adequately address the remainder of your concerns and would appreciate your consideration of an updated score if it addresses all your concerns.
Updated: Thank you again for making our submission stronger with your feedback and follow up experiments :)
Rebuttal
Response: We thank the reviewer for this suggestion. We introduce a matched-information ablation comparing EHR-WM against direct prediction from initial latent states, with and without clinical actions. All models share the same initialization context, state encoder, and evaluation protocol. Importantly, STATES + ACTIONS receives the same recorded action sequence as EHR-WM, but without autoregressive trajectory generation, isolating the contribution of learned latent dynamics rather than additional clinical information.
As shown in Figure 6, EHR-WM improves AUROC over STATES + ACTIONS by 0.090, 0.079, and 0.029 for ICU admission, long LOS, and sepsis, respectively. These results demonstrate the additional predictive value of explicitly modeling patient-state evolution.
Trajectory-Based Prediction Ablation (AUROC, MIMIC-IV)
Revision: Section 4, Figure 6.
Response: We thank the reviewer for this observation. We revise the primary multi-site benchmark to emphasize inpatient outcomes, including ICU admission, sepsis, mortality, length of stay, and readmission where available. We remove long-horizon disease outcomes from the primary comparison (include them in the appendix) and clarify the limitations of disease-incidence labels derived from incomplete longitudinal follow-up.
Revision: Section 4 and Appendix D.
Response: We clarify that medication inputs are subsequently recorded administrations, not treatment plans known at prediction time. Actions are encoded as binary indicators across five treatment classes in six-hour intervals. EHR-WM initializes from two observed windows and generates future states autoregressively without future physiological observations, conditioned on recorded actions. Forecasting covers 6–48 hours after initialization, with all baselines receiving the same initial context and evaluated on identical variables, horizons, and observations.
Revision: Sections 3–4 and Appendix D.3.
Response: We address this directly through the matched-information ablation in Comment 1. Even when direct prediction receives the same initial states and action sequence, EHR-WM consistently performs better, demonstrating that autoregressive latent trajectory modeling contributes predictive value beyond directly using the available representations and actions.
Revision: Section 4, Figure 6.
Response: We thank the reviewer for highlighting this limitation. Disease-incidence labels are defined using the first recorded ICD diagnosis within 365 days, excluding patients with prior diagnoses. We clarify that absence of a recorded diagnosis does not establish absence of disease, since care outside the observed health system is unavailable. We therefore prioritize inpatient outcomes in the primary evaluation and explicitly discuss this limitation and move the remainder of the tasks to the appendix.
Revision: Appendix D.1 and Section 5.
Response: We thank the reviewer for requesting clarification. We have expanded the intervention analysis to specify that trajectories represent simulated latent patient-state evolution under recorded or modified medication sequences as suggested by reviewer
edcc. The matched-cohort experiments initialize trajectories using the first 24 hours of patient history, introduce interventions between 24 and 54 hours, and evaluate physiological responses at 54–60 hours against observed treated–control differences. We have revised the figure captions to clarify intervention timing and interpretation.Revision: Section 4, Figure 4.
Thank you again for the constructive feedback! Please let us know if these additional experiments and analysis provide sufficient evidence to warrant a score raise.