- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose a states based model approach to forecast along the ordered sequence of clinical episodes, derived from the MIMIC dataset. The authors first split each hospital admission from MIMIC into an ordered sequence of clinical episodes and classifying them into either diagnostic or therapeutic. As a proposed method, A GRU keeps a running 512-d latent “belief state" about the patient. At each episode it combines the previous belief with an embedding of the new data, which comes from code embeddings, BioClinicalBERT for text, and image features for chest X-rays. The belief is parameterized as a Gaussian and sampled, and the sample is trained only to predict interventions, timeline, and mortality about the next episode.
Critical strengths
- The work is based on the MIMIC dataset, which is a widely used open dataset. This increases the transparency, which is particularly valuable in the healthcare domain.
- The authors also open source the pipeline that adapts the MIMIC-IV datasets into admissions that structured as episodes, combining with notes and imaging data.
- The categorization of diagnostic vs therapeutic is an intuitional design. And the proposed state model does not need hand-defined states, transitions, or observation model.
Critical weaknesses
- My biggest concern is around the significance and novelty of the study. As one of the main conclusions of the paper, the authors argue that predicting future disease states by updating patient belief states with history sequentially (OICycle), specifically compared to a version without doing clinical history update (OICycle-S). Basically the authors show that the recurrence helps the three specific prediction tasks, which nobody would doubt and is a trivial knowledge. Furthermore, previous studies like Doctor AI (Choi et al., 2016) already adopts a GRU structure over EHR history to predict the next event. Therefore, it is not clear what novelty this paper is bringing into the community.
- Related to the first point, the evidence that the author proposed to support their claims are very thin. Namely, the only baseline that OICycle compared against was a null baseline without the clinical history, which should act as a control that the clinical state models are expected to beat. The paper should include much more stronger baselines to show that the gain of the proposed structure is not trivial, such as EHR models like BEHRT, or plain GRU or transformer over events for future prediction.
- At the step one, there was a single note per patient that was fed into the model. Is that note the discharge note? If so, the note would likely contain information about the endpoint from the series of episode, and it would be leakage for the model that requires additional filtering. If it is mot the discharge note, what note is it that's used at step 1? I'm only asking this because in my experience with MIMIC-IV most visits would only have discharge notes with no entry notes.
- There are many details of the method design that is under specified. For example, no window size, conflict resolution, time bins, or mortality label was defined.
Questions for authors
- Can you add more baseline methods than a null baseline to demonstrate effective gains of OICycle?
- Can you specify the exact type of note was used at step 1 and how was leakage prevented? Can you also specify the window size, conflict resolution, time bins, or mortality definition?
- Please include discussion with adjacent relevant works such as Doctor AI (Choi et al., 2016) and explain the novelty of this work compared with them.