acceptodds
Under review as a conference paper at ICLR 2027

OICycle: Observe, Infer, and Intervene Cycle for Belief-State Learning in Longitudinal Clinical Decision Making

Abstract

Clinical care is a closed loop: clinicians observe partial evidence, infer the patient's state, and intervene, and each intervention determines what is observed next. This makes clinical decision making a partially observable Markov decision process (POMDP), in which a belief over a latent patient state must be updated from irregular, multimodal, and incomplete records. Yet most deep clinical models predict from a single snapshot or from coarsely aggregated history, while POMDP-based clinical methods need hand-specified discrete state spaces and transition models that real electronic health records cannot supply. We introduce OICycle (Observe, Infer, and Intervene Cycle), which learns a stochastic belief state directly from retrospective trajectories. At every episode, a learned recurrent operator fuses a sample of the previous belief with the new multimodal observations and the intervention that produced them. The resulting Gaussian belief is trained to predict clinicians' next interventions, the time to the next episode, and next-episode mortality. To enable reproducible evaluation, we release a benchmark derived from MIMIC-IV, MIMIC-IV-Note, and MIMIC-CXR that organizes 75,075 hospital admissions (5.4M laboratory, microbiology, radiology, prescription, and note records) into temporally ordered therapeutic–diagnostic episodes. Compared with an otherwise identical history-free model, OICycle improves next-intervention adaptive recall from 0.256 to 0.347, time-to-next-episode macro-F1 from 0.293 to 0.351, and next-episode mortality macro-F1 from 0.585 to 0.649. Analysis shows that the learned belief encodes intervention type and trajectory position and evolves smoothly across episodes, and that the benefit of history is concentrated in interventions whose ordering depends on prior findings.

Primary area:
learning on time series and dynamical systems
Keywords:
longitudinal clinical decision-making, multimodal clinical data, Belief-state learning, Latent state representation, POMDP, patient trajectory modeling

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 2 reviews, above 0% of the others.
Reviewer_MZfZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a states based model approach to forecast along the ordered sequence of clinical episodes, derived from the MIMIC dataset. The authors first split each hospital admission from MIMIC into an ordered sequence of clinical episodes and classifying them into either diagnostic or therapeutic. As a proposed method, A GRU keeps a running 512-d latent “belief state" about the patient. At each episode it combines the previous belief with an embedding of the new data, which comes from code embeddings, BioClinicalBERT for text, and image features for chest X-rays. The belief is parameterized as a Gaussian and sampled, and the sample is trained only to predict interventions, timeline, and mortality about the next episode.

Critical strengths
  1. The work is based on the MIMIC dataset, which is a widely used open dataset. This increases the transparency, which is particularly valuable in the healthcare domain.
  2. The authors also open source the pipeline that adapts the MIMIC-IV datasets into admissions that structured as episodes, combining with notes and imaging data.
  3. The categorization of diagnostic vs therapeutic is an intuitional design. And the proposed state model does not need hand-defined states, transitions, or observation model.
Critical weaknesses
  1. My biggest concern is around the significance and novelty of the study. As one of the main conclusions of the paper, the authors argue that predicting future disease states by updating patient belief states with history sequentially (OICycle), specifically compared to a version without doing clinical history update (OICycle-S). Basically the authors show that the recurrence helps the three specific prediction tasks, which nobody would doubt and is a trivial knowledge. Furthermore, previous studies like Doctor AI (Choi et al., 2016) already adopts a GRU structure over EHR history to predict the next event. Therefore, it is not clear what novelty this paper is bringing into the community.
  2. Related to the first point, the evidence that the author proposed to support their claims are very thin. Namely, the only baseline that OICycle compared against was a null baseline without the clinical history, which should act as a control that the clinical state models are expected to beat. The paper should include much more stronger baselines to show that the gain of the proposed structure is not trivial, such as EHR models like BEHRT, or plain GRU or transformer over events for future prediction.
  3. At the step one, there was a single note per patient that was fed into the model. Is that note the discharge note? If so, the note would likely contain information about the endpoint from the series of episode, and it would be leakage for the model that requires additional filtering. If it is mot the discharge note, what note is it that's used at step 1? I'm only asking this because in my experience with MIMIC-IV most visits would only have discharge notes with no entry notes.
  4. There are many details of the method design that is under specified. For example, no window size, conflict resolution, time bins, or mortality label was defined.
Questions for authors
  1. Can you add more baseline methods than a null baseline to demonstrate effective gains of OICycle?
  2. Can you specify the exact type of note was used at step 1 and how was leakage prevented? Can you also specify the window size, conflict resolution, time bins, or mortality definition?
  3. Please include discussion with adjacent relevant works such as Doctor AI (Choi et al., 2016) and explain the novelty of this work compared with them.
Reviewer_LUWAOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper adds to a rich literature of honoring the complexities of clinical time-series (missingness, confounding, temporal dependence, etc) by leveraging intermediate modeling strategies. In this work, inspired by belief states and POMDP formulations also introduces a novel, to me, way of partitioning time series into distinct time intervals centered on therapeutic and diagnostic interventions. The result is a proposed general clinical prediction framework which is applied to a patient cohort derived from MIMIC-IV, inclusive of medical imaging, clinical notes, on top of the structured data contained in the electronic medical records. This multi-modal encoding of patient histories (merging structured and unstructured data) is not in itself novel (see Marzyeh Ghassemi's PhD work) nor is the longitudinal, history based encoding of intermediate patient states (see GRU-D) but their combination in the specific architecture inspired by POMDPs in the pursuit of belief states (something done solely on structured patient data in Killian, et al at ML4H 2020 and Li, et al which was cited in this work) is a nice contribution.

Suresh, H., Hunt, N., Johnson, A., Celi, L. A., Szolovits, P., & Ghassemi, M. (2017, November). Clinical intervention prediction and understanding with deep neural networks. In Machine learning for healthcare conference (pp. 322-337). PMLR.

Ghassemi, Marzyeh, et al. "A multivariate timeseries modeling approach to severity of illness assessment and forecasting in ICU with sparse, heterogeneous clinical data." Proceedings of the AAAI conference on artificial intelligence. Vol. 29. No. 1. 2015

Ghassemi, Marzyeh, et al. "Unfolding physiological state: Mortality modelling in intensive care units." Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014.

Che, Zhengping, et al. "Recurrent neural networks for multivariate time series with missing values." Scientific reports 8.1 (2018): 6085.

Killian, T.W., Zhang, H., Subramanian, J., Fatemi, M. &amp; Ghassemi, M.. (2020). An Empirical Study of Representation Learning for Reinforcement Learning in Healthcare. <i>Proceedings of the Machine Learning for Health NeurIPS Workshop</i>, in <i>Proceedings of Machine Learning Research</i> 136:139-160 Available from https://proceedings.mlr.press/v136/killian20a.html.

Li, L., Komorowski, M., & Faisal, A. A. (2019). Optimizing sequential medical treatments with auto-encoding heuristic search in pomdps. arXiv preprint arXiv:1905.07465.

Critical strengths

The paper has come to a nice set of design decisions in how to incorporate complex, multi modal data prevalent in clinical settings. Aligned with recent work from Shengpu Tang (citation below), which covers temporal misalignment within common paradigms of RL applied to healthcare settings, the choice to set up distinct time intervals around therapeutic and diagnostic interventions was neat.

The modeling choice of a multi-stage prediction of latent states and basing predictions from those was nice. As I detail below, the construction of how this was laid out in the paper needs sufficient revision but I could understand the gist of what the authors intended and do feel that they've made some nice choices. I'm not entirely sure how novel they are in aggregate but I do feel that the authors are on the right track.

Tang, Shengpu, et al. "Off by a beat: the effects of temporal misalignment in reinforcement learning for sepsis treatment." NPJ Digital Medicine 9.1 (2026): 360.

Critical weaknesses

The paper is poorly constructed and core concepts are discussed prior to them being introduced. For example, I feel that most of Section 3 could have been made a part of Section 4 (or at least elevated the definitions of terms and concepts done at the beginning of Section 4 to an earlier position in the paper). The notation and concepts that are used to partition the time-series are non-standard in medical ML, particularly in a sequential decision making setting. The use of "episode" in place of a "state" was confusing and perhaps unnecessary. Traditionally, a patient stay represented by the extracted multi-variable (and in this paper's case multi-modal too, a strength!) time-series is commonly referred to as a trajectory and/or episode (which is still a poor use of the term and concept). I think that the authors attempted to convey that the infrequent (and perhaps prolonged) intervals of time between therapeutic and diagnostic interventions is a cleaner defintion of a "time step" in decision making within this domain. I'm inclined to agree but the paper fails to adequately motivate this choice, choosing to assume that the reader understands the intention of the terms and abbrevations + acronyms chosen.

Along these weaknesses, the paper doesn't adequately describe what features are used to compose the information state under which predictions are made. The initial latent state z0z_0 is not clearly articulated nor discussed on how it is initialized, where it is sampled from, etc. Ultimately, I did not gain any clear understanding of how the clinical data is actually used. There was not an attempt to clearly define the notation and what those concepts represent with regard to the real data (intervention types and subtypes similarly were not adequately described). Based on Figure 1, it's clear that the authors have considered this but there's a major gap in the text describing the set-up between anchoring Figure 1 and Figure 2 and where information is ingested and predictions are provided as output of the OICycle framework. To this end, there are no clear modeling descriptions of how the interventions+predictions y are output. In many cases, this paper did not read as a finished and complete work.

The final example of this last statement is that OICycle was not compared to relevant baseline approaches that provide similar predictive models (most egregious is mortality prediction). I cannot gauge any level of significance of the specific contributions put forward by this paper if it is not used to measure against existing approaches. The concepts contained within are not rigorously presented or evaluated on their own for me to consider this work sufficiently mature for publication.

Questions for authors

I don't have any real questions for the authors. I do like the direction of this paper but found the execution to be significantly lacking. I would recommend a thorough revision and re-submission at a later time. I am usually a major proponent of giving authors the benefit of the doubt but the changes needed to have me adjust my perspective of this paper are significant enough that I do not feel that it would be possible for the authors to do so during the review period. As such, I do not foresee me raising my score above "weak reject". For now, this paper is very clearly a "reject" in my estimation.

While not a question, I think that it's appropriate to raise a concern here that the authors have removed all years from the bibtex fields of the papers they cite. This isn't unethical but it does create a borderline violation of the intellectual honesty I would expect from a proposed scholarly contribution to the field. I appreciate that this was likely done to save space with the inline citation format of ICLRs LaTeX template. This was not a mistake but a clear choice by the authors to adjust or change the template that is expected to be adhered to.

Also, the authors left a public github link to the codebase used for this work, breaking double blind.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.