- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes CLINTIC, a treatment-conditioned latent neural SDE for modeling clinical trajectories. Multi-modal EHR events (assessment, imaging, pathology, treatment, genomics, and context) are encoded by modality-specific sub-encoders and a bidirectional Transformer with continuous-time RoPE into K readout latents, which are then rolled out with Euler–Maruyama integration conditioned on action tokens obtained by averaging treatment-event embeddings. The per-step decoder reads the latent before the action is applied, so that same-week treatment information does not leak into the prediction, and a JEPA loss with an EMA target encoder is used as an auxiliary objective. On MSK CCDE (oncology, weekly) and MIMIC-IV (ICU, hourly), the authors report better performance than existing counterfactual sequence models, EHR foundation models, and frontier LLMs. They further report a Chinchilla-form frontier with N*(C) ∝ C^0.25 from an IsoFLOP sweep, and claim that the progression probability read off the rollout stratifies 52-week survival, and that rollouts with a substituted regimen achieve comparable stratification.
Critical strengths
Timely problem setting. Moving EHR modeling away from static prediction of an outcome at a fixed time and toward modeling how patient state evolves under treatment as an intervention is a timely direction. It is a necessary step toward answering clinical questions that static predictors cannot address, such as regimen selection or whether a patient would meet a synthetic-control criterion.
Critical weaknesses
W1. The possibility that the performance gains come from the treatment actually administered during the prediction window is not ruled out
- Treatment records in the prediction window can leak the outcome: CLINTIC's prediction at week t appears to be conditioned on the treatments actually administered within the dream window, a_0..a_{t-1} (Eq. 2, Figure 2, and the −action ablation). However, treatment switches, dose reductions, and discontinuations within this window are decided in response to the patient's course, such as disease progression or toxicity. Thus, even though the model does not directly receive observations from the dream window, the outcome enters indirectly through the treatment record. Eq. 2 only prevents same-week leakage, and the authors themselves acknowledge that drug-end events frequently encode progression (Section 3.6). In contrast, LogReg, XGBoost, and the LLM baselines only receive information prior to the prediction window (Appendix G), and it is not stated whether CRN and the other sequence baselines receive actions from the same window; even if they do, these are only binary indicators. Indeed, in the −action ablation, DiseaseChange AUROC drops from 0.827 to 0.738, close to the best baseline (0.712), suggesting that a substantial part of the +0.115 gain highlighted in the Abstract comes from this information. The same issue appears to apply to MIMIC-IV, where actions in hours 24–48, such as vasopressors and ventilation, are fed into the mortality and LOS predictions.
- The prediction task is not clinically meaningful: For a treatment-conditioned prediction task to have clinical value, the treatment information given as input must be designed according to the clinical decision the prediction is meant to support. For example, if the goal is to support regimen selection, the input should be the candidate treatments available at the time of the decision. In contrast, information that this paper conditions on, such as discontinuation (drug-end events) decided in response to the patient's course, does not exist at the time the prediction is needed, and by the time it exists, the outcome is already known. In general world models, rollout fidelity can be evaluated with logged actions. In medicine, however, treatments are decided in response to changes in patient state in order to maintain it, so the treatment record itself is indicative of the outcome. In particular, since CLINTIC receives only treatments without any intermediate observations within the dream window, it can achieve high performance by reading the treatment record as a proxy for patient state without learning the dynamics. Therefore, directly transferring the world-model practice of evaluating rollouts under logged actions to the medical domain is not appropriate, and high performance in this setting does not indicate clinically usable predictive ability. This substantially weakens the authors' claims, which are motivated by clinical uses such as predicting the course under a different regimen or determining whether a patient meets a synthetic-control criterion (Section 1).
The authors should therefore (1) redesign the task according to the target clinical decision, using only treatment information available at prediction time, and verify in a clinical context that this input does not leak the outcome, and (2) compare models under conditions where all of them receive the same treatment information.
W2. The prognostic stratification and counterfactual rollout results do not support the claims
- Insufficient comparators and reporting for prognostic stratification: s_13 is computed with the treatments actually administered in weeks 0–12 as input, so whether treatment was changed during this period already reflects prognosis. As the authors acknowledge, this is not a comparison against a multivariate survival model (Section 3.6); still, outperforming univariate biomarkers (PD-L1, EGFR, etc.) alone is not enough to support the claim that the representation learns prognostic information. The score should be compared against standard prognostic factors such as treatment change, baseline ECOG, stage, and line of therapy. In addition, only Nivolumab is reported at T=1, 4, and 13, with the highest value at T=13 (HR 6.35) used as the headline, whereas the other regimens are reported only at T=4 (Table 3). Results at the same landmark should be reported for every regimen.
- Counterfactual rollouts without confounding control cannot be interpreted: Associations between treatment and outcome in observational EHR data are heavily confounded. For example, norepinephrine is mainly administered to patients with severe hypotension, so a model without confounding control may predict that administering it to a typical patient would substantially lower blood pressure. However, apart from conditioning on the history up to the baseline time, this study includes no explicit mechanism for confounding control (e.g., balancing, inverse probability weighting, or g-computation) and no check of identification assumptions such as positivity. In particular, because only treatments are fed into the dream window without intermediate observations, there is structurally no way to condition on the patient state that drove each treatment decision (time-varying confounding).
- The presented results do not support counterfactual validity either: That the median-split HR is preserved under regimen-substituted scores only means that the risk ranking across patients is preserved; it is not evidence that the direction or magnitude of the treatment effect is correct. The authors note that these results should be read as diagnostic proxies for model coherence rather than causal effect estimates (Section 3.6), yet the Introduction presents answering counterfactual questions, such as the course under a different regimen, as a core motivation. The authors should either align the motivation in the Introduction with the scope of what is actually validated, or provide validation such as reproducing the direction of known RCT results.
Questions for authors
See weaknesses above