- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
MIST combines irregular numerical measurements and short timestamped chart events in a unified token stream. ClinicalBERT supplies type and text embeddings, a scalar principal-component projection represents textual values in the triplet stream, and gated cross-attention reinjects full text content alongside self-attention. A separate vital-sign encoder supplies additional physiological features. Experiments cover adverse outcomes, log length of stay, vital forecasting, and MIMIC phenotyping across adult and pediatric cohorts. The reported gains are substantial for several patient-level targets and smaller for aggregate forecasting. The study supports the usefulness of short clinical events, but the specific role of temporal interleaving needs stronger evidence.
Critical strengths
-
The method addresses a useful clinical data type. Short categorical chart events are explicitly distinguished from long clinical documents, and the model accepts numerical and textual records on an irregular timeline. The design offers a practical way to combine these events with an established physiological encoder rather than requiring aligned modalities.
-
The empirical package is broader than a single-task demonstration. On MIMIC, mortality AUROC increases from 0.725 to 0.789 and log-LOS MSE falls from 0.470 to 0.412. The study includes a pediatric cohort, alternative vital encoders, and a forecasting breakdown. Appendix B.4 matches IMM-TSF's ClinicalBERT and GraFITi backbones to MIST, which strengthens the comparison. These are separate within-cohort experiments, not an external transfer evaluation.
Critical weaknesses
-
The evidence does not isolate the central interleaving claim. Table 4 describes replacing seamless fusion with naive concatenation, whereas Section 5.3 and Appendix B.6 describe two streams followed by gated cross-attention. These are different controls. Even under the appendix description, the intervention changes stream organization and the placement of cross-modal interactions together. The .092 mortality AUROC drop therefore cannot be attributed specifically to interleaving or bidirectionality. Please identify the actual comparator, report its capacity and seed-wise results, and separate joint-stream attention from repeated semantic injection and late fusion. A control perturbing relative event timing while retaining content would directly address the motivating claim. The full-text timing mechanism also needs clarification. The triplet tokens carry time, but Equation 2 describes C as content embeddings without an explicit time term. Ordering key/value rows alone does not preserve their event-time binding under attention. The scalar PCA value may supply a content cue, so this is not an absence of all temporal information. Please state whether C contains time or type information and how the full semantics are associated with their original events (Sections 4.1 to 4.2, Table 4, Appendix B.6).
-
The outcome definitions and cohort analysis need a more precise clinical interpretation. Section 5.1 uses the ICU expire flag, whereas Table 1 and Appendix C.2 call the target in-hospital mortality. The exact endpoint must be identified. For CHoA, positive labels include transfer to ICU, but it is unclear whether transfers before the 24-hour prediction landmark are excluded. Truncating inputs alone does not make an already observed transfer a future outcome. In addition, both cohorts require stays longer than 48 hours, and MIMIC eligibility depends on observations between hours 24 and 48. These criteria define a selected population and limit interpretation as prediction for general admissions. Table 7 does not establish absence of meaningful selection bias merely by comparing a few distributions. Appendix C.4 is a useful attempt to examine excluded patients, but training on the full cohort and evaluating on the held-out cohort leaves the patient-disjoint split ambiguous. Please specify the outcome times, landmark eligibility, and expanded-cohort partitions before using these analyses to support prospective prediction and robustness (Section 5.1, Appendices A.1 and C.4, Tables 7, 11, and 12).
Questions for authors
-
Appendix C.2 reports paired tests, but its interpretation as excluding seed-level randomness needs justification. Please explain how three training runs enter each test and define the Wilcoxon pairing unit for repeated forecast observations. The phenotype aggregation should explain how dependence among labels is handled. Patient-level paired intervals that preserve within-patient dependence would clarify the relevant effect sizes. Corresponding uncertainty on CHoA is particularly useful because escalation AUPRC is 0.073 with displayed variation 0.021, versus 0.068 with 0.004 for GRU-D. This does not establish nonsignificance, but Table 9's MIMIC-only tests do not settle that comparison.
-
Please reconcile Figure 2 with Equations 2 and 3. The figure places self-attention before gated cross-attention, while the equations specify the reverse. Figure 3 also uses undefined settings mm=2 versus mm=0, whereas the text describes MIST versus GraFITi. Identify both models and define the percentage calculation and aggregation weights. The claim of superiority over every baseline should be qualified by metric, since Table 3 gives better MIMIC forecasting MAE for IMM-TSF, 0.526 versus 0.530 for MIST.
-
Which implementation produced the first row of Table 4, and does the gain remain against a capacity-comparable joint-stream control with the same inputs and readout? What time, type, or positional information is present in C, and how is full text content bound to each event?
-
Which exact mortality field was used? For CHoA, how many positive transfers or deaths occur before 24 hours, and how are those patients handled? Please specify prediction landmarks and event-availability timestamps.
-
In Appendix C.4, does full-cohort training mean training only on the training partition of an expanded population? Please provide retained/excluded patient counts in training, validation, and testing, and confirm that evaluated patients did not enter fitted preprocessing.
-
What are the paired units and seed-handling rules behind Table 9, including phenotype dependence and repeated forecasting targets? Can you provide corresponding patient-level effect intervals and CHoA uncertainty? These answers would materially affect the scope of the statistical claims.
-
The perturbed patient cases are illustrations, not strong proof of captured cross-modal reasoning. Please state whether Table 5's probabilities were recomputed on the perturbed records and how cases were selected. Without a matched event-removal or timing intervention, the difference between two models does not identify which highlighted events caused the prediction change. The conclusion should reflect this evidential boundary (Section 5.4, Appendix B.5).
2 comments
Q1. Missing details and CHoA significance tests
We provide the significance tests on CHoA here, and clarify the test settings as follows. Each model's per-patient predictions are averaged over its three seeds before the test, and the Wilcoxon pairing unit is the patient. The tests therefore measure uncertainty from patient sampling; seed variability is reported separately as the spread in Tables 1–3, and we will reword Appendix C.2 accordingly. For phenotyping, Fisher's method treats the 25 labels as independent, which we consider a reasonable approximation: the average pairwise correlation among the 25 labels across all patients is 0.04. We do not run seed-to-seed tests, because seeds are not paired across different architectures.
(The results below are the same as in our response to Reviewer kzhs, since both reviewers raise the same question.)
| Task | Escalation | Escalation | LOS | Forecasting |
|---|---|---|---|---|
| Baseline | RAINDROP | GRU-D | GRU-D | GraFITi |
| p-value | 9.9e-7 | 1.7e-10 | 3.2e-41 | 1.8e-26 |
The table above follows the same protocol as Appendix C.2 and is computed on escalation AUROC, LOS MSE, and forecasting MSE. The baselines are the strongest ones on CHoA in Tables 1–3, and MIST significantly outperforms each of them. We note in particular that the forecasting p-value is very small even though the gap itself is small: despite the inherent difficulty of improving forecasting, MIST's improvement over the baseline is consistent across patients.
Regarding AUPRC and patient-level effect intervals, we additionally run a paired patient-level bootstrap (2,000 resamples) on escalation AUPRC and AUROC, and report the 95% percentile interval of the difference (MIST minus baseline) and a two-sided p-value. We acknowledge that MIST's AUPRC gain is borderline, especially against GRU-D. We also note that the labels are extremely imbalanced (1.1% positive, Table 6), which makes AUPRC a noisy metric on this task. We will qualify our claims accordingly in the revision.
| 95% interval of difference (p-value) | Escalation AUPRC | Escalation AUROC |
|---|---|---|
| vs. RAINDROP | [.007, .044] (.006) | [.034, .084] |
| vs. GRU-D | [.000, .044] (.051) | [.057, .109] |
Q2. Attention order clarification
Thank you for pointing these out. The implemented attention order is the one in Figure 2: self-attention followed by gated cross-attention in each layer. We will correct Equations 2–3 and the description in Section 4.2. In Figure 3, mm=0 denotes unimodal GraFITi alone and mm=2 denotes full MIST; we will relabel the figure. Each cell reports the relative change in MSE, (MSE_MIST − MSE_GraFITi) / MSE_GraFITi, computed over the forecast targets of that vital channel and horizon bucket. The aggregate in Table 3 averages over all observed targets, so channels and horizons with more observations carry more weight. We agree that the superiority claim should be qualified by metric (IMM-TSF has a better MIMIC forecasting MAE, .526 vs. .530), and we will correct it in the revision.
Q7. Case study concerns
We acknowledge that the perturbed patients are illustrations rather than quantitative proof. Section 5.4 presents them as case studies "to investigate how MIST benefits downstream patient-outcome prediction": examples of how the cross-modal patterns we describe appear in patient histories, and of how MIST behaves in such scenarios, without revealing actual patient records. We will revise the wording in the conclusion accordingly. The probabilities are computed by feeding the perturbed histories into MIST. The cases were selected manually: we looked for patients with such highly indicative events, identified from clinical knowledge, so they are not a random sample. We also performed an event-removal experiment in which we mask the important events described in Section 5.4. MIST's prediction changes from .63 to .43 for Patient A and from .42 to .57 for Patient B, in both cases moving toward the unimodal GraFITi prediction (.23 and .65).
We thank the reviewer for sharing the detailed comments and concerns, and we kindly address them as following:
W1 & Q3: Dual-stream ablation concerns, and stream specifications
We apologize for the confusion caused by the wording. The first row of Table 4 follows Section 5.3 and Appendix B.6: we replace MIST's unified stream with a dual-stream, late-fusion mechanism, following prior approaches such as UTDE, to show that the unified-stream pipeline is what contributes. The phrase "naive concatenation" in the caption of Table 4 is inaccurate and we will correct it. The requested per-seed results of this variant are: forecasting MSE .588/.591/.592; mortality AUROC .699/.705/.687; LOS MSE .465/.473/.487; phenotype AUROC .692/.696/.692.
We acknowledge that this ablation changes both the stream organization and the fusion technique at once. We therefore add three controls, reported in the table below together with the late-fusion variant: (1) removing the cross-attention layers while keeping the unified stream and its self-attention; (2) removing the self-attention layers while keeping the cross-attention; and (3) randomly permuting the times of textual events while retaining their content. We do not include a variant that changes only the stream organization, because the unified stream is the basis of both attention layers, and removing it is equivalent to the late-fusion variant.
As shown in the table, each control degrades performance relative to full MIST, though none as much as late fusion. Keeping the unified stream and its self-attention without cross-attention ("no cross attn") already recovers a large part of the gap between late fusion and full MIST (mortality AUROC .738, against .697 and .789), which indicates that the unified stream itself carries cross-modal information; cross-attention then adds the full text semantics on top. Removing self-attention ("no self attn") or perturbing event times ("perturb time") brings performance close to the late-fusion level, which indicates that the gain depends on self-attention over a correctly time-ordered interleaved stream. All numbers are means over three seeds.
Regarding C: it contains the full text embedding only, with no temporal term. The time of each event is carried by its token in Z. Cross-attention retrieves text content from C and adds it, as a residual, to the time-stamped tokens in Z. The temporal relations between events are modeled by the self-attention layers over the unified stream. We will make this explicit around Equation 2.
W2 & Q4 & Q5: outcome definitions and cohort analysis details
The mortality label on MIMIC-III is the in-hospital expire flag, and we will use one consistent name throughout the paper. On CHoA, both adverse-outcome labels occur after the prediction landmark: to align with prior work, we exclude patients whose adverse outcome occurs during the input window, so no positive transfer or death falls within the first 24 hours. We will state this explicitly in the revision. We chose the cohort filters and input lengths on clinical grounds that prior work has also adopted, as stated in Section 5.1 and Appendix A.1. We agree that these criteria define a selected population: our results apply to stays longer than 48 hours with a 24-hour prediction landmark, not to general admissions, and we will state this scope and soften the wording around Table 7. For Tables 11 and 12, we evaluate on 2,365 patients (about 30%) of the held-out cohort and train on the remaining held-out patients merged with the training set of our main experiments; the split is patient-disjoint. Normalization statistics and the PCA projection are fit on training patients only. The held-out patients follow the same rule that outcome labels arise after the input window.