- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TRIAGE for clinical risk prediction from irregularly sampled medical time series. It uses outcome-specific rationales to address risk polarization, followed by SFT and GRPO with a batch-level ranking reward. Experiments on five benchmarks report improvements in prediction and calibration.
Critical strengths
The paper addresses an important clinical prediction problem. The observation of risk polarization is interesting, and the experiments cover multiple clinical benchmarks.
Critical weaknesses
-
I am not convinced that the method offers sufficient novelty. The two outcome-specific rationales are generated independently and concatenated, rather than explicitly compared or revised. The subsequent SFT and GRPO largely follow existing methods. It is also unclear what specifically addresses irregular sampling.
-
I find the baseline presentation potentially misleading. Table 3 highlights only explainable methods, although specialized models perform better on some predictive metrics (e.g., P19 AUPRC: 56.6 vs. 53.8). The caption explains this choice, but I do not think explanation availability justifies different rankings for the same metrics. Table 4 also excludes specialized models, while Table 13 shows higher overall explanation ratings for GPT-5.1 than TRIAGE (4.68 vs. 4.48).
-
The ablations are not sufficiently convincing. Answer-only SFT already achieves 53.4 AUPRC versus 56.4 for TRIAGE SFT, and the one-sided baseline uses a different scoring procedure. Appendix G.1 also shows that sample-level rewards already reduce ECE from 0.246 to 0.087, compared with 0.079 for batch-level rewards. The individual contributions remain unclear.
-
I also noticed differences in the P12 results. Tables 3–4 report AUPRC 59.0, ECE 0.044, and Brier score 0.089, while Appendix G.1 reports 64.35, 0.079, and 0.037. The SFT reference results also differ. These differences need a clearer explanation.
Questions for authors
-
What fundamentally distinguishes TRIAGE from outcome-conditioned reasoning supervision followed by GRPO? What is specific to irregular time series?
-
Why are specialized models excluded from the highlighted rankings in Table 3 and the calibration comparison in Table 4?
-
Could the authors provide more controlled ablations to separate the effects of dialectical reasoning and the batch-level reward?
-
Could the authors clarify the different P12 results, particularly the checkpoint selection and evaluation settings?
11 comments
Response to Reviewer pvxz (2/3)
[W1-2, Q1-2] Handling of Irregular Sampling
As clarified in [W1-1, Q1-1], the goal of TRIAGE is to have an LLM produce well-calibrated risk scores with explanations when it is given irregular medical time series. Its text input, however, can represent irregular sampling directly, because a textualized record only needs to contain the values that were actually observed. Following the set-based encoding of Horn et al. (2020), we serialize each observed measurement together with its timestamp (Sec. 5.1), so the input needs no imputation or resampling and the sampling pattern is preserved. Given this input, the LLM can interpret the sampling pattern on its own, and some of its rationales do so in ways that resemble dedicated components of ISMTS architectures (pvxz-Table 1). This behavior appears without any irregularity-specific design, so we see LLM reasoning as a promising direction for irregular clinical time series.
pvxz-Table 1. Irregularity signals modeled by ISMTS architectures, and example TRIAGE rationales that draw on the same signals (P12).
| Irregularity signal | TRIAGE rationale (RecordID) |
|---|---|
| Time since last observation (GRU-D) | “Mechanical ventilation is used early and then not documented beyond 31.2 h, … implying successful weaning” (143187) |
| Observation frequency / intensity (IP-Net) | “Electrolytes (Na, K, Mg, HCO3) … are repeatedly measured, indicating close monitoring” (153430) |
[W2 & Q2] Presentation of specialized ISMTS models in Tables 3 and 4
We thank the reviewer for carefully examining our baseline comparisons and highlighting concerns about the presentation of specialized ISMTS models. We would like to re-emphasize that effective clinical decision support must provide both high predictive performance and explainability. Specialized ISMTS models predict well, but they do not explain their predictions, which makes them hard to adopt in clinical practice [1, 2]. Post-hoc explanations do not resolve this, as the clinicians in our study found them of limited use (Appendix F.1, Table 13). Our primary target for direct comparison is therefore explainable LLM-based systems.
- Highlighted rankings in Table 3. We listed the specialized models as high-performance references. We agree that the highlighting could be misread, and we revised Table 3 to highlight the top performance across all baselines.
- Calibration comparison in Table 4. Table 4 has a different purpose from Table 3. It evaluates whether a probability that an LLM produces together with its reasoning is well calibrated, and this comparison is meaningful only among LLM-based methods. A specialized model outputs a probability without reasoning, and that probability can be freely recalibrated or otherwise adjusted post hoc, so it falls outside this comparison (the full results, requested by Reviewer kw4Q, are in kw4Q-Table 4). For reference, pvxz-Table 2 reports the specialized models averaged over the five benchmarks, and TRIAGE still has the lowest ECE and Brier score.
- Explanation ratings in Table 13. In the original Table 13, GPT-5.1 received a higher explanation rating than TRIAGE (4.68 vs. 4.48). Its explanations are informative, but it does not produce a risk score that discriminates between patients (Table 3). TRIAGE reaches a comparable explanation rating together with higher predictive performance. It does so with a 4B-parameter model, far smaller than a proprietary frontier model such as GPT-5.1.
pvxz-Table 2. Predictive performance and calibration of the specialized ISMTS models and TRIAGE, averaged over the five benchmarks.
| Model | avg. AUROC | avg. AUPRC | avg. ECE | avg. Brier |
|---|---|---|---|---|
| GRU-D | 86.7 | 55.5 | 0.190 | 0.136 |
| KEDGN | 86.8 | 55.4 | 0.191 | 0.136 |
| STraTS | 87.0 | 55.0 | 0.183 | 0.136 |
| TRIAGE$_{\text{SFT}}$ | 86.8 | 55.3 | 0.180 | 0.138 |
| TRIAGE | 87.3 | 57.4 | 0.038 | 0.077 |
Response to Reviewer pvxz (3/3)
[W3-1] Reasoning-structure ablation: gap over answer-only SFT and scoring of the one-sided baseline
We thank the reviewer for the careful examination of our reasoning-structure ablations, particularly the comparison with answer-only SFT and the scoring procedure of the one-sided baseline. We clarify these points below to better explain the contribution of our reasoning design.
Answer-only SFT is not practical for real-world workflows because it lacks explainability. In contrast, TRIAGE$_{\text{SFT}}$ provides explainability and also outperforms it by 3.0 AUPRC points. Furthermore, the one-sided rationale baseline inevitably requires a different scoring procedure. One-sided rationales suffer from risk polarization, which drives the predicted probabilities to extreme values (Sec. 4). As a result, under the same scoring as TRIAGE, its AUROC and AUPRC fall to 75.1 and 33.1 (pvxz-Table 3). To avoid understating this baseline’s performance, we instead scored it by the fraction of 10 sampled responses that predict the positive label. This costs 10 decoding passes per patient and favors the baseline, which still performs substantially worse than both the answer-only model and TRIAGE$_{\text{SFT}}$.
pvxz-Table 3. Ablation of the reasoning structure on P12 (Table 5).
| Method (P12) | AUROC | AUPRC |
|---|---|---|
| Answer-only | 86.4±1.5 | 53.4±2.2 |
| One-sided rationale (10 samples, averaged) | 83.8±1.0 | 43.1±3.6 |
| One-sided rationale (same scoring as TRIAGE) | 75.1±1.0 | 33.1±3.0 |
| TRIAGE$_{\text{SFT}}$ | 86.9±1.0 | 56.4±1.9 |
[W3-2] Reward ablation: calibration of sample-level vs. batch-level rewards (Appendix G.1)
Thank you for this observation. Compared to the sample-level reward, batch-level reward reduces ECE and Brier score by 21.3% and 9.2%, respectively. While we consider these reductions meaningful, we agree that the batch-level reward mainly improves discrimination. We revised the manuscript (Sec. 5.2 and Appendix G.1) to clarify that, although the batch-level reward is designed to improve discrimination performance, the observed improvements in calibration are secondary benefits rather than direct objectives of its design.
[Q3] Controlled ablations that separate dialectical reasoning from the batch-level reward
Thank you for the constructive suggestion. We agree that additional ablations would help clarify the individual contributions of dialectical reasoning and batch-level rewards. To separate the two effects, we ran one additional experiment: a one-sided reasoning model trained with the same batch-level reward and RL setup as TRIAGE, on the same P12 split. It differs from TRIAGE only in the reasoning format.
pvxz-Table 4. Ablation on P12 (single split). Each variant removes one component from TRIAGE.
| Method | AUROC | AUPRC |
|---|---|---|
| w/o Dialectical Reasoning | 87.23 | 60.64 |
| w/o Batch-level Reward | 88.37 | 62.48 |
| TRIAGE (Ours) | 89.23 | 64.35 |
Removing dialectical reasoning lowers AUROC by 2.00 and AUPRC by 3.71. Both dialectical reasoning and the batch-level reward contribute to the performance of TRIAGE.
[W4 & Q4] Different P12 results between Tables 3-4 and Appendix G.1
We thank the reviewer for the careful reading and for raising this inconsistency. The gap comes from a difference in the evaluation setting and a typo in Appendix G.1. We explain both below, together with our checkpoint selection protocol.
- Evaluation setting: Tables 3-4 report the average over all 5 P12 splits, whereas the ablation in Appendix G.1 was run on a single split because of its computational cost. This also applies to the SFT reference, which is why its numbers differ as well. We have added an explanation of this in Appendix G.1 to clarify this.
- Typo in ECE and Brier score: We thank the reviewer for figuring this out. The ECE and Brier score values in Appendix G.1 were swapped. The correct values are ECE 0.037 and Brier score 0.079, close to the 5-split averages in Tables 3-4 (0.044 and 0.089). We have corrected them in the revised version.
- Checkpoint selection: We run validation every 0.25 epochs for SFT and every 15 rollout steps for RL, and report test results at the checkpoint with the best validation AUPRC. Consistent with Kang et al. (2026), which shows that a higher SFT score does not reliably lead to better performance after RL, we empirically initialize RL from the best SFT checkpoint within the first 2 epochs instead of the overall best SFT checkpoint. See Appendix C.4 for details.
Response to Reviewer pvxz (1/3)
We thank the reviewer for the careful review and detailed feedback. We appreciate the recognition that our work addresses an important clinical prediction problem, that our observation of risk polarization is interesting, and that our experiments cover multiple clinical benchmarks.
[W1-1, Q1-1] Novelty of the TRIAGE Framework
We thank the reviewer for raising this important question about the novelty of TRIAGE relative to existing SFT and GRPO approaches. Our contribution centers on a reasoning formulation in which a single LLM produces a clinical risk score that its own rationale explains. SFT and GRPO are standard training methods that we use to instill this formulation. We would like to clarify the line of work that TRIAGE belongs to and what it contributes to that line.
1. Line of work: LLM reasoning for medical time series. Specialized architectures for irregularly sampled medical time series (ISMTS) mostly come from the irregular time series literature, where they are used as baselines. They focus on modeling irregularity and produce strong risk scores, but they give no rationale. Medical decisions are high-stakes, and models that support them are expected to be explainable as well as accurate [1, 2]. LLM-based methods address the same clinical prediction tasks while giving a natural-language rationale for each prediction. TRIAGE belongs to this LLM-based line, so we do not claim a new technique for handling irregularity. Because most other methods in this line output hard labels, they have rarely been compared directly with the specialized architectures, even though both use the same datasets. TRIAGE produces a risk score together with its rationale, so we fill this gap by reporting the specialized architectures alongside our main comparison with LLM-based methods (Table 3).
2. Contribution: a rationale and a risk score from one model. Early warning and triage need a cross-patient comparable continuous risk score, but existing LLM-based methods output hard labels or hand scoring to a separate predictor (Table 1). We designed the reasoning-and-answer structure for the clinical decision process, so that one model provides both the rationale and the score. Our preliminary study (Section 4) reveals the risk polarization problem that collapses the predicted probability due to early commitment and one-sided reasoning. Since a single ICU trajectory often carries signals of both deterioration and stabilization (Arnold et al., 2009; Hernandez et al., 2014), TRIAGE has one LLM reason about both outcomes and then produce the cross-patient comparable risk score, all in a single decoding pass. To the best of our knowledge, prior LLM work on ICU data has not used such a design.
3. Training: a probability-level reward that compares patients within a batch. The training procedure also differs from SFT followed by standard LLM GRPO in that each rollout’s reward depends on the predictions for other patients. The model outputs a continuous probability, and we define the reward directly on this probability. From one patient alone, however, it is difficult to judge whether a probability is appropriate for cross-patient discrimination, because the same value can be too high or too low depending on the probabilities assigned to other patients. We therefore designed a batch-level reward based on the margin between a rollout’s probability and the probabilities assigned to patients with the opposite outcome in the same batch (Sec. 5.2).
[1] Amann, Julia, et al. “Explainability for artificial intelligence in healthcare: a multidisciplinary perspective.” BMC Medical Informatics and Decision Making 20.1. 2020.
[2] Tonekaboni, Sana, et al. “What clinicians want: Contextualizing explainable machine learning for clinical end use.” Machine Learning for Healthcare Conference. 2019.