From Forecast Error to Triage Decisions: Patient-Centered Evaluation of ICU Vital-Sign Forecasting

In intensive care, a vital-sign forecast would be acted on through an early-warning score, a step function that bins each parameter into a few ordinal levels and sums the points. We therefore evaluate ICU vital-sign forecasts through that score. Each forecast is mapped through a four-parameter NEWS2-style risk index (respiratory rate, SpO${}_2$, systolic blood pressure and heart rate), converted into triage levels and scored against two separate ground truths: the index on the observed vitals and events recorded in the 24 h after the forecast origin (termination, vasopressor start, dialysis start, invasive ventilation or any of these). On 347,648 windows from the MIMIC-IV dataset, the 9.3% masked-MSE lead of the best in-domain-trained model over the best zero-shot foundation models shrinks to 1% in weighted $\kappa$. With the same 9.6% of windows flagged in both states, so that every extra event recovered is one fewer false alarm, triage on a zero-shot TimesFM forecast recovers 18% more deterioration-bearing windows than triage on the currently observed vitals at $H=6$ h (sensitivity 12.4% to 14.7%; 95% CI on the gain 13.9 to 22.3, resampled over patients). Ranked by the raw subscore rather than by level, the gain is 9.0%; ranked by level, it decays to 7.2% at 24 h. Squared error does not select the forecast model family that recovers the most events: at a forced 10% budget, GRU, the model with the lowest MSE, recovers fewer deteriorations than every zero-shot model (Chronos-2 minus GRU $+6.2$ relative-gain points, 95% CI 3.0 to 9.4, paired on the same patients) and 27 of 36 paired zero-shot-minus-trained contrasts exclude zero. In addition, we report the harmonised benchmark behind the forecasts (MIMIC-IV-CLIF and eICU-CRD, six vitals, $L=48$ h, $H=1$ to $24$ h, 16 models) and release the risk and triage protocols as secondary evaluations for vital-sign forecasting benchmarks.