- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces S^3M, a multivariate time-series architecture that integrates graph-spectral operators into recurrent state-space transitions. Its key components include dynamic graph construction, B-spline spectral filtering, input-dependent top-(s) routing, and availability-aware pooling. The theoretical analysis establishes diagonalization of the linear transport operator in the graph Fourier basis, conditions for cross-variable mixing, top-(s) support optimality, and a uniform state bound induced by LayerNorm. On PhysioNet 2019 sepsis data, with approximately 89.86% missingness, S^3M achieves an AUROC of (0.9239 \pm 0.0048) across three seeds, outperforming 15 baselines under the authors' controlled evaluation protocol. However, the evaluation is limited to a single clinical dataset, the temporal relationship between trajectory-level labels and input observations remains unclear, and per-step eigendecomposition incurs O(D^3) computational cost, potentially limiting scalability.
Critical strengths
S^3M introduces a mathematically interpretable recurrence by embedding a graph-spectral operator directly into the linear state transport, accompanied by clear theoretical characterizations of both its Graph-Fourier diagonalization and measurement-coordinate cross-variable mixing. Additionally, the empirical protocol is transparent and well-controlled: it maintains a fixed 70/15/15 split under native missingness across three seeds, and candidly reports computational bottlenecks, including the ~60.3-hour RTX A6000 training time and per-step eigendecomposition costs.
Critical weaknesses
- The clinical prediction task rests on an insufficiently specified temporal setup, making it unverified as a prospective early-warning system. The model uses full 72-hour patient trajectories to predict an endpoint label, but the manuscript does not adequately specify the temporal relationship between the input observations and sepsis onset. This formulation differs materially from the original PhysioNet 2019 challenge, which evaluates hourly pre-onset predictions using a timing-sensitive clinical utility function. Without establishing whether post-onset measurements can enter the input trajectories, the reported AUROC cannot be reliably interpreted as evidence of prospective early-warning performance.
- The empirical evidence exhibits limited generalizability and does not establish a rigorously comparable state-of-the-art advantage. The evaluation is confined to a single clinical dataset under one fixed patient-level split, lacking validation on independent datasets, multi-center cohorts, or non-clinical multivariate domains, while component ablations rely on only a single training seed. Furthermore, Table 3 juxtaposes results obtained under conflicting evaluation protocols; although these protocol differences are disclosed, the comparison does not establish a directly comparable state-of-the-art claim across structured time series.
- The theoretical derivations largely restate standard algebraic properties of the chosen design: Lemmas 1–3 and Theorem 1 follow directly from shared-basis eigendecomposition, simplex entropy regularization, and LayerNorm bounds. And, the manuscript does not adequately contrast its core mechanism with existing graph state-space models and dynamic spectral filters, leaving it unproven what concrete capabilities or advantages it provides beyond prior methods.
Questions for authors
- Does the fixed 72-hour input window include observations recorded after sepsis onset, and given the manuscript's explicit acknowledgment that verified onset timestamps are unavailable, how do the authors establish that the evaluation measures prospective early-warning performance rather than retrospective outcome classification using potentially post-onset data?
- In the all-missing boundary case of Eq. (6), finite logits cause the softmax to assign strictly positive weights to missing positions; how is this behavior reconciled with the claimed zero-weight guarantee, particularly given the method section's explicit statement that no separate all-missing fallback is implemented?
- What specific architectural capability or theoretical guarantee does S^3M establish beyond the closest existing graph state-space models and dynamic spectral-filtering approaches, and which aspect of the proposed method cannot be obtained through a straightforward extension of the closest prior approach?
- Filter responses are constructed via dense operations and the ablation shows only a marginal AUROC gain over dense routing; does top- routing provide measurable runtime or memory savings, or a consistent generalization advantage beyond qualitative sparsity?
11 comments
Response to Critical Weaknesses 1 (continued)
- Appendix F.2, “Relation between the 72-hour window and onset”: explains why the absence of a verified onset timestamp prevents us from asserting that every input row is pre-onset. Table 9 reports the audit counts: among 6,051 test trajectories, 440 have a positive endpoint label, 334 have the first-positive proxy within the first 72 rows, 106 have it after the window, and 5,611 have no positive proxy in the available record. The appendix explicitly says that this proxy is an analysis device, not a clinical gold standard.
- Appendix F.3, “Onset-censoring sensitivity audit”: reports Table 10 with the locked-checkpoint results: original AUROC 0.9187; post-onset-censored AUROC 0.8905; from-onset-censored AUROC 0.8917. The strict and inclusive masks change 21,582 and 23,631 test observations in 329 and 334 trajectories, respectively. The appendix explains that the score changes quantify temporal sensitivity but do not establish prospective lead time or challenge utility.
- Appendix F.4, “Retrospective benchmark interpretation”: states exactly what the AUROC and controlled interventions support under the common fixed-window protocol and withdraws the prospective/clinical-utility interpretation.
- Appendix F.5, “Prospective early-warning limitation and required follow-up protocol”: states that prospective early-warning performance cannot be established with the retained data and lists the protocol needed to establish it in future work.
Response to Critical Weaknesses 2
Critical Weaknesses 2. The empirical evidence exhibits limited generalizability and does not establish a rigorously comparable state-of-the-art advantage. The evaluation is confined to a single clinical dataset under one fixed patient-level split, lacking validation on independent datasets, multi-center cohorts, or non-clinical multivariate domains, while component ablations rely on only a single training seed. Furthermore, Table 3 juxtaposes results obtained under conflicting evaluation protocols; although these protocol differences are disclosed, the comparison does not establish a directly comparable state-of-the-art claim across structured time series.
**Response:**Thank you for this important comment. We agree that the previous draft did not make the scope of the evidence sufficiently clear. The three baselines in Table 4—TimeCHEAT, iTimER, and NCDE—were run under the same settings as our algorithm, and their results are reported in the original Table 3. We have added the fixed-split results for Raindrop and a paper-based SPECTRA reimplementation to the rightmost column of Table 3, added a new Appendix K synthesizing the evidence and its limits, and expanded the Reproducibility Statement with the NATOPS retraining protocol and fixed-split baseline settings. Appendix J already reports the four added nonclinical datasets, while Appendix H reports the independently retrained NATOPS ablations across seeds 7, 8, and 9. We revised the Discussion and Conclusion to avoid presenting the results as a broad state-of-the-art claim.
See:
“Appendix J reports held-out results for FingerMovements, NATOPS, AtrialFibrillation, and CharacterTrajectories, with one seed-7 evaluation for each dataset. S$^3$M is competitive on accuracy in these supplied comparisons, but the leading method varies by metric and dataset; it does not consistently lead in AUROC, AUPRC, or Macro-F1. These experiments address the absence of nonclinical domains, but their single-seed design does not quantify run-to-run uncertainty. They also do not add an independent clinical cohort or a multicenter validation. The P19 results therefore remain evidence on one retrospective benchmark and one fixed patient-level split, not evidence of clinical transportability.”
“The P19 Table 4 remains a seed-7 study of one unchanged Full forward pass plus six frozen-checkpoint interventions. To add a multi-seed check of independently retrained variants, Appendix H.3 reports fresh training of the Full model and the same six variants on NATOPS for seeds 7, 8, and 9. Their AUROCs (mean ± sample SD) are: Full S$^3$M, 0.729840 ± 0.229729; identity graph, 0.461284 ± 0.079221; no spectral filter, 0.561679 ± 0.140948; dense routing, 0.517975 ± 0.077934; no Laplacian diffusion, 0.537556 ± 0.104235; no state recurrence, 0.692914 ± 0.031617; and unmasked readout, 0.604957 ± 0.136592. The large seed variation, including for Full S$^3$M, warrants a cautious interpretation. These are genuinely retrained multi-seed component results on a separate nonclinical dataset; they do not convert the P19 frozen-intervention study into a P19 multi-seed retraining experiment.”
“The rightmost column of Table 3 now supplies a common-protocol P19 comparison for Raindrop and a paper-based SPECTRA reimplementation, in addition to the already available matched-split results for TimeCHEAT, iTimER, and NCDE. Under the fixed
neighbor_v170/15/15 split (split seed 7) and training seeds 7, 8, and 9, the mean ± sample-SD AUROCs are 92.39 ± 0.48% for S$^3$M, 91.8418 ± 0.2704% for Raindrop, 87.69 ± 7.63% for SPECTRA, 88.10 ± 3.51% for TimeCHEAT, 83.71 ± 0.27% for iTimER, and 90.84 ± 0.51% for NCDE. The published-source column is retained as context but reflects different splits and evaluation protocols. For S$^3$M, seed 7 is the original run and seeds 8 and 9 are validation-selected warm-start continuations from its EMA checkpoint; its reported SD is descriptive, not an independent-seed uncertainty estimate.”
“The matched column permits a more direct comparison among the listed P19 runs than the source-protocol values alone, but it still represents a single dataset and a single fixed split. The four nonclinical results are single-seed, and the multi-seed ablation check is on NATOPS rather than P19. The S$^3$M seed-8 and seed-9 runs are warm-start continuations, so the reported dispersion is descriptive rather than an independent-replication estimate; the small difference from Raindrop is not a significance test. SPECTRA is a paper-based reimplementation and not an evaluation of the authors’ official implementation. Accordingly, we limit the empirical claim to comparative performance under the stated P19 protocol and report the nonclinical and NATOPS findings as supplementary evidence; we do not claim a robust state-of-the-art advantage across structured time series, clinical sites, or datasets.”
Response to Critical Weaknesses 1
Critical Weaknesses 1. The clinical prediction task rests on an insufficiently specified temporal setup, making it unverified as a prospective early-warning system. The model uses full 72-hour patient trajectories to predict an endpoint label, but the manuscript does not adequately specify the temporal relationship between the input observations and sepsis onset. This formulation differs materially from the original PhysioNet 2019 challenge, which evaluates hourly pre-onset predictions using a timing-sensitive clinical utility function. Without establishing whether post-onset measurements can enter the input trajectories, the reported AUROC cannot be reliably interpreted as evidence of prospective early-warning performance. Response. We thank the reviewer for identifying this important limitation. We agree that the original manuscript did not state the temporal scope precisely enough. Following your suggestion, we added a new Appendix F, “Temporal Protocol, Onset Censoring, and Scope of the Clinical Benchmark,” and revised the Results and Reproducibility Statement in the main paper.
The revised text now defines the evaluated task explicitly as
yendpoint=fθ(X1:72,M1:72,Δ1:72),where the first 72 retained rows are mapped to one trajectory-level endpoint label. The model does not emit an hourly sequence of pre-onset risks, the retained files do not provide an adjudicated hourly sepsis-onset timestamp, and the experiment does not compute the original PhysioNet 2019 timing-sensitive clinical utility function. We therefore narrow the claim: the reported AUROC is a retrospective endpoint-classification benchmark under the fixed P19 protocol, not evidence of prospective early-warning performance or bedside clinical utility.
To quantify the temporal sensitivity rather than leave the issue qualitative, we added an inference-only onset-censoring audit using the locked validation-selected seed-7 checkpoint and the same 6,051-example test split. The uncensored score is AUROC 0.9187. Masking observations strictly after the first positive
SepsisLabelproxy gives AUROC 0.8905 (absolute change -0.0282), and masking the proxy hour and all later hours gives AUROC 0.8917 (absolute change -0.0271). The corresponding AUPRC values are 0.6456, 0.5625, and 0.5571. This shows that the benchmark score is sensitive to observations after the proxy. Because the first positive label is only a proxy and not a verified clinical onset timestamp, we describe this as temporal or post-proxy sensitivity evidence, not as prospective validation or an estimate of challenge utility.We retain the retrospective benchmark value: the same fixed window, missingness channels, endpoint definition, split, and locked test partition are supplied to every compared model, so the protocol still supports controlled endpoint-discrimination and mechanism comparisons. We also state the limitation explicitly and list the requirements for a future prospective evaluation: adjudicated onset times, a declared pre-onset prediction origin, removal of all post-onset observations, hourly risk outputs, a challenge-compatible timing-sensitive utility function, and temporal or prospective external validation.
See: