- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper introduces SurvICL, a tabular foundation model specifically designed for survival analysis, along with a substantial effort to combine 103 datasets for foundation model pre-training using 40 datasets and benchmark on 63 held-out datasets. SurvICL is trained on synthetic uncensored targets and evaluated on real targets. SurvICL is able to outperforms other classical and recent survival models on majority of the held-out datasets. The main novelty lies in adapting TabICLv2 for survival analysis and devising a synthetic data prior that results in good performance.
Critical strengths
- The data used for pre-training SurvICL and benchmarking is very large, to the best of my knowledge. The synthetic pre-training data pipeline also allows scaling up pre-training data.
- SurvICL is compared with an exhaustive set of models, comprising classical and recent survival analysis models. SurvICL outperforming the Cox model on small datasets is particularly interesting for me.
Critical weaknesses
- Sole reliance on synthetic targets in pre-training warrants a deeper analysis of bias that could potentially impact the model.
- It is not really clear to me what the comparison is between the distribution of the synthetic and real data, or whether there is any quality control for checking the quality of the synthetic targets, and an analysis of what will happen when the generated synthetic distribution is close to or far from the real target data.
- Missing error bars, CI, or statistical significance for ablation, and no visualization/diagram of the SurvIVL model that can help in understanding.
Questions for authors
- Figure 2 (a) "by generating an event time and censoring time distribution": Can you explain the distribution used?
- How is statistical significance checked/calculated in Sec. 6.4?
- Line 420-424, SurvialBoost vs SurvICL. How is the comparison on the other two metrics (i.e. MAE-Hinge and KM-Calibration)?
4 comments
Table 1 (W1–W2). Outcome distributions of 500 synthetic and 63 real datasets.
| Outcome summary | Synthetic median | Synthetic 1–99% range | Real median |
|---|---|---|---|
| Censoring rate | 40% | 2% – 93% | 61% |
| Skewness of observed durations | 1.10 | −0.9 – 5.1 | 0.79 |
| p90 / p50 duration ratio | 3.2 | 1.1 – 50 | 2.5 |
| Kaplan–Meier survival at last time | 0.00 | 0.00 – 0.75 | 0.39 |
Table 2 (W1–W2). SurvICL vs Cox on held-out datasets inside and outside the synthetic range. Gain = IBS(Cox) − IBS(SurvICL).
| Held-out datasets | SurvICL better on | Median IBS gain ×10⁻³ | Wilcoxon p |
|---|---|---|---|
| Inside the prior range | 37/54 | 1.5 | 0.006 |
| Outside the prior range | 9/9 | 1.5 | 0.004 |
Table 3 (W3). Prior ablation with uncertainty (40 development datasets, 3 seeds). ΔIBS > 0 means worse than the PO prior. All 13 alternatives are in the appendix.
| Prior | ΔIBS ×10⁻³ [95% CI] | Datasets worse (of 40) |
|---|---|---|
| Proportional odds (SurvICL) | reference | — |
| Cox | +3.1 [+0.9, +5.5] | 29 |
| SurvPFN prior | +2.1 [+0.1, +5.2] | 25 |
| SurvivalPFN prior | +2.1 [−0.4, +4.8] | 30 |
| Survival-IC prior | −0.3 [−2.8, +1.6] | 25 |
Table 4 (Q2). SurvICL vs Cox with paired tests. Δ = SurvICL − Cox.
| Held-out datasets | ΔIBS ×10⁻³ [95% CI] | SurvICL better on | Wilcoxon p |
|---|---|---|---|
| All 63 | −3.7 [−7.5, −0.2] | 46/63 | < 0.001 |
| 44 with n < 1,000 | −4.4 [−9.6, +0.7] | 29/44 | 0.022 (Holm 0.043) |
Table 5 (Q3). SurvICL vs SurvivalBoost, paired Wilcoxon tests.
| Metric | Nb wins of SurvICL on all datasets | Nb wins on the 19 datasets with n ≥ 1,000 |
|---|---|---|
| IBS | 40 (p = 0.012) | 7 (p = 0.15) |
| C-index | 46 (p < 0.001) | 12 (p = 0.12) |
| KM-calibration | 55 (p < 0.001) | 17 (p < 0.001) |
| MAE | 20 (p = 0.087) | 3 (p = 0.055) |
Dear Reviewer,
Thank you for your thoughtful review and contribution to the discussion.
As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.
In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.
Thank you again for your time and effort!
Dear authors. Thank you for your response. I confirm that most of my concerns have been addressed. I keep my positive rating.
We thank the reviewer for the positive assessment and for the questions, which helped us clarify the paper. We addressed the three concerns: presentation, the synthetic-to-real gap, and error bars. A revised version of the manuscript is available for the reviewer on Openreview.
Presentation. A model diagram is now referred to in section 3 and can be found in Appendix and the legend of Figure 2 is now clearer. It shows how the prior generates censored datasets, the three stages of the network, the quantile head, and where the loss applies. We also rewrote the prior description as a step-by-step recipe in appendix (Q1 below).
W1–W2. Synthetic targets and the gap to real data. We measured this gap directly. For 500 synthetic datasets and all 63 real ones, we compared summaries of the outcome distribution: censoring rate, skewness, tail heaviness and length of follow-up. Two findings stand out.
The prior covers most real data. 54 of the 63 held-out datasets fall inside the prior's range on every summary. On the 9 held-out datasets outside that range, SurvICL beats Cox on all 9 (Table 2).
SurvICL therefore generalizes beyond its synthetic training data instead of relying on a close match. Two filters provide quality control during generation: we discard datasets with more than 95% censoring, and datasets with no learnable signal. All design choices were made on development datasets, never on held-out ones. SurvICL's predictions also show no systematic bias. We now report D-calibration (Haider et al., 2020) for all 26 methods. D-calibration tests whether the predicted survival probabilities evaluated at the observed times are uniformly distributed, as they should be for a well-calibrated model. Censored samples are spread across bins according to their predicted conditional survival. A Pearson chi-squared test is then applied to the bin counts, following standard practice (e.g., SurvivalPFN). A model is considered D-calibrated when the test does not reject uniformity (p > 0.05). SurvICL is D-calibrated on 60 of 63 datasets, with a median p-value of 0.77. Its D-calibration is significantly better than that of every other survival foundation model, Survival-IC and SurvivalPFN included. Overall, we find that - in line with a growing literature on tabular foundation models - that training on large synthetic datasets ensures good generalization on real-world datasets. We do not find evidence of any forms of biases induced by training on synthetic data.
W3. Error bars. Every ablation arm now uses 3 training seeds, with error bars (revised Fig. 4c) and paired statistical tests (appendix). No prior significantly beats the PO prior, and the Cox prior is significantly worse.
Q1. The distributions in Fig. 2(a). Event and censoring times both follow a proportional-odds model. Unlike Cox, its hazard ratios can vary over time, as often happens in clinical data.
Q2. Statistical significance in Sec. 6.4. We used the Conover–Friedman post-hoc test (https://scikit-posthocs.readthedocs.io/en/latest/generated/scikit_posthocs.posthoc_conover_friedman.html) on ranks across datasets, shown as critical-difference diagrams. A rank test over 26 methods has little power for any single pair, so we added paired Wilcoxon tests with Holm correction and bootstrap confidence intervals. The main result holds: SurvICL beats Cox in IBS across the held-out datasets (p < 0.001).
Q3. SurvICL vs SurvivalBoost on MAE and KM-calibration. The two models have complementary strengths. SurvICL is much better calibrated, with better KM-calibration on 55 of 63 datasets, and also wins on IBS and C-index overall. SurvivalBoost has lower MAE, with its clearest edge on large datasets. This matches our statement that boosting remains strongest above 1,000 samples.
We hope the clearer presentation and the new analyses address the reviewer's concerns. We would be grateful if the reviewer would consider strengthening their support.