acceptodds
Under review as a conference paper at ICLR 2027

SurvICL: a Foundation Model for In-Context Survival Analysis

Abstract

Tabular foundation models (TFMs) now outperform gradient-boosted trees in classification and regression. Yet in survival analysis, which predicts when an event occurs from censored data, often on small clinical cohorts, we find that no existing TFM beats the half-century-old Cox model. We address this issue by introducing SurvICL, a TFM that predicts survival curves in-context. SurvICL is pretrained on purely synthetic datasets generated from a novel flexible proportional-odds prior that outperforms other prior types in our ablation study. Unlike most other survival models, SurvICL is trained with a plain quantile regression loss, which is possible because its synthetic pre-training data generator can provide uncensored event times. Additionally, we introduce the largest survival benchmark to date spanning 103 datasets. On the 63 datasets held out from all design decisions, SurvICL outperforms 20 tree-based, deep, and foundation-model baselines without any tuning: it wins 82.9% of pairwise comparisons in integrated Brier score, against 71.3% for the runner-up, the Cox model, and is the first model to rank above Cox on small datasets. Our code is available at https://anonymous.4open.science/r/survicl-FFC4.

Primary area:
foundation or frontier models, including LLMs
Keywords:
survival analysis, in-context learning, foundation model, tabular data

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_r7b6OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces SurvICL, a tabular foundation model specifically designed for survival analysis, along with a substantial effort to combine 103 datasets for foundation model pre-training using 40 datasets and benchmark on 63 held-out datasets. SurvICL is trained on synthetic uncensored targets and evaluated on real targets. SurvICL is able to outperforms other classical and recent survival models on majority of the held-out datasets. The main novelty lies in adapting TabICLv2 for survival analysis and devising a synthetic data prior that results in good performance.

Critical strengths
  • The data used for pre-training SurvICL and benchmarking is very large, to the best of my knowledge. The synthetic pre-training data pipeline also allows scaling up pre-training data.
  • SurvICL is compared with an exhaustive set of models, comprising classical and recent survival analysis models. SurvICL outperforming the Cox model on small datasets is particularly interesting for me.
Critical weaknesses
  • Sole reliance on synthetic targets in pre-training warrants a deeper analysis of bias that could potentially impact the model.
  • It is not really clear to me what the comparison is between the distribution of the synthetic and real data, or whether there is any quality control for checking the quality of the synthetic targets, and an analysis of what will happen when the generated synthetic distribution is close to or far from the real target data.
  • Missing error bars, CI, or statistical significance for ablation, and no visualization/diagram of the SurvIVL model that can help in understanding.
Questions for authors
  • Figure 2 (a) "by generating an event time and censoring time distribution": Can you explain the distribution used?
  • How is statistical significance checked/calculated in Sec. 6.4?
  • Line 420-424, SurvialBoost vs SurvICL. How is the comparison on the other two metrics (i.e. MAE-Hinge and KM-Calibration)?
4 comments

We thank the reviewer for the positive assessment and for the questions, which helped us clarify the paper. We addressed the three concerns: presentation, the synthetic-to-real gap, and error bars. A revised version of the manuscript is available for the reviewer on Openreview.

Presentation. A model diagram is now referred to in section 3 and can be found in Appendix and the legend of Figure 2 is now clearer. It shows how the prior generates censored datasets, the three stages of the network, the quantile head, and where the loss applies. We also rewrote the prior description as a step-by-step recipe in appendix (Q1 below).

W1–W2. Synthetic targets and the gap to real data. We measured this gap directly. For 500 synthetic datasets and all 63 real ones, we compared summaries of the outcome distribution: censoring rate, skewness, tail heaviness and length of follow-up. Two findings stand out.

The prior covers most real data. 54 of the 63 held-out datasets fall inside the prior's range on every summary. On the 9 held-out datasets outside that range, SurvICL beats Cox on all 9 (Table 2).

SurvICL therefore generalizes beyond its synthetic training data instead of relying on a close match. Two filters provide quality control during generation: we discard datasets with more than 95% censoring, and datasets with no learnable signal. All design choices were made on development datasets, never on held-out ones. SurvICL's predictions also show no systematic bias. We now report D-calibration (Haider et al., 2020) for all 26 methods. D-calibration tests whether the predicted survival probabilities evaluated at the observed times are uniformly distributed, as they should be for a well-calibrated model. Censored samples are spread across bins according to their predicted conditional survival. A Pearson chi-squared test is then applied to the bin counts, following standard practice (e.g., SurvivalPFN). A model is considered D-calibrated when the test does not reject uniformity (p > 0.05). SurvICL is D-calibrated on 60 of 63 datasets, with a median p-value of 0.77. Its D-calibration is significantly better than that of every other survival foundation model, Survival-IC and SurvivalPFN included. Overall, we find that - in line with a growing literature on tabular foundation models - that training on large synthetic datasets ensures good generalization on real-world datasets. We do not find evidence of any forms of biases induced by training on synthetic data.

W3. Error bars. Every ablation arm now uses 3 training seeds, with error bars (revised Fig. 4c) and paired statistical tests (appendix). No prior significantly beats the PO prior, and the Cox prior is significantly worse.

Q1. The distributions in Fig. 2(a). Event and censoring times both follow a proportional-odds model. Unlike Cox, its hazard ratios can vary over time, as often happens in clinical data.

  1. Baseline. A smooth random hazard curve, plus 1 to 5 random peaks for localized risk periods.
  2. Individual effect. A random nonlinear function of the features shifts each person's odds of the event.
  3. Censoring. A second, independent draw of steps 1–2 gives censoring times that depend on the covariates. We observe the earlier of the two times.
  4. Realism. Half of the datasets get an end-of-study cutoff, and times are occasionally rounded to a grid to create ties.

Q2. Statistical significance in Sec. 6.4. We used the Conover–Friedman post-hoc test (https://scikit-posthocs.readthedocs.io/en/latest/generated/scikit_posthocs.posthoc_conover_friedman.html) on ranks across datasets, shown as critical-difference diagrams. A rank test over 26 methods has little power for any single pair, so we added paired Wilcoxon tests with Holm correction and bootstrap confidence intervals. The main result holds: SurvICL beats Cox in IBS across the held-out datasets (p < 0.001).

Q3. SurvICL vs SurvivalBoost on MAE and KM-calibration. The two models have complementary strengths. SurvICL is much better calibrated, with better KM-calibration on 55 of 63 datasets, and also wins on IBS and C-index overall. SurvivalBoost has lower MAE, with its clearest edge on large datasets. This matches our statement that boosting remains strongest above 1,000 samples.

We hope the clearer presentation and the new analyses address the reviewer's concerns. We would be grateful if the reviewer would consider strengthening their support.

Table 1 (W1–W2). Outcome distributions of 500 synthetic and 63 real datasets.

Outcome summarySynthetic medianSynthetic 1–99% rangeReal median
Censoring rate40%2% – 93%61%
Skewness of observed durations1.10−0.9 – 5.10.79
p90 / p50 duration ratio3.21.1 – 502.5
Kaplan–Meier survival at last time0.000.00 – 0.750.39

Table 2 (W1–W2). SurvICL vs Cox on held-out datasets inside and outside the synthetic range. Gain = IBS(Cox) − IBS(SurvICL).

Held-out datasetsSurvICL better onMedian IBS gain ×10⁻³Wilcoxon p
Inside the prior range37/541.50.006
Outside the prior range9/91.50.004

Table 3 (W3). Prior ablation with uncertainty (40 development datasets, 3 seeds). ΔIBS > 0 means worse than the PO prior. All 13 alternatives are in the appendix.

PriorΔIBS ×10⁻³ [95% CI]Datasets worse (of 40)
Proportional odds (SurvICL)reference—
Cox+3.1 [+0.9, +5.5]29
SurvPFN prior+2.1 [+0.1, +5.2]25
SurvivalPFN prior+2.1 [−0.4, +4.8]30
Survival-IC prior−0.3 [−2.8, +1.6]25

Table 4 (Q2). SurvICL vs Cox with paired tests. Δ = SurvICL − Cox.

Held-out datasetsΔIBS ×10⁻³ [95% CI]SurvICL better onWilcoxon p
All 63−3.7 [−7.5, −0.2]46/63< 0.001
44 with n < 1,000−4.4 [−9.6, +0.7]29/440.022 (Holm 0.043)

Table 5 (Q3). SurvICL vs SurvivalBoost, paired Wilcoxon tests.

MetricNb wins of SurvICL on all datasetsNb wins on the 19 datasets with n ≥ 1,000
IBS40 (p = 0.012)7 (p = 0.15)
C-index46 (p < 0.001)12 (p = 0.12)
KM-calibration55 (p < 0.001)17 (p < 0.001)
MAE20 (p = 0.087)3 (p = 0.055)

Dear Reviewer,

Thank you for your thoughtful review and contribution to the discussion.

As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.

In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.

Thank you again for your time and effort!

Dear authors. Thank you for your response. I confirm that most of my concerns have been addressed. I keep my positive rating.

Reviewer_5mhbOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces SurvICL, an in-context foundation model for survival analysis built on TabICLv2. The model is pretrained on synthetic survival datasets generated from a flexible proportional-odds prior and predicts event-time quantiles using a multi-quantile pinball loss with latent uncensored event-time supervision. The predicted quantiles are sorted and linearly interpolated to construct survival curves.

The authors evaluate SurvICL on 103 survival datasets, including 40 development datasets and 63 held-out datasets, against 20 classical, tree-based, neural, and foundation-model baselines. SurvICL achieves strong overall performance, including an IBS pairwise win rate of 82.9% compared with 71.3% for Cox regression.

However, the paper has two major weaknesses: (1) insufficiently established methodological novelty and inadequate validation of the proposed quantile-based formulation, and (2) incomplete empirical evidence supporting the claimed advantages in small-data survival prediction, calibration, and generalization across training and inference settings.

Critical strengths

1. A practically relevant application of in-context learning to survival analysis.

The paper addresses an important problem: existing survival foundation models often struggle to outperform classical methods such as Cox regression, particularly on small clinical datasets. Enabling survival-curve prediction without dataset-specific fine-tuning or hyperparameter optimization is practically appealing.

2. A comprehensive benchmark and informative investigation of synthetic survival priors.

The evaluation covers 103 datasets, with 40 development datasets separated from 63 held-out datasets, and includes a broad collection of classical, tree-based, deep learning, and foundation-model baselines. The overall IBS and C-index results are encouraging.

Critical weaknesses

1. Limited methodological novelty and insufficient validation of the pinball-loss-based formulation.

SurvICL largely combines existing components: the TabICLv2 backbone, multi-quantile regression using the pinball loss (also known as the check loss), synthetic-data pretraining, and proportional-odds survival modeling. Latent event-time supervision has also been explored in SurvivalPFN. The main methodological contribution therefore appears to be the specific synthetic prior and its integration with an existing architecture.

More importantly, the paper identifies the pinball loss as a major design choice (lines 214–215), yet neither isolates its contribution through a controlled loss ablation nor compares against directly relevant survival methods. These include CQRNN [1] and DeepQuantreg [2], which employ censoring-aware variants of the pinball loss, as well as ALD [3] and ICALD [4], which connect quantile modeling with probabilistic survival prediction and calibration. The authors explicitly cite CQRNN but omit it from the benchmark.

The prior ablations also provide limited evidence for the final model: they use a smaller architecture trained for 10K rather than 200K steps, and report only per-dataset min–max normalized IBS scores, which do not quantify absolute performance improvements.

Without these comparisons and controlled ablations, the paper does not convincingly establish the independent methodological contribution of its pinball-loss-based formulation or synthetic prior.

2. Insufficient empirical evidence for the claimed small-data and calibration advantages.

First, the claimed advantage over Cox on small datasets is not statistically significant. Section 6.4 reports IBS pairwise win rates of 84.9% for SurvICL versus 77.2% for Cox, but explicitly acknowledges that the difference is not statistically significant. Since outperforming Cox on small clinical datasets is a central motivation, the reported ranking advantage does not yet establish a statistically reliable improvement in the primary target setting.

Second, the calibration evaluation is incomplete compared with directly competing survival foundation models. The paper primarily evaluates marginal KM-calibration, which does not establish distributional or conditional calibration. However, both SurvivalPFN (Qi et al., 2026) and Survival-IC (Seletkov et al., 2026) report D-calibration, despite being direct baselines in this paper. Survival-IC further demonstrates that D-calibration can reveal substantial miscalibration before post-hoc correction. SurvICL does not report this metric and is not the strongest method even on its reported KM-calibration evaluation.

The empirical conclusions are further complicated by unexplained differences between pretraining and inference context sizes and time-normalization procedures, as detailed in the questions below.

Together, these issues weaken the evidence that SurvICL provides statistically reliable and well-calibrated survival predictions in the setting it primarily targets.


References

[1] Pearce T, Jeong J H, Zhu J. Censored quantile regression neural networks for distribution-free survival analysis[J]. Advances in neural information processing systems, 2022, 35: 7450-7461.

[2] Jia Y, Jeong J H. Deep learning for quantile regression under right censoring: DeepQuantreg[J]. Computational Statistics & Data Analysis, 2022, 165: 107323.

[3] Sheng D, Henao R. Learning survival distributions with the asymmetric Laplace distribution[J]. Proceedings of machine learning research, 2025, 267: 54772.

[4] Sheng D, Henao R. Learning Survival Distributions with Individually Calibrated Asymmetric Laplace Distribution[C]//The Fourteenth International Conference on Learning Representations. 2026.

Questions for authors

Q1. Missing baselines and contribution of the pinball loss.

SurvICL uses the pinball loss to predict event-time quantiles. Why are closely related methods such as CQRNN [1], DeepQuantreg [2], ALD [3] and ICALD [4] excluded, and can the authors compare them under the same evaluation protocol?

Can the authors isolate the contribution of the pinball loss through a controlled ablation with the backbone and prior fixed, and report raw paired IBS differences and uncertainty estimates for the prior ablations rather than only min–max normalized scores?

Q2. Statistical significance and calibration.

Given that the small-data improvements over Cox are not statistically significant, can the authors report paired absolute IBS improvements and confidence intervals against Cox and other strong baselines on this subset?

D-calibration is one of the most established and widely used metrics for evaluating distributional calibration in survival analysis. Both SurvivalPFN (Qi et al., 2026) and Survival-IC (Seletkov et al., 2026) report D-calibration, yet SurvICL evaluates calibration primarily through marginal KM-calibration. Why is this standard metric omitted? Can the authors report D-calibration for SurvICL and these direct baselines under the same evaluation protocol, together with risk-stratified conditional calibration analyses?

Q3. Pretraining–inference consistency.

SurvICL is pretrained with at most approximately 922 context observations but evaluated with up to 4,096. How are these larger contexts processed, and how does performance change with context length?

Appendix A.1 states that time normalization uses both context and query observations during pretraining but only context observations during inference. Why do these procedures differ, and what is the effect of using consistent context-only normalization?

Q4. Continuous quantile representation and feature scalability.

SurvICL predicts 999 discrete quantiles followed by sorting and linear interpolation. What is the demonstrated advantage of this representation over interpolated predictions from existing survival models, and how frequently do quantile crossings occur before sorting?

Finally, how does SurvICL handle datasets with substantially more than the 100 features used during pretraining, and are competing foundation-model baselines given comparable effective feature access?

4 comments

We thank the reviewer for their constructive review. We ran every experiment requested in Q1–Q4 and have incorporated them in the revised version of the manuscript which we have uploaded on Openreview.

W1/Q1. Novelty and the role of the loss.

As noted by the reviewer, existing survival foundation models trail classical methods. SurvICL is the first to beat Cox across held-out datasets. Our contribution is showing which innovations in the model’s construction drive this performance. The new controlled experiments show that both of our ingredients are necessary:

  • Without our prior and target, the same TabICLv2 backbone (TFM4Survival on TabICLv2) loses to SurvICL on 59 of 63 datasets.

  • Without latent-time supervision, results get worse. We ran a counterfactual experiment in which we used CQRNN's censored pinball loss in SurvICL, keeping everything else fixed. It is worse on 28 of 40 datasets. The losses of Survival-IC, SurvPFN and SurvivalPFN are also all significantly worse. Censoring-aware losses throw away information that the synthetic prior provides for free: the uncensored event time. Those results can be seen in Figure 4c of the revised version.

  • Without the PO prior, results get worse too. With 3 seeds, the Cox prior is worse on 29 of 40 datasets, and none of the 13 alternative priors significantly beats PO.

We also found that a histogram head trained on latent times matches the pinball loss. We therefore revise lines 214–215. To use latent-time supervision, we therefore use simply the pinball loss.

Q1. Quantile baselines. We added CQRNN, DeepQuantreg, ALD and ICALD, re-implemented from the authors' code, with our tuning protocol. SurvICL beats all four on IBS, C-index, KM-calibration and D-calibration. The strongest of them, CQRNN, loses to SurvICL on 46 of 63 datasets. All baselines have been added to the manuscript.

Q1. Raw differences and uncertainty. The revised Fig. 4c shows 3 seeds with error bars, and now includes the loss ablation. The appendix gives raw paired IBS differences with 95% confidence intervals for every ablation.

W2/Q2. Small data. On the 44 held-out datasets with fewer than 1,000 samples, SurvICL has a lower IBS than Cox on 29 (paired Wilcoxon p = 0.022; p = 0.043 after Holm correction over the four strongest baselines). We now report these paired numbers in Section 6.4 and in the appendix.

Q2. D-calibration. We now report D-calibration (Haider et al., 2020) for all 26 methods. D-calibration tests whether the predicted survival probabilities evaluated at the observed times are uniformly distributed, as they should be for a well-calibrated model. Censored samples are spread across bins according to their predicted conditional survival. A Pearson chi-squared test is then applied to the bin counts, following standard practice (e.g., SurvivalPFN). A model is considered D-calibrated when the test does not reject uniformity (p > 0.05). SurvICL is D-calibrated on 60 of 63 datasets. Its D-calibration is significantly better than that of every other survival foundation model, Survival-IC and SurvivalPFN included.

We also report 1-calibration (Haider et al., 2020) at the median observed time of each dataset. Patients are grouped into deciles of predicted survival, and a Hosmer–Lemeshow-style test compares the predicted survival in each group with the Kaplan–Meier estimate. SurvICL passes this test (p > 0.05) on 97% of datasets. Finally, we split patients into tertiles of predicted risk and compute, within each tertile, the KM-calibration: the normalized integrated squared difference between the mean predicted survival curve and the Kaplan–Meier curve. Averaged over the three tertiles, SurvICL's median is 0.0126, comparable to Cox (0.0114), RSF (0.0115) and SurvivalBoost (0.0129). We also understand the point of the reviewer to ask for more metrics to evaluate the different aspects of the model. So, we have also added two other metrics, the CRPS and log-score in Appendix E.1.6 in the new revised version. SurvICL has the highest winrate on both metrics.

Q3. Pretraining–inference consistency.

  • Context length. The model treats rows as a set, with no positional encoding. Its attention uses scalable softmax, which is designed to generalize to longer inputs. Performance keeps improving past the 922 rows seen in pretraining. On the 16 datasets with more training rows than that, using 4,096 rows instead of 922 improves IBS on 13 and C-index on all 16. Like other tabular foundation models, it extrapolates well beyond the dataset sizes seen in pretraining. We also tried continued pretraining on larger datasets, but this did not improve results further.

  • Normalization. We retrained with context-only normalization (3 seeds) and performance is unchanged. The query set contributed only the time unit i.e. a single number.

Q4. Quantiles and feature width.

  • Crossings are rare and tiny: on average, sorting moves a quantile by 0.03% of its range.
  • Wide data. The column-wise embedder has no width limit, so SurvICL sees every column. Reducing dimension with PCA does not improve it. Survival-IC and TFM4Survival see the same columns, as now stated in the paper.

These results answer the reviewer's requests one by one. We hope they resolve the soundness concerns, and would be grateful if the reviewer would reconsider the score.

Table 1 (Q1). Quantile-based baselines on the 63 held-out datasets. Δ = SurvICL − baseline; ΔIBS < 0 and ΔC-index > 0 favor SurvICL.

BaselineΔIBS ×10⁻³ [95% CI]SurvICL better onΔC-index ×10⁻² [95% CI]Mean D-cal p
SurvICL———0.74
CQRNN−8.8 [−13.9, −4.1]46/63+1.5 [+0.5, +2.6]0.52
DeepQuantreg−26.6 [−33.0, −20.9]59/63+10.4 [+8.1, +13.0]0.27
ALD−10.8 [−17.4, −5.0]51/63+1.6 [+0.7, +2.5]0.54
ICALD−9.7 [−16.5, −4.1]47/63+1.7 [+0.7, +2.9]0.55

Table 2 (Q1, Q3). Training objective and time normalization, with backbone and prior fixed (40 development datasets, 3 seeds). ΔIBS > 0 means worse than SurvICL's objective.

Training objectiveΔIBS ×10⁻³ [95% CI]Datasets worse (of 40)
Pinball loss on latent times (SurvICL)reference—
Histogram NLL on latent times−0.5 [−2.8, +1.5]22
Censored pinball loss (CQRNN)+2.2 [+0.6, +4.1]28
Observed times + censoring head (SurvivalPFN)+1.7 [+0.2, +3.3]25
IPCW + ranking loss (SurvPFN)+3.1 [+0.8, +5.6]29
DeepHit loss (Survival-IC)+4.6 [+2.2, +7.4]30
Context-only time normalization−1.1 [−2.9, +0.0]18

Table 3 (Q1). Raw paired IBS differences for the prior ablation, same setting. All 13 priors are in the appendix.

PriorΔIBS ×10⁻³ [95% CI]Datasets worse (of 40)
Proportional odds (SurvICL)reference—
Cox+3.1 [+0.9, +5.5]29
ACH+2.2 [+0.1, +4.3]31
SurvPFN prior+2.1 [+0.1, +5.2]25
SurvivalPFN prior+2.1 [−0.4, +4.8]30
Grand mixture+0.4 [−1.1, +1.7]22
Survival-IC prior−0.3 [−2.8, +1.6]25

Table 4 (Q2). Small data: 44 held-out datasets with n < 1,000. Holm correction over the four baselines shown. On all 63 datasets, SurvICL vs Cox gives ΔIBS −3.7×10⁻³ [−7.5, −0.2], better on 46/63, p < 0.001.

BaselineΔIBS ×10⁻³ [95% CI]SurvICL better onHolm p
Cox−4.4 [−9.6, +0.7]29/440.043
RSF−6.9 [−12.3, −2.4]30/440.023
SurvivalPFN−4.6 [−9.6, −0.6]27/410.043
SurvivalBoost−7.3 [−11.3, −3.8]33/44< 0.001

Table 5 (Q2). D-calibration on the 63 held-out datasets. Paired Wilcoxon test on the per-dataset p-values.

ModelMedian D-cal pPasses (p > 0.05)Paired p vs SurvICL
RSF0.7961/630.77
DeepSurv0.7960/620.50
Kaplan–Meier0.7862/630.85
SurvICL0.7760/63—
SurvPFN0.7656/630.002
Cox0.7461/630.13
SurvivalPFN0.7055/60< 0.001
Survival-IC0.5553/63< 0.001
TFM4Survival (TabICLv2)0.3443/63< 0.001

Table 6 (Q3). Context length at inference. The pretraining maximum is 922 rows.

Context rowsMean IBS, all 63 datasetsMean IBS, 16 datasets with > 922 training rows
2560.14560.1557
5120.14450.1521
9220.14380.1500
2,0480.14290.1466
4,096 (paper setting)0.14260.1454
Full training set0.14240.1446

Thank you for the detailed rebuttal and additional experiments. However, I remain concerned about the limited methodological novelty and the modest practical improvements over strong baselines. Therefore, I have decided to maintain my rating of 2.

The components the reviewer lists were all available before us. SurvivalPFN, SurvPFN, Survival-IC and TFM4Survival were built from similar parts, and, as the reviewer noted in the review, they struggle to outperform Cox. Our contribution is to identify which choices make the difference. Our controlled experiments show that the choices made by earlier work are measurably worse when swapped into SurvICL (Table A).

Table A. Replacing one SurvICL component with the choice made in earlier work. ΔIBS > 0 means worse than SurvICL. Rows 1–5: controlled ablations with backbone and everything else fixed (40 development datasets, 3 seeds). Last row: 63 held-out datasets.

ComponentChoice in earlier workΔIBS ×10⁻³ [95% CI]Datasets worse
ObjectiveDeepHit loss (Survival-IC)+5.6 [+2.8, +8.6]33/40
ObjectiveIPCW + ranking loss (SurvPFN)+4.1 [+1.5, +7.0]31/40
ObjectiveCensored pinball loss (CQRNN)+3.1 [+0.9, +5.9]28/40
PriorCox+3.1 [+0.9, +5.5]29/40
PriorSurvPFN prior+2.1 [+0.1, +5.2]25/40
Survival recipe, same TabICLv2 backboneReduction to classification (TFM4Survival)+25.5 [+19.4, +32.5]59/63

We believe these findings are new and useful to anyone building survival foundation models:

  • Latent-time supervision matters; the form of the loss does not. As the reviewer noted, SurvivalPFN also trains on latent times. Survival-IC, SurvPFN and CQRNN instead use censoring-aware losses on observed times, and each of these losses is significantly worse in SurvICL. With latent targets, a histogram likelihood ties with the pinball loss (+0.5 [−1.7, +2.5]), and so does adding a censoring-time target as SurvivalPFN does (+0.6 [−1.5, +2.7]). To our knowledge, this is the first comparison of the objectives of existing survival foundation models with backbone and prior held fixed.
  • The structure of the prior matters. The Cox prior, which encodes the default assumption in clinical practice, is significantly worse, and none of 13 alternative priors beats proportional odds.
  • The backbone is not the bottleneck. On the same TabICLv2 backbone, the survival recipe alone accounts for a gap seven times larger than SurvICL's margin over Cox.

Beyond pretraining. A survival simulator that transfers to real data is useful in its own right. Simulation studies of survival methods usually draw times from Cox or Weibull models (Bender et al., 2005), and in our ablations these generators transfer worse to real data than the PO prior. 54 of the 63 held-out datasets fall within the PO prior's range on every outcome summary we measured. Unlike real data, the prior also provides each subject's latent event and censoring times, so it can be used to benchmark survival methods and evaluation metrics against ground truth. It is part of our code release.

Correction to our first response. Table 2 of our response attributed the "observed times + censoring head" objective to SurvivalPFN. That arm reproduces the observed-time training mode of SurvivalPFN's released code, but the SurvivalPFN paper trains on latent times. The matching arm is "latent times + censoring head", which ties with our objective. Our statement that SurvivalPFN's loss is significantly worse was therefore inaccurate. The revised paper describes both modes.

2. Size of the improvement

On the 63 held-out datasets:

  • SurvICL has the best IBS of all 26 methods and beats each of the 25 baselines one-on-one (paired Wilcoxon, Holm-adjusted p ≤ 0.012). On C-index, it is tied with Cox and RSF and beats the other 23 baselines.
  • The strongest baselines are close together. Among those that run on all 63 datasets, Cox (0.1464), SurvivalBoost (0.1464), RSF (0.1478) and GBS (0.1478) span 1.5×10⁻³ IBS. SurvICL (0.1426) is 3.7×10⁻³ ahead of the best of them, 2.5 times that spread.
  • Kaplan–Meier ignores covariates. Cox improves on it by 17.8×10⁻³ IBS, and SurvICL by 21.6×10⁻³, a 21% larger gain than the best tuned baseline.
  • Every baseline gets a hyperparameter search on each dataset. SurvICL needs no tuning, makes predictions in a single forward pass, and does not trade calibration for discrimination: its D-calibration is on par with Cox and RSF, and its KM-calibration beats SurvivalBoost on 55 of 63 datasets.

We hope that this controlled evidence about what makes in-context survival prediction work, together with consistent gains over every baseline, addresses the remaining concerns. The validation concerns behind the original soundness assessment are now resolved, so we would be grateful if the reviewer would reconsider their scores.

Reviewer_6FoiOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SurvICL, a tabular foundation model for survival analysis that predicts survival distributions in context without dataset-specific parameter updates. It combines a TabICLv2 backbone with a proportional-odds synthetic data prior and quantile regression supervised by latent uncensored event times. The evaluation covers 63 held-out datasets and 20 baselines, reporting strong overall performance.

Critical strengths

1.SurvICL addresses the practical challenge of obtaining accurate survival predictions on small censored datasets without repeatedly fitting and tuning a model. Its training formulation uses censored observations as context and uncensored query targets supplied by the synthetic generator for supervision. This enables a standard quantile regression objective and provides a useful alternative to discretized survival prediction.

2.The empirical evaluation covers 63 held-out datasets that are kept separate from 40 development datasets. It compares multiple model families across four complementary metrics. Evaluation over ten splits strengthens the evidence for the overall IBS improvement. Detailed descriptions of the architecture and training settings also support reproducibility.

Critical weaknesses

1.The evidence does not sufficiently establish which proposed components explain the final model’s gains. The paper identifies the quantile regression objective and proportional-odds prior as central design choices. However, the backbone and quantile objective are inherited from TabICLv2, and latent-time supervision is also used by SurvivalPFN. The contribution therefore depends substantially on demonstrating the benefit of their particular combination. Appendix C provides useful controlled prior comparisons, but these use a smaller backbone trained for 10,000 steps, whereas the final model uses a larger backbone trained for 200,000 steps. The reported ablations also do not directly compare quantile regression against alternative survival objectives under an otherwise matched configuration. The experiments support the effectiveness of the complete system but do not sufficiently establish the contribution of each key design choice. Controlled comparisons of alternative priors and training objectives using the final backbone and training budget would help clarify the source of the gains.

2.Time normalization differs between pretraining and inference without a corresponding sensitivity analysis. Appendix A.1 states that pretraining computes the mean observed duration from both context and query sets. Inference uses the context set alone. The training normalization therefore incorporates query outcomes unavailable at inference. An ablation using context-only normalization throughout training would help assess its impact on performance.

Questions for authors

See critical weaknesses.

4 comments

We thank the reviewer for their thorough review. In short, latent-time supervision and the PO prior each matter, the backbone alone does not explain the gains, and context-only time scaling does not change the results.

We answer each of the weaknesses in detail:

W1. Which components explain the gains?

Following the reviewer’s suggestions, we have run different experiments, and the results have been added to the final manuscript (we have uploaded a revised version with the changes).

The backbone is not the sole driver of the strong gains. We used TFM4Survival in combination with the TabICLv2 backbone. It loses to SurvICL on 59 of 63 held-out datasets, and does slightly worse than with the older TabICL v1.1. A strong tabular backbone does not by itself produce good survival predictions.

The training objective matters. With backbone, prior and budget fixed, we swapped our objective for the published alternatives:

  • the censored pinball loss of CQRNN;
  • the losses of Survival-IC (DeepHit), SurvPFN (IPCW plus ranking) and SurvivalPFN (observed times plus a censoring head).

All four are significantly worse; each loses on 25 to 30 of the 40 datasets. This is also what separates SurvICL from SurvivalPFN: SurvivalPFN's objective, on observed times with an added censoring target, is worse than ours. A histogram head trained on latent times does as well as the pinball loss. We therefore now present latent-time supervision, not the pinball loss itself, as the key design choice.

The prior matters. With 3 seeds, none of the 13 alternative priors we tested beats PO. The Cox prior, the most common choice, is worse on 29 of 40 datasets.

The choices hold at full scale. Due to time constraints, we trained the final and the ablation-control configurations on the final backbone for 100k steps each. The final configuration wins on 40 of 63 held-out datasets. Training the final configuration for 200k instead of 100k steps changes the IBS by only −0.0001, so the 100k-step comparison reflects the final training budget. Finally, we want to highlight that in most articles about TFMs, the ablations are made on smaller settings due to time / budget constraints.

W2. Time normalization.

We ran the following ablation: context-only normalization throughout pretraining, with 3 seeds. Performance is unchanged on every metric we track (IBS, C-index, D-calibration): all confidence intervals include zero, and IBS is, if anything, slightly better (Table 1).

We believe these experiments resolve both weaknesses, which the reviewer rated as fixable. If so, we would be grateful if the reviewer would consider raising the score.

Table 1. Training objective and time normalization, with backbone and prior fixed (40 development datasets, 3 seeds). ΔIBS > 0 means worse than SurvICL's objective.

Training objectiveΔIBS ×10⁻³ [95% CI]Datasets worse (of 40)
Pinball loss on latent times (SurvICL)reference—
Histogram NLL on latent times−0.5 [−2.8, +1.5]22
Censored pinball loss (CQRNN)+2.2 [+0.6, +4.1]28
Observed times + censoring head (SurvivalPFN)+1.7 [+0.2, +3.3]25
IPCW + ranking loss (SurvPFN)+3.1 [+0.8, +5.6]29
DeepHit loss (Survival-IC)+4.6 [+2.2, +7.4]30
Context-only time normalization−1.1 [−2.9, +0.0]18

Table 2. Prior, same setting. All 13 alternative priors are in the appendix.

PriorΔIBS ×10⁻³ [95% CI]Datasets worse (of 40)
Proportional odds (SurvICL)reference—
Cox+3.1 [+0.9, +5.5]29
SurvPFN prior+2.1 [+0.1, +5.2]25
SurvivalPFN prior+2.1 [−0.4, +4.8]30
Survival-IC prior−0.3 [−2.8, +1.6]25

Table 3. Backbone and scale (63 held-out datasets).

ComparisonΔIBS ×10⁻³ [95% CI]First model better on
SurvICL vs TFM4Survival on the same TabICLv2 backbone−25.5 [−32.5, −19.3]59/63
Final vs ablation-control configuration (100k steps each)−0.9 [−1.8, −0.3]40/63
200k vs 100k training steps−0.1 [−0.3, +0.2]32/63

Thank you for the detailed response. The additional experiments substantially strengthen the paper. In particular, the matched comparisons with CQRNN, DeepQuantreg, ALD, and ICALD, the D-calibration analysis, the context-length study, and the context-only normalization ablation address most of my concrete questions.

However, I still consider W1 only partially resolved. The additional comparison using the final backbone at 100K steps provides useful evidence, but the response does not specify how the ablation control differs from the final SurvICL configuration. Further clarification is needed on how the two configurations differ, as well as on the backbone architecture and training budget used for each ablation. While the final configuration performs similarly at 100K and 200K steps, this comparison does not establish whether the individual contributions of the prior and training objective remain consistent at the final training scale.

Overall, the response addresses many of my concerns. However, further clarification is needed on the experimental configurations and the evidence supporting the separate contributions of the prior and training objective. I would consider raising my score from 2 to 3 if these points are adequately addressed.

AuthorsOpenReview ↗

Even a 10k, the ablation is SOTA; we're running the full 200k

We thank the reviewer for engaging with our response, and we are glad the new experiments address most of the concrete questions. Below we clarify how the configurations differ, why the ablations use a smaller model, and how that model compares with the baselines.

1. Ablation control vs final configuration The configurations of the ablations and the final model are described below. The small architecture is the one used in most of the ablations, while the Cox vs PO ablation is the one on the same architecture as our final model.

Table A. Configurations.

Ablations (small architecture)Cox vs PO ablation (Final architecture, Table 3 of our former answer)Final (SurvICL)
Backbone9.9M params (2 column / 2 row / 4 ICL blocks, 4 heads)27.6M params (3 / 3 / 12 blocks, 8 heads)Same as Table 3 (28.6M with the 999-quantile head)
Training budget10k steps × 32 = 0.32M datasets100k × 64 = 6.4M datasets100k × 64, then 100k more (12.8M total) datasets
Synthetic datasets≤ 512 rows, ≤ 32 features≤ 1,024 rows, ≤ 100 featuresSame as Table 3
Prior / objectiveProportional odds / pinball on latent timesSameSame
Quantiles / draws / discretization99 / 1 / off999 / 5 / on999 / 5 / on
Seeds311

2. The ablation control already outperforms the baselines

The small model is not a weak reference. Table B compares the ablation control and SurvICL with the 25 baselines on the 63 held-out datasets.

Table B. Pairwise win rate (%) against all baselines on the 63 held-out datasets (rank in parentheses).

Ablation control (10k steps, 9.9M)SurvICL (final)Best baseline
IBS, all datasets80.4 (1st)83.3 (1st)Cox, 73.0
IBS, n < 1,000 (44 datasets)86.1 (1st)85.9 (1st)Cox, 79.0
IBS, n ≥ 1,000 (19 datasets)67.4 (5th)77.3 (3rd)SurvivalBoost, 82.8
C-index72.0 (1st)78.7 (1st)RSF, 70.7

Each model is ranked with the 25 baselines only.

The ablation control ranks first on IBS and beats each of the 25 baselines one-on-one (Wilcoxon, Holm-adjusted p ≤ 0.04). We took it as the starting point and scaled up the backbone, the training budget and the size of the synthetic datasets. The ablation control was trained on datasets with at most 512 rows, and scaling up mainly helps on the larger datasets (IBS win rate 67.4% → 77.3%) and on the C-index (72.0% → 78.7%). On the 44 datasets with fewer than 1,000 samples, the two models are tied on IBS (SurvICL better on 20/44, p = 0.22).

3. Why the ablations use a smaller model

As is common practice when pretraining foundation models, we ran the ablations on a smaller backbone and training budget because of computational constraints. Each ablation is a full pretraining run, and every arm is trained with 3 seeds (except the parametric-laws prior, which has 1). This gives nearly 80 pretraining runs. At full scale, each run would use 20 times more synthetic datasets, each with up to twice as many rows and three times as many features, on a backbone nearly three times larger. We are currently running the experiment for 200k steps. However, due to time constraints and limited GPU availability, we will not be able to report the final results within the rebuttal period. Nevertheless, we will include these results in the final version of the manuscript.

In short, the prior and the objective were measured at ablation scale, on a configuration that already outperforms all baselines. The backbone, the output settings and the budget were measured at full scale (Table 3 of our response). "The choices hold at full scale" referred to the output settings only. The revised paper will include Table A and state the scale of each comparison.

Thank you for the clarification. The additional details resolve my concerns about the experimental configurations and make the ablation results more convincing. The strong performance of the ablation control further supports the validity of the analysis. I will raise my score from 2 to 3.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal period highlights

We thank the reviewers for their constructive feedback and for engaging during the rebuttal period. All three reviewers praised the scale of the evaluation (103 datasets, with 63 held out), and thanks to their comments the rebuttal has strengthened the paper with new experiments, addressing comments about ablations, calibration, and generalization. All new results are included in the revised manuscript. We briefly detail the major changes in the revised version.

Stronger evidence of solid lead on benchmark (63 held-out datasets, 26 methods)

To answer 5mhb, we added 4 new baselines, CQRNN, DeepQuantreg, ALD and ICALD, and 5 new metrics strengthening our lead:

  • Best IBS among all 26 methods, significantly beating every baseline, including the new ones (Holm p ≤ 0.012). On C-index, SurvICL ties with Cox and RSF and beats the other 23. (5mhb, r7b6)
  • First survival foundation model to significantly outperform Cox: 46/63 wins (p < 0.001), including on small datasets (29/44, Holm p = 0.043).
  • Added D-calibration, 1-calibration, risk-stratified calibration, CRPS and log-score (an ask from 5mhb, r7b6). SurvICL passes D-calibration on 60/63 datasets, matching Cox and RSF and outperforming other survival foundation models.

Better-controlled ablations

We have added new ablations of the loss, the backbone and the time normalization, with everything else held fixed to answer concerns raised by reviewers 6Foi and 5mhb. Every arm now uses 3 seeds and is reported as raw paired IBS differences with 95% CIs instead of min–max normalized scores, answering remarks by 5mhb and r7b6:

Losses: Censoring-aware losses from Survival-IC, SurvPFN and CQRNN significantly worsen IBS (+5.6, +4.1, +3.1 × 10⁻³). Latent-time objectives perform similarly. This means that latent-time supervision is one key ingredient in our model’s performance.

Prior: None of 13 alternatives beats proportional odds (6Foi, 5mhb, r7b6) showing that our prior works best on the validation datasets. Cox is worse on 29/40 datasets.

Backbone: TFM4Survival with the same TabICLv2 backbone loses on 59/63 datasets. (6Foi)

Robustness: Context-only normalization has no significant effect (6Foi, 5mhb); 200k versus 100k training steps yields no significant improvement (6Foi).

Generalization (r7b6): SurvICL beats Cox on all 9 held-out datasets outside the synthetic prior's measured range against 500 generated datasets in terms of censoring rate and proportion of survival at last time.

Quantiles and wide data (5mhb): Quantile crossings are rare and tiny: sorting moves a quantile by only 0.03% of its range on average. SurvICL has no limit on the number of features and uses every column of datasets with more than 100 features, and PCA does not improve it.

To summarize, SurvICL is the first survival foundation model that outperforms strong classical baselines without dataset-specific tuning even in the small samples setting, while maintaining strong calibration. Our work provides an extensive set of ablations that identify latent time supervision and our tailored prior as key drivers of the model’s performance. We believe the new evidence addresses the reviewers' main concerns and establishes both the methodological and practical value of SurvICL.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.