acceptodds
Under review as a conference paper at ICLR 2027

Amortized Survival Distribution Estimation via In-Context Learning

Abstract

Survival analysis models time-to-event (TTE) outcomes in the presence of *censoring*. Most survival models require dataset-specific fitting, whereas prior-fitted networks (PFNs) enable in-context prediction on new datasets without gradient-based adaptation. However, existing survival PFNs provide limited theoretical guidance for the design of their pretraining objective, *context-size* distribution, and synthetic *prior*. We introduce TTE-PFN, an amortized in-context estimator that maps a labeled *right-censored context* and unlabeled *query* covariates to predictive *event-* and *censoring-time distributions* in a single forward pass. TTE-PFN combines *oracle-distillation* supervision from *analytic* conditional distributions, randomized *context sizes*, and a diverse *prior* over survival data-generating processes. We first show that the *oracle-distillation* and matched *sampled-label* objectives share the same *posterior*-predictive population minimizer, but the latter incurs additional positive-semidefinite gradient covariance, which *oracle distillation* removes to facilitate faster and more stable optimization. We further establish *posterior*-predictive optimality across all represented *context sizes* and derive an excess-risk bound decomposing predictive error into *amortization*, *prior coverage*, and *posterior concentration*. Across 115 survival datasets and seven metrics that span *predictive accuracy*, *concordance*, and *calibration*, TTE-PFN achieves the best aggregate rank against 28 competing methods and ranks first on the *predictive accuracy* metrics. TTE-PFN also has the lowest runtime in our evaluation protocol, requiring only seconds to predict individuals.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Survival Analysis, Prior-Fitted Networks, In-Context Learning, Amortized Bayesian Inference, Tabular Foundation Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LRzhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TTE-PFN, a pretrained network that uses labeled right-censored observations to predict survival distributions for new patients without further gradient-based training. Compared with SurvivalPFN, the main changes are training on known synthetic distributions rather than sampled outcomes, varying the context size, and using a broader synthetic prior. The paper provides theoretical results and comparisons with 28 methods on 115 datasets. However, I am not convinced that the chosen synthetic prior offers a reliable advantage over alternatives (e.g. power prior, emoirical Bayes, Bayesian or transfer learning alternatives). A clearer prior-selection procedure and a consistent evaluation protocol are needed.

Critical strengths

The benchmark and nemuerical studies are extremely extensive. The results suggest that all three proposed changes contribute, especially with the broader prior.

Critical weaknesses
  1. My main concern is the choice of synthetic prior, which needs stronger justification. The authors specify five generators with fixed mixture weights and particular distributions for covariates, effects, and censoring. How were these choices made, and how should a user judge whether they are suitable for a new dataset? The prior seems account for the largest improvement in the numerical evaluation, so its selection is central to the contribution. The leave-one-generator-out analysis is useful, but sensitivity to mixture weights, effect strengths, and censoring settings is also needed. Please clarify whether the prior was designed using separate development datasets or revised after examining benchmark results.
  2. The paper builds on survival-PFN. The added value over SurvivalPFN and simpler regularized methods remains unclear. The oracle and sampled-label objectives have the same population target, so the benefit of oracle supervision is mainly computational. Please distinguish the SurvivalPFN comparison from the controlled ablation that holds the new architecture fixed. In addition, the possibility that prior information improves small-sample prediction is not enough to justify this particular prior. Carefully tuned comparisons with simpler Bayesian, power prior, or empirical-Bayes survival models would help establish its value.
  3. The optimality result holds under the chosen prior. The error bound assumes posterior concentration and overlap between the target-task distribution and the training distribution, while retaining a prior-mismatch term. These results help describe possible errors, but do not show when TTE-PFN should outperform a local estimator or how to recognize a poor prior (i.e. how to prevent negative learning). Can the authors provide a practical check for prior–data conflict or identify settings in which the method performs worse?
  4. Recalibration is applied specifically for IBS and KM-Cal, although the appendix shows that it can worsen MAE-PO. Please evaluate the raw and recalibrated predictors separately on all metrics and allow comparable recalibration for competing methods.
  5. The marginal Kaplan–Meier censoring weights also need justification when censoring depends on covariates. In addition, the log-rank comparison of predicted medians and observed times does not measure patient-level discrimination: permuting predictions among patients leaves that statistic unchanged.
  6. Random splits provide evidence for prediction within a cohort. They do not show that historical patients provide an adequate context for future patients whose outcome relationships may change
  7. The output approximates a posterior-predictive distribution, not a posterior over survival functions. The reported calibration measures are useful, but do not establish uncertainty about the model from variation in patient outcomes. Predictive-interval coverage and width under small samples, heavy censoring, and prior misspecification would strengthen the evaluation.
Questions for authors

please see the weakness section

4 comments

Thank you for your careful review and constructive feedback. We sincerely appreciate your recognition of the extensive numerical evaluation and your thoughtful suggestions regarding the synthetic prior, theoretical guarantees, and evaluation methodology. We apologize for the delayed response, as we conducted extensive additional experiments to thoroughly address your concerns. Your comments have helped us identify several aspects that required further clarification and empirical validation. We will provide point-by-point responses below.

R1 [W1]: (i) Prior design and selection. Our synthetic prior was designed to cover diverse survival DGPs rather than optimize performance for any particular dataset. The five generators provide complementary structures: Tree and DeepMLP capture different nonlinear interactions, \texttt{Narrow} emphasizes strong covariate effects, and Parametric and Bernstein broaden the range of conditional event-time distributions. All generators provide analytically available conditional distributions for oracle distillation. Importantly, we used separate held-out synthetic development tasks for model development and hyperparameter selection (Lines 1339--1342 of the original manuscript). None of the 115 benchmark datasets was used to tune the prior or select the reported model configuration.

(ii) Additional sensitivity analyses. To further assess the impact of prior specification, we have conducted three additional sensitivity analyses varying generator mixture weights, effect strengths, and censoring settings, each evaluated using five independent random seeds. The configurations and results are reported in Rebuttal Tables 5 [https://anonymous.4open.science/r/TTEPFN/Table5.png ] and 6 [https://anonymous.4open.science/r/TTEPFN/Table6.png ], respectively.

For mixture weights, the uniform configuration achieves the best Overall rank (2.70), followed by interaction-light (2.77) and our original configuration (2.99). Notably, the original configuration achieves the best ranks on MAE-PO and both C-indices, indicating trade-offs across predictive metrics. For effect strengths, the original configuration (1.96) performs close to the strong-effect variant (1.93), while the weak-effect variant performs worse (2.12). Similarly, for censoring settings, the original configuration (2.33) performs close to the heavy-censoring variant (2.25), whereas the light-censoring variant performs substantially worse (2.95).

Importantly, these analyses were conducted after fixing the original prior and were not used to reselect the reported configuration. The results confirm that prior specification affects predictive performance and that our original configuration is not universally optimal. Since future datasets are unknown, our objective is to construct a broadly applicable prior rather than select weights optimized for the evaluation benchmark.

(iii) Prior suitability for new datasets. For a new cohort, potential prior--data conflict can be assessed through held-out predictive performance and calibration within the labeled context, including comparisons with locally fitted estimators such as CoxNet. Such diagnostics can help identify settings where the fixed prior may be unsuitable, although they do not provide formal guarantees against negative transfer. We have incorporated the sensitivity analyses and clarified the prior development protocol, its limitations, and opportunities for more adaptive prior selection in the revised manuscript.

R2 [W2]: (i) Contributions beyond SurvivalPFN. We agree that TTE-PFN builds on the amortized inference framework of SurvivalPFN. Our contributions focus on a theoretically motivated pretraining framework combining oracle distillation, context-size randomization, and a structurally diverse synthetic prior. As established in our analysis, oracle distillation and sampled-label supervision share the same population minimizer. The primary benefit of oracle supervision is therefore the reduction of gradient variance associated with label sampling, rather than a different population-level objective. Context-size randomization supports inference across varying amounts of labeled context, while the diverse synthetic prior aims to cover heterogeneous covariate effects and event-time distributions. We do not claim that the specific prior configuration is uniquely optimal, and we separately examine its sensitivity in our response to R1 [W1].

(ii) Official SurvivalPFN comparison versus controlled ablation. We agree that the original presentation did not sufficiently distinguish the official SurvivalPFN comparison from the controlled ablation of our proposed components. We have therefore extended the factorial ablation to explicitly account for backbone architecture, as shown in Rebuttal Table 3 [https://anonymous.4open.science/r/TTEPFN/Table3.png ]. All eight combinations of oracle distillation (A), context-size randomization (B), and the diverse synthetic prior (C) are evaluated using the same two-way attention backbone. We additionally evaluate TTE-PFN with the TabDPT backbone while retaining all three pretraining components, and report the original SurvivalPFN separately.

The complete TTE-PFN achieves the best Overall average rank (4.07), closely followed by TTE-PFN (TabDPT) (4.10). In contrast, the original SurvivalPFN obtains an Overall rank of 7.23. This comparison suggests that the choice between the two backbones has a relatively small effect on aggregate performance in our evaluation, whereas the proposed pretraining framework yields substantial improvements even when using the TabDPT backbone. Within the controlled factorial ablation, removing the diverse prior causes the largest single-component degradation, increasing the Overall rank from 4.07 to 6.95, compared with 4.70 without context-size randomization and 4.26 without oracle distillation.

We have clarified the distinction between backbone-controlled ablations and the original SurvivalPFN comparison in the final version.

(iii) Additional comparisons with Bayesian and regularized survival models. We have conducted additional experiments comparing TTE-PFN against five alternatives: Bayesian Weibull AFT (BayesWeibullAFT, [1]), empirical-Bayes Weibull AFT (EB-WeibullAFT, [2]), power-prior Weibull AFT (PP-WeibullAFT, [3]), prior-projected Weibull AFT (PriorProj-WeibullAFT, [4]), and cross-validated CoxNet (CoxNet-CV, [5]).

As shown in Rebuttal Table 7 [https://anonymous.4open.science/r/TTEPFN/Table7.png ], TTE-PFN achieves the best Overall average rank (3.10) across 115 benchmark datasets, outperforming PriorProj-WeibullAFT (3.34), EB-WeibullAFT (3.43), PP-WeibullAFT (3.44), CoxNet-CV (3.72), and BayesWeibullAFT (3.94). TTE-PFN also achieves the best average ranks on five of the seven evaluation metrics, including MAE-PO, IBS, Harrell's C-index, Uno's C-index, and Log-Rank. The alternative methods remain competitive on calibration metrics, particularly PP-WeibullAFT on D-Cal and KM-Cal. These results demonstrate that TTE-PFN offers competitive overall predictive performance compared with simpler Bayesian and regularized survival estimators, rather than uniformly outperforming them on every metric.

We further investigate small-sample performance across 55 datasets at context sizes N∈32,64,128,256,512N\in \\{32,64,128,256,512\\}, without recalibration (Rebuttal Table 8 [https://anonymous.4open.science/r/TTEPFN/Table8.png ]). TTE-PFN achieves the best Overall ranks at N=32N=32 (2.98), N=64N=64 (3.01), and N=128N=128 (3.10), providing direct empirical evidence of its competitive advantage in low-data settings. As the context size increases, simpler estimators become more competitive, with PriorProj-WeibullAFT ranking first at N=256N=256 (3.20) and CoxNet-CV at N=512N=512 (3.18). These findings suggest that the benefits of amortized prior-based prediction are particularly pronounced when labeled data are scarce, while locally fitted methods can become preferable as more observations become available.

We have incorporated these additional comparisons, implementation details, and a discussion of the relative strengths and limitations of prior-based and conventional survival estimators into the final version.

R3 [W3]: We agree that Theorem 1 provides a conditional error decomposition rather than a guarantee that TTE-PFN always outperforms locally fitted estimators. Importantly, the primary goal of our theoretical analysis is to decompose predictive error into amortization, prior coverage, and posterior concentration, thereby providing principled guidance for PFN design. Specifically, oracle distillation targets amortization error, the diverse synthetic prior aims to improve prior coverage, and context-size randomization supports learning across different cohort sizes. We acknowledge that these guarantees depend on posterior concentration and train--test overlap and may become uninformative under severe prior misspecification.

Nevertheless, a practical way to detect potential prior--data conflict is through held-out validation within the labeled context. Specifically, we can split the context into fitting and validation subsets and compare TTE-PFN against a locally fitted estimator such as CoxNet using censoring-aware predictive metrics. Cases where TTE-PFN performs substantially worse may indicate negative transfer and motivate selecting the local estimator instead.

We have clarified the purpose and limitations of our theoretical analysis and discussed this practical diagnostic in the revised manuscript.

R4 [W4]: We have extended the evaluation to all 29 methods under two consistent protocols: without recalibration (Rebuttal Table 1 [https://anonymous.4open.science/r/TTEPFN/Table1.png ]) and with the same context-based recalibration applied to all methods for IBS and KM-Cal (Rebuttal Table 2 [https://anonymous.4open.science/r/TTEPFN/Table2.png ]). The remaining five metrics are evaluated using raw predictions in both protocols, following the original evaluation setting.

Without recalibration, TTE-PFN ranks third overall (9.31), outperforming SurvivalPFN (10.97). With recalibration available to all methods, TTE-PFN achieves the best overall rank (8.42), ahead of CWGB (8.81) and SurvivalPFN (10.78). In particular, TTE-PFN's IBS rank improves from 8.01 to 6.02, while KM-Cal improves from 11.59 to 7.37. These results demonstrate that TTE-PFN remains competitive without recalibration and achieves the best aggregate performance under the matched recalibration protocol.

We have incorporated these additional comparisons and clarified the recalibration protocol in the revised manuscript.

R5 [W5]: (i) Censoring weights. Our evaluation follows the standard SurvivalEVAL implementation, using marginal Kaplan--Meier estimates consistently across all methods. We have clarified the underlying censoring assumptions and explicitly discussed the limitations of marginal censoring weights when censoring depends on covariates in the revised manuscript.

(ii) Log-Rank metric. We include the Log-Rank metric because it was used in SurvivalPFN, our closest baseline, and we follow the same evaluation protocol to ensure direct comparability. We agree that this metric measures cohort-level distributional agreement rather than patient-level discrimination. Importantly, our evaluation covers three complementary dimensions of survival prediction: predictive accuracy, discrimination, and calibration. Patient-level discrimination is evaluated separately using Harrell's and Uno's C-indices. We have clarified this distinction and revised the interpretation of Log-Rank in the manuscript.

R6 [W6]: We agree that random train--test splits primarily assess within-cohort generalization rather than prospective prediction under temporal distribution shifts. However, our primary objective, as formalized in Eq. (1) of the original manuscript, is to estimate the posterior-predictive distribution for new patients given a labeled context from the same underlying DGP. This is also the central setting of existing in-context survival prediction methods, including SurvivalPFN. Accordingly, we follow the widely adopted random-split evaluation protocol to ensure consistent and comparable evaluation across methods.

At the same time, evaluating temporal generalization is nontrivial, as it requires reliable temporal information and careful consideration of changes in baseline hazards, covariate distributions, and covariate--outcome relationships. We have clarified this limitation in the revised manuscript. We believe that extending in-context survival prediction to temporal distribution shifts represents an important and interesting direction for future work.

R7 [W7]: As formalized in Eq. (1) of the original manuscript, TTE-PFN approximates the posterior-predictive distribution qPPD,Tq_{\mathrm{PPD},T}, from which predictive survival functions and patient-level predictive intervals can be directly derived. While this distribution integrates over posterior uncertainty in the underlying DGP, it does not provide a posterior distribution over survival functions. One training-free approach to further characterize uncertainty in the estimated survival functions is context bootstrapping, which repeatedly resamples the labeled context and performs inference with the frozen TTE-PFN.

Regarding the suggested evaluation, our existing benchmark already covers challenging settings, with sample sizes ranging from 2626 to 52,42252{,}422 and censoring proportions from 00 to 0.940.94 across 101 real-world cohorts, as detailed in Table 3 of the original manuscript. Furthermore, we have added a synthetic case study in the Appendix (Rebuttal Figure 3 [https://anonymous.4open.science/r/TTEPFN/Fig3.png ]) examining 90%90\% pointwise context-bootstrap uncertainty bands across context sizes N∈50,100,200,400N\in \\{50,100,200,400 \\} and censoring rates of 5%5\%, 25%25\%, 50%50\%, and 95%95\%. The results show that increasing context size generally narrows the uncertainty bands and improves agreement with the true survival function, whereas heavier censoring leads to greater uncertainty and larger prediction errors. Under extreme censoring, the bands may fail to contain the true survival function, highlighting the limitations of context bootstrapping as an approximate uncertainty assessment.

We have incorporated this case study into the revised manuscript and clarified the distinction between posterior-predictive uncertainty and context-bootstrap uncertainty, including the limitations of the latter under heavy censoring.

References

[1] Ibrahim J G, Chen M H, Sinha D. Bayesian survival analysis[M]. New York: Springer, 2001.

[2] MacKay D J C. Bayesian interpolation[J]. Neural computation, 1992, 4(3): 415-447.

[3] Ibrahim J G, Chen M H. Power prior distributions for regression models[J]. Statistical Science, 2000: 46-60.

[4] Schmidli H, Gsteiger S, Roychoudhury S, et al. Robust meta-analytic-predictive priors in clinical trials with historical control information[J]. Biometrics, 2014, 70(4): 1023-1032.

[5] Simon N, Friedman J H, Hastie T, et al. Regularization paths for Cox's proportional hazards model via coordinate descent[J]. Journal of statistical software, 2011, 39: 1-13.

Reviewer_hHdCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TTE-PFN, a prior-fitted network that estimates event- and censoring-time distributions on new datasets without dataset-specific training. The method introduces three main components: oracle distillation using analytic conditional distributions, randomized context sizes during pretraining, and a diverse synthetic prior combining five data-generating families. The authors provide theoretical analyses showing that oracle distillation removes additional gradient variance introduced by sampled-label supervision while preserving the posterior-predictive population minimizer. They also derive an excess-risk bound that decomposes prediction error into amortization, prior coverage, and posterior concentration terms. Experiments on 115 survival datasets against 28 competing methods using seven evaluation metrics show that TTE-PFN achieves the best overall average rank and the best average ranks on both predictive accuracy metrics, while demonstrating efficient inference.

Critical strengths
  1. The experimental evaluation covers 115 survival datasets, including 101 real-world datasets, and compares TTE-PFN against 28 competing methods using seven evaluation metrics. TTE-PFN achieves the best overall average rank and the best average ranks on both predictive accuracy metrics. Furthermore, the factorial ablation study systematically evaluates the contributions of the three proposed components while keeping the backbone and optimization settings fixed. These experiments provide substantial empirical support for the effectiveness of the proposed approach across diverse survival datasets.

  2. The authors demonstrate that oracle distillation preserves the posterior-predictive population minimizer while removing the additional logit-gradient covariance introduced by sampled-label supervision. They also derive an excess-risk bound that decomposes prediction error into amortization, prior coverage, and posterior concentration terms under explicit assumptions. These analyses provide a theoretical foundation for the proposed pretraining strategies and help explain the motivation behind the design choices.

Critical weaknesses
  1. The paper applies context-based recalibration to TTE-PFN when evaluating IBS and KM-Cal, and Appendix D.2 demonstrates that this procedure improves both metrics on many datasets. However, the main benchmark does not provide a comprehensive comparison of TTE-PFN before and after recalibration against the competing methods. It is also unclear whether comparable recalibration procedures were applied to the baselines. Consequently, it is difficult to determine how much of the reported improvement in predictive accuracy and overall ranking can be attributed to the proposed model itself rather than the additional recalibration procedure.

  2. The ablation study indicates that the proposed diverse synthetic prior contributes the most to the performance improvement among the three proposed components. However, the original SurvivalPFN already explored combinations of synthetic data-generating processes through its Kitchen-Sink Meta Prior. The paper does not directly compare the proposed prior with this existing approach under controlled conditions, making it difficult to establish the additional benefits of the specific prior design. Furthermore, Table 2 describes the removal of all three proposed components as recovering SurvivalPFN, although the ablation keeps the proposed backbone fixed, which differs from the original SurvivalPFN architecture.

Questions for authors
  1. Could the authors provide the benchmark results of TTE-PFN before and after context-based recalibration and clarify whether comparable recalibration procedures were applied to the baselines? In particular, how much does the overall ranking change when recalibration is excluded?

  2. Appendix D.2 constructs the recalibration transformation using predicted survival curves from the labeled context set. How are these predictions generated? Specifically, are the corresponding observed times and event indicators masked or held out when predicting each context sample? How is potential in-sample bias in the recalibration procedure addressed?

  3. How does the proposed diverse synthetic prior differ from the Kitchen-Sink Meta Prior introduced in SurvivalPFN? Could the authors provide a direct comparison under comparable backbone and training settings to clarify the additional contribution of the proposed prior? Furthermore, does the SurvivalPFN configuration in Table 2 represent the original model or an ablation variant using the TTE-PFN backbone?

2 comments

Thank you for your careful review and constructive feedback. We sincerely appreciate your recognition of our extensive empirical evaluation and theoretical contributions. We apologize for taking additional time to respond, as we conducted extensive additional experiments and analyses to thoroughly address your concerns. We provide point-by-point responses below.

R1 [W1 & Q1]: We have conducted two additional evaluations across all 115 datasets to assess the impact of context-based recalibration and ensure a fair comparison across methods. Specifically, we consider (i) a fully uncalibrated comparison, in which no method receives recalibration, and (ii) a matched recalibration comparison, in which the same procedure is applied to all eligible methods for IBS and KM-Cal. The remaining five metrics are evaluated without recalibration in both settings.

As shown in Rebuttal Table 1 [https://anonymous.4open.science/r/TTEPFN/Table1.png ], TTE-PFN achieves an Overall average rank of 9.31 without recalibration, ranking third and outperforming SurvivalPFN (10.97). Under the matched recalibration protocol (Rebuttal Table 2 [https://anonymous.4open.science/r/TTEPFN/Table2.png ]), TTE-PFN achieves the best Overall rank (8.42), followed by CWGB (8.81), while SurvivalPFN obtains 10.78. These results demonstrate that recalibration improves TTE-PFN's aggregate ranking, while its competitive performance is not solely attributable to this procedure.

We have incorporated both evaluation protocols into the final version and clarified the application of recalibration across methods and metrics to ensure a fair comparison.

R2 [Q2]: We clarify that the recalibration procedure uses only the labeled context and does not access any held-out query outcomes. For each context subject, we generate its predicted survival curve by treating that subject as an unlabeled query while excluding its observed time and event indicator from the labeled context. The remaining context observations provide the support information for prediction. This leave-one-out procedure ensures that each subject's prediction does not directly use its own outcome, thereby avoiding self-label leakage.

We then average these out-of-sample survival predictions and align the resulting curve with the Kaplan--Meier estimate of the labeled context to construct the recalibration transformation. The same transformation is subsequently applied to held-out query predictions without using their outcomes. We have clarified this prediction and recalibration protocol in the final version.

R3 [W2 & Q3]: (i) Synthetic prior design and sensitivity. We acknowledge that SurvivalPFN already explored diverse synthetic DGPs through its Kitchen-Sink Meta Prior. Our contribution is not the general idea of combining multiple generators, but a specific five-family prior designed to provide complementary structural coverage and analytic conditional distributions for oracle distillation. Specifically, Tree and DeepMLP capture nonlinear interactions, Narrow emphasizes stronger covariate effects, Parametric broadens event-time distribution families, and Bernstein provides flexible conditional distributions.

We have already examined the contributions of individual generator families through a leave-one-family-out analysis in the original manuscript (Figure 8). The results show that removing any individual generator worsens the Overall rank, with increases ranging from +0.05+0.05 to +0.35+0.35, while restricting the prior to Bernstein alone leads to a substantially larger degradation (+1.78+1.78). These findings indicate that the five generators provide complementary coverage rather than relying on a single dominant family.

We have further conducted three additional sensitivity analyses varying (a) generator mixture weights, (b) effect strengths, and (c) censoring settings, each evaluated across 115 datasets using five independent random seeds. The configurations and results are presented in Rebuttal Tables 5 [https://anonymous.4open.science/r/TTEPFN/Table5.png ] and 6 [https://anonymous.4open.science/r/TTEPFN/Table6.png ]. For mixture weights, the uniform configuration achieves the best Overall rank (2.70), followed by interaction-light (2.77) and the original prior (2.99). Notably, the original prior achieves the best ranks on MAE-PO and both C-indices, indicating trade-offs across metrics. For effect strengths, the original configuration (1.96) performs close to the strong-effect variant (1.93), while the weak-effect variant performs worse (2.12). For censoring settings, the original configuration (2.33) performs close to the heavy-censoring variant (2.25), whereas light censoring yields a substantially worse rank (2.95).

These analyses demonstrate that prior specification affects predictive performance and that our original configuration is not universally optimal. Importantly, the original prior hyperparameters were determined using synthetic development datasets, independently of the 115 benchmark datasets. The additional sensitivity analyses were conducted after fixing the original configuration and were not used to reselect the reported prior.

(ii) Controlled ablation and backbone comparison. We have extended the factorial ablation to explicitly distinguish the effects of pretraining components from backbone architecture. As shown in Rebuttal Table 3 [https://anonymous.4open.science/r/TTEPFN/Table3.png ], all eight combinations of the three pretraining components are evaluated using the same two-way attention backbone. We additionally evaluate TTE-PFN with the TabDPT backbone while retaining all three components and separately report the original SurvivalPFN.

TTE-PFN achieves the best Overall rank (4.07), closely followed by TTE-PFN (TabDPT) (4.10), whereas the original SurvivalPFN obtains 7.23. The small difference between the two TTE-PFN backbones suggests that the architectural change is not the primary driver of the aggregate improvement. Within the controlled factorial ablation, removing the diverse prior causes the largest single-component degradation, increasing the Overall rank from 4.07 to 6.95, compared with 4.70 without context-size randomization and 4.26 without oracle distillation. These results provide additional evidence for the contribution of the proposed pretraining components while separating their effects from the backbone architecture.

We have corrected the original ambiguity in Table 2 by explicitly distinguishing the two-way attention ablation without A, B, and C from the original SurvivalPFN, which uses the TabDPT backbone. We have incorporated the additional sensitivity analyses and architecture-controlled ablations into the final version, together with a clearer discussion of the relationship between our synthetic prior and SurvivalPFN's Kitchen-Sink Meta Prior.

Reviewer_6tzUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces time-to-event prior-data-fitted networks (TTE-PFNs), an amortized Bayesian method for right-censored survival analysis with the goal of estimating survival distributions (along with censoring distributions). It builds on existing methods such as SurvivalPFN [1], and introduces some ideas, including randomizing the context lengths, using the ground-truth analytic conditional event/censoring distributions instead of sampled event/censoring times, a more diverse prior, and a two-way attention mechanism over columns/rows to achieve better performance across several survival analysis metrics on 115 datasets, achieving the best overall rank, and first on MAE-PO and IBS.

The paper also accompanies these ideas with some theoretical insights, showing how the error between the trained posterior predictive distribution from TTE-PFN and the true survival distributions can be decomposed into amortization error, prior coverage, and posterior concentration. Furthermore, it theoretically shows how using the true conditional event/censoring distribution instead of sampled realizations can reduce gradient variance without changing the population target.

[1] Qi, S., Balazadeh, V., Cooper, M., Greiner, R., & Krishnan, R.G. (2026). SurvivalPFN: Amortizing Survival Prediction via In-Context Bayesian Inference. ArXiv, abs/2605.15488.

Critical strengths

Strong empirical results

The evaluation covers 115 datasets against 28 baselines over seven metrics. This is indeed a very extensive evaluation package and also shows better performance compared to SurvivalPFN (taking into account some of the evaluation weaknesses described below, of course). Moreover, a good set of ablations is provided to check the design choices empirically, although there are important issues with their implementation and interpretation, as discussed below.

Oracle distillation is clean and reasonable

Given that the prior is fully simulated, one can obtain the conditional event/censoring distributions for prior datasets. Using them, instead of just the sampled ground truths, is a natural idea to help with more informative gradients. Lemma 2 gives a clear reasoning for why that is the case. This is an idea similar to distillation from soft labels in supervised learning, going back to Hinton et al. (2015).

  • Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the Knowledge in a Neural Network. arXiv:1503.02531.
Critical weaknesses

Novelty, contributions, and theory

First, I notice some overclaims in the paper. For example, the paper claims in lines 064–066 that it "... formulate[s] in-context survival prediction as amortized Bayesian inference." This is not something new; SurvivalPFN---the main method this work builds on---has already done that, and the Background section (lines 090–107) already follows a very similar framing.

The remaining contributions are mostly new design choices on top of SurvivalPFN to achieve better performance. Those are certainly impactful, as we see better reported performance compared to SurvivalPFN. However, they seem to be mostly ideas that have already been explored in the community, applied in this setting. The claim that these design choices are motivated by the theory developed in the paper seems somewhat misleading:

  1. Context-size randomization and Corollary 1: First, I'd emphasize that randomization of context length to improve length generalization, along with other tricks such as QK-normalization, has already been explored in SOTA tabular foundation models such as TabPFN, TabICL, and Kumo-Tabular. Claiming that this is driven by the theory in this paper (such as Corollary 1) seems misleading. Specifically, Corollary 1 only shows that, over all measurable predictors, the population minimizer equals the posterior predictive for context sizes in the support of PNP_N, i.e., [200, 2000] (Appendix C.3, line 1333). It says nothing about sizes outside this range and provides no finite-capacity or finite-training guarantee even within this range. Although Eq. (74) provides a bound involving 1/PN(n)1/P_N(n), it does not justify the particular choice of a log-uniform distribution or explain the empirical generalization across context sizes. Hence, the question of how well the model generalizes to different context sizes remains primarily empirical, as was already the case in SurvivalPFN.

  2. Prior diversity, misspecification, and Assumption 4: The idea of diversifying the prior to support a broader set of data-generating processes (DGPs) is already known in PFNs (e.g., see the priors used in TabICLv2 or MITRA by Zhang et al.). The paper does a good job of using classes like trees, MLPs, parametric distributions, etc. However, it claims that these choices are theoretically grounded and provides an excess-risk decomposition in Theorem 1 to back this up (lines 059–062). It even states that the theory allows the true DGP to be completely outside the prior support (line 236). However, the quantities that would actually provide insights are mostly just assumed in Assumptions 2 to 4. In particular, Assumption 3 already assumes posterior concentration at a rate ϵn\epsilon_n, rather than deriving such a rate for the proposed prior. Assumption 4 also requires a bounded density ratio between the test-task law and the prior-predictive law. Under misspecification, this ratio can become extremely large or even fail to exist when the required absolute continuity does not hold, making the first term in Theorem 1 vacuous or the theorem inapplicable. In particular, the theory does not provide actionable insights on what to do when such prior misspecification happens; no theory or practical approach is provided to identify scenarios where we have misspecified priors. Hence, making the prior more diverse remains largely an engineering effort, and the question of whether this is sufficient is again empirical, as it already was in SurvivalPFN.

As a result, statements such as "these results align with the design guidance provided by Theorem 1 and Corollary 1" (line 397) seem overstated. The theory mainly provides a qualitative error decomposition under substantial assumptions, rather than quantitative, model-specific guidance for the proposed design choices. One would expect more concrete insights, such as convergence rates that predict empirical behavior as we increase context length, guidance on when to stop training, or a way to detect whether a new inference dataset (without ground truth) lies far from the prior, and what to do in such scenarios. None of these is provided. Similarly, there are no calibration guarantees (frequentist or otherwise).

Evaluation

There are certain contradictions between what is written or claimed in the paper and what the released code provides.

First, in the released configuration in the supplementary materials (configs/ablation_backbone.yaml), we see that all the factorial (Table 2) and leave-one-family-out (Fig. 8) ablation arms were trained on a different, smaller model than the reported TTE-PFN (approximately 3.6M parameters and 50 bins instead of 6.97M and 1,024 bins). Moreover, the ablation backbone uses per-row attention and a different time transformation. Although the backbone is held fixed across the ablation arms, these experiments do not isolate the contributions of the proposed components to the actual reported TTE-PFN model. Furthermore, the last row of Table 2, labeled SurvivalPFN, is not the released SurvivalPFN but an ablation backbone where all three design choices (A, B, C) are removed. The paper nevertheless states that removing these components "recovers SurvivalPFN" (lines 383–384). Finally, the two-way attention mechanism, which is another major difference from SurvivalPFN, is never ablated.

Besides the above, certain changes in how evaluation is performed for TTE-PFN, compared to other methods, raise questions about fair comparison. For example, TTE-PFN's IBS and KM-Cal results on real cohorts are computed after a post-hoc recalibration of the predicted time scale (Appendix C.3), whereas the other baselines, including SurvivalPFN, are not given the same recalibration procedure. This choice is consequential, as recalibration improves IBS on 89 cohorts but worsens MAE-PO on 77 of the 115 cohorts (Appendix D.2).

Moreover, D-Cal seems to be calculated using a different formula for TTE-PFN (a custom formula in ttepfn/pretrain.py) than SurvivalEVAL's chi-square statistic (_dcal_chi2 in ttepfn/metrics.py), which is used for the baselines. A similar inconsistency exists for the concordance metrics: TTE-PFN's main evaluation uses a mean-time prediction as its risk score (Y_FROM_T_STRATEGY = "mean"), whereas the common SurvivalEVAL implementation uses the predicted median time. These inconsistencies make it difficult to assess how much of the reported improvement comes from the model itself rather than differences in the evaluation protocol.

There is also another reproducibility concern about the synthetic evaluation protocols. TTE-PFN and the released SurvivalPFN runner generate 4,000 observations with a 2,000/2,000 context–test split, whereas the other baseline runner generates 2,800 observations with a random 70/30 split and different seeds. This contradicts the claim that all methods use the same five train–test partitions.

Finally, there is some concern about the benchmarks being used as part of the development loop. The censoring-head weight λc=1\lambda_c=1 is reported as achieving the best overall rank in the sensitivity analysis on the evaluation benchmark (Appendix D.2). It is unclear whether this value was selected independently of the benchmark or whether any held-out datasets were used for model selection. In addition, the Parametric generator includes the same four distribution families as the 14 synthetic test datasets (Appendix C.3), with covariates in the same range of (0, 2) as the CQRNN generators. Although these are standard distributional choices and this overlap does not by itself imply data leakage, evaluating on synthetic DGP families not explicitly represented in the prior would provide stronger evidence of generalization.

Questions for authors

0. Can the authors show a concrete case where Theorem 1 or Corollary 1 led to a design decision, or predicts an empirical result, that would not have been reached without them? For example, can the authors derive a quantitative prediction-error bound as a function of context size, or justify the choice of PNP_N (e.g., why log-uniform on [200, 2000])? Otherwise, I suggest the authors present these choices as empirically motivated and revise the theoretical claims accordingly.

1. How can BnB_n in Assumption 4 be finite on real cohorts with one-hot categorical covariates, given the continuous covariate distributions in the prior? More generally, the prior uses effective covariate dimensions {2,4,8,12}\{2,4,8,12\} with zero-padding. For example, a dataset with three nondegenerate continuous covariates can have a padding pattern with probability zero under the prior, violating the absolute-continuity requirement in Assumption 4. Can any term of Theorem 1 be estimated on a real dataset, for example to flag a cohort that is poorly covered by the prior?

2. How does performance change on benchmark datasets outside the prior support? How is it possible to detect that before outputting predictions? Does Theorem 1 provide any actionable insights in such cases?

3. Can the authors confirm that Table 2 and Figures 7 and 8 used a different backbone and discretization than the reported model? If so, could they repeat these ablations using the reported two-way attention backbone and 1,024-bin discretization? Moreover, could the authors ablate the two-way attention backbone against a row-tokenized one (as used in SurvivalPFN), holding the prior and loss fixed? This would help distinguish the contribution of the architecture from the prior, which seems to account for a large part of the improvement.

4. Could the authors re-run the TTE-PFN results without the recalibration procedure (line 2114), or apply equivalent recalibration to all baselines for a fair comparison? Moreover, which D-Cal formula produced Table 6? Can D-Cal be recomputed using SurvivalEVAL's chi-square statistic for all methods? Similarly, can Harrell's and Uno's C-indices be recomputed using the same median-based SurvivalEVAL implementation?

5. How does performance change as the absolute context size varies below, within, and above the training range [200, 2000], on the same datasets? This would help distinguish context-size generalization from differences in dataset difficulty.

6. Can the authors clarify why the synthetic benchmark uses different dataset sizes and splits across the TTE-PFN and baseline evaluation runners, and repeat the comparison using identical generated samples and train–test partitions?

4 comments

Thank you for your detailed review and constructive feedback. We sincerely appreciate your recognition of our extensive empirical evaluation and theoretical analysis of oracle distillation. We apologize for the additional time it took to respond, as we conducted additional experiments and analyses to address your concerns carefully. We provide point-by-point responses below.

R1 [W1 & Q0, Q5]: (i) Scope of the theoretical contributions. Our theoretical contributions are not intended to establish these general ideas as new, but to characterize the proposed survival-specific pretraining framework. In particular, our oracle-distillation analysis establishes that analytic conditional supervision preserves the posterior-predictive population minimizer while eliminating the additional gradient variance introduced by sampled-label supervision. The excess-risk decomposition further separates amortization error, prior coverage, and posterior concentration under explicit assumptions.

We clarify that context-size randomization was introduced to expose the model to dynamically varying labeled contexts during pretraining, rather than training it with a single fixed context size. We adopted log-uniform sampling to provide broad coverage across different context-size scales, allowing the model to learn from both relatively small and large contexts. The range [200,2000][200,2000] was selected empirically using held-out synthetic development datasets, independently of the 115 benchmark datasets. We do not claim that Corollary 1 establishes the optimality of log-uniform sampling or the selected range. Rather, Corollary 1 characterizes the population minimizer for context sizes within the support of the training distribution, while generalization under finite model capacity and beyond this range remains an empirical question. Similarly, Theorem 1 provides a qualitative error decomposition under explicit assumptions, rather than model-specific convergence rates or hyperparameter prescriptions. We have revised the corresponding claims in the final version to distinguish theoretical characterization from empirically motivated design choices.

(ii) Additional experiments on context-size generalization. We have conducted an additional controlled experiment on the SUPPORT dataset to directly evaluate generalization across context sizes below, within, and above the pretraining range. We use the reported TTE-PFN checkpoint without retraining and evaluate context sizes N∈32,64,128,256,512,1024,1433,2000,3000,4000N\in\\{32,64,128,256,512,1024,1433,2000,3000,4000\\}. The dataset contains 6,211 training observations and 2,662 test observations per split, with approximately 32% censoring. We report results averaged over five random splits, varying the number of labeled context observations while evaluating on the held-out test set.

As shown in Rebuttal Table 9 [https://anonymous.4open.science/r/TTEPFN/Table9.png ], performance generally improves as the context size increases. Harrell's C-index increases from 0.5481 at N=32N=32 to 0.6083 at N=2000N=2000, while IBS decreases from 0.2126 to 0.1921 and MAE-PO decreases from 1.5466 to 1.2973. Importantly, performance remains stable and improves modestly beyond the pretraining range: at N=4000N=4000, Harrell's C-index reaches 0.6109, and IBS decreases to 0.1898. These results provide empirical evidence of context-size robustness, including at sizes outside the pretraining support, although improvements are not monotonic for every metric or context size.

We emphasize that these observations are empirical and are not guaranteed by Corollary 1, particularly outside the training range. We have incorporated the additional context-size experiment into the final version and clarified the scope and limitations of our theoretical claims.

R2 [W1 & Q1, Q2]: (i) Assumption 4 and the train--test overlap condition. We agree that Assumption 4 is not automatically satisfied on arbitrary real-world cohorts. In particular, one-hot categorical covariates and deterministic zero-padding patterns may induce test-task distributions that are not absolutely continuous with respect to the synthetic prior-predictive law. In such cases, a finite density-ratio bound BnB_n may not exist, and Theorem 1 does not provide a nonvacuous guarantee.

We clarify that allowing the evaluation DGP to lie outside the parameter support of the training prior does not imply that arbitrary distribution shifts are covered. Theorem 1 applies only when the stated identification, regularity, posterior-concentration, and train--test overlap assumptions hold. In particular, the prior-coverage term characterizes approximation error under these assumptions, rather than establishing a universal guarantee under prior misspecification.

(ii) Practical detection and empirical behavior under prior mismatch. We also acknowledge that the quantities appearing in Theorem 1, including BnB_n, the pseudo-true DGP, and the posterior-concentration rate, are generally not directly estimable from a single unlabeled evaluation cohort. Consequently, the theorem does not currently provide a calibrated procedure for detecting inadequate prior coverage before inference or an actionable threshold for rejecting out-of-prior datasets.

We have conducted additional sensitivity analyses varying the generator mixture weights, effect strengths, and censoring settings across 115 datasets using five independent random seeds. As reported in Rebuttal Tables 5 [https://anonymous.4open.science/r/TTEPFN/Table5.png ] and 6 [https://anonymous.4open.science/r/TTEPFN/Table6.png ], predictive performance changes under different prior specifications, indicating that prior coverage remains an important practical factor. However, these analyses do not establish performance guarantees for arbitrary distributions outside the training prior, nor do they provide a formal out-of-support detector.

We have revised the theoretical discussion in the final version to explicitly state these limitations and distinguish the qualitative error decomposition from guarantees of practical out-of-prior generalization. We will also identify reliable prior-mismatch detection and adaptive prior refinement as important directions for future work.

R3 [W2 & Q3]: Controlled ablations and backbone architecture. We have rerun both the factorial and leave-one-family-out ablations using the reported full-scale TTE-PFN configuration, including the two-way attention backbone and 1,024-bin discretization, rather than the smaller configuration used in the original ablations. We added a controlled backbone comparison to isolate the contribution of the architecture.

(i) Factorial ablation and backbone comparison. We have repeated the complete 232^3 factorial ablation while keeping the full-scale backbone, discretization, and training settings fixed across all eight combinations of (A) oracle distillation, (B) context-size randomization, and (C) the diverse synthetic prior. We additionally evaluate TTE-PFN with the TabDPT backbone while retaining all three components and separately include the original SurvivalPFN.

As shown in Rebuttal Figure 1 [https://anonymous.4open.science/r/TTEPFN/Fig1.png ] and Rebuttal Table 3 [https://anonymous.4open.science/r/TTEPFN/Table3.png ], TTE-PFN achieves the best Overall average rank (4.07), closely followed by TTE-PFN (TabDPT) (4.10), whereas the original SurvivalPFN obtains 7.23. The small difference between the two TTE-PFN backbones suggests that the architectural change is not the primary driver of the aggregate improvement. Among the single-component ablations, removing the diverse prior causes the largest degradation, increasing the Overall rank from 4.07 to 6.95, compared with 4.70 without context-size randomization and 4.26 without oracle distillation.

(ii) Contributions of individual prior components. We repeated the leave-one-family-out analysis using the aligned full-scale configuration, keeping the backbone, discretization, and training settings fixed across the prior variants. As shown in Rebuttal Figure 2 [https://anonymous.4open.science/r/TTEPFN/Fig2.png ] and Rebuttal Table 4 [https://anonymous.4open.science/r/TTEPFN/Table4.png ], the complete five-family mixture achieves the best Overall rank (3.42). Removing any individual generator worsens aggregate performance, with increases of +0.40+0.40 without Tree, +0.62+0.62 without DeepMLP, +0.42+0.42 without Narrow, +0.22+0.22 without Parametric, and +0.08+0.08 without Bernstein. These results provide additional evidence that the five generator families make complementary contributions to the overall performance.

We have also corrected the original statement that removing all three components recovers SurvivalPFN. The w/o A, B, C configuration retains the two-way attention backbone (Overall rank 7.16), whereas the original SurvivalPFN uses the TabDPT backbone (7.23). We have updated the ablation results and experimental configurations in the final version, explicitly distinguishing the original SurvivalPFN from the controlled ablation variants.

R4 [W2 & Q4]: In the original experiments, we allowed method-specific evaluation choices to reflect the strongest performance of each method. For example, some methods used the predicted mean survival time for concordance evaluation, while others used the predicted median. However, we recognize that these differences may affect the relative rankings and make direct comparisons less consistent. We have therefore rerun the evaluations using a unified protocol across all methods.

(i) Unified evaluation metrics. We have standardized the computation of D-Cal using SurvivalEVAL's chi-square statistic for all methods, replacing the method-specific implementations used previously. Similarly, we have recomputed Harrell's and Uno's C-indices using the same SurvivalEVAL implementation, with predicted median survival time consistently used as the risk-score basis across all methods. These changes eliminate differences arising from metric implementations and mean-versus-median choices.

(ii) Fair comparison of recalibration. We conducted two evaluations across all 115 datasets: (a) a fully uncalibrated comparison, in which no method receives recalibration, and (b) a matched recalibration comparison, in which the same context-based procedure is applied to all eligible methods for IBS and KM-Cal. The remaining five metrics are evaluated without recalibration in both settings.

As shown in Rebuttal Table 1 [https://anonymous.4open.science/r/TTEPFN/Table1.png ], TTE-PFN achieves an Overall average rank of 9.31 without recalibration, ranking third and outperforming SurvivalPFN (10.97). Under matched recalibration (Rebuttal Table 2 [https://anonymous.4open.science/r/TTEPFN/Table2.png ]), TTE-PFN achieves the best Overall rank (8.42), followed by CWGB (8.81), while SurvivalPFN obtains 10.78. These results indicate that recalibration improves TTE-PFN's aggregate ranking, while its competitive performance is not solely attributable to recalibration.

We have updated all the results and the evaluation protocol in the final version, ensuring consistent metric definitions and recalibration treatment across methods.

R5 [W2 & Q6]: Synthetic benchmark reproducibility. The apparent inconsistency originated from legacy configurations inherited from the CQRNN repository, which were retained in some scripts but were not used in our reported experiments. Our actual synthetic benchmark evaluation used a 70/30 train--test split, rather than the 2,000/2,000 split indicated by those legacy configurations.

Nevertheless, we have rerun all synthetic benchmark experiments using identical generated samples, a common 70/30 train--test split, and matched random seeds across all methods to eliminate any potential inconsistencies. The updated results remain consistent with our original findings, confirming that the reported conclusions are not driven by differences in data generation or train--test partitioning.

We have updated the evaluation scripts, removed the obsolete configurations, and clarified the data-generation procedures, train--test partitions, and random-seed settings in the final version to ensure reproducibility.

R6 [W2]: (i) Hyperparameter selection and benchmark independence. We clarify that the pretraining hyperparameters, including the censoring-loss weight λc\lambda_c, were selected using held-out synthetic development datasets, independently of the 115 evaluation datasets. These held-out synthetic tasks were also used for checkpoint monitoring, as described in the original manuscript. None of the 115 benchmark datasets was used for hyperparameter selection or checkpoint monitoring.

We understand the reviewer's concern that λc=1\lambda_c=1 achieves the best Overall rank in the benchmark sensitivity analysis. However, this value had already been fixed based on the independent synthetic development data before the benchmark evaluation. The sensitivity analysis was conducted to assess the robustness of this choice rather than to select the optimal value on the evaluation datasets. As shown in Figure 9, performance remains relatively stable for moderate censoring-loss weights: λc=0.25\lambda_c=0.25 and λc=0.5\lambda_c=0.5 increase the Overall rank by only +0.15+0.15 and +0.16+0.16, respectively, compared with λc=1\lambda_c=1. Larger deviations produce greater degradation, with increases of +0.33+0.33, +0.58+0.58, and +0.96+0.96 for λc=0\lambda_c=0, 22, and 44, respectively. These results indicate that the model does not require precise tuning of λc\lambda_c to achieve competitive performance.

(ii) Synthetic prior overlap and generalization. We acknowledge that the Parametric generator includes standard distribution families that also appear in the 14 synthetic datasets from the CQRNN benchmark. However, these benchmark datasets were generated independently and were never used during pretraining or model selection.

As shown in Rebuttal Tables 5 [https://anonymous.4open.science/r/TTEPFN/Table5.png ] and 6 [https://anonymous.4open.science/r/TTEPFN/Table6.png ], the original prior is not uniformly optimal across all configurations and metrics. These analyses help characterize the sensitivity of predictive performance to prior specification, while the evaluation on real-world cohorts provides complementary evidence beyond the synthetic benchmark. We do not claim that these experiments establish a universal guarantee under arbitrary prior misspecification.

We have clarified the separation between synthetic development data and evaluation datasets, the hyperparameter-selection protocol, and the limitations of synthetic benchmark overlap in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.