- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes BayCSD, a survival prediction method that transfers information from an external teacher model to a student trained on a smaller target dataset. The method combines the target outcomes, teacher predictions, and a prior through a generalized Bayesian update. It preserves the original event and censoring times and produces uncertainty estimates for survival predictions. The authors evaluate the approach using synthetic experiments and real clinical datasets.
Critical strengths
1- The paper addresses a useful practical problem. When few events are observed in the target population, information from an external survival model can help improve prediction without requiring access to the original source records.
2- The method retains the original event and censoring times while borrowing information from teacher predictions at selected time points. This is a meaningful design choice, and the controlled comparisons help demonstrate its value.
3- The experiments show improvements over target-only learning on several datasets. In the kidney-to-kidney–pancreas transfer experiment, BayCSD also improves over the teacher and the compared distillation methods.
4- The authors investigate both compatible and mismatched teachers. I appreciate that they discuss how borrowing can reduce estimation variability but also introduce bias when the teacher does not match the target population.
Critical weaknesses
1- The main idea could be explained more clearly. Although the teacher and student are defined, their roles, training procedures, and implementation details are spread across several sections. A simple example would help explain how the generalized posterior uses teacher guidance and why this could improve target prediction.
2- The advantage over directly using the teacher needs clearer justification. In Figure 2, teacher-only predictions appear competitive with or better than BayCSD in the compatible and calibration-shifted settings, although BayCSD performs better under risk-function shift. This does not invalidate the method, but it raises the question of when its additional complexity is worthwhile. A comparison with simple target-data recalibration of the teacher would be helpful.
3- The uncertainty estimates are not consistently reliable under teacher mismatch. In Appendix Table 5, the nominal 95% intervals contain the true survival probabilities approximately 75.6% of the time under calibration shift, compared with 83.4% for the target-only model. The authors acknowledge that nominal coverage is not guaranteed. Nevertheless, narrower intervals alone do not establish better uncertainty estimates, which is important given the paper’s emphasis on uncertainty-aware prediction.
4- Comparisons with other transfer methods could be broader. The paper includes relevant baselines and controlled variants, but the broader comparison is concentrated in one real source–target application. Additional comparisons on the other datasets, or under different population shifts, would help establish the general usefulness of the method.
Questions for authors
1- Why does BayCSD not improve on teacher-only predictions under calibration shift in Figure 2? How would it compare with simply recalibrating the teacher using the target data?
2- How much of the predictive benefit comes from Bayesian posterior averaging rather than the teacher-guidance term? Could the authors compare against a single fitted model using the same objective?
3- How should users interpret the uncertainty intervals when the teacher is mismatched? Could the borrowing-weight selection be modified to consider interval reliability as well as predictive fit?
4- Do the improvements over other distillation methods hold across additional source–target settings?
5 comments
(W2 / Q1. Teacher-only predictions and recalibration continued)
Table R1. Teacher-mismatch simulation: BayCSD versus the teacher recalibrated on the target data (baseline only / baseline and slope). Means over 20 paired replicates.
| Teacher regime | Method | C-index | IBS |
|---|---|---|---|
| Compatible | Teacher only | 0.764 | 0.174 |
| Recal. teacher (baseline / +slope) | 0.764 | 0.177 / 0.179 | |
| BayCSD | 0.757 | 0.179 | |
| Calibration shift | Teacher only | 0.764 | 0.193 |
| Recal. teacher (baseline / +slope) | 0.764 | 0.177 / 0.179 | |
| BayCSD | 0.752 | 0.199 | |
| Risk-function shift | Teacher only | 0.632 | 0.218 |
| Recal. teacher (baseline / +slope) | 0.632 | 0.210 / 0.210 | |
| BayCSD | 0.664 | 0.210 |
- Calibration shift is a pure baseline multiplier, exactly the family recalibration corrects. Recalibration removes the imposed baseline shift, so its fits coincide across the two regimes.
- Risk-function shift. Recalibration uses the teacher score as its only risk input, so it cannot learn a target-specific risk ordering. BayCSD learns target-specific covariate effects and improves the C-index in 18/20 replicates (, 95% CI ) at the same IBS. Recalibration also treats the teacher's risk function as known: its nominal 95% intervals contain the true survival probability for only 48–49% of predictions, compared with 82% for BayCSD.
These comparisons clarify when the additional complexity is worthwhile. Direct teacher deployment may suffice when the teacher transports well and point predictions meet the analytical goal. Recalibration is effective when the shift lies in its parametric family; BayCSD is most relevant when target adaptation requires learning covariate effects beyond a baseline adjustment or global slope, together with uncertainty in the adapted risk function and baseline hazard. The type of shift need not be specified in advance. In the KI-to-KP application, where the shift is genuine, BayCSD has lower mean IBS and higher mean than both recalibrated KI teachers (Table 2). We have added the recalibrated KI teachers to Table 2 and the simulation comparison to Appendix J.
W3 / Q3. Interpreting the intervals under teacher mismatch. The intervals quantify the uncertainty that remains in the target survival model after the teacher's predictions are incorporated at the selected strength. The joint generalized posterior over the risk function and baseline hazard induces a distribution of individualized survival probabilities and RMST, whose spread reflects how sharply the target likelihood and the teacher's pseudo-observations together determine these quantities. Their practical value is to provide patient- and horizon-specific measures of estimation precision beyond a point prediction. Under teacher mismatch, however, this conditional precision should not be interpreted as accounting for all sources of prediction error.
Under teacher compatibility and a fixed borrowing weight, the generalized-posterior analysis accompanying Proposition 1 (Appendix B.3) gives the local posterior covariance , where combines the curvature of the target-likelihood criterion with the curvature contributed by the teacher, and this uncertainty propagates through the student model to individualized survival probabilities and RMST. The borrowing weight is selected for target predictive fit, and the posterior at the selected weight quantifies the uncertainty that remains after borrowing.
How these intervals relate to the true survival probability depends on teacher alignment, and the theory identifies the two cases. Under compatibility, borrowing preserves the population center, and Proposition 1 and the accompanying posterior analysis (Appendix B.3) show that the local posterior covariance is no smaller than the sampling covariance of the fitted center, . This explains the conservative truth inclusion observed under compatible borrowing (98.8%, Table 5; Appendix G.4.1). Under mismatch, borrowing displaces the center by to first order (Eq. 12), where is the gradient of the teacher–student discrepancy at the target parameter . This displacement is not contained in the posterior spread, so truth inclusion is governed by this displacement relative to the spread.
(W3 / Q3. Interpreting the intervals under teacher mismatch continued)
The two mismatch regimes show how target-guided selection acts on this balance. Under risk-function shift, strong borrowing from the shifted teacher worsens target predictive fit, so WAIC selects weak borrowing (median ), the displacement remains small, and inclusion stays close to target-only learning (82.1% versus 83.4%); recalibration, which treats the teacher's risk function as known, reaches only 48–49% (W2 / Q1). Under calibration shift, the teacher predicts well but carries a shifted survival level, so WAIC selects borrowing intermediate between the compatible and risk-function-shift regimes yet substantial enough to carry the shifted level, and cross-validation borrows as strongly as under compatibility and yields lower inclusion (66.5% versus 75.6% over ten replicates). This is the case in which predictive accuracy and interval accuracy diverge.
For users, a larger selected indicates greater reliance on the teacher's predictive information, and the target-only fit (), always part of the selection grid, shows how much of the interval reduction is due to borrowing. Because a shifted survival level is reflected in how well predictions are calibrated against observed target outcomes, incorporating censoring-adjusted calibration in the target population into the selection of , alongside predictive fit, is a natural extension.
Q2. Posterior averaging versus teacher guidance. To separate the contributions of posterior averaging and teacher guidance, we consider both the statistical role of the posterior mean and a direct comparison with a single fitted model. BayCSD predicts with the posterior predictive survival probability, , which averages the survival curves implied by the posterior draws (Gelman et al., 2013). This summary has two statistical motivations. First, the posterior mean minimizes the posterior expected squared-error loss, which connects it to the Brier score underlying the IBS (Gneiting and Raftery, 2007). Second, because it averages over the parameter configurations supported by the posterior rather than relying on a single fitted configuration, its predictions are less dependent on any one configuration, which is particularly relevant for a neural risk function learned from few target events (Wilson and Izmailov, 2020). The single fitted model the reviewer describes is the joint MAP of the same generalized objective, which initializes each BayCSD fit (Appendix J); Table R2 compares the two at the same WAIC-selected .
Table R2. Single fitted model (MAP) versus posterior-mean prediction at the WAIC-selected in the teacher-mismatch simulation. Means (SD) over 20 paired replicates.
| C-index (MAP) | C-index (Posterior mean) | IBS (MAP) | IBS (Posterior mean) | |
|---|---|---|---|---|
| Target-only Bayesian PWE | 0.566 (0.034) | 0.611 (0.030) | 0.239 (0.009) | 0.225 (0.009) |
| BayCSD, compatible | 0.762 (0.004) | 0.757 (0.009) | 0.176 (0.004) | 0.179 (0.005) |
| BayCSD, calibration shift | 0.737 (0.062) | 0.752 (0.019) | 0.201 (0.019) | 0.199 (0.007) |
| BayCSD, risk-function shift | 0.609 (0.050) | 0.664 (0.020) | 0.223 (0.013) | 0.210 (0.006) |
The results are consistent with these motivations and show that the gains do not come from averaging alone. Teacher guidance substantially improves the single fitted model itself, most clearly under compatibility (MAP C-index ; IBS ). Posterior averaging adds further accuracy under weak borrowing, namely for target-only learning and under risk-function shift (C-index ; IBS ), and it makes predictive performance more stable across replicates under both mismatch regimes (C-index SD under calibration shift and under risk-function shift). Under strong compatible borrowing, the posterior concentrates and the two predictions nearly coincide. Teacher guidance thus improves the single fitted model itself, while posterior averaging can further improve accuracy and stability when borrowing is weak or the teacher is mismatched; the joint posterior in addition quantifies uncertainty in individualized survival probabilities and RMST.
We thank the reviewer for the constructive comments and for recognizing the practical problem, the design of the controlled comparisons, and the analysis of compatible and mismatched teachers.
W1. How the generalized posterior uses teacher guidance. We thank the reviewer for this helpful suggestion. We agree that the teacher’s role and the student’s training procedure should have been explained together more clearly. The teacher is fitted once, outside the target analysis, and held fixed; BayCSD needs only its survival predictions for the target-training subjects at the comparison endpoints. The student is a Bayesian survival model fitted to the target data, whose posterior (Eq. 7) multiplies the target likelihood by a teacher factor and a structural prior on the student parameters. The target likelihood uses the original event and censoring times, while the teacher factor encourages agreement with the external predictions.
Eq. (8) gives this factor a simple reading. Suppose the teacher predicts that subject i, event-free at τk−1, fails within the next comparison interval with probability 0.1. Assume that this comparison is resolved by the observed follow-up, so Wik=1: the subject is followed to the interval endpoint or experiences an observed event within the interval. With η=10, this comparison contributes the equivalent of one pseudo-event and nine pseudo-non-events for that subject and interval, in addition to the subject's actual observed outcome in the target likelihood. Equivalently, its teacher factor is proportional to pik(ϑ){1−pik(ϑ)}9, favoring student interval probabilities near 0.1 without forcing equality. These are algebraic contributions to the posterior, not additional observed patients or events.
Across included subject–interval comparisons, compatible teacher guidance can add information where sparse target events leave the model poorly determined; because the student's interval probability depends on both the risk function and the baseline hazard (Eq. 6), both components can be stabilized. The weight η sets how many pseudo-observations the teacher contributes and is selected from the target data, with η=0 recovering target-only learning. Joint posterior draws then provide individualized survival predictions and measures of their conditional uncertainty, rather than only a single fitted estimate. We have added this example to Section 3.3.
W2 / Q1. Teacher-only predictions and recalibration. We thank the reviewer for this helpful suggestion. We agree that the added complexity should be justified against both direct teacher use and simple recalibration, rather than by improvement over target-only learning alone.
In Figure 2, the compatible teacher is fitted to 50,000 subjects from the target law, and the calibration-shifted teacher is constructed by multiplying its baseline hazard by 1.5 while retaining its fitted risk function. Both therefore retain the same strong risk ordering. Although the calibration-shifted teacher's absolute survival level is shifted, it remains highly predictive, and the target data accordingly select an intermediate borrowing weight (median η=75, between 100 under compatibility and 5 under risk-function shift). Because BayCSD compares conditional failure probabilities, which depend on both relative risk and absolute survival level, borrowing at this strength transfers the teacher's accurate risk ordering together with its shifted survival level, which the sparse target outcomes only partly correct.
These comparisons should also be read in light of the information each predictor uses and provides. The teacher is fitted to 50,000 source subjects, whereas BayCSD uses 200 target subjects ($\approx$51 events) and only the teacher's survival predictions for them. Used directly, the teacher provides point predictions that neither incorporate the target outcomes nor quantify uncertainty for target subjects. BayCSD learns a target model with posterior uncertainty, improves on target-only learning under all three teachers, and improves on the teacher when its covariate effects are shifted.
As suggested, we added target-data recalibration of the teacher (Appendix J), fitted to the same target cohort with the same continuous-time PWE likelihood, baseline partition and prior as BayCSD: (a) re-estimating the baseline hazard with the teacher log risk as an offset, and (b) additionally estimating a calibration slope on the teacher log risk. Both treat the teacher's risk score as known, so their posterior intervals reflect uncertainty in the baseline hazard (and slope) only. Results are in Table R1.