acceptodds
Under review as a conference paper at ICLR 2027

Post-Hoc Self-Correction for Conditional Calibration of Survival Models

Abstract

We propose a post-hoc procedure that moves a fitted survival model toward conditional calibration by transforming its estimated conditional cumulative distribution function (CDF) separately for each covariate value. The transformation is a nonparametric calibration map built from the conditional D-calibration criterion in survival analysis. At the population level, a single application recovers the true conditional CDF when there is no censoring. Under right censoring, each application reduces the discrepancy from the true conditional CDF by a factor determined by the censoring mechanism, and repeated applications converge to the true conditional CDF under conditions on the time horizon. We estimate the map using kernel weighting in a reduced covariate space. Experiments on ten datasets and six survival models show that a single application of the estimated map improves conditional calibration and the integrated Brier score in most pairwise comparisons with existing post-hoc methods, while maintaining comparable discrimination.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
survival analysis, conditional calibration, post-hoc calibration, D-calibration, right censoring

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uPiYOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper converts the conditional D-calibration criterion into a covariate-dependent recalibration map and composes that map with a fitted survival CDF. Without censoring, the population map recovers the true conditional CDF in one step under strict monotonicity. With conditionally independent censoring and suitable horizon assumptions, the paper derives a geometric contraction in a censoring-weighted L2 norm for repeated population updates. The practical CoSC method estimates the map using kernel weights, optionally after PCA, and selects bandwidth by cross-validated marginal KS-cal. The main experiments use one correction step across ten datasets and six survival models, with comparisons against KSP, CSD, and CSD-iPOT; additional simulations examine iteration, sample size, and projection error.

Critical strengths

S1. The method has a meaningful population interpretation for censored outcomes. The derivation in Appendices B.1–B.2 connects a D-calibration residual map to self-consistency, rather than introducing an unexplained transformation. The contraction factor explicitly relates progress to the censoring CDF over a finite horizon. The paper also acknowledges that the uncensored one-step identity is classical, which helps locate the more interesting contribution in the censored analysis.

S2. The empirical scope and ablations are substantial, and several limitations are reported candidly. Sixty dataset–model combinations, comparisons with a conditional-calibration method, a marginal-map ablation, and simulations with known event and censoring distributions provide a useful body of evidence. The paper does not claim uniform superiority: KS-cal against KSP is evenly split, repeated empirical updates can deteriorate, and PCA has no consistent benefit across metrics. Those qualifications make the reported IBS and Calws gains more credible than an undifferentiated claim of better calibration everywhere.

Critical weaknesses

W1. The population result and the finite-sample estimator are not yet connected by a sufficiently precise theorem. Proposition 2 concerns exact maps conditional on the full covariates and contraction in L2(dG_z). The implemented method uses a projected-space map, a data-selected bandwidth, and one empirical update. The paper acknowledges these distinctions, but the kernel theorem does not by itself establish the accuracy or contraction of that practical estimator.

There are specific gaps to resolve. Lemma 2 gives a uniform bound with a log n factor, whereas Section 4.3 subsequently quotes a rate without that factor. The proof first obtains interior-point bias control, then takes a supremum over sample covariates. Appendix B.4 acknowledges boundary effects, but a vanishing fraction of boundary-near observations does not justify a bound on their maximum error. The discretization over the probability argument also needs an explicit bracketing argument controlling the empirical process between grid points. Finally, the stated bandwidth order is not shown to hold for the bandwidth selected by minimizing cross-validated KS-cal. These issues may be repairable, but the exact theorem supported by the proof should be stated without implying that it automatically covers Algorithm 1.

The PCA approximation argument in Appendix B.5 is particularly restrictive. Condition C2 requires the discarded-component norm to lie within epsilon_n tending to zero with probability tending to one. For a fixed covariate distribution with nondegenerate discarded components, that cannot hold; it would require those components to vanish almost surely, or an explicitly sample-size-dependent distribution. Low explained variance alone is not this condition. The main text correctly admits persistent projection error, so the appendix should retain that error or formulate a clearly identified asymptotic setting rather than suggest that ordinary 90%-variance PCA makes it negligible.

The practical implications also need narrower wording. The contraction is in a censoring-weighted norm, not directly in IBS, pointwise CDF error, or every subgroup-calibration criterion. It does not imply that a single estimated correction improves those quantities. Strictly increasing initial CDFs also require discussion for stepwise or flat predictions such as Cox-based outputs; linear interpolation of MTLR and DeepHit alone does not address every possible flat segment or tail.

W2. The empirical evaluation does not fully separate conditional calibration gains from censoring assumptions, additional local fitting, and retrospective task construction. Appendix A.1 computes IBS with a marginal Kaplan–Meier estimate of the censoring survival function. The theory, however, permits censoring that is independent of event time only conditional on covariates. In that setting, marginal censoring weights need not recover the intended Brier risk. Marginal KM-cal has a related interpretation problem if censoring and event time are dependent after averaging over covariates. This matters because IBS is a major source of claimed superiority. In addition, evaluation up to the maximum observed time requires an explicit support/positivity rule to avoid unstable or zero censoring-survival denominators. Conditional censoring weights, restricted supported horizons, and ground-truth scoring in the existing simulations would make the accuracy claim substantially stronger.

The comparison also needs a local conditional-survival control with similar covariate access and smoothing capacity. A Beran/conditional Kaplan–Meier estimator is an established kernel-based way to estimate conditional survival; see, for example, the treatment of conditional survival estimation in [Ausset et al. (2022)]. A comparison using the same representation and comparable tuning budget would help determine how much comes from the proposed correction of a fitted model rather than additional local estimation on the calibration set. The existing CSD-iPOT comparison is valuable and should be retained; it does not answer this particular attribution question.

Finally, Appendix G.4 constructs the MIMIC-III response as time from ICU admission to discharge and aggregates repeated covariates over the stay. This is not an ordinary baseline prediction task if those covariates include observations after the intended prediction time. It should be explicitly characterized as retrospective length-of-stay modeling, or rebuilt with a prediction-time cutoff. This affects one benchmark rather than negating all results. More generally, the significant-win counts from directional paired tests over overlapping random splits should be supported by paired effect sizes and a multiplicity/dependence sensitivity analysis, rather than treated as independent replications of improved conditional calibration.

Questions for authors

Q1 — W1, theoretical clarification. What exact uniform-estimation statement is justified after accounting for the log factor, boundary points, and interpolation between probability-grid points? Does it apply to a fixed bandwidth, to Algorithm 1's selected bandwidth, or only to a deterministic asymptotic bandwidth sequence? Please distinguish these cases explicitly.

Q2 — W1, projection and implementation. Is C2 intended for a fixed covariate distribution or a triangular-array model in which discarded variances shrink with sample size? If the former, how can it hold for nondegenerate discarded PCs? Can the conclusion instead retain a nonvanishing projection term, and can you specify how flat CDF segments and tails are handled in the implementation?

Q3 — W2, essential scoring evidence. Do the IBS gains persist using conditional censoring weights and a prespecified horizon with adequate censoring support? In the existing synthetic experiments, can you also report ground-truth, unweighted CDF/Brier error and subgroup calibration, rather than only the censoring-weighted norm and D-calibration diagnostics? This would establish whether the claimed improvements survive a scoring procedure aligned with the assumptions.

Q4 — W2, attribution and benchmark clarification. How does one-step CoSC compare with a local conditional-survival estimator using the same representation and comparable tuning budget? For MIMIC-III, what is the prediction time and which measurements enter each patient's covariates? Please also report paired effect-size uncertainty so the robustness of the broad benchmark result can be assessed without relying primarily on significant-win counts.

8 comments

We thank the reviewer for their thoughtful and constructive comments. We respond to each point below and will make all necessary clarifications and corrections in the final version.

Q1. How does Proposition 2 relate to the empirical one-step estimator with PCA and a CV-selected bandwidth?\textsf{Q1. How does Proposition 2 relate to the empirical one-step estimator with PCA and a CV-selected bandwidth?}

Thank you for your helpful comments. We agree that the connection between our theoretical results and the practical algorithm needs clarification. Proposition 2 is stated for a fixed value of the full covariates. The relationship between the original covariates and the kernel approach is outside the scope of this discussion. PCA can be helpful in our datasets because of its potential denoising effect.

The revised appendix considers the theoretical arguments in the PCA-projected space. Proposition 2 can also apply in this space with the corresponding conditions in terms of z~\tilde{z} and Gz~(⋅)G_{\tilde{z}}(\cdot), under the assumptions that Fz≡Fz~F_z \equiv F_{\tilde{z}} and Gz≡Gz~G_z \equiv G_{\tilde{z}}. The latter assumption holds exactly in the signal-plus-noise setting and is approximately satisfied as n→∞n \to \infty under the approximation conditions.

Q2. Can you reconcile the log factor in Lemma 2 with the rate stated in Section 4.3?\textsf{Q2. Can you reconcile the log factor in Lemma 2 with the rate stated in Section 4.3?}

Thank you for pointing this out. Section 4.3 now includes the log⁡n\log{n} factor in the rate stated for the kernel estimator, Op(n−2/(p+4)log⁡n)O_p(n^{-2/(p+4)}\log{n}), consistent with the order in Lemma 2. The local comparison with marginal calibration also includes the factor, giving Op(n−(1−α)/2log⁡n)O_p(n^{-(1-\alpha)/2}\log{n}) when the region has probability of order n−αn^{-\alpha}. The kernel statement concerns the projected calibration target and the deterministic bandwidth order used in the proof. We discuss the scope of the uniform bound in Q3.

Q3. How do you control uniform error near covariate boundaries and between probability grid points?\textsf{Q3. How do you control uniform error near covariate boundaries and between probability grid points?}

Thank you for pointing this out. A sharp supremum-norm bound requires control of the error at all sample points, including those near the boundary. A vanishing proportion of near-boundary points can at most support a sharp bound on the MAE over the sample points.

With a conventional kernel, the bias is larger at points within roughly one bandwidth of the boundary. This region shrinks as the bandwidth tned to zero with n→∞n \to \infty, and the estimation error there can converge to zero more slowly than at interior points. As noted in the remarks, boundary-adapted kernels such as Gasser-Muller and Chen-Gamm kernels can be used near the boundary. We revise the proposition and discuss how to combine conventional and boundary-adapted kernels.

Q4. Does the bandwidth order assumed in the proof apply to Algorithm 1’s CV-selected bandwidth?\textsf{Q4. Does the bandwidth order assumed in the proof apply to Algorithm 1's CV-selected bandwidth?}

Thanks for your comments. Section 4.3 now distinguishes the bandwidth order used in the proof from the rule in Algorithm 1. The proof takes a deterministic sequence hn≍n−1(p+4)h_n \asymp n^{-1(p+4)}. We use this bandwidth order to derive the kernel estimator's convergence rate and to illustrate how borrowing information from nearby covariate profiles can improve estimation in shrinking covariate regions. With a fixed positive bandwidth, a smoothing bias can persist. Algorithm 1 minimizes cross-validated KS-cal over a search space specified independently of the theoretical scaling. Establishing a rate for the selected h^\hat{h} requires a separate analysis of that selection procedure. Appendix C shows the cross-validation loss curves and the selected bandwidths (Figures 3 and 4).

Q5. When does Condition C2 hold, and how do you account for persistent PCA projection error?\textsf{Q5. When does Condition C2 hold, and how do you account for persistent PCA projection error?}

We have revised Appendix B.5 to clarify the role of PCA. Appendix B.5.1 uses a signal-plus-noise model to illustrate how PCA can reduce noise. Appendix B.5.2 complements this discussion by anlyzing how the projected calibration map approximates the full-covariate map under conditions C1 and C2, with C2 imposing a strong asymptotic concentration requirement on the discarded components.

Consider a signal-plus-noise modelk for the centered covariates, z=S+e\boldsymbol{z} = \boldsymbol{S} + \boldsymbol{e}, where the signal S\boldsymbol{S} and the noise e\boldsymbol{e} are independent, and e∼Nd(0,σ2Id)\boldsymbol{e} \sim N_d(\boldsymbol{0}, \sigma^2\boldsymbol{I}_d). The covariance matrices of S\boldsymbol{S} and z\boldsymbol{z} share the same eigenvectors, and isotropic noise increases each eigenvalue by $\sigma^2%. Their spectral decompositions are

ΣS=Pdiag⁡(λ1,...,λd)P⊤\Sigma_{\boldsymbol{S}} = \boldsymbol{P} \operatorname{diag}(\lambda_1,...,\lambda_d) \boldsymbol{P}^{\top}

Σz=Pdiag⁡(λ1+σ2,...,λd+σ2)P⊤\Sigma_{\boldsymbol{z}} = \boldsymbol{P} \operatorname{diag}(\lambda_1+\sigma^2,...,\lambda_d+\sigma^2) \boldsymbol{P}^{\top}

where P=[u1,...,ud]\boldsymbol{P} = [\boldsymbol{u}_1,...,\boldsymbol{u}_d] and λ1≥⋯≥λd≥0\lambda_1 \ge \cdots \ge \lambda_d \ge 0.

Suppose the signal covariance has rank pp, so that λi=0\lambda_i = 0 for i>pi > p, and that λp>σ2\lambda_p > \sigma^2. The last d−pd-p eigenvalues of Σz\Sigma_{\boldsymbol{z}} are then equal to the noise variance σ2\sigma^2, and the leading pp eigenvectors span the signal subspace.

Let P1:p=[u1,...,up]\boldsymbol{P}_{1:p} = [\boldsymbol{u}_1,...,\boldsymbol{u}_p]. The principal component scores are P⊤z\boldsymbol{P}^{\top}\boldsymbol{z}. Reconstructing z\boldsymbol{z} from the leading pp scores gives P1:pP1:p⊤z\boldsymbol{P}_{1:p} \boldsymbol{P}_{1:p}^{\top} \boldsymbol{z}. The corresponding mean squared reconstruction error is

E[∥P1:pP1:p⊤z−S∥2]=σ2tr⁡(P1:pP1:p⊤)=pσ2,\mathbb{E}\left[\left\lVert \boldsymbol{P}_{1:p} \boldsymbol{P}_{1:p}^{\top} \boldsymbol{z} - \boldsymbol{S} \right\rVert^2\right] = \sigma^2 \operatorname{tr}\left(\boldsymbol{P}_{1:p} \boldsymbol{P}_{1:p}^{\top}\right) = p\sigma^2,

compared with E[∥z−S∥2]=dσ2\mathbb{E}\left[\left\lVert \boldsymbol{z} - \boldsymbol{S} \right\rVert^2\right] = d\sigma^2 for the original covariates. The lower mean squared reconstruction error shows that PCA can reduce the effect of noise and provide a more accurate approximation of the underlying signal S\boldsymbol{S} in this setting.

In practice, with a learning algorithm that can represent linear functions, we can use either the reconstructed covariates P1:pP1:p⊤z\boldsymbol{P}_{1:p} \boldsymbol{P}_{1:p}^{\top} \boldsymbol{z} or the retained scores P1:p⊤z\boldsymbol{P}_{1:p}^{\top} \boldsymbol{z}. The theoretical results give function-approximation rates under their stated assumptions. Denoising can help reveal the association between the signal and the response. The quantity λp−σ2\lambda_p - \sigma^2 measures how much the weakest retained signal variance exceeds the noise variance.

Q6. The practical implications need narrower wording.\textsf{Q6. The practical implications need narrower wording.}

We agree. Proposition 2 is a population-level statement about the exact maps in the norm L2([0,τ],Gz)L^2([0,\tau], G_z). It does not imply a reduction in IBS, in the pointwise CDF error, or in subgroup calibration criteria, and it does not cover a single estimated correction. We have narrowed the wording accordingly.

∙Theoretical statements now name the norm.\bullet \textsf{Theoretical statements now name the norm.} The abstract, the second contribution, and the Conclusion previously said that the correction "reduces the discrepancy" or "converges" without qualification. They now state convergence "in a censoring-weighted L2L_2 norm".

∙The scope is stated next to Proposition 2 (Section 4.2).\bullet \textsf{The scope is stated next to Proposition 2 (Section 4.2).} New text: "This guarantee concerns the exact population-level maps Hz(k)H_z^{(k)} and the norm L2([0,τ],Gz)L^2([0,\tau], G_z). It does not by itself imply a reduction in the integrated Brier score, in the pointwise error of the CDF, or in subgroup calibration criteria, in particular for a single estimated correction; we assess these quantities empirically in Section 5."

∙Empirical claims are labeled as empirical.\bullet \textsf{Empirical claims are labeled as empirical.} The third contribution and the Conclusion now present the gains in IBS and conditional calibration of the one-step estimator as empirical findings in most pairwise comparisons.

What we retain from the theory is limited to the following: since GzG_z is strictly increasing on [0,τ][0,\tau] (Assumption 1(b)), the norm vanishes only if F^z=Fz\hat{F}_z = F_z on [0,τ][0, \tau], so the population iteration has the correct limit.

Q7, How do you handle stepwise CDFs, flat segments, and tails under the strict monotonicity assumption?\textsf{Q7, How do you handle stepwise CDFs, flat segments, and tails under the strict monotonicity assumption?}

Thank you for pointing this out. We adjust the initial CDFs from MTLR, DeepHit, and SurvivalBoost to address monotonicity. For DeepSurv, we use the Breslow CDF directly.

∙MTLR and DeepHit.\bullet \textsf{MTLR and DeepHit.} We interpolate the MTLR and DeepHit CDFs between adjacent prediction times (Section 5.1 and Appendices F.2 and F.5). Interpolation is strictly increasing between unequal adjacent CDF values and flat over tied values. A constant extension beyond the prediction grid has flat tail.

∙SurvivalBoost.\bullet \textsf{SurvivalBoost.} For SurvivalBoost, isotonic regression yields a nondecreasing CDF and can create flat segments (Appendix F.7).

∙DeepSurv.\bullet \textsf{DeepSurv.} Appendix F.1 describes the DeepSurv CDF obtained using the Breslow estimator. It increases at distinct observed event times and is flat between them. On a fixed interval where every subinterval of positive length has positive probability of containing an observed event, the longest flat segment tends to zero in probability as the training sample grows.

∙Flat segments under CoSC.\bullet \textsf{Flat segments under CoSC.} For a fixed query, composing the initial prediction with the CoSC map preserves a flat segment wherever the input CDF is constant. Strict monotonicity remains a population-level assumption.

Q8. What censoring assumptions justify marginal KM weights for IBS and the pooled KM reference for KM-cal?\textsf{Q8. What censoring assumptions justify marginal KM weights for IBS and the pooled KM reference for KM-cal?}

Thank you for raising this point. We recomputed IBS using conditional IPCW estimated by Cox regression and a random survival forest. Both schemes yielded nearly the same head-to-head comparisons as marginal Kaplan-Meier weighting, and CoSC retained its IBS advantage under both conditional schemes.

∙Censoring assumptions behind the metrics.\bullet \textsf{Censoring assumptions behind the metrics.} Appendix A.1 now distinguishes the assumptions behind KM-cal and IBS. KM-cal measures agreement with the pooled Kaplan-Meier reference, which estimates marginal event-time survival under marginal independent censoring. For IBS, conditional IPCW estimates the Brier risk under conditional independence when censoring survival is consistently estimated and positive over the evaluation horizon. Independence of censoring from both the event time and the covariates is a sufficient condition for marginal IPCW.

∙IBS under conditional censoring weights.\bullet \textsf{IBS under conditional censoring weights.} Appendix A.4 (Table 5) compares marginal Kaplan-Meier IPCW with conditional IPCW estimated by Cox regression and a random survival forest, treating censoring as the event. All three schemes select the same best method in all 70 dataset-model combinations, with CoSC first in 64. The full ranking obtained with marginal weights agrees with that obtained with Cox weights in 57 combinations and with RSF weights in 54.

∙Pairwise outcomes.\bullet \textsf{Pairwise outcomes.} For each conditional scheme, 279 of CoSC's 280 head-to-head outcomes agree with those under under marginal weights. The only reversal is against CSD-iPOT for DeepHit on SEER-stomach, where the absolute mean IBS gap is below 3×10−53\times 10^{-5} under all three schemes. For SurvivalBoost, CoSC improves on the uncalibrated model on all ten datasets and ranks first on eight under each weighting scheme. The common horizon in Appendix A.4 also imposes the support rule on both conditional censoring estimates (see Q9), which explains the different win counts from the main analysis.

Q9. How do you restrict the IBS horizon to ensure censoring support, and do the gains persist with conditional IPCW and ground-truth scoring?\textsf{Q9. How do you restrict the IBS horizon to ensure censoring support, and do the gains persist with conditional IPCW and ground-truth scoring?}

Thank you for the suggestion. We revised the IBS evaluation horizon to account for censoring support. We have added True IBS and Calws\mathrm{Cal}_{\mathrm{ws}} results to Appendix D based on the simulation repetitions completed to date.

∙Restricting the IBS evaluation horizon.\bullet \textsf{Restricting the IBS evaluation horizon.} Appendix A.1 sets the main IBS horizon η\eta to the minimum of the maximum observed time and the latest evaluation time at which the estimated marginal censoring survival is at least 0.01. For the sensitivity analysis in Appendix A.4, we choose one horizon per split. It is the latest grid time at which marginal KM censoring survival and both conditional estimates for every test subject are at least 0.01. All prediction methods and weighting schemes in that analysis use this same horizon.

∙Ground-truth scoring and subgroup calibration in the simulations.\bullet \textsf{Ground-truth scoring and subgroup calibration in the simulations.} Appendix D.1 gives the expected Brier score at time tt using the known conditional event-time CDF FzF_z,

F^z(t)−Fz(t)2+Fz(t)1−Fz(t)\\{\hat{F}_z(t) - F_z(t)\\}^2 + F_z(t)\\{1 - F_z(t)\\}.

We compute True IBS by averaging this score over the test covariates and integrating it over the evaluation horizon, normalized by the length of that horizon. On a common test set and horizon, differences in True IBS equal differences in time-averaged squared CDF error. Proposition 2 establishes population-level contraction in the censoring-weighted L2(dGz)L^2(dG_z) norm. The simulations in Appendix D.2 also show that True IBS often decreases after the first CoSC update, with further reductions for several models at 90% censoring. Calws\mathrm{Cal}_{\mathrm{ws}} often decreases after the first update and varies more over subsequent iterations.

Q10. How does CoSC compare with Beran using the same covariate representation and a comparable tuning budget?\textsf{Q10. How does CoSC compare with Beran using the same covariate representation and a comparable tuning budget?}

We fitted the Beran estimator and compared it with one-step CoSC. CoSC achieved lower IBS and Calws\mathrm{Cal}_{\mathrm{ws}} in most comparisons, suggesting that self-correction of a fitted survival model can improve on direct kernel estimation of conditional survival.

∙Setup of the comparison.\bullet \textsf{Setup of the comparison.} Appendix I compared CoSC with the Beran conditional Kaplan-Meier estimator [1]. In each split, we fit Beran on the validation set using the same PCA representation and Laplacian kernel as CoSC. We choose the bandwidth by minimizing cross-validated KS-cal using the same procedure (Appendix C). Beran uses the resulting kernel weights to estimate conditional survival directly from observed outcomes. CoSC uses kernel weights to correct predictions from a fitted survival model.

[1] Beran (1981). Nonparametric Regression with Randomly Censored Survival Data.

Reviewer_4jbLOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This work highlights the conditional D-calibration as an important metric in survival analysis, and proposes a post-hoc procedure to recalibrate survival predictions. They propose a theoretical results with and without censoring, and an empirical evaluation of the proposed method.

Critical strengths

The theory seems well established, and the experiments are strong, and support an empirical gain of the proposed approach.

Critical weaknesses

Although well-written, the article could be more pedagogical, especially with examples illustrating the mentioned situation that the model is calibrated at the population level, but that conditional calibration could highlight subgroups or variables with poor calibration. Also, the limitations of the previous methods overcome by the proposed approach are not very clear. Finally, the experiments could be completed by a tree-based baseline, like random survival forests or survivalboost.

Questions for authors
  • can you clarify the position of the proposed method with respect to the existing methods more clearly?
  • can you add a tree-based baseline?
  • can you provide (toy) examples of the case for using conditional calibration because of limitations of population calibration?
3 comments

We thank the reviewer for their thoughtful and constructive comments. We respond to each point below and will make all necessary clarifications and corrections in the final version.

Q1. How does CoSC relates to existing post-processing methods?\textsf{Q1. How does CoSC relates to existing post-processing methods?}

Thank you for the comment. We revised Section 2 to position CoSC among methods that recalibrate full survival distributions.

∙Relation to marginal recalibration.\bullet \textsf{Relation to marginal recalibration.} CSD adjusts predicted quantile times, and KSP learns a shared probability-scale transformation by minimizing KS-cal.

∙Relation to CSD-iPOT.\bullet \textsf{Relation to CSD-iPOT.} CSD-iPOT also targets conditional distribution calibration and uses pooled conformity scores to construct a common probability-level adjustment. Its asymptotic conditional calibration result assumes a consistent initial survival estimator ([1], Theorem 3.2).

∙Position of CoSC.\bullet \textsf{Position of CoSC.} CoSC uses the conditional D-calibration criterion to estimate a separate correction at each query covariate profile. In the uncensored population setting, the map recovers the true conditional CDF in one step from a strictly increasing initial CDF, even when the initial CDF is miscalibrated. For right-censored data, Proposition 2 characterizes repeated population updates in a censoring-weighted L2L_2 norm.

Q2. Can you add a tree-based baseline?\textsf{Q2. Can you add a tree-based baseline?}

Thank you for the suggestion.

∙A tree-based model is added.\bullet \textsf{A tree-based model is added.} We added SurvivalBoost [2], a gradient-boosted tree model, as the seventh survival model in the main benchmark. Appendix F.7 describes the model and the isotonic regression used to obtain nondecreasing initial CDFs. The main benchmark now includes 70 dataset-model combinations. Across these combinations, CoSC achieves lower IBS in 56-69 comparisons per comparator and lower Calws\mathrm{Cal}_{\mathrm{ws}} in 51-63 (Table 1).

∙Results on SurvivalBoost.\bullet \textsf{Results on SurvivalBoost.} CoSC lowers KS-cal Calws\mathrm{Cal}_{\mathrm{ws}} relative to the baseline on all ten datasets and KM-cal on nine. The unrounded mean IBS is lower on all ten, with both values rounding to 0.0266 on MIMIC-III (Table 18). The median IBS decreases from 0.1362 to 0.1346, and the median Calws\mathrm{Cal}_{\mathrm{ws}} decreases from 0.000921 to 0.000687 (Table 3). CoSC has the lowest median PSR, KSP leads on KS-cal, KM-cal, and Calws\mathrm{Cal}_{\mathrm{ws}}, and CSD has the highest median C-index and the lowest median IBS (0.1334).

Q3. Can you give a toy example where marginal calibration masks subgroup miscalibration?\textsf{Q3. Can you give a toy example where marginal calibration masks subgroup miscalibration?}

Thank you for the suggestion. The revised Figure 1 and Section 4.1 show a two-group example. A shared marginal correction matches the pooled survival curve closely because overestimation in one group and underestimation in the other offset each other. A covariate-dependent correction brings each group's mean prediction closer to its own reference. The pooled and group-specific panels display these two aspects of calibration separately.

[1] Qi et al. (2024). Toward Conditional Distribution Calibration in Survival Prediction. NeurIPS.

[2] Alberge et al. (2025). Survival Models: Proper Scoring Rule and Stochastic Optimization with Competing Risks. PMLR.

Dear Reviewer,

Thank you for your thoughtful review and contribution to the discussion.

As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.

In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.

Thank you again for your time and effort!

The additions have strengthened the paper so I maintain my good rating

Reviewer_fJptOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a post-hoc calibration procedure for time to event models that transforms the conditional cumulative distribution function (CDF) for each covariate against conditional D-calibration. The proposed calibration approach recovers the true conditional CDF under no censoring and reduces the discrepancy to the true CDF by a factor as a function of the censoring mechanism. Experiments on 10 datasets and 6 survival models show that the post-hoc approach improves conditional calibration and IBS relative to most existing methods without sacrificing discrimination.

Critical strengths

The theoretical results are well motivated, described and illustrated empirically in the Appendix, especially those in Section 4.3 concerning the kernel estimator.

The experiments are substantial, convincing and provided with sufficient detail in the Appendix. The statistics in the empirical results are sound, cover multiple metrics including computational cost and ablation studies.

Critical weaknesses

Although it is clear why some of the details of theoretical results are relegated to the Appendix, making space for the assumptions in exchange for some of the background details is likely to improve the presentation.

The space in the experiments section can be used more effectively. For example, Table 3 basically shows that the proposed model is relatively insensitive to sensible hyperparameter choices, which can be easily moved to the Appendix to make room for additional details and results shown in the Appendix.

The theory predicts improvements for multiple correction rounds specially when the censoring mechanism is not simple, however, results are only shown for one correction round no experiments illustrate the impact of censoring or the censoring mechanism on performance.

Questions for authors

Why call the expression in Section 4.1 conditional D-calibration instead of conditional KS-calibration?

Reconsider the contents and caption of Figure 1 that as presented is not terribly informative.

Can the authors comment on the significant differences between the proposed method and CSD-iPOT considering that such differences are not immediately obvious when looking at the calibration plots and performance tables in the Appendix?

3 comments

We thank the reviewer for their thoughtful and constructive comments. We respond to each point below and will make all necessary clarifications and corrections in the final version.

Q1. Can you state the key theoretical assumptions in the main text?\textsf{Q1. Can you state the key theoretical assumptions in the main text?}

Thank you for the suggestion. We agree: the assumptions for the population-level results are now stated in the main text (Assumption 1, Section 4.2), and we shortened the background in Section 3 to make room. Specifically,

∙The assumptions for the population-level results are now stated in the main text.\bullet \textsf{The assumptions for the population-level results are now stated in the main text.} We moved them into Assumption 1 in Section 4.2. They specify conditionally independent censoring, continuity and strict monotonicity of Fz,GzF_z, G_z, and F^z\hat{F}_z on [0,τ][0,\tau], and the endpoint conditions P(Y>τ∣Z=z)>0\mathbb{P}(Y>\tau|Z=z)>0 and F^z(τ)<1\hat{F}_z(\tau)<1. Proposition 2 now refers directly to Assumption 1. The endpoint conditions give the strict contraction factor Gz(τ)<1G_z(\tau)<1.

∙Background is shortened to make room.\bullet \textsf{Background is shortened to make room.} We moved the hazard definition to Appendix F and the C-index definition and interval form of D-calibration to Appendix A.1.

∙Regularity conditions for the kernel estimator remain in the appendix.\bullet \textsf{Regularity conditions for the kernel estimator remain in the appendix.} They remain with the analysis in Appendices B.3-B.4.

Q2. Can you move the ablation table to the appendix to make room for other results?\textsf{Q2. Can you move the ablation table to the appendix to make room for other results?}

Thank you for the suggestion. We agree: the ablation and runtime tables are moved to the appendix, and Section 5.2 now reports median performance overall and by dataset size (new Table 2). Specifically,

∙The ablation results are moved to Appendix A.3 (Table 4).\bullet \textsf{The ablation results are moved to Appendix A.3 (Table 4).} Section 5.1 states the default settings, and Appendix C gives the bandwidth-selection algorithm.

∙The runtime results are moved to Appendix J (Table 32-33).\bullet \textsf{The runtime results are moved to Appendix J (Table 32-33).} CoSC's median runtime is 5.46 seconds overall and ranges from 5.32 to 6.25 seconds across the size groups.

∙The space newly available shows median performance (Table 2, Section 5.2).\bullet \textsf{The space newly available shows median performance (Table 2, Section 5.2).} The new Table 2 reports median performance overall and by dataset size. CoSC has the best median performance on five metrics both overall and in the Large group. CSD has the highest median C-index in both summaries. In the Medium group, CoSC leads on C-index, KM-cal, Calws\mathrm{Cal}_{\mathrm{ws}} and KSP leads on PSR, KS-cal, and IBS. In the Small group, KSP or CSD has the best median on every metric, and CoSC matches CSD on IBS at the reported precision.

Q3. How does repeated correction perform across censoring rates and mechanisms?\textsf{Q3. How does repeated correction perform across censoring rates and mechanisms?}

Thank you for the comment. We evaluated repeated correction on real data. Across 60 dataset-model combinations (still progressing in SurvivalBoost), multi-step CoSC lowered KS-cal and Calws\mathrm{Cal}_{\mathrm{ws}} in 44 combinations each and KM-cal in 42. It also lowered the C-index in 40 combinations and increased median runtime from 5.44 to 22.89 seconds. Given this trade-off between calibration, discrimination, and runtime, the real-data results support using a single correction step. We also added a lognormal censoring simulation in Appendix D.5. At matched target censoring rates and calibration set sizes, the correction trajectories are broadly similar to those under Weibull censoring.

∙We evaluated multiple corrections on real data.\bullet \textsf{We evaluated multiple corrections on real data.} Appendix I (Tables 22-31) compares one-step and multi-step CoSC on ten datasets and six survival models. At each iteration, we reselect the bandwidth using cross-validated KS-cal and use the updated predictions as input to the next iteration. We stop at the first iteration that fails to lower cross-validated KS-cal and evaluate the preceding predictions. Across the 60 combinations, the average number of retained corrections is 3.14.

∙Repeated correction improves calibration at a cost in PSR and runtime.\bullet \textsf{Repeated correction improves calibration at a cost in PSR and runtime.} Table 1 below summarizes the results. Repeated correction lowers KS-cal and Calws\mathrm{Cal}_{\mathrm{ws}} in 44 of 60 combinations each. The median paired IBS reduction is 0.0001 at the displayed precision, and one-step CoSC has lower PSR in 59 combinations. Repeated correction increases median runtime from 5.44 to 22.89 seconds. This trade-off motivates using one step in the main experiments. Multi-step results for SurvivalBoost are in progress.

Table 1. Multi-step versus one-step CoSC across 60 dataset-model combinations.

\hline & \text{C-index} & \text{PSR} & \text{KS-cal} & \text{KM-cal} & \text{IBS} & \mathrm{Cal}_{\mathrm{ws}} \\\\ \hline \text{Better / worse / tied} & 17/40/3 & 1/59/0 & 44/16/0 & 42/17/1 & 33/19/8 & 44/16/0 \\\\ \hline \end{array}$$

∙Impact of the censoring rate and distribution.\bullet \textsf{Impact of the censoring rate and distribution.} Appendix D.2 uses covariate-dependent Weibull censoring, and Appendix D.5 (Figures 26-32) uses covariate-dependent lognormal censoring. Both simulations use the same event-time model and target censoring rates of 30%, 60%, and 90%. At 30% and 60% censoring, reductions in KS-cal and True IBS often occur in the first few updates. At 90% censoring, further updates generally reduce True IBS for MTLR, S-CRPS, DeepHit, and SurvivalBoost under both distributions. The Calws\mathrm{Cal}_{\mathrm{ws}} trajectories differ in some settings. For example, for AFT at 90% censoring with ncal∈500,1000n_{cal} \in \\{500, 1000\\}, the final Calws\mathrm{Cal}_{\mathrm{ws}} is below its initial value under Weibull censoring and above it under lognormal censoring.

Q4, Why call the criterion conditional D-calibration, and how does it relate to conditional KS-calibration?\textsf{Q4, Why call the criterion conditional D-calibration, and how does it relate to conditional KS-calibration?}

Thank you for the question. "D-calibration" is the property\textit{property} (uniformity of F^Z(T)\hat{F}_Z(T) under censoring) and "KS-cal" is a metric\textit{metric} that measures its violation by a supremum over xx. Section 4.1 defines a property, with no supremum or norm, so we call it conditional D-calibration; this matches "conditional distribution calibration" in [1]. The [0,x][0, x] form used there is equivalent to the usual interval form of D-calibration, since the criterion for [a,b][a, b] is the difference of the criteria at x=bx=b and x=ax=a. The misleading phrase at the end of Section 3.2 ("its conditional counterpart") is corrected, and the equivalence is added to Appendix A.1.

Q5. Reconsider the contents and caption of Figure 1 that as presented is not terribly informative.\textsf{Q5. Reconsider the contents and caption of Figure 1 that as presented is not terribly informative.}

Thank you for the suggestion. We have redesigned Figure 1 and revised its caption.

∙The revised figure shows how errors within groups can cancel in the population average.\bullet \textsf{The revised figure shows how errors within groups can cancel in the population average.} Both rows start from the same uncalibrated predictions. After marginal calibration (top row), the overall mean prediction closely matches the reference curve because overestimation in Group A and underestimation in Group B offset one another. The group-dependent (bottom row) brings the mean predictions closer to the reference curves within both groups and maintains agreement at the population level.

∙The revised caption explains the panel layout and curve types.\bullet \textsf{The revised caption explains the panel layout and curve types.} It identifies the baseline and corrected predictions and explains how the two rightmost columns compare mean predicted survival with reference survival, overall and within each group.

Q6. How does CoSC differ from CSD-iPOT, and how are those differences reflected in the results?\textsf{Q6. How does CoSC differ from CSD-iPOT, and how are those differences reflected in the results?}

Thank you for the comment. We agree that the calibration plots of CSD-iPOT and CoSC are visually similar in many settings. We will clarify the methodological distinction and discuss how to interpret these plots.

∙CoSC estimates a covariate-dependent recalibration map.\bullet \textsf{CoSC estimates a covariate-dependent recalibration map.} Section 2 now clarifies how the corrections differ. CSD-iPOT pools conformity scores to estimate a shared probability-scale adjustment. CoSC estimates a map at each query covariate profile using kernel weights in the projected space. Subjects with the same predicted probability can therefore receive different corrections.

∙The calibration plots show calibration after pooling.\bullet \textsf{The calibration plots show calibration after pooling.} The curves in Appendix E summarize calibration after pooling across covariate profiles. The curves are often close, with small differences at some tail probabilities, such as for S-CRPS on NB-SEQ (Figure 37).

∙The quantitative comparison captures differences across settings.\bullet \textsf{The quantitative comparison captures differences across settings.} The worst-slab metric Calws\mathrm{Cal}_{\mathrm{ws}} evaluates calibration within subregions of the covariate space. Across the 70 dataset-model combinations, CoSC has lower IBS in 68 and lower Calws\mathrm{Cal}_{\mathrm{ws}} in 56 compared with CSD-iPOT (Table 1 in the main manuscript). The overall medians for CoSC and CSD-iPOT are 0.1343 and 0.1381 for IBS, and 0.001059 and 0.001202 for Calws\mathrm{Cal}_{\mathrm{ws}} (Table 2). After the Nadeau-Bengio correction [2], 56 IBS wins and 6 Calws\mathrm{Cal}_{\mathrm{ws}} wins reach the nominal one-sided threshold of 0.05 (Appendix A.5 and our response to uPiY Q12).

[1] Qi et al. (2024). Toward Conditional Distribution Calibration in Survival Prediction. NeurIPS.

[2] Claude Nadeau and Yoshua Bengio (1999). Inference for the generalization error. NeurIPS.

Dear Reviewer,

Thank you for your thoughtful review and contribution to the discussion.

As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.

In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.

Thank you again for your time and effort!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response

We thank the reviewers for their constructive feedback. The revised manuscript marks changes in blue and tags them by reviewer and question number. We highlight four changes.

∙Benchmark.\bullet \textsf{Benchmark.} We add SurvivalBoost as a seventh survival model, bringing the main benchmark to 70 dataset-model combinations. (4jbL Q2)

∙Theory.\bullet \textsf{Theory.} We state the population assumptions in Section 4.2, clarify the scope of the contraction result, and include the log factor in the kernel rate (fJpt Q1 and uPiY Q1-Q6).

∙Figure 1.\bullet \textsf{Figure 1.} The revised two-group example shows how calibration errors can cancel after pooling (fJpt Q5 and 4jbL Q3).

∙Additional analyses.\bullet \textsf{Additional analyses.} We compare three IPCW weighting scheme, assess sensitivity to overlapping splits, and compare one-step CoSC with repeated correction and the Beran estimator (Appendices A.4, A.5, and I).

The main benchmark, the IPCW and split-overlap analyses, and the Beran comparison include SurvivalBoost. We are extending the ablation, and multi-step comparison to cover SurvivalBoost. Appendix D.1 specifies the ground-truth Brier score. Work in progress in Appendix D (results to date are included) includes extending the simulations to SurvivalBoost, reporting True IBS and Calws\mathrm{Cal}_{\mathrm{ws}}, and testing a different censoring family.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.