- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper converts the conditional D-calibration criterion into a covariate-dependent recalibration map and composes that map with a fitted survival CDF. Without censoring, the population map recovers the true conditional CDF in one step under strict monotonicity. With conditionally independent censoring and suitable horizon assumptions, the paper derives a geometric contraction in a censoring-weighted L2 norm for repeated population updates. The practical CoSC method estimates the map using kernel weights, optionally after PCA, and selects bandwidth by cross-validated marginal KS-cal. The main experiments use one correction step across ten datasets and six survival models, with comparisons against KSP, CSD, and CSD-iPOT; additional simulations examine iteration, sample size, and projection error.
Critical strengths
S1. The method has a meaningful population interpretation for censored outcomes. The derivation in Appendices B.1–B.2 connects a D-calibration residual map to self-consistency, rather than introducing an unexplained transformation. The contraction factor explicitly relates progress to the censoring CDF over a finite horizon. The paper also acknowledges that the uncensored one-step identity is classical, which helps locate the more interesting contribution in the censored analysis.
S2. The empirical scope and ablations are substantial, and several limitations are reported candidly. Sixty dataset–model combinations, comparisons with a conditional-calibration method, a marginal-map ablation, and simulations with known event and censoring distributions provide a useful body of evidence. The paper does not claim uniform superiority: KS-cal against KSP is evenly split, repeated empirical updates can deteriorate, and PCA has no consistent benefit across metrics. Those qualifications make the reported IBS and Calws gains more credible than an undifferentiated claim of better calibration everywhere.
Critical weaknesses
W1. The population result and the finite-sample estimator are not yet connected by a sufficiently precise theorem. Proposition 2 concerns exact maps conditional on the full covariates and contraction in L2(dG_z). The implemented method uses a projected-space map, a data-selected bandwidth, and one empirical update. The paper acknowledges these distinctions, but the kernel theorem does not by itself establish the accuracy or contraction of that practical estimator.
There are specific gaps to resolve. Lemma 2 gives a uniform bound with a log n factor, whereas Section 4.3 subsequently quotes a rate without that factor. The proof first obtains interior-point bias control, then takes a supremum over sample covariates. Appendix B.4 acknowledges boundary effects, but a vanishing fraction of boundary-near observations does not justify a bound on their maximum error. The discretization over the probability argument also needs an explicit bracketing argument controlling the empirical process between grid points. Finally, the stated bandwidth order is not shown to hold for the bandwidth selected by minimizing cross-validated KS-cal. These issues may be repairable, but the exact theorem supported by the proof should be stated without implying that it automatically covers Algorithm 1.
The PCA approximation argument in Appendix B.5 is particularly restrictive. Condition C2 requires the discarded-component norm to lie within epsilon_n tending to zero with probability tending to one. For a fixed covariate distribution with nondegenerate discarded components, that cannot hold; it would require those components to vanish almost surely, or an explicitly sample-size-dependent distribution. Low explained variance alone is not this condition. The main text correctly admits persistent projection error, so the appendix should retain that error or formulate a clearly identified asymptotic setting rather than suggest that ordinary 90%-variance PCA makes it negligible.
The practical implications also need narrower wording. The contraction is in a censoring-weighted norm, not directly in IBS, pointwise CDF error, or every subgroup-calibration criterion. It does not imply that a single estimated correction improves those quantities. Strictly increasing initial CDFs also require discussion for stepwise or flat predictions such as Cox-based outputs; linear interpolation of MTLR and DeepHit alone does not address every possible flat segment or tail.
W2. The empirical evaluation does not fully separate conditional calibration gains from censoring assumptions, additional local fitting, and retrospective task construction. Appendix A.1 computes IBS with a marginal Kaplan–Meier estimate of the censoring survival function. The theory, however, permits censoring that is independent of event time only conditional on covariates. In that setting, marginal censoring weights need not recover the intended Brier risk. Marginal KM-cal has a related interpretation problem if censoring and event time are dependent after averaging over covariates. This matters because IBS is a major source of claimed superiority. In addition, evaluation up to the maximum observed time requires an explicit support/positivity rule to avoid unstable or zero censoring-survival denominators. Conditional censoring weights, restricted supported horizons, and ground-truth scoring in the existing simulations would make the accuracy claim substantially stronger.
The comparison also needs a local conditional-survival control with similar covariate access and smoothing capacity. A Beran/conditional Kaplan–Meier estimator is an established kernel-based way to estimate conditional survival; see, for example, the treatment of conditional survival estimation in [Ausset et al. (2022)]. A comparison using the same representation and comparable tuning budget would help determine how much comes from the proposed correction of a fitted model rather than additional local estimation on the calibration set. The existing CSD-iPOT comparison is valuable and should be retained; it does not answer this particular attribution question.
Finally, Appendix G.4 constructs the MIMIC-III response as time from ICU admission to discharge and aggregates repeated covariates over the stay. This is not an ordinary baseline prediction task if those covariates include observations after the intended prediction time. It should be explicitly characterized as retrospective length-of-stay modeling, or rebuilt with a prediction-time cutoff. This affects one benchmark rather than negating all results. More generally, the significant-win counts from directional paired tests over overlapping random splits should be supported by paired effect sizes and a multiplicity/dependence sensitivity analysis, rather than treated as independent replications of improved conditional calibration.
Questions for authors
Q1 — W1, theoretical clarification. What exact uniform-estimation statement is justified after accounting for the log factor, boundary points, and interpolation between probability-grid points? Does it apply to a fixed bandwidth, to Algorithm 1's selected bandwidth, or only to a deterministic asymptotic bandwidth sequence? Please distinguish these cases explicitly.
Q2 — W1, projection and implementation. Is C2 intended for a fixed covariate distribution or a triangular-array model in which discarded variances shrink with sample size? If the former, how can it hold for nondegenerate discarded PCs? Can the conclusion instead retain a nonvanishing projection term, and can you specify how flat CDF segments and tails are handled in the implementation?
Q3 — W2, essential scoring evidence. Do the IBS gains persist using conditional censoring weights and a prespecified horizon with adequate censoring support? In the existing synthetic experiments, can you also report ground-truth, unweighted CDF/Brier error and subgroup calibration, rather than only the censoring-weighted norm and D-calibration diagnostics? This would establish whether the claimed improvements survive a scoring procedure aligned with the assumptions.
Q4 — W2, attribution and benchmark clarification. How does one-step CoSC compare with a local conditional-survival estimator using the same representation and comparable tuning budget? For MIMIC-III, what is the prediction time and which measurements enter each patient's covariates? Please also report paired effect-size uncertainty so the robustness of the broad benchmark result can be assessed without relying primarily on significant-win counts.
8 comments
Suppose the signal covariance has rank , so that for , and that . The last eigenvalues of are then equal to the noise variance , and the leading eigenvectors span the signal subspace.
Let . The principal component scores are . Reconstructing from the leading scores gives . The corresponding mean squared reconstruction error is
compared with for the original covariates. The lower mean squared reconstruction error shows that PCA can reduce the effect of noise and provide a more accurate approximation of the underlying signal in this setting.
In practice, with a learning algorithm that can represent linear functions, we can use either the reconstructed covariates or the retained scores . The theoretical results give function-approximation rates under their stated assumptions. Denoising can help reveal the association between the signal and the response. The quantity measures how much the weakest retained signal variance exceeds the noise variance.
We agree. Proposition 2 is a population-level statement about the exact maps in the norm . It does not imply a reduction in IBS, in the pointwise CDF error, or in subgroup calibration criteria, and it does not cover a single estimated correction. We have narrowed the wording accordingly.
The abstract, the second contribution, and the Conclusion previously said that the correction "reduces the discrepancy" or "converges" without qualification. They now state convergence "in a censoring-weighted norm".
New text: "This guarantee concerns the exact population-level maps and the norm . It does not by itself imply a reduction in the integrated Brier score, in the pointwise error of the CDF, or in subgroup calibration criteria, in particular for a single estimated correction; we assess these quantities empirically in Section 5."
The third contribution and the Conclusion now present the gains in IBS and conditional calibration of the one-step estimator as empirical findings in most pairwise comparisons.
What we retain from the theory is limited to the following: since is strictly increasing on (Assumption 1(b)), the norm vanishes only if on , so the population iteration has the correct limit.
Thank you for pointing this out. We adjust the initial CDFs from MTLR, DeepHit, and SurvivalBoost to address monotonicity. For DeepSurv, we use the Breslow CDF directly.
We interpolate the MTLR and DeepHit CDFs between adjacent prediction times (Section 5.1 and Appendices F.2 and F.5). Interpolation is strictly increasing between unequal adjacent CDF values and flat over tied values. A constant extension beyond the prediction grid has flat tail.
For SurvivalBoost, isotonic regression yields a nondecreasing CDF and can create flat segments (Appendix F.7).
Appendix F.1 describes the DeepSurv CDF obtained using the Breslow estimator. It increases at distinct observed event times and is flat between them. On a fixed interval where every subinterval of positive length has positive probability of containing an observed event, the longest flat segment tends to zero in probability as the training sample grows.
For a fixed query, composing the initial prediction with the CoSC map preserves a flat segment wherever the input CDF is constant. Strict monotonicity remains a population-level assumption.
Thank you for raising this point. We recomputed IBS using conditional IPCW estimated by Cox regression and a random survival forest. Both schemes yielded nearly the same head-to-head comparisons as marginal Kaplan-Meier weighting, and CoSC retained its IBS advantage under both conditional schemes.
Appendix A.1 now distinguishes the assumptions behind KM-cal and IBS. KM-cal measures agreement with the pooled Kaplan-Meier reference, which estimates marginal event-time survival under marginal independent censoring. For IBS, conditional IPCW estimates the Brier risk under conditional independence when censoring survival is consistently estimated and positive over the evaluation horizon. Independence of censoring from both the event time and the covariates is a sufficient condition for marginal IPCW.
Appendix A.4 (Table 5) compares marginal Kaplan-Meier IPCW with conditional IPCW estimated by Cox regression and a random survival forest, treating censoring as the event. All three schemes select the same best method in all 70 dataset-model combinations, with CoSC first in 64. The full ranking obtained with marginal weights agrees with that obtained with Cox weights in 57 combinations and with RSF weights in 54.
For each conditional scheme, 279 of CoSC's 280 head-to-head outcomes agree with those under under marginal weights. The only reversal is against CSD-iPOT for DeepHit on SEER-stomach, where the absolute mean IBS gap is below under all three schemes. For SurvivalBoost, CoSC improves on the uncalibrated model on all ten datasets and ranks first on eight under each weighting scheme. The common horizon in Appendix A.4 also imposes the support rule on both conditional censoring estimates (see Q9), which explains the different win counts from the main analysis.
Thank you for the suggestion. We revised the IBS evaluation horizon to account for censoring support. We have added True IBS and results to Appendix D based on the simulation repetitions completed to date.
Appendix A.1 sets the main IBS horizon to the minimum of the maximum observed time and the latest evaluation time at which the estimated marginal censoring survival is at least 0.01. For the sensitivity analysis in Appendix A.4, we choose one horizon per split. It is the latest grid time at which marginal KM censoring survival and both conditional estimates for every test subject are at least 0.01. All prediction methods and weighting schemes in that analysis use this same horizon.
Appendix D.1 gives the expected Brier score at time using the known conditional event-time CDF ,
.
We compute True IBS by averaging this score over the test covariates and integrating it over the evaluation horizon, normalized by the length of that horizon. On a common test set and horizon, differences in True IBS equal differences in time-averaged squared CDF error. Proposition 2 establishes population-level contraction in the censoring-weighted norm. The simulations in Appendix D.2 also show that True IBS often decreases after the first CoSC update, with further reductions for several models at 90% censoring. often decreases after the first update and varies more over subsequent iterations.
We fitted the Beran estimator and compared it with one-step CoSC. CoSC achieved lower IBS and in most comparisons, suggesting that self-correction of a fitted survival model can improve on direct kernel estimation of conditional survival.
Appendix I compared CoSC with the Beran conditional Kaplan-Meier estimator [1]. In each split, we fit Beran on the validation set using the same PCA representation and Laplacian kernel as CoSC. We choose the bandwidth by minimizing cross-validated KS-cal using the same procedure (Appendix C). Beran uses the resulting kernel weights to estimate conditional survival directly from observed outcomes. CoSC uses kernel weights to correct predictions from a fitted survival model.
[1] Beran (1981). Nonparametric Regression with Randomly Censored Survival Data.
We thank the reviewer for their thoughtful and constructive comments. We respond to each point below and will make all necessary clarifications and corrections in the final version.
Q1. How does Proposition 2 relate to the empirical one-step estimator with PCA and a CV-selected bandwidth?
Thank you for your helpful comments. We agree that the connection between our theoretical results and the practical algorithm needs clarification. Proposition 2 is stated for a fixed value of the full covariates. The relationship between the original covariates and the kernel approach is outside the scope of this discussion. PCA can be helpful in our datasets because of its potential denoising effect.
The revised appendix considers the theoretical arguments in the PCA-projected space. Proposition 2 can also apply in this space with the corresponding conditions in terms of z~ and Gz~(⋅), under the assumptions that Fz≡Fz~ and Gz≡Gz~. The latter assumption holds exactly in the signal-plus-noise setting and is approximately satisfied as n→∞ under the approximation conditions.
Q2. Can you reconcile the log factor in Lemma 2 with the rate stated in Section 4.3?
Thank you for pointing this out. Section 4.3 now includes the logn factor in the rate stated for the kernel estimator, Op(n−2/(p+4)logn), consistent with the order in Lemma 2. The local comparison with marginal calibration also includes the factor, giving Op(n−(1−α)/2logn) when the region has probability of order n−α. The kernel statement concerns the projected calibration target and the deterministic bandwidth order used in the proof. We discuss the scope of the uniform bound in Q3.
Q3. How do you control uniform error near covariate boundaries and between probability grid points?
Thank you for pointing this out. A sharp supremum-norm bound requires control of the error at all sample points, including those near the boundary. A vanishing proportion of near-boundary points can at most support a sharp bound on the MAE over the sample points.
With a conventional kernel, the bias is larger at points within roughly one bandwidth of the boundary. This region shrinks as the bandwidth tned to zero with n→∞, and the estimation error there can converge to zero more slowly than at interior points. As noted in the remarks, boundary-adapted kernels such as Gasser-Muller and Chen-Gamm kernels can be used near the boundary. We revise the proposition and discuss how to combine conventional and boundary-adapted kernels.
Q4. Does the bandwidth order assumed in the proof apply to Algorithm 1’s CV-selected bandwidth?
Thanks for your comments. Section 4.3 now distinguishes the bandwidth order used in the proof from the rule in Algorithm 1. The proof takes a deterministic sequence hn≍n−1(p+4). We use this bandwidth order to derive the kernel estimator's convergence rate and to illustrate how borrowing information from nearby covariate profiles can improve estimation in shrinking covariate regions. With a fixed positive bandwidth, a smoothing bias can persist. Algorithm 1 minimizes cross-validated KS-cal over a search space specified independently of the theoretical scaling. Establishing a rate for the selected h^ requires a separate analysis of that selection procedure. Appendix C shows the cross-validation loss curves and the selected bandwidths (Figures 3 and 4).
Q5. When does Condition C2 hold, and how do you account for persistent PCA projection error?
We have revised Appendix B.5 to clarify the role of PCA. Appendix B.5.1 uses a signal-plus-noise model to illustrate how PCA can reduce noise. Appendix B.5.2 complements this discussion by anlyzing how the projected calibration map approximates the full-covariate map under conditions C1 and C2, with C2 imposing a strong asymptotic concentration requirement on the discarded components.
Consider a signal-plus-noise modelk for the centered covariates, z=S+e, where the signal S and the noise e are independent, and e∼Nd(0,σ2Id). The covariance matrices of S and z share the same eigenvectors, and isotropic noise increases each eigenvalue by $\sigma^2%. Their spectral decompositions are
ΣS=Pdiag(λ1,...,λd)P⊤
Σz=Pdiag(λ1+σ2,...,λd+σ2)P⊤
where P=[u1,...,ud] and λ1≥⋯≥λd≥0.