acceptodds
Under review as a conference paper at ICLR 2027

Proper Scoring Rules for Right-Censored Survival Data

10 views

Abstract

Proper scoring rules provide a rigorous theoretical basis for the training and evaluation of probabilistic forecasts. In survival analysis, such forecasts describe the distribution of the time until an event occurs. However, this event time is often only partially observed because follow-up may end before the event occurs, resulting in right censoring. We propose a framework for proper scoring of right-censored survival outcomes based on a simple idea: first, map the predictive distribution through the censoring mechanism, then apply the underlying proper score on the induced observed-data law. This yields localized scores for fixed censoring times and marginalized scores when the censoring time is random or only partially observed. The resulting construction recovers familiar right-censored likelihood and IPCW-type criteria within a coherent framework, while also yielding right-censored versions of the CRPS, pinball loss, Brier score, and energy score. We show that the marginalized construction is proper under conditional independent censoring and, for strictly proper base scores, identifies the latent joint CDF on the identifiable region. The same principle also leads to censored engression, a sample-based learning objective for multivariate right-censored survival modeling. In experiments, our scores correctly rank the oracle forecast across several censoring regimes, whereas forecast-dependent plug-in weighted scores can exhibit ranking reversals. Censored engression likewise substantially improves over naive training on censored outcomes.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
proper scoring rules, right censoring, survival analysis, probabilistic forecasting, multivariate survival, distributional regression, energy score, engression

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kV9nOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes a framework for proper scoring of right-censored survival outcomes: the predictive distribution is first mapped through the censoring mechanism, and the underlying proper score is then applied to the induced observed-data law. This yields localized scores for fixed censoring times and marginalized scores for random or partially observed censoring, with the censoring law held fixed independently of the forecast. Proposition 1 establishes propriety under conditionally independent censoring and identification of the latent joint CDF on the identifiable region. The construction recovers the censored NLL and IPCW Brier score as special cases and gives right-censored versions of the CRPS, pinball loss, and energy score. The censored energy score is further used as a training objective (censored engression) for likelihood-free modeling of multivariate, co-occurring event times. Validation includes controlled simulations showing the proposed scores preserve the oracle ranking while forecast-dependent plug-in scores can reverse it, synthetic multivariate experiments, and an ICU AKI case study on MIMIC-IV.

Critical strengths
  1. Starting from the principle of first censoring the predictive distribution according to the observation mechanism and then applying a proper scoring rule, the paper derives right-censored versions of commonly used evaluation scores. It covers both fixed censoring through localization and random or partially observed censoring through marginalization. Existing criteria, including the censored NLL and IPCW Brier score, are recovered as special cases within a unified framework. These scores can also serve as learning objectives. Building on this connection, the authors propose censored engression, extending engression to multivariate right-censored outcomes and enabling likelihood-free modeling of potentially co-occurring event times.
  2. The experimental results support the effectiveness of the proposed approach. The theory is clearly presented.
Critical weaknesses

None. I did not find any major weaknesses. The notation is heavy and easy to confuse; I suggest adding a unified table of notation.

Questions for authors

The naive baseline treats censored values as event times. Could the authors also compare against complete-case engression (training only on uncensored samples) and an IPCW-weighted energy score, which are more standard naive alternatives?

3 comments

We thank the reviewer for the positive assessment of our paper.

We also appreciate the suggestion to include complete-case engression and IPCW-weighted energy-score training as additional baselines. We have added both in the synthetic experiments and the real-data AKI use case. Censored engression consistently achieves the lowest censored energy score. For example, at k=2, censored engression scores 0.4904 versus 0.5236 for IPCW and 0.5710 for complete-case engression. At k=10, the respective scores are 1.3382, 2.3976, and 2.5101. IPCW improves over naive engression at low dimensions but performs worse at higher dimensions.

Finally, we have added a unified notation table in the appendix to improve readability.

Dear Reviewer,

Thank you for your thoughtful review and contribution to the discussion.

As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.

In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.

Thank you again for your time and effort!

Thanks for the detailed response and revision. The new baselines and the notation table address my questions. I have no further concerns and am happy to maintain my positive score.

Reviewer_g6RfOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors extend an approach to build a localized proper scoring rule to account for censoring for several proper scoring rules. They also derive an estimation strategy that optimizes those proper scoring rules using engressions. This framework is able to handle multiple events that are not incompatible contrary to the competing risks setting (which they call multivariate, which in my own humble opinion would refer more to the features used for prediction, maybe the multiclass word wouldbe more suited?). They illustrate the performances of their proposed proper scoring rules both for evaluation and optimization.

Critical strengths

The proposed work is very thorough, with strong theoretical results, and elegantly extends very recent results to settings with right censoring.

Critical weaknesses

The experimental results critically lack of comparisons to baselines. The invoked motive to handle multivariate prediction does not prevent comparing to existing methods in particular settings where it is possible: classical survival analysis with one event, and the competing risks setting. It could be expected and acceptable that the proposed approach is slightly outperformed by approaches designed for those settings, but it is critical for the reader and reassuring for the user to know that the results are close to the SOTA.

Questions for authors
  • can you provide more experiments comparing to baselines in the standard survival analysis setting, and to the competing risks setting, on real datasets? I will definitely raise my assessment to acceptance with convincing empricial results compared to the baselines. Required computational time and resoulrces should also be compared to baselines.
  • I am not sure to understand why the loss from rindt is not evaluated in your work? and also the loss of alberge in cases where it could be applied.
2 comments

We thank the reviewer for the positive assessment of our paper and the constructive suggestions. We address the individual points below.

  1. Multivariate predictive distributions. We appreciate the suggestion to clarify our terminology. Multivariate survival analysis is established terminology for modelling multiple potentially dependent event times, including distinct events occurring within the same individual (Wei et al., 1989; Oakes, 1994). Our censored engression framework explicitly models the joint distribution of a vector of event times, rather than a categorical outcome. We therefore believe multivariate is the appropriate term, whereas multiclass would be misleading. We added the aforementioned references to clarify this distinction. 2.The loss in Rindt et al. We agree that the connection should be clearer. Rindt et al. establish the propriety of the right-censored logarithmic score, which is recovered by our framework and evaluated in Section 5.1. We also use this score to train our likelihood-based baselines. We identified an inconsistency in our notation: Table 1 refers to the logarithmic score, whereas the experiments use censored NLL. We have standardized this terminology to logarithmic score throughout the experimental tables. Additionally, we now benchmark SuMo-net (Rindt et al., 2022) in Appendix D.7.

  2. Competing risks is a different problem from the one we tackle here. In the competing-risks setting, only the first event time and its cause are observed, so the joint distribution of the potentially co-occurring event times is generally not identifiable. Consequently, competing-risks models cannot directly be compared to our approach in terms of the full joint distribution.

  3. The score of Alberge et al. Alberge et al. (2025) develop a proper censoring-adjusted, horizon-specific score for univariate survival and competing risks. Our framework addresses a different and more general objective: constructing censored versions of a broad class of proper scoring rules, including rules for continuous multivariate distributions. We clarify this distinction in the related work. Additionally, we now include their SurvivalBoost method in our univariate benchmarks (Appendix D.7).

  4. Single/univariate time-to-event distribution benchmarking. We agree that additional univariate benchmarks provide useful empirical context. We therefore added experiments on SUPPORT and METABRIC (Appendix D.7), comparing censored engression against SuMo-net, SurvivalBoost, DeepHitSingle, and an MLP–Weibull model across five data splits. We report censored CRPS, IPCW integrated Brier score, C-index, and training time. Censored engression achieves the best CRPS and Brier score on METABRIC and remains competitive on SUPPORT, where SurvivalBoost performs best. We also acknowledge its higher computational cost (61 seconds versus 18 seconds for SurvivalBoost on SUPPORT). These results demonstrate competitiveness with established univariate methods, while our primary contribution remains flexible joint multivariate survival prediction.

[1] Wei, Lee-Jen, Danyu Y. Lin, and Lisa Weissfeld. "Regression analysis of multivariate incomplete failure time data by modeling marginal distributions." Journal of the American Statistical Association 84.408 (1989): 1065-1073.

[2] Oakes, David. "Multivariate survival distributions." Journal of Nonparametric Statistics 3.3-4 (1994): 343-354.

I thank the authors for their answers which address my comments.

Reviewer_pLceOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on proper rules in a survival analysis context, where observations are right-censored. The procedure is as follows: pass the predicted distribution through the censoring mechanism, then apply the underlying proper score on the induced observed-data law. This gives localized scores for a fixed censoring time, and marginalized scores with random censoring.

The construction recovers two classical survival losses (Brier score, censored NLL as examples) + an energy score, pinball, etc.. The energy score is used to train for multivariate event times (the engression component).

They validate the approach multiple ways, including a Weibull simulation checking whether scores rank the true forecast first and a MIMIC-IV experiment.

Critical strengths
  • The paper provides one a principled approach to both scoring and training. The censoring adjustment never depends on the forecast being scored so the candidate forecast cannot influence its own weights.

  • Standard competing risk models (DeepHit e.g.) approaches assume mutually exclusive events. Censored engression (the proposed mechanism) doesn't have that limitation.

Critical weaknesses
  • The empirical evidence is lacking in parts. Section 5.1 results come from a single set of N=1000 examples; it would benefit from multiple seeds / confidence intervals, etc. Several margins are very small, for example a Brier score of 0.0457 versus 0.0460 in Table 5
Questions for authors
  • Can the Section 5.1 study be repeated over many test samples or DGP seeds, reporting how often each score ranks the oracle first?
2 comments

We thank the reviewer for the positive assessment of our paper. To further assess robustness, we repeated the experiments in Section 5.1 using 200 independent test sets of N=1000 observations. Table 2 now reports the percentage of repetitions in which the oracle forecast achieves the best score. Our logarithmic score (NLL in the previous version), CRPS, and pinball score rank the oracle first in 100% of repetitions across all regimes, while the Brier score does so in 78–100%, providing additional evidence of ranking stability.

Dear Reviewer,

Thank you for your thoughtful review and contribution to the discussion.

As the discussion period approaches its conclusion, I would like to kindly remind you to review the authors' rebuttal and engage in the discussion if you have not already done so.

In particular, it would be helpful if you could indicate whether the authors have adequately addressed your concerns and, if appropriate, reconsider your assessment and score.

Thank you again for your time and effort!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.