- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies whether numerical changes in a survival training loss reliably track changes in the C-index, rather than merely sharing an optimum. It introduces value-monotonicity, gives a Cox rescaling counterexample, and proposes a corresponding argument for discrete-time likelihoods. For sigmoid concordance loss (SCL), it derives a two-sided approximation bound whose usefulness depends on the mass of near-tied score pairs. Across 18 datasets and four modalities, SCL shows stronger loss-metric coupling and closer agreement between loss-based and C-index-based checkpoint selection than the baselines, with comparable discrimination. The claimed contribution is with regards to the interpretation of loss values during training, not improved predictive accuracy.
Critical strengths
-
The distinction between agreement at the optimum and agreement between loss and metric values during training is clearly stated. The Cox rescaling example makes the issue concrete, while the SCL approximation bound identifies a condition under which the loss value can be interpreted. I also appreciate that the authors explicitly discuss where this interpretation fails, rather than claiming unconditional alignment.
- The empirical study covers a broad set of survival datasets and includes a useful control separating the training objective from the monitoring signal. The near-tie diagnostic connects the analysis to a quantity that can be checked on validation predictions. The authors are also transparent about preprocessing leakage, pooling bias and implementation corrections, although documenting these issues does not eliminate their effects on the reported results.
Critical weaknesses
-
The practical benefit beyond monitoring the validation C-index remains limited. Appendix B.12 shows that evaluating SCL on Cox-trained predictions raises coupling from 0.51 to 0.96, and the authors acknowledge that monitoring SCL is not preferable to computing the C-index directly. Table A2 also shows that closer agreement between the two selection rules does not establish better test performance. I recognize that the paper claims interpretability rather than improved discrimination, so I do not view this as a contradiction. My concern is whether making the training objective and monitoring signal coincide provides enough additional value beyond that interpretability. Evidence that the proposed error bounds are informative at the scale of actual checkpoint differences would help establish this value.
- The fixed-risk fibre argument in Proposition A2 does not hold as stated. A positive-dimensional fibre need not approach the boundary where the observed bin has zero probability. For example, with weights and a predicted distribution , fixing the risk at forces every distribution in the fibre to satisfy . If the event occurs in bin 1, its negative log-likelihood cannot diverge along this fibre. The required feasibility condition is that the fixed risk lie in the convex hull of the remaining bin weights. Censored observations require an analogous condition for the total surviving mass. This affects the stated discrete-time extension, not the Cox rescaling result or the SCL approximation bound, but it requires a corrected statement or a different construction.
Questions for authors
-
Can Proposition A2 be restated with the necessary feasibility conditions, or established through a different construction? Which discrete-time models in the experiments satisfy the corrected assumptions?
- Along the measured training trajectories, how often does the loss decrease between checkpoints exceed the sum of their computable approximation-error bounds? This would clarify whether the improvement certificate is informative in practice, beyond the observed loss-metric correlation.