- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how to train models that predict which patients are at higher risk of an event, such as death, when each 3D medical image is so large that only a few patients fit in a training batch. Instead of using the usual Cox loss, the paper trains the model by comparing pairs of patients whose event order is known. It explains when these comparisons are valid despite incomplete follow-up and shows that small batches can estimate the gradient of the new pairwise loss without bias. The paper also explores giving stronger labels to patient pairs with larger observed time gaps.
Critical strengths
-
The problem is practical and worth studying. Training on full-size CT images makes ordinary Cox training difficult because each patient's loss involves other patients who may not fit in the same batch.
-
The pairwise method is easy to understand, and the paper clearly explains how to use comparisons even when some patients' final event times are unknown. I also appreciate the experiment on full-resolution 3D CT images.
Critical weaknesses
-
I am not convinced that the main method is sufficiently different from earlier work. Cox-CC and BigSurvSGD already allow stochastic training using closely related pairwise comparisons. The paper needs to show what is gained by its particular way of selecting patients and combining their pairs.
-
The experiments do not yet establish a clear advantage. The CT results are competitive but mixed across batch sizes. I also find the claim about stronger labels based on time gaps unconvincing: a large observed gap does not necessarily mean a large difference in patients' risks, and the evaluation does not clearly show improved survival-time predictions.
Questions for authors
-
Section 2 and Algorithm 1: Unless I am missing something, BigSurvSGD already uses a pairwise survival loss and stochastic training, while Cox-CC can also train with minibatches. What exactly is new here? Can you show that comparing all available pairs within a batch works better than these existing ways of sampling pairs? The description of Cox-CC as needing all events at every update also seems inaccurate.
-
Table 1: The paper says that when a batch contains only two patients, minibatched Cox and Bradley-Terry use the same pairwise loss. Why, then, are their parameter-recovery errors different? Is this because the losses are scaled differently or because of other training settings?
-
Section 5.2: Why should a longer observed time gap mean a stronger difference in risk? Two patients could have similar relative risks but very different absolute survival times, and one patient may stop being followed before the event happens. Could the improvement simply come from using softer labels, rather than from the actual time gaps?
-
Section 3.3 and Appendix D, Table 4: The soft labels lower the reported error in predicted survival time, but do not improve the integrated Brier score, which evaluates survival predictions over time. Also, the time-error calculation uses only patients whose event was observed. How would the conclusion change if the evaluation also accounted for patients whose follow-up ended before the event?
3 comments
The response clarifies the b = 2 results and adds useful comparisons with Cox-CC and BigSurvSGD. However, the authors agree that the loss is the same as in prior work, and the CT gains appear only at one batch size. They also admit that the graded labels are a heuristic and do not separate their effect from label smoothing, while IBS gets slightly worse. The main concerns therefore remain, and therefore I maintain my original score.
Thank you for your follow-up. We would like to respectfully clarify that we did not state or agree that our loss is the same as those used in prior work. The fact that both objectives contain pairwise logistic terms does not make the overall losses identical. This distinction is fundamental to our formulation and is explicitly discussed in the manuscript.
We thank the reviewer for highlighting the practical relevance of the problem and the simplicity of our approach. We have updated the manuscript in response to your feedback, clarifying the relationship to existing methods, adding baseline comparisons, and revising the evaluation and interpretation of graded preference labels.
Across the evaluated batch sizes, BT at b=4 achieves the highest mean Harrell’s C, Uno’s C, and IPCW-AUC among the reported configurations (Table 2). At this batch size, it also yields the largest estimated high-versus-low hazard ratios across the two-, three-, and four-group stratification analyses (Appendix E, Figures 2–4). Below, we address your four questions.
Relationship to Cox-CC and BigSurvSGD. We explicitly acknowledge the shared pairwise logistic loss when Cox-CC uses one control or BigSurvSGD uses strata of size two. Our practical distinction concerns uniform patient sampling and the reuse of encoded volumes across all comparable pairs:
We now provide direct comparisons in both the parameter-recovery and whole-volume CT experiments. On the high-dimensional DBCD and CHOP cohorts, BT achieves lower mean relative ℓ2 error than Cox-CC and BigSurvSGD for b≥4 (Table 1). For example, on DBCD at b=16, the errors are 1.809 for BT, 2.709 for BigSurvSGD, and 13.331 for Cox-CC. The imaging comparisons use the same backbone and hardware, with the strongest mean discrimination results obtained by BT at b=4 (Table 2) and comparable training times across these methods (Table 5).
Different parameter-recovery results at b=2. Thank you for highlighting this detail. At b=2, the two methods share the same pairwise loss, but their implementations use different normalization constants, as shown in our code. This changes the gradient scaling and can lead to different fitted estimates.
Motivation for graded labels and the role of label smoothing. We clarified in Appendix D that graded labels are a heuristic: nearby observed times receive softer targets, while larger gaps receive stronger targets. As the revised manuscript states:
When the later observation is censored, the observed gap is only a lower bound on the true event-time separation. We also agree that softer labels themselves may contribute to the improvement. The present experiments evaluate the combined graded-label procedure; they do not isolate the contribution of time-dependent grading from that of label smoothing.
Censoring-aware evaluation and interpretation of IBS. We added IPCW weighting to the RMST prediction-error evaluation and defined it explicitly in Section 3.3, Eq. (4). Under this evaluation, soft BT retains lower mean RMST error than hard BT on all three cohorts (Table 4): improvements of 1.34 months on METABRIC, 6.6 days on SUPPORT, and 1.34 months on GBSG. Mean Harrell’s and Uno’s concordance differ by at most 0.002.
IBS is slightly higher under soft BT, and SUPPORT IPCW-AUC decreases. We therefore revised our conclusion accordingly:
We hope these revisions clarify the practical contribution and provide a more precise interpretation of the empirical results.