- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a stochastic optimization framework for large-scale Cox regression by combining grouped risk sets with a softplus-based LogSumExp surrogate. The authors establish improved convergence rates for convex LogSumExp optimization, derive second-order gradient-error bounds for risk-set grouping, and show that the compressed Cox estimator can retain the asymptotic distribution of the full estimator under regularity assumptions. Experiments on synthetic and real-world survival datasets demonstrate improved optimization accuracy over several stochastic Cox baselines.
However, the paper has two major weaknesses: (1) insufficient evidence of practical computational advantages over established exact Cox solvers, and (2) inadequate experimental validation of the contribution of risk-set grouping and the theoretical convergence guarantees.
Critical strengths
1. A technically meaningful contribution to stochastic Cox optimization.
The paper exploits the nested structure of Cox risk sets to reduce the number of auxiliary normalization variables while controlling the resulting approximation error. In particular, the second-order gradient-error bound for risk-set grouping is an interesting theoretical result. It provides a principled explanation for why neighboring risk sets can share normalization variables without introducing a first-order gradient perturbation.
The analysis also establishes curvature-preservation bounds and conditions under which the compressed estimator retains the first-order asymptotic behavior of the full Cox estimator. These results constitute a meaningful technical contribution beyond simply introducing a heuristic grouping strategy.
2. Careful theoretical analysis and systematic optimization experiments.
The paper provides explicit convergence guarantees for the softplus surrogate, including an improved averaged objective rate and a last-iterate result under additional curvature assumptions.
The experimental protocol uses matched vector-work budgets, validation-selected configurations, and multiple optimizer seeds. The proposed method achieves the lowest median terminal full-Cox training gap on all five evaluated datasets. Additional experiments separately investigate the approximation errors introduced by risk-set grouping and softplus smoothing.
These results support the effectiveness of the proposed approach in approximating the full-Cox optimization objective under the evaluated stochastic computational budgets.
Critical weaknesses
1. The practical computational advantage over established Cox solvers is not demonstrated.
The primary motivation of the paper is scalable Cox regression, particularly in settings where full-data optimization is computationally expensive. However, the experimental comparisons focus exclusively on stochastic baselines, including Batch LSE, Minibatch Cox, BigSurv, and Cox-CC, without directly evaluating optimized exact Cox solvers.
For linear Cox regression, the nested risk-set structure already allows efficient full-gradient evaluation using cumulative sums, with established optimization methods such as L-BFGS and coordinate descent. Therefore, the absence of a direct comparison against efficient full-batch Cox implementations makes it difficult to assess the practical significance of the proposed method.
Although the proposed method achieves smaller terminal full-Cox training gaps under matched vector-work budgets, this advantage is not consistent across different target accuracy levels. For example, Appendix Table 5 shows that BigSurv reaches a training gap of with less median vector work on four of the five datasets. The proposed method becomes more competitive at stricter accuracy targets, particularly on the large correlated synthetic datasets. However, the paper does not establish when such additional optimization accuracy provides a meaningful practical advantage.
Furthermore, the motivating stochastic-access scenarios, such as distributed or out-of-core learning, are not directly evaluated. The method assumes that event times have been sorted and that subjects can be sampled efficiently from indexed risk sets. While these assumptions are compatible with some stochastic-access settings, their preprocessing, indexing, and data-access costs may be important in practice. The primary vector-work measure does not fully capture these costs.
The paper also acknowledges that it does not establish an end-to-end runtime advantage over optimized cumulative-sum full-Cox solvers.
Finally, improvements in predictive performance are limited. Appendix Table 6 shows that the proposed method achieves the best mean test Cox loss on three of five datasets and the best mean C-index on only two. Although substantial predictive improvements are not necessary for an optimization-focused paper, these results further emphasize the need to demonstrate a clear computational advantage.
Overall, the paper convincingly demonstrates improved optimization accuracy relative to several stochastic baselines, but does not establish that the proposed approach provides a practically meaningful scalability advantage over existing efficient Cox regression methods.
2. The contribution of risk-set grouping and the theoretical convergence guarantees are insufficiently validated experimentally.
The paper's main methodological contribution is the grouping of neighboring Cox risk sets to reduce the number of auxiliary normalization variables while controlling approximation error. However, the most direct ablation of this contribution is missing.
Specifically, the experiments do not compare against the ungrouped softplus formulation (), which introduces one auxiliary variable per event, or against related variational LogSumExp optimization methods such as SCENT (Wei et al., 2026b).
This omission is important because the benefits of grouping are not limited to reducing auxiliary storage. In the theoretical analysis, the optimization constants depend on the number of normalization variables . Reducing through grouping may therefore substantially improve the stochastic optimization behavior. Without an ungrouped comparison under the same computational budget, it remains unclear how much of the observed optimization advantage comes specifically from risk-set grouping rather than the underlying softplus formulation or optimization procedure.
A second concern is the gap between the theoretical convergence analysis and the experimental implementation. Corollary 4.2 establishes the end-to-end mean-squared parameter error rate using approximation parameters and learning rates scheduled according to the optimization horizon, with a last-iterate guarantee.
However, the experiments instead use fixed values of and , a decreasing learning rate of the form , and averaged coefficient iterates. Consequently, the headline end-to-end convergence guarantee does not directly characterize the reported experimental procedure, and its predicted scaling behavior is not empirically examined.
Moreover, the theoretical analysis relies on uniform curvature assumptions, bounded risk-weight dispersion, and a condition requiring the event risk sets to contain at least a fixed positive fraction of the sample. These conditions can be restrictive, particularly when risk sets shrink rapidly or risk weights are highly heterogeneous. The experiments do not systematically characterize the method's behavior under such less favorable conditions.
Although Figure 2 and Appendix Table 8 provide useful approximation diagnostics, the observed grouping-error slopes range from 1.37 to 1.87, and the pointwise checks do not establish that the theorem's assumptions hold uniformly over the parameter domain. These observations do not contradict the second-order theoretical upper bound, but they limit the extent to which the empirical results validate its applicability.
Overall, the theoretical analysis is interesting, but the experiments do not sufficiently isolate the benefits of the proposed grouping mechanism or demonstrate that the theoretical convergence guarantees are informative under the practical settings considered.
Questions for authors
Q1. Practical computational advantage over established Cox solvers.
The proposed method achieves smaller full-Cox optimization gaps than the evaluated stochastic baselines, but optimized full-batch Cox solvers are not included in the comparisons.
Can the authors provide vector-work and end-to-end runtime comparisons against efficient full-Cox implementations, such as cumulative-sum-based L-BFGS or coordinate-descent solvers?
In particular, under what concrete conditions involving dataset size, memory constraints, data-access patterns, or target optimization accuracy does the proposed method outperform these established approaches?
The method assumes sorted event times and efficient random sampling from indexed risk sets. How are the associated preprocessing and data-access costs handled in genuinely distributed or out-of-core settings?
Q2. Contribution of risk-set grouping relative to existing LogSumExp optimization methods.
The key methodological contribution is the sharing of auxiliary normalization variables across neighboring risk sets. However, an ungrouped softplus baseline () is not evaluated.
Can the authors compare the proposed method against the ungrouped softplus formulation and SCENT under matched computational budgets?
In particular, how much of the improvement in optimization accuracy and convergence speed can be attributed specifically to risk-set grouping, rather than to the underlying softplus surrogate or the choice of optimizer?
Q3. Applicability and empirical relevance of the theoretical convergence guarantees.
Corollary 4.2 establishes an mean-squared parameter error bound using horizon-dependent approximation and learning-rate schedules. However, the experiments use fixed approximation parameters, a different learning-rate schedule, and averaged iterates.
Can the authors clarify how the theoretical guarantees relate to the implemented algorithm and provide experiments using the theoretical parameter schedules?
It would also be important to report whether the resulting optimization or parameter errors exhibit behavior consistent with the predicted rate.
Furthermore, can the authors provide empirical estimates of the curvature and risk-weight dispersion quantities appearing in the theoretical bounds, and clarify whether the resulting finite-time guarantees are informative at the experimental optimization horizons?
Q4. Robustness to less favorable risk-set structures.
The theoretical guarantees require conditions controlling minimum risk-set sizes and risk-weight heterogeneity.
How does the proposed method perform when risk sets shrink rapidly, when event rates are high, or when risk weights are strongly heterogeneous?
Can the authors provide controlled experiments varying these factors to determine when risk-set grouping remains effective and when its computational advantage diminishes?
Dear Reviewer kauy,
Thank you for recognizing the theoretical contribution of our work and for your constructive comments. We have expanded the experiments and revised the convergence analysis. Changes are highlighted in dark blue.
Q1: Practical computational advantage. We added experiments on CASCADE (24.6 million training rows) and Taobao (53.8 million), whose feature matrices do not fit in the available RAM (out-of-core training). We plot the full-Cox optimality gap against optimization time. At 120 seconds, our median gaps are 1.7×10−4 and 2.5×10−4, respectively, lower than all tested baselines, including optimized cumulative-sum L-BFGS, SCENT, ungrouped softplus and BigSurv.
Sampling avoids the repeated full-data scans needed by L-BFGS. BigSurv and several other controls are already stochastic, so sampling alone does not explain the advantage. On the original in-memory benchmarks, L-BFGS is strongest and the stochastic methods have mixed rankings. Our method is intended for larger problems, where data access, compressed state and the improved iteration complexity studied in the theory matter together.
Q2: Contribution of grouping. The softplus construction is prior work. Our main algorithmic contribution is to let neighboring events share a normalization variable while controlling the resulting gradient error to second order. The analysis also controls curvature and the effect on the estimated coefficients.
Ungrouped softplus and SCENT retain one auxiliary variable per event. Grouping reduces this count from m, the number of events, to KT, the number of groups. Under uniform sampling, each variable is visited with probability 1/KT rather than 1/m per draw. More frequent updates can keep normalization estimates current and improve convergence. The new baseline comparisons and the NWTCO training ablation test the effect of grouping in practice.
Q3: Theory and implemented schedules. Revised Corollary 4.2 uses ηt=1+(t−1)/t0η0 and suffix averaging: an average over a final portion of the iterates, starting at or after ⌈T/4⌉. For a planned budget of T updates, grouping tolerance δT≍T−1/5 and softplus smoothing ρT=δT2 remain fixed during each run. The result gives O(T−4/5) mean squared parameter error relative to the full Cox solution. Batches of events and weighted risk-set blocks are covered in the appendix.
The practical runs use the same step-size and averaging rules with tuned numerical constants and approximation levels. The theoretical prescriptions are based on worst-case bounds and are conservative at practical budgets, as is common in optimization theory. The tuned values are not claimed to satisfy the theorem's sufficient ranges. The finite-time bound separates optimization error from the bias due to fixed grouping and smoothing.
A separate illustration uses theoretical values on an unpenalized problem with 4,000 subjects and 1,600 events, with all assumptions checked over the whole parameter domain. Across four update budgets and 32 runs each, mean squared parameter error decreases from 3.9×10−5 to 9.4×10−7. The final general bound is 1.6×10−4, compared with initial squared error 0.061.
Q4: Unfavorable risk sets and heterogeneity. The computational analysis no longer requires every event risk set to contain a fixed positive fraction of the sample. Appendix B retains the dependence on H=1+log(N/nmin), where N is sample size and nmin the smallest event risk-set size, giving O((H/T)4/5). The original Correlated 1M training sample already has a minimum risk fraction of 8.3×10−6.
Rapidly shrinking risk sets can reduce compression, while concentrated risk weights worsen the approximation and convergence bounds. Real-data weight and curvature diagnostics use selected parameter values; the theory illustration checks bounds over the whole domain. Ridge supplies curvature for penalized optimization; the unpenalized statistical result needs curvature near the population parameter and estimators that stay in that neighborhood. Optimizer seeds measure variation on fixed data, so robustness across independent datasets or repeated splits remains a limitation. We hope these revisions support reconsideration of your overall assessment.