acceptodds
Under review as a conference paper at ICLR 2027

Scalable Cox Regression via Grouped Risk Sets and Sharper LogSumExp Rates

Abstract

Motivated by the computational challenges of large-scale Cox regression, we study stochastic minimization of LogSumExp objectives over large sets. Mini-batch normalizer estimates generally yield biased gradients. We instead use a softplus surrogate that introduces one auxiliary scalar per normalizer and admits unbiased single-sample gradients. For smooth convex LogSumExp objectives, we prove an averaged objective bound, improving the previous analysis. With a strongly convex regularizer on the original variable, we also obtain a last-iterate squared-error rate of without strong convexity in the auxiliary variables. For Cox regression, the normalizers are defined over nested risk sets. We exploit this structure by grouping neighboring failures and sharing one auxiliary variable per group. The resulting compressed objective admits uniform score and curvature bounds that control the errors from grouping and softplus approximation. Together with the general optimization result, these bounds give a mean-square rate of , up to logarithmic factors, relative to the full Cox solution. The compressed estimator also matches the full estimator's asymptotic distribution. Experiments on synthetic and real survival datasets with slowly decreasing risk sets show a favorable performance relative to stochastic baselines.

Primary area:
optimization
Keywords:
Cox regression, LogSumExp, stochastic optimization, survival analysis

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kauyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a stochastic optimization framework for large-scale Cox regression by combining grouped risk sets with a softplus-based LogSumExp surrogate. The authors establish improved convergence rates for convex LogSumExp optimization, derive second-order gradient-error bounds for risk-set grouping, and show that the compressed Cox estimator can retain the asymptotic distribution of the full estimator under regularity assumptions. Experiments on synthetic and real-world survival datasets demonstrate improved optimization accuracy over several stochastic Cox baselines.

However, the paper has two major weaknesses: (1) insufficient evidence of practical computational advantages over established exact Cox solvers, and (2) inadequate experimental validation of the contribution of risk-set grouping and the theoretical convergence guarantees.

Critical strengths

1. A technically meaningful contribution to stochastic Cox optimization.

The paper exploits the nested structure of Cox risk sets to reduce the number of auxiliary normalization variables while controlling the resulting approximation error. In particular, the second-order gradient-error bound for risk-set grouping is an interesting theoretical result. It provides a principled explanation for why neighboring risk sets can share normalization variables without introducing a first-order gradient perturbation.

The analysis also establishes curvature-preservation bounds and conditions under which the compressed estimator retains the first-order asymptotic behavior of the full Cox estimator. These results constitute a meaningful technical contribution beyond simply introducing a heuristic grouping strategy.

2. Careful theoretical analysis and systematic optimization experiments.

The paper provides explicit convergence guarantees for the softplus surrogate, including an improved O(T−1/2)O(T^{-1/2}) averaged objective rate and a last-iterate result under additional curvature assumptions.

The experimental protocol uses matched vector-work budgets, validation-selected configurations, and multiple optimizer seeds. The proposed method achieves the lowest median terminal full-Cox training gap on all five evaluated datasets. Additional experiments separately investigate the approximation errors introduced by risk-set grouping and softplus smoothing.

These results support the effectiveness of the proposed approach in approximating the full-Cox optimization objective under the evaluated stochastic computational budgets.

Critical weaknesses

1. The practical computational advantage over established Cox solvers is not demonstrated.

The primary motivation of the paper is scalable Cox regression, particularly in settings where full-data optimization is computationally expensive. However, the experimental comparisons focus exclusively on stochastic baselines, including Batch LSE, Minibatch Cox, BigSurv, and Cox-CC, without directly evaluating optimized exact Cox solvers.

For linear Cox regression, the nested risk-set structure already allows efficient full-gradient evaluation using cumulative sums, with established optimization methods such as L-BFGS and coordinate descent. Therefore, the absence of a direct comparison against efficient full-batch Cox implementations makes it difficult to assess the practical significance of the proposed method.

Although the proposed method achieves smaller terminal full-Cox training gaps under matched vector-work budgets, this advantage is not consistent across different target accuracy levels. For example, Appendix Table 5 shows that BigSurv reaches a training gap of 10−310^{-3} with less median vector work on four of the five datasets. The proposed method becomes more competitive at stricter accuracy targets, particularly on the large correlated synthetic datasets. However, the paper does not establish when such additional optimization accuracy provides a meaningful practical advantage.

Furthermore, the motivating stochastic-access scenarios, such as distributed or out-of-core learning, are not directly evaluated. The method assumes that event times have been sorted and that subjects can be sampled efficiently from indexed risk sets. While these assumptions are compatible with some stochastic-access settings, their preprocessing, indexing, and data-access costs may be important in practice. The primary vector-work measure does not fully capture these costs.

The paper also acknowledges that it does not establish an end-to-end runtime advantage over optimized cumulative-sum full-Cox solvers.

Finally, improvements in predictive performance are limited. Appendix Table 6 shows that the proposed method achieves the best mean test Cox loss on three of five datasets and the best mean C-index on only two. Although substantial predictive improvements are not necessary for an optimization-focused paper, these results further emphasize the need to demonstrate a clear computational advantage.

Overall, the paper convincingly demonstrates improved optimization accuracy relative to several stochastic baselines, but does not establish that the proposed approach provides a practically meaningful scalability advantage over existing efficient Cox regression methods.

2. The contribution of risk-set grouping and the theoretical convergence guarantees are insufficiently validated experimentally.

The paper's main methodological contribution is the grouping of neighboring Cox risk sets to reduce the number of auxiliary normalization variables while controlling approximation error. However, the most direct ablation of this contribution is missing.

Specifically, the experiments do not compare against the ungrouped softplus formulation (δ=0\delta=0), which introduces one auxiliary variable per event, or against related variational LogSumExp optimization methods such as SCENT (Wei et al., 2026b).

This omission is important because the benefits of grouping are not limited to reducing auxiliary storage. In the theoretical analysis, the optimization constants depend on the number of normalization variables KK. Reducing KK through grouping may therefore substantially improve the stochastic optimization behavior. Without an ungrouped comparison under the same computational budget, it remains unclear how much of the observed optimization advantage comes specifically from risk-set grouping rather than the underlying softplus formulation or optimization procedure.

A second concern is the gap between the theoretical convergence analysis and the experimental implementation. Corollary 4.2 establishes the end-to-end mean-squared parameter error rate O((log⁡TT)4/5)O\left(\left(\frac{\log T}{T}\right)^{4/5}\right) using approximation parameters and learning rates scheduled according to the optimization horizon, with a last-iterate guarantee.

However, the experiments instead use fixed values of δ=0.05\delta=0.05 and ρ=10−4\rho=10^{-4}, a decreasing learning rate of the form ηt=η0(1+t/1000)−1/2\eta_t=\eta_0(1+t/1000)^{-1/2}, and averaged coefficient iterates. Consequently, the headline end-to-end convergence guarantee does not directly characterize the reported experimental procedure, and its predicted scaling behavior is not empirically examined.

Moreover, the theoretical analysis relies on uniform curvature assumptions, bounded risk-weight dispersion, and a condition requiring the event risk sets to contain at least a fixed positive fraction of the sample. These conditions can be restrictive, particularly when risk sets shrink rapidly or risk weights are highly heterogeneous. The experiments do not systematically characterize the method's behavior under such less favorable conditions.

Although Figure 2 and Appendix Table 8 provide useful approximation diagnostics, the observed grouping-error slopes range from 1.37 to 1.87, and the pointwise checks do not establish that the theorem's assumptions hold uniformly over the parameter domain. These observations do not contradict the second-order theoretical upper bound, but they limit the extent to which the empirical results validate its applicability.

Overall, the theoretical analysis is interesting, but the experiments do not sufficiently isolate the benefits of the proposed grouping mechanism or demonstrate that the theoretical convergence guarantees are informative under the practical settings considered.

Questions for authors

Q1. Practical computational advantage over established Cox solvers.

The proposed method achieves smaller full-Cox optimization gaps than the evaluated stochastic baselines, but optimized full-batch Cox solvers are not included in the comparisons.

Can the authors provide vector-work and end-to-end runtime comparisons against efficient full-Cox implementations, such as cumulative-sum-based L-BFGS or coordinate-descent solvers?

In particular, under what concrete conditions involving dataset size, memory constraints, data-access patterns, or target optimization accuracy does the proposed method outperform these established approaches?

The method assumes sorted event times and efficient random sampling from indexed risk sets. How are the associated preprocessing and data-access costs handled in genuinely distributed or out-of-core settings?

Q2. Contribution of risk-set grouping relative to existing LogSumExp optimization methods.

The key methodological contribution is the sharing of auxiliary normalization variables across neighboring risk sets. However, an ungrouped softplus baseline (δ=0\delta=0) is not evaluated.

Can the authors compare the proposed method against the ungrouped softplus formulation and SCENT under matched computational budgets?

In particular, how much of the improvement in optimization accuracy and convergence speed can be attributed specifically to risk-set grouping, rather than to the underlying softplus surrogate or the choice of optimizer?

Q3. Applicability and empirical relevance of the theoretical convergence guarantees.

Corollary 4.2 establishes an O((log⁡T/T)4/5)O((\log T/T)^{4/5}) mean-squared parameter error bound using horizon-dependent approximation and learning-rate schedules. However, the experiments use fixed approximation parameters, a different learning-rate schedule, and averaged iterates.

Can the authors clarify how the theoretical guarantees relate to the implemented algorithm and provide experiments using the theoretical parameter schedules?

It would also be important to report whether the resulting optimization or parameter errors exhibit behavior consistent with the predicted rate.

Furthermore, can the authors provide empirical estimates of the curvature and risk-weight dispersion quantities appearing in the theoretical bounds, and clarify whether the resulting finite-time guarantees are informative at the experimental optimization horizons?

Q4. Robustness to less favorable risk-set structures.

The theoretical guarantees require conditions controlling minimum risk-set sizes and risk-weight heterogeneity.

How does the proposed method perform when risk sets shrink rapidly, when event rates are high, or when risk weights are strongly heterogeneous?

Can the authors provide controlled experiments varying these factors to determine when risk-set grouping remains effective and when its computational advantage diminishes?

1 comment

Dear Reviewer kauy,

Thank you for recognizing the theoretical contribution of our work and for your constructive comments. We have expanded the experiments and revised the convergence analysis. Changes are highlighted in dark blue.

Q1: Practical computational advantage. We added experiments on CASCADE (24.6 million training rows) and Taobao (53.8 million), whose feature matrices do not fit in the available RAM (out-of-core training). We plot the full-Cox optimality gap against optimization time. At 120 seconds, our median gaps are 1.7×10−41.7\times10^{-4} and 2.5×10−42.5\times10^{-4}, respectively, lower than all tested baselines, including optimized cumulative-sum L-BFGS, SCENT, ungrouped softplus and BigSurv.

Sampling avoids the repeated full-data scans needed by L-BFGS. BigSurv and several other controls are already stochastic, so sampling alone does not explain the advantage. On the original in-memory benchmarks, L-BFGS is strongest and the stochastic methods have mixed rankings. Our method is intended for larger problems, where data access, compressed state and the improved iteration complexity studied in the theory matter together.

Q2: Contribution of grouping. The softplus construction is prior work. Our main algorithmic contribution is to let neighboring events share a normalization variable while controlling the resulting gradient error to second order. The analysis also controls curvature and the effect on the estimated coefficients.

Ungrouped softplus and SCENT retain one auxiliary variable per event. Grouping reduces this count from mm, the number of events, to KTK_T, the number of groups. Under uniform sampling, each variable is visited with probability 1/KT1/K_T rather than 1/m1/m per draw. More frequent updates can keep normalization estimates current and improve convergence. The new baseline comparisons and the NWTCO training ablation test the effect of grouping in practice.

Q3: Theory and implemented schedules. Revised Corollary 4.2 uses ηt=η01+(t−1)/t0\eta_t=\frac{\eta_0}{1+(t-1)/t_0} and suffix averaging: an average over a final portion of the iterates, starting at or after ⌈T/4⌉\lceil T/4\rceil. For a planned budget of TT updates, grouping tolerance δT≍T−1/5\delta_T\asymp T^{-1/5} and softplus smoothing ρT=δT2\rho_T=\delta_T^2 remain fixed during each run. The result gives O(T−4/5)O(T^{-4/5}) mean squared parameter error relative to the full Cox solution. Batches of events and weighted risk-set blocks are covered in the appendix.

The practical runs use the same step-size and averaging rules with tuned numerical constants and approximation levels. The theoretical prescriptions are based on worst-case bounds and are conservative at practical budgets, as is common in optimization theory. The tuned values are not claimed to satisfy the theorem's sufficient ranges. The finite-time bound separates optimization error from the bias due to fixed grouping and smoothing.

A separate illustration uses theoretical values on an unpenalized problem with 4,000 subjects and 1,600 events, with all assumptions checked over the whole parameter domain. Across four update budgets and 32 runs each, mean squared parameter error decreases from 3.9×10−53.9\times10^{-5} to 9.4×10−79.4\times10^{-7}. The final general bound is 1.6×10−41.6\times10^{-4}, compared with initial squared error 0.0610.061.

Q4: Unfavorable risk sets and heterogeneity. The computational analysis no longer requires every event risk set to contain a fixed positive fraction of the sample. Appendix B retains the dependence on H=1+log⁡(N/nmin⁡)H=1+\log(N/n_{\min}), where NN is sample size and nmin⁡n_{\min} the smallest event risk-set size, giving O((H/T)4/5)O((H/T)^{4/5}). The original Correlated 1M training sample already has a minimum risk fraction of 8.3×10−68.3\times10^{-6}.

Rapidly shrinking risk sets can reduce compression, while concentrated risk weights worsen the approximation and convergence bounds. Real-data weight and curvature diagnostics use selected parameter values; the theory illustration checks bounds over the whole domain. Ridge supplies curvature for penalized optimization; the unpenalized statistical result needs curvature near the population parameter and estimators that stay in that neighborhood. Optimizer seeds measure variation on fixed data, so robustness across independent datasets or repeated splits remains a limitation. We hope these revisions support reconsideration of your overall assessment.

Reviewer_YfNuOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper improves convergence guarantees for softplus LogSumExp optimization and applies risk-set grouping to linear Cox regression. Second-order gradient approximation error yields an end-to-end computational rate under stated curvature and risk-weight conditions, with a separate first-order statistical equivalence result.

Critical strengths

The paper has certain novelties.

Critical weaknesses
  1. Direct controls are missing. The experiments omit ungrouped softplus (K = m) and SCENT, leaving the benefit of grouping insufficiently isolated. The baselines optimize different objectives, so full-Cox gaps combine optimization error with objective mismatch. Figure 2 diagnoses approximation error but does not replace an optimization ablation.

  2. The practical advantage remains limited in scope. BigSurv reaches a 0.001 full-Cox gap with less work on four of five datasets (Table 5). An optimized exact Cox solver is used only as a reference. Under a rough three-vector-pass estimate per full gradient, the final caps correspond to about 80–620 full-gradient evaluations in vector work. Moreover, the experiments use fixed grouping/smoothing and averaged iterates, whereas Corollary 4.2 uses horizon-dependent choices and the last iterate. The accuracy and cost regimes favoring the proposed method remain unclear.

Questions for authors
  1. Could you compare grouped and ungrouped softplus at matched vector work, report memory, and include SCENT as a variational baseline?

  2. Could you compare accuracy versus total cost with an optimized cumulative-sum full-Cox solver, including setup and memory?

  3. Could a controlled experiment evaluate the horizon-dependent schedule and last iterate in Corollary 4.2 alongside the fixed-parameter averaged implementation?

1 comment

Dear Reviewer YfNu,

Thank you for your review and constructive suggestions. We have expanded the experiments and revised the convergence analysis. Changes are highlighted in dark blue.

Large-scale comparisons and direct controls. We added experiments on CASCADE (24.6 million training rows) and Taobao (53.8 million), whose feature matrices do not fit in the available RAM (out-of-core training). We plot the full-Cox optimality gap against optimization time. At 120 seconds, our median gaps are 1.7×10−41.7\times10^{-4} and 2.5×10−42.5\times10^{-4}, respectively, lower than all tested baselines, including optimized cumulative-sum L-BFGS, SCENT, ungrouped softplus and BigSurv.

Several of these methods are already stochastic. Our additional contribution is grouping neighboring risk sets so that their events share a normalization variable. Compared with ungrouped softplus and SCENT, this reduces the number of auxiliary variables from one per event to one per group. Each variable is then updated more frequently, which can improve convergence. The NWTCO training ablation varies grouping and smoothing separately. Relative to L-BFGS, sampled updates also avoid repeated full-data scans.

The expanded baseline set is included on the original in-memory datasets too. L-BFGS is strongest there, and the stochastic methods have mixed rankings. Full scans are comparatively cheap in this setting. Our method is designed for larger problems, combining efficient data access and compressed normalization state with the improved iteration complexity studied in the theory.

Theory and practical parameters. Revised Corollary 4.2 uses ηt=η01+(t−1)/t0\eta_t=\frac{\eta_0}{1+(t-1)/t_0} and suffix averaging, which averages only a final portion of the iterates starting at or after ⌈T/4⌉\lceil T/4\rceil. With TT updates, grouping tolerance δT≍T−1/5\delta_T\asymp T^{-1/5} and softplus smoothing ρT=δT2\rho_T=\delta_T^2, it gives O(T−4/5)O(T^{-4/5}) mean squared parameter error relative to the full Cox solution. The appendix also covers batches of events and weighted blocks of risk-set subjects.

The practical runs follow the same step-size and averaging rules, with tuned numerical constants and approximation levels. The theoretical values are based on worst-case bounds and are conservative at practical budgets, as is common in optimization theory. The tuned values are not claimed to meet the sufficient theorem ranges. The finite-time bound retains the bias from fixed grouping and smoothing separately from optimization error.

To illustrate the theorem with its prescribed values, we added an unpenalized problem with 4,000 subjects and 1,600 events, checking all assumptions over the whole parameter domain. Across four update budgets and 32 runs each, mean squared parameter error falls from 3.9×10−53.9\times10^{-5} to 9.4×10−79.4\times10^{-7}; the general bound falls from 6.0×10−36.0\times10^{-3} to 1.6×10−41.6\times10^{-4}. This illustration separately reports optimization error and the approximation bias from grouping and smoothing.

Reviewer_WVpvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses large-scale Cox regression by combining softplus approximation with risk-set grouping and optimizing the compressed objective using unbiased stochastic gradients. Under the corresponding assumptions and parameter schedules, the authors establish an O(T−1/2)O(T^{-1/2}) bound on the expected objective gap at the averaged iterate for general LogSumExp objectives, a O~(T−1)\widetilde O(T^{-1}) last-iterate mean squared error bound relative to the minimizer of the softplus surrogate objective, and a O~(T−4/5)\widetilde O(T^{-4/5}) mean squared error rate relative to the full Cox solution, together with asymptotic equivalence of the estimators. Experiments on two real and three synthetic datasets show that the method achieves a smaller final full-Cox objective gap under the same vector computation budget while reducing the number of auxiliary variables.

Critical strengths

S1. The research objective is clear, and the computational approximation of the full Cox objective is of practical significance.

The paper distinguishes computational approximation error from statistical model error and uses the full Cox objective and its numerical optimum as the main evaluation references. This setting is appropriate for examining whether an algorithm can approximate the full-data solution under a limited computational budget and provides a clear objective for comparing different stochastic training procedures.

S2. Risk-set grouping and its error analysis contain substantial technical contributions.

Theorem 4.1 provides control over the gradient, curvature, and optimal parameter bias. In particular, the gradient error induced by grouping is of second order, which is a key result supporting the compression of auxiliary states. Together with Theorem 3.2, the paper further establishes connections among approximation error, optimization error, and parameter estimation error.

Critical weaknesses

W1. The significance of the new contributions relative to existing softplus methods and SCENT has not been sufficiently demonstrated.

The paper adopts the softplus formulation introduced by Gladin et al., https://arxiv.org/html/2509.24894v4\#S2.SS2, Improved Stochastic Optimization of LogSumExp, with the main additions being expected-smoothness analysis, last-iterate guarantees, and risk-set grouping. Compared with Assumption 2.8 and Theorem 2.9 in version 4 of that work, the T−1/2T^{-1/2} bound in Theorem 3.1 of the present paper introduces smoothness assumptions and uniform boundedness conditions on individual losses and gradients. The earlier theorem allows nonsmooth losses and requires only bounded second moments of subgradients, although it additionally requires conditions on the second moments of normalized weights and bounds on the optimal value. Under the bounded domain assumed in the present paper, one can derive the stochastic gradient weight bound w≤e2B+1w\le e^{2B+1}. Standard projected SGD with ρ≍T−1/2\rho\asymp T^{-1/2} could also achieve a guarantee of the same order, although the constants may grow exponentially.

SCENT by Wei et al., https://arxiv.org/html/2602.02877v2, A Geometry-Aware Efficient Algorithm for Compositional Entropic Risk Minimization, ICML 2026 has already established an O(T−1/2)O(T^{-1/2}) bound for related convex problems under its Assumption 3.2 (Theorem 3.6), where the expression includes an average variance term along the iterate trajectory. Section 2 of the present paper already discusses its advantages in terms of explicit constants and last-iterate guarantees. However, a comparison of the dependence on B,κˉ,KB,\bar\kappa,K and the implementability of parameter choices under common assumptions, access models, and error metrics is still needed.

W2. Key baselines and grouping ablations are insufficient, and the practical computational benefits have not been adequately established.

The experiments lack comparisons with the original softplus method, SCENT, and optimized full-Cox solvers that exploit cumulative sums, as well as comparisons between grouped and ungrouped variants within the same implementation. The pointwise approximation errors in Figure 2 cannot replace training ablations. Although Appendix D.5 uses L-BFGS and Newton corrections to compute reference solutions, these methods are not included in the timing comparisons. Section 6 also acknowledges that a runtime advantage over mature full Cox solvers has not yet been established.

The advantage also depends on the required accuracy. At the 10−310^{-3} threshold in Table 5, BigSurv (Tarkhan and Simon, https://arxiv.org/abs/2003.00116, BigSurvSGD: Big Survival Data Analysis via Stochastic Gradient Descent) uses fewer computations on four of the five datasets. The vector counts in Appendix D.4 omit some practical overhead, while the timing comparisons mix NumPy and PyTorch implementations. I suggest comparing runtime to reach the same accuracy, memory usage, and the number of runs that fail to reach the target accuracy within the allotted budget on a small number of representative datasets, using comparable objectives, regularization, sampling and data-access conditions, and timing rules. The random-access constraints should also be clarified.

W3. The applicability of the convergence conditions and the error constants requires more specific diagnostics. Corollary 4.2 requires ∣Ri∣≥cN|R_i|\ge cN. For a fixed dataset, a very small cc can be chosen, but a complexity interpretation uniform in problem size requires that cc does not deteriorate as NN increases. Small risk sets near the end of follow-up may not satisfy this requirement. The authors should clarify whether the fixed observation-time window described in Appendix C is used, report the minimum risk-set proportion, and discuss complexity bounds that retain the dependence on log⁡(N/nmin⁡)\log(N/n_{\min}) when necessary.

Different theoretical results rely on different curvature conditions. The basic gradient and Hessian error bounds in Theorem 4.1 do not require strong convexity, whereas its parameter-bias result and Corollary 4.2 require global curvature conditions. Theorem 4.3 instead uses local curvature and localization. Ridge regularization in the experiments ensures strong convexity of the full objective, while the asymptotic equivalence of unpenalized estimators still relies on curvature provided by the data (Appendix C). Furthermore, the error bounds depend on Lq,κ,XL_q,\kappa,X, and the pointwise checks in Appendix F are insufficient to certify global conditions. Representative diagnostics of weight moments and curvature should be provided.

W4. The theoretical parameter schedules are inconsistent with the experimental parameters and output rules. Corollary 4.2 sets δT=(log⁡T/T)1/5\delta_T=(\log T/T)^{1/5} and ρT,ηT\rho_T,\eta_T in equation (23) according to a prespecified budget TT, and analyzes the last iterate. In contrast, the experiments fix δ=0.05\delta=0.05 and ρ=10−4\rho=10^{-4}, use ηt=η0(1+t/1000)−1/2\eta_t=\eta_0(1+t/1000)^{-1/2}, and report averaged iterates. Therefore, the experiments cannot directly verify this guarantee. Fixed approximation parameters may introduce residual bias relative to the full Cox solution. The authors should compare these two settings and use ablations on grouping strength, the number of groups, runtime, and final error to distinguish optimization error from approximation bias.

W5. The experimental coverage and evidence of robustness across datasets are limited. The two real datasets contain only 8,873 and 4,028 samples, and the large-scale evidence comes from simulations. In Appendix D.1, the correlated and independent covariate settings have censoring rates of approximately 90%90\% and 30%30\%, respectively, while the standard deviation of the linear predictor is 0.50.5 in both settings. The datasets and hyperparameter search menu were influenced by exploratory results, and the final experiments use a single split. Controlled experiments and repeated data splits are recommended to examine the benefits under varying risk sets and weight heterogeneity.

Questions for authors

Q1. What is the main algorithmic novelty beyond the existing softplus formulation? Compared with Gladin et al.\ and SCENT, what new algorithmic or theoretical contributions are provided by the grouping mechanism, expected-smoothness analysis, explicit constants, and last-iterate guarantees, respectively? Can the authors compare the complete complexity bounds and parameter information requirements under common assumptions?

Q2. Why are comparisons with optimized full-Cox solvers and recent LogSumExp methods missing? Can the authors provide same-accuracy runtime comparisons with mature full Cox solvers, the original softplus method, and SCENT on one or two representative datasets, together with grouped versus ungrouped ablations? If there are access or adaptation limitations, please explain them.

Q3. How representative are the assumptions required for the main convergence guarantees? What is the minimum risk-set proportion cc in the experiments, and is the observation-time window restricted? Can the authors separately clarify the curvature conditions required for ridge-regularized optimization and for establishing asymptotic equivalence between the grouped-softplus and full-Cox estimators in the unpenalized setting? Can the authors also report diagnostics for Lq,κL_q,\kappa, including whether they are pointwise or uniform?

Q4. Do the theoretical parameter schedules provide practical improvements? How do the theoretical (δT,ρT,ηT)(\delta_T,\rho_T,\eta_T) schedules compare with the fixed-parameter settings, and how do the last iterate and averaged iterate perform relative to each other? Can the authors show the relationships among grouping strength, auxiliary states, runtime, and final error, and analyze the approximation bias potentially introduced by fixed parameters?

Q5. At what accuracy levels does the method become computationally advantageous? In Table 5, BigSurv requires fewer computations in most cases at the 10−310^{-3} threshold. At what accuracy level or computational budget does the proposed method begin to show an advantage? Which practical tasks require higher solution accuracy sufficient to offset the additional computational cost?

Q6. How robust is the method to risk-set shrinkage and heterogeneous risk weights? With sample size, dimension, and regularization fixed, would changing the censoring mechanism or the dispersion of the linear predictor substantially affect the number of groups, approximation error, and runtime advantage? Can the authors provide a small-scale sensitivity analysis and report the actual changes in risk sets?

Q7. How stable are the conclusions across data realizations and splits? The current ten random seeds mainly reflect optimization randomness under the same data split. After fixing the hyperparameter search menu, do a small number of independent simulated datasets or repeated data splits still support the main conclusions? Please distinguish optimizer variability from uncertainty arising from the data themselves.

2 comments

Dear Reviewer WVpv,

Thank you for your careful review and for recognizing the theoretical contribution of our work. We have expanded the experiments and revised the convergence result. Manuscript changes are highlighted in dark blue.

Large-scale experiments (W2, W5; Q2, Q5). We added experiments on CASCADE (24.6 million training rows) and Taobao (53.8 million), whose feature matrices do not fit in the available RAM (out-of-core training). We plot the full-Cox optimality gap against optimization time. At 120 seconds, our median gaps are 1.7×10−41.7\times10^{-4} and 2.5×10−42.5\times10^{-4}, respectively, lower than all tested baselines, including optimized cumulative-sum L-BFGS, SCENT, ungrouped softplus and BigSurv.

Several baselines, including BigSurv, are also stochastic. Our additional mechanism is risk-set grouping: neighboring events share a normalization variable, reducing auxiliary state while controlling the error from approximating the full risk sets. Relative to L-BFGS, sampling also avoids repeated full-data scans.

We expanded the original in-memory benchmarks as well. L-BFGS is strongest there, and the stochastic methods have mixed rankings. Your observation about BigSurv's lower work at the original 10−310^{-3} threshold is valid. Our method is intended for larger problems, where full scans are expensive and the benefits of fewer normalization variables and improved iteration complexity become more relevant.

Schedules and output rule (W4; Q4). Revised Corollary 4.2 uses ηt=η01+(t−1)/t0\eta_t=\frac{\eta_0}{1+(t-1) / t_0} and suffix averaging: only a final part of the iterates, starting at or after ⌈T/4⌉\lceil T/4\rceil, is averaged. Here TT is the planned number of updates. With grouping tolerance δT≍T−1/5\delta_T\asymp T^{-1/5} and softplus smoothing parameter ρT=δT2\rho_T=\delta_T^2, fixed during each run, the result gives O(T−4/5)O(T^{-4/5}) mean squared parameter error relative to the full Cox solution. The appendix also covers batches of events and weighted blocks of risk-set subjects.

The practical runs use the same step-size and averaging rules, but tune their numerical constants, grouping tolerance and smoothing level. The theoretical prescriptions use worst-case bounds and are conservative at practical budgets, as is common in optimization theory. The tuned values are not claimed to satisfy the theorem's sufficient ranges. For fixed grouping and smoothing, the finite-time bound keeps their approximation bias separate from optimization error.

Illustration with theoretical parameters. A separate unpenalized Cox problem with 4,000 subjects and 1,600 events uses the prescribed values, with all assumptions checked over the entire parameter domain. Across four update budgets and 32 runs per budget, mean squared parameter error decreases from 3.9×10−53.9\times10^{-5} to 9.4×10−79.4\times10^{-7}. The general theoretical bound decreases from 6.0×10−36.0\times10^{-3} to 1.6×10−41.6\times10^{-4}, compared with initial squared error 0.0610.061.

Comparison with prior work (W1; Q1). SCENT's published bound contains trajectory-dependent variances; a coarse bounded-domain specialization gives the exponential constants shown below. Our analysis states its dependence on fixed problem parameters directly. In the table, the profile is the objective after minimizing over the auxiliary variables.

We compare expected optimality gaps for the original objective. The general LogSumExp comparison uses KK equally weighted log-partition functions of convex losses satisfying ∣Lk∣≤B\lvert L_k\rvert\le B, plus a convex regularizer. We hold the domain diameter and the applicable derivative, (sub)gradient second-moment, and curvature bounds fixed. Here κˉ\bar\kappa bounds the second moment of the normalized exponential weights.

AnalysisAdditional requirementsExpected objective gapAuxiliary variables in Cox
Original softplus (Gladin et al.)Nonsmooth losses allowedO((1+B)κˉ K1/4T1/4)O\left(\frac{\sqrt{(1+B)\bar\kappa}\,K^{1/4}}{T^{1/4}}\right)mm
Projected SGD (W1)Nonsmooth losses allowedO((1+B)Ke2B+κˉT)O\left(\frac{(1+B)\sqrt{K}e^{2B}+\bar\kappa}{\sqrt{T}}\right)mm
SCENT (Wei et al.)DifferentiabilityO(e2BK(1+B)T)O\left(\frac{e^{2B}\sqrt{K(1+B)}}{\sqrt{T}}\right) (coarse)mm
Our convex analysis (Thm. 3.1)Uniformly bounded, Lipschitz gradientsO(K(1+B)2+κˉT)O\left(\frac{K(1+B)^2+\bar\kappa}{\sqrt{T}}\right)mm
Our strongly convex analysis (Thm. 3.2 and Cor. 3.3)Thm. 3.1 regularity and a uniformly strongly convex profile for small ρ\rhoO(K[(1+B)4+κˉ2]T)O\left(\frac{K[(1+B)^4+\bar\kappa^2]}{T}\right) if the original minimizer is stationarymm
Our grouped Cox analysis (Cor. 4.2 and Appendix B)Strong convexity of the full Cox objectiveO((1+B)4+κˉ2T4/5)O\left(\frac{(1+B)^4+\bar\kappa^2}{T^{4/5}}\right) if the full Cox minimizer is interiorKT=O(T1/5)K_T=O(T^{1/5}), KT≤mK_T\le m

The first row extends Gladin et al.'s K=1K=1 argument to uniform sampling and balances the softplus parameter ρ\rho; their original theorem also allows unbounded losses. The SCENT entry uses bounded shifts to upper-bound its trajectory-dependent variances. Per sampled normalizer, the SCENT theorem uses two independent inner draws, and the general softplus comparison uses one. In Cox regression, direct application uses one auxiliary variable per observed event (K=mK=m), and grouped Cox uses one per risk-set group. The last two rates also bound mean squared parameter error to the respective original minimizers.

Theoretical improvements are thus as follow. With derivative, curvature and normalized-weight moment bounds fixed, our constants depend polynomially on the loss range BB. The fast rates use a curvature argument that controls joint parameter/auxiliary error without assuming uniform strong convexity in the auxiliary directions. Grouping also gives each auxiliary variable more frequent updates: probability 1/KT1/K_T per uniform draw, versus 1/m1/m with one variable per event. This can keep normalization estimates current and improve convergence at scale. The Cox row fixes the feature, curvature and risk-weight bounds and H=1+log⁡(N/nmin⁡)H=1+\log(N/n_{\min}), where NN is sample size and nmin⁡n_{\min} the smallest event risk-set size. Appendix B retains O((H/T)4/5)O((H/T)^{4/5}) when NN varies.

Assumptions and robustness (W3, W5; Q3, Q6, Q7). The computational analysis no longer requires every risk set to contain a fixed positive fraction of the sample. No additional follow-up cutoff was imposed on the original five benchmarks. Their minimum training risk-set fractions are 0.02160.0216, 0.1760.176, 8.3×10−58.3\times10^{-5}, 8.3×10−68.3\times10^{-6} and 1.7×10−51.7\times10^{-5} (SUPPORT2, NWTCO, Correlated 100k, Correlated 1M, Independent 100k).

Gradient/Hessian approximation bounds require no strong convexity; bounds on minimizer error and the computational rate use global curvature. The unpenalized statistical result needs curvature near the population parameter and estimators that stay in that neighborhood. Ridge supplies curvature for regularized optimization. Real-data diagnostics of weight concentration (Lq,κL_q,\kappa) and curvature are evaluated at selected parameter values; the theory illustration checks bounds over the whole domain.

Rapidly shrinking risk sets can reduce compression, and more concentrated risk weights worsen approximation bounds. Optimizer seeds measure variation on fixed data; robustness across independent datasets or repeated splits remains a limitation. We hope the new practical evidence and clarified guarantees support reconsideration of your overall assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.