- Rating
- 2
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SMOOTHCUM, a neural estimator of the cumulative mean model , where each is a -dimensional nonlinear and unknown function. SMOOTHCUM keeps the cumulative structure of (1) but does not impose additivity. It replaces the separate per-visit output layers of the sub-network estimator by one shared network and a fixed set of smooth basis functions of time in the output layer, so that the contribution of each visit is an explicit smooth function of time with coefficients. For a sample-split version of the estimator, the paper derives an exact decomposition of the risk into squared bias and variance, in which the variance term is proportional to , and gives an approximation-error decomposition under the low time-rank, time-basis approximation and completeness assumptions. Simulations compare SMOOTHCUM with DLNM, the sub-network estimator, the two-step estimator, and sequence models; further experiments study irregular observation times, the smoothness sweep, and model averaging. Cohort analyses provide real-data evidence; however, no cohort in the paper's set has a non-additive target, so the real-data evidence covers the additive class only.
Critical strengths
S1: Appropriate inductive bias for small longitudinal cohorts. The shared feature extractor combined with a low-dimensional temporal basis preserves the cumulative-over-visits structure while avoiding the restrictive additivity-across-covariates constraint of DLNM. This effectively targets the critical regime between overly restrictive DLNM and flexible but data-hungry sequence models.
S2: Mechanism-focused evaluation. The empirical analysis goes well beyond a standard benchmark ranking. By decomposing the risk into squared bias and variance over repeated refits, isolating the individual effects of body sharing and the smooth temporal basis, evaluating irregular observation times, and honestly reporting regimes where DLNM or sequence models win, the paper makes its finite-sample claims highly transparent and interpretable.
Critical weaknesses
W1: Disconnect between the theoretical analysis and the deployed estimator. Theorem 1 conditions on a representation trained on a separate fold and uses a fixed-design least-squares projection. However, the empirical method is jointly trained and evaluated out-of-sample. Thus, the exact in-design decomposition offers no corresponding guarantee for the practical procedure. This gap is compounded in Appendix A, where approximation control depends on a realized feature-completeness event, without showing that the optimization process can attain it. This substantially limits the theoretical explanation of the method's success.
W2: Incomplete ablation of performance drivers. SMOOTHCUM differs from the primary DLNM baseline along multiple dimensions simultaneously: it uses an absolute-visit-time representation versus lag indexing, and allows covariate interactions while DLNM does not. Furthermore, the main architecture grid fixes (Appendix G), relegating the adaptive- procedure largely to real-data tuning. More controlled comparisons and a clear end-to-end evaluation of the adaptive- procedure are needed to isolate which specific architectural choices drive the claimed gains.
Questions for authors
Q1: Theory / practice. Can you quantify the gap between the sample-split, frozen-feature least-squares estimator analyzed in Theorem 1 and the jointly trained estimator used in practice? Specifically, can you compare in-design and held-out-subject risks under matched settings, or derive an additional bound for the jointly trained procedure?
Q2: Dependence and shared representation. In Appendix A.5, which precise assumptions justify the application of the multitask representation bound to within-subject correlated observations? How do the constants and the apparent benefits of larger change under strong within-subject dependence?
Q3: Controlled baselines. Can you evaluate two additional baselines: (i) an additive model with the same absolute-time spline basis, and (ii) a cumulative neural model with a lag-indexed representation? This would disentangle the effects of covariate interactions from absolute-time modeling.
Q4: Basis selection. Which headline results rely on the fixed versus cross-validated ? Given the separate -sweep and real-data tuning, can you provide an aggregate comparison of fixed- and adaptive- SMOOTHCUM on the principal test grid?
Q5: Motivation for the condition-free design matrix claim. Theorem 1 explicitly states that "no structural assumption and no lower eigenvalue condition on enters." Since no such condition was emphasized earlier, could you elaborate on the motivation behind this specific claim? Given that the use of the Moore-Penrose pseudo-inverse mathematically bypasses the need for invertibility, does this condition-free property come at the cost of the variance bound becoming highly conservative in practical regimes with collinear covariates or redundant basis functions? How should practitioners reconcile this theoretical generality with the practical necessity of regularization?
4 comments
W2, Q3 and Q4 (the comparison with DLNM confounds time indexing and interactions, and the grid fixes K=3, so add controlled baselines and compare fixed with adaptive K).
We agree on both points: the comparison with DLNM confounded two design choices, and the main grid fixed . The revision addresses the first with the two controlled baselines of Q3 and the second with the adaptive- procedure of Q4, run end to end on the full grid.
Controlled baselines. Both baselines are run on the full comparison grid (Table 19, page 33): DLNM with our absolute-time B-spline in place of its lag polynomial, and SmoothCum with its basis evaluated at the lag . On the non-additive , moving DLNM from lag to absolute time changes its ISE by a factor of ( to ) and moving SmoothCum from absolute to lag time by ( to ). Holding the time representation fixed at absolute time, allowing interactions reduces the ISE by a factor of ( to ). On the additive , changing the time index moves the error by at most , while DLNM leads by a factor of . The gain is covariate interactions. Time indexing contributes only a few percent either way.
Fixed against cross-validated . We add the comparison the reviewer asks for. Table 20 (page 33) runs the adaptive procedure end to end on all settings of the principal grid (first replicates, paired with the fixed- estimator in the same replicate). Cross-validated has the lower ISE in of settings, by on average ( against , paired difference with SE ), so the fixed of the main tables understates the estimator rather than flattering it. The selected tracks the true coefficient: in half the fits where is constant, in three quarters where it increases, and picks either or where it alternates, since an alternating coefficient is too rough for an intermediate to follow.
Both additions agree with the conclusions of the initial submission. The rule of Section 6.3, SmoothCum for non-additive targets and DLNM for additive ones, stands and rests on covariate interactions rather than on the time index, and the fixed of the main tables is conservative for SmoothCum, so no reported comparison moves against it.
Q2 (assumptions under within-subject correlation, and the benefit of larger K under strong dependence).
The multitask bound is removed (see our reply to Reviewer 2qrW, W1). In the bounds that remain, within-subject dependence enters only through , the largest eigenvalue of a subject's residual covariance, which multiplies the whole noise term and leaves the approximation term untouched (Appendix A.6, page 21, lines 1130–1133). Two consequences follow.
The constant in front of the noise term. Dependence inflates the noise term by a factor we measured: the ratio rises from under independent residuals to under AR(1) and under exchangeable correlation for SmoothCum (, and for the sub-network estimator) and is flat in (Table 18, page 32).
The benefit of a larger . A larger lowers the approximation term and adds of noise per basis function. Therefore, strong dependence raises the noise cost of each added basis function and makes the bound-optimal of Corollary A2 smaller, since decreases in . The benefit of a larger therefore shrinks under strong dependence, while the variance bound at remains times the bound of the per-visit output layer, a ratio that does not depend on . The experiments agree: the -setting stress study with residual correlation (Appendix C) keeps the method ranking, and in the -sweep arm with the advantage over the per-visit estimator flattens with , as the cancelled in the noise term predicts.
Q5 (the condition-free claim, the pseudo-inverse and regularization).
The claim says only that the bound of Theorem 1 does not depend on the smallest eigenvalue of , and a new remark explains why (Appendix A.6, page 21, lines 1120–1129). Because is a projection, the identity and the trace bound need no lower eigenvalue condition, so rank deficiency from dead or duplicated hidden units costs nothing in the theorem. Collinearity lowers the rank and tightens the trace bound, so the condition-free statement is not conservative in design. Where conditioning matters is the transfer from in-design to held-out risk, which needs the Gram matrix of the trained features to be stable between training and test subjects, and that is where regularization earns its place. We measured where collinearity bites in practice (Table 17, page 32): the exact least-squares refit changes held-out ISE by more than in of replicates, always for the worse, when a near-singular direction of the trained design receives a large coefficient that the jointly trained layer never acquires. That is why we deploy the jointly trained output layer, which does not reach the exact least-squares solution, state Theorem 2 for the refit layer, where the normal equations hold exactly, and report the difference between the two in Table 17.
We thank the reviewer again for questions that each produced a table we would not otherwise have made, and we would be happy to answer any further questions.
I appreciate the authors' substantial revision. However, my main concerns remain, and I found several additional issues in the revised manuscript.
- The gap between Theorem 2 and the deployed estimator remains.
Although Theorem 2 extends the analysis to the jointly trained network, its exact risk decomposition requires the output layer to satisfy the least-squares normal equations. This condition is guaranteed by least-squares refitting, but not necessarily by the jointly trained output layer used in deployment. Table 17 shows that refitting increases held-out ISE by 6.3% on average and performs worse in 148 of 150 settings. Thus, the difference between the analyzed and deployed estimators is not negligible. Moreover, Theorem 2 only addresses in-design risk and does not provide a held-out risk guarantee or control the approximation term of the jointly trained features. My original theory–practice concern therefore remains only partially addressed.
- Unclear specification of the evaluated estimator.
The general experimental protocol states that is selected by cross-validation, and Appendix C.3 states that is never fixed in advance. However, Appendices C.2 and C.10 indicate that the main comparison grid uses fixed . These are different estimation procedures. The authors should clearly state which procedure was used for each main result and which conclusions are supported by the cross-validated estimator.
- Inconsistency in the bias–variance decomposition.
The manuscript states that the variance across 60 refits is computed with a divisor of 59. However, the ISE values in Table 6 appear to be obtained by directly adding squared bias and variance. Under the stated variance convention, the corresponding ISE values should be approximately 12.41 and 24.47, rather than 12.50 and 24.71. This difference is larger than rounding error, contrary to the authors' explanation. Since the bias–variance decomposition is central to the paper, this inconsistency needs to be addressed.
- Inconsistent experimental results across tables.
Tables 2 and 17 report different deployed-estimator ISE values for the same 150-setting comparison grid: 15.99 versus 16.49 for SMOOTHCUM, and 23.86 versus 24.45 for Subnet-Cum. The real-data results also differ across tables. For example, the HRS SMOOTHCUM MSE is reported as 0.8628, 0.870, and 0.862 in Tables 27–29, while the CUMBLEND MSE changes from 0.856 to 0.849. Although the manuscript mentions rerunning experiments, these differences are not clearly explained.
- Empirical claims are not fully supported by the reported results.
Several claims appear inconsistent with the tables. The manuscript states that all competing methods on HRS have error ratios between 0.99 and 1.01 relative to SMOOTHCUM, but Table 26 includes substantially larger ratios, reaching approximately 1.09 for some history-based methods. The broad claim of superiority under irregular observation times also needs qualification, since Table 16 shows that DLNM clearly outperforms SMOOTHCUM on both predominantly additive settings. Furthermore, the statement that every single-model MSE on the food-group panel is within 4% of the mean-only predictor is inconsistent with the SMOOTHCUM MSE of 1.296 in Table 29.
Although the revision includes meaningful improvements, the remaining theory–practice gap and the newly identified experimental inconsistencies weaken my confidence in the paper's soundness. Therefore, I keep my original rating.
We are grateful to the reviewer for a careful reading of our manuscript and for five constructive questions, especially Question Q1, which asked us to quantify the gap between the analysed estimator and the deployed one. In this revision we have addressed this gap with a new theorem and two new tables, and we respond to each question below.
W1 and Q1 (theory analyses a frozen, sample-split estimator, so quantify the gap and compare in-design and held-out risk).
We did both. New Theorem 2 (page 6, lines 288–313, and Appendix A.6, pages 19–22) gives the exact decomposition for the jointly trained network with its output layer refit by least squares, so that the normal equations hold, with the noise term bounded through the pseudo-dimension. The gap to the deployed estimator is then one step, and we measure it in two tables.
Refit against deployed (Table 17, page 32, all 150 settings, 200 replicates, paired on the same trained networks). The jointly trained output layer sits 1.5% from stationarity in gradient norm. Refitting it lowers the in-design risk and raises the held-out ISE by 6.3% for SmoothCum (worse in 148 of 150 settings), by 7.3% for the sub-network estimator and by −0.3% for the variant with time as an input. The ordering of SmoothCum against DLNM changes in 3 of 150 settings and against the GRU in none.
In-design against held-out (Table 18, page 32, reference setting, 60 refits, K=1,…,5 and the sub-network estimator). The identity holds to machine precision in every fit. The realized noise term exceeds the fixed-network reference tr(PΣ)/N by a factor of 1.32 to 1.76, and both terms grow linearly in the design rank. The held-out ISE is about twice the in-design risk for every estimator, with the same ordering of K and of the two architectures. The deployed layer is within 0.05 of the refit layer in-design and 1 to 5% better held-out.
The new theorem and the two added tables are what make the paper's explanation of the gain, variance reduction through the output layer and the shared network, stand for the estimator we deploy. Theorem 2 makes the mechanism, noise passed through a projection whose size scales with K and with the parameter count, a property of the estimator we deploy rather than of a stand-in. Table 17 shows that the analysed estimator and the deployed one differ by about 6% in held-out error and never change order against the competitors, so the theorem speaks about the estimator the tables report. Table 18 shows that the trained network's noise term exceeds the fixed-network value by a bounded factor, which is the cost of fitting the features on the same noise, and that both terms grow linearly in the design rank as Theorem 1 predicts, so the mechanism at work is the one the theory describes. It also shows that in-design and held-out risk differ by a factor and not in their ordering, so a conclusion drawn from the quantity the theorems bound carries over to the quantity the experiments report. The small held-out advantage of the deployed layer over the refit layer is the implicit regularization of stopping short of the least-squares solution. A convergence guarantee for the trained features is beyond current theory for networks of this size, so we measure the approximation term instead (Table 18), as the limitations now state (Section 8, page 10, lines 524–525).