- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an implementation method for pooled Lasso in continual learning scenarios without replay. First, it points out that under L1 regularization, the previous Lasso solution cannot be directly used as the center of the quadratic historical penalty because its shrinkage bias accumulates with each task, causing the cumulative estimate to deviate from the joint training objective. The paper proposes the SP-CL algorithm, which maintains a cumulative curvature , satisfying (i.e., a first-order score without penalty contribution), and a cumulative sample size at each task boundary. In each step, it solves a quadratic penalized Lasso centered at , and proves that this yields a minimum point of the pooled Lasso. On the evaluation side, it decomposes the evaluation metrics into task conflict, information loss, and specialization for analysis. Experiments on both synthetic and real datasets validate the proposed mechanism.
Critical strengths
The paper proposes an accurate solution for pooled Lasso in continual learning scenarios without replay, and uses experiments to verify that the obtained cumulative solution is indeed consistent with the offline pooled Lasso trained on all historical data, while the method of directly passing the center is significantly degraded.
Critical weaknesses
Replay-based methods were excluded from the paper's protocol, but they remain a typical and strong family of methods in continual learning. The paper did not compare against any replay baselines (such as ER, Chaudhry et al., 2019) with equivalent memory budgets as a reference.
Questions for authors
Q1 [Suggestion]. Could a replay-based method be added as a reference under the same memory budget, and could a storage efficiency analysis of SP-CL be added?
4 comments
Response to Reviewer gXmG (2/2)
Response to Reviewer gXmG, part 2 of 2.
Permanent holdouts assess historical retention under a distinct protocol (Tables 7–9). Both replay versions retain 157 daily rows for NP/PJM and 200 rows for SPX. Generalist has lower final Overall and historical MSE than either replay under both validation criteria on all three streams. SPX historical MSEs for generalist/equal/weighted replay are .43947/.44640/.45841 under current-task validation and .42387/.45396/.44953 under history-aware validation. Both criteria use development data only, with frozen rules and ten seed-specific losses per replay version. Appendix I.4.1 also reports current-task metrics and BWT/FWT. BWT uses each method's own task-time baseline and is interpreted alongside absolute historical MSE; FWT compares with current-only Lasso after current-task training (Section 2; Proposition 2).
3. Agreement with full-history replay
When all history fits, , so both replay weightings give the same pooled-Lasso objective as SP-CL at a matched penalty. A separate full-buffer check gives an objective gap below . Financial text's sparse CSR history fits the matched budget; replay and generalist agree at the reported precision. This equality checks the historical objective, whereas the finite-buffer comparisons evaluate a different information constraint.
The permanent-company-holdout text evaluation also retains all 21,775 eligible historical reports. Replay/generalist predictions agree within across all evaluation updates (Appendix I.5.1, Table 12). Equal and weighted replay share this full-history row.
Dear Reviewer gXmG,
As the discussion period is approaching its end, we would like to kindly follow up on our response. We sincerely appreciate your thoughtful feedback and hope that our rebuttal and revisions have addressed your concerns.
If any concerns remain, we would be happy to provide further clarification.
Thank you again for your time and consideration.
Best regards, Authors of Submission 46828
Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts in addressing my concerns.
The replay comparison under a strictly matched memory budget is thorough and convincing. SP-CL's consistent advantage on the synthetic streams and most real-data benchmarks, together with the full-history equivalence verification, satisfactorily resolves my concern.
Overall, I have decided to raise my score from 2 to 3.
Response to Reviewer gXmG (1/2)
Response to Reviewer gXmG, part 1 of 2.
Thank you for suggesting a replay comparison under the same memory budget.
Our primary focus is regularization-based continual learning under a protocol that retains approved historical summaries. DataSHIELD provides a related access setting: individual records remain at their source, and the analyst receives disclosure-controlled aggregate outputs. We additionally evaluate experience replay under the same retained-memory budget, allowing the learner to retain historical samples.
1. Replay implementation and memory budget
Our primary baseline is experience replay (ER; Chaudhry et al., 2019) with reservoir sampling (Vitter, 1985), using Lasso for fitting. A uniform reservoir Mt−1 contains mt past labeled rows before task t. Each current and cached sample receives weight one:
\left\{\frac{\|Y_t-X_tb\|_2^2+\|Y_{\mathcal M_{t-1}}-X_{\mathcal M_{t-1}}b\|_2^2}{2(n_t+m_t)} +\lambda_t^{\mathrm{ER}}\|b\|_1\right\}.$$ This is equal weighting per sample, not equal total weight per group. The reservoir is updated after fitting. We separately tune replay's penalty using the actual training size $n_t+m_t$; current observations are not counted twice. We retain historical-weighted replay as a supplementary control: it multiplies the historical residual sum by $N_{t-1}/m_t$ and normalizes by $2N_t$. The versions use identical buffer capacities, seeds, and splits, with separate validation-only penalty selection and the same Lasso KKT tolerance. In the long-stream experiment, arrays, both prediction heads, schedules, bookkeeping, and metadata give SP-CL a budget of 468,723 bytes. Both replay versions use the largest feasible buffer: 240 rows taking 467,177 bytes; one more row exceeds the budget. Current inputs and temporary solver workspace are accounted for separately. Penalties are selected on four independent pilot streams before evaluation, without formal-test outcomes or true coefficients. SP-CL's $O(p^2)$ state preserves historical squared loss independently of stream length; the comparison tests its benefit once a fixed reservoir cannot retain all history. **2. Results after the reservoir fills** Appendix I.1 uses $p=240$, $s=8$, $n_t=40$ and $T=30$, giving $N_T=1,200$. Replay retains all history when fitting task 7 and first subsamples it at task 8. Final population excess risks exclude noise and report mean $\pm$ SE over 20 paired streams: | Setting and evaluation target | SP-CL generalist | Replay (equal) | Replay (weighted) | | --- | ---: | ---: | ---: | | Shared coefficients | $0.033\pm0.002$ | $0.173\pm0.014$ | $0.239\pm0.020$ | | Task shift: current task | $0.695\pm0.016$ | $0.878\pm0.055$ | $1.030\pm0.061$ | | Task shift: previous-task average | $0.226\pm0.002$ | $0.384\pm0.016$ | $0.423\pm0.022$ | The shift path keeps support fixed with $\|\beta_T^\star-\beta_1^\star\|_1=3$. Equal replay improves on the weighted control, while generalist retains lower current and historical risks in these settings. Figure 8 shows both replay trajectories across the capacity threshold. The separate chronological forecasts yield the following prospective response MSEs. Penalties use earlier validation data; replay losses average five electricity or ten SPX reservoir seeds. | Stream | Generalist | Specialist | Replay (equal) | Replay (weighted) | | --- | ---: | ---: | ---: | ---: | | Nord Pool | 16.691254 | 17.529177 | 17.225040 | 17.861476 | | PJM | 24.593975 | 25.150766 | 26.051670 | 28.984537 | | Oxford-Man SPX | 0.424512 | 0.381271 | 0.405277 | 0.431155 | | Financial text | 0.150242 | 0.143020 | 0.150242 | 0.150242 | Equal replay improves on weighted replay in all three market streams. On SPX it also outperforms generalist, while specialist remains better than both replay variants.