- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1(−1)
- Soundness
- 2
Read review
Paper summary
This paper investigates how replay timing affects sample retention in constant-step online logistic regression. The authors establish a sharp worst-case replay-gap threshold for separable interfering pairs under fixed-size bursts and prove eventual all-time retention under subcritical replay-time ratios. The analysis is extended to a coupled multi-example family, providing a geometric sign criterion and a local guarantee under persistent ratio variation. Furthermore, the paper demonstrates that replay schedules with nearly identical cumulative update counts can lead to opposite retention outcomes. The results are supported by theoretical proofs and a limited numerical evaluation of the derived sufficient conditions.
Critical strengths
The paper provides a well-structured theoretical analysis of replay-gap thresholds in online logistic regression. The authors present a set of formally stated results with extensive proofs that examine the effects of replay timing and burst scheduling on sample retention. The paper also includes a numerical evaluation of the derived sufficient conditions in Table 1. Overall, the theoretical development appears technically rigorous and constitutes a strength of the submission.
Critical weaknesses
-
Limited Significance and Practical Relevance. The paper does not sufficiently establish the broader relevance of its theoretical results to continual learning. The analysis focuses on logarithmically sparse replay schedules, but it remains unclear which practical continual learning scenarios motivate such schedules or how the derived replay-gap thresholds could guide replay strategies. Furthermore, Section 2 defines retention solely in terms of positive training margins, while Section 8 acknowledges that the normalized rare margin can still converge to zero. The paper does not establish how this notion of retention relates to forgetting, generalization, or downstream task performance. Consequently, their significance for understanding practical continual learning remains insufficiently justified.
-
Limited Generalizability and overly idealized assumptions. Although the paper extends its analysis to varying replay-time ratios and a coupled multi-example family, the theoretical framework still relies on restrictive assumptions, including fixed-feature logistic regression, deterministic replay schedules, and specific sample geometries. In particular, the multi-example extension in Theorem 3 and Appendix F requires orthogonal common examples and a single interfering rare example. As acknowledged in Section 8, the results do not cover arbitrary larger datasets, multiple rare directions, stochastic replay laws, or representation learning. While these simplifications facilitate rigorous theoretical analysis, the paper provides insufficient justification that the identified replay-timing phenomena capture more general mechanisms of forgetting or replay effectiveness in continual learning systems.
Questions for authors
-
Could the authors clarify which practical continual learning scenarios motivate the logarithmically sparse replay schedules considered in this paper? How could the derived thresholds inform the design or analysis of practical replay strategies?
-
To what extent do the theoretical conclusions depend on the restrictive sample geometries and deterministic replay assumptions? Is there evidence or a theoretical argument suggesting that the identified phenomena persist in more general continual learning settings?
-
The paper primarily relies on mathematical derivations, with limited intuitive explanations or visual illustrations of its main findings. Could the authors provide more intuitive explanations or visualizations to help readers understand the theoretical results? For example, a comparison of replay schedules and their corresponding margin trajectories in Section 6 would help illustrate why schedules with nearly identical cumulative update counts can lead to opposite retention outcomes.
-
The Reproducibility Statement explicitly refers to an accompanying Python script for reproducing the numerical results in Table 1. However, there are no submission materials or anonymous link for code. Could the authors clarify its availability and provide the referenced artifact?