acceptodds
Under review as a conference paper at ICLR 2027

The Sequential Price of Continual Learning

Abstract

Sequential task updates are fundamental to continual learning, but their recency bias can impose a lasting performance cost. We study this cost in an overparameterized linear-regression model with i.i.d. task sampling. We prove that distribution-level forgetting and population loss converge to the same stationary limit. We quantify the additional loss incurred by sequential exact fitting, or the sequential price. In more homogeneous task geometries, it equals the intrinsic loss asymptotically attained by joint training, making the total loss twice as large. We further analyze fixed-strength elastic weight consolidation (EWC) under general task curvatures and characterize its stationary sequential price at every regularization strength. Under strong regularization, the price decays inversely with EWC strength while the mean-square coupling horizon grows proportionally. Experiments on Jester and Rotated MNIST support the predicted sequential price and its reduction by EWC, with quantitative agreement on real-world tasks satisfying the theory's assumptions and qualitative agreement under nonlinear finite-step training.

Primary area:
learning theory
Keywords:
continual learning, catastrophic forgetting, sequential learning, elastic weight consolidation, learning theory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nRENOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the price of sequential task updates in continual learning. The authors build on independently task sampling procedure and an overparameterized linear regression model. Theoretically, they prove that distribution-level forgetting and population loss converge to the same stationary limit, with the gap between them vanishing at rate of O(1/T). They further decompose the limiting loss into the unavoidable loss caused by task conflicts and an additional sequential price. They also analyze a fixed-strength, population-curvature EWC regularizer and show that, in the strong-regularization regime, the stationary excess loss decreases at rate of O(1/\lambda). Experiments on real dates provide quantitative and qualitative support, respectively, for the predicted sequential price and its reduction.

Critical strengths

I think the main contribution of this paper lies in its theoretical insights. The authors assess the cost of sequential learning and characterize its asymptotic behavior. They distinguish the unavoidable loss caused by conflicting tasks from the excess loss introduced by sequential exact fitting. This unavoidable loss, or minimal achieved loss, provides a lower bound, giving a meaningful benchmark for evaluating sequential learning. The results clarify how task curvature and long-run parameter fluctuations contribute to persistent error, and identify homogeneous curvature conditions under which sequential exact fitting incurs twice the loss of joint training. The EWC analysis further shows how regularization can reduce the stationary excess loss. While the dependence will decay slowly.

Critical weaknesses

The main limitation is that the theoretical insights rely on linear models, quadratic loss, independent task sampling, and exact per-task optimization. Although it is a standard starting point, I am not sure whether the findings hold in complex settings. It will be interesting to provide a more detailed discussion of how violations of these assumptions affect the results and the technical difficulties involved in extending the theory.

Questions for authors

In the EWC analysis, it assumes that the population curvature is known. While sample curvature is typically used in application. Could the authors clarify the rate of estimation error vanishes. how this error affects the theoretical results and finite-sample performance? Would the resulting rates depend jointly on the sample size and lambda.

The paper specifically focuses on EWC. Would similar findings hold for other penalties

1 comment
AuthorsOpenReview ↗

Revised manuscript: assumptions and extensions

We have substantially revised the manuscript to clarify the assumptions behind each result and address the scope questions you raised. Section 6 now explains which conclusions depend on linearity, quadratic loss, independent task sampling, and the specific update rule. Section 2.3 shows how real Jester ratings, with a natural encoding and training procedure, satisfy the theoretical assumptions, while Appendix G examines finite-step nonlinear training and discusses where the linear-model predictions cease to hold. Regarding estimated curvature, we clarify that the full Jester user population defines our experimental task distribution, so its population curvature is available exactly; our earlier response separately addresses estimation from independent tasks and the joint dependence on sample size and regularization strength. Appendix E.4 also explains how the stationary calculations extend to other fixed positive-definite quadratic penalties, including an independently estimated curvature matrix held fixed during training. Throughout the main text, we have made the motivation and interpretation of the results more explicit. We would appreciate your assessment of whether these revisions and our earlier response address your questions.

Reviewer_VBA8OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper quantifies the sequential price of continual learning in an overparameterized linear-regression model. It further characterizes how fixed-strength EWC changes this price under general task curvatures, with experiments on Jester and Rotated MNIST supporting the theoretical predictions.

Critical strengths

The paper makes a theoretical contribution by proving that distribution-level forgetting converges to population loss. It further extends this analysis to fixed-strength EWC and supports the predictions empirically.

Critical weaknesses
  1. The theoretical analysis is built on a linear model, which is quite restrictive given that current models are highly nonlinear.

  2. The paper is framed as a theoretical analysis paper, but the exposition of the theory and the proofs rely heavily on AI, as the authors acknowledge in the AI USE STATEMENT. This substantially reduces the authors' contribution.

  3. The paper lists a series of theorems. Each theorem is followed by a proof sketch, but there is no discussion of the theorem itself. As a result, readers are left with a set of hard-to-follow theorems and cannot really tell what the authors want to present.

  4. The related work section misses many recent theoretical analyses of CL and does not adequately compare with recent theoretical work based on linear models, neural tangent kernel, etc.

Questions for authors

As the paper relies heavily on AI, could the authors provide the prompts used and explain specifically how AI was used in developing the theory and proofs?

1 comment
AuthorsOpenReview ↗

Revised manuscript: insights and scope

We have substantially rewritten the main text to clarify the continual-learning insights and the scope of our results. In response to your concern about the linear setting, Section 4 now clarifies that comparing sequential learning with the joint-training optimum does not require a linear model: this comparison separates the loss inherent to the tasks from the additional loss caused by sequential training. Our linear analysis then characterizes how large this additional loss is and how EWC reduces it. Section 2.3 also explains how real Jester ratings, with a natural encoding and training procedure, satisfy our theoretical assumptions. Regarding the insights and exposition, we have reorganized the paper around why forgetting and population loss share a common limit, how the joint-training benchmark separates unavoidable task conflict from the sequential price that continual-learning methods should aim to reduce, and how EWC changes that price. We have replaced much of the proof-sketch-heavy presentation with explanations of what the results establish and why they matter, and clarified their relationship to prior work. We would appreciate your assessment of whether the revised manuscript addresses your concerns about the paper’s insights and presentation.

Reviewer_3mt7OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies continual learning in overparameterized linear regression under i.i.d. task arrivals from a distribution Π\Pi when tasks are individually realizable but globally incompatible. Analyzing minimum-norm sequential exact fitting as a contractive random affine recursion, the authors show that historical loss FTF_T and population loss GTG_T converge to the same stationary limit G∞G_\infty at rate O(T−1)O(T^{-1}) (Theorem 3.1). They decompose $G_\infty$ into the joint-training optimum R∗R^\ast and a "sequential price" Δseq\Delta_{\text{seq}} (Theorem 4.1), which satisfies Δseq=R∗\Delta_{\text{seq}} = R^\ast under homogeneous nonzero task curvatures (Corollary 4.2). Finally, they analyze fixed-strength EWC weighted by the population curvature Cˉ\bar{C}, showing that as λ→∞\lambda \to \infty, the sequential price decays as κΠ/λ\kappa_\Pi/\lambda while the coupling horizon grows as λ2log⁡(1/η)\frac{\lambda}{2}\log(1/\eta) (Theorem 5.2), with numerical illustrations on Jester and Rotated MNIST.

Critical strengths
  • It raises the concept of steady-state sequential oscillation. This is a concept that would have lead to clear acceptance if demonstrated correctly and clearly.
Critical weaknesses
  • Ill-Defined Limiting Random Variable W∞W_\infty and Unspecified Probability Spaces: In Section 2, the core object of the paper is introduced simply as W∞∼νW_\infty \sim \nu. For a theory paper with W∞W_\infty present in the main Theorem 4.1, this is a fatal flaw: the subscript ∞\infty implies the iterate sequence WtW_t converges almost surely or in L2L^2 to a random variable W∞W_\infty, whereas WtW_t actually oscillates perpetually under fresh i.i.d. task arrivals and only converges in distribution to ν\nu. A stationary random variable W0(∞)W_0^{(\infty)} is only constructed in Appendix C by extending time backward to −∞-\infty. Furthermore, Theorem 4.1 writes W∞′=PτW∞+ZτW_\infty' = P_\tau W_\infty + Z_\tau without specifying the underlying joint probability space, and the expectation operator E\mathbb{E} is used throughout without subscripts to distinguish averaging over a single task τ∼Π\tau \sim \Pi, the training history τ1:T\tau_{1:T} and initialization W0W_0, or the stationary law ν\nu.

  • Pervasive Notational Ambiguity and Collisions:

  1. Circular task indexing: In Section 2, τ=(Xτ,yτ)\tau = (X_\tau, y_\tau) is simultaneously the task tuple and its own index subscript, before switching to time subscripts tt in Pt,ZtP_t, Z_t, leaving CtC_t in Lemma C.2 undefined, and switching to integer subscripts ii in Section 6.

  2. Inconsistent overbars: Overbars denote deterministic population expectations in Cˉ,Pˉ,Qˉ\bar{C}, \bar{P}, \bar{Q}, and hˉ\bar{h}, even though hτh_\tau is never defined, E[Zτ]\mathbb{E}[Z_\tau] lacks a bar, and norms like ∥⋅∥Cˉ−12\Vert{}\cdot\Vert{}_{\bar{C}^{-1}}^2 are visually cluttered. Yet Lemma C.2 uses CˉT\bar{C}_T and hˉT\bar{h}_T for random empirical averages, while Appendix F.1 uses Dˉ\bar{\mathcal{D}} for an input mixture distribution.

  3. Symbol overloading: Section 5 and Appendix D.4 reuse WtW_t and W∞W_\infty without λ\lambda-indexing for a completely different EWC stochastic process. The symbol hˉ\bar{h} is a cross-moment vector in Section 2 while hjh_j is a scalar second moment in Appendix E.4. The active subspace projector QτQ_\tau in Corollary 4.2 collides with the matrix operator QτQ_\tau in Lemma C.4 and the forcing covariance QQ in Theorem 5.2's proof. The dimensionless condition number κ\kappa in Corollary 4.2 collides with κΠ\kappa_\Pi in Theorem 5.2, which has units of loss. Finally, ∥SΠ∥op,F\Vert{}S_\Pi\Vert{}_{\text{op}, F} in Section 3 is used without defining the (op,F)(\text{op}, F) super-operator norm.

  • Unrealistic Assumption in Corollary 4.2 and Degeneracy of the Twofold Law: Corollary 4.2 assumes mQτ⪯Cτ⪯LQτm Q_\tau \preceq C_\tau \preceq L Q_\tau almost surely for deterministic constants 0<m≤L<∞0 < m \le L < \infty, requiring all nonzero eigenvalues of CτC_\tau to be bounded away from 00 and ∞\infty with probability one. This fails even for standard i.i.d. Gaussian designs where extreme singular values occur with positive probability, explodes near the interpolation threshold nτ≈dn_\tau \approx d, and yields loose bounds under power-law spectra. Moreover, the headline twofold law G∞=2R∗G_\infty = 2R^* requires either Cτ=cQτC_\tau = c Q_\tau almost surely—restricting feature matrices to scaled partial isometries with flat singular values—or Cτ≡C≻0C_\tau \equiv C \succ 0 almost surely. In the latter case, Pτ=0P_\tau = 0 and Wt=ZtW_t = Z_t as noted in Appendix D.3, reducing continual learning to memoryless single-task overwriting.

  • Misleading Definition of Forgetting FTF_T: Defining FTF_T as the raw average historical loss in Section 2 only measures true forgetting when acquisition loss ℓτs(Ws)\ell_{\tau_s}(W_s) is zero. Under conflicting tasks, EWC with λ>0\lambda > 0, or finite-step training, FTF_T is merely historical empirical risk: under this metric, even the static offline joint-training oracle w∗w_* suffers positive forgetting equal to R∗R^*.

  • Incremental Novelty Over Inconsistent Kaczmarz and Constant-Step SGD: The recursion Wt=PtWt−1+ZtW_t = P_t W_{t-1} + Z_t with R∗>0R^* > 0 is identical to randomized block Kaczmarz on inconsistent linear systems, where the steady-state error decomposition of Theorem 4.1 is classic. Similarly, Theorem 3.1 follows immediately by summing the geometric mixing bound ρΠ(T−s)/2\rho_\Pi^{(T-s)/2}, and Theorem 5.2 is a standard 1/λ1/\lambda small-step expansion of constant-step-size quadratic SGD Markov chains, where the O(1/λ)O(1/\lambda) variance versus O(λ)O(\lambda) mixing horizon trade-off is textbook.

Questions for authors
  1. Formalizing W∞W_\infty and Resolving Notational Collisions: Will the authors revise the main text to state weak convergence Wt→dνW_t \xrightarrow{d} \nu explicitly rather than writing W∞∼νW_\infty \sim \nu as if WtW_t converged as a random variable, specify the probability spaces for all expectations, and fix the notation collisions across τ\tau, overbars, Wt/W∞W_t/W_\infty under EWC, hˉ/hj\bar{h}/h_j, Qτ/QQ_\tau/Q, and κ/κΠ\kappa/\kappa_\Pi?

  2. Justification of Corollary 4.2 and Novelty Over Kaczmarz/SGD: Since mQτ⪯Cτ⪯LQτm Q_\tau \preceq C_\tau \preceq L Q_\tau almost surely fails for standard Gaussian designs and Cτ≡C≻0C_\tau \equiv C \succ 0 degenerates into memoryless overwriting Wt=ZtW_t = Z_t, what realistic non-diagonal feature distributions satisfy Corollary 4.2 with small κ\kappa? Additionally, what is the core mathematical novelty of Theorems 4.1 and 5.2 beyond inconsistent randomized Kaczmarz and constant-step quadratic SGD?

1 comment
AuthorsOpenReview ↗

Revised manuscript and clarified exposition

We have substantially rewritten the main text to present the sequential price and the role of each theoretical result more clearly. In response to points 1 and 2, we have made the stationary-distribution formulation and probabilistic definitions explicit, clarified task indexing, and standardized notation throughout the paper and proofs. For point 3, we now present the general sequential-price decomposition before the curvature result and explain directly how real Jester tasks satisfy the assumptions. For point 4, we clarify why historical loss measures forgetting under exact fitting and distinguish the historical-loss metric used in finite-step experiments. For point 5, we have reorganized the introduction and theoretical exposition around the connection between forgetting and population loss, the separation of sequential price from intrinsic loss using the joint-training benchmark, and the effect of EWC on that price. We would appreciate your assessment of the revised presentation and contribution.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised manuscript: overview of changes

We have uploaded a substantially revised manuscript and updated supplementary material. We thank the reviewers for identifying places where the motivation, notation, and interpretation of our results needed clearer presentation.

The revised main text follows a clearer progression: connecting forgetting with population loss, using the joint-training optimum to separate intrinsic loss from the sequential price, and analyzing how EWC reduces that price. We have clarified the probabilistic definitions and assumptions, standardized notation, and replaced much of the proof-sketch-heavy exposition with explanations and examples of what the results mean for continual learning.

We also explain directly how real Jester ratings, with a natural encoding and training procedure, satisfy the theoretical assumptions, and place the corresponding experiments alongside the results they illustrate. The discussion distinguishes conclusions that require our linear setting from the broader use of the joint-training benchmark; finite-step nonlinear experiments are presented in the appendix.

We have posted reviewer-specific follow-ups indicating how the revision addresses the concerns raised. We would appreciate the reviewers’ and area chair’s assessment of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.