acceptodds
Under review as a conference paper at ICLR 2027

Towards Eliminating Catastrophic Forgetting in the Curriculum Learning of Math Reasoning Tasks

Abstract

Curriculum learning has found broad application across numerous domains. Nevertheless, its effectiveness is intrinsically curtailed by catastrophic forgetting, driven by the shifts in model parameter distributions between curriculum tasks. In this paper, we investigate the phenomenon of catastrophic forgetting in this training paradigm, building on the established efficacy of curriculum learning. Our theoretical analyses of parameter update dynamics demonstrate that catastrophic forgetting in curriculum learning stems from the divergence of task optima, which is generally essential to the faster convergence of curriculum learning; therefore, forgetting cannot be completely eliminated. Based on this finding, we augment the training process and propose IV-EWC, which incorporates Elastic Weight Consolidation (EWC) into the curriculum learning objective to curb catastrophic forgetting in mathematical reasoning, a prototypical curriculum learning scenario. IV-EWC employs the influence function to construct a representative validation set from the curriculum's training data, which is used to drive dynamic regularization during training. We further present an extended theoretical analysis to show that EWC-based regularization methods mitigate catastrophic forgetting in curriculum learning, thereby providing theoretical support for IV-EWC. Empirical evaluations on three backbone models and three benchmarks indicate that curriculum learning exhibits catastrophic forgetting. IV-EWC alleviates this issue, reducing forgetting by 162% on average relative to vanilla curriculum learning and yielding positive backward transfer, as evidenced by improved performance on easier tasks after subsequent training on challenging tasks.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Curriculum Learning, Catastrophic Forgetting

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_B4oROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes IV-EWC a method to improve curriculum learning. The aim of IV-EWC is to prevent forgetting when learning on subsequent tasks in the curriculum. IV-EWC works by applying EWC with a specific regularization coefficient schedule and a particular way of selecting points to construct the estimate of the fisher information. The authors evaluate performance on 3 datasets and compare to 5 baselines. IV-EWC achieves a marginal accuracy performance improvement.

Critical strengths
  • Looking at problems like forgetting due to the distribution shifting between different parts of the curriculum is an interesting perspective. Though has been explored before.

  • I think the case study is a nice analysis.

Critical weaknesses
  • No mention of previous work on forgetting in curriculum learning is given. There have been many works on this, including theoretical work on convergence speedup , for example Saglietti, Mannelli & Saxe, An Analytical Theory of Curriculum Learning in Teacher-Student Networks NeurIPS 2022

  • Paper claimed main novelty of IV-EWC is using influence functions in the estimation of the fisher information in EWC. But, nowhere in the main paper is this presented.

  • The theory section has many problems:

    A) The main claim is that curriculum learning improves convergence speed. This is not tested in the experiments. In fact, as the experiments use the same learning rate per curriculum step it renders the theory empty, in the sense the only reason you would expect an improved convergence speed is if you could use larger learning rates earlier on. This makes the theory and method/experiments seem quite unrelated.

    B) The theory section is standard analysis and no references or indication of this is given in the paper. For example, Lemmas 1 and 2 and Theorem 3 are not novel. This means that the theory section could be shortened to a paragraph or two.

    C) Theorem 2 proof looks to have an error where eq. 4 should be for a single gradient step where in the proof this is assumed to be over learning the whole curriculum task t.

  • Performance improvement is very marginal, baselines are outdated and no comparison to doing no curriculum learning IV-EWC not tested over different curriculum methods and the curriculum method used in the paper is unclear

Questions for authors
  • Why does figure 1. c) not seem to link up with the results in table 1, am I missing something?

  • Why was the main claim of the paper, using influence functions, not described in the main text nor fully in the appendix?

Reviewer_8TUUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the phenomenon of catastrophic forgetting in training paradigm, building on the established efficacy of curriculum learning. This paper theoretically analyzes parameter update dynamics to demonstrate that catastrophic forgetting in curriculum learning stems from the divergence of task optima, which is generally essential to the faster convergence of curriculum learning; therefore, forgetting cannot be completely eliminated. The main contributions can be summarized as:

a. Identify and formalize the phenomenon of catastrophic forgetting in curriculum learning, providing theoretical analysis from the parameter update perspective in gradient descent to establish that forgetting can be attributed to the differences of optima across curriculum tasks.

b. Propose IV-EWC, which integrates influence-function-guided selection of representative training samples with a dynamic EWC regularizer.

c. Investigate EWC-based regularization in curriculum learning and prove that it attenuates catastrophic forgetting by constraining the distribution shifts of the parameters.

d. Conduct extensive experiments with three base models on three math reasoning datasets

Critical strengths

a.This paper is well written and easy to follow.

b.Interesting ideas. This paper proposes IV-EWC, which incorporates EWC into the curriculum learning objective to curb catastrophic forgetting in mathematical reasoning.

c. This paper further presents an extended theoretical analysis to show that EWC-based regularization methods mitigate catastrophic forgetting in curriculum learning.

Critical weaknesses

The presentation and analysis of the experimental results are confusing and not convincing enough. According to the metrics provided by Section 5.1, the value of Forget should be negative, and the closer it is to 0, the less forgetting occurs. However, the experimental result of UPGD in the first column (-0.0008) is actually better than that of IV-EWC (-0.0103). Moreover, positive forgetting values are better than negative forgetting values. So why does the author explain that IV-EWC achieves superior performance relative to all baselines on both forgetting and overall accuracy in nearly all tested configurations?

Questions for authors

Please address the concerns in Critical Weaknesses.

Reviewer_qMW9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies catastrophic forgetting in curriculum learning (data D\mathcal{D} split into subtasks D=D1∪⋯∪DT\mathcal{D} = \mathcal{D}_1\cup \dots \cup \mathcal{D}_T of different levels, presented sequentially in training), with a focus on mathematical reasoning tasks performed by large language models. The authors first give a theoretical analysis of a simplified sequential learning problem where each subtask is a convex optimisation problem (quadratic bowl loss landscape with unique optimal parameter for each subtask), arguing that under the condition that the 'easier' (or earlier) task is better conditioned and leaves the parameters closer to the final task optimum, the benefit of curriculum learning comes in the form of fewer steps to convergence. They further argue in this toy model that differences between the optima of successive curriculum tasks necessarily lead to forgetting.

Motivated by this, the authors propose IV-EWC, a modification of Elastic Weight Consolidation. IV-EWC makes two modifications to vanilla EWC: (1) the use of influence functions to select a progressively expanding subset of influential training samples which is then used to estimate the Fisher Information Matrix, (2) and scales the EWC regularisation coefficient as the curriculum progresses by a factor of 1subtask number\frac{1}{\textrm{subtask number}} (1t\frac{1}{t}) to allow for greater plasticity in later stage training. The method is evaluated on three LLM models and three mathematical reasoning datasets and compared with vanilla curriculum baseline, random replay, vanilla elastic weight consolidation (with same regularisation parameters), averaged gradient episodic memory, utility-based perturbed gradient descent. Empirically, IV-EWC generally produces lower measured forgetting and competitive or improved final accuracy.

Critical strengths

The empirical evaluation is reasonably broad. The authors test three different models across three mathematical reasoning datasets and compare against several continual-learning baselines. This shows that their method is robust and can consistently deliver results, which report to often be best performing or second place when compared to other continual-learning methods.

Critical weaknesses

(In order of priority)

Largest concern is that the theoretical and empirical parts of the paper feel totally disconnected with no clear motivation between the two sections. The theoretical analysis is developed for a very narrow toy problem with strongly convex and L-smooth losses, and later for isotropic quadratic losses of the form ft(θ)=Lt2∣θ−θt∗∣2f_t(\theta)=\frac{L_t}{2}|\theta-\theta_t^\ast|^2 which is already very far removed from post-training LLMs. More importantly, the novel IV-EWC algorithm that the authors introduce is not actually analysed in this theory. The influence-function sample selection (for which the process is ambiguously specified with no clear detail in the paper) does not enter the theoretical model, and neither does the proposed 1t\frac{1}{t} scaling of the EWC regularisation coefficient meaning that the final theoretical result is essentially just a result about standard EWC in a simplified quadratic problem.

I am not convinced that the current experiments identify the specific mechanism responsible for IV-EWC's improvement. The ablation study compares replay from only the previous task, cumulative replay across curriculum levels, and cumulative replay using influential samples with improvement seeming to come simply from retaining examples across levels, and then with an additional improvement from influence-based selection. However, IV-EWC uses selected examples to estimate the Fisher rather than replaying them directly.

I have further concerns about the correctness of some theoretical claims - I do not see how the assumptions of Theorem 1 are sufficient to prove that curriculum learning always requires fewer iterations. They claim that because δ\delta is small and κe<κ\kappa_e<\kappa, the sum in Eq. 36 is negative which does not follow, the positive second term can be arbitrarily large, e.g. ϵe\epsilon_e very small. Furthermore, the claim in Theorem 3 that forgetting becomes zero for λ=1\lambda=1 is inconsistent with Eq. 9 and the proceeding statement which concludes forgetting cannot be eliminated when task optima are distinct (which is assumed throughout). The correct statement I assume they meant was that λ=1\lambda=1 minimises their average task loss (not to zero).

The toy model also makes the curriculum notion rather artificial. Each task is essentially just optimisation towards a different point θt∗\theta_t^\ast in a quadratic loss landscape which makes forgetting feel to me almost trivial as if the final parameters move towards a different optimum, performance on previous quadratic objectives necessarily worsens. I therefore found it unclear how much new understanding the theoretical result provides beyond this construction. Furthermore, the connection between Theorems 1 and 2 is unclear as the former's notion an "easier" task is a smaller condition number, κe<κ\kappa_e<\kappa but in the quadratic loss model their smoothness and strong-convexity constants are both LtL_t, so every task has κt=LtLt=1\kappa_t=\frac{L_t}{L_t}=1 making every task equally easy.

Questions for authors

In Theorem 1, why do the conditions κe<κ\kappa_e<\kappa and small ∣θe∗−θ∗∣|\theta_e^\ast-\theta^\ast| guarantee Eq. 36?

In Theorem 3, how can forgetting be zero when the task optima are distinct?

What is the intended theoretical connection between Theorem 3 and IV-EWC when the influence-selected validation set and the λ=γt\lambda=\frac{\gamma}{t} schedule do not enter the analysis?

How exactly is the influence score used to select the top-kk samples defined? This feels wholely under explained whilst being one of the two central pillars in the novelty of your proposed algorithm.

Could you provide a direct ablation of the two main modifications to EWC? e.g using randomly selected cumulative samples against EWC using influence-selected samples, as well as fixed regularisation against the proposed 1t\frac{1}{t} schedule.

Fix Eq. (13). θLt′∗\theta^\ast_{L_{t'}} appears to be a typesetting error with intended θt′∗\theta^\ast_{t'}.

A minor issue is the definition of metric Forget=FP−APAP\mathrm{Forget}=\frac{FP-AP}{AP} using conflicting sign notation when compared to the results which interprets positive values as forgetting and negative values as positive backward transfer. Under the displayed definition, I would expect the opposite sign. Can you clarify this.

Can you provide an IID / randomly shuffled / interleaved curriculum baseline? This would help distinguish the benefit of curriculum ordering itself from the effects of sequential fine-tuning, and would better contextualise the proposed transfer/forgetting trade-off.

Reviewer_sQa8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies catastrophic forgetting in curriculum learning under an idealized smooth and strongly convex setting, arguing that curriculum learning accelerates convergence but induces forgetting because different curriculum tasks have non-coincident optima; it proposes IV-EWC, which uses influence functions to select a progressively expanding validation set for Fisher-based dynamic EWC regularization, aiming to protect parameters important for easier tasks while adapting to harder ones. Its contributions are formalizing forgetting in curriculum learning, proposing IV-EWC, providing theoretical analysis of EWC-based forgetting mitigation, and reporting empirical improvements on mathematical reasoning tasks. The validation combines theoretical lemmas and theorems with experiments on GSM8K, MATH, and cn_k12 using three base models, comparing IV-EWC against vanilla curriculum, Random Replay, EWC, A-GEM, and UPGD, together with ablation studies, hyperparameter sensitivity analyses, and case studies.

Critical strengths
  1. The paper gives a structural account of forgetting in curriculum learning by linking accelerated convergence to non-coincident optima across curriculum tasks. This turns forgetting from an incidental empirical phenomenon into a property of the curriculum’s task sequence and gives the proposed method a clear target.
  2. IV-EWC is a coherent method design. Influence-function-guided validation selection focuses the Fisher estimate on curriculum-relevant samples, while dynamic EWC regularization relaxes protection as tasks become harder. The design directly addresses the tension between retaining earlier-task knowledge and adapting to harder tasks, and it can be evaluated independently of the theoretical claims.
Critical weaknesses
  1. Limited novelty—it reads as a combination of existing techniques. IV-EWC is essentially influence-function selection plus EWC plus a dynamic regularization coefficient. The paper does not clearly establish a new mechanism beyond composing known components, and the ablation does not isolate the contribution of each IV-EWC component.
  2. The theory does not adequately support the method and contains inconsistencies. Theorem 3’s claim that λ=1 reduces forgetting to zero is not valid under Eq. 12 when task optima differ; its EWC anchor differs from Eq. 10; Theorem 2 uses a proximal update while Lemmas 1–2 use gradient descent; and the analysis does not cover influence-based selection or dynamic λ.
Questions for authors
  1. Theory. Regarding Theorem 3, under Eq. 12, does the residual constant C′C' vanish when λ=1\lambda=1? Also, Eq. 10 anchors EWC to θt−1\theta_{t-1}, but Theorem 3 uses θt′∗\theta_{t'}^*. Additionally, Theorem 2 uses a proximal update while Lemmas 1–2 use gradient descent. Could the authors clarify these inconsistencies and confirm if the theoretical claims hold under the experimental optimizer?
  2. Details. Could the authors provide a framework figure or pseudo-code detailing the influence-based sample selection, validation-set expansion, and Fisher estimation? Also, regarding baseline fairness, does the EWC baseline use the exact same dynamic λ\lambda, validation set, and Fisher procedure as IV-EWC?
  3. Ablation. Since IV-EWC combines influence-function selection, validation-based Fisher estimation, and dynamic EWC, could the authors provide component-level ablations that isolate the effects of these individual modules? This would clarify how much each component contributes to the overall performance beyond merely combining existing techniques.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.