acceptodds
Under review as a conference paper at ICLR 2027

When Is Forgetting a Capacity Problem? Dissecting Catastrophic Forgetting under Controlled Capacity Pressure

Abstract

Catastrophic forgetting is often attributed either to exhausted capacity or to optimisation interference despite sufficient capacity. The two explanations imply different remedies, and telling them apart calls for a measured capacity axis, which natural data do not provide: neither the stream's information content nor the model's effective capacity is directly known. To place forgetting on this axis, we build a continual-learning testbed from random facts, rules and noise, with known information content and calibrated model capacity. We define load as learnable information relative to measured capacity and vary it while holding each fact's training exposure fixed, which separates capacity limits from under-training. Three findings emerge in the testbed. First, storage of the old facts fails once they alone exceed capacity. Up to a load of 1 they are still stored almost in full, but as load grows a new linear readout needs more examples to decode them, and the model's own readout decodes them with narrower margins. Second, far below capacity, new facts in the old facts' format collapse old-fact recall within tens of steps, while less than 3% of their information is stored; training without new facts erodes recall much more slowly. During warm-up, collapse comes at a fixed cumulative learning rate, once the parameters have moved by about 1% of their norm; random perturbations of that size leave recall intact. Third, information survives the loss of recall and fades on separate clocks: the model's ranking of the correct answer outlasts exact recall, and brief retraining on a few old facts still restores others (a recoverable trace) once that ranking is at chance. During the collapse, unseen facts also become transiently hard to learn, while old ones stay easy to relearn. Among the consolidation methods we compare at three loads, utility-weighted replay retains the most useful facts and gives the highest future-task utility at each, and, at matched weighting, replay outperforms sleep-phase distillation with less compute. Weighting redirects retention toward useful facts even when all facts fit, but no method stores more than the measured plateau. A pretrained language model (Pythia-160M) fine-tuned on natural-language facts reproduces the collapse, its onset at a fixed cumulative learning rate and the recoverable trace; there, rendering the new facts as statements rather than in the old facts' question-answer format delays the collapse.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, catastrophic forgetting, capacity, interference, synthetic testbed, knowledge storage, linear probing, replay, consolidation, language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_b1XPOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how capacity affects forgetting during sequential training. The authors use synthetic facts with known information content and estimate capacity under a training protocol. The paper suggests that recall can fail while some information remains recoverable, and that replay focused on useful facts improves retention, reducing forgetting. Capacity experiments use 5M and 11M models, while Pythia-160M experiments test interference and recovery.

Critical strengths

** Controlled experiments**: The synthetic facts experiments make sense in studying this topic, especially as they can vary the amount of information relative to the capacity. This helps test when 'capacity' shapes forgetting in (language) models.

Critical weaknesses
  1. Limited advance over existing work: Allen-Zhu and Li (2025) already measure a model's capacity using synthetic random facts and prediction loss. Marek et al. (2026) also studied how capacity affects forgetting. This paper combines these established directions and adds tests of answer ranking, recovery, and relearning. However, the measurement framework builds on Allen-Zhu and Li. I am not convinced that the additional findings establish a substantial advance over these prior studies. This makes it difficult to find the significance of the contribution.

  2. Insufficient evidence across model sizes: The capacity study varies information load but uses only two small models, with 5M and 11M parameters. A concurrent work (Marek et al., ) studied a similar problem with more (5+) model sizes. The pythia-160M experiments test interference and recovery without repeating the capacity study. The authors also acknowledge that the two small models have similar storage per parameter. Thus, the results cannot clearly separate the effect of measured load from the number of facts per parameter. A broader study of model sizes and training choices is needed to establish the claimed capacity relationships. The current evidence is insufficient to show how these relationships change as models become larger.

Questions for authors
  1. Which specific observation provides an advance over the previous works? The idea of studying forgetting from a model capacity perspective has already been studied, with similar synthetic data and thoroughly on real datasets.
  2. Could the paper's results separate the effect of capacity from different training choices (e.g., data, model, hyperparams)? The authors perform their experiments using a single set of hyperparameters (App. C.). Furthermore, if capacity was indeed the main focus of the paper, it would be sound to vary the amount of pretraining data. I'm also curious if the authors have considered studying the effect of pretraining on forgetting. Learning rates are only tested on values of 1e-4 and 1e-3, which is too narrow of a scope and rather unfit for the training of an LLM.
Reviewer_3T2vOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies catastrophic forgetting using random facts with known information content, compositional rules, and noise. It defines load relative to a measured storage plateau and controls training exposure per fact. Experiments distinguish exact recall, answer ranking, recovery on unrehearsed facts, and relearning. They show rapid forgetting even when joint training can learn both phases. Recovery remains possible after recall collapses. Increasing load also reduces the accuracy of probes trained on fixed sample sizes. The paper compares consolidation methods and finds that utility-weighted replay performs best under its evaluation. Pythia-160M experiments reproduce rapid forgetting and recovery using synthetic facts expressed in natural language.

Critical strengths

The experiments convincingly demonstrate interference when both fact sets can fit. At load 0.25, joint training learns both sets fully, while sequential training rapidly destroys old-fact recall. Random perturbations with matched displacement preserve recall. Removing new facts substantially slows forgetting. These controls support a meaningful distinction between insufficient storage and harmful updates. The joint-learning control in Pythia strengthens this result. Evidence: Section 4; Appendices F.1, F.2, and K.

The recovery experiments test knowledge beyond the rehearsed examples. Recovery uses 1% of old facts and evaluates other old facts. Controls train on unseen facts or noise; Pythia also includes shuffled answers. In Pythia, recovery reaches 33–37%, compared with 2–3% for controls. This supports surviving information after recall failure. The separate analysis of unseen-fact learning also prevents relearning advantage from being mistaken for a pure retention measure. Evidence: Sections 4 and 6; Appendices F.3 and K.

Critical weaknesses
  1. The “stored bits” measure does not establish the claimed amount of stored information. The paper clips each fact’s score before summing:

    I^=∑imax⁡(0,16−NLLi/ln⁡2).\widehat I=\sum_i\max\left(0,16-\mathrm{NLL}_i/\ln 2\right).

    This can yield positive values without memorizing any fact. Consider a predictor independent of the keys and training data. Its first-token probabilities are 1.5/2561.5/256 for half the values and 0.5/2560.5/256 for the rest. Its second-token distribution is uniform. Under uniformly sampled answers, the expected score is:

    12log⁡2(1.5)≈0.292 bits per fact.\tfrac12\log_2(1.5)\approx0.292\text{ bits per fact}.

    This exceeds the reported 0.24–0.27 bits per new fact at collapse. The example establishes a measurement concern, not the actual bias in these experiments. A shuffled-value baseline and an unclipped aggregate would clarify the effect on the reported results.

    Moreover, poor decoder likelihood cannot upper-bound information retained in the weights. The paper’s own recovery results illustrate this distinction. Therefore, “less than 3% stored” needs a narrower interpretation.

    The capacity plateau also remains specific to the training protocol. Table 4 reports 7.1 Mbit under longer training, compared with the adopted 6.5 Mbit. Appendix F.1 reports only 62% joint recall at load 1. These results limit how cleanly the axis separates capacity from optimization. The authors acknowledge this limitation, but several main-text claims remain stronger. This matters because the capacity interpretation is central to the paper’s contribution.

    References: Section 2; Table 4; Appendices F.1 and F.3.

  2. The replay–distillation comparison changes several factors together. Replay supplies old data throughout training, whereas sleep distillation supplies it afterward. For TT wake steps, replay allocates 0.20T0.20T batch-equivalents to old data. Sleep allocates approximately 0.25T×0.5=0.125T0.25T\times0.5=0.125T. Thus, sleep receives fewer old-data exposures despite its greater compute cost.

    The methods also begin phase B with different recall. At load 1, uniform replay starts at 0.95, while uniform sleep starts at 0.64. These results establish a ranking of the tested procedures. They provide weaker evidence about the relative merits of replay and distillation objectives. A comparison with a common starting checkpoint, matched old-data exposure, and matched intervention timing would clarify this conclusion.

    References: Section 5; Appendix C; Table 8.

Questions for authors
  1. How do shuffled-value baselines and unclipped aggregate scores change the capacity estimates and new-fact storage measurements at collapse?
  2. What convergence evidence separates the reported capacity plateau from remaining optimization limits, particularly near loads 1–2.5?
  3. Does replay retain its advantage with a common starting checkpoint, matched old-data exposure, and matched intervention timing?
Reviewer_satZOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates whether catastrophic forgetting reflects limited storage capacity or optimization interference under sufficient capacity. For controlled study, authors uses synthetic facts, rules and noise and estimated model capacity to study the forgetting phenomenon in model training. The experiment shows that recall can collapse far below capacity when new information is learning. Further, some information is recoverable, but it's harder to retrieve than the time before new learning process starts. This paper also compares consolidation methods and reproduces key interference findings in Pythia-160M.

Critical strengths
  1. Authors use well-controlled experiment design. The information content, exposure of each fact and utility and frequency are known and assigned independently. This helps separate the confounding variables between capacity pressure, under-training and useful memory.

  2. The complementary diagnosis is convincing. Exact recall, answer ranking, probing, and controlled recovery experiments provide strong evidence that failure in recall does not directly imply complete forgetting, or information loss. Further, the shared training, matching random perturbations, variable learning rate and pre-trained model replica provides strong evidence that there is interference below capacity.

Critical weaknesses
  1. One weakness is limited generalizability. Experiments in the paper mainly involve small models, synthetic facts, and only two learning phases. Pythia model's result indeed validates interference but not the capacity transition. And also, the capacity is still estimated, which needs stronger support. Probably varying the amount of facts or information can test the relative content usage with respect to the theoretical model capacity.

  2. I also concern about the novelty of this research. First, the catastrophic forgetting in machine learning under sufficient capacity is a well-known phenomenon. Also, replay is also an established way to resolve this problem. Further, the observation that failed to recall doesn't imply completely forgetting also has precedent (https://proceedings.iclr.cc/paper_files/paper/2024/file/692ae28fda9bfbde7c01b13bf5a03395-Paper-Conference.pdf), which shows some learned instructions or rules can be recovered. Thus, this paper mainly adds some controlled study on catastrophic forgetting.

Questions for authors
  1. How would you frame your core contribution and novelty of this paper?

2.How stable are the capacity estimate and transition if we train the model for substantially longer, under different optimizers, and different LR schedules?

  1. Based on the experiments, what do you think causes rapid forgetting?
Reviewer_dn7DOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to distinguish capacity limitations from optimization interference as causes of catastrophic forgetting. The authors construct a controlled stream of random facts, rules, and noise, and vary information load relative to calibrated model capacity while controlling training exposure. The experiments show that updates on new facts in the same format can collapse recall despite sufficient capacity, while recoverable memory traces survive this collapse. Replay weighted by utility best preserves useful knowledge among the evaluated interventions, without exceeding the measured storage plateau. Partial replication in a pretrained language model supports the broader value of the framework for diagnosing different forms of forgetting.

Critical strengths
  • Known factual information content, calibrated capacity, and controlled training exposure make comparisons across information loads more interpretable. This design reduces the risk of mistaking insufficient training for a storage limitation.

  • Recall, probing, ranking, and recovery measurements reveal information that survives recall failure. Evaluating recovery on old facts excluded from retraining provides particularly useful evidence of residual memory.

  • The pretrained-model experiments reproduce key interference and recovery phenomena, increasing confidence that these effects extend beyond the original testbed. Comparing formats at matched new-fact storage strengthens the evidence that presentation affects interference.

Critical weaknesses
  • Most experiments use synthetic data, so even with the partial validation in a pretrained model, it is still unclear how well the findings carry over to real language data and other models.
  • Could the authors use these insights to develop continual learning methods that improve overall performance, maintain plasticity, and reduce forgetting on real tasks? This could offer an indirect test of the framework's practical value.
Questions for authors

Please refer to the Weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.