acceptodds
Under review as a conference paper at ICLR 2027

A Retention Curve Certifies the Rehearsed Sentence, Not the Fact

Abstract

Language models are kept current by continual maintenance rather than retraining: new facts are written into the weights, the training sentences are replayed every round, and a stabiliser such as weight interpolation toward the pre-trained model limits drift. Retention is then measured on the replayed sentences themselves, on the premise that the item rehearsed every round is the item held best. We show that this measurement certifies a verbatim completion and nothing beyond it. On Qwen3-8B with 150 facts rehearsed for thirty rounds under a per-round weight interpolation, every training sentence is completed word for word every round, while the same fact in a held-out paraphrase is answered on .120 of the facts against .473 with weight interpolation off. The mechanism is the interaction of the two ingredients: weight interpolation alone lowers both measurements, rehearsal alone lowers neither, and together they rebuild exactly the training sentence. Even the training sentence, asked through the chat template, is answered on .087 of the facts against .560. The gap recurs on Mistral-7B, OLMo-2-7B and Qwen3-14B, on Wikidata facts and on 600 CounterFact and zsRE edits, under a periodic merge, and under later updates with no stabiliser. The standard rule for choosing a stabiliser strength, applied to the retention curve, picks a strength at which most of the paraphrase is already lost. At one distance from the pre-trained weights the retention curve scores every stabiliser family alike while the paraphrase separates them: on a float32 master an L-SP penalty answers the paraphrase on .393 of the facts where weight interpolation answers it on .140. And a per-round weight interpolation applied in place to bfloat16 weights delivers .02 of its nominal step by the thirtieth round, a shortfall the curve cannot show. We propose a release check that measures every maintained fact in a held-out paraphrase against the pre-trained baseline, holds some facts out of rehearsal, asks the training sentence through the chat template, and reports the stabiliser strength delivered rather than the one set.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
continual learning, knowledge editing, retention curve, paraphrase, rehearsal, LoRA, WiSE-FT, bfloat16

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_j2SrOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper explores the problem of integrating new information into a pre-trained model. It explores whether the new information is really learned by the model, or whether the model just memorized the exact sentence that expresses the new information. It brings evidence that the latter is true.

Critical strengths

None

Critical weaknesses
  • The paper is very unclear and hard to follow, to the point that it is difficult for me to properly review its contribution. The problem setting is unclear, the experimental setup is hard to understand. I think a substantial editing work has to be done, readability should be much improved, and that is an understatement.
Questions for authors
Reviewer_og2EOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the area of continuously improving an LLM’s temporal knowledge; they study the setting in which an LLM is periodically fine-tuned on new facts (sentences) with and without weight interpolation. They do an analysis by assessing the model performance both on the training sentences and their paraphrases to show that while the model can (and often does) verbatim memorize training sentences, that knowledge doesn’t aid in performance on paraphrased sentences and in the interpolation setting can actually hurt model performance. This work primarily proposes this evaluation methodology of using synthetic facts and their paraphrases to measure new knowledge abilities in LLMs and assess the acquisition of new knowledge under varying settings (varying interpolation weights, models, training rounds). They largely find a negative result: LLMs are more prone of memorizing new synthetic facts rather then learning them in a way that makes them amenable for use in related settings (e.g., paraphrased sentence completions).

Critical strengths

Maintaining model knowledge is an important area of study as foundation models (such as LLMs) are being used for their temporal knowledge; continuously updating a model (e.g., LLM) with up-to-date information is important. This paper tried to tackle this challenge by proposing a methodology to evaluate if new knowledge is actually learned by the model, and disambiguate if instead the model simply memorizes new knowledge verbatim.

Critical weaknesses
  1. Why would we expect the answer to a paraphrased version of a “fake fact” the model was trained on to be learned during fine-tuning? The authors themselves write that prior work has not considered “paraphrase of a fact rehearsed every round under a stabiliser” (line 100); is there a reason for this? Since these synthetic facts are not true and not related to the model’s pretraining data, it isn’t clear to me that a model should be able to do an accurate completion to a paraphrase of these facts. And therefore, I am a little worried the using the discrepancy between the training sentences and their paraphrased counter-parts might not be a meaningful measure of a model's ability to learn new information.

  2. The main argument for a model interpolation is to retain, "pre-rehearsal" model performance. However, the accuracy of “pre-rehearsal” knowledge is not tracked via performance of some sort of pre-training relevant data. Therefore, its not clear if the interpolation is even necessary. This is important since it looks like interpolation is hurting both training sentences and the paraphrased facts (figure 2); perhaps the simplest solution is just not to interpolate.

Questions for authors
  1. Figure 1 B, C do not have y-axis columns so I am guessing as to what the metrics are supposed to be. The X-axis on Figure 1C is also confusing.

  2. In figure 2B it seems interpolation also hurts the non-paraphrased fact retention…so I am not surprised that it also hurts the paraphrased fact retention. Figure 1B is a little misleading because it separates the interpolated vs. non-interpolated variants for the paraphrased facts, but not for the non-paraphrased facts; I am now curious if the same trend repeats? And if so, maybe the simpler solution is just to not interpolate?

  3. If I am reading figure 1,B correctly, then the first data point (x = 0) is after the first round of training has occurred on the fake facts. There is a large gap between the actual fake fact accuracy (which is already near ~100%) and the paraphrases (~50%). Over the course of non-interpolated rehearsal, the paraphrased facts never really improve in accuracy. This raises two concerns: first, (see first weakness), its not clear to me that measuring rehearsal performance on paraphrases of fake facts is a good metric since the performance of the paraphrased facts don’t seem related to the training sentences. Second, what was the model’s performance on the fake facts and their paraphrases before any rehearsal? Was there any increase in the performance of paraphrased facts before and after the first rehearsal?

Reviewer_MgkYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors address the problem of incremental knowledge edition of pre-trained LLMs. The main strategy is to expose the model to the new facts while repeating the original sentences to avoid catastrophic forgetting, and to pull the model closer to its original weights, to avoid drift. The paper claims that the standard strategies hide the loss by showing that the original sentences can be retrieved verbatim. The LLM asked using different wordings show a clearer effect. The paper suggests that the stabilizer keeps erasing the knowledge while the rehaersal puts it back verbatim (i.e. not necessarily retaining the underlying fact)

Critical strengths
  • A practical message, that informs practitioners on what to look after when dealing with this problem
  • Several ablations help isolate the problem and findings
  • The paper checks its claims in different scenarios, models and regimes
  • Concrete and actional suggestions for practitiones aiming to overcome the problem
  • The bfloat16 finding, although a side discovery, is a useful
  • Strong reproducibility
Critical weaknesses
  • Writing seems too condensed and hard to follow.
  • float32 and bfloat16 runs are displayed mixed across tables.
  • The main claim of the paper, that "the retention curve hides the knowledge loss" holds for the main experiment, where they train with single-wording facts. This regime is associated by Allen-Zhu & Li with poor knowledge extraction. The authors address this, in Table 2, via a four-paraphrase write that is then tested against a fifth held-out wording. My impression is that in this other scenario, which is closer to what a practitioner would choose, the retention curve does show a noticeable drop (i.e. doesnt hide the knowledge loss), which weakens the main claim. Moreover, this four-paraphrase experiment is ran at bfloat16, which is not directly comparable.
  • The interpolation regime is very aggressive: They pull the model 4.4% closer to its original 31 times, which compounds the interpolation and leads to a ~75% pull by round 30. This number matches the reported 27.5% kept. Note: WiSE-FT is a one-time interpolation.
  • Anchoring to the weights of the previous round removes the loss (0.493 vs 0.447). This fact weakens the relevance of the loss.
  • Rehearsal is performed with low strength (LoRa of rank 8, 90 steps, and 10x smaller LR than round 0). To me it is plausible that this regime is too weak to pull the whole knowledge behind a fact, so it might only be able to pull the exact wording being rehearsed and nothing else. This might be a side effect of rehearsing too weakly, not a general law. The authors test this by strengthening the rehearsal, but use different settings (e.g. bfloat16) making it hard to draw definitive conclusions.
  • Knowledge-editing benchmarks already score paraphrase generalisation, weakening the novelty of this contribution.
Questions for authors

Could you please report the bfloat16 and float32 experiments in both the 1-wording setting and the multiple-wording setting? This could help readers drawing definitive conclusions

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.