acceptodds
Under review as a conference paper at ICLR 2027

Did Your LLM Really Forget? Measuring Factual Recall During LLM Post-Training

Abstract

Post-training shapes LLM behavior, e.g., by aligning models with user preferences and scaling test-time compute through long reasoning trajectories. While post-training rarely teaches models new facts about the world, it might induce forgetting of factual knowledge acquired during pretraining. Previous studies of this phenomenon either rely on aggregate measures that mask individual facts or compare only two checkpoints. We propose a different way to approach this problem, investigating how the probability of recalling individual facts is altered across the entire post-training trajectory. Using repeated rollouts across intermediate checkpoints, we show that individual facts' recall probabilities fluctuate substantially during post-training, repeatedly increasing and decreasing even when aggregate accuracy remains relatively stable. Although changes in the underlying probability of recalling each fact largely explain the magnitude of these fluctuations, whether a fact is classified as forgotten or reappearing is driven primarily by sampling variability. Thus, we show that prior methods can substantially overestimate persistent knowledge loss. Finally, we formalize a lower bound to certify the knowledge loss in bits. We show that our metric is more consistent than existing ones and can be easily extended to account for recall fluctuations.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
forgetting, llm, sampling

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Yqi9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether LLMs really lose factual knowledge during post-training, or whether much of the measured loss comes from how forgetting is measured. Recall of each fact is treated as a probability. The authors estimate it with 128 rollouts per question at each of 55 public checkpoints of Olmo-3-7B, SmolLM3-3B and K2-Horizon-7B. They use three QA sets: EntityQuestions, SimpleQA Verified and Incompressible Knowledge Probes.

Per-fact recall moves a lot between neighboring checkpoints while aggregate accuracy stays roughly flat. A stationary-binomial null is rejected for all nine model-dataset pairs. Facts that drop to 0/128 and later come back appear at rates close to that null. When only the post-trained model's budget is doubled, 30.7% of binary forgetting decisions flip, and 51.9% flip at 4x. The paper then proposes a certified lower bound on knowledge loss in bits, based on Clopper-Pearson bounds. It comes with a closed-form rollout budget and a multi-checkpoint version. Answers are scored by exact match, and an LLM re-check of 1000 random negatives finds 3.4% false negatives.

Critical strengths
  1. The main message is useful and well supported. Fig. 5 shows that a 0/k "forgotten" label depends heavily on how many samples were drawn. The experiment behind this is large, with about 100M rollouts in total, and the stationary-binomial null is a reasonable reference. The bound itself is simple and correct, and Eq. 4 tells a practitioner how many rollouts to plan for. I expect others who measure forgetting to pick up this framing.
Critical weaknesses
  1. The measured quantity has not been checked on the cases that drive the conclusions. Scoring is exact match, and the false-negative check in App. C.1 used random negatives. These are probably dominated by facts the model never recalls, but the appendix gives no breakdown. Table 7 includes canonical facts that drop sharply, such as "Salvador Dali" for Olmo-3-7B on IKP (107/128 to 0/128). The raw outputs are not shown, and that LLM re-check was not applied to these events. Prompting also differs between base and post-trained checkpoints. Base models get the three exemplars as plain text. Post-trained models get them as a chat with a system prompt, with thinking enabled (except SmolLM3-3B) and up to about 32k tokens. The certified losses may therefore mix knowledge change with changes in format and elicitation.

  2. Two of the main claims go further than the evidence. The claim in §4.2 that most forget-and-reappear events are explained by sampling compares aggregate rates with a stationary null that the paper itself rejects earlier. In Table 4 the difference is significant in 4 of 9 pairs, and in another 4 the observed rate is below the null. In Fig. 5 only the post-trained budget grows, so a flipped decision just means one correct answer within up to 1024 samples. That does not show that nothing was lost. In Fig. 6b the paper's own bound still certifies at least 1 bit of loss for 182 to 267 facts per model.

Questions for authors
  1. Could you apply your existing LLM re-check to the facts labeled forgotten, the forget-and-reappear events and the Table 7 entries? What is the false-negative rate on these cases?
  2. How many post-trained rollouts run out of tokens and fall back to "Final Answer:"? Do Fig. 5 and Fig. 6b still hold when base and post-trained models are prompted the same way, for example with thinking turned off?
  3. Could you test individual forget-and-reappear events, e.g. each zero-count checkpoint against its neighbors with FDR control, instead of comparing aggregate rates only?
6 comments

Question 1

We thank the reviewer for the suggestion. As discussed in the limitations section (Appx. F), evaluating LLM-generated rollouts is inherently noisy, and this affects every study that measures forgetting from model outputs. This project evaluated about 100M rollouts, so judging all of them with an LLM would be prohibitively expensive, and the judge would add its own errors. Nonetheless, to demonstrate that our results hold either way, we re-ran the LLM judge on answers that the exact match marked as wrong, sampling 1000 answers from each of the three groups suggested by the reviewer. The judge considered correct 13.1% of rollouts from the forgotten pool (Fig. 6b), 1.9% of forgotten-and-reappear rollouts, and 16.7% of rollouts of Tab. 7. These false negatives are concentrated in a few facts rather than spread across the data. That is, among forgotten facts, 154 of the 188 sampled facts (82%) contain no false negatives. Thus, our core findings consistently hold. In Table 7, only three rows out of 18 would be replaced (rows 1 and 2 for Olmo3 in IKP, and row 2 for SmolLM3 in IKP as well) by other facts, while the others would remain unchanged. IKP, in practice, is the dataset where most false negatives occur. Instead, Tab. F below shows how Fig. 6b would change if we account for the false negatives proposed by the LLM. The number of facts certified to have lost at least 1 bit remains substantial, and our conclusions remain invariant; i.e., no fact is certified below k = 16, with the count increasing with the budget.

Table F. Facts certified to have lost at least 1 bit, before → after removing flagged facts.

ModelFactsk=128k=1024EQ / SQA / IKP at k=1024
Olmo-3-7B331187 → 66182 → 15752 → 44 / 46 / 84 → 67
SmolLM3-3B3311102 → 87267 → 25163 → 60 / 32 → 31 / 172 → 160

Question 2

The number of rollouts that run out of tokens and fall back to “Final Answer” is negligible and mostly occurs on questions the model does not know the answer to, leading to a degenerate loop that hits the max tokens allowed. Table G: rollouts hitting the length limit and ran into the “Final Answer” fallback

ModelEQSQA-VIKP
Olmo-3-7B-Think0.024%0.025%0.010%
SmolLM3-3B<0.001%<0.001%<0.001%
Qwen3.5-35B-A3B1.169%0.894%0.061%

We thank the reviewer for suggesting prompting base and post-trained models in the same way. Unfortunately, while we can compare them under the same prompting, it would be an unfair comparison, as the two model variants are trained differently. Thus, removing the chat template from a post-trained model would be an unfair evaluation. Nonetheless, we can compare think-on and think-off performance as suggested. As we write in Appendix A.3 (paragraph “Rollout sampling”), the main paper already shows results on SmolLM3-3B with thinking turned off, as both variants show comparable performance; thus, we went for the cheaper option. We therefore recomputed rollout sampling for Qwen3.5-35B-A3B with thinking off and SmolLM3-3B with thinking on, reporting the results in Tables H and I below. We skipped comparing Olmo3 with thinking off, as the non-think variant is a different model, trained on a different dataset.

Table H shows that while overturn rates are slightly lower with think-off for Qwen3.5, they are still significantly higher than SmolLM3, and end up saturating (>99%) either way at high rollout budgets. Additionally, as can be seen in the table, SmolLM3 factual knowledge with think-off is only marginally lower than with thinking enabled, supporting our claim in the appendix.

Table H. Results overturned by additional sampling: think-on vs. think-off

ModelDataset2× k_p: off2× k_p: on4× k_p: off4× k_p: onk_b=1 → k_p=1024: offk_b=1 → k_p=1024: on
SmolLM3-3BEQ23.224.041.743.191.491.1
SmolLM3-3BSQA-V24.024.642.743.789.891.1
SmolLM3-3BIKP25.626.343.845.492.594.2
Qwen3.5-35B-A3BEQ34.645.458.669.599.799.8
Qwen3.5-35B-A3BSQA-V34.136.458.561.299.599.6
Qwen3.5-35B-A3BIKP34.440.956.466.399.399.9

Table I compares facts certified to have lost at least 1 bit between think-on and think-off. SmolLM3 shows similar results between the two variants, while Qwen3.5 surprisingly shows more certified forgotten facts with thinking enabled. Running a simple aggregate analysis reveals that while the post-trained version of Qwen3.5 with thinking enabled is more likely to sample correct answers than its base variant in EntityQuestions and SimpleQA-Verified on average, in IKP it has a 5% lower probability of answering correctly than the base model and an 8% lower chance than the think-off variant. This suggests that models should be tested in depth and in different configurations. However, this does not invalidate the claims made in our paper and our proposed metric, as we tested the configurations that should expose more knowledge from the model according to the literature [1]. Rather, it opens a new research direction that would extend our work and [1].

Table I. Facts certified to have lost at least 1 bit: think-on vs. think-off

ModelThinkingk=8k=16k=32k=64k=128k=256k=512k=1024EQ / SQA-V / IKP at k=1024
SmolLM3-3Boff018386710213920426763 / 32 / 172
SmolLM3-3Bon01739659813218925061 / 27 / 162
Qwen3.5-35B-A3Boff01025518412517121449 / 69 / 96
Qwen3.5-35B-A3Bon049821251672092282515 / 49 / 197

We added these results and discussions in Appendix C.9.

[1] Gekhman, Zorik, et al. "Thinking to recall: How reasoning unlocks parametric knowledge in llms." COLM, 2026.

Question 3

We want to clarify that Figure 5 further stresses that zero-correct rollouts do not imply that factual knowledge is forgotten, i.e., it does not claim that there is no forgetting. On the other hand, Figure 6b shows our metric to be effective, i.e., separating real knowledge loss from sampling noise. Therefore, it does not contradict the claim.

Regarding Table 4, although the difference reaches statistical significance in 4 out of 9 pairs, its magnitude remains very small (under 3% across most pairs, with a single case peaking at 4.9%). Thus, to further prove that f&r events are mostly caused by sampling variance, we follow the reviewer’s suggestion and tested each forget-and-reappear (f&r) event individually. Following the manuscript, we define an event as LL consecutive checkpoints with 0 correct rollouts out of k=128k=128, preceded and followed by checkpoints with at least one correct rollout. For each event, we compare the preceding and the following checkpoint with the zero-count run (pooling its 128L128L rollouts) via two one-sided Fisher exact tests. We mark an event significant only if both tests are significant (i.e., p=max⁡(pbefore,pafter)p=\max(p_{\text{before}},p_{\text{after}})). We control the false discovery rate with Benjamini–Hochberg at q=0.05q=0.05 across all events of each model–dataset pair.

Table J. Significance of individual forget-and-reappear events under BH correction.

ModelDatasetEventsSig. f&r
Olmo-3-7BEQ8650
SQA11090
IKP20611
SmolLM3-3BEQ3528
SQA3531
IKP66229
K2-Horizon-7BEQ2521
SQA3930
IKP5934

Only 44 of the 6,640 events (0.66%) are significant, and 6 of the 9 model–dataset pairs have at most one. Most events cannot be significant regardless of the correction. Since the run has zero correct rollouts, the p-value depends only on the neighbouring counts, and a single zero-count checkpoint requires at least 5/1285/128 correct rollouts on both sides to reach p<0.05p<0.05. 96.5% of events fall below this threshold, i.e., a fact recalled 1–4 times out of 128 is statistically indistinguishable from one recalled 0 times. Moreover, we note that this test is anti-conservative, as events are selected because a zero was observed. Results are consistent when excluding the greedy rollout (45 of 6,645 events) and when applying Benjamini–Hochberg jointly across all nine model–dataset pairs instead of within each pair (37 of 6,640 events).

These results support our claim: 0 correct rollouts out of kk indicate a low sampling probability rather than forgetting, and almost all apparent f&r events are explained by sampling variance. We added this analysis in Section 4.2.

Reviewer_j9ifOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates factual knowledge recall during LLM post-training, focusing on the limitations of existing forgetting metrics based on aggregate accuracy or binary correct-to-incorrect transitions. The paper further analyzes the role of sampling stochasticity using a stationary-binomial null model. Finally, the paper introduces a certified knowledge-loss metric based on the logarithmic ratio of correct-answer probabilities between base and post-trained models.

Critical strengths
  1. The paper addresses an important and timely problem: whether apparent factual forgetting during LLM post-training reflects genuine changes in recallability or artifacts of limited evaluation.
  2. The paper demonstrates that binary forgetting-and-reappearance events can occur frequently even under a stationary recall-probability model. This observation provides a useful caution against interpreting unsuccessful recall from a limited number of generations as definitive evidence of knowledge loss.
Critical weaknesses
  1. The proposed certified knowledge-loss metric relies on Clopper–Pearson confidence bounds, which assume i.i.d. Bernoulli observations. However, Appendix A.3 (Lines 773–775) states that each set of 128 rollouts includes one greedy-decoded response, while the remaining responses are generated through stochastic decoding. Furthermore, Proposition 1 establishes a pointwise confidence guarantee for an individual fact, whereas the experiments report large numbers of facts with certified knowledge loss.

  2. The stationary-binomial experiments demonstrate that binary forgetting-and-reappearance events can frequently arise from finite sampling, even when the underlying recall probabilities remain unchanged. However, the similarity between observed and simulated aggregate event rates does not establish that most observed forgetting events are actually caused by sampling stochasticity. This distinction is particularly important because the paper independently finds significant changes in recall probabilities across checkpoints. In addition, the proposed knowledge-loss metric measures the logarithmic change in correct-answer generation probabilities under a specific evaluation protocol, rather than directly quantifying the loss of internally stored factual knowledge.

Questions for authors
  1. The Clopper–Pearson bounds used in Proposition 1 assume i.i.d. Bernoulli observations, while the evaluation includes one greedy-decoded response among the 128 rollouts. Could the authors clarify the impact of this discrepancy and recompute the main certification results using only i.i.d. stochastic rollouts?
  2. The stationary-binomial experiments show that many forgetting-and-reappearance events can occur without changes in the underlying recall probabilities. However, how can the authors determine what proportion of the observed events is actually attributable to sampling stochasticity rather than genuine changes induced by post-training?
5 comments

Weakness 1 (part 1) and Question 1

While it is true that we include one greedy-decoded response to ensure the most probable answer is always sampled from the model, this in practice is not a problem. As we report in the footnote of Appendix A.3, we sample hundreds, if not thousands, of rollouts; thus, the impact of the greedy rollout is negligible. To demonstrate this, we recomputed Fig. 6b results when removing the greedy rollouts. As Tab. D shows, the difference is negligible:

Table D. Number of facts for which our metric reports ≥ 1 bit lost (original / greedy-rollout-free):

kOlmo-3-7BSmolLM3-3BQwen3.5-35B-A3B
1–80 / 00 / 00 / 0
1616 / 1518 / 1949 / 48
3238 / 3738 / 3582 / 84
12887 / 84102 / 102167 / 167
1024182 / 182267 / 266251 / 250

We updated the footnote in Appendix A.3, further clarifying that the greedy rollout does not change the results.


Weakness 1 (part 2)

Proposition 1 is indeed a per-fact guarantee. A count of NN facts certified to have lost bb bits could contain false certifications, and the naive bound on their expected number is α⋅N\alpha \cdot N. However, this bound is very loose in practice, because a false certification of bb bits requires the true factual knowledge loss to lie just below the threshold bb. In other words, for a fact with true loss of 0 (ΔK=0\Delta K = 0), for ΔK~\Delta \tilde{K} to certify at least 1 bit (ΔK~≥1\Delta \tilde{K} \geq 1) implies overshooting by a full bit or more, which is highly unlikely. We demonstrate this with two simulations. (i) Exact computation shows that the probability of certifying ≥1\geq 1 bit when ΔK<1\Delta K < 1 never exceeds 3.5×10−33.5 \times 10^{-3}, even for ΔK=0.99\Delta K = 0.99. That is more than an order of magnitude below the nominal α=0.05\alpha = 0.05. For facts with no true loss, it is below 2×10−62 \times 10^{-6}. (ii) We then ran a parametric simulation calibrated to the observed counts of every fact (3,311 per model), drawing true probabilities from the Jeffreys posterior. Of the facts with true loss below 1 bit, only about 40 per model (out of 2,200--2,700) lie within 0.1 bits of the threshold, where false certifications are plausible. As a result, at k=1024k = 1024, the expected absolute number of false certifications is 0.046 / 0.058 / 0.054 for Olmo-3-7B / SmolLM3-3B / Qwen3.5-35B-A3B, against the 182 / 267 / 251 reported certified facts. This corresponds to an average per-fact rate of about 1.5×10−51.5 \times 10^{-5}, more than three orders of magnitude below α\alpha. Instead, the probability that any reported fact is a false certification is 4--6%, close to the nominal α\alpha. Finally, results are unchanged under alternative priors (Beta(0.1,0.1)\mathrm{Beta}(0.1, 0.1) and Beta(1,1)\mathrm{Beta}(1, 1)). The per-fact guarantee therefore carries over, in practice, to the reported counts as a family-wise guarantee at about level α\alpha. We included this analysis in Appendix C.7.

Weakness 2 (part 1) and Question 2

We thank the reviewer for this question. To disentangle which portion of f&r events are due to sampling, and which to real sampling probability changes, at each checkpoint, we can split the sampled rollouts into two disjoint halves, A and B, generated with different seeds. We detect f&r events on A and measure them on B. If an event is caused by sampling, the success rate of B, where the fact "disappeared", should match the probability of a correct answer at the neighbouring checkpoints. In other words, for a real change we should observe a drop in correct answer sampling probability as we do in A.

Table E. Share of f&r events attributable to real changes in sampling probabilities.

ModelEQSQAIKP
Olmo-3-7B21%13%19%
SmolLM3-3B60%12%20%
K2-Horizon-7B58%34%57%

Table E shows that the share of real changes varies a lot across models. For instance, in Olmo-3-7B, real changes account for only 13–21% of f&r events, so sampling explains most of them, while in K2-Horizon-7B real changes are more prevalent, accounting for 34–58% of f&r events. Note that these are aggregate statistics; thus, they tell us the proportion of f&r events that are real changes in recall probability, but they cannot pinpoint which fact changed nor determine whether a change amounts to forgetting.

Finally, we clarify that our metric addresses both sampling variance and transient changes (fluctuations). First, ΔK~\Delta\tilde{K} compares bounds and not estimated probabilities, so a fact never recalled in kk rollouts certifies a loss only when kk is large enough to exclude sampling (Proposition 1). Second, certifying the loss over a window of mm consecutive checkpoints filters out transient changes (Table 5). In SmolLM3-3B on IKP, contradicted certifications drop from 24.0% (m=1m=1) to 6.0% (m=2m=2) and 0% (m=4m=4), and at m=4m=4 they are <0.4%<0.4\% for every model–dataset pair.

In summary, the vast majority of individual events are statistically indistinguishable from sampling noise, and our certified metric with a checkpoint window is robust to both. We included these results in Appendix C.8.


Weakness 2 (part 2)

We thank the reviewer for this observation. We acknowledge that LLMs can store more information than they expose through generation [1]. We measure extractable knowledge, i.e., the factual knowledge observable from the model’s output distribution, which is what a user experiences when interacting with the LLM. This is the definition adopted by most works in the field [2, 3, 4], and it is the one that matters in practice: knowledge the model stores but can no longer express is, from the user’s perspective, lost. Under this definition, the change in the probability of generating the correct answer is a direct measure of knowledge loss, not a proxy for it. The metric also does not depend on how this probability is obtained. If internal knowledge were available as a probabilistic measure, our metric would apply to it unchanged. To avoid any ambiguity, we have updated the manuscript in Section 2 accordingly.

[1] Gekhman, Zorik, et al. “Inside-out: Hidden factual knowledge in LLMs.” COLM, 2025.
[2] Petroni, Fabio, et al. "Language models as knowledge bases?" EMNLP-IJCNLP, 2019.
[3] Dong, Qingxiu, et al. "Statistical knowledge assessment for large language models." NeurIPS, 2023.
[4] Zhao, Wayne Xin, Naoki Yoshinaga, and Daisuke Oba. "What matters in memorizing and recalling facts? multifaceted benchmarks for knowledge probing in language models." EMNLP-F, 2024.

We thank the reviewer again for reviewing our work. As the discussion period continues, we would like to ask whether our responses have addressed the points raised in the review. Should the reviewer have any remaining questions, we would welcome the chance to clarify them.

Reviewer_4nk9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper highlights that existing methods for measuring knowledge forgetting may overestimate knowledge loss. The authors report that increasing the number of samples allows a substantial proportion of facts initially deemed forgotten to be recalled again.
  • The paper argues that sampling variability must be taken into account when measuring knowledge loss during post-training. To this end, the authors introduce a lower bound that certifies knowledge loss in bits. They argue that this metric provides a more consistent measure of forgetting and quantifies changes in the probability of recalling a fact.
Critical strengths
  • The paper identifies a potential overestimation of knowledge loss by existing methods for measuring knowledge forgetting. To investigate this issue, the authors evaluate recall for 100 randomly selected facts across 55 intermediate checkpoints.
  • Through this analysis, the authors demonstrate that overall accuracy is not a reliable measure of knowledge retention and that fact-level recall probabilities do not change monotonically. They also report that increasing the number of samples allows a substantial proportion of facts initially deemed forgotten to be recalled again.
  • The paper argues that sampling variability must be accounted for when measuring knowledge loss during post-training and proposes a lower bound to certify knowledge loss based on this principle. By comparing the proposed metric with a binary forgetting measure, the authors demonstrate that their metric provides a more consistent assessment and can quantify changes in fact-recall probabilities.
Critical weaknesses
  • Recent frontier models undergo complex post-training pipelines involving multiple stages. These pipelines differ substantially from the training stages and procedures of the relatively small open-weight models examined in this paper. The authors should clarify whether, and under what conditions, the proposed metric can be applied to frontier models.
Questions for authors
  • In Equation (3), why is a logarithmic function necessary? What are the theoretical or practical advantages of using the log transformation in the proposed metric?
  • Changes in the probability of generating a correct answer may arise not only from knowledge forgetting, but also from changes in the probability of combining multiple pieces of knowledge or performing the reasoning required to produce the correct answer. Would it therefore be more appropriate to characterize changes in answer-generation probability as one of several indicators of potential knowledge loss, rather than as a direct measure of knowledge loss itself?
3 comments

Weakness 1

We want to clarify that our metric applies to any LLM, as it is derived from first principles and makes no assumptions about the model being evaluated. Instead, whether our observations on factual knowledge fluctuations transfer to frontier models needs empirical evidence. To show that frontier models are also affected by fluctuations, we evaluated K2-Horizon-375B-A23B [1], a very large recent MoE model that performs similarly to closed-weight models from OpenAI and Anthropic and which undergoes SFT, RLVR, and model merging, allowing us to get as close as possible to frontier complex pipelines. Given its size and the limited time for the rebuttal, we evaluated 4 checkpoints, i.e., mid-training, RL-merged, SFT-1, and the final model, and compared them against K2-Horizon-7B on the same checkpoints. Table A shows that K2-Horizon-375B-A23B fluctuates about 2.5$\times$ more than K2-Horizon-7B on the same four checkpoints (e.g., 10.0% vs. 4.4% RMS difference on EQ). Its expected sampling noise is also higher, but the observed fluctuations remain about 3 above it. F&r events (Table B), instead, are comparable between the two models and are not statistically significant on any of the three datasets. Finally, Table C shows that fewer facts are certified to have lost at least 1 bit for K2-Horizon-375B-A23B than for the 7B variant (18 vs. 31 at k=128). As for the 7B, no fact is certified below k=16.

Table A: K2-Horizon-7B vs -375B-A23B fluctuations.

Dataset7B RMS diff7B RMS max diff7B binomial375B RMS diff375B RMS max diff375B binomial
EQ4.4%6.8%1.5%10.0%14.4%3.7%
SQA-V1.7%2.6%1.2%7.4%11.1%2.8%
IKP10.0%15.1%3.1%11.7%16.6%3.9%

Table B: K2-Horizon-7B vs -375B-A23B forgotten-and-reappeared facts.

Dataset7B obs. f&r7B sim. f&r7B p_adj375B obs. f&r375B sim. f&r375B p_adj
EQ3.0%3.9%1.0003.0%5.0%1.000
SQA-V4.5%6.1%1.0005.7%6.6%1.000
IKP6.3%8.3%1.0004.4%4.3%1.000

Table C: K2-Horizon-7B vs -375B-A23B in facts certified to have lost at least 1 bit.

Modelk=1k=2k=4k=8k=16k=32k=64k=128EQ / SQA-V / IKP at k=128
K2-Horizon-7B000041320315 / 1 / 25
K2-Horizon-375B00001510183 / 2 / 13

We included these results in Appendix C.10

[1] IFM. Introducing k2 horizon: Frontier performance, radically open. https://ifm.ai/blog/k2/, 2026.


Question 1

There is no specific advantage in terms of findings: since the logarithm is monotonic, it does not change which facts are identified as forgotten or their ranking. The log is nonetheless a deliberate choice, for two reasons. (i) Interpretation: it expresses factual knowledge loss in bits. (ii) Aggregation: it maps multiplicative changes to additive ones, so that equal-factor increases and decreases cancel symmetrically; it compresses the heavy tail arising when ppostp_\mathrm{post} is small so that a few facts do not dominate aggregate statistics; averaging log-ratios corresponds to a geometric mean of ratios, which is appropriate for multiplicative changes.


Question 2

We agree with the reviewer that the model can internally store more information than what it exposes [2] and that forgetting might be caused by multiple concurring events (e.g., [3]). We align with most work in the field [4, 5, 6] and measure extractable knowledge, i.e., the observable factual knowledge a user experiences when interacting with the LLM. In other words, what can be measured from its output distribution. Nonetheless, if the internal model knowledge is provided in a probabilistic form, then our proposed metric can be readily applied in that case as well. We updated the manuscript in Section 2 and made the distinction between external, extractable knowledge and internal knowledge clearer.

[2] Gekhman, Zorik, et al. "Inside-out: Hidden factual knowledge in llms." COLM, 2025.
[3] Ren, Yi, and Danica Sutherland. "Learning dynamics of llm finetuning." International Conference on Learning Representations. Vol. 2025. 2025.
[4] Petroni, Fabio, et al. "Language models as knowledge bases?." EMNLP-IJCNLP, 2019.
[5] Dong, Qingxiu, et al. "Statistical knowledge assessment for large language models." NeurIPS, 2023.
[6] Zhao, Wayne Xin, Naoki Yoshinaga, and Daisuke Oba. "What matters in memorizing and recalling facts? multifaceted benchmarks for knowledge probing in language models." EMNLP-F, 2024.

We thank the reviewer once more for the time spent on our paper. We would be grateful to know whether our rebuttal has addressed their concerns. If anything in our response is still unclear, we are happy to provide further details.

As the rebuttal approaches its end, we thank the reviewer for their feedback. We hope the additional details address their remaining concerns, and we would be glad to clarify any point that is still unclear before the end of the discussion period.

Comments on the paper

1 comment

We thank all reviewers and the AC for their efforts and appreciate that the reviewers recognized that prior works can overestimate forgetting (4nk9), that aggregate accuracy is not a reliable metric for knowledge forgetting (4nk9), that sampling variance should be accounted for (4nk9, j9if, Yqi9), and that our metric provides a more consistent assessment than existing ones (4nk9, Yqi9).

Rebuttal Summary

During the rebuttal, we provided detailed responses to all reviewers. In summary:

  • We provided further evidence supporting the main paper claim that forget-and-reappear events are mostly a sampling artifact.
  • We showed that the proposed metric is statistically sound even when extended to multiple-fact certification and frontier models.
  • We demonstrated that our results are robust to the evaluation choices made, showing that our conclusions remain invariant when using an LLM-as-judge instead of exact match.
  • We clarified that the paper targets extractable knowledge, and that the proposed metric also applies to internal knowledge when expressed probabilistically.

Finally, we updated the manuscript with the reviewers’ suggestions (edits are in blue).

We sincerely thank the reviewers for their thoughtful comments and respectfully invite the AC to consider these clarifications.

Best regards,
Authors of Submission 23742

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.