- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether LLMs really lose factual knowledge during post-training, or whether much of the measured loss comes from how forgetting is measured. Recall of each fact is treated as a probability. The authors estimate it with 128 rollouts per question at each of 55 public checkpoints of Olmo-3-7B, SmolLM3-3B and K2-Horizon-7B. They use three QA sets: EntityQuestions, SimpleQA Verified and Incompressible Knowledge Probes.
Per-fact recall moves a lot between neighboring checkpoints while aggregate accuracy stays roughly flat. A stationary-binomial null is rejected for all nine model-dataset pairs. Facts that drop to 0/128 and later come back appear at rates close to that null. When only the post-trained model's budget is doubled, 30.7% of binary forgetting decisions flip, and 51.9% flip at 4x. The paper then proposes a certified lower bound on knowledge loss in bits, based on Clopper-Pearson bounds. It comes with a closed-form rollout budget and a multi-checkpoint version. Answers are scored by exact match, and an LLM re-check of 1000 random negatives finds 3.4% false negatives.
Critical strengths
- The main message is useful and well supported. Fig. 5 shows that a 0/k "forgotten" label depends heavily on how many samples were drawn. The experiment behind this is large, with about 100M rollouts in total, and the stationary-binomial null is a reasonable reference. The bound itself is simple and correct, and Eq. 4 tells a practitioner how many rollouts to plan for. I expect others who measure forgetting to pick up this framing.
Critical weaknesses
-
The measured quantity has not been checked on the cases that drive the conclusions. Scoring is exact match, and the false-negative check in App. C.1 used random negatives. These are probably dominated by facts the model never recalls, but the appendix gives no breakdown. Table 7 includes canonical facts that drop sharply, such as "Salvador Dali" for Olmo-3-7B on IKP (107/128 to 0/128). The raw outputs are not shown, and that LLM re-check was not applied to these events. Prompting also differs between base and post-trained checkpoints. Base models get the three exemplars as plain text. Post-trained models get them as a chat with a system prompt, with thinking enabled (except SmolLM3-3B) and up to about 32k tokens. The certified losses may therefore mix knowledge change with changes in format and elicitation.
-
Two of the main claims go further than the evidence. The claim in §4.2 that most forget-and-reappear events are explained by sampling compares aggregate rates with a stationary null that the paper itself rejects earlier. In Table 4 the difference is significant in 4 of 9 pairs, and in another 4 the observed rate is below the null. In Fig. 5 only the post-trained budget grows, so a flipped decision just means one correct answer within up to 1024 samples. That does not show that nothing was lost. In Fig. 6b the paper's own bound still certifies at least 1 bit of loss for 182 to 267 facts per model.
Questions for authors
- Could you apply your existing LLM re-check to the facts labeled forgotten, the forget-and-reappear events and the Table 7 entries? What is the false-negative rate on these cases?
- How many post-trained rollouts run out of tokens and fall back to "Final Answer:"? Do Fig. 5 and Fig. 6b still hold when base and post-trained models are prompted the same way, for example with thinking turned off?
- Could you test individual forget-and-reappear events, e.g. each zero-count checkpoint against its neighbors with FDR control, instead of comparing aggregate rates only?
6 comments
Question 2
The number of rollouts that run out of tokens and fall back to “Final Answer” is negligible and mostly occurs on questions the model does not know the answer to, leading to a degenerate loop that hits the max tokens allowed. Table G: rollouts hitting the length limit and ran into the “Final Answer” fallback
| Model | EQ | SQA-V | IKP |
|---|---|---|---|
| Olmo-3-7B-Think | 0.024% | 0.025% | 0.010% |
| SmolLM3-3B | <0.001% | <0.001% | <0.001% |
| Qwen3.5-35B-A3B | 1.169% | 0.894% | 0.061% |
We thank the reviewer for suggesting prompting base and post-trained models in the same way. Unfortunately, while we can compare them under the same prompting, it would be an unfair comparison, as the two model variants are trained differently. Thus, removing the chat template from a post-trained model would be an unfair evaluation. Nonetheless, we can compare think-on and think-off performance as suggested. As we write in Appendix A.3 (paragraph “Rollout sampling”), the main paper already shows results on SmolLM3-3B with thinking turned off, as both variants show comparable performance; thus, we went for the cheaper option. We therefore recomputed rollout sampling for Qwen3.5-35B-A3B with thinking off and SmolLM3-3B with thinking on, reporting the results in Tables H and I below. We skipped comparing Olmo3 with thinking off, as the non-think variant is a different model, trained on a different dataset.
Table H shows that while overturn rates are slightly lower with think-off for Qwen3.5, they are still significantly higher than SmolLM3, and end up saturating (>99%) either way at high rollout budgets. Additionally, as can be seen in the table, SmolLM3 factual knowledge with think-off is only marginally lower than with thinking enabled, supporting our claim in the appendix.
Table H. Results overturned by additional sampling: think-on vs. think-off
| Model | Dataset | 2× k_p: off | 2× k_p: on | 4× k_p: off | 4× k_p: on | k_b=1 → k_p=1024: off | k_b=1 → k_p=1024: on |
|---|---|---|---|---|---|---|---|
| SmolLM3-3B | EQ | 23.2 | 24.0 | 41.7 | 43.1 | 91.4 | 91.1 |
| SmolLM3-3B | SQA-V | 24.0 | 24.6 | 42.7 | 43.7 | 89.8 | 91.1 |
| SmolLM3-3B | IKP | 25.6 | 26.3 | 43.8 | 45.4 | 92.5 | 94.2 |
| Qwen3.5-35B-A3B | EQ | 34.6 | 45.4 | 58.6 | 69.5 | 99.7 | 99.8 |
| Qwen3.5-35B-A3B | SQA-V | 34.1 | 36.4 | 58.5 | 61.2 | 99.5 | 99.6 |
| Qwen3.5-35B-A3B | IKP | 34.4 | 40.9 | 56.4 | 66.3 | 99.3 | 99.9 |
Table I compares facts certified to have lost at least 1 bit between think-on and think-off. SmolLM3 shows similar results between the two variants, while Qwen3.5 surprisingly shows more certified forgotten facts with thinking enabled. Running a simple aggregate analysis reveals that while the post-trained version of Qwen3.5 with thinking enabled is more likely to sample correct answers than its base variant in EntityQuestions and SimpleQA-Verified on average, in IKP it has a 5% lower probability of answering correctly than the base model and an 8% lower chance than the think-off variant. This suggests that models should be tested in depth and in different configurations. However, this does not invalidate the claims made in our paper and our proposed metric, as we tested the configurations that should expose more knowledge from the model according to the literature [1]. Rather, it opens a new research direction that would extend our work and [1].
Table I. Facts certified to have lost at least 1 bit: think-on vs. think-off
| Model | Thinking | k=8 | k=16 | k=32 | k=64 | k=128 | k=256 | k=512 | k=1024 | EQ / SQA-V / IKP at k=1024 |
|---|---|---|---|---|---|---|---|---|---|---|
| SmolLM3-3B | off | 0 | 18 | 38 | 67 | 102 | 139 | 204 | 267 | 63 / 32 / 172 |
| SmolLM3-3B | on | 0 | 17 | 39 | 65 | 98 | 132 | 189 | 250 | 61 / 27 / 162 |
| Qwen3.5-35B-A3B | off | 0 | 10 | 25 | 51 | 84 | 125 | 171 | 214 | 49 / 69 / 96 |
| Qwen3.5-35B-A3B | on | 0 | 49 | 82 | 125 | 167 | 209 | 228 | 251 | 5 / 49 / 197 |
We added these results and discussions in Appendix C.9.
[1] Gekhman, Zorik, et al. "Thinking to recall: How reasoning unlocks parametric knowledge in llms." COLM, 2026.
Question 3
We want to clarify that Figure 5 further stresses that zero-correct rollouts do not imply that factual knowledge is forgotten, i.e., it does not claim that there is no forgetting. On the other hand, Figure 6b shows our metric to be effective, i.e., separating real knowledge loss from sampling noise. Therefore, it does not contradict the claim.
Regarding Table 4, although the difference reaches statistical significance in 4 out of 9 pairs, its magnitude remains very small (under 3% across most pairs, with a single case peaking at 4.9%). Thus, to further prove that f&r events are mostly caused by sampling variance, we follow the reviewer’s suggestion and tested each forget-and-reappear (f&r) event individually. Following the manuscript, we define an event as consecutive checkpoints with 0 correct rollouts out of , preceded and followed by checkpoints with at least one correct rollout. For each event, we compare the preceding and the following checkpoint with the zero-count run (pooling its rollouts) via two one-sided Fisher exact tests. We mark an event significant only if both tests are significant (i.e., ). We control the false discovery rate with Benjamini–Hochberg at across all events of each model–dataset pair.
Table J. Significance of individual forget-and-reappear events under BH correction.
| Model | Dataset | Events | Sig. f&r |
|---|---|---|---|
| Olmo-3-7B | EQ | 865 | 0 |
| SQA | 1109 | 0 | |
| IKP | 2061 | 1 | |
| SmolLM3-3B | EQ | 352 | 8 |
| SQA | 353 | 1 | |
| IKP | 662 | 29 | |
| K2-Horizon-7B | EQ | 252 | 1 |
| SQA | 393 | 0 | |
| IKP | 593 | 4 |
Only 44 of the 6,640 events (0.66%) are significant, and 6 of the 9 model–dataset pairs have at most one. Most events cannot be significant regardless of the correction. Since the run has zero correct rollouts, the p-value depends only on the neighbouring counts, and a single zero-count checkpoint requires at least correct rollouts on both sides to reach . 96.5% of events fall below this threshold, i.e., a fact recalled 1–4 times out of 128 is statistically indistinguishable from one recalled 0 times. Moreover, we note that this test is anti-conservative, as events are selected because a zero was observed. Results are consistent when excluding the greedy rollout (45 of 6,645 events) and when applying Benjamini–Hochberg jointly across all nine model–dataset pairs instead of within each pair (37 of 6,640 events).
These results support our claim: 0 correct rollouts out of indicate a low sampling probability rather than forgetting, and almost all apparent f&r events are explained by sampling variance. We added this analysis in Section 4.2.
Question 1
We thank the reviewer for the suggestion. As discussed in the limitations section (Appx. F), evaluating LLM-generated rollouts is inherently noisy, and this affects every study that measures forgetting from model outputs. This project evaluated about 100M rollouts, so judging all of them with an LLM would be prohibitively expensive, and the judge would add its own errors. Nonetheless, to demonstrate that our results hold either way, we re-ran the LLM judge on answers that the exact match marked as wrong, sampling 1000 answers from each of the three groups suggested by the reviewer. The judge considered correct 13.1% of rollouts from the forgotten pool (Fig. 6b), 1.9% of forgotten-and-reappear rollouts, and 16.7% of rollouts of Tab. 7. These false negatives are concentrated in a few facts rather than spread across the data. That is, among forgotten facts, 154 of the 188 sampled facts (82%) contain no false negatives. Thus, our core findings consistently hold. In Table 7, only three rows out of 18 would be replaced (rows 1 and 2 for Olmo3 in IKP, and row 2 for SmolLM3 in IKP as well) by other facts, while the others would remain unchanged. IKP, in practice, is the dataset where most false negatives occur. Instead, Tab. F below shows how Fig. 6b would change if we account for the false negatives proposed by the LLM. The number of facts certified to have lost at least 1 bit remains substantial, and our conclusions remain invariant; i.e., no fact is certified below k = 16, with the count increasing with the budget.
Table F. Facts certified to have lost at least 1 bit, before → after removing flagged facts.