- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies data-free continual pre-training and fine-tuning (Pythia 160M-12B, Qwen2.5-0.5B, TinyLlama-1.1B, OLMo-2-1B; Korean, math, code, Alpaca, KoAlpaca; 40M or 12.6M tokens; three seeds for references and the Table 5 intervention except two seeds for the Pythia-12B and OLMo baselines (Table 5 caption, line 378); two seeds per freezing arm except Qwen, Table A3 lines 648-653) and asks where forgetting, defined as the mean increase in held-out cross-entropy on old domains (Eq. 3), is located. Freezing one (site, band) parameter group at a time (Section 4.1) shows that freezing the lowest- band of the output embeddings removes 31.3-51.0% of forgetting at five settings with no learning cost, while the same band of the body holds a negligible share (-1.3% to +2.7%) and mass-matched body freezing 0.4-12.3% (Table 2); learning lives in the upper body bands. The site is governed by vocabulary deficiency, the share of old-domain evaluation tokens under-exposed () in the new corpus (Table 3), and the mechanism offered is one-sided softmax gradients on absent tokens (Eq. 1) amplified by Adam's normalisation (Eq. 2). The intervention raises AdamW's to on the output projection only; Table 5 reports 39.4-67.9% forgetting reduction at seven of eight settings with learning change +0.4% to +10.9%, additive composition with replay (Table 6), a 91% rescue of released-head LoRA (Section 5.3), and that post-hoc row edits recover under 5% (Table 7). Appendix A.2 (Table A10) shows a lower embedding learning rate traces the same frontier (67.5% vs 66.7%). The localisation grid is a genuine and well-executed finding and the intervention is compared at the same learning rate within each setting (Table 5; paired across the rate grid in Table A11); the claims are weakened because forgetting is measured only as loss, the diagnostic needs an old-domain corpus the data-free setting does not grant, the remedy is empirically a per-tensor learning-rate reduction already known from the rare-token literature, and one setting is bimodal across seeds while one is excluded from the summary.
Critical strengths
- Originality: The causal-share grid is a real contribution. Partitioning three sites by three bands, freezing one group at a time with fixed baseline masks and mass-matched body controls (Section 4.1, lines 206-215), gives a necessity test rather than a correlational drift metric (two seeds per freezing arm except Qwen, Table A3), and the result is clean: the bottom output band holds 31.3-51.0% of forgetting at five settings with +0.1% to +5.2% learning change, the same band of the input embeddings -1.5% to +1.4%, and mass-matched body freezing 0.4-12.3% (Table 2, lines 235-247). The split of forgetting (bottom output band) from learning (upper body bands, 8.8-40.7% of learning, line 249) is the kind of localisation the field has been missing, and the vocabulary-deficiency rank association (Table 3) suggests it can be anticipated before training, with the caveat that each Table 3 row changes both the selected learning rate and the evaluation pool (lines 216-224, 264-269; Appendix A.1 lines 628-634), so the ordering is a within-grid association and does not isolate deficiency as the cause.
- Significance: If the intervention's effect survives the controls requested below, a parameter-group change is the cheapest forgetting defence proposed for data-free adaptation: zero compute or memory overhead, no token lists, no replay data (lines 328-329), composing additively with replay (79.8% on Qwen/Korean, Table 6) and rescuing released-head LoRA from a 23-fold forgetting surge (lines 419-421).
Critical weaknesses
- Soundness: Forgetting is measured only as held-out loss, the diagnostic needs data the setting excludes, and stability is uneven. Every reduction in the eight-setting main grid is a change in cross-entropy in nats (Eqs. 3-4). The paper does report two capability-based results (Appendix A.5 lines 786-797, Table A12): 83.6% reduction at Pythia-6.9B against 73.9% loss-based on the same pair, and 49.0% vs 49.1% at Pythia-1.4B, but at on math with 1% replay, which the authors say "predates the grid" and is "not data-free Korean" (lines 470-472). Those two cells are encouraging and reduce the weight of this concern, but they do not cover the data-free Korean grid on which every headline number rests. At 6.9B and 12B the baseline forgetting is about 0.1 nats (Table 5), so a 60-65% reduction is 0.06 nats whose effect on any task is unmeasured; the title's "catastrophic forgetting" is a loss-drift claim. The pre-retraining diagnostic, vocabulary deficiency, is defined on old-domain evaluation positions (lines 119-123) sampled from for Pythia and OLMo or from proxies for closed models (lines 181-186), and the deployment rule (lines 343-348) again needs the share of old-domain positions; under the paper's own zero-access-to- assumption (lines 112-115) this statistic cannot be computed from the new corpus alone, so "risk ranking from token counts alone" (abstract) requires a representative old-domain sample the data-free framing does not grant. Finally the Pythia-410M arm is bimodal: across six seeds reduction spans 4.6-46.6% and learning change -32.9% to +16.8% (lines 283-286; Table 5 footnote 32.7% and -7.0% at six seeds vs 43.8% and +10.9% at three), and Pythia-160M collapses at its peak rate and is excluded from the summary range while remaining in the "eight settings" count (lines 253-260, 356-357).
- Significance: The remedy is, on the paper's own evidence, interchangeable with a per-tensor learning-rate reduction, and its mechanism (arresting under-exposed rows) is the one rare-token gating and freezing already use, so the contribution is the diagnosis more than the instrument. Appendix A.2 states that freezing rows with , per-step gating and output-scoped "agree closely" (lines 334-338), and Table A10 shows scaling the embedding learning rate by 0.10 gives 67.5% reduction at +0.7% learning versus 66.7% at +2.2% for . Table A10 is a single Qwen/Korean campaign, so the equivalence is an empirical trade-off match in one setting, not a theorem. Section 6 acknowledges that Yu et al. 2022's rare-token gating "matches output " (line 461), and Kwok et al. 2024 freeze rare-token rows for continual ASR; neither proposed the specific optimizer setting, but both act on the same rows by the same logic (lines 334-338 list freezing and gating as equivalents). The paper's framing ("a single-line optimizer adjustment may serve as the primary defense," abstract; "identify Adam as the amplifier and as the brake," lines 105-106) presents the instrument as the discovery. The same-rate comparisons also show the intervention improving target learning by +5.4% to +10.9% (Table 5) and +10.0% / +18.4% (Table 3, Qwen math/code), which a pure brake on absent-token rows should not do; the paper does not analyse why, and without that analysis the "no learning cost" claim is an observation rather than an understood property. What survives is the localisation (strength above), which is valuable but is not what the abstract and title lead with.
Questions for authors
Questions that could change my assessment
- Please add a downstream evaluation (e.g. the Pythia evaluation suite or a small set of old-domain tasks such as LAMBADA, HellaSwag and a code or math task) for the reference and intervention arms at the eight settings of Table 5, with three seeds. Score-change criterion: if the intervention preserves old-task accuracy relative to the reference by a margin consistent with the loss reductions, Soundness rises to 3 and I would raise the Rating to 4; if the loss reductions do not translate into accuracy, the claims must be restated as loss drift and the Rating falls to 2.
- Please compare against a reference tuned over a two-rate grid (body rate output-embedding rate), i.e. the best per-tensor learning-rate baseline, and against whole-output-embedding freezing and MoFO in the Section 5.3 frontier, at the same body rate as the intervention. Score-change criterion: if output-scoped still gives a non-trivial reduction at zero learning cost against the best two-rate reference, Informativeness stays at 3 and the Rating stays at 3; if the two-rate reference matches it, the contribution must be reframed as the localisation plus a per-tensor rate rule, and the Rating falls to 2 unless question 1 succeeds.
- Report Table 5 with six seeds for every arm, include Pythia-160M at its tuned rate in the summary range, give the per-seed distribution for 410M, and explain the +5% to +18% target-learning improvements at the same rate. Score-change criterion: a range that includes all eight settings at six seeds with the bimodal arm resolved and the learning gain explained would raise Soundness to 3.
Clarifications and minor points (do not affect my assessment)
- State in the abstract that the localisation and the remedy apply to high vocabulary-deficiency injections; under in-distribution tuning forgetting is in the MLPs (Table 4) and body-scoped is free at only one of two cells (Section 5.5). The title claims the general case.
- State early that vocabulary deficiency requires an old-domain evaluation sample or proxy, and what to do when none exists.
- Table 3's caption says rows use different evaluation pools and are "comparable with neither," but the text reads cross-row trends from the same table; either match across rows or drop the cross-row prose.
- Table A11 (same-rate pairing and rate invariance) and Table A10 (equivalence to learning-rate scaling) belong in the main text; they answer the first question any reader asks about the protocol.
- State the seed counts of the freezing grid (two per arm except Qwen, Table A3) in Section 4.1 rather than only in the appendix caption.
- Release the corpora splits, rate grids per arm and seed lists; the intervention is one line, but the grid is not reproducible from the text.
5 comments
Response 2
(Edited 2026-11-15 22:02: table and section numbers updated to match the revised PDF.)
Question 2 and Weakness (Significance)
- Question 2:
Please compare against a reference tuned over a two-rate grid (body rate × output-embedding rate), i.e. the best per-tensor learning-rate baseline, and against whole-output-embedding freezing and MoFO in the Section 5.3 frontier, at the same body rate as the intervention. Score-change criterion: if output-scoped ϵ still gives a non-trivial reduction at zero learning cost against the best two-rate reference, Informativeness stays at 3 and the Rating stays at 3; if the two-rate reference matches it, the contribution must be reframed as the localisation plus a per-tensor rate rule, and the Rating falls to 2 unless question 1 succeeds. - Weakness (Significance):
The remedy is, on the paper's own evidence, interchangeable with a per-tensor learning-rate reduction, and its mechanism (arresting under-exposed rows) is the one rare-token gating and freezing already use, so the contribution is the diagnosis more than the instrument.
Thank you for this request. We ran the two-rate grid. The best output-embedding learning rate matches ε on forgetting, so we accept the reframing you suggest. The reason is in the update rule. ε multiplies each step by √v̂ / (√v̂ + ε). Within the output embeddings, this lowers the step of rows with small √v̂, such as the rows of rare tokens, by orders of magnitude, while rows with √v̂ above 1e-4 keep at least half of it. ε therefore acts as a lower learning rate that each parameter sets by itself from its √v̂ (Section 5.1).
What we changed. We ran the grid at Qwen2.5-0.5B and Pythia-1.4B on Korean, each at two body learning rates (lr 1e-4 and 3e-5). The output-embedding learning rate was scaled by 1, 0.3, 0.1, 0.03 and 0 (frozen output embeddings). We also ran MoFO. Each condition has three seeds (Section 5.1, Table A22). Section 5.1 now says that output-scoped ε gives the result of the best output-embedding learning rate, with no second learning rate to tune. The abstract and the Introduction now say the same. Related Work now states that freezing the output layer (Gu & Feng, 2020) and freezing rare-token rows act on the same rows. What the paper adds is which rows carry the forgetting, and why.
Evidence. Each lower output-embedding learning rate removes more forgetting. At ×0.03 it removes 44.9–71.8%, against 45.3–69.5% for ε (Table A22). Freezing the output embeddings removes nearly the same forgetting as ε, within 2.5 points. At Pythia, whose output embeddings are not tied to its input embeddings, the learning change is also nearly the same. Qwen's output embeddings are tied to its input embeddings. At Qwen, freezing loses 4.3% of target learning at body lr 3e-5, while ε loses none. MoFO removes 51.5–58.6% but loses 3.6–13.6% of target learning.
Table A22. Korean, three seeds per condition. Each cell: share of reference forgetting removed (%), with the learning change (%) in parentheses. Output-embedding lr ×k: output-embedding learning rate scaled by k relative to the body learning rate. ×0 freezes the whole output embedding.
| Qwen2.5-0.5B, body lr 1e-4 | Qwen2.5-0.5B, body lr 3e-5 | Pythia-1.4B, body lr 1e-4 | Pythia-1.4B, body lr 3e-5 | |
|---|---|---|---|---|
| Ref. forget / learn (nats) | 0.614 / 0.376 | 0.136 / 0.320 | 0.463 / 0.331 | 0.107 / 0.287 |
| Output-scoped ε | 67.7 (+5.8) | 69.5 (+0.4) | 45.3 (+2.3) | 52.0 (+0.6) |
| Output-embedding lr ×0.3 | 54.4 (+3.7) | 57.4 (+0.6) | 32.6 (+1.5) | 38.3 (+0.5) |
| Output-embedding lr ×0.1 | 65.8 (+4.6) | 69.6 (−0.5) | 40.9 (+2.0) | 50.0 (+0.5) |
| Output-embedding lr ×0.03 | 69.0 (+3.8) | 71.8 (−2.8) | 44.9 (+2.1) | 52.7 (+0.5) |
| Output-embedding lr ×0 (frozen) | 69.2 (+0.9) | 72.0 (−4.3) | 46.4 (+2.1) | 52.9 (+0.4) |
| MoFO | 58.6 (−3.6) | 51.5 (−13.6) | 56.1 (−4.4) | 53.3 (−11.2) |
Over 160M tokens, freezing and ε still remove nearly the same forgetting at the four models tested (Table A7). At Qwen, the frozen tied embeddings have a new-domain loss 0.018 nats higher at the end of training, in every seed, and the gap grows during training (Appendix A.5).
Response 3
(Edited 2026-11-15 22:02: table and section numbers updated to match the revised PDF.)
Question 3 (six seeds, Pythia-160M, per-seed values) and Weakness (Soundness, last part)
- Question 3:
Report Table 5 with six seeds for every arm, include Pythia-160M at its tuned rate in the summary range, give the per-seed distribution for 410M, and explain the +5% to +18% target-learning improvements at the same rate. Score-change criterion: a range that includes all eight settings at six seeds with the bimodal arm resolved and the learning gain explained would raise Soundness to 3. - Weakness (Soundness, last part):
Finally the Pythia-410M arm is bimodal: across six seeds reduction spans 4.6-46.6% and learning change -32.9% to +16.8% (lines 283-286; Table 5 footnote 32.7% and -7.0% at six seeds vs 43.8% and +10.9% at three), and Pythia-160M collapses at its peak rate and is excluded from the summary range while remaining in the "eight settings" count (lines 253-260, 356-357).
Thank you for this request. All eight settings are now at six seeds, Pythia-160M is in the reduction range, and the 410M seeds are no longer bimodal.
What we changed. We reran every run of Table 6 at six paired seeds (seeds 100–105) with one pipeline, including Pythia-6.9B and 12B. Every row now reports n and a 95% paired-bootstrap confidence interval. Pythia-160M is reported at its tuned learning rate (lr 3e-5) in Table 6, because training at its peak lr 3e-4 collapses (Table A16). The abstract now gives the reduction range over all eight settings and states that target learning is not degraded at seven of them. The per-seed values for 410M are in Appendix A.3 (Table A2). We also corrected the TinyLlama row (item 5(a) of the general response).
Evidence. Item 2 of the general response (part 1) shows the rerun Table 6. Over all eight settings at six seeds, the reduction is 39.0–67.8%, including Pythia-160M at its tuned learning rate (56.5% [49.5, 62.6]). The learning change is −0.2% to +5.7% at the other seven settings, and none of their intervals lies entirely below zero. At Pythia-160M it is −6.6% [−11.5, −1.6], so the paper does not call that setting free of a loss in target learning. At these seeds, Pythia-410M is not bimodal: its six seeds span 31.2–46.7% (Table A2). The seed with the smallest reduction is also the one that learns least.
Table A2, two rows. Reduction (%) per seed, with the learning change (%) in parentheses.
| Setting | seed 100 | 101 | 102 | 103 | 104 | 105 |
|---|---|---|---|---|---|---|
| Pythia-160M (lr 3e-5) | 61.5 (+3.9) | 58.8 (−6.1) | 52.8 (−12.5) | 41.3 (−15.6) | 67.6 (−3.0) | 57.0 (−6.6) |
| Pythia-410M | 36.6 (−2.6) | 46.7 (+13.8) | 46.2 (+0.6) | 31.2 (−9.4) | 41.2 (−3.2) | 32.3 (−0.1) |
Question 3 (learning gain)
- Question 3:
explain the +5% to +18% target-learning improvements at the same rate.
Thank you. The gain does not mean that the ε run learns Korean better. Without ε, the reference pushes down the output rows of tokens that are rare in the new data. This is the drift that causes forgetting. Some of these rare tokens are Korean, so the reference also predicts them worse. ε stops this drift, and the learning change counts what the reference loses as a gain for ε.
What we changed. After training, we split the tokens of the Korean validation text into three groups by how often each token occurs in the first 40M tokens of the Korean training pool: absent (c(t) = 0), rare (1 to 999 times) and frequent (1,000 times or more). We then measured the loss of each run on each group separately (Table A3).
Evidence. In the rerun Table 6, the learning gains are at most +5.7%, smaller than in the submission. Rare tokens are a small part of the Korean validation text (Positions column). On frequent tokens, the two runs reach the same loss at every setting except Pythia-160M. The ε run is better only on rare tokens, and most on tokens absent from those 40M tokens (c(t) = 0). Across models, the gap is largest at Qwen and OLMo, whose tokenizers produce the most rare tokens in Korean.
Table A3, selected columns. Six paired seeds. Loss is cross-entropy per token (nats). Ref. − ε is the reference's loss minus the ε run's loss; a positive value means the ε run predicts these tokens better.
| Setting | Positions: c(t) = 0 / 1–999 / ≥ 1,000 | Ref. − ε, c(t) = 0 | Ref. − ε, 1 ≤ c(t) < 1,000 | Ref. − ε, c(t) ≥ 1,000 |
|---|---|---|---|---|
| Pythia-160M (lr 3e-5) | 132 / 7,861 / 401,207 | +1.086 | +0.040 | −0.051 |
| Pythia-410M | 132 / 7,861 / 401,207 | +0.797 | +0.257 | −0.007 |
| Qwen2.5-0.5B | 402 / 18,462 / 390,336 | +2.058 | +0.434 | 0.000 |
| TinyLlama-1.1B | 87 / 6,509 / 402,604 | +0.529 | +0.122 | 0.000 |
| Pythia-1.4B | 132 / 7,861 / 401,207 | +0.971 | +0.295 | +0.001 |
| OLMo-2-1B | 331 / 11,638 / 397,231 | +1.586 | +0.513 | 0.000 |
| Pythia-6.9B | 132 / 7,861 / 401,207 | +0.343 | +0.095 | 0.000 |
| Pythia-12B | 132 / 7,861 / 401,207 | +0.358 | +0.120 | −0.001 |
Response 1
(Edited 2026-11-15 22:02)
Thank you for the time and care you put into this review. We also thank you for seeing the causal-share grid as a real contribution. Your three questions each came with a clear criterion, and we followed them. Responses 1 to 3 answer them in order.
Question 1 and Weakness (Soundness, first part)
Please add a downstream evaluation (e.g. the Pythia evaluation suite or a small set of old-domain tasks such as LAMBADA, HellaSwag and a code or math task) for the reference and intervention arms at the eight settings of Table 5, with three seeds. Score-change criterion: if the intervention preserves old-task accuracy relative to the reference by a margin consistent with the loss reductions, Soundness rises to 3 and I would raise the Rating to 4; if the loss reductions do not translate into accuracy, the claims must be restated as loss drift and the Rating falls to 2.At 6.9B and 12B the baseline forgetting is about 0.1 nats (Table 5), so a 60-65% reduction is 0.06 nats whose effect on any task is unmeasured; the title's "catastrophic forgetting" is a loss-drift claim.Thank you for this request. We ran the downstream evaluation at all eight settings, with six paired seeds. The ε run keeps more old-task accuracy than the reference at every setting, and the gain is largest where the reference loses most.
What we changed. We now evaluate every reference and ε run of Table 6 zero-shot, in FP32, with the LM Evaluation Harness (v0.4.12). The evaluation runs inside the training process. We used six paired seeds instead of the three requested. The old-domain tasks are LAMBADA, HellaSwag, PIQA, ARC-Easy and WinoGrande. KoBEST measures the new domain. GSM8K and HumanEval are also reported (Sections 3.3 and 5.2, Table 7, Tables A24 and A25).
Tables A24 and A25, LAMBADA rows. LAMBADA accuracy (%). Zero-shot, FP32, mean over six paired seeds. Pythia-160M is at lr 3e-5, because training at its peak lr 3e-4 collapses (Table A16).
LAMBADA is the task where the reference loses most: 6.6 to 35.1 points against the base model. The ε run gets back 2.8 to 11.9 points of this at every setting. At 6.9B and 12B, it ends within 0.5 points of the base model.
Outside LAMBADA, the ε run also gains on HellaSwag, PIQA and ARC-Easy at Qwen2.5-0.5B and OLMo-2-1B, whose references lose the most. The largest of these gains is 5.9 points (Qwen, ARC-Easy). In the remaining cells, the two runs differ by at most 1.6 points. Across the forty cells (eight settings × five tasks), the more accuracy the reference loses on a task, the more the ε run gets back: the correlation between the two is 0.87 (Section 5.2). The KoBEST mean stays within 0.5 points of the reference at every setting, so the gain on English does not reduce accuracy on Korean. GSM8K and HumanEval are near zero for most base models at these scales (Tables A24 and A25), so they cannot show a difference. The exception is HumanEval at Qwen2.5-0.5B: 28.7 (base), 10.3 (reference) and 14.9 (ε run).
On your criterion: where the reference loses at least 3 points, the ε run gets back 35% of the drop on average, while it removes 39.0–67.8% of the loss increase (Table 6). The accuracy gain is smaller than the loss reduction. But on the five-task English mean it is positive at all eight settings, and across the forty cells it grows with the size of the drop. At 6.9B and 12B, the 0.06-nat difference is visible in accuracy: the ε run gets back 1.3 of the 1.7 points the reference loses on the English mean at 6.9B, and 1.5 of 2.5 at 12B. At these two models, forgetting also grows with longer training, and the reduction grows with it (Table A6, Section 5.2).
Table A6, selected columns. Runs at lr 3e-5 scheduled to 160M tokens, two paired seeds per model, evaluated at four checkpoints. The 40M row is a checkpoint of these runs, not the 40M-scheduled runs of Table 6.