acceptodds
Under review as a conference paper at ICLR 2027

The Geometry of Forgetting in nGPT: Disentangling Weight Rotations and Scale Dynamics

Abstract

nGPT achieves high training efficiency by constraining its major vector-valued parameters to unit hyperspheres while learning explicit scale parameters. This parameterization naturally decomposes parameter updates into weight rotations and scale changes. However, when a pretrained model is further trained, it remains unclear which of these geometric changes harm the behavior learned during pretraining. We study this question in domain-adaptive continued pretraining, analyzing which kinds of rotations and scale changes drive forgetting. We show that, at matched rotation angles, rotational damage varies strongly with alignment to task-sensitive directions. At matched displacement sizes, scale damage varies with the spread of per-token logit reweighting. Based on these channel-specific mechanisms, we design a two-term regularizer and use it to intervene on the two channels separately. Weight rotations drive most net forgetting, while scale damage is largely offset by output-projection interactions. With both penalty terms, suppressing task-sensitive rotations exposes this scale damage, which the scale penalty then controls. nGPT's hyperspherical parameterization thus provides not only a mechanism for training efficiency but also a coordinate system for understanding and controlling change during continued pretraining.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
nGPT, Continued Pretraining, Forgetting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aQrcOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses nGPT’s native parameterization, with unit-norm weight vectors on hyperspheres and explicit Euclidean scale parameters, to break forgetting during domain-adaptive continued pretraining into a rotation channel and a scale channel. Using a Fisher/KL local model around the pretrained point θA, the authors argue for two claims:

  1. Rotational damage at matched angle depends on alignment with a low-rank task-A-sensitive tangent subspace.
  2. Scale damage, carried mainly by the output logit scales s_z, depends on the variance under p_A of the per-token logit reweighting (E_S), not on the norm of the displacement.
Critical strengths
  1. Overall, using nGPT’s native geometry as a coordinate system for forgetting is a natural and underexplored idea.
  2. The matched-angle rotation probes and the 280 synthetic s_z probes over eleven orders of magnitude in E_S are proper interventions. The sensitive subspaces and E_S are fixed from θA beforehand, so the tests are not circular.
  3. The cancellation finding is the most interesting result.
Critical weaknesses
  1. The author state “This localizes the cancellation but does not explain why it is nearly exact.” I_end ≈ −K_s holds to within a few percent on code and medicine and is left as an observation. Several hypotheses seem testable with the existing infrastructure: (1) Does the optimizer drive output scales and output rows toward a logit-preserving equivalence class, given that z_v = s_v⟨h, w_v⟩ has approximate symmetry in which magnitude can be traded between s_v and the effective projection after h shifts? (2) Is the cancellation specific to nGPT, where s_z exists explicitly, or would an unnormalized model show the same effect inside the row norms of W_out?
  2. there's only one model size (0.5B) in in the paper. Such an undertrained model may sit in an unusual curvature regime.
Questions for authors
  1. Can you explain why I_end ≈ −K_s so precisely on code and medicine? Is there an approximate reparameterization symmetry among s_z, the output rows, and the hidden state that the optimizer exploits?
  2. How does the rotation penalty compare with a full K-FAC EWC penalty in ambient (non-tangent) coordinates, using the same statistics? This would separate the contribution of nGPT geometry from that of non-diagonal curvature.
  3. Does the normalization miscalibration (Table 3) persist if you use the true Fisher (labels sampled from p_A) instead of the empirical Fisher for the K-FAC factors?
4 comments
AuthorsOpenReview ↗

Response to Reviewer aQrc

We thank the reviewer for the careful review and the precise, testable suggestions (we ran all three), and for recognizing the probes as non-circular interventions and the cancellation as the most interesting result.

Summary. Q1 / W1: a token-selective insensitivity, formed during training; why it forms remains open. Q2: the EWC gap is lm_head row weighting; no advantage from the geometry. Q3: Table 3 was our unit error.

Q1 / W1. Why I_end ≈ −K_s, and is it specific to nGPT? It is realized as a token-selective insensitivity to the learned s_z change, and appears, much weaker, in unnormalized GPT; why optimization produces it remains open. We tested hypothesis (1) at the logit level.

Measurement. Since z_v = s_v⟨h, w_v⟩ (unit h, w_v), all four combinations of {θ_A, endpoint} × {s_z, all other parameters} follow exactly from two forward passes. Let p_R be the model with all non-s_z parameters at the endpoint and s_z at θ_A, and p_B the endpoint. The endpoint's s_z change alone costs K_sz = 1.57 (code) / 1.71 (medicine) nats at θ_A (the all-scale K_s is 1.88). Two distinct measurements: (a) θ_A-referenced, the s_z-only interaction is I_z/K_sz = −1.00 / −0.96; (b) directly, KL(p_R‖p_B), how much the same s_z change still moves the adapted model, is only 3% / 2% of K_sz. (b) is not algebraically identical to I_z; both show this insensitivity.

It is not a general insensitivity. For any scale change d we use M(d) = KL(p_R‖p_R+d) / KL(p_A‖p_A+d). The same s_z log-ratios permuted across tokens give 77–89%, and another domain's s_z change 40–49%. Transplanted onto the rotation-penalized code model, the unregularized run's own s_z change keeps 89% of its effect, so it is largely absent under the rotation penalty.

Not simple reciprocal rescaling. A trade s_B,v⟨h_B, w_B,v⟩ ≈ s_A,v⟨h_A, w_A,v⟩ predicts slope −1 between per-token log scale ratio and log change in mean output cosine; we find −6.0 (code) and −5.6 (medicine). This rules out that simple form, not every approximate symmetry.

It develops during training (reply to k9Kf, W4); these controls concern the endpoints. For (2): the GPT interaction is negative at all six endpoints, but its norm channel is small (reply to k9Kf, W4), so nGPT exposes the size and precision of the cancellation, not its existence. Revised: §5.3, App. C.7.

Q2. Full ambient K-FAC EWC with the same statistics. It forgets more at matched adaptation, a gap largely explained by its lm_head row weighting in our code experiment; with equalized rows the two reach the same trade-off. We implemented λ Σ tr(R ΔW C ΔWᵀ) with both K-FAC factors from the task-A statistics, per-matrix normalization and unpenalized scales (code):

λ3e-61e-53e-51e-4
code loss0.7990.8180.8510.899
ΔL_A, EWC+1.05+0.76+0.61+0.52
ΔL_A, rotation penalty (same code loss)+0.66+0.52+0.38+0.31

Last row: our raw rotation-only sweep, interpolated; EWC forgets 0.21–0.39 nats more.

The gap is localized to lm_head row weighting. Ambient K-FAC weights lm_head rows by E[δ_v²] and wte rows by input-token frequency, so rare-token rows are nearly free; our penalty weights rows equally. Under EWC lm_head rotates 41–51° at every λ (rotation-only: 2–6°). We changed only this, replacing the diagonal row factor by its mean:

rows equalizedλcode lossΔL_Arotation penalty (same code loss)
lm_head + wte3e-50.853+0.38+0.38
lm_head + wte1e-40.912+0.31+0.29
lm_head only3e-50.849+0.39+0.39

lm_head now rotates 3° and the gap vanishes (within 0.013 nats); with lm_head alone equalized, wte still rotates 12°, so in this code run (one seed) lm_head row weighting explains nearly all of the gap. With equal row weights, the two penalties reach the same trade-off but split the damage differently (EWC: K_w 0.14 vs. 0.24, K_s 0.26 vs. 0.18). We claim no performance advantage from the geometry; it makes this split observable. Revised: §5.2, App. C.5, App. A.

Q3. Does the normalization miscalibration persist with the true Fisher? No, and the miscalibration itself was an error on our side.

  • Training. Predictive-Fisher (label-sampled) bases and magnitudes give the same endpoints as empirical ones (normalized μ = 1; raw μ = 1, 3): within 0.004 in code loss and 0.003 in ΔL_A.
  • Basis. Predictive and empirical bases differ by about 5% in sensitive damage.
  • Cause of Table 3. The gap came from a unit mismatch between K-FAC factors, not the label convention (reply to WfWA 1/3, Correction 1). Corrected, raw and normalized coincide at every μ tested (code, 0.1–3). Revised: Table 3, App. C.2.

W2. One model size. Agreed: all analyses are at 0.5B; the limitations say so, and we make no scaling claim.

AuthorsOpenReview ↗

Follow-up to Reviewer aQrc: Responses to Your Questions

Thank you again for your insightful review and suggestions. We have conducted additional experiments addressing all three of your questions, including analyses of the cancellation phenomenon, full ambient K-FAC EWC comparisons, and predictive-Fisher tests. As the discussion period nears its end, we would greatly appreciate any feedback on our responses.

I thank the author for thorough rebuttal. The new logit-level measurement and the K-FAC EWC comparison are informative. However, the core question of why the cancellation is nearly exact remain open, and the single-model-size concern is acknowledged but not addressed. The rebuttal also weakens two stated contributions: the EWC gap comes from row weighting rather than nGPT geometry, and held-out subspace coverage is far below the reported in-sample figures. I therefore maintain my score.

AuthorsOpenReview ↗

Thank You for Your Feedback and Clarifying Our Research Focus

Thank you for your thoughtful follow-up and for carefully considering our additional analyses. We agree that understanding why the near-exact cancellation emerges and validating our findings at larger model scales remain important future directions. We also appreciate your comments on the EWC comparison and Fisher subspace coverage, which have helped us refine our claims and clarify their limitations. We hope the revised manuscript better conveys our central aim: to develop a clearer mechanistic understanding of forgetting through the interplay of rotation and scale, rather than to establish the superiority of a particular regularization method. We believe the additional analyses help clarify both what this perspective reveals and what remains to be understood. Thank you again for your constructive feedback, which has been valuable in sharpening the focus of our work.

Reviewer_k9KfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates catastrophic forgetting in nGPT by analyzing weight rotations and scale changes using Fisher Information. The authors show that forgetting depends on the direction of weight rotations and the relative distortion of token logits. Based on these findings, they propose two regularization terms to reduce forgetting. Experiments on code, math, and medicine demonstrate reductions in forgetting with only minor degradation in domain adaptation performance.

Critical strengths
  1. The paper is well-organized and easy to follow. The motivation, theoretical analysis, and experimental results are clearly presented.

  2. The paper provides useful insights into how weight rotations and scale changes contribute differently to forgetting in nGPT.

  3. The authors conduct controlled experiments to validate their findings across three domains. The ablation studies and analysis of rotation–scale interactions further support their conclusions.

Critical weaknesses
  1. Since the experiments cover code, mathematics, and medicine, I encourage the authors to include domain-specific downstream evaluations, such as HumanEval or MBPP for code generation, GSM8K for mathematical reasoning, and MedQA or PubMedQA for medical question answering. Additionally, general-domain benchmarks such as HellaSwag, PIQA, or ARC-Easy could help assess whether the proposed regularization preserves pretrained capabilities.

  2. While the geometric analysis of forgetting in nGPT is interesting, the proposed framework is specific to nGPT's parameterization. Moreover, the observation that changes along Fisher-sensitive directions lead to greater forgetting is closely related to existing Fisher-based methods. The authors should better clarify what fundamentally new insights their analysis provides beyond these established principles, particularly regarding the interaction between weight rotations and scale changes.

  3. The proposed Fisher-based approximation is accurate only for small weight rotations, while actual continued pretraining results in much larger rotations. Although the authors show that sensitive directions remain important throughout training, it is unclear how well the local analysis explains forgetting at these larger displacements.

  4. The interaction between weight rotations and scale changes is one of the paper's key findings. However, while the authors empirically identify strong cancellation effects, the underlying reason why these effects emerge during continued pretraining remains unclear. A deeper explanation of this behavior would strengthen the main contribution.

  5. The method relies on an approximate Fisher subspace with a fixed rank of 64. However, the approximation captures only 50-74% of the top-64 Fisher energy for some MLP matrices. It would be helpful to examine how sensitive the findings and regularization performance are to the approximation quality and the selected rank.

  6. Most of the forgetting reduction comes from the rotation penalty, while the additional benefit of scale regularization varies considerably across domains and is nearly negligible for math. A clearer explanation of when and why the scale penalty provides additional benefits would strengthen the motivation for combining the two terms.

Questions for authors

Could the authors address the concerns raised in the Weaknesses section, particularly regarding downstream evaluation and sensitivity to Fisher subspace estimation?

5 comments
AuthorsOpenReview ↗

Response to Reviewer k9Kf (1/2): downstream evaluation (W1) and rank sensitivity (W5)

We thank the reviewer for the careful review and the concrete benchmark and rank suggestions, all of which we ran, and for finding the paper clear and the rotation–scale analysis useful.

Summary. W1: scores on three general multiple-choice benchmarks follow the loss-level picture, while domain-specific tasks are at floor or near chance at 0.5B, before and after adaptation. W5: the Fisher approximations preserve the sensitive damage; rank 64 is uneven across matrix types, yet ranks 32–128 give the same trade-off.

W1. Downstream evaluation. General benchmarks follow the loss-level picture, while domain-specific tasks are at floor at 0.5B before and after adaptation. We evaluated θ_A and all 12 continued-pretraining endpoints zero-shot on the full labeled evaluation splits: HellaSwag validation (10,042), PIQA validation (1,838) and ARC-Easy test (2,376). Each answer is chosen by the log-likelihood of its continuation, as in lm-eval-harness. For HellaSwag and PIQA we normalize the log-likelihood by the continuation's UTF-8 byte length; ARC-Easy uses unnormalized accuracy. θ_A standard errors are ±0.005 / ±0.011 / ±0.010.

HellaSwagPIQAARC-E
θ_A0.3490.6440.472
code: unregularized0.2810.5520.290
code: rotation-only0.3350.6200.420
code: scale-only0.2810.5480.275
code: both0.3420.6160.425
math: unregularized / rotation-only0.306 / 0.3460.585 / 0.6290.475 / 0.487
medicine: unregularized / rotation-only0.299 / 0.3390.530 / 0.6110.358 / 0.448
  • Unregularized adaptation generally degrades pretrained benchmark performance (math ARC-E is the exception); on code, ARC-E falls to near chance (0.29 vs. 0.25).
  • The rotation penalty recovers 71–92% of each drop across domains and benchmarks. Math ARC-E does not drop.
  • The scale-only penalty recovers almost nothing (−8% to +17%), and "both" matches rotation-only at benchmark resolution.
  • Domain-specific tasks were at floor (≤ 0.03; GSM8K for θ_A and the math models, HumanEval/MBPP for θ_A and the code models), and MedQA was near chance for every model. Continued pretraining lowers code loss from 2.95 to 0.80 nats, yet sampled generations remain repetitive or incorrect: exact-match and pass@1 require capabilities that this 0.5B model lacks before and after adaptation. We therefore keep domain loss as the adaptation measure.

Revised: §5.2; full table, including math and medicine scale-only/both, PubMedQA and MedQA, in App. C.8 (Tables 17–18).

W5. Sensitivity to the Fisher approximation and the rank. The estimator and shared-basis steps preserve the sensitive damage, and the trade-off is stable for ranks 32–128, although rank 64 covers the output projections only partially.

  • Estimator. It barely matters: a 5° rotation into the production basis causes 1.03× the held-out damage of one into the per-vector predictive-Fisher basis on average (0.89–1.29 across matrices); the other estimators give 0.95–1.06× (reply to WfWA, 1/3). Revised: App. C.3, Table 4.
  • Shared basis vs. exact per-row subspaces (the 50–74% you cite). The shared basis does at least as well as the exact per-row basis (1.01–1.07×) once basis and energy are measured on disjoint tokens: attention matrices stay at 80–81% and MLP-up becomes 39–74%, consistent with estimation noise in per-row subspaces estimated from few effective tokens. Revised: App. C.2.
  • Rank. Coverage is uneven across matrix types: measured held-out, the top-64 subspace carries 45–66% of the energy for Q/K/V, 42% for MLP-up, but only 24% for attention-output and 14% for MLP-down columns. Revised: Figure 6, App. C.2.
  • Regularization performance vs. rank. The trade-off is stable from k = 32 upward. We retrained the rotation-only penalty (code, μ = 1) with k = 16, 32 and 128 (subspaces re-estimated for k = 128). A larger k constrains more directions, so the runs move along the trade-off; we therefore compare each run with the k = 64 sweep at the same code loss:
kcode lossΔL_Ak = 64 at same code lossdifference
160.811+0.62+0.56+0.06
320.831+0.47+0.48−0.01
640.857+0.37+0.370
1280.895+0.31+0.32−0.01

At matched code loss, k = 16 forgets 0.06 more than k = 64, whereas k = 32 and 128 stay within about 0.01 of the k = 64 curve (single seed). The held-out energy share of the output projections (layers 2/11/20) rises from 8–26% at k = 32 to 26–52% at k = 128 with no gain relative to the k = 64 curve, so energy capture in these projections alone does not predict regularization performance here. Revised: §3.3, App. C.3 (Table 6), App. A.

AuthorsOpenReview ↗

Response to Reviewer k9Kf (2/2): why the cancellation emerges (W4); novelty, large rotations and the scale penalty (W2, W3, W6)

Summary. W4: the cancellation forms during training and is realized as a token-selective insensitivity; why it forms remains open. W2: what is new is what nGPT's coordinates reveal, not the Fisher principle. W3: far from θ_A the local analysis identifies which directions cause forgetting, not how much. W6: the scale penalty helps once suppressing rotations exposes cancelled scale damage.

W4. Why the cancellation emerges. We can now say when it develops and how it is realized at the endpoints, though not why: it develops during training and is realized as a token-selective insensitivity of the adapted model. At the code endpoint, rotations alone cause K_w = 2.43 nats of KL damage and scales alone K_s = 1.88, but together only 2.36, because the interaction I_end = −1.95 offsets almost all of the scale damage.

The cancellation develops during training. We audited 24 checkpoints (iterations 0–4500) of each unregularized run (code shown):

iteration50010001500200030004500
median rotation25°34°38°41°43°43°
K_w1.822.492.542.602.492.43
K_s0.160.500.921.311.781.88
I / K_s−1.25−1.21−1.08−1.08−1.05−1.04

From iteration 1000 on, the rotation barely grows while K_s nearly quadruples; the negative interaction grows with it, keeping the total nearly constant (2.39 → 2.36). Medicine shows the same pattern (I_end/K_s = −1.00); in math the scale damage is small (K_s = 0.21, I_end = −0.28). Unnormalized GPT also shows a negative interaction between output-row norms and all other parameters, but its norm channel carries only 5–11% of the deviation on code and medicine.

At the endpoints, the cancellation appears as a token-selective insensitivity to the learned output-scale change. With all other parameters at θ_A, it substantially changes the predictions; with all other parameters at their adapted values, it moves them by only 2–3% as much. This is not a general insensitivity to scale: the same changes permuted across tokens keep 77–89% of their effect, and the original change transplanted onto a rotation-penalized model keeps 89%. The insensitivity thus depends on which tokens receive which scale changes and is largely absent when rotations are constrained (details: reply to aQrc, Q1).

These experiments do not explain why optimization produces this coordination or which changes drive the others; in particular, the s_z swap changes all other parameters together, so it does not isolate rotations as the cause. Nor is it simple reciprocal scaling. Revised: §5.3, App. C.7 (Fig. 10, Tables 14–16), App. A.

W2. What is new beyond Fisher-based methods. The new part is what nGPT's coordinates make measurable, not the Fisher principle: "changes along Fisher-sensitive directions cause more forgetting" is closely related to EWC-type principles, and we do not claim it as new. What is new: directions found sensitive at θ_A stay damaging far beyond the local regime, which the local principle alone does not guarantee; scale damage follows its own law, the spread E_S of per-token logit reweighting; and the channels nearly cancel on code and medicine (W4), so suppressing rotations exposes the scale damage the interaction had been cancelling, which is when a scale penalty matters (W6). We do not claim superiority over a properly row-balanced EWC baseline, which reaches a comparable trade-off (reply to aQrc, Q2). Revised: §1, §2.

W3. Local approximation vs. large rotations. At the endpoints (~40°) the local analysis explains which directions cause forgetting, not how much. Re-measured with float32 attention (the small-angle excess in Fig. 5a is sensitive to the bfloat16 rounding of attention inputs), the quadratic model is accurate to within 5% up to 2° for sensitive directions and up to about 5° for random/orthogonal ones, so it does not predict endpoint damage; but θ_A-sensitive directions remain about 7× more damaging than random there (Fig. 4c), and the held-out layer-wise probe finds the anisotropy in every layer (reply to WfWA, 2/3). Revised: §4.2, App. C.1.

W6. When the scale penalty helps. It helps when the rotation penalty exposes scale damage that the interaction was cancelling. On medicine, the rotation-only endpoint leaves an all-scale K_s = 0.50 with I_end ≈ 0 (§5.3), so that damage counts in full, and the scale term removes it (Fig. 3, Table 1; ΔL_A 0.87 → 0.55; code 0.37 → 0.27; math 0.14 → 0.13). Math has little scale damage to expose (unregularized K_s = 0.21 vs. 1.88 on code), consistent with its smaller additional benefit. We now present the scale penalty as a probe of this exposure effect, not as a generally necessary component. Revised: §1, §5.3.

AuthorsOpenReview ↗

Follow-up to Reviewer k9Kf: Rebuttal and Additional Evaluations

Thank you again for your constructive review. We have added downstream benchmark evaluations, rank-sensitivity experiments, and further analyses of the cancellation phenomenon, and incorporated the results into the revised manuscript. As the discussion period nears its end, we would greatly appreciate any feedback on whether our responses have addressed your concerns.

Reviewer_WfWAOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies forgetting in nGPT during domain-adaptive continued pretraining. It separates parameter changes into rotations of normalized weights and changes to explicit scales, then uses Fisher-guided probes and channel-specific penalties to test how those two kinds of change affect forgetting. The experiments use a 0.5B-parameter model adapted to code, mathematics, and medicine. My understanding is that the regularizer is mainly used to test and control the proposed geometric account.

Critical strengths

The rotation/scale decomposition gives a coherent way to organize the analysis of forgetting in nGPT. This is consequential because it turns a broad question about parameter change into two interpretable channels that can be probed separately.

Critical weaknesses
  1. The theoretical motivation uses the predictive Fisher, while the implemented rotation penalty uses a substantially modified surrogate: observed-label empirical-Fisher statistics, a shared-basis/K-FAC approximation, rank-64 truncation, and per-matrix normalization. The current evidence does not make clear how much of the result validates the canonical Fisher account rather than this particular surrogate. This matters because the theory-to-intervention connection is central to the paper’s contribution.
  2. The analysis does not sufficiently localize the reported directional effect by layer or matrix group. Figure 6 reports layer-level retained Fisher energy, but I did not find a corresponding layer-wise analysis of how directionally relevant rotations affect forgetting. No code or supplementary archive was available with the submission materials I inspected, which also limits verification of the implementation-dependent claims.
Questions for authors
  1. Can you report a layer-wise or matrix-group analysis of the effect of sensitive-direction rotations on forgetting? In particular, is the headline effect broadly distributed across the model or concentrated in a small number of layers or projections, and how would that change the interpretation of the mechanism?
  2. How much of the intervention result survives each step from the predictive-Fisher objective to the implemented empirical-Fisher, shared-basis, rank-truncated, normalized surrogate? A coefficient sweep for the raw-magnitude variant would help distinguish an inferior geometry from a simple penalty-scale mismatch.
6 comments
AuthorsOpenReview ↗

Response to Reviewer WfWA (1/3): from the predictive Fisher to the implemented surrogate (W1, Q2)

We thank the reviewer for the careful review and for recognizing that the rotation/scale decomposition organizes the analysis into two separately probed channels. We address informativeness separately in (3/3).

Summary. At every approximation step, sensitive directions remain more damaging than random, and ranks 32–128 and raw or normalized magnitudes give the same training trade-off, though rank 64 covers the output projections only partially. We correct Table 3 (a unit error), Figure 6 (in-sample shares) and Figs. 1a/5a (bfloat16 sensitivity); the main training results stand.

Corrections.

  1. Table 3 (raw vs. normalized magnitudes). The gap was a unit error in our raw variant. That variant used the eigenvalues of only one K-FAC factor per matrix: input covariances for row-type matrices but mean-loss output-gradient covariances for the output projections and the embedding, so the latter got relative weights of 1e-10 to 1e-6 and were effectively unpenalized. We rebuilt raw magnitudes from both factors in per-token units; spot checks against directly measured per-row Fisher (three layers, five matrix types) give ratios of 0.52–1.34. The normalized production penalty is unchanged (same basis; weights differ by at most 9e-7). Revised: Table 3, §5.1, App. C.2.
  2. Figure 6 / §3.3 (energy concentration at k = 64). The reported shares overstated concentration: the medians (90% / 85% / 63%) were measured in-sample, on the same tokens used to estimate the subspace. We withdraw "low-dimensional in every matrix type"; held-out numbers are in (b). Revised: Figure 6 (in-sample and held-out), §3.3.
  3. Figs. 1a,5a / App. C.1. The small-angle excess disappears under float32 evaluation, and the validity radius is unchanged; it is sensitive to the bfloat16 rounding of attention inputs in the official implementation. Fig. 1a and the §4.2 curvature ratios now come from the same float32 rerun (coefficient ratios, θ ≤ 0.5°: 7.5×/13×; previously 7.3×/12.4×). Revised: Figs. 1 and 5, §4.2, App. C.1.

Answer to W1 / Q2: what survives each step from the predictive Fisher to the implemented surrogate.

(a) Each approximation step, measured functionally. The estimator and shared-basis steps preserve the sensitive damage. For 15 matrices (layers 2/11/20 × Q, V, attention-out, MLP-up, MLP-down), we built five bases, from per-vector predictive Fisher (the canonical object) to the production basis, and measured the held-out KL of a 5° rotation into each. Geometric means over matrices, relative to the reference:

basissensitive KLsensitive / random
per-vector predictive (reference)1.005.8×
per-vector empirical0.955.6×
shared K-FAC, predictive1.066.2×
shared K-FAC, empirical1.036.0×
production basis1.036.0×

Per matrix, the production basis gives 0.89–1.29, and training with predictive-Fisher statistics gives the same endpoints (reply to aQrc, Q3). Revised: App. C.3, Table 4.

(b) Rank-64 truncation. This is the clearest limitation: rank 64 covers the output projections only partially, yet ranks 32–128 give the same training trade-off (reply to k9Kf, W5). Evaluated on tokens disjoint from those used to estimate it, the top-64 subspace carries a median 42–66% of the per-vector Fisher energy for Q/K/V and MLP-up rows, but only 24% (attention-output) and 14% (MLP-down) for the output projections (in-sample: 63–92%, biased upward by fitting 64 of 1023 tangent dimensions). Functionally, the strongest direction left outside the subspace costs 6–16% of the in-subspace damage for most Q/V matrices (layer-20 query: 38%) and 42–95% for the output projections. Directional sensitivity still holds in every layer and matrix type (3–14× over random; reply 2/3). Revised: App. C.2–C.3, Tables 5–6.

(c) Per-matrix normalization: the coefficient sweep you asked for. With correct two-factor raw magnitudes, raw and normalized penalties fall on the same adaptation–forgetting trade-off at every μ we tested (code, rotation-only; code loss / ΔL_A):

μnormalizedraw (two-factor)
0.10.788 / +0.800.786 / +0.79
0.30.811 / +0.560.808 / +0.57
10.857 / +0.370.852 / +0.38
30.919 / +0.290.913 / +0.29

Rotational damage also matches (K_w at μ = 1: 0.238 vs. 0.230), and a second seed of raw μ = 1 reproduces its point (0.853 / +0.38). Raw only redistributes rotation across matrices (e.g. query 8.0° → 12.5° at μ = 1). So the earlier Table 3 gap was a penalty-scale error, not an inferior geometry, and we withdraw the claim that normalization is needed. In this code sweep, which directions are protected matters more for the trade-off than per-matrix magnitude scaling. Revised: Table 3, App. C.2.

AuthorsOpenReview ↗

Response to Reviewer WfWA (2/3): localization by layer and matrix group (W2, Q1), and code

Summary. The directional effect appears in every layer and matrix type (3.4–13.7× over random) but is graded: strongest in Q/K/V and MLP-up, weakest in the output projections. Code: anonymized repository at the end.

W2/Q1. The directional effect is distributed across the model, but graded. It appears in all 24 layers and every matrix type, strongest in Q/K/V and MLP-up and weakest in the output projections. We ran the matched-angle probe one layer or one matrix type at a time. Every vector in the group is rotated by the same angle toward its top-64 sensitive subspace, its complement, or a random tangent direction, and KL is measured on held-out OpenWebText text.

group (all 24 layers)sensitive / random at 5°sensitive / complement
Q, K, V13.7×60×
MLP up8.5×16×
lm_head5.7×8.5×
attention output4.4×6.0×
MLP down3.6×4.5×
all block matrices at once (0.5° / 2° / 5°)8.1× / 8.3× / 9.6×16.5–19.5×

(The last row rotates every Q/K/V, attention-output, MLP-up and MLP-down vector in all 24 blocks together; wte and lm_head are excluded.)

  • Every layer shows the effect. Sensitive/random per layer ranges from 3.4× (layer 22) to 13.7× (layer 5).
  • By depth, it is strongest in layers 5–9 and weakest in layers 17–22.
  • By matrix type, it is strongest in the matrices that read the residual stream (Q/K/V, MLP up) and weakest in those that write to it (the output projections). The output projections are also where rank 64 captures the least held-out energy (reply 1/3, b).
  • We do not convert per-group KLs into shares of total forgetting, because the groups interact non-additively.

What this changes in the interpretation. The mechanism is not a property of a few projections. It is a model-wide anisotropy whose strength varies with depth and with the role of the matrix. The penalty is therefore best supported where this anisotropy is strongest (Q/K/V, MLP-up) and protects the output projections only partially. Revised: App. C.3 (Tables 8–9), §4.2, App. A.

Code. Anonymized repository: https://anonymous.4open.science/r/ngpt-forgetting-51C1/. It contains the training patch, Fisher/K-FAC estimation, the probes and channel audits, and the scripts for the new experiments in this rebuttal. Its README maps each figure and table of the submitted and the revised paper to an entry-point script and the command that renders it, and lists the corrections given in reply 1/3.

AuthorsOpenReview ↗

Response to Reviewer WfWA (3/3): what the paper finds (informativeness)

Summary. The main finding of our paper is that the magnitude of parameter changes alone does not explain forgetting in nGPT. Instead, forgetting depends on which directions the weights rotate toward, how scale changes distort the output distribution, and how these two effects interact and cancel. We have reorganized the abstract, introduction, related work, and conclusion around these three findings, making them the central contributions rather than presenting the regularization results as the main outcome.

What the analysis reveals.

  • Rotation direction matters more than rotation magnitude. At matched angles, Fisher-sensitive directions cause 7.5× more damage than random ones (§4.2), and remain about 7× more damaging at the endpoints, far beyond the local approximation's validity range (Fig. 4c). Median rotations differ by less than 10% across the three domains, yet forgetting differs 3.7× (Fig. 1b).
  • Scale damage depends on output distortion, not scale drift. Across 280 synthetic displacements spanning eleven orders of magnitude in E_S, damage follows ½E_S over the bulk of the sweep, with departures explained by the exact scale-only theory (Fig. 2a). In actual training, medicine has less scale drift than code (0.42 vs. 0.48) but greater scale damage, consistent with their E_S (Fig. 2b).
  • Rotation and scale effects nearly cancel. On code and medicine, scale changes alone cost K_s = 1.88 nats, yet a negative interaction (I_end = −1.95 / −1.87) almost entirely offsets it. The cancellation develops during training and is realized as a token-selective insensitivity: with all other parameters at their adapted values, the learned output-scale change has only 2–3% of the effect it has at θ_A, while the same change permuted across tokens retains 77–89%. Why optimization produces it remains open (§5.3, App. C.7).

These findings explain why the rotation and scale penalties have asymmetric effects on forgetting (Table 1): a scale-only penalty removes damage that was already cancelled (at most 3.3% change), whereas a rotation penalty removes uncancelled damage (69–84% less forgetting) and exposes the cancelled scale damage. More broadly, they show what the nGPT parameterization makes observable: not merely how much the model changes, but which changes are damaging and which compensate for one another.

The local Fisher principle and nGPT's rotation/scale parameterization are established ideas. Our contribution is to establish the behavior of sensitive directions beyond the local regime, identify what governs scale-induced output distortion, and characterize the large cancellation between the two channels. Revised: Abstract, §1, §2, §6.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.