- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies catastrophic forgetting during replay-free fine-tuning of language models. The focus is on projection-based methods such as Adam-NSCL/GPM. The authors argue that using a single layer-wise input subspace shared across all rows is too coarse for ReLU MLP layers because most unit-token pairs are inactive, so constraining every row with the same subspace wastes budget on closed units. They derive an exact decomposition of per-layer output change into in-cell, open->closed, and closed->open terms, present a gate-aware per-unit projector that preserves each unit’s pre-activation on selected tokens where that unit is active, and prove a budget-separation result comparing per-unit versus shared subspaces. They also apply the same decomposition to pruning repair, arguing that local curvature-based repair objectives are blind to closed->open errors. Empirically, on OPT-1.3B the proposed projector reduces old-probe NLL forgetting relative to Adam-NSCL at matched average budget. On OPT-6.7B it roughly matches forgetting while learning slightly more. Selector ablations support the claim that the open/closed partition is the key variable.
Critical strengths
The underlying method is simple and sensible. The neuron-specific projector is a natural relaxation of existing null-space projection methods. The exact output-change decomposition is correct, and the linear algebraic arguments explaining the reduction in constrained dimensions are plausible.
The comparisons between open, random, sign-blind, anti-gate, within-open random, and first-order selection are particularly informative. They provide evidence that the activation partition itself is responsible for much of the improvement.
There are nontrivial empirical improvements in the target setting. On OPT-1.3B, the proposed method reduces forgetting relative to Adam-NSCL. The results on OPT-6.7B indicate that the method can improve learning while maintaining comparable forgetting.
The paper provides detailed information about datasets, hyperparameters, training configurations, and evaluation procedures. The supplementary material contains implementation details and some code, with a complete release promised, ensuring reproducability of the results.
Critical weaknesses
I think the overall idea is good and could potentially improve fine-tuning of pre-trained models. However, the paper as it is now is not suited to a top-tier ML conference. I am willing to raise my score if the two largest problems I see (overly elaborate theory and lacking experimental evaluation) are addressed.
The tropical geometry framework is unnecessary and substantially obscures the contribution. The central idea follows directly from the ReLU Jacobian: since inactive neurons have zero derivatives, the null-space constraint can be constructed separately for each neuron using only its active calibration inputs. The projector then follows from standard linear algebra. I cannot identify any result where tropical geometry, Newton polytopes, or zonotope duality is actually needed to derive the algorithm, prove some properties, or explain the experiments or results. In particular, Proposition 2 appears disconnected from the subsequent methodological development. The tropical machinery makes an otherwise simple and intuitive contribution unnecessarily difficult to understand. I strongly believe this framework should be removed from the title, abstract, and main technical development. The paper should be rewritten around neuron-specific null-space projection and ReLU activation patterns.
The theoretical presentation overcomplicates relatively elementary observations. Theorem 5 is essentially a case distinction over ReLU activation states. Theorem 9 follows from comparing the ranks of neuron-specific input matrices with that of their union, while Proposition 10 follows from a standard eigendecomposition argument. These are useful observations, but they do not require these elaborate formulations. The paper repeatedly presents straightforward mathematical properties through unnecessarily complicated constructions. The abstract is particularly problematic, combining specialized terminology, theorem claims, numerical results, and a second application before the reader has any clear understanding of the actual method. Overall, the writing makes the contribution harder to evaluate than necessary.
The experimental scope is substantially narrower than the framing suggests. The main fine-tuning experiments focus on ReLU-based OPT models with updates restricted to fc1. The method is not convincingly evaluated on contemporary language models with SwiGLU or GELU activation functions, nor in general full-model fine-tuning. In particular, the OPT-6.7B experiments update only the fc1 matrices of the final four transformer blocks. The broader applicability to modern language models remains unclear.
The theoretical guarantees do not directly extend to the experimental setting. The preservation results apply to individual layers evaluated at fixed calibration inputs. However, the experiments jointly update multiple layers, so their input representations drift during fine-tuning. Moreover, the projector does not prevent previously inactive neurons from becoming active. The authors acknowledge these limitations, but their practical consequences are not sufficiently investigated.
The budget matching is not entirely convincing. Comparisons with Adam-NSCL use the mean number of selected tokens, rather than the actual rank of the neuron-specific constraints. Furthermore, the proposed operator is not an exact projector: due to the ridge regularization, it attenuates directions rather than eliminating them. Consequently, equal nominal dimensions do not necessarily correspond to equal effective constraints.
The selection strategy appears more complicated than necessary. The experiments show that randomly selecting tokens from within a neuron's active set performs equivalently to selecting those with the largest activations. This suggests that the activation partition is important, but the additional ranking by activation magnitude is not. The method could potentially be simplified further, and this finding deserves a more prominent discussion.
The pruning-repair application is disconnected from the main contribution and takes up space that would be better spent establishing the method's broader applicability. Section 5 introduces a largely separate problem and additional theoretical machinery without substantially strengthening the central contribution on knowledge-preserving fine-tuning. I would strongly prefer the authors to remove this section and instead develop the theory and experiments for modern MLP architectures, particularly SwiGLU. Modern MLPs do not produce exactly zero activations, so extending the neuron-specific projection principle is nontrivial and would constitute a much more meaningful theoretical contribution. Demonstrating that the method remains effective on contemporary language models would also considerably strengthen the empirical claims and practical relevance of the paper.
Questions for authors
How does the method compare against the per-unit PCA projector implied by Proposition 10? This seems like a particularly natural baseline for determining whether the benefits arise from neuron-specific constraints or the proposed token-selection strategy. Relatedly, given the equivalence between random-within-open and activation-magnitude selection, is the latter necessary?
How well does the approach generalize to contemporary architectures and more realistic continual-learning settings? The paper discusses relevance to gated architectures, but the main fine-tuning results concern ReLU OPT fc1 layers. Results on SwiGLU/GELU models, full-model fine-tuning, and multiple sequential adaptation tasks would considerably strengthen the claims.
What are the computational and memory costs compared with Adam-NSCL? Constructing and factorizing separate systems for potentially thousands of neurons is not free.
How much preservation error is caused by representation drift and previously inactive neurons becoming active? The theoretical results distinguish these effects, but a quantitative decomposition during end-to-end fine-tuning would help establish how well the fixed-input analysis explains the observed forgetting.
5 comments
(2/3)
Q1. Per-neuron PCA, token selection and budget matching
Per-neuron PCA baseline (Proposition 10)
At the C=64 budget, we implemented the optimal per-neuron subspace of Proposition 10 by computing principal directions from all active tokens of each neuron. To separate the effect of neuron-specific directions, we compare it with two controls: (i) a bridge control that uses gate-weighted shared directions with matched per-row numerical rank and ridge shrinkage (23.17 directions and 23.02 effective dimensions on average), and (ii) gate-weighted shared projection with . The table gives means over seeds (128 steps, WikiText-103).
| C=64 budget | Per-neuron PCA (Prop. 10) | Bridge: shared directions, matched rank | Gate-weighted shared () |
|---|---|---|---|
| Set A forgetting (↓) | 0.0391 | 0.0435 | 0.0508 |
| Set B forgetting (↓) | 0.0410 | 0.0436 | 0.0528 |
| Original probe forgetting (↓) | 0.0194 | 0.0373 | 0.0506 |
| New-task learning (↑) | 0.6188 | 0.6200 | 0.6239 |
Per-neuron PCA forgets less than the bridge control in 3/3 seeds on Set A and on the probe, and in 2/3 seeds on Set B (seed 1: +0.0010 nats/token), with learning matched within 0.2%. Relative to gate-weighted sharing at , per-neuron PCA forgets less in all 3/3 seeds on all three retention measures.
Relation to the C=256 selector
In OPT-1.3B, the median neuron is active on only 11 calibration tokens, and only 7.59% of neurons reach the C=256 cap. For the other 92.4% of units, our selector constrains every active token. It therefore realizes the per-neuron subspace of Proposition 10 at full rank without an eigendecomposition. At 128 steps, C=256 forgot less than both standard and gate-weighted shared projection () in 3/3 seeds on Sets A and B (Set A: 0.0331 vs. 0.0378 and 0.0373; Set B: 0.0339 vs. 0.0396 and 0.0391), with learning matched within 0.2%. Over 512 steps (LR , chosen on validation text), C=256 forgot 34% less than gate-weighted sharing on the 520 documents (0.0182 vs. 0.0276 nats/token) and learned more, in all 3/3 seeds.
Random selection within the open set
We agree that random tokens from the open set of a neuron perform equivalently to the tokens with the largest activation magnitudes. On the original probe at C=64, random open-token selection changed forgetting by only -0.0015 nats/token (90% CI [-0.0051, +0.0021], well within the TOST equivalence margin). What matters is restricting the constraints to the open set: rank-matched random tokens drawn globally increased forgetting by 0.0317 nats/token in all 6/6 seeds. Section 4.2 states that membership in , not the ranking inside it, governs retention.
Budget matching and effective dimensions
The table reports the average number of selected tokens , the exact numerical rank and the effective ridge dimension .
| Budget setting | Selected | Numerical rank | Effective dimension | Shared budget |
|---|---|---|---|---|
| C=64, WikiText | 23.519 | 23.168 | 23.018 | 24 |
| C=256, WikiText | 47.257 | 46.672 | 46.151 | 47 |
| C=128, Python | 34.421 | 33.959 | 33.657 | 34 |
Since and shared projection removes full orthogonal directions, our projector constrains fewer effective dimensions per row than its shared baseline in every comparison.
Q4. Layer-by-layer representation drift and closed-to-open crossings
To measure how the fixed-input guarantee interacts with representation drift through the whole network, we decomposed the squared post-activation drift in all 24 layers of the step-128 models on the 520 documents as follows: (i) direct drift, from the updated weights on the original inputs, and (ii) input drift, from the updated weights on the drifted inputs that preceding layers actually produce. Values are sums over all 24 layers and all positions, in millions (means over seeds).
| Method | Direct drift | Input drift | Total layer drift | Closed→open crossings (% of total) |
|---|---|---|---|---|
| Unconstrained fine-tuning | 10.459 | 72.110 | 82.722 | 15.305 (18.50%) |
| Our projector (C=64) | 2.643 | 25.848 | 28.846 | 3.853 (13.36%) |
All four quantities decrease in all 3/3 seeds on both document sets. Direct drift falls by 74.7% and total layer drift by 65.1%. Although input drift accounts for 89.6% of the remaining error, it also falls by 64.2%, because each protected upstream layer passes less representation drift downstream. In addition, the closed-to-open term, which our rule leaves unconstrained, is a small share of the total (13.36%) and falls by 75% without an explicit constraint. This confirms the near-wall containment of Proposition 11. Across the 520 documents, C=64 forgot 82.2% less than unconstrained training.
(3/3)
Q2. Architecture diversity, full-model fine-tuning, scale and sequential adaptation
We evaluated our method on recent architectures, under full-parameter fine-tuning, at the 6.7B parameter scale and on a multi-stage sequential benchmark.
| Setting | Comparator | Forgetting, Sets A / B (↓) | New-task learning (↑) | Seeds favoring ours |
|---|---|---|---|---|
| Qwen2.5-0.5B SwiGLU, up projection, 128 steps | Unconstrained | 0.136 / 0.124 vs. 0.249 / 0.233 | 0.401 vs. 0.369 | 3/3 |
| Pythia-1.4B GELU, first MLP matrix, C=256, 128 steps | Unconstrained | 0.073 / 0.062 vs. 0.285 / 0.259 | 0.542 vs. 0.499 | 3/3 |
| OPT-1.3B, all parameters, WikiText, 128 steps | Plain AdamW | 0.174 / 0.177 vs. 0.410 / 0.405 | 0.622 vs. 0.568 | 3/3 |
| OPT-1.3B, all parameters, Python, 128 steps | Plain AdamW | 0.023 / 0.026 vs. 0.124 / 0.126 | 0.210 vs. 0.144 | 3/3 |
| OPT-1.3B, all parameters, WikiText, 512 steps | Shared () | 0.024 / 0.025 vs. 0.030 / 0.034 | 0.688 vs. 0.689 | 3/3 (forgetting) |
- SwiGLU and GELU. On Qwen2.5-0.5B (native SwiGLU with frozen gates), constraining the up-projection rows on the 24 tokens with the largest reduced forgetting by 45.3% / 46.7%. On Pythia-1.4B (GELU), our rule reduced forgetting by 74.3% / 76.1%. Compared with shared projection with the same budget of 175 directions per row, our rule matched retention on the 520 documents (0.0675 vs. 0.0676), forgot less on the original probe (0.0047 vs. 0.0154) and learned more in all 3/3 seeds (0.5420 vs. 0.5234).
- Full-parameter fine-tuning. When all 1.3B parameters are updated (our rule on fc1, AdamW on all other matrices), our method reduced forgetting by 57% on WikiText and by 80% on Python over 128 steps. Over 512 steps, it forgot 22.8% less than shared projection on Sets A and B at matched learning, and it removed 86% of the forgetting incurred by plain AdamW.
- OPT-6.7B scale. We updated fc1 in the last four blocks (C=128, 68.7 directions per row vs. shared ). Forgetting on the original probe was equivalent to Adam-NSCL under TOST (0.0192 vs. 0.0184, within the margin; both remove 85% of unconstrained forgetting), whereas new-task learning was higher in all three seeds (+0.0109 nats/token, paired ). Rank-matched random tokens forgot 0.0098 more in all 3/3 seeds.
- Three-stage continual sequence (WikiText → Python → Turkish Wikipedia). This experiment shows the capacity-plasticity trade-off predicted by Theorem 9. Shared projection removes the same 85 directions from every row in the layer, which blocks learning on the third domain: Turkish NLL worsened by +0.0142 nats/token in all 3/3 seeds. In contrast, our per-unit projector allocates constraints per neuron and preserves plasticity: Turkish NLL decreased by 0.0167 nats/token, a net learning advantage of 0.0309 nats/token (, 3/3 seeds). In exchange for this forward learning capacity, our method kept mean forgetting on the earlier domains below 0.023 nats/token (0.0222 vs. 0.0143).
Q3. Computational and memory footprint
All measurements use an NVIDIA L40S GPU in fp32.
- Construction time. Building the C=256 projector takes 6.3 seconds (9.8 s with cached factors), comparable to the 6.7 seconds of shared projection (), whereas explicit PCA bases require about 13 minutes.
- Throughput and memory. With cached Cholesky factors, training runs at 84% of shared throughput (159.1 vs. 189.9 tokens/second) and peaks at 34.1 GiB of memory because of tensor padding in our current code. With block recomputation, training runs at 52% of shared throughput and peaks at only 7.7 GiB, close to the 7.4 GiB of shared projection. Both modes give identical numerical results.
- Factor storage. The unpadded Cholesky factors take 3.15 GB in fp16 across all 24 layers, roughly the size of the AdamW momentum buffers of fc1 (3.22 GB). The factors exist only during training, so inference has no added parameters and no added latency.
Conclusion
On the two conditions named in the review:
- Presentation. We have rewritten the theory around the direct 3-line Jacobian derivation, tied each formal result to a concrete empirical question, moved tropical geometry and zonotope duality to Appendix A and compressed pruning repair to a single paragraph. We will remove "tropical geometry" from the title in the final revision.
- Empirical evaluation. The evaluation now covers SwiGLU, GELU, full-model fine-tuning over 128 and 512 steps, the OPT-6.7B scale, a 3-stage sequence, 520 independent documents, per-neuron PCA and a quantitative drift analysis.
We would be grateful if the reviewer would consider whether these revisions and new results meet the two conditions stated in the review and update the score accordingly. We are glad to discuss any further point.
Response to Reviewer pdKz
(1/3)
We thank the reviewer for the detailed review and for finding the method sensible, the exact decomposition and the budget separation conceptually sound, and the selector ablations informative.
The review names two conditions for reassessing the submission: (1) a more streamlined theoretical presentation, and (2) a broader empirical evaluation that covers recent architectures, full-model fine-tuning, sequential adaptation and representation drift. We agree with both points. This part describes the restructured theory and the shorter pruning-repair section. Part 2 answers Q1 (PCA), token selection, budget matching and Q4 (drift), and Part 3 answers Q2 (architectures, scale and sequences) and Q3 (cost).
Unless stated otherwise, the experiments fine-tune fc1 of OPT-1.3B on WikiText-103 with paired seeds 0, 1 and 2. Forgetting is the increase in old-text NLL relative to the base model, and learning is the decrease in new-task NLL, both in nats/token (lower forgetting and higher learning are better).
Theoretical restructuring and the role of geometry
We agree that the algorithm follows directly from the local Jacobian of the ReLU layer and should be presented before the tropical geometry. We have reorganized the manuscript accordingly.
Direct Jacobian derivation first
Section 3 of the revision now opens with a three-line derivation for a single neuron that does not use tropical formalism:
This derivation makes the method easy to follow and to implement.
Role of the formal results
Each formal result (numbered as submitted) is now introduced by the role it plays:
The formal tropical hypersurface and zonotope duality constructions (Propositions 2 and 4, Corollary 6) are now in Appendix A. Following the reviewer's recommendation, we will also remove "tropical geometry" from the title in the final revision.
Compressing the post-pruning repair section
We agree with the suggestion and have compressed Section 5 to a single paragraph. The space saved is used for modern MLPs and full-model fine-tuning. We keep this short discussion because pruning repair is the main setting in which closed-to-open crossings govern performance.
One-shot pruning displaces weights by large discrete amounts, which pushes previously inactive pre-activations across the threshold. Closed tokens have zero derivatives at the base weights, so derivative-based local models are blind to this curvature (Lemma 13). On OPT-1.3B at 70% sparsity, the exact gated refit increases the loss by 1.94 nats/token relative to no repair, and pre-activation refitting gives a WikiText-2 perplexity of 153.64 (dense model: 14.62).
Theorem 5 identifies closed-to-open crossings as the cause of the failure of the gated refit, and our convex one-sided penalty penalizes these crossings explicitly. On fixed Wanda masks, keeping the mask alone is preferable at 50% and 60% sparsity (perplexity 19.76 and 29.37). At 70% sparsity, however, one-sided refitting is effective and lowers perplexity from 97.25 to 84.50. Within SparseGPT, refitting fc1 with our one-sided objective lowered perplexity from 30.21 to 28.04 at 60% sparsity, from 79.93 to 57.55 at 70% and from 763.3 to 395.4 at 80%, in all 3/3 seeds. These values are lower than those of an equal refit with the SparseGPT objective (28.81, 77.38 and 738.3).