- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies which parts of a LoRA update are needed to keep the mathematical skills learned during fine-tuning. The authors use a shared sparse autoencoder (SAE) to find internal features that change after training. They then split the LoRA update into rank-one components and score them using answer prediction loss and a loss that matches the fine-tuned model’s feature values. The highest-scoring components are kept under a fixed storage budget. Experiments cover addition, subtraction, and simple equations on four models. Using 25% of the adapter storage, the combined score outperforms full LoRA on three models and reduces their loss of GSM8K accuracy. The paper also intervenes on selected features to test their role in later computation, including whether deleting them still causes errors after the correct next token is supplied.
Critical strengths
(1) The main comparison shows a consistent benefit from feature-based scoring. Behavior, SAE-only, and Joint use the same candidate components, storage budget, and allocation rule. SAE-only outperforms Behavior on all four models, and the advantage holds across the tested budgets. This provides useful evidence that feature matching adds information beyond answer prediction loss.
(2) The intervention experiments include meaningful controls. Deleting selected features still harms later computation when the correct next token is supplied, so the damage cannot be explained solely by an incorrect immediate prediction. Restoring carry-related features also affects carry problems much more than matched no-carry problems. These results give causal evidence that some identified features contribute to computation.
Critical weaknesses
(1) The experiments do not establish what SAE specifically contributes. SAE-only outperforming Behavior shows that matching the fine-tuned model’s internal features helps, but it does not show that SAE features—or selecting those that change most—are important. A simpler hidden-state matching objective could potentially provide the same benefit. Without this comparison, the main gain could come from additional supervision from the fine-tuned model rather than the proposed feature analysis. This leaves the paper’s central technical contribution insufficiently supported.
(2) The practical value of the full pipeline remains unclear. The method requires a fully trained LoRA adapter, activation collection, SAE training, and gradient scoring before producing a smaller adapter. The reported storage reduction covers only the adapter. The experiments do not establish whether this process offers enough benefit over directly training a similarly sized adapter or using simpler compression methods to justify the additional work. Beating the full adapter on these tasks is interesting, but it does not by itself establish a compelling reason to use this pipeline.
Questions for authors
(1) The feature-matching objective looks quite similar to directly matching the fine-tuned model’s hidden states. Have the authors tried this simpler baseline? I would like to know whether SAE actually performs better, since it adds another model to train and another set of features to select.
(2) I understand that removing the carry-related features hurts Joint, but how does this explain why Joint works better than Behavior? Does Joint preserve these features better? If the same features are removed from both models, does Joint still have an advantage?
(3) For practical use, why train a rank-64 LoRA and then reduce it to the storage size of rank 16, rather than train rank 16 directly? Have the authors compared the two? I would like to see how much performance this extra process gains.
(4) The method scores components using local gradients, but then removes many components at once. Have the authors checked whether removing high-scoring components actually hurts performance more? Also, does averaging gradients along the path work better than just computing them at one point?
(5) GSM8K seems to be the only benchmark used to check whether existing capabilities are retained. I do not think results on this one benchmark are enough to draw a broader conclusion about reducing interference, even with several models. Have the authors checked whether the same benefit holds on other mathematical tasks?
Dear Reviewer nANG,
We understand that this is a busy period and would like to gently follow up on our response. When you have a chance, we would be grateful to hear whether the additional experiments and clarifications have addressed your concerns, or whether any questions remain. We remain happy to provide further clarification.
Thank you again for your time and consideration!