acceptodds
Under review as a conference paper at ICLR 2027

Understanding and Mitigating Forgetting in LLM Full Fine-Tuning at the Parameter Level

Abstract

Full fine-tuning enables large language models to acquire new knowledge but can cause them to forget previously mastered knowledge. Existing gradient-projection methods constrain overall update directions without explicitly identifying which parameter updates contribute to forgetting. We decompose gradient similarity into parameter-wise contributions to distinguish Conflicting Parameters, whose updates increase the loss on mastered examples to first order, from Collaborative Parameters, whose updates do not. Based on this analysis, we propose Collaborative Parameter Learning (CPL), which uses a gradient from reference examples to block conflicting updates while retaining collaborative ones. Experiments span five models and six datasets across three modalities. Across held-out evaluations, CPL compares favorably with existing forgetting-mitigation baselines, achieving a strong balance between accuracy and knowledge retention. Further analyses suggest that CPL’s strong performance is likely related to the recovery of previously mastered answers and greater preservation of upper-layer representation structure.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Models, Full Fine-Tuning, Forgetting, Parameter-Level Gradient Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bnJaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies forgetting in full fine-tuning of LLMs, where the model learns new knowledge but loses knowledge. The authors split the inner product between the gradient of mastered examples and the actual Adam update direction into per-parameter terms. Based on this, they propose CPL, which blocks the updates of conflicting parameters and keeps the updates of collaborative parameters during training. Experiments show that CPL achieves the highest accuracy and a relatively low forgetting rate on four text datasets with Qwen 1.5B.

Critical strengths
  1. The idea is simple and easy to implement. The method only compares the signs of the reference gradient and the Adam update at each coordinate and applies a mask, with almost no change to the training pipeline.
  2. The analysis is based on the actual Adam update direction rather than the raw gradient, which better matches how the model parameters really change.
Critical weaknesses
  1. The key steps of the method are a Taylor expansion, splitting the inner product into per-coordinate products, and masking coordinates with conflicting signs. These steps are fairly direct. The paper should explain more clearly what CPL offers that existing methods based on gradient sign agreement, coordinate-level gradient selection, gradient masking, or conflict resolution do not.

  2. The link between loss change and catastrophic forgetting is not well supported. An increase in cross-entropy loss does not always turn a correct answer into a wrong one, and a decrease in average loss can hide forgetting on some examples. So loss change cannot be directly treated as forgetting or retention. The authors should provide theoretical or empirical evidence that this loss-based criterion can reliably explain and predict actual forgetting.

  3. The method uses all training examples from the same dataset that the model originally answered correctly to protect old knowledge. In practice, such labels are usually not available, so the results may not transfer to broader real-world settings.

  4. The reference set (M_train) and the set used to measure forgetting (M_test) come from the same dataset and the same distribution. As a result, the reference gradient is naturally close to the true gradient, which gives the method its key assumption for free. In practice, forgetting on other domains and on general abilities is of greater concern, but the paper does not evaluate forgetting across domains or on general abilities.

  5. The forgetting rate of CPL is still 33.9% on MedQA and 18.0% on MMLU, while NTK-CL reaches 6.4% and 7.8%. CPL mainly obtains higher accuracy within the accuracy–forgetting trade-off rather than clearly reducing forgetting.

  6. The paper claims to cover five models, but the full comparison of CPL is only done on text tasks with Qwen 1.5B. The multimodal experiments compare against only one baseline, OGD.

Questions for authors

See above

3 comments

Dear Reviewer:

Thank you for your constructive feedback. We have conducted additional comparisons and analyses to clarify CPL's relationship to existing methods, examine the connection between loss and actual forgetting, and broaden the evaluation.

Question 1: The key steps of the method are a Taylor expansion, splitting the inner product into per-coordinate products, and masking coordinates with conflicting signs. These steps are fairly direct. The paper should explain more clearly what CPL offers that existing methods based on gradient sign agreement, coordinate-level gradient selection, gradient masking, or conflict resolution do not.

Response 1: We compare CPL with Cautious (Liang et al., ICLR 2026), a representative coordinate-wise sign-agreement optimizer. Cautious selects updates based on sign agreement between the current fine-tuning gradient and the proposed Adam update, whereas CPL uses gradients from previously mastered examples to evaluate the first-order effect of each coordinate update on the reference loss. This enables CPL to identify and suppress updates that may interfere with previously acquired knowledge while preserving beneficial updates. Cautious, however, lacks such a reference-based mechanism to explicitly assess the impact of coordinate updates on previously acquired knowledge.

To evaluate this distinction, we compared CPL with Cautious using the same Adam backbone. As shown in Rebuttal Table 1, CPL achieves higher accuracy and lower forgetting than Cautious across all four datasets, demonstrating the effectiveness of mastered-reference-guided coordinate selection in balancing new knowledge acquisition and previous knowledge retention.

Additional comparisons with Cautious are reported in Appendix E. Section 3.2 states: “The effects of learning rate and coordinate selection are examined in Appendix E.”

Rebuttal Table 1. Comparison between Cautious and CPL on Qwen2.5-1.5B (Accuracy (%) / Forgetting (%)).

MethodMMLUMedQAARC-CCSQA
Cautious56.8 / 28.141.8 / 48.177.3 / 9.875.7 / 14.6
CPL61.0 / 18.045.1 / 33.979.1 / 6.379.7 / 7.7

[1] Liang et al. Cautious Optimizers: Improving Training with One Line of Code. ICLR, 2026.

Question 2: The link between loss change and catastrophic forgetting is not well supported. An increase in cross-entropy loss does not always turn a correct answer into a wrong one, and a decrease in average loss can hide forgetting on some examples. So loss change cannot be directly treated as forgetting or retention. The authors should provide theoretical or empirical evidence that this loss-based criterion can reliably explain and predict actual forgetting.

Response 2: We apologize for the confusion caused by our original description. Accuracy depends on discrete correctness decisions and does not provide useful gradients for direct optimization. Following existing approaches, including A-GEM (Chaudhry et al., ICLR 2019), EWC (Kirkpatrick et al., PNAS 2017), and Learning without Forgetting (Li and Hoiem, ECCV 2016), we use cross-entropy loss as a differentiable surrogate to guide optimization. We agree that an increase in loss does not necessarily turn a correct answer into an incorrect one, and that a decrease in average loss does not guarantee the preservation of every previously correct answer.

To examine whether loss changes can identify actual forgetting, we used per-example cross-entropy changes as scores for distinguishing previously correct examples that became incorrect after fine-tuning from those that remained correct. As shown in Rebuttal Table 2, the high AUC values across all four datasets support a strong empirical association between per-example loss changes and actual forgetting.

We have added these results to Appendix G and included the following statement in Section 4.2: “Appendix G examines the association between loss changes and actual forgetting.” We have also revised Section 3.1 to clarify the distinction: “The Adam step increases the mastered-set loss, and this loss increase is used as a loss-based surrogate for forgetting.”

Rebuttal Table 2. AUC of loss-change-based forgetting prediction.

DatasetAUC
MMLU0.9996
MedQA0.9968
ARC-C0.9443
CSQA0.9658

Question 3: The method uses all training examples from the same dataset that the model originally answered correctly to protect old knowledge. In practice, such labels are usually not available, so the results may not transfer to broader real-world settings.

Response 3: We investigated reference-label dependence by replacing the reference targets with answers generated by DeepSeek-V3.1. We retained all original reference questions, including teacher mistakes, without filtering for agreement with gold labels.

As shown in Rebuttal Table 3, although using automatically generated reference labels leads to some performance degradation, the changes in both accuracy and forgetting remain relatively small across all four datasets. These results suggest that CPL is reasonably robust to the use of automatically generated reference labels.

Furthermore, as shown in Table 4, reducing the reference set size by 50% leads to only minor changes in training performance. This further supports the robustness of CPL to reference set.

The reference-label comparison is included in Appendix F. Section 4.4.2 states: “Reference-set composition and label sources are investigated in Appendix F.”

Rebuttal Table 3. CPL performance with different reference-label sources (Accuracy (%) / Forgetting (%)).

Reference labelsMMLUMedQAARC-CCSQA
DeepSeek label59.13 / 19.3145.08 / 33.0979.02 / 6.5679.21 / 7.99
Gold label61.0 / 18.045.1 / 33.979.1 / 6.379.7 / 7.7

Question 4: The reference set (M_train) and the set used to measure forgetting (M_test) come from the same dataset and the same distribution. As a result, the reference gradient is naturally close to the true gradient, which gives the method its key assumption for free. In practice, forgetting on other domains and on general abilities is of greater concern, but the paper does not evaluate forgetting across domains or on general abilities.

Response 4: To evaluate whether CPL can preserve previously acquired knowledge beyond the reference distribution, we conducted an additional cross-dataset evaluation under distribution shift.

Specifically, we used MedQA for adaptation and reference construction, while evaluating retention on MedMCQA, which was excluded from training and reference-gradient computation.

As shown in Rebuttal Table 4, CPL achieves the highest accuracy (48.50%) among all compared methods. This provides evidence that CPL can preserve knowledge beyond its reference dataset and generalize to an out-of-dataset evaluation setting.

The cross-dataset retention results are now included in Appendix J. Section 4.4.1 states: “Cross-dataset retention is evaluated in Appendix J.”

Rebuttal Table 4. Cross-dataset retention evaluation on MedMCQA after adaptation and reference construction on MedQA (Accuracy (%) and Forgetting (%)).

MethodMedMCQA Acc ↑MedMCQA FT ↓
BLF42.1536.23
O-LoRA33.6059.36
A-GEM43.6534.47
GPM42.1536.12
EWC43.4036.89
OGD43.1534.91
OPGD41.1540.31
NTK-CL45.506.72
Smart Replay46.2023.68
CPL48.5025.22

Question 5: The forgetting rate of CPL is still 33.9% on MedQA and 18.0% on MMLU, while NTK-CL reaches 6.4% and 7.8%. CPL mainly obtains higher accuracy within the accuracy–forgetting trade-off rather than clearly reducing forgetting.

Response 5: We agree with the reviewer that CPL does not achieve the lowest forgetting on every dataset. Its advantage lies in achieving a more favorable trade-off between knowledge acquisition and retention. Although NTK-CL achieves lower forgetting on MMLU and MedQA, it also exhibits substantially lower learning performance, indicating that it acquires considerably less new knowledge. Consequently, NTK-CL achieves lower overall accuracy than CPL on both datasets. These results highlight the advantage of CPL in balancing knowledge acquisition and retention rather than minimizing forgetting alone.

Section 4.4.1 emphasizes this balance: “These results suggest that CPL achieves a favorable trade-off between learning and retention.”

Rebuttal Table 5. Comparison of accuracy, learning, and forgetting (Accuracy (%) / Learning (%) / Forgetting (%)).

DatasetMethodAccuracy ↑Learning ↑Forgetting ↓
MMLUNTK-CL58.211.17.8
MMLUCPL61.032.118.0
MedQANTK-CL44.95.76.4
MedQACPL45.128.233.9
ARC-CNTK-CL69.511.82.7
ARC-CCPL79.148.86.3
CSQANTK-CL69.423.010.1
CSQACPL79.751.27.7

Question 6: The paper claims to cover five models, but the full comparison of CPL is only done on text tasks with Qwen 1.5B. The multimodal experiments compare against only one baseline, OGD.

Response 6: We agree with the reviewer that our original claim regarding evaluation across five models was too strong, given the limited scope of the comprehensive baseline comparisons.

To further strengthen the multimodal evaluation, we conducted additional comparisons with multiple forgetting-mitigation baselines on MMBench. As shown in Rebuttal Table 6, CPL achieves the highest accuracy and the lowest forgetting among all compared methods, demonstrating its effectiveness in multimodal fine-tuning.

We fully acknowledge the reviewer's concern and have removed the claims regarding comprehensive evaluation across five models from the revised manuscript to ensure that our claims accurately reflect the experimental scope.

The additional multimodal comparisons are reported in Appendix H. Section 4.4.1 states: “Appendix H extends the multimodal comparison to additional baselines.”

Rebuttal Table 6. Multimodal comparison on MMBench (Accuracy (%) / Forgetting (%)).

MethodMMBench Accuracy ↑MMBench Forgetting ↓
BLF93.54.2
O-LoRA93.65.3
A-GEM94.13.5
GPM93.83.9
EWC93.63.9
OGD74.812.7
OPGD93.73.9
NTK-CL79.81.3
Cautious93.45.6
CPL97.70.2

We sincerely appreciate your valuable feedback, which has helped us improve our manuscript. We will incorporate the corresponding details into the revised manuscript and look forward to your further comments.

Reviewer_rq1EOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

Brief Summary: The paper tackles the task of avoiding forgetting in LLMs/VLMs when fine-tuning. Basically, suppose the model can initially do 5/100 tasks in the dataset correctly already. But if naively fine-tuned its possible that the model gets say 80/100 but forgets the 5 original tasks that it already knew. To achieve this the authors propose CPL (Collaborative Parameter Learning) which only blocks conflicting Adam gradient updates. The authors first motivate the reason for doing CPL showing that the difference in "top"/"bottom" aka conflicting information and collaborative information can be quite drastic on forgetting rate going ranging from 5 to 60 points. Then authors show across multiple image/text datasets the proposed CPL method achieves sota against competitive baselines in accuracy and forgetting.

Critical strengths

Pros:

  1. This is overall a fascinating and interesting problem on how to avoid forgetting. The authors motivate the problem very well through their experiments (Section 4.3). This is very timely given industry might move to fine-tuning open-source models for their own datasets, and should avoid forgetting known information in base models.

  2. The proposed CPL method is intuitive, and also shows sota performance compared to strong baselines on multiple image and text datasets.

  3. The authors do careful ablation of forgetting over epochs and showing that OGD (orthogonal gradients) can be better at earlier epochs, CPL significantly outperfoms with more epochs (Fig2).

Critical weaknesses

Cons:

  1. My main concern is that the relative section is very thing (only 2 paragraphs). Moreover, the main idea of masking out gradients is very similar to Cautious Optimizers [Ref1] which is using Adams second moment to do the masking instead of using "mastered gradient" which is derived from the dataset. Authors should show a direct comparison with Cautious Optimizers in the tables, it is unclear how important having the mastered gradient is if Adam momentum can also give similar numbers.

  2. It is unclear what happens if base model itself has learned incorrect things, or perhaps some factual information is outdated.

  3. Even though the updates prevent conflicting changes, the forgetting rate is still quite high for instance ~34% on MedQA (Table 3). Why is this the case? Is it because the Mastered Gradients are computed from an older update?

  4. (Minor) It would be great to see if this affects subsequent RL or RLHF training? Does it show proportional increase or no change?

  5. (Minor) Given that CPL is blocking many updates, it would be good to add a baseline which randomly blocks a subset of updates for comparison to see if the forgetting improvements are due to CPL or just amount of updates.

======

[Ref1]: Liang, Kaizhao, Lizhang Chen, Bo Liu, and Qiang Liu. "Cautious optimizers: Improving training with one line of code." In International Conference on Learning Representations, vol. 2026, pp. 106538-106563. 2026.

Questions for authors

see weakness section

1 comment

Question 4: (Minor) It would be great to see if this affects subsequent RL or RLHF training? Does it show proportional increase or no change?

Response 4: To leave more room for improvement during subsequent GRPO training, we first fine-tuned Qwen 1.5B on MedQA for three epochs using either ordinary FT or CPL. We then applied GRPO to both models for two epochs (144 steps). As shown in Rebuttal Table 4, both initializations benefit from GRPO. After two GRPO epochs, CPL maintains higher accuracy (48.00% vs. 44.70%) and lower forgetting (25.81% vs. 40.27%) than ordinary FT. These results show that CPL’s advantage persists through subsequent GRPO training.

The subsequent GRPO results are included in Appendix I, and Section 4.4.1 now states: “The effect of subsequent reinforcement learning is studied in Appendix I.”

Rebuttal Table 4. Subsequent GRPO on MedQA with Qwen 1.5B.

MethodAccuracy (%)Forgetting (%)
Ordinary FT (3 epoch)42.1446.56
Ordinary FT (3 epoch) + GRPO (1 epoch)43.5242.85
Ordinary FT (3 epoch) + GRPO (2 epoch)44.7040.27
CPL (3 epoch)46.4630.66
CPL (3 epoch) + GRPO (1 epoch)47.9027.71
CPL (3 epoch) + GRPO (2 epoch)48.0025.81

Question 5: (Minor) Given that CPL is blocking many updates, it would be good to add a baseline which randomly blocks a subset of updates for comparison to see if the forgetting improvements are due to CPL or just amount of updates.

Response 5: We evaluated a random-mask control that, at each optimizer step, retains exactly the number of coordinates that CPL would retain at the random control’s current state, but selects their positions uniformly at random.

As shown in Rebuttal Table 5, CPL achieves higher accuracy and lower forgetting on all four datasets. This indicates that CPL’s coordinate-selection criterion contributes beyond simply reducing the number of updated coordinates.

We have added the count-matched random-mask comparison to Appendix E. Section 3.2 states: “The effects of learning rate and coordinate selection are examined in Appendix E.”

Rebuttal Table 5. Comparison with count-matched random masking on Qwen 1.5B. Each cell reports accuracy (%) / forgetting (%).

MethodMMLUMedQAARC-CCSQA
Random mask56.08 / 28.8343.39 / 41.4175.42 / 10.0475.20 / 15.00
CPL59.83 / 18.2045.78 / 31.3479.54 / 4.6380.12 / 7.58

We sincerely appreciate your constructive feedback and will incorporate the corresponding revisions into the updated manuscript. We look forward to your further comments.

Reviewer_Sj1LOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies forgetting of initially correct examples when an LLM is fully fine-tuned on initially incorrect ones. It splits the first-order change of the mastered-set loss along Adam's update into per-coordinate terms, calling coordinates with a negative term conflicting and the rest collaborative. CPL then zeroes the Adam coordinates that conflict with a buffer gradient computed on mastered training examples. The method is compared with nine baselines on Qwen2.5-1.5B over four multiple-choice text datasets, and with OGD on one image and one audio task.

Critical strengths
  • Simple rule. The mask acts directly on Adam's update, needs only the sign of the reference gradient, and uses less memory than projection methods.
  • Careful protocol. All reference-based methods share the same buffer, held-out mastered examples never enter training, and CPL reaches the highest mean accuracy on all four text datasets.
Critical weaknesses

1. Missing reference rows. The nine-baseline comparison uses only Qwen 1.5B, and Table 3 has no row for the model before fine-tuning or for ordinary fine-tuning. Reading the starting accuracy from Fig. 6:

MMLUMedQA
Before fine-tuning (Fig. 6)≈ 57.5≈ 43.3
CPL (Table 3(a))61.045.1
Baselines at or below the starting point6 of 95 of 9

The net gain of CPL is thus about 3.5 and 1.8 points, and since each baseline uses a single hyperparameter setting without a validation split (Appendix B.3), the ranking is difficult to interpret.

2. Unclear source of the gain. The mask in Eq. 14 has the form of the cautious optimizer rule [1] with the buffer gradient in place of the current gradient, and about half of the coordinates are conflicting at initialization (Table 2), so CPL zeroes a large share of each early update. Because freezing half of the parameters [2] or updating only the largest-momentum coordinates [3] already reduces forgetting, the gain is not yet attributed to the collaborative/conflicting partition; Table 4 varies only the buffer size and refresh interval.

Typos. L468 Fig. 4 axis label "FT" → "Ordinary" (FT already denotes the forgetting rate); L480 "representations.Our" → "representations. Our"; L516 "try arc, the ai2" → "Try ARC, the AI2"; L591 "Commonsenseqa" → "CommonsenseQA".

Questions for authors
  1. Please state which epoch Table 3(a) reports and how the baseline hyperparameters were selected, and reconcile the OGD forgetting rate in Table 3(a) (26.2 on MMLU) with Fig. 2, where the OGD curve stays above about 28.
  2. A random mask matched to CPL's masked fraction, ordinary fine-tuning with a learning-rate sweep, and a cautious-style mask built from the current gradient [1] should be reported, preferably also on Qwen 7B or LLaMA 8B.

References

[1] Cautious Optimizers: Improving Training with One Line of Code. ICLR 2026.
[2] HFT: Half Fine-Tuning for Large Language Models. ACL 2025.
[3] MoFO: Momentum-Filtered Optimizer for Mitigating Forgetting in LLM Fine-Tuning. TMLR 2025.

2 comments

Dear Reviewer,

We sincerely appreciate your careful review and valuable suggestions. We have revised our manuscript in response to your comments and detail the changes point by point below.

Question 1: Please state which epoch Table 3(a) reports and how the baseline hyperparameters were selected, and reconcile the OGD forgetting rate in Table 3(a) (26.2 on MMLU) with Fig. 2, where the OGD curve stays above about 28.

Response 1: Table 3(a) reports the fixed epoch-10 results. The baseline hyperparameters were set to fixed values based on the original papers, as documented in Appendix B.3.

To provide a more thorough comparison, we introduced a validation set to tune baselines that are sensitive to hyperparameter choices across all four text datasets. Specifically, we searched over {500, 5000, 50000, 500000} for EWC’s regularization coefficient, {2, 4, 8, 16} for O-LoRA’s rank, and {0.02, 0.05, 0.1, 0.2} for OPGD’s perturbation radius. The hyperparameter settings were selected based on validation performance.

Despite hyperparameter tuning, these baselines still exhibit a clear performance gap compared with CPL across all four datasets in terms of both accuracy and forgetting. Moreover, CPL does not require setting regularization coefficients or similar additional constants, which reduces the tuning burden and provides an additional practical advantage.

We have added these hyperparameter tuning results to Appendix D. Section 4.1 now states: “Baseline hyperparameter sensitivity is examined in Appendix D.”

Moreover, we have corrected Figure 2 accordingly, and the conclusion remains unchanged: CPL consistently recovers more previously forgotten questions than OGD and ordinary FT.

Rebuttal Table 1. Hyperparameter tuning results for EWC’s regularization coefficient. Each cell reports accuracy (%) / forgetting (%) on Qwen 1.5B.

EWC coefficientMMLUMedQAARC-CCSQA
50054.98 / 30.1941.29 / 50.7976.19 / 10.2377.39 / 13.63
500055.39 / 29.5342.27 / 47.0176.19 / 9.4678.72 / 10.87
5000056.50 / 26.6342.27 / 46.4076.96 / 7.9278.34 / 11.12
50000022.69 / 93.0923.67 / 93.6477.73 / 8.4975.92 / 13.53

Rebuttal Table 2. Hyperparameter tuning results for O-LoRA’s adapter rank. Each cell reports accuracy (%) / forgetting (%) on Qwen 1.5B.

O-LoRA rankMMLUMedQAARC-CCSQA
227.21 / 71.2225.64 / 77.2168.85 / 20.2725.52 / 77.87
443.45 / 50.1232.45 / 68.9670.53 / 17.7625.52 / 77.87
847.18 / 44.7933.73 / 65.3372.46 / 19.6969.83 / 24.05
1647.21 / 43.1832.88 / 66.1373.93 / 12.3670.21 / 21.85

Rebuttal Table 3. Hyperparameter tuning results for OPGD’s perturbation radius. Each cell reports accuracy (%) / forgetting (%) on Qwen 1.5B.

OPGD radiusMMLUMedQAARC-CCSQA
0.0255.34 / 31.7240.64 / 50.1977.09 / 8.8877.11 / 13.77
0.0554.91 / 29.6540.31 / 51.4877.09 / 9.2776.87 / 13.33
0.154.58 / 31.9741.19 / 50.8777.09 / 9.6576.91 / 13.63
0.255.34 / 30.5640.77 / 50.8776.45 / 9.2776.29 / 13.82

Question 2: A random mask matched to CPL's masked fraction, ordinary fine-tuning with a learning-rate sweep, and a cautious-style mask built from the current gradient should be reported, preferably also on Qwen 7B or LLaMA 8B.

Response 2: Thank you for this suggestion. We conducted all three comparisons on Qwen 1.5B and LLaMA 8B across four datasets.

(1) Random masking. We evaluated a random mask that masks the same proportion of parameter coordinates as the CPL. CPL achieves higher accuracy and lower forgetting on every dataset for both models. These results support the importance of CPL’s coordinate-selection criterion beyond simply reducing the number of updated coordinates.

(2) Learning-rate sweep. We swept the learning rate of Ordinary Adam on Qwen 1.5B and LLaMA 8B, as reported in Rebuttal Tables 4 and 5. On Qwen 1.5B, accuracy on MMLU and MedQA remained below the pre-fine-tuning level across all tested learning rates. This suggests that the decline is not simply an artifact of the original learning-rate choice. The prior model-refinement studies that we follow (Jin & Ren, ICML 2024; Zhou et al., NeurIPS 2025) also report a similar phenomenon: the number of forgotten examples can exceed the number of newly learned examples, leading to lower overall accuracy after fine-tuning.

(3) Cautious-style masking. We evaluated a Cautious-style mask based on sign agreement between the current training gradient and momentum (Liang et al., ICLR 2026). CPL achieves higher accuracy and lower forgetting on both models across all four datasets, supporting the effectiveness of our masking strategy relative to current-gradient sign filtering.

These comparisons are now included in Appendix E. Section 3.2 states: “The effects of learning rate and coordinate selection are examined in Appendix E.”

Rebuttal Table 4. Learning-rate sweep and comparison of masking strategies on Qwen 1.5B. Each cell reports Accuracy ↑ / Forgetting ↓ (%).

MethodMMLUMedQAARC-CCSQA
Before fine-tuning57.5 / 0.043.3 / 0.066.7 / 0.069.6 / 0.0
Ordinary Adam (2.5e-7)51.0 / 36.640.1 / 47.675.7 / 9.574.6 / 15.6
Ordinary Adam (5e-7)55.7 / 29.242.8 / 45.276.1 / 8.974.1 / 16.2
Ordinary Adam (1e-6)56.7 / 28.142.6 / 44.777.0 / 9.376.2 / 14.5
Ordinary Adam (2e-6)55.8 / 29.641.3 / 51.477.7 / 9.777.0 / 14.5
Random mask56.1 / 28.843.4 / 41.475.4 / 10.075.2 / 15.0
Cautious55.7 / 28.041.4 / 49.277.1 / 9.376.5 / 14.0
CPL61.0 / 18.045.1 / 33.979.1 / 6.379.7 / 7.7

Rebuttal Table 5. Learning-rate sweep and comparison of masking strategies on LLaMA 8B. Each cell reports Accuracy ↑ / Forgetting ↓ (%).

MethodMMLUMedQAARC-CCSQA
Before fine-tuning60.2 / 0.064.2 / 0.032.4 / 0.041.6 / 0.0
Ordinary Adam (1e-7)56.1 / 38.668.0 / 23.478.2 / 34.171.8 / 39.9
Ordinary Adam (2.5e-7)60.8 / 30.765.9 / 26.079.9 / 29.078.1 / 25.2
Ordinary Adam (5e-7)62.7 / 27.963.4 / 29.680.8 / 25.476.7 / 29.9
Ordinary Adam (1e-6)63.4 / 27.154.9 / 39.877.0 / 38.574.7 / 34.8
Ordinary Adam (2e-6)57.8 / 35.747.2 / 48.275.9 / 40.172.9 / 36.4
Random mask54.9 / 37.466.3 / 24.976.6 / 34.966.8 / 50.0
Cautious62.6 / 23.266.4 / 22.880.7 / 23.473.1 / 32.9
CPL68.8 / 7.573.8 / 10.281.6 / 10.380.8 / 11.5

Question 3: Typos. L468 Fig. 4 axis label "FT" → "Ordinary" (FT already denotes the forgetting rate); L480 "representations.Our" → "representations. Our"; L516 "try arc, the ai2" → "Try ARC, the AI2"; L591 "Commonsenseqa" → "CommonsenseQA".

Response 3: Thank you for pointing out these presentation issues. We will replace “FT” with “Ordinary” in Fig. 4 to avoid ambiguity with the forgetting rate, add the missing space in “representations. Our,” and correct the capitalization to “Try ARC, the AI2” and “CommonsenseQA.”

We sincerely appreciate the time and effort you have devoted to reviewing our work. We will carefully incorporate your valuable suggestions into the revised manuscript and look forward to your further feedback.

References

[1] Xisen Jin et al. What Will My Model Forget? Forecasting Forgotten Examples in Language Model Refinement. ICML, 2024.

[2] Yuxuan Zhou et al. Investigating and Mitigating Catastrophic Forgetting in Medical Knowledge Injection through Internal Knowledge Augmentation Learning. NeurIPS, 2025.

[3] Kaizhao Liang et al. Cautious Optimizers: Improving Training with One Line of Code. ICLR, 2026.

Comments on the paper

1 comment

Dear Reviewer,

We sincerely thank you for your constructive feedback. Below, we clarify the relationship with Cautious Optimizers and present additional comparisons, reference-set diagnostics, and subsequent reinforcement-learning results.

Question 1: My main concern is that the relative section is very thing (only 2 paragraphs). Moreover, the main idea of masking out gradients is very similar to Cautious Optimizers [Ref1] which is using Adams second moment to do the masking instead of using "mastered gradient" which is derived from the dataset. Authors should show a direct comparison with Cautious Optimizers in the tables, it is unclear how important having the mastered gradient is if Adam momentum can also give similar numbers.

Response 1: Thank you for highlighting this connection. Cautious Optimizers constructs its mask from sign agreement between the current training gradient and Adam’s first moment. CPL instead evaluates the proposed update using a reference gradient from previously mastered examples. This enables CPL to explicitly account for the potential impact of new updates on previously acquired knowledge, thereby better mitigating catastrophic forgetting.

We directly compared Cautious and CPL on all four text datasets. As shown in Rebuttal Table 1, CPL achieves higher accuracy and lower forgetting on every dataset, providing a better acquisition-retention trade-off in this setting.

The comparison with Cautious is reported in Appendix E. Section 3.2 now states: “The effects of learning rate and coordinate selection are examined in Appendix E.”

Rebuttal Table 1. Comparison with Cautious on Qwen 1.5B. Each cell reports accuracy (%) / forgetting (%).

MethodMMLUMedQAARC-CCSQA
Cautious55.74 / 27.9641.42 / 49.2177.09 / 9.2776.50 / 13.97
CPL61.0 / 18.045.1 / 33.979.1 / 6.379.7 / 7.7

Question 2: It is unclear what happens if base model itself has learned incorrect things, or perhaps some factual information is outdated.

Response 2: Thank you for raising this point. Our focus is on preserving previously mastered knowledge while learning new information. In our protocol, mastery is determined using the task’s current gold labels, so initially incorrect answers belong to the learning set rather than the protected reference set. When facts change, the reference labels should be updated to reflect the knowledge we intend to preserve.

We have added the following sentence to the Conclusion to discuss this issue: “CPL defines mastered knowledge using current reference labels, so initially incorrect answers remain learning targets and outdated facts require updated labels.”

Question 3: Even though the updates prevent conflicting changes, the forgetting rate is still quite high for instance ~34% on MedQA (Table 3). Why is this the case? Is it because the Mastered Gradients are computed from an older update?

Response 3: We believe that the remaining forgetting on MedQA may be partly due to a mismatch between the mastered training examples used to construct the reference gradient and the initially correct test examples being evaluated.

The mastered training and test subsets are not perfectly aligned: the cosine similarity between their initial gradients is 0.4914. Consequently, protecting the aggregate training-reference loss does not necessarily preserve all initially correct test answers.

To examine this mismatch, we replaced the training-mastered reference set with an oracle reference set containing the initially correct test examples themselves. When the reference set exactly matches the mastered examples being evaluated, forgetting decreases from 31.34% to 0.15%, with only 2 of 1,321 examples forgotten. These results suggest that the mismatch between the reference and evaluation sets contributes to the remaining forgetting.

These reference-set diagnostics are now reported in Appendix F. Section 4.4.2 states: “Reference-set composition and label sources are investigated in Appendix F.”

Rebuttal Table 2. Cosine similarity between the mean CE-gradient representations of the two reference sets at the initial model.

Reference-set comparisonCosine similarity
Train-correct vs. Test-correct0.4914

Rebuttal Table 3. Reference-set comparison on MedQA with Qwen 1.5B.

Reference setAccuracy (%)Forgetting (%)
Train-correct as reference gradient45.7831.34
Test-correct as reference gradient60.840.15

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.