- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies forgetting in full fine-tuning of LLMs, where the model learns new knowledge but loses knowledge. The authors split the inner product between the gradient of mastered examples and the actual Adam update direction into per-parameter terms. Based on this, they propose CPL, which blocks the updates of conflicting parameters and keeps the updates of collaborative parameters during training. Experiments show that CPL achieves the highest accuracy and a relatively low forgetting rate on four text datasets with Qwen 1.5B.
Critical strengths
- The idea is simple and easy to implement. The method only compares the signs of the reference gradient and the Adam update at each coordinate and applies a mask, with almost no change to the training pipeline.
- The analysis is based on the actual Adam update direction rather than the raw gradient, which better matches how the model parameters really change.
Critical weaknesses
-
The key steps of the method are a Taylor expansion, splitting the inner product into per-coordinate products, and masking coordinates with conflicting signs. These steps are fairly direct. The paper should explain more clearly what CPL offers that existing methods based on gradient sign agreement, coordinate-level gradient selection, gradient masking, or conflict resolution do not.
-
The link between loss change and catastrophic forgetting is not well supported. An increase in cross-entropy loss does not always turn a correct answer into a wrong one, and a decrease in average loss can hide forgetting on some examples. So loss change cannot be directly treated as forgetting or retention. The authors should provide theoretical or empirical evidence that this loss-based criterion can reliably explain and predict actual forgetting.
-
The method uses all training examples from the same dataset that the model originally answered correctly to protect old knowledge. In practice, such labels are usually not available, so the results may not transfer to broader real-world settings.
-
The reference set (M_train) and the set used to measure forgetting (M_test) come from the same dataset and the same distribution. As a result, the reference gradient is naturally close to the true gradient, which gives the method its key assumption for free. In practice, forgetting on other domains and on general abilities is of greater concern, but the paper does not evaluate forgetting across domains or on general abilities.
-
The forgetting rate of CPL is still 33.9% on MedQA and 18.0% on MMLU, while NTK-CL reaches 6.4% and 7.8%. CPL mainly obtains higher accuracy within the accuracy–forgetting trade-off rather than clearly reducing forgetting.
-
The paper claims to cover five models, but the full comparison of CPL is only done on text tasks with Qwen 1.5B. The multimodal experiments compare against only one baseline, OGD.
Questions for authors
See above
3 comments
Question 3: The method uses all training examples from the same dataset that the model originally answered correctly to protect old knowledge. In practice, such labels are usually not available, so the results may not transfer to broader real-world settings.
Response 3: We investigated reference-label dependence by replacing the reference targets with answers generated by DeepSeek-V3.1. We retained all original reference questions, including teacher mistakes, without filtering for agreement with gold labels.
As shown in Rebuttal Table 3, although using automatically generated reference labels leads to some performance degradation, the changes in both accuracy and forgetting remain relatively small across all four datasets. These results suggest that CPL is reasonably robust to the use of automatically generated reference labels.
Furthermore, as shown in Table 4, reducing the reference set size by 50% leads to only minor changes in training performance. This further supports the robustness of CPL to reference set.
The reference-label comparison is included in Appendix F. Section 4.4.2 states: “Reference-set composition and label sources are investigated in Appendix F.”
Rebuttal Table 3. CPL performance with different reference-label sources (Accuracy (%) / Forgetting (%)).
| Reference labels | MMLU | MedQA | ARC-C | CSQA |
|---|---|---|---|---|
| DeepSeek label | 59.13 / 19.31 | 45.08 / 33.09 | 79.02 / 6.56 | 79.21 / 7.99 |
| Gold label | 61.0 / 18.0 | 45.1 / 33.9 | 79.1 / 6.3 | 79.7 / 7.7 |
Question 4: The reference set (M_train) and the set used to measure forgetting (M_test) come from the same dataset and the same distribution. As a result, the reference gradient is naturally close to the true gradient, which gives the method its key assumption for free. In practice, forgetting on other domains and on general abilities is of greater concern, but the paper does not evaluate forgetting across domains or on general abilities.
Response 4: To evaluate whether CPL can preserve previously acquired knowledge beyond the reference distribution, we conducted an additional cross-dataset evaluation under distribution shift.
Specifically, we used MedQA for adaptation and reference construction, while evaluating retention on MedMCQA, which was excluded from training and reference-gradient computation.
As shown in Rebuttal Table 4, CPL achieves the highest accuracy (48.50%) among all compared methods. This provides evidence that CPL can preserve knowledge beyond its reference dataset and generalize to an out-of-dataset evaluation setting.
The cross-dataset retention results are now included in Appendix J. Section 4.4.1 states: “Cross-dataset retention is evaluated in Appendix J.”
Rebuttal Table 4. Cross-dataset retention evaluation on MedMCQA after adaptation and reference construction on MedQA (Accuracy (%) and Forgetting (%)).
| Method | MedMCQA Acc ↑ | MedMCQA FT ↓ |
|---|---|---|
| BLF | 42.15 | 36.23 |
| O-LoRA | 33.60 | 59.36 |
| A-GEM | 43.65 | 34.47 |
| GPM | 42.15 | 36.12 |
| EWC | 43.40 | 36.89 |
| OGD | 43.15 | 34.91 |
| OPGD | 41.15 | 40.31 |
| NTK-CL | 45.50 | 6.72 |
| Smart Replay | 46.20 | 23.68 |
| CPL | 48.50 | 25.22 |
Question 5: The forgetting rate of CPL is still 33.9% on MedQA and 18.0% on MMLU, while NTK-CL reaches 6.4% and 7.8%. CPL mainly obtains higher accuracy within the accuracy–forgetting trade-off rather than clearly reducing forgetting.
Response 5: We agree with the reviewer that CPL does not achieve the lowest forgetting on every dataset. Its advantage lies in achieving a more favorable trade-off between knowledge acquisition and retention. Although NTK-CL achieves lower forgetting on MMLU and MedQA, it also exhibits substantially lower learning performance, indicating that it acquires considerably less new knowledge. Consequently, NTK-CL achieves lower overall accuracy than CPL on both datasets. These results highlight the advantage of CPL in balancing knowledge acquisition and retention rather than minimizing forgetting alone.
Section 4.4.1 emphasizes this balance: “These results suggest that CPL achieves a favorable trade-off between learning and retention.”
Rebuttal Table 5. Comparison of accuracy, learning, and forgetting (Accuracy (%) / Learning (%) / Forgetting (%)).
| Dataset | Method | Accuracy ↑ | Learning ↑ | Forgetting ↓ |
|---|---|---|---|---|
| MMLU | NTK-CL | 58.2 | 11.1 | 7.8 |
| MMLU | CPL | 61.0 | 32.1 | 18.0 |
| MedQA | NTK-CL | 44.9 | 5.7 | 6.4 |
| MedQA | CPL | 45.1 | 28.2 | 33.9 |
| ARC-C | NTK-CL | 69.5 | 11.8 | 2.7 |
| ARC-C | CPL | 79.1 | 48.8 | 6.3 |
| CSQA | NTK-CL | 69.4 | 23.0 | 10.1 |
| CSQA | CPL | 79.7 | 51.2 | 7.7 |
Question 6: The paper claims to cover five models, but the full comparison of CPL is only done on text tasks with Qwen 1.5B. The multimodal experiments compare against only one baseline, OGD.
Response 6: We agree with the reviewer that our original claim regarding evaluation across five models was too strong, given the limited scope of the comprehensive baseline comparisons.
To further strengthen the multimodal evaluation, we conducted additional comparisons with multiple forgetting-mitigation baselines on MMBench. As shown in Rebuttal Table 6, CPL achieves the highest accuracy and the lowest forgetting among all compared methods, demonstrating its effectiveness in multimodal fine-tuning.
We fully acknowledge the reviewer's concern and have removed the claims regarding comprehensive evaluation across five models from the revised manuscript to ensure that our claims accurately reflect the experimental scope.
The additional multimodal comparisons are reported in Appendix H. Section 4.4.1 states: “Appendix H extends the multimodal comparison to additional baselines.”
Rebuttal Table 6. Multimodal comparison on MMBench (Accuracy (%) / Forgetting (%)).
| Method | MMBench Accuracy ↑ | MMBench Forgetting ↓ |
|---|---|---|
| BLF | 93.5 | 4.2 |
| O-LoRA | 93.6 | 5.3 |
| A-GEM | 94.1 | 3.5 |
| GPM | 93.8 | 3.9 |
| EWC | 93.6 | 3.9 |
| OGD | 74.8 | 12.7 |
| OPGD | 93.7 | 3.9 |
| NTK-CL | 79.8 | 1.3 |
| Cautious | 93.4 | 5.6 |
| CPL | 97.7 | 0.2 |
We sincerely appreciate your valuable feedback, which has helped us improve our manuscript. We will incorporate the corresponding details into the revised manuscript and look forward to your further comments.
Dear Reviewer:
Thank you for your constructive feedback. We have conducted additional comparisons and analyses to clarify CPL's relationship to existing methods, examine the connection between loss and actual forgetting, and broaden the evaluation.
Question 1: The key steps of the method are a Taylor expansion, splitting the inner product into per-coordinate products, and masking coordinates with conflicting signs. These steps are fairly direct. The paper should explain more clearly what CPL offers that existing methods based on gradient sign agreement, coordinate-level gradient selection, gradient masking, or conflict resolution do not.
Response 1: We compare CPL with Cautious (Liang et al., ICLR 2026), a representative coordinate-wise sign-agreement optimizer. Cautious selects updates based on sign agreement between the current fine-tuning gradient and the proposed Adam update, whereas CPL uses gradients from previously mastered examples to evaluate the first-order effect of each coordinate update on the reference loss. This enables CPL to identify and suppress updates that may interfere with previously acquired knowledge while preserving beneficial updates. Cautious, however, lacks such a reference-based mechanism to explicitly assess the impact of coordinate updates on previously acquired knowledge.
To evaluate this distinction, we compared CPL with Cautious using the same Adam backbone. As shown in Rebuttal Table 1, CPL achieves higher accuracy and lower forgetting than Cautious across all four datasets, demonstrating the effectiveness of mastered-reference-guided coordinate selection in balancing new knowledge acquisition and previous knowledge retention.
Additional comparisons with Cautious are reported in Appendix E. Section 3.2 states: “The effects of learning rate and coordinate selection are examined in Appendix E.”
Rebuttal Table 1. Comparison between Cautious and CPL on Qwen2.5-1.5B (Accuracy (%) / Forgetting (%)).
[1] Liang et al. Cautious Optimizers: Improving Training with One Line of Code. ICLR, 2026.
Question 2: The link between loss change and catastrophic forgetting is not well supported. An increase in cross-entropy loss does not always turn a correct answer into a wrong one, and a decrease in average loss can hide forgetting on some examples. So loss change cannot be directly treated as forgetting or retention. The authors should provide theoretical or empirical evidence that this loss-based criterion can reliably explain and predict actual forgetting.
Response 2: We apologize for the confusion caused by our original description. Accuracy depends on discrete correctness decisions and does not provide useful gradients for direct optimization. Following existing approaches, including A-GEM (Chaudhry et al., ICLR 2019), EWC (Kirkpatrick et al., PNAS 2017), and Learning without Forgetting (Li and Hoiem, ECCV 2016), we use cross-entropy loss as a differentiable surrogate to guide optimization. We agree that an increase in loss does not necessarily turn a correct answer into an incorrect one, and that a decrease in average loss does not guarantee the preservation of every previously correct answer.
To examine whether loss changes can identify actual forgetting, we used per-example cross-entropy changes as scores for distinguishing previously correct examples that became incorrect after fine-tuning from those that remained correct. As shown in Rebuttal Table 2, the high AUC values across all four datasets support a strong empirical association between per-example loss changes and actual forgetting.
We have added these results to Appendix G and included the following statement in Section 4.2: “Appendix G examines the association between loss changes and actual forgetting.” We have also revised Section 3.1 to clarify the distinction: “The Adam step increases the mastered-set loss, and this loss increase is used as a loss-based surrogate for forgetting.”
Rebuttal Table 2. AUC of loss-change-based forgetting prediction.