- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how gradient-kernel evolution affects forgetting during full fine-tuning of LLMs. It separates changes in kernel scale from changes in response direction and develops a decay-calibrated NTK approximation for correct-answer log-odds drift. An adjoint-based optimizer then uses the surrogate to select epoch-wise replay weights. The experiments find that decay calibration improves prediction magnitude, while replay schedules obtained without calibration achieve comparable downstream performance in the evaluated control setting.
Critical strengths
- The three questions in the introduction connect coherently to kernel measurements, response prediction, and replay control. The main contribution is the empirical distinction between accurate prediction magnitudes and useful optimization guidance.
- Tables 2 and 5 evaluate nine models on three datasets. Section 4.1 separates parameter fitting from held-out evaluation and reports variability across three data partitions. Table 1 supports the direction–magnitude distinction: calibration substantially reduces the parallel residual while leaving directional agreement largely unchanged.
- Table 3 reports the highest mean accuracy for PG-NoDe on all three datasets while showing that O-LoRA achieves lower forgetting. Appendix F uses a shared replay buffer, and Table 7 accounts for kernel-construction overhead.
Critical weaknesses
- Appendix F describes OPGD through output-gradient projection without explaining whether the cited method’s GAM-based gradient-norm penalty is retained. This leaves an implementation ambiguity rather than establishing an incorrect implementation. Selected values and tuning procedures for other baseline hyperparameters, including EWC and DER++, are also unspecified.
- The replay-control conclusion has narrower evidence than the prediction analysis: Table 3 evaluates only Qwen 1.5B, and Appendix B uses SGD throughout. The finding that magnitude calibration adds little control benefit should therefore remain explicitly scoped to this setting.
- The empirical remainder in Table 1 is estimated as observed log-odds change minus the prediction, as stated at lines 184–185. It can therefore include errors from the first-order dynamics, residual linearization, and continuous-time approximation, in addition to kernel evolution beyond shared scaling. Its attribution specifically to kernel evolution needs qualification.
Questions for authors
- Does OPGD include GAM and its gradient-norm penalty? Please provide baseline configurations.
- How should the empirical remainder be interpreted given the preceding approximation errors?
- Are additional replay-control results already available? Otherwise, please maintain the stated scope of the control conclusion.
5 comments
Question 2: How should the empirical remainder be interpreted given the preceding approximation errors?
Response 2: We appreciate this important observation and agree that the empirical remainder should not be exclusively attributed to kernel evolution. To examine how the preceding approximations affect the empirical remainder, we conducted additional experiments on MMLU, MedQA, and CSQA.
Replacing the linearized CE residual with its nonlinear form reduces the parallel remainder on all three datasets, with little change in the orthogonal remainder or directional agreement. Moreover, residual linearization produces a smaller change in the parallel component than that observed for kernel evolution in Table 1 of the manuscript.
Rebuttal Table 4. Effect of CE residual linearization.
| Dataset | Predictor | ||||
|---|---|---|---|---|---|
| MMLU | Linear | 237.62 | 114.86 | 19.79 | 0.9873 |
| MMLU | Nonlinear | 202.06 | 79.29 | 19.73 | 0.9873 |
| MedQA | Linear | 119.35 | 72.56 | 8.11 | 0.9853 |
| MedQA | Nonlinear | 102.84 | 56.08 | 8.24 | 0.9848 |
| CSQA | Linear | 383.65 | 212.00 | 33.34 | 0.9817 |
| CSQA | Nonlinear | 286.86 | 115.11 | 32.84 | 0.9822 |
Replacing the continuous solution with discrete recurrence has a small effect on both the parallel and orthogonal remainders.
Rebuttal Table 5. Continuous versus discrete dynamics.
| Dataset | Predictor | ||||
|---|---|---|---|---|---|
| MMLU | Continuous | 237.62 | 114.86 | 19.79 | 0.9873 |
| MMLU | Discrete | 241.22 | 118.46 | 19.85 | 0.9872 |
| MedQA | Continuous | 119.35 | 72.56 | 8.11 | 0.9853 |
| MedQA | Discrete | 121.55 | 74.76 | 8.08 | 0.9854 |
| CSQA | Continuous | 383.65 | 212.00 | 33.34 | 0.9817 |
| CSQA | Discrete | 389.70 | 218.07 | 33.47 | 0.9815 |
Explicitly incorporating the full higher-order Taylor expansion is impractical in our setting. We therefore examine the remaining discrepancy after incorporating kernel decay, nonlinear residuals, and discrete recurrence. These changes substantially reduce the parallel remainder, while the orthogonal remainder remains largely unchanged. This persistent orthogonal remainder may partly reflect higher-order Taylor terms omitted by the first-order approximation.
Rebuttal Table 6. Jointly incorporating kernel decay, nonlinear residuals, and discrete recurrence.
| Dataset | Predictor | ||||
|---|---|---|---|---|---|
| MMLU | No decay + Linear + Continuous | 237.62 | 114.86 | 19.79 | 0.9873 |
| MMLU | Decay + Nonlinear + Discrete | 123.03 | 0.33 | 20.24 | 0.9867 |
| MedQA | No decay + Linear + Continuous | 119.35 | 72.56 | 8.11 | 0.9853 |
| MedQA | Decay + Nonlinear + Discrete | 67.64 | 20.81 | 7.86 | 0.9862 |
| CSQA | No decay + Linear + Continuous | 383.65 | 212.00 | 33.34 | 0.9817 |
| CSQA | Decay + Nonlinear + Discrete | 154.03 | 17.39 | 34.54 | 0.9803 |
We fully agree that the empirical remainder cannot be attributed exclusively to kernel evolution. Accordingly, in Section 3.2, we replaced “two remainders capturing the effects of kernel evolution beyond this shared scaling” with “two remainder components describing the deviation from this prediction.” We have also added the corresponding analyses to Appendix L.
Question 3: Are additional replay-control results available? Otherwise, please maintain the stated scope of the control conclusion?
Response 3: We have conducted three additional sets of replay-control experiments beyond the original Qwen 1.5B setting, including experiments with AdamW, a 5× larger learning rate, and an additional evaluation on LLaMA 8B.
As shown in Rebuttal Tables 4-6, PG-NoDe and PG-De maintain strong performance across all three additional settings. In particular, one of the two PG variants achieves the highest accuracy across all nine dataset-setting combinations. These additional experiments provide further evidence that our proposed replay-control approach generalizes beyond the original Qwen 1.5B model, optimizer, and training configuration.
We have incorporated these additional replay-control results into Tables 14-16 of the revised manuscript. We have also added the following sentence at the end of Section 4.4: “Additional replay-control comparisons with AdamW, a 5× larger learning rate, and LLaMA 8B support the same conclusion (Appendix I).”
Meanwhile, we agree that this conclusion should be carefully scoped. Accordingly, we have revised the manuscript to emphasize replay control as a methodological contribution motivated by our kernel analysis, rather than a general finding. Specifically, we have removed statements suggesting that “magnitude calibration offers limited additional benefit” from the Abstract, Introduction, and other related sections.
Rebuttal Table 7. Method comparison under AdamW (Accuracy (%) / Forgetting (%)).
| Method | MMLU | MedQA | CSQA |
|---|---|---|---|
| GPM | 44.66 / 46.24 | 26.23 / 75.93 | 56.93 / 36.94 |
| OPGD | 42.35 / 51.82 | 30.55 / 70.02 | 22.07 / 88.38 |
| EWC | 44.23 / 48.55 | 29.44 / 68.13 | 22.65 / 88.06 |
| O-LoRA | 57.48 / 0.91 | 43.06 / 1.36 | 75.95 / 0.23 |
| BLF | 43.69 / 49.30 | 28.62 / 70.10 | 21.86 / 89.37 |
| ER | 53.10 / 33.04 | 33.69 / 60.56 | 45.40 / 55.63 |
| ER+DWA | 53.13 / 33.00 | 33.79 / 60.86 | 46.53 / 54.19 |
| DER++ | 54.91 / 29.94 | 45.68 / 28.24 | 48.96 / 52.03 |
| Smart Replay | 56.10 / 25.81 | 46.07 / 27.02 | 57.48 / 37.61 |
| PG-NoDe | 59.57 / 17.33 | 47.35 / 22.18 | 78.00 / 6.98 |
| PG-De | 58.98 / 20.43 | 47.25 / 23.77 | 78.07 / 6.98 |
Rebuttal Table 8. Method comparison with a 5× larger learning rate (Accuracy (%) / Forgetting (%)).
| Method | MMLU | MedQA | CSQA |
|---|---|---|---|
| GPM | 48.32 / 37.76 | 31.73 / 59.65 | 68.39 / 19.19 |
| OPGD | 46.09 / 43.01 | 31.20 / 61.09 | 67.43 / 21.26 |
| EWC | 49.96 / 35.19 | 33.30 / 55.56 | 66.92 / 21.40 |
| O-LoRA | 57.65 / 2.52 | 43.39 / 3.94 | 75.47 / 1.26 |
| BLF | 48.11 / 39.29 | 31.76 / 59.27 | 66.47 / 21.98 |
| ER | 53.86 / 27.30 | 35.66 / 48.90 | 71.40 / 15.00 |
| ER+DWA | 53.84 / 27.30 | 35.99 / 48.22 | 71.33 / 15.00 |
| DER++ | 55.84 / 22.75 | 41.00 / 32.93 | 72.25 / 13.38 |
| Smart Replay | 56.29 / 20.80 | 42.37 / 21.73 | 71.40 / 14.95 |
| PG-NoDe | 58.88 / 13.28 | 44.53 / 10.83 | 77.45 / 6.49 |
| PG-De | 58.81 / 13.73 | 44.99 / 11.88 | 75.95 / 7.48 |
Rebuttal Table 9. Method comparison on LLaMA 8B (Accuracy (%) / Forgetting (%)).
| Method | MMLU | MedQA | CSQA |
|---|---|---|---|
| GPM | 53.06 / 36.68 | 68.63 / 20.04 | 67.57 / 24.31 |
| OPGD | 54.17 / 35.18 | 67.13 / 22.95 | 65.96 / 26.48 |
| EWC | 53.08 / 37.04 | 60.71 / 27.38 | 65.96 / 25.85 |
| O-LoRA | 60.98 / 0.83 | 65.29 / 1.33 | 76.12 / 0.27 |
| BLF | 52.56 / 37.75 | 59.00 / 29.53 | 65.38 / 26.52 |
| ER | 60.50 / 25.10 | 69.84 / 19.68 | 70.00 / 21.02 |
| ER+DWA | 60.52 / 24.98 | 69.78 / 19.68 | 69.93 / 21.02 |
| DER++ | 61.21 / 23.83 | 69.81 / 19.07 | 70.37 / 20.25 |
| Smart Replay | 61.69 / 22.92 | 72.27 / 15.55 | 71.88 / 18.27 |
| PG-NoDe | 67.23 / 15.06 | 71.94 / 15.40 | 78.28 / 9.11 |
| PG-De | 65.11 / 18.14 | 72.63 / 13.77 | 77.35 / 10.10 |
We sincerely thank you again for highlighting these important issues. Your suggestions have helped us improve the clarity of our implementation details, the rigor of our empirical interpretation, and the scope of our experimental conclusions. We sincerely look forward to your further feedback.
Dear Reviewer:
We sincerely thank you for your constructive feedback and insightful comments. In response, we have clarified the baseline implementation details, refined our interpretation of the empirical remainder, and conducted additional replay-control experiments.
Question 1: Does OPGD include GAM and its gradient-norm penalty? Please provide baseline configurations.
Response 1: Thank you for pointing out the ambiguity in our description of OPGD. Our OPGD implementation includes GAM and its gradient-norm penalty. Following the original OPGD paper (Zheng et al., 2026), we set the penalty strength to α=0.5 and the perturbation radius to b=0.05. For EWC (Kirkpatrick et al., 2016), we use λ=5000. For DER++ (Buzzega et al., 2020), we use (α,β)=(0.5,1). These method-specific configurations were selected with reference to the respective original papers.
To examine hyperparameter sensitivity, we conducted additional experiments. For OPGD, we varied b∈0.02,0.05,0.2. For EWC, we evaluated λ∈500,5000,50000. For DER++, we evaluated (α,β)∈(0.5,1),(1,0.5),(0.75,0.75). Even the best configurations remain substantially behind PG-NoDe and PG-De in the overall accuracy–forgetting trade-off.
We have revised Appendix F to explicitly document GAM in OPGD and the hyperparameter values used for baselines, and added the sensitivity results in Tables 8-10.
Rebuttal Table 1. OPGD hyperparameter sensitivity (Accuracy (%) / Forgetting (%)).
Rebuttal Table 2. EWC hyperparameter sensitivity (Accuracy (%) / Forgetting (%)).
Rebuttal Table 3. DER++ hyperparameter sensitivity (Accuracy (%) / Forgetting (%)).
References
[1] Guodong Zheng, Peng Wang, Shengchao Hu, Quan Zheng, and Li Shen. Understanding the dynamics of forgetting and generalization in continual learning via the neural tangent kernel. In International Conference on Learning Representations, 2026.
[2] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796, 2016.
[3] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. In Advances in Neural Information Processing Systems, volume 33, 2020.