- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work theoretically characterizes the ability of pre-trained models to learn from new fine-tuning data, and their performance on pre-trained data after fine-tuning. They first establish fundamental limits for linear models, and then extend their analysis to family of models that have gated correction under gradient flow algorithms. The authors provide numerical experiments to support their theory.
Critical strengths
S1) For a simple linear model, the authors provide fundamental limits of the MMSE.
S2) The authors have analyzed the DiSeL method under realistic algorithms like gradient flow and have established the bounds on the pre-training and fine-tuning data.
Critical weaknesses
W1) The theory of adaptation, and retention seems to very closely related to paradigms in continual learning such as plasticity, and stability. Despite the relatedness the authors did not sufficiently compare their theory with existing bounds in continual learning literature.
W2) The authors consider a very particular class of non-linear predictors, that is, gated corrected to analyze their theory. However, in practice commonly non-linear feed-forward networks, and transformers are models of interest. The current draft seems to lack discussion of scope of their work.
W3) The authors provide motivation for gated corrected non-liner model using Theorem 2. However, in section 5 the authors did not provide guarantees whether the single-neuron form converges to the likelihood ratio.
Questions for authors
Q1) Does Theorem 1 recover Proposition 1? If so, what is the slackness and if not, why is it the case?
Q2) Can the authors provide how does equation (10) compare with standard total variation?
Q3) Can the authors clarify why Theorem 2 has merit in its own regard as it would appear to be restating equation (7), and (8) ?
Q4) In line 250-251, substituting equation (DISeL) into (4) does not give equation (17). Are the authors modifying the objective in section 5?
Q5) What is the distribution of fine-tuning data in Proposition 2? As of the current state, it appears to be applicable for any distribution with i.i.d. access.
Q6) The title, introduction, and abstract motivate the problem of adaption and retention in a very general sense. However, their theory is applied to model (DiSeL) which is typically not a standard model used in practice. Therefore, I would ask the authors to reconsider their writing and appropriated clarify the exact model for which their theory and insights apply to.
Q7) In line 2207-2208, the authors compute that their experimental setup requires for theory to hold true. However, their Figure 1 only sweep the up-to 8 which is not in the regime where theoretical results hold. Therefore, making theory Theorem 3 not verifiable with the experiments in the main draft.
Q8) The theory and experiments fixes , and . Are their theoretical conclusions to changes in these parameters in permissible range? Sensitivity analysis of these parameters could benefit the draft.
Q9) Can authors provide more concrete comparison with existing theoretical works in continual learning?
Q10) Does the single-neuron form indeed converges to the likelihood ratio?
Update: The authors have addressed all my concerns, I have changed my evaluation to (3/4) Weak Acceptance.
3 comments
Thank you for addressing all of my concerns so thoroughly. I am satisfied with the authors’ responses and, as a result, am willing to increase my rating of the paper.
We are glad that we addressed your concerns and will incorporate the remaining revisions discussed above into the final version.
Thank you for the careful review.
Existing theory [R1]-[R4] studies how task similarity, order, sample size, model size, and regularization affect forgetting in linear or linearized models.
Unlike the shared-predictor setting in [R1], we allow the populations to require different predictions at the same input. We bound the unavoidable error even when both populations are known and nonlinear corrections are allowed. Our bounds concern input overlap where predictions must differ, rather than feature-subspace overlap as in [R4]. We also analyze nonlinear gate training directly, showing that under our stated conditions, FT-only training achieves adaptation while preserving retention throughout training. We have added these references and comparisons to the main paper's Related Work section.
Thank you for this suggestion. We chose DISeL because it is a low-rank method specifically designed to reduce forgetting without PT samples. It also has useful mathematical properties: its sigmoid gate is smooth and bounded, and its form is motivated by the MMSE correction. These properties let us study how training can apply a correction on FT inputs while limiting changes on PT inputs. Our training analysis considers the single-neuron form with a fixed correction matrix, not end-to-end transformer training. The linear and MMSE loss bounds apply more broadly within our regression model. We will make this motivation and scope clearer in the final version.
You are right that Theorem 2 rewrites Eq. 7,8. Our intended emphasis was the explicit Gaussian form in Corollary 1. We will incorporate the sigmoid identity into Equation (8) and present the current Corollary 1 as Theorem 2. Theorem 3 does not claim convergence to the likelihood ratio. With unequal Gaussian covariances, that ratio's logarithm is generally quadratic, while the learned score is linear. Moreover, regularized FT-only training does not minimize the total loss that defines MMSE. The guarantee is instead that training reaches small FT loss while preserving small PT loss, without needing to recover the optimal population-assignment rule.
They optimize over different classes. Proposition 1 gives the exact minimum among linear corrections. Theorem 1 bounds the minimum among all corrections including non-linear ones. Disjoint populations can have zero MMSE loss while the best linear correction still has positive loss. Both results are needed for different purposes: Proposition 1 identifies the limitation of linear updates, while Theorem 1 identifies the fundamental limit faced by every method.
Standard TV compares input distributions without considering their required predictions. Our weighting by q(x)=∥Mx∥2 distinguishes harmless overlap from overlap that forces prediction error. It recovers standard TV for constant positive q, but neither is always larger. Our contribution is to turn this distinction, together with the average correction size, into upper and lower bounds on the best achievable total loss for arbitrary densities. This provides a benchmark for every method within our model, not just DISeL, and explains why input overlap alone does not determine the difficulty of adaptation and retention.
We are sorry for the unclear wording. We substitute DISeL into the FT loss, not the total loss, and use a regularized version for training. Our text incorrectly suggests that substitution alone gives this training objective. We will correct it to make this distinction clear.
Yes, the FT distribution can be arbitrary. The proof uses only the observed training objective and its decrease.
Figure 1 tests whether adaptation and retention remain achievable beyond the theorem's sufficient conditions. Appendix D separately tests settings that satisfy those conditions, with all checked trajectory and loss bounds holding at every recorded point. We will make this check more visible and clarify the different purposes of these experiments.
Theorem 3 allows any fixed M with Eft[q]>0, any B≥10, and every λ in its stated interval, provided the other conditions hold. Changing M changes the loss bounds and regularization interval. Increasing B improves the loss bounds but requires greater separation and can delay adaptation. We will add a sensitivity study varying B and λ under the theorem's conditions, with no PT-based tuning. The added Figures 4 and 5 also examine learning the correction matrix rather than keeping it fixed.
If we managed to address your concerns, we appreciate if you consider increasing your score.
[R1] How catastrophic can catastrophic forgetting be in linear regression?
[R2] Theory on Forgetting and Generalization of Continual Learning.
[R3] Fixed Design Analysis of Regularization-Based Continual Learning.
[R4] A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix.