acceptodds
Under review as a conference paper at ICLR 2027

Theory of Adaptation and Retention

Abstract

Fine-tuning (FT) should help a model learn new tasks without losing capabilities acquired during pre-training (PT). We develop a theoretical framework for understanding when these two goals can be achieved together. Using a linear prediction model, we first consider the favorable setting where both PT and FT data are available. Even in this setting, linear updates, including full FT and low-rank adaptation (LoRA), can face an unavoidable tradeoff between adaptation and retention. We then ask what the best possible correction can achieve. We characterize the minimum mean-square error (MMSE) predictor and derive upper and lower bounds on its loss that match up to a universal constant factor for arbitrary population densities. These bounds explain how population overlap and the required correction determine the minimum loss achievable by any method. We next turn to a setting closer to practice where only FT data are available. We study a practical input-dependent FT method motivated by the MMSE predictor, whose default behavior favors preserving pre-trained knowledge. With the correction held fixed, we prove that suitable separation and training conditions in a Gaussian model allow this method to achieve small FT loss while keeping forgetting small throughout training. These results explain how learning when to apply a correction can support adaptation and retention without access to PT examples.

Primary area:
learning theory
Keywords:
Fine-tuning, Retention, Forgetting

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_x4NROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work theoretically characterizes the ability of pre-trained models to learn from new fine-tuning data, and their performance on pre-trained data after fine-tuning. They first establish fundamental limits for linear models, and then extend their analysis to family of models that have gated correction under gradient flow algorithms. The authors provide numerical experiments to support their theory.

Critical strengths

S1) For a simple linear model, the authors provide fundamental limits of the MMSE.

S2) The authors have analyzed the DiSeL method under realistic algorithms like gradient flow and have established the bounds on the pre-training and fine-tuning data.

Critical weaknesses

W1) The theory of adaptation, and retention seems to very closely related to paradigms in continual learning such as plasticity, and stability. Despite the relatedness the authors did not sufficiently compare their theory with existing bounds in continual learning literature.

W2) The authors consider a very particular class of non-linear predictors, that is, gated corrected to analyze their theory. However, in practice commonly non-linear feed-forward networks, and transformers are models of interest. The current draft seems to lack discussion of scope of their work.

W3) The authors provide motivation for gated corrected non-liner model using Theorem 2. However, in section 5 the authors did not provide guarantees whether the single-neuron form converges to the likelihood ratio.

Questions for authors

Q1) Does Theorem 1 recover Proposition 1? If so, what is the slackness and if not, why is it the case?

Q2) Can the authors provide how does equation (10) compare with standard total variation?

Q3) Can the authors clarify why Theorem 2 has merit in its own regard as it would appear to be restating equation (7), and (8) ?

Q4) In line 250-251, substituting equation (DISeL) into (4) does not give equation (17). Are the authors modifying the objective in section 5?

Q5) What is the distribution of fine-tuning data in Proposition 2? As of the current state, it appears to be applicable for any distribution with i.i.d. access.

Q6) The title, introduction, and abstract motivate the problem of adaption and retention in a very general sense. However, their theory is applied to model (DiSeL) which is typically not a standard model used in practice. Therefore, I would ask the authors to reconsider their writing and appropriated clarify the exact model for which their theory and insights apply to.

Q7) In line 2207-2208, the authors compute that their experimental setup requires ρ≥91.14\rho \geq 91.14 for theory to hold true. However, their Figure 1 only sweep the ρ\rho up-to 8 which is not in the regime where theoretical results hold. Therefore, making theory Theorem 3 not verifiable with the experiments in the main draft.

Q8) The theory and experiments fixes BB, and λ\lambda. Are their theoretical conclusions to changes in these parameters in permissible range? Sensitivity analysis of these parameters could benefit the draft.

Q9) Can authors provide more concrete comparison with existing theoretical works in continual learning?

Q10) Does the single-neuron form indeed converges to the likelihood ratio?

Update: The authors have addressed all my concerns, I have changed my evaluation to (3/4) Weak Acceptance.

3 comments

Thank you for the careful review.

W1, Q9

Existing theory [R1]-[R4] studies how task similarity, order, sample size, model size, and regularization affect forgetting in linear or linearized models.

Unlike the shared-predictor setting in [R1], we allow the populations to require different predictions at the same input. We bound the unavoidable error even when both populations are known and nonlinear corrections are allowed. Our bounds concern input overlap where predictions must differ, rather than feature-subspace overlap as in [R4]. We also analyze nonlinear gate training directly, showing that under our stated conditions, FT-only training achieves adaptation while preserving retention throughout training. We have added these references and comparisons to the main paper's Related Work section.

W2, Q6

Thank you for this suggestion. We chose DISeL because it is a low-rank method specifically designed to reduce forgetting without PT samples. It also has useful mathematical properties: its sigmoid gate is smooth and bounded, and its form is motivated by the MMSE correction. These properties let us study how training can apply a correction on FT inputs while limiting changes on PT inputs. Our training analysis considers the single-neuron form with a fixed correction matrix, not end-to-end transformer training. The linear and MMSE loss bounds apply more broadly within our regression model. We will make this motivation and scope clearer in the final version.

W3, Q3, Q10

You are right that Theorem 2 rewrites Eq. 7,8. Our intended emphasis was the explicit Gaussian form in Corollary 1. We will incorporate the sigmoid identity into Equation (8) and present the current Corollary 1 as Theorem 2. Theorem 3 does not claim convergence to the likelihood ratio. With unequal Gaussian covariances, that ratio's logarithm is generally quadratic, while the learned score is linear. Moreover, regularized FT-only training does not minimize the total loss that defines MMSE. The guarantee is instead that training reaches small FT loss while preserving small PT loss, without needing to recover the optimal population-assignment rule.

Q1

They optimize over different classes. Proposition 1 gives the exact minimum among linear corrections. Theorem 1 bounds the minimum among all corrections including non-linear ones. Disjoint populations can have zero MMSE loss while the best linear correction still has positive loss. Both results are needed for different purposes: Proposition 1 identifies the limitation of linear updates, while Theorem 1 identifies the fundamental limit faced by every method.

Q2

Standard TV compares input distributions without considering their required predictions. Our weighting by q(x)=∥Mx∥2q(\mathbf x)=\|\mathbf M\mathbf x\|^2 distinguishes harmless overlap from overlap that forces prediction error. It recovers standard TV for constant positive qq, but neither is always larger. Our contribution is to turn this distinction, together with the average correction size, into upper and lower bounds on the best achievable total loss for arbitrary densities. This provides a benchmark for every method within our model, not just DISeL, and explains why input overlap alone does not determine the difficulty of adaptation and retention.

Q4

We are sorry for the unclear wording. We substitute DISeL into the FT loss, not the total loss, and use a regularized version for training. Our text incorrectly suggests that substitution alone gives this training objective. We will correct it to make this distinction clear.

Q5

Yes, the FT distribution can be arbitrary. The proof uses only the observed training objective and its decrease.

Q7

Figure 1 tests whether adaptation and retention remain achievable beyond the theorem's sufficient conditions. Appendix D separately tests settings that satisfy those conditions, with all checked trajectory and loss bounds holding at every recorded point. We will make this check more visible and clarify the different purposes of these experiments.

Q8

Theorem 3 allows any fixed M\mathbf M with Eft[q]>0\mathbb E_{\rm ft}[q]>0, any B≥10B\ge10, and every λ\lambda in its stated interval, provided the other conditions hold. Changing M\mathbf M changes the loss bounds and regularization interval. Increasing BB improves the loss bounds but requires greater separation and can delay adaptation. We will add a sensitivity study varying BB and λ\lambda under the theorem's conditions, with no PT-based tuning. The added Figures 4 and 5 also examine learning the correction matrix rather than keeping it fixed.

If we managed to address your concerns, we appreciate if you consider increasing your score.

[R1] How catastrophic can catastrophic forgetting be in linear regression?

[R2] Theory on Forgetting and Generalization of Continual Learning.

[R3] Fixed Design Analysis of Regularization-Based Continual Learning.

[R4] A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix.

Thank you for addressing all of my concerns so thoroughly. I am satisfied with the authors’ responses and, as a result, am willing to increase my rating of the paper.

We are glad that we addressed your concerns and will incorporate the remaining revisions discussed above into the final version.

Reviewer_4EQiOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies adaptation and retention in a two-population regression model. It derives sharp bounds on the optimal prediction error and proves that, with a fixed task correction, regularized gate training can achieve adaptation while controlling forgetting without pre-training samples.

Critical strengths
  • Theorem 3 is the strongest contribution. It controls the entire gradient-flow trajectory and guarantees retention throughout training, without requiring alignment between the mean, covariance, and correction matrix.
  • The weighted-overlap bounds give a clear interpretation of unavoidable error. The experiments demonstrate their tightness and examine learning beyond the theorem’s sufficient conditions.
Critical weaknesses
  • The main guarantee covers a restrictive learning problem. Theorem 3 assumes the true correction matrix is fixed, uses population gradient flow, and requires strong Gaussian separation. At B=10B=10, the required mean separation is at least 60 times the largest standard-deviation scale. The finite-sample result guarantees retention alone. The MNIST experiment also fixes the correction and constructs regression targets, so it provides limited evidence about learning the correction itself.
  • Theorem 2 follows directly from Bayes’ rule, while Theorem 1 extends established divergence inequalities. These results provide useful context but modest additional novelty.
Questions for authors
  • How sensitive are the guarantees to errors in the fixed correction matrix?
  • Which proof step creates the large separation requirement, and could it be substantially tightened?
  • What prevents extending the adaptation guarantee to finite-sample gradient descent?
3 comments

Thank you for the helpful review.

Learning and estimating the correction matrix

With M\mathbf M fixed, we study how to learn when to apply a correction while preserving PT predictions. We can also learn the matrix first. In our noiseless Gaussian model, least squares recovers it exactly if the FT covariance is positive definite. Holding this estimate fixed then allows population GF for the gate under Theorem 3's conditions.

We can also quantify the effect of learning the matrix imperfectly. Suppose a gate achieves loss LL with the true matrix M\mathbf M. We now replace only the matrix by M^\widehat{\mathbf M}, keeping the gate unchanged. If the average squared difference between M^x\widehat{\mathbf M}\mathbf x and Mx\mathbf M\mathbf x is at most ε2\varepsilon^2 on the population being evaluated, the new loss is at most

(L+ε2)2.\left(\sqrt{L}+\frac{\varepsilon}{\sqrt{2}}\right)^2.

This applies separately to FT and PT. The bound follows from the triangle inequality and the fact that the gate cannot amplify the matrix error (since it's between 0 and 1). Extending the training guarantee also requires controlling how estimation errors affect the gate's gradient.

The MNIST experiment tests the linear and MMSE bounds on image features, rather than matrix recovery. To address joint learning directly, we added Figures 4 and 5, training both w\mathbf w and M^\widehat{\mathbf M} from random starts with fixed bias and L2 penalties on both. Training and parameter selection use only FT information. Figure 4 achieves small FT and PT losses at ρ=8\rho=8. Figure 5 examines finite-sample joint training and its optimization variability.

Where the separation requirement comes from

Separation matters because some inputs need different predictions for PT and FT. Theorem 1 shows that if there is substantial overlap on inputs requiring a large correction, no method can make both losses small. What is needed is separation where the correction matters, not necessarily everywhere. This explains why a separation condition is needed, without making our particular Gaussian threshold necessary.

The factor of 60 at B=10B=10 comes from a conservative step in Appendix C.3.4. We require the variance of the gate's linear input, w⊤x\mathbf w^\top\mathbf x, to stay below B/40B/40 throughout the controlled region. Combining this with ∥w∥≤3B/ρ\|\mathbf w\|\le3B/\rho produces the constant 360=9×40360=9\times 40. Appendix C.3.6 applies the same argument to PT inputs. This controls variation across inputs so the gate can open on FT while remaining mostly closed on PT.

Keeping a sharper norm bound already allows reducing 360 to 201, with corresponding proof updates and unchanged final guarantees. At B=10B=10, the separation factor becomes about 45 rather than 60. Further tightening is a useful direction for future work, motivated by the success observed with less separation in our experiments.

Finite-sample adaptation

The difficulty is that fitting a small FT dataset need not give low loss on new FT inputs. However, our proof leaves room for small errors. When λmin⁡<λ<λmax⁡\lambda_{\min}<\lambda<\lambda_{\max}, Appendix C.3.4 shows that training makes progress toward adaptation with a positive margin. Appendix C.3.5 also keeps the parameters within a region where retention is controlled. A finite-sample extension would require sampling errors small enough to preserve both properties throughout this region, together with sufficiently small GD steps. This gives a concrete starting point for extending the guarantee. Determining how many samples ensure these conditions is a useful direction for future work. Proposition 2 already guarantees finite-sample retention under its stated conditions.

Role and novelty of the MMSE results

Theorem 2 uses Bayes' rule to identify the best possible correction when both populations are known. Theorem 1 also uses classical divergence inequalities, which we cite. Our contribution is to characterize adaptation and retention using the correction FT requires, not just population overlap. If a shared input needs no FT correction, both populations already agree on its prediction, so their overlap causes no unavoidable error. Our bounds account for this by weighting inputs according to the correction they require. They hold for arbitrary densities, characterize the smallest total loss within a factor of two, and can be attained by explicit examples.

Theorem 3 then moves from what is achievable to what training actually achieves. It shows that, under the stated conditions, FT-only training learns to adapt while preserving PT predictions throughout training, without alignment assumptions.

If we managed to address your concerns, we appreciate if you consider increasing your score.

As the rebuttal deadline approaches, we would appreciate your thoughts on our response. We would be happy to address any remaining concerns or provide further clarification.

Thanks for the clear response. The explanation of where the separation constant comes from and the bound for an imperfect correction matrix are helpful. However, the main restrictions of Theorem 3 remain: even at about 45 standard deviations the two populations barely overlap, while the experiments succeed at much weaker separation; the matrix-error bound applies to a fixed gate rather than to training; joint learning is supported only empirically; and finite-sample adaptation is left for future work. I'll keep my score.

Reviewer_4sgpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies adaptation and retention through a two-population regression model for one linear map. It characterizes the best fixed linear correction, derives correction-weighted overlap bounds on MMSE loss, and analyzes a sigmoid-gated correction trained without pretraining examples.

The main learning guarantee concerns a fixed correction matrix and population gradient flow under separated Gaussian populations. The paper consistently distinguishes achievable limits from what training can attain.

Critical strengths

The exposition is clear about what the simplified model captures. Weighting population overlap by the required correction energy provides a more informative criterion than unweighted distribution separation.

The paper carefully distinguishes an oracle MMSE solution, existence of a suitable gate, and convergence of gate training. The anytime-retention guarantee and trajectory analysis are its most substantial technical contributions.

Numerical studies examine bound tightness, finite-sample behavior and nontrivial trajectories, including experiments outside the theorem’s sufficient conditions. The MNIST-feature study is appropriately presented as prediction-error analysis. Overall, the experiments illuminate the mathematical claims without requiring a new algorithmic contribution.

Critical weaknesses

The most important limitation is that the desired correction MM is fixed and already available. The analysis therefore explains learning when to apply a correction, while leaving joint learning of the correction and gate unresolved. This is explicitly acknowledged, but limits direct transfer to practical fine-tuning.

The learning guarantee also requires a zero-mean Gaussian pretraining population, a covariance bound and strong separation:

ρ2≥360Bmax⁡{∥Σft∥op,R}.\rho^2 \geq 360B\max\{\|\Sigma_{\mathrm{ft}}\|_{\mathrm{op}},R\}.

These conditions are sufficient rather than necessary, as the authors state, but the gap between the guaranteed and useful empirical regimes is considerable. Finite-sample adaptation is demonstrated experimentally rather than guaranteed by Theorem 3.

Finally, the posterior/sigmoid identities are largely standard, and DISeL is an existing method that is properly credited. The acceptance case consequently rests mainly on the weighted-risk analysis and gradient-flow guarantee. A sharper treatment of the retention-versus-adaptation-time tradeoff would strengthen that case, since a strongly closed gate can suppress the learning signal.

Questions for authors

Can the guarantees tolerate an estimated correction matrix, or simultaneous learning of MM and the gate, even in a low-dimensional setting?

Can the population adaptation result be extended to finite-sample training with an explicit sample requirement?

How do bias, separation and regularization jointly determine the time needed to achieve a specified adaptation error while preserving retention?

2 comments

Thank you for the helpful review.

Fixed correction matrix

We added Figures 4 and 5, which jointly learn w\mathbf w and M^\widehat{\mathbf M} with fixed BB and regularization on both parameters. Training and parameter selection use only FT information. Figure 4 shows small FT and PT losses at separation 8 in the 2D setting, while Figure 5 tests learning from finite FT datasets.

Zero-mean PT population and separation

Zero mean simplifies the analysis. With a known nonzero PT mean, we can center the gate inputs and adjust the retention bounds, using mean information rather than PT samples.

If PT and FT contain many of the same inputs but need very different predictions, no method can achieve small error on both (Theorem 1). Our theorem gives conditions that guarantee both adaptation and retention, and the experiments show success even with less separation.

Novelty

We agree that the posterior identities are standard and DISeL is an existing method. Our contributions are the correction-weighted MMSE bounds, the general linear-loss characterization, and the FT-only retention and GF guarantees, not the posterior formulas.

We address your three questions below, including the finite-sample and training-time concerns.

Question 1: Learning the correction matrix

A simple approach is to learn the correction matrix first, then train the gate. In our noiseless Gaussian model, ordinary least squares recovers the true matrix with probability one when the FT covariance is positive definite and the number of independent examples is at least the input dimension:

=\left[\sum_{i=1}^n(\mathbf y_i-\mathbf W_0\mathbf x_i)\mathbf x_i^\top\right] \left[\sum_{i=1}^n\mathbf x_i\mathbf x_i^\top\right]^{-1} =\mathbf M.$$ After recovering $\mathbf M$, we keep it fixed and train the gate by population GF, as in Theorem 3. Under the theorem's conditions, this gives adaptation and retention without PT samples. It is a two-stage result, rather than a guarantee for learning both parameters together. To study learning both parameters together, we added Figures 4 and 5. We keep the bias fixed and add a penalty on the learned matrix alongside the gate penalty: $$\mathcal L_\lambda(\mathbf w,\widehat{\mathbf M}) =\mathbb E_{\mathbf x\sim p_{\mathrm{ft}}}\!\left[ \left\\|\sigma(\mathbf w^\top\mathbf x-B)\widehat{\mathbf M}\mathbf x-\mathbf M\mathbf x\right\\|^2 \right] +\frac{\lambda}{2}\left(\\|\mathbf w\\|^2+\\|\widehat{\mathbf M}\\|_F^2\right),\qquad\lambda>0.$$ These penalties also let us extend the retention guarantee. Let $E_0$ be the initial objective. Under Proposition 2's PT assumptions, any training procedure that keeps this objective non-increasing satisfies $$\sup_{t\ge0}\mathcal L_{\mathrm{pt}}(t) \le\frac{RE_0}{\lambda}(1+B)e^{-B/2}, \quad\text{when}\quad 2RE_0\le\lambda B.$$ The argument is simple: decreasing the objective limits both parameter norms, and Proposition 2 then bounds the PT loss. This also applies to the sample-average FT objective, with $E_0$ defined accordingly. Proving adaptation during joint training is a promising direction for future work. > Question 2: Finite-sample training Proposition 2 already guarantees finite-sample retention under its stated conditions. For adaptation, the next step is to show that gradients computed from FT samples stay close to the population gradients throughout training. Appendix C.3.4 leaves room for small errors in these gradients. Determining how many samples are needed to preserve the adaptation guarantee is a direction for future work. > Question 3: Bias, separation, and training time Under Theorem 3's conditions, PT loss stays small throughout training. Before $T$, FT loss may still be large, but afterward both losses satisfy the theorem's bounds. For a desired FT error, choose $B$ to meet that bound while satisfying the separation condition. Larger $B$ improves the loss bounds but weakens the initial learning signal. Greater separation helps distinguish FT from PT and is required for larger $B$. Regularization limits the weights but opposes opening, so taking $\lambda$ close to $\lambda_{\max}$ worsens the time bound. For $\lambda=\lambda_{\max}/2$, Appendix C.3.5 gives $$T\le \frac{16B\left(1+\frac{1}{30B}\right)}{\rho^2\mathbb E_{\mathrm{ft}}[q(\mathbf x)]} \exp\left(B+\frac1{20}+\sqrt{\frac{3B}{10}}\right).$$ The exponential factor in $B$ shows the possible cost of stronger protection. This is an upper bound, not a necessary training time. If we managed to address your concerns, we appreciate if you consider increasing your score.

As the rebuttal deadline approaches, we would appreciate your thoughts on our response. We would be happy to address any remaining concerns or provide further clarification.

Comments on the paper

2 comments

Dear Reviewers,

As the rebuttal deadline is approaching, please review the authors' responses and participate in the discussion. Kindly indicate whether your concerns have been addressed and update your scores if appropriate.

Thank you for your time and effort!

Best regards,
Area Chair

We thank all the reviewers for their constructive feedback. To summarize our contributions and the discussion we had in the rebuttal, we would like to mention the following points.

We study when a model can learn a new task while preserving its previous behavior, and whether training on FT data alone can achieve this. This is a practical concern in LLM fine-tuning, where PT data are often unavailable.

As reviewer x4NR highlighted, our MMSE analysis identifies a fundamental limit. For arbitrary population densities within our model, Theorem 1 gives a tight lower bound that no correction can beat, including nonlinear ones with full access to both populations. Our bound goes beyond overlap alone: it accounts for the required correction, recognizing that shared inputs needing no change are harmless. This is particularly relevant to LLM fine-tuning, where a new task need not require changes on every input. Both losses cannot be small if PT and FT substantially overlap where large corrections are required. Separation where these corrections matter is therefore necessary, although our GF theorem's sufficient separation threshold could be potentially improved.

Reviewers 4sgp and 4EQi highlighted the trajectory analysis and retention guarantees. With a fixed correction matrix and under the stated Gaussian and separation conditions, Theorem 3 proves that population GF achieves adaptation while keeping PT loss small throughout training. As reviewer 4EQi noted, it requires no alignment between the mean, covariance, and correction matrix. Proposition 2 separately guarantees retention from finite FT datasets, for arbitrary FT distributions under the stated conditions.

In response to the reviews:

  • We have added Figures 4 and 5, which learn both the correction matrix and gate using only FT information, with fixed bias. Our replies also give a simple two-stage matrix-recovery argument and a retention extension for regularized joint training.
  • We have expanded the comparison with continual-learning theory, corrected the explanation of the regularized FT objective, and incorporated the standard sigmoid identity into Equation (8), making the explicit Gaussian result Theorem 2.
  • We will clarify the scope of the learning guarantees, make the existing experiment within the theorem's conditions more visible, and add a bias and regularization sensitivity study. We will also explain more clearly how stronger retention protection can delay adaptation.

Finite-sample adaptation and joint-training convergence are the main next directions. In our replies, we already identify where the current proof tolerates gradient errors, providing a concrete starting point for the finite-sample extension.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.