- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper theoretically characterizes when a task vector can be transferred exactly from one model to another under a proximal fine-tuning formulation. Specifically, exact transfer occurs if and only if the parameter difference between the two original models lies in the null space of the task-loss Hessian averaged along the path connecting their fine-tuned counterparts, under suitable assumptions. The authors further show that the magnitude of the violation of this condition bounds the transfer loss gap, and empirically demonstrate a strong correlation between a K-FAC-based proxy for this violation and the loss gap relative to direct fine-tuning. They also propose CurvTV, which introduces a curvature-based correction to vanilla task-vector transfer, and derive a tighter quartic upper bound on its optimality gap in the local regime, compared with the quadratic upper bound for vanilla transfer. Empirically, CurvTV outperforms vanilla task-vector transfer, Re-Basin, and GradFix on RoBERTa-family models, and improves upon vanilla task-vector transfer on Llama-3.1-8B-based models.
Critical strengths
-
Clear motivation and relevance. The paper addresses a well-motivated and practically relevant problem: task-vector transfer works well in some settings but fails in others, and understanding when and why this happens is valuable.
-
Insightful theoretical characterization. The paper provides an elegant theoretical characterization of when task-vector transfer is exact, offering a useful explanation for why task-vector transfer can succeed or fail under the stated assumptions.
-
Theory-guided method. The proposed CurvTV method is well motivated by the theoretical analysis, directly addressing the identified source of transfer error. The additional theoretical guarantees on its optimality gap are also appreciated.
-
Solid empirical evaluation. The experiments demonstrate the effectiveness of the proposed method across multiple models and downstream tasks, including language models with up to 8B parameters.
Critical weaknesses
-
Mismatch between theoretical assumptions and practical fine-tuning. The theoretical analysis assumes proximal regularization toward the original pretrained model, , whereas the experiments use standard weight decay, which corresponds to an L2 penalty under ordinary SGD. This distinction is consequential: replacing proximal regularization with standard L2 regularization changes the stationarity condition to . Under the nonsingularity assumption, this implies , so exact task-vector transfer would require , rather than the proposed directional curvature condition .
This mismatch also affects the proposed CurvTV method. In theory, and represent the proximal regularization strengths used during fine-tuning. However, the practical fine-tuning procedure does not use these proximal regularizers, and the authors instead select and through empirical performance tuning without establishing their connection to the actual fine-tuning procedure. This weakens the connection between the theoretical derivation and the proposed method. Although the authors acknowledge this limitation, a clearer theoretical justification or additional experiments using explicit proximal regularization would help bridge the gap between theory and practice.
-
Local nature of the theoretical guarantee. The theoretical advantage of CurvTV over vanilla task-vector transfer is local, as acknowledged by the authors. Specifically, the quartic upper bound provides a stronger asymptotic guarantee only when the displacement between the two anchor models is sufficiently small. This limits the scope of the theoretical improvement, as task-vector transfer may also be applied between models with substantial parameter differences.
A minor additional concern is the comparison of theoretical upper bounds between CurvTV and vanilla TV. The authors argue that CurvTV offers a theoretical advantage by comparing its quartic upper-bound branch in the local regime with the quadratic upper bound of vanilla TV in terms of their asymptotic orders. However, since explicit upper bounds are available for both methods, directly comparing these bounds would be more informative than comparing their asymptotic orders alone. Such a comparison yields an additional threshold on the anchor displacement, below which CurvTV's quartic branch provides a tighter upper bound than vanilla TV's quadratic bound. Beyond this threshold, the quartic bound is not necessarily tighter. Although this does not change the conclusion that CurvTV admits a tighter upper bound in a sufficiently local regime, explicitly discussing this threshold would make the theoretical comparison more precise.
-
Minor figure presentation issues. In Figure 1(a), several data points are obscured by the legend. In Figure 5, some data points are obscured by the Pareto frontier curve.
Questions for authors
In Figure 2(a), only the values of are explicitly indicated in the legend, although the text discusses trends with respect to both and . How are the different values of represented in the figure? It would be helpful to clarify this so that the claimed trends with respect to can be verified.
2 comments
Q5: Figure presentation
Thank you for the careful reading of our figures. We would like to clarify that each curve in Fig. 2(a) fixes one value of and connects five points for in increasing order. Thus, the increase of the native gap with for fixed corresponds to moving right along a single curve, whereas its decrease with for fixed is seen by comparing corresponding points across curves. In the revision, we will label the values in the figure.
Thank you also for pointing out the obscured data points. In the revision, we will move the legend of Fig. 1(a) away from the data points, and draw the Pareto frontier in Fig. 5 beneath the data points and make it semi-transparent.
References
[1] Ali et al. A Continuous-Time View of Early Stopping for Least Squares. AISTATS 2019.
[2] Daheim et al. Model Merging by Uncertainty-Based Gradient Matching. ICLR 2024.
[3] Yu et al. Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch. ICML 2024.
Q1: Proximal regularization versus practical fine-tuning
Thank you for this careful derivation, which is exactly why we use the proximal term to model the overall effect of standard fine-tuning in practice. Under L2 regularization alone, fine-tuning from any initialization would converge to the same solution, so the transfer problem would not arise. However, practical fine-tuning does not converge to the same solution. Techniques such as early stopping, a limited number of training steps and small learning rates keep the solution close to its initialization, which approximately acts as proximal regularization centered at the initialization [1]. The proximal objective captures this key effect while keeping the analysis simple, a modeling choice shared by related work such as Daheim et al. [2].
Moreover, our experiments already test the theory under standard fine-tuning. On RoBERTa, θ1⋆ and θ2⋆ are trained with weight decay, without a proximal term, for a fixed number of epochs [3]. In this setting, the proxy for the violation of the directional-curvature condition is highly correlated with the loss gap (rˉ=0.959), and CurvTV raises the average gain from 0.353 to 0.465 without increasing the native gap. This shows that the proximal model also explains standard fine-tuning.
Q2: Connection between α1,α2 and fine-tuning
Thank you for this question. We would like to clarify that αi models the strength of the implicit regularization in the fine-tuning on each side, which is determined by factors such as the learning rate and the number of training steps. For least squares, for example, gradient flow run up to time t has a risk close to that of ridge regression with coefficient 1/t [1]. For deep networks this correspondence is only qualitative and αi cannot be computed in advance, so we treat α1 and α2 as hyperparameters.
Q3: Improvement beyond small displacements
Thank you for raising this point. We agree that the fourth-order improvement of CurvTV is local. However, an improvement that does not depend on the size of the displacement follows directly from our existing results. Let u=θ2⋆−θ1⋆, and let H~ be the curvature used by CurvTV. Comparing the CurvTV upper bound at θ~CurvTV with the vanilla lower bound given by the identity in Thm. 3.3 shows that, under the conditions of that theorem, if H~⪰0 and ∣(Hˉ−H~)u∣<α/(α+β)∣Hˉu∣, then L2(θ~CurvTV)<L2(θtv). This condition does not involve the size of the displacement. It only requires the curvature estimate to approximate the path-averaged curvature along u with relative error below α/(α+β). Therefore, even when the two anchor models differ substantially, CurvTV strictly outperforms vanilla TV as long as the curvature changes little along the path. In the quadratic case with the exact Hessian, H~=Hˉ and CurvTV exactly recovers θ2⋆ for displacements of any size, so the condition holds whenever vanilla TV is not exact (Hˉu=0). In the revision, we will state this sufficient condition after Thm. 4.4.
Q4: Direct comparison of the upper bounds
Thank you for this helpful suggestion. We directly compare the fourth-order branch of Thm. 4.4 with the vanilla upper bound in Thm. 4.3, which gives
\iff \\|\theta_2-\theta_1\\|<\frac{2\\|\psi_\alpha(\bar H)\\|}{\rho\Upsilon_\alpha}.$$ That is, the CurvTV upper bound is strictly smaller than the vanilla upper bound whenever the error of CurvTV caused by the change of the Hessian along the path is small compared with the error of vanilla TV caused by ignoring curvature. Hence, the more slowly the Hessian changes along the path, the larger the displacement for which CurvTV has the smaller upper bound. Beyond the threshold, one can instead use the sufficient condition in our response to Q3, which does not depend on the size of the displacement. In the revision, we will state this threshold after Thm. 4.4.