Directional Curvature Governs Task-Vector Transfer: Theory and Curvature Compensation

Amid rapid iteration of pretrained models, task vectors—the parameter difference between a base model and its fine-tuned counterpart—offer a training-free transfer rule: adding the task vector learned on one model to another to impart similar task capabilities. However, prior empirical studies are mixed: task-vector transfer can succeed in some settings but fail dramatically in others. In this paper, under suitable regularity conditions, we theoretically characterize both when vanilla transfer is exact and how its suboptimality is controlled. Specifically, we prove that vanilla transfer is exact if and only if the model displacement ${\theta }_2-{\theta }_1$ lies in the null space of the path-averaged Hessian $\bar{H}$, i.e., the directional curvature condition $\bar{H}({\theta }_2-{\theta }_1)=0$. We further show that the violation magnitude $$\begin{gathered} |\bar{H}({\theta }_2-{\theta }_1) \\ {|}^2 \end{gathered}$$ controls the suboptimality of transfer and provide proxy-based empirical support for this relationship in neural networks. To overcome the intrinsic limitations of vanilla transfer, we propose the Curvature-compensated Task Vector (CurvTV), which enhances task-vector transfer by compensating for parameter deviations induced by violations of the directional curvature condition. Theoretically, CurvTV retains a quadratic optimality-gap envelope while admitting a sharper quartic refinement in the local regime, and this local gain persists under controlled Hessian estimation error. Extensive experiments across RoBERTa and Llama-family models demonstrate that CurvTV yields consistent improvements in task-averaged performance over vanilla task-vector transfer.