Predicting When Model Merging Beats Reverting to the Initialization

Model merging is typically evaluated by whether the merged model outperforms the two fine-tuned models it combines. We show that this comparison is misleading whenever both fine-tuned models score below their shared initialization on the reported metric, as is common after self-training, continual fine-tuning, or fine-tuning for accuracy. Interpolating a single one of them back toward the initialization, by the same distance the merge moves, can then match or exceed the merge. We propose this interpolation, the matched regression-to-ancestor control, as a baseline that merging evaluations should report. We also derive a criterion that predicts which of the two will score higher from the initialization and the two fine-tuned models alone, before either candidate is evaluated. Both candidates lie in the plane spanned by the two task vectors, where a first-order approximation, which we test and whose error we bound, reduces the comparison to a single inequality with no fitted parameters. On a synthetic self-training testbed with exactly known structure, the criterion correctly predicts the better candidate on 28 of 28 lineage pairs. We further fine-tune three experts from each of six base models spanning three families and evaluate every pair on four benchmarks, for 72 cells in total. The control outperforms the merge in 97% of the cells in which the initialization outscores both experts, compared with 44% of the remaining cells. On third-party fine-tunes of Llama-3.1-8B, the control recovers a median 69% of the gain of a merge that outperforms both of its parents. The criterion does not uniformly favor the control: on the standard 8-task CLIP benchmark, which we reproduce, the best expert outscores the initialization, the criterion accordingly predicts no advantage for the control, and the merge outperforms it by 12.5 points. The control requires a single interpolation, and the criterion selects the better of the two candidates in 86% of cells, compared with 32% for the common practice of always selecting the merge. Code is available at https://anonymous.4open.science/r/merge-plane-geometry-4591.