Is Flat Merging Enough? Evolutionary Search for Tree-Structured Model Merging

Model merging provides an efficient way to integrate independently fine-tuned models without costly retraining or access to their original training data. Most existing methods, however, focus primarily on designing merging operators while combining a fixed set of candidate experts through a typically flat aggregation structure. This leaves a largely overlooked structural design space: how experts should be organized, which experts should participate, and whether experts should be reused throughout the merging process. Through systematic empirical studies, we show that, even when the candidate experts and merging operator are held fixed, changing only the merging structure can substantially affect performance, with markedly different sensitivities across operator families. Identifying effective configurations in this structural space is challenging, as merging topology, expert selection, and expert reuse jointly define a large discrete and combinatorial search space. We therefore propose EvoTM, an evolutionary framework based on genetic programming for optimizing tree-structured model merging. EvoTM represents a complete merging procedure as a hierarchical tree, where leaves correspond to expert models and internal nodes apply a given merging operator. Structure-aware evolutionary operators jointly explore tree topology and expert composition, naturally accommodating expert selection and reuse within a unified framework. Experiments across diverse merging operators and both vision and language models show that EvoTM can improve over conventional flat merging in multiple settings. Further analyses reveal operator-dependent structural sensitivities and provide additional insights into how different classes of merging operators respond to tree-structured composition. These results highlight merging structure as a complementary and previously underexplored design dimension of model merging.