Dynamic Model Merging with Multi-Layer Aggregated Routing Evidence

Dynamic model merging aims to retain the efficiency of a unified model while better preserving task-specific expertise through input-dependent routing at both the task and head levels. However, even the routing components of the state-of-the-art approach remain limited: the task routing is highly sensitive to the choice of routing layer, whereas the head routing relies on simple max-logit competition. To address these limitations, we propose multi-layer aggregated routing evidence (MATE) as a unified mechanism for improving both task and head routing. Specifically, we first introduce a metric to quantify task routing separability across different layers. Empirical analysis of this metric reveals that different layers provide complementary evidence for identifying the tasks relevant to an input. Motivated by this observation, MATE selects multiple informative layers according to their routing separability and aggregates their layer-wise routing scores to produce a more robust signal that can be directly used for task selection. Moreover, we observe that this signal also encodes useful information about the suitability of the corresponding classification heads. This motivates us to further combine the MATE signal with the max-logit score to guide classification head selection. Extensive experiments on vision benchmarks demonstrate that MATE consistently achieves state-of-the-art performance.