A Controlled Benchmark for Weight-Space Model Merging Across Medical Imaging Domains

Consolidating medical-imaging specialists into a shared model requires preserving performance across distinct imaging domains. Weight-space model merging offers a way to combine independently fine-tuned encoders without retraining, but comparisons between merging methods are sensitive to validation protocols. Official evaluation scripts for TSV-Merge and isotropic merging use wider scaling-coefficient ranges for selected spectral methods than for other tunable baselines. We introduce a controlled benchmark spanning dermoscopy, chest radiography, and histopathology, with three binary tasks and $2,000$ training images per specialist. We compare 11 methods across seven vision-transformer backbones and three seeds using a shared scaling-coefficient grid. The best observed merger reaches $0.811$ mean AUROC against $0.853$ for task-aware specialist routing, with tuned Task Arithmetic close behind at $0.808$. We also introduce Gram-LS, a closed-form method based on task-vector projection constraints that performs comparably to tuned summation. Within-method validation tuning improves Gram-LS and Task Arithmetic by approximately $0.030$–$0.035$ AUROC, exceeding the $0.019$ spread among five leading tuned methods, while search bounds can change method rankings. Iso-C and Iso-CTS fall $0.10$–$0.13$ AUROC below tuned summation. These findings establish tuned summation as a strong baseline in this setting and show that specialist references and validation protocols are essential to assessing the performance cost of consolidation.