Neuron-Aligned Model Merging for Cross-Modal Reasoning Transfer

Existing neuron-level model merging methods transfer reasoning capabilities from LLMs to VLMs by selectively merging reasoning-relevant neurons at the same neuron indices. However, reasoning training and multimodal adaptation can reshape neuron responses, breaking the same-index correspondence assumption: same-index neurons may become functionally mismatched, while cross-index correspondences may be overlooked. To address this limitation, we propose Targeted Alignment and Neuron Grafting via Optimal Transport (TANGO), a training-free framework for cross-modal reasoning transfer through neuron-aligned model merging. TANGO uses activations on shared inputs and optimal transport to establish functional correspondences between source and target neuron spaces, rather than relying on fixed index-wise matches. The resulting correspondences guide both the selection of reasoning-relevant transfer locations and the alignment of source parameters for selective merging. Across four VLMs from the Qwen and InternVL families, TANGO consistently improves multimodal mathematical reasoning and general multimodal understanding, achieving the best or tied-best result in 27 of 28 mathematical reasoning settings and ranking first in 18 of 20 general multimodal and text-only reasoning comparisons.