- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TANGO, a training-free framework for transferring reasoning capabilities from LLMs to VLMs through neuron-aligned model merging. Instead of relying on same-index neuron correspondence, TANGO uses shared-input activations and optimal transport to identify functional correspondences between source and target neurons. The learned transport plan guides both task-relevant neuron selection and source-parameter alignment for selective merging. Experiments across Qwen and InternVL models show consistent improvements on multimodal mathematical reasoning and general reasoning benchmarks.
Critical strengths
- The paper identifies a reasonable limitation of index-wise neuron merging, as neurons with the same indices may no longer have matching functions after different training and adaptation processes.
- TANGO combines activation-based neuron alignment, task-aware neuron selection, and selective parameter merging into a unified framework.
- The method achieves the best average mathematical reasoning performance across four VLMs and shows generally positive results on broader multimodal and text-only benchmarks.
- The paper evaluates alignment strategies, soft versus hard transport, and several hyperparameters. The controlled permutation experiment also provides evidence that OT can recover known neuron correspondences.
Critical weaknesses
- TANGO largely combines existing activation-based optimal transport alignment with task-aware neuron selection and selective model merging. While the integration is reasonable, the paper does not sufficiently establish what fundamentally distinguishes the proposed framework from prior OT-based model merging and neuron-level reasoning transfer methods.
- Table 3 shows that perturbing the selected parameters severely degrades model performance, but lacks matched-budget random or low-importance neuron controls. Therefore, the results do not demonstrate that the identified neurons are specifically important for reasoning rather than essential to general model functionality.
- The controlled permutation experiment verifies OT alignment under artificial neuron permutations, but does not establish that the learned correspondences are functionally meaningful between independently trained LLMs and VLMs. More direct evidence on real model pairs is needed to support this central assumption.
- Table 4 does not fully disentangle OT-based neuron selection from OT-based parameter alignment. Without evaluating these two components independently, it is unclear which contributes most to the reported improvements.
Questions for authors
- Beyond the controlled permutation experiment, the paper provides insufficient evidence that the learned OT correspondences are functionally meaningful between independently trained LLMs and VLMs.
- The individual contributions of OT-based neuron selection and OT-based weight alignment are not sufficiently isolated, making it difficult to determine their respective effects.
- The paper does not adequately investigate how different calibration data distributions affect neuron correspondence and downstream reasoning performance.
- The paper lacks a detailed analysis of TANGO's computational and memory overhead.
6 comments
Dear Reviewer VJ7N,
Thank you for suggesting additional controls for TANGO’s selection and alignment mechanisms. The new experiments show that both design choices improve transfer, and TANGO achieves higher average mathematical reasoning accuracy than SNRF with less offline compute on both tested backbones.
The choice of neurons matters. Zeroing TANGO-selected positions caused greater degradation across all three benchmarks than zeroing the same number of random positions. With OT fixed, multimodal profiling improved all five evaluated metrics. With the selection mask fixed, OT parameter alignment improved both reasoning benchmarks over same-index merging.
Correspondence holds on real model pairs. The frozen text-derived OT plan remained predictive on paired image and description inputs, and all three tested text calibration corpora yielded high held-out response correlations.
Less offline compute. Compared with SNRF, TANGO reduced the combined time for profiling, alignment or selection, and merging by 34.9% on Qwen3-8B and 15.0% on InternVL3.5-8B, with comparable peak GPU memory.
If you have a chance to revisit the rebuttal, we would appreciate your taking these results into account in your final assessment. We would be glad to discuss any concerns that remain.
Best regards, The authors
Thank you to the authors for providing additional experimental results, which address part of my concerns. Regarding W4, I would like to further understand the effectiveness of OT-based parameter alignment in the absence of neuron selection. Specifically, what would the performance be if the selection mask (M) were removed and the OT-aligned source weights were directly merged with the target weights?
We thank the reviewer for the careful assessment and constructive suggestions.
W1: Distinction from prior OT merging and neuron-level transfer
TANGO combines target-side multimodal relevance ui with OT-transported source reasoning relevance to select neurons. Cui et al. (2026a) selects neurons by activation magnitude in text-only merging. TANGO also replaces SNRF’s same-index matching with OT-based functional alignment. When transferring from DeepSeek-R1-0528-Qwen3-8B to Qwen3-VL-8B-Instruct, removing only ui with the OT plan, merge coefficients, and update budget fixed lowers all five scores.
W2: Functional importance of selected neurons
Zeroing PG-OT positions lowers all three scores more than matched random zeroing. Both runs start from the same Qwen3-VL-8B-Instruct checkpoint and zero the same number of positions per layer and projection. The zeroing runs use the MME perception score. Original TANGO is shown for reference.
W3 / Q1: Functional correspondence in real LLM–VLM pairs
Text-derived OT predicts neuron responses when the VLM processes images. In Figure 1, a cross-index pair has Pearson correlation 0.851 versus 0.640 for the same-index pair. Across 36 Q-projection layers, 54.26% of OT mass is off-diagonal. On 500 Geometry3K pairs, the VLM receives images and questions and the LLM receives faithful descriptions and the same questions. Frozen GSM8K OT yields higher weighted correlations than same-index and shuffled matching. Image-based recalibration gives no consistent downstream advantage, supporting shared text as the default.
W4 / Q2: Neuron selection and parameter alignment
Both components contribute. Selection chooses target positions (Eqs. 6–7), and alignment maps source weights to them (Eqs. 8–9). We will clarify these roles in Table 4. The full method scores 67.92 versus 66.38 without OT alignment, 66.91 with shuffled correspondence, and 66.28 with hard transport. Fixing the mask isolates the benefit of OT parameter mapping below. W1 fixes OT and isolates the contribution of ui to selection.
Q3: Sensitivity to calibration data
All three corpora yield high held-out correlations, and GSM8K leads on the three downstream benchmarks. We recompute OT, selection, and merging using 500 questions per corpus while fixing the model pair, task profiling, update budgets, and merging coefficients. We test each frozen plan on the same target neurons and held-out sets (100 questions each).
Q4: Compute and GPU memory
On one A100 80GB in BF16, TANGO cuts three-stage compute time versus SNRF by 34.9% on Qwen3-8B and 15.0% on InternVL3.5-8B, with comparable peak memory. Faster merging offsets alignment cost. TANGO and SNRF average 55.94% and 55.06% on Qwen3-8B, and 56.67% and 56.22% on InternVL3.5-8B, respectively, across the mathematical reasoning tasks. The merge runs once offline. Inference uses the original target architecture without the source expert or OT.