- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces a model merging method for multilingual reasoning models to improve their multilingual reasoning ability. Specifically, authors first need train a continue-pretraining model in the target language and a supervised fine-tuning model in English. Then, they locate and merge the most important neurons for each model to obtain the final reasoning model working on the target language. Experimental results show that the proposed method outperforms baseline methods across most of involved languages.
Critical strengths
- The proposed method is easy to follow and obtains promising results across ten languages and three reasoning datasets.
Critical weaknesses
-
The motivation and findings are not supported from the analyses of this work. Take Figure 3 as an example, the curve is mostly above than the y=x (random sample baseline). It means that the most language-sensitive neurons are more likely to overlap with the most reasoning-sensitive neurons, rather than the claim in this paper (the most language-sensitive neurons induced by CPT are largely distinct from the most reasoning-sensitive neurons emphasized by SFT). For example, the top 10% neurons of CPT have 20% more overlap with the ones of SFT, which are two times higher than random baseline.
-
"Many neurons are highly important for only one training stage" is not supported. If that were the case, we would find data points clustering distinctly near the axes (i.e., high CPT paired with low SFT, or vice versa). Instead, the actual data form a coherent, moderately dispersed cloud-like distribution. To make the claim solid, the authors need to report Spearman correlations per module type, use rank-normalized axes, mark top-k overlap (for example, the fraction of the top 1% CPT neurons that are also in the top 1% SFT neurons), and include a same-stage, different-seed control.
-
Missing multitask learning baseline method that training on both CPT and SFT samples. This is also an important baseline for comparison.
Questions for authors
- Please refer to the wrong claims and the missing baseline method in the weakness part.