- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper targets the cross-lingual gap in LLM mathematical reasoning. NAPO (i) selects FFN neurons that are language-balanced (high entropy of activation probability across languages) and strongly active in every language; (ii) represents each response by position-weighted pooled activations of these neurons in intermediate layers; (iii) scores each target-language candidate by layer-weighted cosine similarity to an English reference response; and (iv) trains with DPO plus an NLL term for three iterations. Experiments on MNumGLUESub, MSVAMP and MGSM (10 languages) with MistralMathOctopus report gains over MAPO, CM-Align and LaCoMSA (+4.5 and +2.2 average points over MAPO on MNumGLUESub and MSVAMP), plus ablations, consistency analyses, t-SNE plots, and results on Qwen2.5-3B and Llama3.1-8B.
Critical strengths
- Three benchmarks, ten languages and three backbones, with iteration curves and several analyses. If the comparisons are fair, the Mistral results are strong.
- Which internal representation space gives the most useful cross-lingual reward is a timely question, and a sparse language-balanced subspace is a reasonable candidate worth testing.
Critical weaknesses
1.Limited novelty. The reward paradigm follows MAPO/LaCoMSA, the neuron selection reuses LAPE (Tang et al., 2024) with the entropy criterion inverted, and the iterative DPO+NLL objective is essentially Iterative RPO (Pang et al., 2024, not cited). The method only changes the representation space in which an existing reward is computed. Claims of "reasoning-relevant neurons" and of independence from hidden-state signals are not supported by the implementation. 2.Insufficient experimental evidence and missing methodological details. The paper does not state whether the reimplemented baselines were trained under the same settings as NAPO. Beyond this, the paper provides no direct evidence that the neuron-activation-based reward is effective. Position-weighted pooling, presented as a main contribution, lacks comparison experiments against alternative pooling strategies in terms of downstream performance. Finally, key parameters and methodological details are insufficiently described, which makes the method difficult to reproduce and its results difficult to verify.
Questions for authors
- Some reported values are inconsistent across tables (e.g., Table 3 vs. Tables 1 and 6). Please verify and correct all reported results, and explain any discrepancies.
- The manuscript contains broken cross-references, typos, inconsistent terminology and notation, unreadable figures, and claims that are not supported by the reported results. Please revise it carefully.