- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Safety-Neuron Preference Optimization (SNPO) for multilingual safety alignment. It identifies candidate safety-related neurons from activation differences between harmful and benign prompts, groups languages by resource level, and alternately masks one group’s safety neurons while updating only the other group’s exclusive neurons using multilingual preference data. All neurons are restored at inference. The main contribution is a neuron-level training strategy for directing safety updates across language groups. Evaluation covers seven models and eight languages using MULTI-STRONGREJECT for safety and MGSM/M-MMLU for utility, with comparisons against MPO, SWE, and additional training baselines. Results show consistent average safety improvements, while utility varies by model. Ablation and pruning studies support the strategy’s effectiveness and the functional importance of some neurons that become shared across languages, although safety specificity and over-refusal remain insufficiently validated.
Critical strengths
Strengths: 1.Consistent safety improvements across multiple models. SNPO achieves the largest average reduction in unsafe responses among the compared methods on all seven evaluated models across two model families and eight languages. 2.A distinctive training intervention supported by meaningful ablations. SNPO combines language-group-specific activation masking with restricted parameter updates to encourage cross-lingual safety learning.
Critical weaknesses
Weaknesses: 1.Insufficient validation that the detected neurons specifically mediate safety behavior. The detection procedure selects neurons whose activations distinguish harmful from benign prompts, which may also capture differences in topic or task rather than safety functionality. The post-training pruning experiment provides useful functional evidence for the Specific→Shared subset, but does not establish the safety specificity of the original detected sets; it also lacks matched functional controls and utility measurements after pruning. This matters because identifying language-specific safety carriers is central to both SNPO’s design rationale and its mechanistic interpretation. 2.Missing evaluation of over-refusal leaves the safety–helpfulness trade-off unresolved. The paper measures unsafe responses to harmful prompts but does not evaluate whether SNPO also rejects legitimate requests, particularly benign requests involving safety-sensitive topics across the target languages. MGSM and M-MMLU do not directly measure this behavior, and Table 1 already shows a substantial MGSM decline on Qwen2-1.5B, from 20.36 to 14.89.
Questions for authors
Did you validate the detected neuron sets in the original models through ablation, using controls matched for neuron count and layer distribution, while measuring both safety and utility? Such evidence would help distinguish safety-related functionality from general topic or task processing. Did you evaluate refusal rates on benign requests across the target languages, particularly safety-sensitive but legitimate requests? If so, how does SNPO compare with the original models and baselines? This would clarify whether the ASR improvements reflect better safety discrimination or broader refusal.
5 comments
Response to Reviewer hyMp(2/3): Weaknesses1 - Part 2
(Continued from our previous comment.)
(iii) Cross-lingual overlap.
We further compute the Jaccard similarity between the safety-neuron sets of each language pair. The overlap is far from complete (0.052–0.391) and particularly low for Bengali (0.052–0.123); moreover, it is markedly higher among high-resource languages (average 0.308) than among low-resource languages (0.179). This shows that multilingual safety relies on both shared and language-associated neurons, with safety carriers unevenly distributed across languages, which supports SNPO's design of routing alignment signals to different neuron groups rather than treating all safety neurons uniformly.
| Language | EN | ZH | VI | JA | TH | ID | BN | HE |
|---|---|---|---|---|---|---|---|---|
| EN | 1.000 | 0.390 | 0.248 | 0.284 | 0.191 | 0.219 | 0.052 | 0.133 |
| ZH | 0.390 | 1.000 | 0.333 | 0.389 | 0.273 | 0.266 | 0.072 | 0.194 |
| VI | 0.248 | 0.333 | 1.000 | 0.279 | 0.360 | 0.391 | 0.080 | 0.232 |
| JA | 0.284 | 0.389 | 0.279 | 1.000 | 0.256 | 0.232 | 0.080 | 0.187 |
| TH | 0.191 | 0.273 | 0.360 | 0.256 | 1.000 | 0.293 | 0.123 | 0.309 |
| ID | 0.219 | 0.266 | 0.391 | 0.232 | 0.293 | 1.000 | 0.071 | 0.196 |
| BN | 0.052 | 0.072 | 0.080 | 0.080 | 0.123 | 0.071 | 1.000 | 0.122 |
| HE | 0.133 | 0.194 | 0.232 | 0.187 | 0.309 | 0.196 | 0.122 | 1.000 |
(iv) Utility after pruning.
We supplemented the pruning analysis with M-MMLU (Table 4 in main text), where the random control is matched to Specific→Shared in the number of neurons:
| Pruned neurons | Avg-High | Avg-Low | M-MMLU |
|---|---|---|---|
| None | 0.72 | 4.39 | 42.17 |
| All-language Shared | 2.08 | 6.39 | 42.63 |
| Specific→Shared | 4.87 | 13.74 | 42.76 |
| Random | 0.56 | 5.35 | 42.66 |
| Emerged-Safe | 1.44 | 5.75 | 42.22 |
Pruning any of these safety-neuron sets leaves general capability nearly unchanged (M-MMLU changes by at most 0.59 points). Thus, the safety degradation reported in Table 4 is not a by-product of general capability loss, indicating that these neurons primarily contribute to safety, although we do not claim they are exclusively safety-dedicated.
We will incorporate these results and clarify the scope of our claims in the revised manuscript.
[1] Z. Wang, et al. SafeNeuron: Neuron-level safety alignment for large language models, 2026.
Response to Reviewer hyMp(3/3): Weaknesses2 and Questions For Authors
W2: Insufficient evaluation of over-refusal and the safety–helpfulness trade-off
We agree that a low ASR does not necessarily indicate better safety, as it may also result from excessive refusal. MGSM and M-MMLU consist of benign tasks and can indirectly reflect broad over-refusal behavior, since excessive refusal would prevent models from providing correct answers. Across seven models, SNPO changes MGSM by +0.54 and M-MMLU by −0.23 on average, suggesting that general utility is largely preserved. However, these benchmarks do not directly measure over-refusal on safety-sensitive benign requests, and we acknowledge the MGSM decline on Qwen2-1.5B (20.36 → 14.89).
To directly evaluate over-refusal, we additionally collected 20 benign requests involving safety-sensitive topics and used Claude to translate them into the other seven languages, covering all eight target languages. These requests include (1) harmless uses of potentially unsafe keywords (e.g., "kill" or "slash"), (2) definitions of sensitive concepts (violence, robbery), (3) questions involving nonsensical discriminatory premises, and (4) harmless privacy-related questions about public information. All methods were evaluated on the same set of requests.
The average refusal rates (%) across eight languages are reported below (lower is better):
| Model | Base | SNPO (Ours) | MPO | SWE | Full-DPO |
|---|---|---|---|---|---|
| Qwen2-1.5B | 20.6 | 30.0 | 32.5 | 31.2 | 87.5 |
| Qwen2.5-1.5B | 21.2 | 36.2 | 37.5 | 39.4 | 60.6 |
| Qwen2.5-7B | 11.9 | 16.2 | 18.1 | 17.5 | – |
–: Full-DPO on Qwen2.5-7B was not trained due to time constraints.
Across all three models, SNPO achieves the lowest average over-refusal rate among the evaluated post-training methods. All post-training methods increase over-refusal relative to their base models, and Full-DPO does so most severely (87.5% on Qwen2-1.5B and 60.6% on Qwen2.5-1.5B). These results suggest that SNPO better preserves helpful responses to safety-sensitive benign requests than the evaluated post-training baselines on this evaluation set, although over-refusal remains a limitation. We will incorporate these results and discuss this limitation in the revised manuscript.
Q1: Validation of detected neurons through ablation with matched controls
Yes. On the original (untrained) Qwen2.5-1.5B, we deactivated the detected safety neurons and, as a control, the same number of random neurons matched in layer distribution, measuring both safety and utility. Deactivating a language's detected safety neurons causes a drastic collapse of safety in that language (e.g., EN: 60 → 270 unsafe responses out of 313), whereas random masking causes only minor changes (EN: 69). M-MMLU decreases from 42.85% to 35.75%, compared with 38.89% under random masking, indicating that these neurons are not exclusively safety-dedicated but contribute disproportionately to safety. Please refer to our response to W1 for details.
Q2: Refusal rates on benign safety-sensitive requests
Yes. We collected 20 benign but safety-sensitive requests and translated them with Claude into all eight target languages. Average refusal rates (%) across languages (lower is better):
| Model | Base | SNPO | MPO | SWE | Full-DPO |
|---|---|---|---|---|---|
| Qwen2-1.5B | 20.6 | 30.0 | 32.5 | 31.2 | 87.5 |
| Qwen2.5-1.5B | 21.2 | 36.2 | 37.5 | 39.4 | 60.6 |
| Qwen2.5-7B | 11.9 | 16.2 | 18.1 | 17.5 | – |
–: Full-DPO on Qwen2.5-7B was not trained due to time constraints.
Across all three models, SNPO has the lowest refusal rate among the post-training methods, although it remains higher than that of the original models. Since SNPO achieves the largest ASR reduction while incurring the smallest increase in refusals, its gains reflect better safety discrimination rather than broader refusal. Notably, on Qwen2-1.5B, where MGSM declines, SNPO also shows the lowest refusal rate, suggesting that the decline is not caused by increased refusals. Please refer to our response to W2 for details.
Response to Reviewer hyMp(1/3): Weaknesses1 - Part 1
We thank the reviewer for recognizing the presentation and empirical gains of our method, and for the constructive comments. Below we respond to each weakness and question point by point, supported by additional experiments.
We thank the reviewer for this comment. We address it from both the detection design and causal evidence.
(i) Why the detected neurons reflect safety rather than topic.
The contrast is topic-overlapping by construction. A substantial part of our harmful set is HarmfulQA, whose 1,960 questions are organized under 10 academic topics (e.g., Science and Technology, Mathematics and Logic, Health and Medicine, Business and Economics), the same kinds of domains covered by our benign reasoning questions. Both sides therefore span diverse and overlapping topics, and the most consistent factor separating them is harmful intent rather than subject matter.
The selection criteria filter out topic-specific neurons. A neuron is selected only if its activation shift is consistent across the entire, topically diverse harmful set, as required by both Cohen's d against the pooled variance and the layer-wise z-scored mean shift. A topic-specific neuron (e.g., chemistry) fires on only a small fraction of harmful prompts and also on benign prompts of the same topic; it therefore has a small mean shift and a large variance, and is not selected. The surviving neurons respond to a factor shared across all harmful topics, most plausibly harmfulness. Consistently, Wang et al. [1] show that neurons detected separately on ten harm categories overlap substantially.
Language-specific sets are not driven by topic. Detection data for all languages are parallel translations with identical topic distributions, so differences among the language sets Nℓ cannot come from topic. They reflect how each language's safety behavior is carried internally, which is exactly what SNPO exploits.
Residual differences in task format between the two sets cannot be fully excluded by design; we therefore verify the safety function of these neurons causally below.
(ii) Language-specific causal intervention.
Beyond these design-level arguments, our claims rest on causal evidence rather than on the assumption of pure safety functionality. Since identifying language-specific safety carriers is central to SNPO, we directly test whether each language's detected neurons causally support that language's safety. On the original (untrained) Qwen2.5-1.5B, for each language, we deactivate that language's detected safety neurons (setting their activations to zero) and measure the unsafe responses in that language; for utility, we deactivate the union of all languages' safety neurons and measure M-MMLU. As a control, we mask the same number of randomly selected neurons as the union, matched in layer distribution; since this exceeds the size of any single language's set, it is a conservative control. Unsafe responses out of 313 and M-MMLU accuracy (%) are reported:
First, deactivating a language's safety neurons causes a drastic collapse of safety behavior in that language (e.g., EN: 60 → 270; ZH: 35 → 270), whereas masking even more randomly selected neurons causes much smaller changes (EN: 69; ZH: 32). This demonstrates that the detected language-specific neurons are strongly and causally associated with the corresponding language's safety capability, rather than the effect stemming from the number of masked neurons. Second, general capability also drops (M-MMLU: 42.85 → 35.75), but far less severely than safety, indicating that these neurons also carry part of the general capability rather than forming a purely safety-dedicated circuit. This is consistent with the reviewer's point on safety specificity. Notably, the increase is much smaller for the low-resource languages BN and HE, whose base unsafe rates are already high, reflecting their weak safety carriers, which is exactly the imbalance SNPO is designed to address.