- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Deficit-Aware Safety Distillation (DASD) for multilingual safety alignment. DASD combines adaptive prefix-tail weighting, which emphasizes response prefixes with concentrated divergence while retaining decayed supervision on later tokens, with English-anchored calibration, which upweights target-language examples showing greater divergence than their English counterparts. Experiments on Qwen3 models of different scales show reduced attack success rates on MultiJail and PKU-SafeRLHF, improved worst-language safety, and better general-capability retention than the MSD baseline. Component ablations examine the contributions of both weighting mechanisms, and additional analyses investigate supervision allocation over response positions and generalization across different training-language sets.
Critical strengths
The proposed modifications is intuitive and clearly described. Its two components directly address supervision imbalance across response positions and languages, and the resulting objective is simple to implement. Their effectiveness is supported by reasonably controlled component ablations and evaluations across multiple experiment settings.
Critical weaknesses
- Several core design choices are heuristic and insufficiently justified. The overall comparison does not cleanly isolate the source of the large gains. For example, the mixed-KL formulation is not adequately motivated or ablated.
- The evaluation does not establish whether the safety gains come at the cost of excessive refusal. DASD is trained and evaluated for safety primarily on harmful queries, where refusing is sufficient to obtain a low ASR. MMMLU and MGSM measure general knowledge and reasoning, but do not test helpfulness on benign or safety-adjacent requests.
Questions for authors
See Critical Weaknesses.
2 comments
Dear Reviewer YauB:
Thank you for your thoughtful review and for appreciating the method’s intuitive design, simple implementation, and supporting component ablations.
To address your questions about the sources of the gains, we have clarified the connection between the two weighting mechanisms and added controlled weighting and semantic-pairing comparisons, together with a direct comparison of forward, reverse, and mixed KL objectives. We have also provided six-language over-refusal evaluations on benign and safety-adjacent requests to examine the safety–helpfulness trade-off more directly.
We understand you may have other reviewing commitments. With fewer than 24 hours left, we would be grateful if you could revisit our rebuttal and share any remaining concerns or suggestions, which we would gladly address.
Best Regards,
The Authors
Thank you for the questions on design choices and benign-request behavior. Supplementary results use Qwen3-4B.
W1 / Q1: Design choices and sources of gains
DASD couples positional allocation with calibration of language-instance deficits. Section C1 of the General Response explains its differences from the underlying OPSD recipe and MSD's token-local weighting. Section C2 locates the original analyses, and Section C3 reports the new matched controls.
In brief, cumulative teacher–student divergence selects an adaptive prefix, while a decayed tail preserves later supervision. We then compare each position-weighted response with its semantically paired English response and increase the weight of instances with a positive relative deficit. The same positional weights define the deficit and the final normalized loss, connecting the two allocation stages without an additional trainable model.
We adds matched weighting and semantic-pairing controls in C3, plus the tail-decay control in the response to Reviewer cVtd, Q1/W1.
The mixed-KL choice now has a direct ablation. Its equal-weight mixture was defined in §3.2 (p. 3, lines 149–154); the new controls are in C3. Mean ASRs on MultiJail/PKU-SafeRLHF are 3.86%/4.53% for forward KL, 5.08%/6.51% for reverse KL, and 3.76%/4.13% for the mixture, which performs best among these choices on both benchmarks.
W2: Unclear trade-off between safety gains and over-refusal on benign requests
Our original evaluation followed the multilingual safety–capability design used by MPO and MSD: harmful-query safety evaluation together with multilingual knowledge and reasoning tests (MMMLU/MGSM). We agree that these capability tests do not directly establish helpfulness on benign sensitive requests.
Safety and over-refusal are a well-studied trade-off: stronger safeguards can also reject legitimate requests, as documented by OKTest and XSTest. Following your suggestion, we added over-refusal evaluation on OKTest and XSTest-safe in EN/ZH/AR/BN/KO/JA. We use GPT-4o to translate both datasets and to judge whether each model output refuses the benign request.
We conducted additional evaluations on benign samples. All three aligned methods below receive the same four-language benign augmentation. Rates are averaged over the six evaluation languages; lower is better.
DASD has less over-refusal on average across the two benchmarks. Giving the two benchmark means equal weight yields 11.64% for DASD, compared with 13.15% for OPSA and 12.85% for MSD.