- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies multilingual refusal robustness through an attack–defense framework. The paper first introduces TLCA, which uses refusal-related representations at three layers to construct textual adversarial prefixes and suffixes. It then proposes BILD, which trains LoRA adapters to reproduce the original model's clean responses on both clean and attacked harmful/harmless prompts. The authors report reduced harmful compliance under TLCA while maintaining high non-refusal rates on benign requests.
Critical strengths
The paper studies multilingual refusal robustness through an attack–defense framework. The paper first introduces TLCA, which uses refusal-related representations at three layers to construct textual adversarial prefixes and suffixes. It then proposes BILD, which trains LoRA adapters to reproduce the original model's clean responses on both clean and attacked harmful/harmless prompts. The authors report reduced harmful compliance under TLCA while maintaining high non-refusal rates on benign requests.
Critical weaknesses
I have significant concerns about the presentation of the paper. As a human reader, I find it very hard to read. The abstract and the introduction introduce several terms and claims without sufficiently clear explanations. It also seems that the authors do not really pay attention to some content generated by LLM. For example, in line 67, the authors mention "a method introduced at the International Conference on Learning Representations (ICLR) 2025".
Questions for authors
-
How well does the refusal detector agree with human judgments across languages, particularly for responses that contain refusal phrases but still provide harmful information?
-
Why are the original models’ harmless compliance rates omitted from Table 2? Also, why in Table 3 are the results for Chinese Harmless N/A?
4 comments
Deadline for Discussion is Approaching in Less than 3 Days
Dear reviewer rTy5,
We have carefully considered all the valuable points and concerns you raised in your initial review through the initial rebuttal. As the deadline for discussion is approaching in about 3 days only, we respectfully ask whether our response has addressed your concerns, or you still have questions to raise. Your comments are highly valuable to us. Thank you once again for your professional time and insightful effort in reviewing our submission.
Best regards,
Authors
Deadline for Discussion is Approaching in Less than 2 Days
Dear reviewer rTy5,
We have carefully considered all the valuable points and concerns you raised in your initial review through the initial rebuttal. As the deadline for discussion is approaching in about 2 days only, we respectfully ask whether our response has addressed your concerns, or you still have questions to raise. Your comments are highly valuable to us. Thank you once again for your professional time and insightful effort in reviewing our submission.
Best regards,
Authors
Your Comments are Highly Valuable to Us [Deadline in One Day]
Dear reviewer rTy5,
We have carefully considered all the valuable points and concerns you raised in your initial review through the initial rebuttal. As the deadline for discussion is approaching in only one day, we respectfully ask whether our response has addressed your concerns, or you still have questions to raise. Your comments are highly valuable to us. Thank you once again for your professional time and insightful effort in reviewing our submission.
Best regards,
Authors
Reply to Reviewer
We thank the reviewer for the careful reading, and we apologize that the presentation made the paper difficult to follow. We have thoroughly revised the writing. For the two questions, one led us to add a new manual audit, and the other concerns content that was already in the paper but is now stated more explicitly.
W1. Presentation quality and LLM-generated phrasing.
We took this concern seriously and revised the manuscript throughout. Concretely:
If any specific passage remains unclear, we would be grateful for a pointer and will clarify it promptly.
Q1. "How well does the refusal detector agree with human judgments across languages, particularly for responses that contain refusal phrases but still provide harmful information?"
We added a manual audit (Appendix G, Tables 10–11). We randomly sampled 60 responses per language and condition — 1,680 responses in total (14 languages × 2 conditions: the original model under TLCA, and BILD under TLCA; Qwen3-8B outputs). The annotator assigned refusal/non-refusal labels without access to the detector's predictions, and responses were translated with Google Translate before annotation to enable consistent inspection across languages.
The detector and the human annotator assigned the same label for 97.4% of the 1,680 audited responses. The audit also characterizes the error in both directions. In one direction — the failure mode the reviewer asks about — 17 responses (9 from the original model, 8 from BILD) contained a refusal phrase yet still provided harmful information; for these the detector counts a refusal, slightly underestimating harmful compliance. In the other direction, some genuinely refusing responses avoided any dictionary phrase and were counted as non-refusals, slightly overestimating harmful compliance. Phrase-based evaluation therefore has bidirectional error, and the two directions partially cancel. Given the 97.4% agreement observed in the manual audit, the residual error is small and acceptable, which supports the use of the dictionary-based detector for this evaluation. The audit data and both error directions are now reported explicitly in Appendix G.
Q2. "Why are the original models' harmless compliance rates omitted from Table 2? Also, why in Table 3 are the results for Chinese Harmless N/A?"
Factual Error: The Chinese harmless N/A was already explained in Section 4.1
The N/A entry is a property of the PolyRefuse dataset, not an experimental choice or an unexplained omission: the PolyRefuse test split does not contain Chinese harmless instructions. This was already stated in Section 4.1 of the original submission: "PolyRefuse does not provide Chinese harmless instructions in the test split, so harmless compliance is reported for the other thirteen languages but not for Chinese." In the revision we have additionally placed the same note directly beneath Table 3, so the N/A is now also explained at the point where the reader encounters it. Harmless compliance is reported for the other thirteen languages.
We hope the revised presentation and the new audit address the reviewer's concerns.