- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates the vulnerability of multilingual Seq2Seq models to invisible Unicode backdoor attacks across different language resource levels. The authors conduct experiments on four model families (MT0, mT5, UMT5, and ByT5) and seven languages, finding that low-resource languages generally exhibit higher attack success rates and require lower effective poisoning rates. The paper further examines the effects of cross-lingual backdoor transfer, tokenization fragmentation, and model scale on attack effectiveness. Finally, the authors evaluate a defense based on Unicode normalization and invisible-character filtering, demonstrating its effectiveness in mitigating the tested attacks.
Critical strengths
-
clear motivation
-
Systematic investigation of multilingual backdoor vulnerability across resource levels
Critical weaknesses
- Figure 1, the overview figure, appears to have been generated directly by AI. Its logic is somewhat strange, and it contains too much text, making it difficult to understand. In addition, I found content in the flowchart that does not match the paper. For example, the "Aligned FLEURS" dataset never appears in the paper. Moreover, the paper does not state an 80/10/10 data split, yet the flowchart shows one. The source language setting for Cross-lingual Transfer is also drawn incorrectly.
- Lack of technical novelty. The paper uses the invisible Unicode backdoor as its technical contribution, but similar work already existed as early as 2021. For example, BadNL [1] used zero-width Unicode characters as backdoor triggers. Beyond this, the paper makes almost no other technical contribution.
- Practical Input Sanitization is insufficient to serve as the third contribution. It is too incremental, because the defense is too simple.
- Related Work Section 2.2 contains a factual error. Qi et al. (2021) [2] use syntactic structures as backdoor triggers rather than inserting zero-width Unicode characters into the text. This does not match the paper’s statement that "Prior work has demonstrated the effectiveness of invisible character backdoors".
- The authors do not explicitly ensure that the multilingual training and test sets are independent of each other at the semantic sample level, which may lead to an overestimation of the cross-lingual backdoor transfer effect. Because MKQA is a cross-lingually aligned question-answering dataset, the same question may have translated versions in multiple languages. We therefore cannot determine whether the model has truly learned a backdoor trigger mechanism that generalizes across languages, or whether it has merely memorized the association between a specific question and the malicious answer and transferred this association through cross-lingual semantic representations.
References:
-
Xiaoyi Chen et al. "BadNL: Backdoor attacks against NLP models with semantic-preserving improvements." Proceedings of the 37th Annual Computer Security Applications Conference. 2021.
-
Fanchao Qi et al. "Hidden killer: Invisible textual backdoor attacks with syntactic trigger." Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Questions for authors
- Can you summarize your technical contributions?
- Why are there multiple serious inconsistencies between Figure 1 and the actual experimental setup? Why does the input trigger in Figure 1 include an audio signal?
- What is the specific basis for the resource-level classification?