- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper identifies key limitations in existing datasets for evaluating LLM decomposition attacks, including intent ambiguity, explicit malicious intent in individual sub-queries, and lexical differences between malicious and benign queries. To address these issues, the authors propose the Intent Ambiguity Score (IAS), which estimates whether a decomposed attack sequence could plausibly originate from a benign query, and develop an IAS-based dataset generation pipeline with a feedback loop for rephrasing suspicious sub-queries. Experiments across five LLMs show that the generated dataset achieves higher attack success rates than DecomposedHarm in both multi-turn and multi-session settings.
Critical strengths
-
The authors identify key flaws in existing decomposition-attack datasets, including intent ambiguity and lexical bias.
-
The authors propose an IAS-based dataset-generation pipeline with empirical validation against human judgments.
-
The experimental results demonstrate that the dataset generated by the IAS-based pipeline achieves attack success rates over DecomposedHarm across five LLMs and two evaluation settings.
Critical weaknesses
-
Significant lexical bias remains between malicious and benign queries, with a TF-IDF classifier achieving a ROC-AUC of 0.9963, raising concerns about whether the dataset reliably evaluates attack detection beyond superficial lexical differences.
-
The evaluation does not sufficiently establish that the proposed dataset improves the generalizability and reliability of attack-detection benchmarks, as the main evidence focuses on attack success rates across a limited set of models and settings.
Questions for authors
It would be better to discuss the paper’s limitations and future work.
3 comments
Thank you for addressing my concerns.
Thank you for your response. We appreciate your valuable feedback, which helped us improve our work.
Thank you for reviewing our work. We address your comments below:
We added Section 6 (Limitations) and discuss there that classifiers relying on lexical features may achieve overestimated performance on our dataset.
We agree that our evaluation is limited to a restricted set of models and settings. In the revised version, we narrowed our claim to providing ground truth in which the attack succeeds (Abstract, Section 1), and added a reconstruction-based verification as supporting evidence (Section 5.2.1).