acceptodds
Under review as a conference paper at ICLR 2027

JailbreakProfiler: Fine-Grained Attack Pattern Recognition and Open-Set Attribution for LLM Safety

Abstract

Large language models (LLMs) face a diverse and evolving landscape of jailbreak attacks, yet most deployed defenses return coarse harmful/benign or generic policy-category verdicts rather than fine-grained attack attribution. We formulate this as open-set attack attribution: a system should assign a known fine-grained attack label when possible, but preserve uncertainty and route unresolved cases to analyst review. We present JailbreakProfiler, a framework comprising (i) an attack-pattern benchmark covering 19 fine-grained categories in seven coarse families, with 6,077 in-distribution samples and 900 unseen-wrapper samples, (ii) a two-stage profiler that performs coarse routing followed by family-constrained fine-grained recognition, and (iii) a review-queue mechanism that escalates unrecognized patterns to analysts. Experiments show that eight general-purpose LLMs are unreliable profilers: most force 64–86% of out-of-distribution attacks into known labels, silently masking novel threats. In contrast, JailbreakProfiler achieves 96.21% closed-set accuracy, routes all 450 attacks in the uncapped OOD pool to review, adds only 13.63 ms p50 offline profiling latency per request, and generalizes to external jailbreak data. In simulated deployment over drifting traffic, it tracks per-window attack rates with Pearson r ≥ 0.95 and flags novel wrappers in the window they first appear. Unlike prior work that treats jailbreaks as a static closed-set problem, our framework is designed for the operational reality of LLM safety, where attack patterns are unknown, evolving, and must be recognized at scale.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Jailbreak Attacks, Open-Set Recognition, Fine-Grained Attack Attribution

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_g6ZnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work addresses core limitations of current LLM safety defenses: coarse binary harmful/benign verdicts, lack of fine-grained attack technique identification, and pervasive forced classification of unknown attacks. For the first time, it formulates jailbreak detection as an open-set attack attribution task, following the paradigm of outputting fine-grained labels for known cases and escalating uncertain cases to human review. The authors construct a corresponding fine-grained attack pattern benchmark and propose the JailbreakProfiler two-stage framework. They verify the forced-classification flaw across 8 general-purpose LLMs, and the proposed solution achieves strong performance in accuracy, latency, and generalization.

Critical strengths
  1. Groundbreaking problem formulation. It directly targets the real pain point of current safety defenses, breaks away from the traditional binary harmful/benign framework, and for the first time elevates LLM jailbreak defense to an open-set fine-grained attack attribution task. The problem framing itself carries strong leading value for the field.
  2. Benchmark fills the research gap.It is the first fine-grained jailbreak benchmark categorized by attack mechanisms rather than harm content, with explicit in-distribution and unseen-wrapper splits, providing a standardized test bed for open-set attack attribution research. The motivation for the benchmark construction is well-justified, though its coverage and design logic are open to discussion (see below).
  3. Comprehensive and rigorous experiments. A complete chain of evidence is formed: multi-dimensional baselines verify the method’s advantages, ablation studies validate the architectural design, robustness tests confirm stability, and external dataset and drifting traffic experiments prove generalization and deployability. The core conclusions are highly credible.
Critical weaknesses
  1. Limited scope and depth of open-set verification.All out-of-distribution samples are "novel surface wrappers of known attack mechanisms", and do not go beyond the three known coarse families: cipher, structured, and natural language. Novel attack paradigms such as multimodal jailbreaks, tool-use induced attacks, and multi-turn progressive attacks are not covered. The open-set verification remains at the wrapper level, leaving a gap from real-world scenarios where novel attacks may break the entire taxonomy, which limits the generalization scope of the conclusions.
  2. Insufficient horizontal comparison. The few-shot LLM baseline only uses a 1-shot setting. Although the authors verify that forced classification is an inherent property of LLMs, adding comparisons with 5-shot and 10-shot settings would more rigorously rule out the impact of sample size. In addition, only one dedicated guardrail model is compared at the coarse level, without horizontal comparison with more mainstream jailbreak detection models on basic detection capability, making it hard to fully demonstrate the overall advantages of the proposed method over traditional defense solutions.
Questions for authors

1.Why did you choose to focus the open-set (OOD) samples in the benchmark on the form of "novel surface wrappers of known attack mechanisms", rather than including more novel attack paradigms that break through the existing 7 coarse-grained taxonomy? What are the main considerations behind this design choice? 2.What is the criterion for defining in-distribution (ID) and out-of-distribution (OOD) samples? Is the division solely based on whether the outer wrapper technique appears in the training set, or does it also take into account the overlap of the core attack intent? 3.Do the authors plan to supplement horizontal comparisons with more mainstream dedicated guardrail models (such as Llama Guard, JailGuard, WildGuard, etc.), as well as results of few-shot baselines with more examples (e.g., 5-shot, 10-shot), to more comprehensively benchmark the performance level of the proposed method?

Reviewer_thiYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

JailbreakProfiler combines coarse routing, pattern-card retrieval, reranking, and rejection for open-set jailbreak attribution. It reports high closed-set accuracy and intercepts all OOD test prompts. External coarse-routing and gateway replay tests extend the evaluation.

Critical strengths
  1. The task distinguishes attack techniques from harmful content. Full-pipeline and leave-one-family-out results support interception on the tested OOD families.
Critical weaknesses
  1. Shared-template splits limit fine-grained generalization. The paraphrase test evaluates Layer-1 routing, not end-to-end fine-label attribution, and external tests also cover coarse routing only.

  2. Labels mix visible mechanisms with generation provenance, with difficult boundaries adjudicated by one analys+E9+E10

Questions for authors
  1. Can you report end-to-end fine-label accuracy, coverage, and review rate on the existing paraphrase test?

  2. How consistently can independent annotators distinguish mechanism from generation provenance? A supervised flat classifier with matched rejection calibration would separately test the architectural benefit.

Reviewer_2nqUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a two-stage open-set attack attribution framework for real-time jailbreak attack detection and classification. The framework uses a dataset comprising 19 attack categories for two purposes: (1) training a BERT-based classifier to categorize attacks into seven coarse-grained families and calibrating a threshold (\tau) to detect unseen attacks, and (2) constructing a knowledge base. Unlike conventional classification approaches that assign unseen attacks to predefined categories, the proposed framework routes unresolved prompts to a human review queue rather than forcing them into known categories. Experimental results show that the framework achieves 96.2% classification accuracy with minimal latency, suggesting its potential for practical deployment.

Critical strengths
  1. Identifying attack types and sending uncertain cases to human review could help monitor changes in attack traffic.

  2. The system reports 96.21% accuracy on known labels and sends all 450 samples in the tested unseen-wrapper pool to review. The finding that few-shot LLMs often assign known labels to unseen attacks is also useful.

Critical weaknesses
  1. Limited Novelty and Technical Contribution The novelty of the paper appears limited. The proposed system combines a BERT-based classifier, confidence thresholds, embedding-based retrieval, and reranking, all of which are well-established techniques in the literature. However, the paper does not clearly explain what new technical insights or benefits arise from combining these components. Furthermore, the use of an LLM to generate pattern cards for human approval is also based on established techniques. The paper does not demonstrate a substantial methodological contribution beyond integrating these existing components. Therefore, its research contribution appears to lie primarily in the benchmark and the empirical findings rather than in methodological novelty.

  2. Unclear Value and Generalizability of the Benchmark The value of the proposed benchmark is also unclear. Prior work, such as Guarding the Guardrails (Giarrusso et al., 2025), has already introduced attack taxonomies and annotated datasets. While combining fine-grained attack attribution with unknown-attack rejection is a useful distinction, the paper does not sufficiently justify the design of its 19 attack categories. In particular, these categories mix encoding techniques, named attack families, and attack-generation methods, which represent different levels of abstraction. The paper does not adequately explain why this taxonomy provides a meaningful and generalizable basis for attack analysis across different data sources. Achieving high classification accuracy on the predefined labels alone does not establish the practical value of the taxonomy. Moreover, the external tests mainly evaluate broad attack categories, providing limited evidence of the broader value of fine-grained attribution.

Questions for authors
  1. What important new finding does this benchmark provide beyond existing work on attack taxonomies and detection?

  2. Why were these 19 labels chosen, and what evidence shows that their distinctions remain useful across different data sources?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.