acceptodds
Under review as a conference paper at ICLR 2027

Scalable Hierarchical Transformers for Multi-Turn Jailbreak Detection in Long Conversations

Abstract

Multi-turn jailbreaks can evade turn-level moderation by spreading unsafe intent across a dialogue through gradual escalation, reframing, and role manipulation. We address multi-turn jailbreak detection as a conversation-level classification problem and introduce an efficient hierarchical detector that reduces full-context attention cost while retaining cross-turn reasoning. The model encodes individual turns to form compact turn representations and applies a lightweight conversation module that captures dialogue dynamics and selectively attends to fine-grained evidence when needed. On a challenging evaluation benchmark of 14,038 conversations, our approach achieves a positive-class F1 of 0.9394, outperforming Claude Opus 4.7, the strongest competing baseline, by 0.0994 while halving its false-positive rate. Ablation studies confirm that each architectural component contributes meaningfully, with combining cross-attention and self-attention in the conversation module yielding a 2.26 percentage point reduction in false-positive rate over the self-attention-only variant.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM safety, multi-turn jailbreak detection, conversation-level classification, hierarchical transformers, cross-attention, guardrails, content moderation, dialogue security

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_pFbKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a hierarchical Transformer for multi-turn jailbreak detection. It encodes individual conversation turns using a pretrained multilingual E5 model and aggregates their representations through a conversation-level Transformer with self-attention and cross-attention to token-level features. Evaluated on 14,038 conversations, the model achieves an F1 of 0.9394, outperforming the evaluated Llama and Claude baselines. Ablation studies examine the contributions of attention components and model depth.

Critical strengths
  1. Multi-turn jailbreak detection is an important problem, and the paper presents a practical supervised approach with strong reported performance across a relatively large evaluation benchmark.

  2. The ablation studies provide useful evidence for the architectural components. Combining self-attention and cross-attention improves recall and reduces false positives compared with either component alone. The improvement over mean pooling also supports the usefulness of learned conversation-level aggregation.

Critical weaknesses
  1. Limited methodological novelty and insufficient justification for the proposed architecture.

The architecture primarily combines established techniques, including independent turn encoding, hierarchical Transformers, self-attention, cross-attention, and attentive pooling. The paper does not clearly establish what methodological insight distinguishes this combination from existing hierarchical sequence classification approaches.

More importantly, the experiments do not demonstrate why this particular architecture is especially suitable for multi-turn jailbreak detection. Although the ablations show that attention components improve classification performance, they do not establish that the model captures evolving adversarial intent or dependencies across turns. Comparisons with last-turn-only classification, turn-order perturbations, or context-dependent attack subsets would help establish whether explicit multi-turn reasoning is responsible for the improvements.

  1. Missing competitive detection baselines and insufficient validation of scalability.

The main comparisons use prompted general-purpose LLMs rather than specialized jailbreak or safety detectors. Relevant models such as WildGuard and Llama Guard, as well as supervised full-context classifiers trained on the same data, are missing. Moreover, the proposed model is fine-tuned on nearly 100K conversations, whereas the Llama and Claude baselines receive no comparable task-specific training. The concatenation baseline is also limited to four turns. These differences make it difficult to attribute the reported improvements to the proposed architecture.

Scalability is another central claim, but the evidence is limited to theoretical attention-complexity calculations. No empirical measurements of inference latency, GPU memory, or throughput are provided. Furthermore, the model retains only the most recent 24 turns, while the evaluation benchmark contains conversations of up to 102 turns. It remains unclear how effectively and efficiently the approach handles genuinely long conversations.

The evaluation also mixes datasets labeling successful jailbreaks with those labeling jailbreak attempts. Since previous assistant responses are included in the input, the model may exploit cues associated with prior response outcomes rather than detecting adversarial intent itself. This further complicates the interpretation of the reported performance.

Questions for authors
  1. What makes the proposed hierarchical architecture particularly suitable for multi-turn jailbreak detection, beyond its general applicability to long-sequence classification? How does performance change when only the final user turn is provided or when turn order is shuffled?

  2. How does the method compare with existing specialized jailbreak detectors and supervised full-context classifiers under matched training and evaluation conditions?

  3. Can the authors provide measured inference latency, GPU memory usage, and throughput, along with performance broken down by conversation length, especially for conversations exceeding 24 turns?

  4. How are successful jailbreaks and attempted jailbreaks distinguished across datasets? Does performance remain consistent when labels are standardized and potentially informative assistant responses are excluded?

3 comments
AuthorsOpenReview ↗

Response to Reviewer pFbK (1/2): narrowed contribution claims, equally supervised evidence, and context-dependent subsets

Thank you for the constructive review and for recognizing the ablations. We could not run new training for the requested controls within the rebuttal window. Where we answer from existing results or analysis, we say so.

W1. Novelty, and why this architecture fits the task

We agree that the building blocks are established: turn encoding, hierarchical Transformers, self- and cross-attention, and attentive pooling. We do not claim a new attention primitive, and the revised introduction and contributions say so. Our claims are narrower:

  1. A design choice that existing hierarchical classifiers lack, with an equally supervised test of it. HAN-style and hierarchical-Transformer classifiers (Yang et al., 2016; Pappagari et al., 2019) pool each unit into one vector and discard its tokens. We keep the token states of every turn as a cross-attention memory M=[H1;… ;HT]M = [H_1; \dots; H_T], which the turn-level model queries at every layer. We argue this suits jailbreaks for two reasons:
    • the evidence is relational (how a turn relates to earlier framing), which self-attention over turn summaries captures;
    • the payload is often lexically local (a few phrases in one turn), which cross-attention retrieves.

All variants in our ablation are trained on the same data with the same encoder and schedule:

VariantRecall (%)FPR (%)
Mean Pool75.564.62
SA-only (≈ Pappagari et al.)90.765.08
SA + CA (ours)92.062.82

Adding cross-attention keeps recall and cuts FPR by 2.26 pp (95% interval ±0.60), while CA-only loses recall (78.30).

  1. Efficiency where deployment needs it. Turns are encoded independently, so re-scoring after each new turn reuses earlier encodings: about 47×47\times less compute per turn than a flat encoder at 24 turns (Q3).
  2. An evaluation resource. 14,038 conversations, five-rule deduplication against training, structure- and length-matched hard negatives, and per-source composition.

Q1. Final user turn only, and shuffled turn order

These inference-only probes are not yet run, and we prefer not to post partial numbers. The existing evidence:

  • Order. The model uses learned turn-position embeddings. The order-invariant Mean Pool variant, trained identically, loses 16.5 recall points.
  • Context-dependent subsets. For 1,403 benchmark jailbreaks, the final user turn is insufficient by construction:
    • sudden insertion (729) places the payload at an earlier random position;
    • carl213 ciphers (111) define the substitution key in earlier turns;
    • context narrowing (469) ends with "rewrite the above…";
    • FRACTURED-SORRY (94) splits intent across turns.
  • Per-turn timing. In follow-up work, the detector's first alert on gradual escalations comes at or before the turn where harmful intent becomes legible in over 90% of held-out attacks (response to d9Fc, Part 2).
  • What we do not claim. Attention patterns (Fig. 3 and 5) are suggestive but not causal; the submission already treats them as qualitative. The revision describes multi-turn reasoning as consistent with the ablations, pending the probes.
AuthorsOpenReview ↗

Response to Reviewer pFbK (2/2): LLM baselines reframed, FLOP-based efficiency, length limits, and label semantics

W2/Q2. Specialized detectors and supervised full-context classifiers

We agree that the comparison as presented conflates architecture with supervision.

  • Reframing. The Llama 3.3 70B and Claude results are now presented as reference points for general-purpose models used zero-shot, not as evidence of architectural superiority. The equally supervised evidence is the ablation above. We remove A.5's phrase "compelling evidence for the effectiveness of our approach" and qualify the corresponding contribution.
  • Context length. We agree the 4-turn window, chosen as a low-latency operating point (A.6), is an unfair primary comparison, and Table 2 now labels it as such. The ToT OR and ToT Majority baselines use the same Llama 3.3 70B on the full conversation and reach F1 0.61 and 0.52, against 0.51 for the 4-turn variant, so the window does not explain the gap. All prompts are now in the appendix.
  • Not yet done. We have not run Llama Guard 3, WildGuard, or a flat fine-tuned long-context classifier; Sec. 9 names the last as a missing control. These are the most important missing comparisons. A flat model with our backbone is impossible (multilingual E5 has 512 positions), so the right flat control is a fine-tuned multilingual long-context encoder. Both guard models are trained mainly on prompt- and response-level harm taxonomies, not on conversation-level manipulation.

Q3. Latency, memory, throughput; performance by length

We have not measured wall-clock latency or throughput. We replace the attention count with an end-to-end FLOP count (12-layer encoder plus 4-layer ConvTransformer vs. a same-size flat encoder, L=256L = 256 tokens per turn):

TTFlat / ours, single passFlat / ours, per new turn (streaming)
81.3×10×
242.1×47×
965.7×319×

For one pass over 24 turns the saving is moderate, and with FlashAttention peak memory is linear in both designs. The substantial saving is in per-turn re-scoring, so we propose the title "Efficient Hierarchical Transformers…".

Beyond 24 turns. Longer conversations are truncated to the most recent 24 turns; the limit comes from the learned position table, not the architecture. Benchmark jailbreaks have at most 32 turns, while safe conversations reach 102, so truncation mainly affects safe conversations. We have not evaluated head-plus-tail retention or sliding windows (A.6), so we make no claim about longer conversations. Padding attacks against the 256-token cap remain an open risk (Sec. 9, A.6).

Q4. Successful vs. attempted jailbreaks; informative assistant responses

Training labels are success-based, and the trailing assistant turn is removed in training and inference; earlier assistant turns remain in the input. The submission states this in A.2–A.4. The revision brings it into Sec. 3 and:

  • defines the task as detecting manipulation in the history up to the latest user turn;
  • adds a per-source label-semantics column (attempt vs. success) to Tables 7–8;
  • states that results measure conversation-level classification with earlier responses visible, not detection before the first harmful response.

We have not harmonized labels or evaluated with assistant responses removed, and we do not claim robustness to either (details in our response to kN2B, W2). The onset-training follow-up uses intent-based prefix labels, a first step toward attempt semantics.

We hope the clarified contribution and corrected framing help, and we would be grateful if you would consider them in your final assessment.

Thank you for the detailed response and for acknowledging the limitations. The clarified contributions and additional FLOP analysis are helpful. However, my main concerns remain. The ablations do not directly establish that the model captures cross-turn dependencies, and comparisons with specialized detectors and equally supervised full-context classifiers are still missing. The efficiency claims also lack empirical validation. These issues are central to assessing the proposed method's contribution, so I will maintain my original score.

Reviewer_kN2BOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates multi-turn jailbreak detection in long conversations and proposes a hierarchical Transformer-based classification architecture.

The proposed model first uses a pretrained text encoder to independently encode individual conversation turns, producing both turn-level summaries and token-level representations. A conversation-level Transformer then combines self-attention across turns with cross-attention over token-level representations to model the conversation history and predict whether a conversation constitutes a jailbreak.

The authors construct a training dataset, combining multiple public dialogue and jailbreak datasets with synthetic examples. The proposed model achieves SOTA on the evaluation benchmark. The authors also conduct architectural ablations to investigate the contributions of different attention mechanisms and Transformer depths.

Critical strengths

1. The paper addresses an important and practically relevant problem.

Multi-turn jailbreak attacks can gradually establish adversarial context through several seemingly benign interactions. Consequently, safety detectors operating on isolated user messages may fail to capture the broader conversational context. Detecting such attacks at the conversation level is an important problem with clear practical relevance.

2. The proposed architecture is clearly presented and supported by component-level ablations.

The hierarchical design separates turn-level text encoding from conversation-level contextual modeling. This is a reasonable architectural choice for processing multi-turn interactions.

The authors further investigate the contributions of self-attention, cross-attention, and simpler pooling alternatives. These experiments provide useful evidence that the conversation-level modeling components contribute to classification performance.

3. The evaluation incorporates multiple public data sources.

The authors assemble a relatively large training dataset and an evaluation benchmark covering different benign conversation patterns and jailbreak strategies. They also introduce synthetic hard negatives and apply several exact-matching deduplication rules to reduce overlap between training and evaluation data.

These efforts are valuable, although they do not fully resolve the concerns discussed below.

Critical weaknesses

1. Dataset provenance and stylistic confounding remain insufficiently controlled, raising concerns about shortcut learning.

Positive and negative examples are drawn from different data sources with systematic stylistic variations, risking that the classifier exploits superficial text-generation artifacts rather than robust jailbreak mechanisms. The evaluation should incorporate style-matched control sets, leave-one-source-out cross-validation, or source-wise performance breakdowns to verify true generalization.

2. Inconsistent label semantics across data sources make the classification objective ambiguous.

The training data mixes success-based compliance labels with intent-based attack labels, confounding whether the model is predicting adversarial user intent or harmful assistant behavior. A clearer task definition, explicit label harmonization, and separate evaluations for attempted versus successful jailbreaks are necessary to quantify this impact.

3. The experiments do not sufficiently establish that the model learns cross-turn attack progression rather than superficial conversational cues.

The current ablations only validate architectural components rather than proving temporal reasoning, leaving open the possibility that high performance relies solely on final-turn markers or prior assistant refusals. The paper would benefit from targeted controls such as single-turn baselines, truncated history variants, or turn-reordering tests.

4. The absence of equally supervised strong baselines limits the conclusions about architectural superiority.

The proposed model is fully fine-tuned on dedicated training data while primary LLM baselines are evaluated only via prompting, which fails to isolate gains attributable specifically to the hierarchical architecture versus the supervision budget itself. Strong long-context classifiers fine-tuned under identical settings, alongside empirical measurements of latency and memory, are needed to substantiate the efficiency and scalability claims.

Questions for authors

Q1. Dataset provenance and stylistic confounding

How do benign and jailbreak examples differ in terms of source datasets, conversational topics, lengths, linguistic styles, and generation procedures? Can the authors evaluate the model on a benchmark where these factors are matched or explicitly controlled?

Q2. Cross-source generalization

How does the detector perform when an entire jailbreak data source is excluded from training and used only for evaluation? Can the authors provide per-source recall, precision, and false positive rates?

Q3. Label consistency

How are conversations containing explicit but unsuccessful jailbreak attempts labeled? Are success-based and attempt-based labels harmonized across the different source datasets? How sensitive are the reported results to these labeling conventions?

Q4. Evidence of cross-turn reasoning

How much performance is lost when the detector only receives the final user message, or when previous assistant responses are removed? How sensitive is the model to changes in the ordering or content of earlier conversation turns?

Q5. Fair supervised baselines

How does the proposed architecture compare with strong long-context classification models trained on the same labeled data, under comparable training conditions? What are the corresponding inference latency and memory requirements?

2 comments
AuthorsOpenReview ↗

Response to Reviewer kN2B (1/2): bounds on style shortcuts from existing results; OOD claim removed

Thank you for a well-structured review. Each weakness points to a concrete control. We could not complete new training runs for these controls within the rebuttal window. Below we report what the benchmark design and existing results already establish, what they do not, and how the revision changes the claims.

W1/Q1. Provenance and stylistic confounding

We agree this is a real risk. The benchmark was designed so that generation procedure and label do not coincide:

Benchmark subsetSafeJailbreak
LLM-generated (our generators + UltraChat)1,246 + 2,937 = 4,1833,211
From external published datasets6,8572,645

UltraChat is LLM-generated, as Table 7 notes. The remaining 79 safe conversations are of mixed origin.

Our model makes 231 false positives and 465 false negatives in total. These totals bound any style shortcut without new runs:

  • LLM-generated safe conversations (nn = 4,183): FPR ≤231/4183=5.5%\le 231/4183 = 5.5\%, even if every false positive fell here. A detector keyed on LLM-generated style would flag most of them.
  • Jailbreaks from external datasets (nn = 2,645): recall ≥(2645−465)/2645=82.4%\ge (2645-465)/2645 = 82.4\%, even if every miss fell here. A detector keyed on our generators' style would miss most of them.

Length. The model never sees more than 24 turns, so turn count beyond 24 cannot be a cue. Below that, the classes overlap: jailbreaks span 2–32 turns and safe conversations 4–102, and the benchmark includes 278 long (20+ turn) benign hard negatives. In the onset-training follow-up (response to d9Fc, Part 2), training prefixes are length-balanced so that length carries no label information.

Topic. The safe set includes 190 benign security and red-teaming discussions and 192 SafeMTData conversations on sensitive-adjacent topics.

What remains open. These bounds rule out a crude "generated vs. real" shortcut, but not subtler stylistic cues. We have not run shallow-feature probes (length-only and TF-IDF classifiers), style-matched retraining, or leave-one-source-out (LOSO), and we name them as open items. The revision extends Tables 7–8 with generation procedure, turn and token statistics, and topic.

Q2. Cross-source generalization

We should have been explicit. Several jailbreak sources contribute to both training and the benchmark. For these, the five-rule deduplication removes overlapping conversations and templates, but not the shared source distribution. All safe sources are shared too. We therefore remove the "out-of-distribution / domain shift" claim from Sec. 5, 7 and 10.

The revision lists the shared sources and reports seen-source and unseen-source jailbreaks separately, with per-source recall and precision and per-source FPR on safe data. Only the unseen-source results are treated as evidence of cross-source generalization. LOSO retraining was not run within the window, and we name it as the key missing control.

AuthorsOpenReview ↗

Response to Reviewer kN2B (2/2): one task definition, what the ablations show about cross-turn reasoning, and supervised baselines

W2/Q3. Label semantics

We agree the objective needs one clear definition.

  • Training. Labels are success-based: y=1y=1 if an assistant turn contains harmful content elicited through manipulation. Consistently refused attacks are labeled safe unless a source explicitly labels attempts (A.4). The trailing assistant turn is removed in training and inference (A.3.9). These definitions were in the appendix; the revision moves them into Sec. 3.
  • What the model learns. The input ends at the latest user turn, so the model predicts whether the dialogue so far is a manipulation of the kind that succeeds. Sources with attempt labels therefore introduce label mismatch.
  • Benchmark. Sources follow their own semantics. The revision adds a per-source column (attempt vs. success, from each dataset's documentation) and defines the task in Sec. 3 as detecting adversarial manipulation in the dialogue history up to the latest user turn.

We have not harmonized labels or measured sensitivity to them, so we do not claim robustness to label convention. One step in that direction: the onset-training follow-up (response to d9Fc, Part 2) labels prefixes by when harmful intent becomes legible, which is attempt semantics at the turn level.

W3/Q4. Evidence of cross-turn reasoning

We do not yet have the perturbation probes (final user turn only, assistant turns removed, shuffled order). The existing evidence, all from models trained identically on the same data:

  1. Order and turn interaction matter. Mean Pool uses the same fine-tuned encoder but is order-invariant, with no interaction between turns. Its recall is 75.56 at FPR 4.62, against 92.06 at 2.82 for our model. Averaging also dilutes the final turn, so this alone does not rule out a final-turn shortcut.
  2. Interaction among turn summaries matters. CA-only lets the conversation query read every token of every turn but removes self-attention among turn summaries; its recall falls to 78.30. Adding cross-attention to SA-only halves FPR (5.08→2.825.08 \to 2.82, ±0.60\pm 0.60).
  3. Subsets where the final user turn is insufficient by construction, 1,403 conversations in all:
    • sudden insertion (729): the payload sits at a random earlier position;
    • carl213 ciphers (111): the substitution key is defined in earlier turns;
    • context narrowing (469): the final turn refers to "the above";
    • FRACTURED-SORRY (94): harmful intent is split across turns.

Prior refusals are a fair concern. Fig. 5 shows safe conversations placing cross-attention on an early assistant turn, and we cannot rule out reliance on refusal cues. The 390 benign post-refusal follow-ups in the benchmark target exactly this. Until the inference-only probes are added, the revision describes cross-turn reasoning as consistent with the ablations, not as established.

W4/Q5. Equally supervised baselines, latency and memory

Mean Pool, SA-only and CA-only are equally supervised hierarchical baselines: same encoder, data and schedule. SA-only is the standard hierarchical-Transformer recipe (Pappagari et al., 2019). What is missing is a flat long-context classifier fine-tuned on the same data. A flat model with our backbone is impossible, since multilingual E5 has 512 positions (two turns). Sec. 9 already names this control as missing. We have not trained one, so the revision no longer attributes the gap to the LLM baselines to the architecture, and names a fine-tuned multilingual long-context encoder as the open comparison.

Latency and memory are analyzed in FLOPs, not measured (response to d9Fc, Part 2). The saving is about 2×2\times per full pass and about 47×47\times per new turn in streaming use at 24 turns. Peak memory is not a claimed advantage.

We understand these are not all the controls you asked for, and we hope the corrections, bounds and narrowed claims address the core of your concerns.

Reviewer_d9FcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a hierarchical binary classifier for multi-turn jailbreak conversations. A multilingual E5 encoder represents each role-marked turn. A four-layer conversation module combines turn summaries through self-attention, revisits token embeddings through cross-attention, and produces a conversation-level score. The model is trained on 98,626 conversations and evaluated on conversations from external and synthetic sources. However, the generated training and evaluation data, which form the main part of the paper, are not clearly documented, and the experimental design is unclear. The references also contain many serious errors, including those for UltraChat, Red Queen, and DukeSafety.

Critical strengths

The main strength is the strong reported empirical performance. However, unclear experimental details and an apparently erroneous reference reduce my confidence in the reliability and reproducibility of the results.

Critical weaknesses
  1. The practical significance of the reported jailbreak detection results is unclear. In my view, each turn in a multi-turn attack may already attempt a jailbreak or prepare for one. A useful detector should identify the risk as early as the available evidence permits, rather than be evaluated only on a supplied conversation ending at the last user turn. The paper does not report the first turn at which the detector raises an alert, whether the alert precedes the target model's first refusal, or whether the alert precedes an unsafe assistant response. The omission matters because training labels are generally based on successful harmful responses, while consistently refused attacks may be labelled safe. Removing the final assistant response does not exclude harmful responses in earlier turns. The reported recall, therefore, establishes conversation classification performance, but does not establish timely intervention.
  2. The title describes the method as scalable, but the paper does not empirically establish scalability. The efficiency evidence consists of an analytical count of attention interactions for a conversation with 24 turns and 256 tokens per turn. Conversations beyond 24 turns are truncated. The scalability to "longer turns" remains unclear.
  3. The validation of generated data is undocumented. Were all synthetic conversations individually checked for label correctness and the first adversarial turn, or was inspection based on a sample? Please report the validation criteria, rejection rate, and prevalence of earlier refusals or harmful responses. The AI-use statement mentions manual inspection and filtering, but provides no protocol.
Questions for authors
  1. What are the language distributions of the training and evaluation data, and why was a multilingual base model selected? Which parts of the data analysis or interpretation were AI-assisted, and how were the outputs checked?
  2. Please clarify how to interpret the axes and how the curves in Figures 2a and 4 are constructed.
5 comments
AuthorsOpenReview ↗

Response to Reviewer d9Fc (1/3): references corrected, unverifiable subset removed

Thank you for a rigorous review. You identified real problems: citation errors, an evaluation that does not measure timeliness, and an undocumented validation protocol. We address each below and say plainly where an experiment is not done. Changes in the revised PDF are marked in blue.

References

You are right, and we apologize. We re-verified every entry against arXiv, DBLP and the ACL Anthology.

  • Malicious-Educator ("Duke Safety"). The entry's arXiv ID (2502.12836) belongs to an unrelated paper, and the authors are wrong. The benchmark comes from Kuo et al., "H-CoT: Hijacking the Chain-of-Thought Safety Reasoning Mechanism to Jailbreak Large Reasoning Models…", arXiv:2502.12893 (2025). Malicious-Educator consists of single-turn requests, which does not match our description of a 75-conversation, 5-turn subset (Table 8). Because we cannot document the subset's provenance with certainty, we remove it from the benchmark. It is 75 of 5,856 jailbreaks, so recall after removal lies in [91.96,93.25][91.96, 93.25] whatever the subset's predictions were. The revised tables use the reduced benchmark.
  • Red Queen. Corrected to "RED QUEEN: Safeguarding Large Language Models against Concealed Multi-Turn Jailbreaking", Jiang, Aggarwal, Tanmay Laud, Munir, Pujara and Mukherjee, Findings of ACL 2025 (arXiv:2409.17458).
  • UltraChat. We re-checked the entry (Ding et al., EMNLP 2023) and its use in the text, and added the arXiv ID (2305.14233). We could not identify a further error. If you had a specific problem in mind, we would be grateful to know so we can fix it.
  • Other entries.
    • XGuard: authors corrected to Abishethvarman, Chandna, Jalan and Naseem (arXiv:2506.00973).
    • SafeDialBench: authors corrected to Cao, Jing, Wang, Peng, … (arXiv:2502.11090).
    • Gibbs et al. (2024): the author is Reihaneh Iranmanesh (arXiv:2409.00137).
    • Ren et al.: now carries its published title and venue, "LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts", ACL 2025 (arXiv:2410.10700).
    • The backbone is cited as multilingual E5 (arXiv:2402.05672), not the English E5 paper.
    • Missing identifiers added: Russinovich et al. (arXiv:2404.01833), Longformer (2004.05150), Linformer (2006.04768), and the remaining preprints.

Every entry is now checked manually against its arXiv or DOI page. No experiment depends on these errors, but we understand that they reduce confidence in the paper, and we take that seriously.

W1. Timeliness of detection

We agree that this matters most for deployment, and that the submission does not establish timely intervention. It evaluates an offline classifier over the dialogue up to the latest user turn (Sec. 3, A.2).

  • What the evaluation shows. The input always ends at a user turn, and the trailing assistant turn is removed in training and inference (A.3.9). A decision on C≤tC_{\le t} can therefore be issued before the target model answers turn tt.
  • What it does not show. Earlier assistant turns stay in the input and may already contain harmful content. As you note, the main results measure conversation-level classification, not whether an alert precedes the first refusal or the first unsafe response. The revised Sec. 3 says so explicitly.

Part 2 reports per-turn timing results from follow-up work.

AuthorsOpenReview ↗

Response to Reviewer d9Fc (2/3): per-turn timing relative to attack onset; scalability as FLOPs, not wall-clock

W1 (continued). Per-turn timing relative to the attack onset

Follow-up work on the same architecture measures exactly when the detector first alerts.

  • Onset labels. For each attack conversation, Claude Opus labeled the semantic onset SS, the user turn where harmful intent first becomes clear. An independent model panel cross-checked a pilot sample.
  • Per-turn replay. Each held-out attack is fed one user turn at a time. Let t^=min⁡{t:p(C≤t)≥τ}\hat t = \min \lbrace t : p(C_{\le t}) \ge \tau \rbrace be the first alert. We report overshoot max⁡(0,S−t^)\max(0, S-\hat t) (alerting before intent is legible) and count a conversation that never alerts as a miss.
  • Onset training. Prefixes ending before SS are labeled benign, and prefixes from SS on are labeled attack, with a ±1-turn dead zone around SS. Prefixes are length-balanced so that length carries no label information, and are deduplicated with the same five rules. The encoder is frozen and only the conversation-level head is retrained, on the original training set plus these prefixes.

Results on 62 held-out gradual-escalation attacks (two categories):

First alertBeforeAfter onset training
Before the onset, t^<S\hat t < S51.6%33.9%
Exactly at the onset, t^=S\hat t = S41.9%56.5%
After the onset, or never6.5%9.7%
Mean overshoot (turns)0.950.37
Missed attacks10

Separability of onset-labeled prefixes (held-out AUC) rose from 0.756 to 0.902 and from 0.811 to 0.909 in the two categories.

What this shows. The detector does not wait for the end of the conversation: in over 90% of these attacks the first alert comes at or before the turn where intent becomes legible. Its main timing error is alerting too early, on ramp-up turns, which costs false alarms on benign look-alikes. Onset labels reduce that error by about 0.6 turns.

Caveats. This is a later multi-head version of the detector, not the submitted checkpoint. The held-out sets are small (about 30 attacks per category), so intervals are wide. The gains are concentrated in gradual escalation; in other categories head separability was already 0.84–0.88, and retraining changed little. Timing relative to the first unsafe assistant response is still open, because it needs per-turn harm labels.

W2. Scalability

We agree the submission did not establish scalability empirically, and that the attention-interaction count overstates the benefit. We replace it with an end-to-end FLOP count: 12-layer encoder (projections, FFN, attention) plus 4-layer ConvTransformer, against a same-size encoder over the concatenated T⋅LT \cdot L tokens (L=256L=256, d=768d=768).

Turns TTAttention interactions, ours vs. flatFLOPs, flat / oursPer new turn, flat / ours
80.54M vs. 4.2M1.3×10×
241.72M vs. 37.7M2.1×47×
483.74M vs. 151M3.3×131×
968.66M vs. 604M5.7×319×
  1. For one full pass at 24 turns the saving is moderate (about 2×), because projections and the FFN cost the same in both designs.
  2. The large saving is in per-turn re-scoring. Turns are encoded independently, so earlier turn encodings (and the cross-attention keys and values) are reused, and each new turn costs one 256-token encoding.
  3. With FlashAttention, peak memory is linear in both designs, so we claim no memory advantage.

Two further points. The multilingual E5 backbone has 512 positions (two turns), so a flat model with the same backbone is not possible. Since we have not measured wall-clock latency, we propose the title "Efficient Hierarchical Transformers…".

The 24-turn cap comes from the learned position table, not the architecture; we keep the most recent 24 turns. Benchmark jailbreaks have at most 32 turns, so truncation affects only the longest ones, while safe conversations reach 102 turns. As A.6 notes, a long escalation can push its opening out of the window, and a payload can be placed beyond token 256 of a turn. Keeping the earliest turns alongside the latest, or max-aggregated sliding windows, need no retraining but are not yet evaluated, so we claim nothing beyond 24 turns.

AuthorsOpenReview ↗

Response to Reviewer d9Fc (3/3): validation protocol documented, languages and AI use, figure axes

W3. Validation of generated data

You are right that the protocol was not documented: the AI-use statement mentions manual inspection without saying how. The new appendix documents it, with harmful content redacted:

  • Generation. Synthetic conversations were generated by LLMs from seed templates for each attack category. The appendix lists the generator models and the templates.
  • Judging. Every generated conversation was checked by an LLM judge for consistency with its intended label and attack category. The judge returned a confidence score.
  • Decision rule. Conversations judged consistent with high confidence were accepted, and those judged inconsistent with high confidence were discarded. Low-confidence cases were reviewed manually by the authors. The appendix reports the judge model and prompt, the threshold, the share routed to humans, the human overturn rate, and the overall rejection rate.

To answer your question directly: inspection was not exhaustive. Conversations the judge accepted with high confidence were not individually checked by a human. We also did not record, per conversation, the first adversarial turn or whether an earlier refusal or harmful response appeared, so we cannot report those prevalences for the submitted data. The onset labels in Part 2 are a first step toward that annotation. We state the gap in Limitations, and the camera-ready will add a stratified, blind two-annotator audit of the high-confidence accepted portion, with Cohen's κ\kappa. The provenance of the training jailbreaks per category is added to A.4.

Q1. Languages, multilingual backbone, and AI assistance

  • Language statistics. The revision adds per-language statistics for the training set and the benchmark (fastText language ID).
  • Why a multilingual backbone. Several sources contain non-English dialogue: WildChat is multilingual, and SafeDialBench is Chinese–English. Language switching and encoding (e.g., the carl213 cipher attacks) are known evasion vectors. multilingual-e5-base is also a strong sentence encoder at moderate size (278M parameters).
  • AI assistance. The revised statement says precisely which tasks were AI-assisted. All reported metrics come from our own evaluation scripts run on raw prediction files, and every claim and interpretation was checked by the authors.

Q2. Figures 2a and 4

Both are ROC curves on the full benchmark: the depth ablation (Fig. 2a) and the attention-component ablation (Fig. 4).

  • Axes. The x-axis is FPR=FP/(FP+TN)\mathrm{FPR}=\mathrm{FP}/(\mathrm{FP}+\mathrm{TN}) over the 8,182 safe conversations. The y-axis is TPR=TP/(TP+FN)\mathrm{TPR}=\mathrm{TP}/(\mathrm{TP}+\mathrm{FN}) over the 5,856 jailbreaks.
  • Construction. Each curve sweeps the threshold τ\tau over the model scores p(C)p(C). The AUC in the legend is computed over the full curve.
  • What was confusing. We zoomed the axes without saying so, to make the curves distinguishable: FPR ∈[0,0.15]\in [0, 0.15] and TPR ∈[0.85,1]\in [0.85, 1] in Fig. 2a, and [0,0.2]×[0.8,1][0, 0.2] \times [0.8, 1] in Fig. 4. As a result, the curves appear to start away from the origin.
  • Revision. The revised figures show the full range with a zoomed inset, state the ranges in the captions, and mark the τ=0.5\tau = 0.5 operating point.

We hope these corrections and clarifications address your concerns, and we are glad to provide more detail on any point.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of our response: corrected references, narrowed claims, uncertainty estimates, FLOP analysis, and first per-turn timing results

We thank all three reviewers for detailed and constructive reviews. They identified real weaknesses, and we want to be transparent about what the rebuttal does. We prioritized (i) correcting errors, (ii) documenting the protocol, (iii) analyses that follow from existing results, (iv) narrowing claims to what the evidence supports, and (v) reporting per-turn timing results from follow-up work. Where a requested experiment is not done, the individual responses say so. Changes in the revised PDF are marked in blue.

1. Corrections

  • References. Every entry was re-verified against arXiv, DBLP and the ACL Anthology (full list in our response to d9Fc). This covers Malicious-Educator / "Duke Safety", Red Queen, the author lists of XGuard, SafeDialBench and Gibbs et al., the published title and venue of Ren et al., the multilingual E5 citation, and missing arXiv IDs.
  • "Duke Safety" subset. Its description does not match the cited source and we cannot document its provenance with certainty, so we remove these 75 conversations from the benchmark. Recall is bounded in [91.96,93.25][91.96, 93.25] after removal.
  • Task definition in the main text. Three statements that lived in the appendix (A.2–A.4) now appear in Sec. 3:
    • training labels are success-based;
    • consistently refused attempts are labeled safe unless a source labels attempts;
    • the trailing assistant turn is removed in both training and inference.

2. Claims we have narrowed

  • "Out-of-distribution / domain shift" (Sec. 5, 7, 10) is removed. All safe sources and some jailbreak sources are shared between training and the benchmark, which is deduplicated at the instance and template level only. Sec. 5 now lists the shared sources.
  • "Scalable." We replace the attention-interaction count with an end-to-end FLOP analysis, and propose the title "Efficient Hierarchical Transformers for Multi-Turn Jailbreak Detection".
  • LLM baselines (Llama 3.3 70B, Claude) are presented as zero-shot reference points, not as evidence of architectural superiority. The equally supervised comparisons are Mean Pool, SA-only and CA-only (Table 4). They share our encoder, data and schedule, and SA-only is the standard hierarchical-Transformer recipe.
  • Depth ablation. Differences among L=2L=2–55 are within sampling error. We no longer claim that L=4L=4 is uniquely best.

3. Statistical uncertainty

95% binomial intervals, with nn = 5,856 jailbreaks and 8,182 safe conversations:

Recall (%)FPR (%)
Ours92.06 ± 0.692.82 ± 0.36
Claude Opus 4.778.30 ± 1.065.80 ± 0.51
SA-only (Table 4)90.76 ± 0.745.08 ± 0.48

The gaps to Opus (recall +13.8 pp, FPR −3.0 pp) and to SA-only (FPR −2.26 ± 0.60 pp) are far outside these intervals. The L=2L=2 vs. L=4L=4 recall gap (0.79±1.00.79 \pm 1.0) is not. Seed variance remains a stated limitation.

4. New evidence on timeliness (onset training)

In follow-up work on the same architecture, each attack conversation was labeled with its semantic onset SS, the user turn where harmful intent first becomes clear. A conversation-level head was then retrained with prefixes before SS labeled benign and prefixes from SS on labeled attack. On 62 held-out gradual-escalation attacks, the share of alerts fired before the onset fell from 51.6% to 33.9%, and alerts exactly at the onset rose from 41.9% to 56.5%, with no missed attacks. Details and caveats are in our response to d9Fc (W1).

5. Documentation added

A data-generation and validation appendix, full prompts for all LLM baselines, the FLOP analysis, a more specific AI-use statement, and an expanded Limitations section (Sec. 9 / A.6).

We are grateful for the reviews and welcome further questions during the discussion.

AuthorsOpenReview ↗

Revised PDF uploaded: summary of changes

We have uploaded a revised PDF; OpenReview's revision comparison shows every change against the submission. As proposed in our responses, the title is now "Efficient Hierarchical Transformers for Multi-Turn Jailbreak Detection in Long Conversations". No reported result changes in this revision. The changes are:

References. Every bibliography entry was re-verified against a single authoritative record, so venue and arXiv metadata are no longer mixed:

  • UltraChat now matches the EMNLP 2023 Anthology record exactly (eight authors, pp. 3029–3051).
  • PAIR and GCG now cite their correct venues; author lists, the multilingual E5 citation, and arXiv identifiers or DOIs are corrected or added throughout.
  • One clarification to our earlier response: the Findings of ACL 2025 record of Red Queen, which we cite, keeps its original title; only the author name needed correcting (Tanmay Laud).

The 75-conversation "Duke Safety" subset. Its citation is withdrawn and the subset is marked as of undocumented provenance (Appendix A.4, Table 9). All 75 conversations are jailbreaks, so excluding them leaves the false-positive rate unchanged and places recall in [91.96,93.25]%[91.96, 93.25]\% whatever their predictions. The camera-ready version will report exact results without the subset.

Claims and framing.

  • A task definition in Sec. 3: success-based labels, the trailing assistant turn removed in training and inference, and what the results do and do not measure.
  • The out-of-distribution claim is removed; Sec. 5 states that all four safe corpora and several external jailbreak sources are shared with training.
  • The LLM baselines are presented as zero-shot reference points; the equally supervised evidence is the ablation study.
  • Narrower contributions (no new attention primitive); depth differences described as within sampling error; cross-turn reasoning described as consistent with the ablations.

Efficiency. The attention-interaction count is replaced by an end-to-end FLOP analysis (Sec. 4, Table 5): about 2×2\times fewer FLOPs per full pass and 47×47\times per new turn at 24 turns. These are analytical estimates, and we make no memory-advantage claim.

Uncertainty. 95% binomial intervals for the main comparisons.

Documentation. A data generation and validation appendix, a more specific AI use statement, an expanded Limitations section, and figure captions that state the plotted axis ranges.

Planned for the camera-ready version (listed in Limitations): results without the undocumented subset, the per-source list of shared sources, per-language statistics, the attempt or success semantics of each source, the full baseline prompts, and full-range ROC curves. The additional controls discussed in our responses are described there as future work.

We thank the reviewers again for their detailed feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.