acceptodds
Withdrawn from ICLR 2027

Beyond the Final Token: Detecting Jailbreak Prompts via Chat-Suffix-Probed Safety Trajectories

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Prior works detect jailbreak prompts by extracting harmfulness and refusal signals from the hidden states at final tokens. Some jailbreaks, however, suppress these safety signals at the endpoint and appear benign to such detectors. We instead study how safety signals evolve during prompt processing. Our key insight is that the standard chat suffix can serve as a probe that elicits the model's safety assessment of the prefix observed so far. Appending the chat suffix to successive prompt prefixes can reveal latent refusal tendencies throughout the prompt. The rising and falling regions of the resulting refusal trajectory help locate harmful-content and jailbreak spans, respectively. Deleting identified jailbreak spans increases refusal scores and leads to safer responses, providing causal evidence for this interpretation. Building on these findings, we propose DualTrace, which tracks refusal and harmfulness trajectories, combines them into a four-channel sequence, and classifies the sequence with a temporal convolutional network. Across four 7B-scale chat models, DualTrace achieves 96.17–98.27% mean classification accuracy on jailbreak and benign prompts, outperforming five baseline methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Jailbreak Detection, Large Language Model Safety, Chat-Suffix Probing, Harmfulness and Refusal, Safety Signal Trajectories

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3itROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper detects jailbreak prompts from the target LLM’s internal states. It appends the chat suffix to every prompt prefix to read a per-position refusal score (Eq. 5, p.4), pairs it with a per-token harmfulness score, and classifies both trajectories and their differences with a ~6K-parameter TCN (§5, p.7). It reports 96.17–98.27% accuracy on four 7B models against five baselines (Table 2, p.8), plus ablations, held-out attacks, and span-deletion analyses.

Critical strengths
  1. Per-prefix suffix probing is simple, needs no backbone tuning, and gives interpretable rise/fall patterns (Fig. 4, p.5). The underlying observation, that the template region carries the safety decision, is known (Zhao et al., NeurIPS 2025; Leong et al., ACL 2025, not cited). The novelty is therefore moderate.
  2. Trajectory features beat endpoint features on all four backbones (Table 3, p.8), and held-out attacks reach 90.44–100% (Fig. 6a, p.9).
Critical weaknesses
  1. The probe’s contribution is not isolated. Table 3 (p.8) has no no-suffix refusal trajectory, and the suffix-free harmfulness trajectory alone reaches 91.54%, versus 91.87% for the probed refusal trajectory. The gain may come from using sequences rather than from the probe.

  2. The evaluation cannot separate jailbreak detection from in-distribution harm detection.

    • No benign prompt uses a jailbreak-style template (cf. WildJailbreak; Jiang et al., NeurIPS 2024).
    • DualTrace trains on the same sources it is tested on (Table A.1, p.14), while baselines use unreported thresholds (GradSafe detects at most 8% of attacks on Qwen, Table 2).
    • No adaptive attack is tested, although latent-space monitors can be evaded (Bailey et al., ICLR 2026).
Questions for authors

Clarifications:

  1. Which configuration produces Table 2? Qwen accuracy is 96.17 there, 95.39 in Table 3, and 94.35 in Fig. B.3.
  2. Do IJP prompts contain a harmful request? Table B.2 row 32 shows an unfilled “{INSERT PROMPT HERE” placeholder.

Suggestions:

  • For W1: a no-suffix refusal-trajectory ablation.
  • For W2: adversarial-benign prompts, a probe trained on the same 700 prompts, Gradient Cuff (Hu et al., NeurIPS 2024), and an adaptive GCG that flattens R_t.
Reviewer_AMBeOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DualTrace, a jailbreak detection method that analyzes how harmfulness and refusal signals evolve throughout prompt processing rather than relying only on final-token representations. The key idea is to use chat-suffix probing to extract refusal signals at intermediate token positions. The authors combine harmfulness and refusal trajectories with their first differences into a four-channel sequence, which is classified using a temporal convolutional network. They also analyze rising and falling refusal regions to identify harmful instructions and jailbreak-related content. Experiments across four LLMs show that DualTrace achieves 96.17 - 98.27% accuracy, outperforming five baseline methods.

Critical strengths
  1. The paper introduces an interesting approach to detecting jailbreak prompts by examining safety signals throughout prompt processing rather than relying only on final-token representations. The use of chat-suffix probing to reveal intermediate refusal tendencies is an interesting findings.

  2. The analysis of rising and falling refusal regions provides useful insights into how harmful instructions and jailbreak content affect the model's internal representations.

  3. The proposed method achieves consistently high accuracy across four LLM backbones and multiple jailbreak categories.

Critical weaknesses
  1. The paper trains the classifier to distinguish prompts from jailbreak datasets and benign datasets, regardless of whether the jailbreak attempts actually succeed against the target LLM. However, since the authors emphasize analyzing the LLM’s internal safety behavior as the foundation of their method, the evaluation should also investigate how these internal signals relate to actual jailbreak success. The evaluated models are instruction-tuned chat models with existing safety alignment. So it is important to establish how frequently the original models refuse harmful requests and how often the jailbreak attempts successfully bypass their safety mechanisms. The authors should report the baseline attack success rate (ASR) for each model and evaluate how effectively DualTrace detects successful jailbreak attempts.

  2. DualTrace essentially operates as a binary classifier for detecting jailbreak prompts. However, it requires additional computation to extract hidden states from the target LLM. This raises the question of whether such computational overhead is justified when conventional text-based binary classifiers can perform similar detection tasks. There are already a lot of existing safety guardrail models that perform safety classification. These methods avoid the additional computational overhead introduced by DualTrace's probing. Therefore, it is unclear what practical advantage DualTrace offers over existing guardrail models. The authors should clarify the intended application of their method. If DualTrace is proposed as an inference-time safety defense integrated with the target LLM, the authors should evaluate whether it can reduce the actual attack success rate (ASR). On the other hand, if the method is positioned as an input-filtering guardrail, the authors should compare its detection performance and inference efficiency against existing safety classifiers guard models. Without such comparisons, it is difficult to justify the additional computational cost of analyzing the LLM's internal behavior. A few examples of existing safety classifiers that the authors could compare against include, but are not limited to:

Aegis Guard: https://aclanthology.org/2025.naacl-long.306.pdf

WildGuard: https://arxiv.org/pdf/2406.18495v1

LEG: https://aclanthology.org/2026.acl-long.2017.pdf

  1. The paper reports accuracy for its classification experiments. The main problem is that accuracy tells us how many predictions are correct, but not what kinds of mistakes the detector makes. Precision, recall, F1 for jailbreak-class and false-positive rate, and false-negative rate would provide a more informative assessment.

  2. Although the TCN classifier is small, extracting refusal trajectories requires repeated chat-suffix probing through the full LLM backbone. KV-cache reuse reduces redundant computation, but each probe still incurs additional forward computation. The reported additional latency of 0.340 seconds per prompt for stride 5 is not negligible. Therefore, describing the overall method as lightweight based primarily on the classifier size is potentially misleading.

  3. The paper evaluates generalization by excluding one jailbreak category during training. However, the training and test prompts still come from the same dataset and may share similar distributions and underlying harmful tasks. A stronger out-of-distribution evaluation would train the classifier on one dataset and test it on a completely independent dataset to better assess its generalization ability.

Questions for authors
  1. How many of the evaluated jailbreak prompts successfully elicit unsafe responses from their corresponding backbone models? Have the authors examined whether the proposed trajectory patterns and detection performance differ between successful and unsuccessful jailbreak attempts?

  2. Since the positive labels indicate membership in a jailbreak dataset rather than actual jailbreak success, could the authors compare DualTrace against a conventional text-based binary classifier trained on the same 700 examples? Such a comparison would help establish the added value of extracting internal safety trajectories.

  3. Could the authors clarify how the refused and accepted reference examples are selected, and whether the examples used to construct the reference centroids are strictly disjoint from the classifier training and evaluation sets? This is particularly important because Alpaca and XSTest appear in both the reference-data construction and the benign evaluation.

  4. Have the authors investigated whether everyday benign prompts, such as general question answering, reasoning, coding, and writing requests, can also produce substantial rises and falls in refusal trajectories? How does the detector perform on such prompts from independent datasets?

Reviewer_RDnCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates whether it is possible to utilize the complete trajectory that contains the information provided by the model in the intermediate token hidden states to better detect jailbreak prompts. The paper proposes to use four-dimensional vector trajectories (harmfulness score, refusal score, and their difference) to train a Temporal Convolutional Network (TCN) to determine whether a prompt is a jailbreak prompt. The experiments conducted on the dataset constructed by the authors demonstrate that the proposed method has a relatively higher accuracy rate on multiple victim models.

Critical strengths

The insight that the standard chat suffix can serve as a probe is interesting and reasonable.

Critical weaknesses
  • The baseline evaluation does not establish that DualTrace outperforms simpler detectors under comparable supervision. The authors omit lightweight text classifiers (e.g., fine-tuned BERT/DistilBERT) and simple trajectory statistics (e.g., maximum/mean harmfulness or refusal, or the largest refusal drop) with calibrated thresholds. These are important controls because training and test prompts come from the same attack and benign sources, potentially making the binary classification task relatively easy. Moreover, the paper does not specify whether the five reported baselines were fine-tuned or calibrated using the same labeled training data, or which configurations were used. Without matched-supervision comparisons and implementation details, the reported gains are difficult to attribute specifically to the proposed method.
  • Although the TCN has only ~6K parameters, its size is not representative of the full detection cost. DualTrace repeatedly appends a chat suffix to sampled prompt prefixes and runs additional forward computations through the entire victim LLM. KV-cache reuse avoids recomputing each prefix, but probes still add model computation and latency as the number of sampled positions grows; they cannot be obtained from a standard single prefill alone. A separate guard model can instead process the complete prompt in one forward pass. The paper reports added latency but does not provide a matched end-to-end comparison of latency, throughput, and compute against standalone detectors, especially considering small classifiers may also work. Thus, its practical efficiency advantage remains unsubstantiated.
  • The leave-one-attack-out experiment tests unseen attack methods, but does not rule out shortcuts based on harmful semantics, attack formatting, or dataset origin. Training and test prompts may share underlying harmful behaviors, while benign prompts come from distinct sources. To test whether the TCN truly learns jailbreak-related safety trajectories, the authors should evaluate held-out harmful behavior categories and held-out benign distributions, and construct counterfactual pairs: benign requests with attack-style wrappers and harmful requests without wrappers. These controls would help disentangle content harmfulness from adversarial formatting; wrapper-only prompts require clearly defined labels when they explicitly instruct the model to bypass safeguards. Per-victim held-out results would also clarify how consistently the claimed generalization holds.
Questions for authors

Please refer to the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.