- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper detects jailbreak prompts from the target LLM’s internal states. It appends the chat suffix to every prompt prefix to read a per-position refusal score (Eq. 5, p.4), pairs it with a per-token harmfulness score, and classifies both trajectories and their differences with a ~6K-parameter TCN (§5, p.7). It reports 96.17–98.27% accuracy on four 7B models against five baselines (Table 2, p.8), plus ablations, held-out attacks, and span-deletion analyses.
Critical strengths
- Per-prefix suffix probing is simple, needs no backbone tuning, and gives interpretable rise/fall patterns (Fig. 4, p.5). The underlying observation, that the template region carries the safety decision, is known (Zhao et al., NeurIPS 2025; Leong et al., ACL 2025, not cited). The novelty is therefore moderate.
- Trajectory features beat endpoint features on all four backbones (Table 3, p.8), and held-out attacks reach 90.44–100% (Fig. 6a, p.9).
Critical weaknesses
-
The probe’s contribution is not isolated. Table 3 (p.8) has no no-suffix refusal trajectory, and the suffix-free harmfulness trajectory alone reaches 91.54%, versus 91.87% for the probed refusal trajectory. The gain may come from using sequences rather than from the probe.
-
The evaluation cannot separate jailbreak detection from in-distribution harm detection.
- No benign prompt uses a jailbreak-style template (cf. WildJailbreak; Jiang et al., NeurIPS 2024).
- DualTrace trains on the same sources it is tested on (Table A.1, p.14), while baselines use unreported thresholds (GradSafe detects at most 8% of attacks on Qwen, Table 2).
- No adaptive attack is tested, although latent-space monitors can be evaded (Bailey et al., ICLR 2026).
Questions for authors
Clarifications:
- Which configuration produces Table 2? Qwen accuracy is 96.17 there, 95.39 in Table 3, and 94.35 in Fig. B.3.
- Do IJP prompts contain a harmful request? Table B.2 row 32 shows an unfilled “{INSERT PROMPT HERE” placeholder.
Suggestions:
- For W1: a no-suffix refusal-trajectory ablation.
- For W2: adversarial-benign prompts, a probe trained on the same 700 prompts, Gradient Cuff (Hu et al., NeurIPS 2024), and an adaptive GCG that flattens R_t.