- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This study explores the issue of final-token safety probes failing to detect jailbreak prompts. The potential reasons are analyzed through representation geometry and token-level analysis. The manuscript uses PCA-HMM to model token trajectories, which recovers approximately 90%–95% of the previously undetected samples. And it simultaneously reduces the false positive rate.
Critical strengths
- This study demonstrates the limitations of final-token safety probes in detecting threats across different input formats.
- This study analyzes instances of detection failure from multiple perspectives, and it highlights the potential of trajectory-based diagnostics.
Critical weaknesses
- This study used only three 7-8B models to test and did not test smaller (3B) or larger (28B) models. Regarding the proposed failure of the "final-token probe" to detect certain features, it is unclear whether this is a common failure mode, or just limited by the scale and capabilities of models. This limitation constrains the generalizability of the findings. It's hard to determine whether the mentioned failure mode applies to other models, especially there are a lot of advanced open models (Qwen3.8-27B, released in Aug) and closed models (Opus, Sol, ...)
- The study shows that PCA-HMM could recover a number of jailbreak prompts missed by final-token probes, but it lacks direct comparisons with different settings. For example, a stronger final-token classifier? Or is it possible to use probes trained on jailbreak samples? Or is it possible to test with other multi-token aggregation methods? It is unclear whether the performance improvement is from the sequential information within token trajectories. Maybe it could be simply from the use of more token data or a different method. It's hard to tell how much value trajectory modeling adds compared with simpler methods.
Questions for authors
- If the computational resources are limited, is it possible to at least run experiments on smaller models (e.g., 1.8B, 3B)? Do the changes in model scale and capability affect the performance of final-token probes?
- Although Appendix F shows augmented static readouts, could the author provide more controlled comparisons? For example, a stronger final-token classifier, probes trained on jailbreak samples, or other multi-token aggregation methods? These comparisons would help explain the unique contribution of trajectory modeling.