- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes CORD, a jailbreak detection method that compares the prediction errors of benign and jailbreak hidden-state dynamics using conditional Koopman operators. The method models transitions along both token and layer dimensions and uses their residual contrast for detection. Experiments on four LLMs and ten attack families show strong performance, achieving 93.94% macro TPR@1.
Critical strengths
-
The idea of detecting jailbreaks through contrasting hidden-state dynamics is interesting and well motivated, offering a different perspective from existing representation-based methods.
-
The experimental results are strong across multiple models and attack families, with substantial improvements over existing baselines.
-
The matched-input comparisons, ablation studies, and theoretical analysis provide useful evidence supporting the proposed design.
Critical weaknesses
-
The evaluation is limited to leave-one-attack-family-out settings. It remains unclear how well the method generalizes to genuinely novel or adaptive jailbreak attacks.
-
The benefits of the Koopman formulation are not fully isolated. Comparisons with simpler predictive models, such as MLP-based or unconstrained linear predictors, would help clarify its necessity.
-
The false positive rates increase on challenging benign inputs when using the original calibration thresholds, raising concerns about robustness under benign distribution shifts.
-
The theoretical results rely on assumptions that are not empirically verified, and their connection to the observed detection improvements could be better established.
Questions for authors
Please refer to weaknesses.
2 comments
Author Response to Reviewer LeRY(2/2)
Q3: How robust is CORD to benign distribution shifts at the original calibration threshold?
Response: Thank you for your valuable comment on benign distribution shift. We have evaluated robustness to benign distribution shifts by applying each detector's original threshold to ordinary benign prompts, OKTest, and XSTest-safe without recalibration. Attack detection remains above 92% on all four models. Ordinary-benign FPR stays close to 1%, and FPR on OKTest and XSTest-safe stays below 5%. The hard-benign prompts cause more false alarms, but the increase is moderate. CORD therefore maintains a useful balance between detection and false positives under this distribution shift.
| Target model | Attack TPR | Ordinary-benign FPR | OKTest FPR | XSTest FPR |
|---|---|---|---|---|
| Llama2-7B-Chat | 93.27% | 1.67% | 4.40% | 3.88% |
| Llama3-8B-Instruct | 95.63% | 1.17% | 3.93% | 2.64% |
| Qwen2.5-7B | 92.23% | 1.20% | 1.57% | 3.32% |
| Mistral-7B-Instruct-v0.2 | 94.63% | 1.27% | 4.57% | 2.20% |
Q4: Are the theoretical assumptions empirically supported, and how do they relate to detection gains?
Response: Thanks for your insightful suggestion regarding the connection between theory and empirical results. Proposition 1 predicts a positive expected residual difference, , when the separation between the two predictors’ forecasts exceeds the prediction bias of the correct-class predictor. Proposition 2 describes when this advantage carries over to held-out families. To test these predictions, we compared the two predictors' errors on benign prompts, seen attacks, and held-out attacks, using the same targets in the same observable space. For a more controlled comparison, we also matched seen and held-out attacks by their recorded harmful goals. We excluded GPTFuzz, IJP, and JailJudge because they have no exact goal matches, leaving seven attack families.
Across four models and ten attack families, the 95% paired bootstrap interval is above zero in all 40 benign and all 40 seen-family combinations at each horizon. For held-out attacks, the counts range from 37 to 38 of 40 across the three prediction horizons. The correct-class prediction advantage therefore persists on most held-out families, consistent with the predicted transfer. In the matched-goal comparison, the 95% interval remains above zero in 26-27 of 28 held-out combinations across the three prediction horizons. The advantage thus persists when attacks share the same goal but use different methods. These observations are consistent with the theoretical premise of a transferable relative prediction advantage and provide an empirical basis for the detection gains.
| Evaluation setting | Cohort | 1 step | 2 steps | 4 steps |
|---|---|---|---|---|
| 4 models x 10 families | benign | 40/40 | 40/40 | 40/40 |
| 4 models x 10 families | Non-fit seen-family attacks | 40/40 | 40/40 | 40/40 |
| 4 models x 10 families | Held-out-family attacks | 37/40 | 38/40 | 38/40 |
| 4 models x 7 matchable families | Seen-family attacks | 28/28 | 28/28 | 28/28 |
| 4 models x 7 matchable families | Held-out-family attacks | 26/28 | 27/28 | 27/28 |
Thank you again for your valuable time and efforts in reviewing this paper! We have revised the paper following your constructive and insightful comments. Your valuable feedback has helped us present our work more clearly and comprehensively. If any questions remain unresolved, we appreciate the opportunity for further discussion and clarification.
Author Response to Reviewer LeRY(1/2)
Thank you very much for your positive comments and insightful suggestions. We have carefully considered each point and provide our responses below.
Response: Thank you for your valuable suggestion! Following your suggestion, we have conducted an additional comparison against four baselines on five new attacks published in 2026, using Qwen2.5-7B. The results show that CORD achieves the best or joint-best TPR@1 on all five attacks. We would like to clarify that all five attacks were excluded from training and tuning. AGILE yields CORD's lowest detection rate among the five attacks, but CORD still clearly outperforms every baseline. These empirical results demonstrate that CORD generalizes to new attacks beyond the training families.
Adaptive attacks
Thank you for raising this concern. Your comment has encouraged us to examine this setting more carefully. As suggested, we tested score-feedback suffix search on Llama3, and TPR@1 fell from 96.20% to 0.00%. Since attackers can use score queries to guide black-box search, they can optimize prompts to evade detection. The detection score is also differentiable with respect to hidden representations. This result indicates room for improvement in defending against adaptive attacks. As per your comment, we have added a Limitations section to the revised manuscript and discussed this limitation.
References:
[1] AGILE: Activation-Guided Local Editing for Jailbreaking Attacks
[2] SpatialJB: How Text Distribution Art Becomes The "Jailbreak Key" for LLM Guardrails
[3] LogiBreak: Logic Jailbreak: Efficiently Unlocking LLM Safety Restrictions Through Formal Logical Expression
[4] StegoAttack: Hiding in Plain Sight: A Steganographic Approach to Stealthy LLM Jailbreaks
[5] CAGE: Concept Algebra for Guardrail Evasion and Jailbreak Attacks via Lexical-Semantic Decoupling
Response: Thank you for this insightful suggestion regarding the predictive controls. To examine the contribution of the Koopman structure, we compared CORD with three simpler predictors. CORD achieves the highest ten-family mean TPR@1 on the four evaluated models, including in comparison with the nonlinear dual predictor. The class-conditional MLP keeps the two-predictor residual score but replaces the Koopman operators with nonlinear networks. The unconstrained autonomous linear predictor uses Kz without control inputs or spectral constraints. Persistence repeats the current state and uses the absolute prediction residual as the detection score.
Empirical results show that CORD exceeds the MLP predictor by 11.43%-21.53%. Interestingly, the more flexible nonlinear MLP performs worse than the linear predictor on three models. In this comparison, the more flexible predictor does not consistently improve detection on held-out families. Some controls perform particularly poorly on individual attacks. Persistence reaches 1.67% on Llama2-Base64, where CORD reaches 100.00%. The unconstrained linear predictor reaches 1.67% on Qwen2.5-DrAttack, where CORD reaches 90.67%. The advantage is evident both in mean detection rates across attack families and in the results on individual attacks that simpler predictors largely fail to detect. These comparisons support the use of structured Koopman prediction as an effective design choice for detecting unseen attack families.