- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1(−1)
- Soundness
- 2(−1)
Read review
Paper summary
This work investigates whether improvements in jailbreak defense reflect real prevention of harmful behavior or merely increased attack cost. The authors propose a reachability certificate that replays previously successful queries against defended models to establish an upper bound on attainable-risk reduction. They also introduce ANCHOR, a controlled benchmark with known defense effects, and evaluate the framework on synthetic controls and four existing defenses.
Critical strengths
The paper addresses an important evaluation problem by distinguishing genuine prevention from increased attack difficulty, which is often overlooked in fixed-budget jailbreak evaluations.
The use of negative and positive controls with known ground truth provides a clear experimental setting for validating the evaluation procedure.
Critical weaknesses
The proposed methodology has limited novelty.
The certificate provides a straightforward one-sided bound by counting verified reachable behaviors, without establishing whether uncertified behaviors are actually prevented. The propose problem seems not be solved in this case, as a failed replay is still not be able to prove the defense method block the harmful request.
Moreover, the main experiment uses REPAR, a reversible transformation known to provide no prevention by construction. Thus, the results offer limited evidence of the certificate's added value for evaluating practical defenses.
Although the main claim of prevention vs. delay is somewhat supported during the experiment and also have some metric related insight, it does not provide an new angle in evaluating LLMs' performance in Jailbreaking Success, it is merely just an application of established statistic model.
While it could stands out with a claim to be a diffferent way in characterizing defense effectiveness, it is unclear whether this new metric could leads to more actionable insight for evaluating or improving LLM safety (which is important since metric will be the guidance for future LLM safety research's evaluation and guidance). Since the underlying objective behind this new metric remains unchanged, i.e., reducing the probability of harmful behaviors, the proposed metric also does not provide new guidance to the future LLM training.
Questions for authors
see weakness
Thank you for your review.
We agree that a failed replay leaves the behavior unresolved. A successful replay shows that the defense did not prevent that particular behavior. By counting how many behaviors the same attack produces both before and after the defense is added, we can bound how much of the drop in success at a fixed budget could be due to prevention.
We use REPAR to check the method because its true effect is known. Our re-implementation of a paraphrase defense on ANCHOR shows what we can learn when the effect is unknown (Section 6.4, Table 5). At 32 queries per behavior, paraphrasing reduced best-of-n success by 13.5 percentage points. Given more attempts, the same attack produced 222 of 223 behaviors on both systems. Even if paraphrasing prevented the remaining behavior, prevention could explain at most 0.45 percentage points of the drop. That rules out prevention as an explanation for more than 96% of the measured gain.
The SmoothLLM-style result suggests a different step for evaluators. Most sorting tasks remained unresolved, so we tested whether the model could still sort the words when we removed the instruction to withhold the answer. All unperturbed sorting responses were correct. Only 8.7% of responses to individual perturbed inputs were correct. This gives a concrete reason to check ordinary task accuracy before interpreting fewer successful attacks as a safety benefit. These results concern our implementations on benign tasks. Their implications for harmful requests with a released defense and for changes to model training remain open.