- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SEEK, an inference-only framework for discovering residual unsafe behaviors in safety-aligned language models through stochastic input perturbations and repeated response sampling. Its central premise is that random perturbations can make unsafe responses more accessible despite dominant refusal behavior. SEEK collects unsafe responses for safety training, emphasizing behavioral coverage rather than finding a single successful jailbreak. The paper establishes monotonic coverage growth and an expected coverage advantage of resampling perturbations over reusing a fixed perturbation under its stated assumptions. Experiments across five models and two benchmarks show strong discovery performance at larger sampling budgets and favorable computational efficiency relative to optimization-based baselines. Finally, DPO fine-tuning on SEEK-generated preference data improves resistance to multiple attack families on Qwen2.5-7B, while maintaining accuracy on an MMLU subset.
Critical strengths
-
Practical simplicity and strong discovery performance. SEEK requires neither gradient optimization nor an auxiliary attacker model, yet achieves high discovery rates across five models and two benchmarks. This makes it a potentially useful baseline for generating safety-training data.
-
Demonstrated downstream safety benefits. The paper evaluates whether discovered failures improve robustness against structurally different attacks, connecting discovery to a practical safety objective. The observed improvements make the approach more consequential than a method evaluated solely through jailbreak success.
Critical weaknesses
-
The incremental contribution over existing stochastic methods is insufficiently isolated. The paper lacks a matched-cost comparison in which BoN also retains all failures and supplies equally sized training datasets. Consequently, it remains unclear whether SEEK offers a distinct advantage beyond different perturbations and continued collection.
-
The safety-hardening gains are not clearly attributable to SEEK-generated data. Training also includes affirmative-prefix augmentation, without controls separating its contribution from DPO and the choice of negative-response source. Likewise, the one-versus-twenty-response comparison changes both dataset size and diversity.
-
The harmfulness evaluation lacks independent validation. A single automated classifier determines unsafe outcomes, without reported human auditing; the second example in Table 8 does not clearly fulfill the harmful request. This uncertainty matters particularly for success rates based on finding at least one classified failure among 1,024 generations.
Questions for authors
-
Are matched-cost, matched-data-size safety-tuning comparisons with BoN or other negative-response sources available, and was affirmative-prefix augmentation ablated?
-
Was any human validation of the harmfulness labels performed, and what supports classifying the second example in Table 8 as a successful jailbreak?
-
How were the fractional ASR values computed from 100 prompts, and were attacks regenerated against the fine-tuned model? Please also reconcile the differing HarmBench averages between Table 1 and Appendix H.
The incremental contribution over existing stochastic methods
We respectfully disagree with the characterization that SEEK offers only an incremental variation of existing stochastic attack methods. This interpretation overlooks the fundamental difference in the objectives of the two approaches. BoN is primarily designed as a jailbreak attack methodology: its objective is to find a successful harmful response, and its evaluation is centered on attack success. Accordingly, its perturbations are deliberately constrained to preserve human intelligibility, and its conventional first-success criterion does not explicitly optimize the collection of diverse residual failures.
In contrast, SEEK introduces a coverage-driven negative-response mining framework for safety hardening. Its objective is not merely to establish whether a prompt can be jailbroken, but to systematically discover and retain a diverse collection of residual unsafe behaviors that can provide richer negative supervision for subsequent safety tuning. This difference also motivates SEEK's broader perturbation distribution, which is not constrained to produce human-intelligible prompts, and its coverage-based formulation that explicitly values collecting multiple distinct failures rather than identifying a single successful attack.
Our experiments demonstrate why this distinction matters. At a generation budget of 1024, retaining all unsafe responses discovered by SEEK achieves more than 13 percentage points higher behavioral coverage than retaining only the first unsafe response per prompt (Appendix I). More importantly, the collected failures provide effective supervision for safety hardening: DPO training on SEEK-discovered responses reduces vulnerability not only to SEEK but also to distinct attack families, including GCG, ASETF, PiF, TAP, and PAIR. These results demonstrate that the value of SEEK extends beyond attack success to discovering residual behaviors that generalize across attack mechanisms and can be used to improve model robustness.
We agree that a matched-cost comparison in which BoN also retains all discovered failures and supplies an equally sized training dataset would provide an additional controlled comparison. However, such a comparison would evaluate a coverage-oriented adaptation of BoN, rather than its conventional attack objective. We view the distinction as important: SEEK explicitly formulates and evaluates stochastic exploration as a negative-response mining problem, with behavioral coverage and downstream safety hardening as its central objectives. The contribution is therefore not simply continued collection under a different perturbation strategy, but the formulation, analysis, and empirical demonstration of a coverage-driven discovery framework for safety hardening. We will clarify this distinction and the scope of our comparison in the revised manuscript.
The safety-hardening gains are not clearly attributable to SEEK-generated data
We agree that the current experiments do not fully isolate the contributions of affirmative-prefix augmentation, DPO, and the negative-response source. We plan to conduct controlled ablations with matched dataset sizes and training configurations to disentangle these effects, including the contribution of response diversity.
The harmfulness evaluation lacks independent validation
We have evaluated several ASR using an alternative judge, observing differences of mostly approximately ±5 percentage points, and found only marginal changes after denoising responses. We will add these results in future.