- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper explores how to transform a general-purpose LLM into a guardrail that balances safety and utility by optimizing the System Prompt, without requiring any fine-tuning of the LLM's parameters. This method first generates a fixed set of candidate prompts, then filters them to satisfy safety constraints using a statistical upper confidence bound, and finally selects the prompt with the highest benign utility. Subsequently, the method incorporates risk-certification feedback into an iterative prompt search process, enabling the model to proactively revise prompts that fail to meet safety requirements.
Critical strengths
-
This paper is well organized and clearly written.
-
The motivation of the paper is attractive.
-
The experiments are extensive and helpful to validate the effectiveness of the model.
Critical weaknesses
-
The components of the method are similar to existing work, please present the novelty of the proposed technique better for understanding.
-
Risk certification holds only under the IID assumption and cannot guarantee adversarial robustness.
-
The optimized prompt may have learned spurious patterns present in the dataset.
Questions for authors
Please refer to the above weaknesses.
2 comments
Thank you for the follow-up. Part of the evidence for your four points was posted in our replies to the other reviewers, so we gather it here.
Novelty. LTT and PRC require the prompt family to be fixed before calibration. CRISP generates prompts from certification feedback, and Prop. 2 proves the certificate remains valid for every certified prompt, regardless of the number of rounds. To our knowledge, no prior LTT-style result covers adaptively generated hypotheses.
Contribution of each component (Deepset, gpt-5.4-mini judge, α = 0.20, T = 10):
- Single population: one pool ranked by the penalized training score , instead of the two populations.
- ReduceRisk only: both rewrites each round go to the lowest-risk uncertified prompt.
- ImproveUtility only: both rewrites go to the highest-utility prompt, certified or not.
- Random parents: both populations and branches, parents drawn uniformly.
| Variant | Test risk / utility | Certified / pool |
|---|---|---|
| CRISP | 0.091 / 0.987 | 10/19 |
| Single population | 0.036 / 0.883 | 3/21 |
| ReduceRisk only | 0.055 / 0.948 | 16/21 |
| ImproveUtility only | – | 0/21 |
| Random parents | 0.091 / 0.961 | 10/18 |
- Both objectives matter: ImproveUtility-only certifies nothing (0/21). ReduceRisk-only certifies the most candidates (16/21), but reduces utility.
- Two-population split matters most: The single-population variant certifies only 3/21 candidates. We hypothesize penalized training risk poorly separates certifiable candidates, so the search cannot specialize.
- Parent selection matters less: Random selection matches risk but reduces utility. Most gains come from the population split and complementary rewrite branches.
Moreover, replacing Guess-and-Check by direct reuse of the calibration set (BIPIA, α = 0.10) returns a prompt with test risk 0.113, above the target.
Fair comparisons. All methods share the same seed prompt, rewriter, judge, splits and filtered test set. We added three controls: CPS with a matched budget (N = 41), risk-aware GEPA and ProTeGi followed by LTT, and PRC on 12 published prompts. At α ≥ 0.20, budget-matched CPS matches CRISP, and we report this. At α = 0.10 (gpt-5.4-mini judge), none of these baselines certifies and CRISP does.
Realistic scenarios. Prompts certified on BIPIA and Deepset transfer without re-certification to LLMail-Inject, a public adaptive challenge in which participants attacked an LLM email assistant (missed-attack rates 0.072 and 0.010), and to Qualifire (0.033 and 0.002). We also report Best-of-N, Crescendo and PAIR attacks, a comparison with deployed classifiers (PIGuard, Prompt Guard, Model Armor), and 50 independent re-splits.
If a specific baseline or scenario is missing, we would be glad to run it.
We thank the reviewer for the positive assessment of the motivation and the experiments.
CPS combines an existing optimizer with LTT/PRC (Zollo et al., ICLR 2024). We present it to expose the limits of this direct approach; its failure mode motivates CRISP (CPS returns an empty certified set in 4 of 12 prompt-injection settings).
The contributions are:
The table below shows that at a tight risk level (α=0.10), every simpler combination of existing pieces fails to certify. Setting: α=0.10, gpt-5.4-mini judge; entries show test risk / utility. "–" means no certified prompt. We will add this to the paper.
We agree that the certificate is stated under i.i.d. calibration. We add three things.
Formally:
Following robust validation and conformal prediction beyond exchangeability (Cauchois et al., 2024; Barber et al., 2023), suppose the deployment malicious distribution satisfies TV(π1,π1′)≤η, where TV(P,Q)=supA∣P(A)−Q(A)∣ is the total variation distance. Then ∣Rπ1′(p)−Rπ1(p)∣≤η for every prompt p, since the missed-attack loss is bounded in [0,1]. Hence running CPS or CRISP at risk level α - η certifies α under any η-bounded shift.
If the shift is only in the label distribution (i.e., π(y) changes but π(x∣y) does not), the guarantee is unchanged, since R and U are class-conditional. We will add this as a corollary in App. B and state in Limitations that η must be specified.
Empirically, under distribution shift:
We took the CRISP prompts certified at α=0.20 on BIPIA and Deepset and measured their test missed-attack rate on two other prompt-injection benchmarks, without re-certification: Qualifire (Qualifire, 2025; Ivry & Nahum, 2025) and LLMail-Inject (Abdelnabi et al., 2025).
Risk stays below α=0.20 in every pair, suggesting the certified prompts are not overfit to the calibration attack distribution.
Empirically, under adaptive attacks:
Figure 3 already reports Best-of-N jailbreaking on BIPIA. We also ran two adaptive attacks, Crescendo and PAIR, each with up to 5 turns, on 30 malicious Deepset seed prompts. Judge: gpt-4o-mini; entries are attack success rates (lower is better).
Crescendo is multi-turn attack shows the guardrail the full dialogue. PAIR is iterative attack.
CRISP yields the most robust prompt under both attacks. We report this as a robustness diagnostic, not a guarantee, and will keep adaptive robustness as an explicit limitation. It is worth mentioning that certifying guardrails against adaptive attacks remains an open research problem beyond restricted threat models.
We thank the reviewer for this point. We address this concern with the following evidence:
We hope these clarifications address the reviewer's concerns.