acceptodds
Under review as a conference paper at ICLR 2027

CICD-Guard: Contextual Intent Consensus for Robust Defense against LLM Jailbreak Attacks

Abstract

Large language models (LLMs) are increasingly deployed in safety-critical applications, yet jailbreak attacks can conceal harmful intent through contextual packaging such as role-playing, fictional scenarios, or seemingly benign motivations. Existing input-level defenses often struggle to balance Defense Success Rate (DSR) and Over-refusal Rate (ORR), either missing disguised attacks or excessively refusing legitimate requests. We propose CICD-Guard, a parameter-free, black-box defense framework that exposes contextual intent through semantic rewriting and performs consensus-based risk assessment. Given an input prompt, CICD-Guard generates five semantically related rewrites that preserve the underlying objective while removing unnecessary contextual packaging. The original prompt and rewrites are then jointly evaluated by multiple independent LLM-based safety scorers using a continuous risk scale. A decision module combines absolute risk thresholds with relative risk changes between the original and rewritten prompts to adaptively intercept the request, preserve the original prompt, or return a more explicit representation with a safety warning. Our experiments show that CICD-Guard achieves average DSRs of 91.44% and 100% on the two target models, with ORRs of only 8.00% and 12.00%, respectively. Compared with the baselines, CICD-Guard provides a substantially more favorable DSR–ORR trade-off: on Llama-3-8B-Uncensored, it improves average DSR by 30.30 percentage points while reducing ORR by 10.04 percentage points; on DeepSeek-V4-Flash, it further improves average DSR by 15.67 percentage points while reducing ORR by 10.22 percentage points.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Defense, LLMs, Rewrite, Intent-detect, Joint-scoring

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cZDiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

CICD-Guard is a black-box jailbreak defense that rewrites user prompts, scores the original and rewritten versions with multiple LLMs, and uses absolute risk and risk changes to decide whether to block or modify the request.

Critical strengths

The authors evaluate two target models with different safety behaviors and provide ablations and sensitivity analyses.

Critical weaknesses

(1) Deployment feasibility is not demonstrated. The default configuration generates five rewrites and scores all six representations with four models, requiring 24 scoring evaluations per request. The computational overhead is tremendous. Despite using several large auxiliary models, the paper reports no end-to-end latency, token consumption, throughput, or cost comparison. This overhead applies to benign requests as well, which is in fact the majority of user request, making the trade-off particularly important for ordinary conversational workloads.

(2) The central robustness assumption remains unvalidated. The method assumes that rewriting reliably exposes harmful intent while preserving legitimate constraints. However, increased risk after rewriting may reflect semantic distortion rather than previously concealed intent. Moreover, an attacker could target the rewriter or scorers, including through combined contextual and adversarial-suffix attacks.

The paper tests three attacks, but does not clearly establish generalization to held-out attack families or robustness against attacks optimized against the complete defense. This is a concern about the mechanism’s security assumptions, not merely a request for more benchmarks. Adaptive evaluation has established precedent in adversarial ML [1, 2], and jailbreak defense papers like Circuit Breakers evaluate diverse LLM attack familys [3].

(3) The ensemble configuration is insufficiently justified beyond one setting. Table 2 studies scorer and rewrite counts only under CIPHER on Llama-3-8B-Uncensored. Adding a fifth scorer reduces DSR from 85% to 66% and increases ORR from 8% to 32%, suggesting substantial sensitivity to model composition. The paper should clarify model-subset selection, score calibration, and whether configuration selection was separated from final evaluation.

(4) Over-refusal does not capture benign task quality. Rewriting could remove constraints or change query meaning. Downstream answer quality and rewriting fidelity remain unmeasured. Favorable benign-task results would address this concern.

(5) Presentation needs improvement. Figure 3’s caption provides little explanation, uncertainty reporting is unclear, and the related-work discussion needs a sharper comparison with closely related defenses.

[1] Athalye, Carlini, and Wagner. Obfuscated Gradients Give a False Sense of Security. ICML, 2018.

[2] Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. ICLR, 2025

[3] Zou et al. Improving Alignment and Robustness with Circuit Breakers. NeurIPS, 2024.

Questions for authors
  1. How robust is the system to held-out attack families and attacks targeting rewriting or scoring?

  2. What are the measured latency, token usage, and cost relative to the baselines? Does the selected configuration remain preferable when cost is considered?

Reviewer_i8UrOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CICD-Guard, a training-free, black-box defense framework against LLM jailbreak attacks that aims to improve defense effectiveness while reducing over-refusal. The framework consists of three components: a semantic rewriting module that generates multiple prompt variants to expose concealed harmful intent, a joint scoring module that uses multiple LLMs to evaluate the risks of original and rewritten prompts, and a risk-adaptive decision module that determines defensive actions based on risk scores and their changes after rewriting. Experiments on two target LLMs against three jailbreak attacks show improved Defense Success Rates and lower Over-Refusal Rates compared with the baselines. The paper also includes ablation studies and sensitivity analyses to evaluate the contributions of different components.

Critical strengths
  1. The paper addresses a practically important problem. Balancing jailbreak defense effectiveness and over-refusal is a critical challenge in LLM safety, and the paper considers both aspects.

  2. The proposed framework is flexible and easy to integrate. CICD-Guard operates as an external defense layer without requiring access to or modification of the target model's internal parameters.

  3. The experimental results are promising. CICD-Guard achieves high DSRs with relatively low ORRs across two target models. Ablation studies and sensitivity analyses further support the effectiveness of its components.

Critical weaknesses
  1. Lack of semantic fidelity and benign task utility evaluation. Semantic rewriting may alter the user's original intent or remove essential context, potentially degrading benign task performance. However, the paper only reports the ORR, without evaluating semantic preservation or downstream response quality.

  2. Lack of adaptive attack evaluation against the rewriting mechanism. The experiments only consider existing jailbreak attacks, without evaluating adversarial prompts specifically optimized to circumvent the rewriting mechanism. If an attacker can construct prompts that prevent the rewriting model from effectively exposing harmful intent, how robust would CICD-Guard remain?

  3. High computational overhead without efficiency evaluation. Each request requires one rewriting step and 24 LLM-based risk evaluations. However, the paper does not report inference latency or computational cost, making it unclear whether the substantial overhead would negatively affect user experience in practical deployments.

Questions for authors
  1. What proportions of benign and malicious requests fall into the gray zone? How often are rewritten prompts forwarded to the target model instead of the original prompts?

  2. How does each safety scorer perform individually? Does the ensemble's effectiveness depend disproportionately on particular scoring models?

  3. How sensitive is CICD-Guard to the choice of rewriting model? Would replacing DeepSeek-V4-Flash-Thinking with other models, particularly smaller or less capable ones, significantly affect defense performance?

Reviewer_EvAXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CICD-Guard, an external black-box jailbreak defense that does not require changes to the target model's parameters. The system generates multiple semantic rewrites of an input, uses several models to assess risk, and combines absolute risk thresholds with changes in risk after rewriting. It then rejects the request, preserves the original input, or uses a rewritten input with a safety reminder.

The experiments evaluate defense success against three attack methods on two target models, measure over-refusal on benign requests, and include component ablations and sensitivity analyses for the numbers of rewrites and scoring models. The paper reports high defense success and relatively low over-refusal. However, the extent to which these results depend on the specific risk-change mechanism, and whether they hold against adaptive attacks, remain to be established.

Critical strengths

1. The method has a clear black-box use case and considers both defense effectiveness and over-refusal.

The system does not require modifying the target model's parameters and can be deployed as an external defense. Reporting attack defense and benign over-refusal together helps distinguish useful protection from apparent safety gains obtained simply by rejecting more requests. This is a practically relevant evaluation objective.

2. Ablation and sensitivity studies cover the main design choices and provide useful initial evidence.

The paper examines rewriting, joint scoring, and the decision module, as well as the numbers of rewrites and scoring models. The degradation after removing rewriting suggests that input representation matters for this system, while adding more components does not always improve performance. These experiments provide an initial account of the system's behavior and identify concrete questions for further validation.

Critical weaknesses

1. Incomplete output-judging and statistical protocols limit the reproducibility of the main defense results.

Some settings report 100% defense success, but the paper does not sufficiently explain how final outputs are classified as harmful or as refusals, or how that judge relates to the defense's internal risk scorers. It also needs to clarify whether the plus/minus values denote standard deviations, standard errors, or confidence intervals, and how many repetitions they summarize. Prompts for scoring input risk do not substitute for an output-evaluation protocol. These omissions make the meaning and stability of the high defense success rates difficult to verify independently.

The experiments evaluate specified attack methods but do not report attacks adapted to the full rewriting, scoring, and routing pipeline. This does not invalidate performance in the tested settings, but it limits the evidence to those evaluations rather than supporting stronger robustness claims.

2. The independent contribution of risk changes and the value of the additional computation are not sufficiently established.

The paper lacks a sufficiently discriminating comparison with multi-representation defenses such as SemanticSmooth. The component ablations also leave confounding factors: replacing the ensemble with a particular single scorer does not fully separate ensemble effects from scorer capability, while removing the entire decision module does not isolate whether the original-to-rewrite risk difference is necessary.

The full system generates several rewrites and repeatedly evaluates them with multiple scoring models, but actual inference cost, latency, and comparisons under matched computational budgets are not sufficiently reported. It is therefore difficult to determine whether the gains come from the specific risk-change mechanism or from additional computation and model capability, and whether they justify the deployment overhead.

Questions for authors

1. Output judging and statistics: clarification of the existing evaluation protocol.

Please specify the models or rules and prompts used to judge harmful outputs and benign refusals, and whether they are independent of the defense's internal scorers. What do the plus/minus values represent, and how many repetitions were performed? Was there any manual spot-checking? These details would directly affect my assessment of the results' reproducibility.

2. Mechanism and cost: controlled comparisons within the existing settings.

Could you retain the same rewrites, scorers, and computational budget while removing the risk difference or replacing the routing rule with a simpler alternative? Could you also compare against a comparable configuration of a directly related method such as SemanticSmooth? Please report defense success, over-refusal, token usage, and latency to establish whether the core mechanism provides an independent benefit.

3. Robustness: a bounded adaptive evaluation.

For one existing target model, could you let the attacker adjust requests using feedback from the complete defense and report results under a fixed attack-query budget? This is a limited test of the current robustness claim, not a request to expand to many new models or attack categories.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.