acceptodds
Under review as a conference paper at ICLR 2027

Defending Multi-turn Attacks On Large Language Models Through Active Intention Verification

Abstract

Multi-turn jailbreaks spread a malicious goal across seemingly benign turns, so defense hinges on inferring the user's hidden intent. Existing safeguards are passive: they classify whatever the attacker chooses to reveal. We argue that when a conversation looks equally plausible under benign and malicious intent, no passive rule can avoid trading unnecessary refusals against unsafe assistance; the defender must instead act to gather evidence. We propose Active Intention Verification (AIV), which maintains an evolving intent profile and, when a sensitive request is ambiguous, asks targeted probes before deciding to assist or refuse. Across four multi-turn attacks and six open- and closed-source models, AIV achieves defense success rates of 0.83–0.98 with only 2–5% over-refusal. Against intention deception, where passive baselines defend fewer than 22% of attempts on open-source models, AIV defends 83–90%, and removing the probes erases this gain. Our analysis shows why probing works: benign users resolve ambiguity with little friction, whereas attackers must commit to a benign frame that constrains their later requests. As a result, even protocol-aware attackers with pre-staged answers reduce AIV's success only to 0.78–0.94. These findings suggest that multi-turn safety should be built as interactive protocols that make intent identifiable, not as filters over whatever has been revealed.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-turn jailbreak attack, LLM safety, adversarial attack and defense

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SDpNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Active Intention Verification (AIV), which maintains an intent profile and asks targeted probes before assisting or refusing ambiguous, sensitive requests. It combines profiling, risk scoring, retrieval, and clarification, supported by theoretical analysis and multi-turn attack evaluations.

Critical strengths
  1. The distinction between passive classification and active evidence gathering provides a useful perspective on multi-turn defense.

  2. Component ablations and protocol-aware attacks provide evidence that probing contributes to the reported gains.

  3. Evaluations include multiple model families, benign benchmarks, and computational overhead measurements.

Critical weaknesses
  1. Limited attack and reasoning-model coverage. ActorAttack is omitted without explanation. Most model–attack combinations use older open-source models, including Qwen2.5-7B, Llama-3-8B, and Mistral-7B, which have limited multi-turn alignment. Frontier closed-source evaluations cover only Intention Deception and 100 tasks per model. Broader testing on contemporary reasoning models, such as Qwen3.5, across attacks, benign tasks, and reasoning-budget settings is needed to support the general robustness claims.

  2. Strong assumptions underlying the theoretical guarantee. Assumption 1 requires high benign acceptance and low downstream harmful assistance after verification. The guarantee is therefore conditional on probing already producing the desired safety properties. Aggregate empirical estimates do not establish the assumption uniformly across ambiguous histories and malicious objectives.

Questions for authors

See Weakness

4 comments
AuthorsOpenReview ↗

Response to Reviewer SDpN (1/2)

We thank the reviewer for the detailed comments. We want to address your concerns as follows:

W1 - Limited attack and reasoning-model coverage We have conducted additional experiments covering ActorAttack, Qwen3.5-9B in both thinking and non-thinking modes, and higher reasoning-effort settings for proprietary models. We also include component ablations to examine whether AIV's individual mechanisms remain effective across additional attacks and model configurations.

1. Additional Evaluation on ActorAttack

We extend our evaluation to ActorAttack across four models, including three open-source models and GPT-4o, with component ablations.

ModelTCANBFX-BoundaryAIVNo-QNo-RAGNo-profile
Qwen2.5-7B0.3620.5460.4820.9200.2440.8720.564
Llama-3-8B0.3940.4780.6460.8840.4680.8560.644
Mistral-7B0.3760.5240.7100.9120.4260.8240.666
GPT-4o0.4360.492–0.9340.4740.8520.688

Takeaways:

AIV generalizes effectively to ActorAttack. AIV achieves DSRs of 0.884–0.934 across four models, outperforming all passive baselines. ActorAttack evaluation extends attack coverage beyond the four multi-turn attack methods originally evaluated.

Active probing remains the most important component of AIV under ActorAttack. Removing active questioning reduces average DSR from 0.913 to 0.403, while removing intention profiling or retrieval reduces DSR to 0.641 and 0.851, respectively. Active probing contributes the largest improvement, with profiling and retrieval providing complementary benefits under ActorAttack.

2. Evaluation on Contemporary Reasoning Models Across Multiple Attacks

We further evaluate Qwen3.5-9B in both thinking and non-thinking modes across five multi-turn attacks: Crescendo, Intention Deception, Opposite-Day, Chain-of-Attack, and ActorAttack, including component ablations.

Defense Success Rate (DSR ↑)

ModeAttackTCANBFX-BoundaryAIVNo-QNo-RAGNo-profile
ThinkingCrescendo0.4580.6460.7120.9100.5640.8860.494
ThinkingIntention Deception0.1420.1600.1640.9240.1380.8720.394
ThinkingOpposite-Day0.6760.8260.8320.9400.7020.9120.802
ThinkingChain-of-Attack0.4740.7280.6440.9620.5660.9380.524
ThinkingActorAttack0.3960.5780.6280.9420.4440.8940.698
Non-thinkingCrescendo0.4280.4860.6400.9180.4100.8800.468
Non-thinkingIntention Deception0.1300.1220.1460.9060.1320.8560.338
Non-thinkingOpposite-Day0.5280.7940.7640.9020.6360.8700.766
Non-thinkingChain-of-Attack0.3680.3820.5840.9220.5300.8580.692
Non-thinkingActorAttack0.3820.4660.4500.8740.4180.8320.560

Takeaways:

AIV maintains strong defense performance across both reasoning configurations and all five attacks. AIV outperforms all passive baselines across all ten configurations, achieving average DSRs of 0.936 in thinking mode and 0.904 in non-thinking mode. Under Intention Deception, AIV achieves 0.924 and 0.906, compared with only 0.164 and 0.146 for the strongest passive baselines, demonstrating robustness against concealed malicious intent in both reasoning configurations.

Model reasoning capability does not eliminate the benefits of active verification and intention profiling. Across both modes and five attacks, removing questioning reduces average DSR from 0.920 to 0.454, while removing profiling and retrieval reduces DSR to 0.574 and 0.880, respectively. The consistent performance degradation demonstrates that model reasoning capability does not eliminate the benefits of active probing and cross-turn intention tracking.

3. Benign Helpfulness on Qwen3.5-9B

We additionally evaluate over-refusal on benign tasks under both Qwen3.5-9B configurations to assess whether improved defense robustness compromises benign helpfulness.

Over-refusal Rate (OR ↓)

ModelTCANBFX-BoundaryAIVNo-QNo-RAGNo-profile
Qwen3.5-9B (thinking)0.1740.6560.1220.0440.1520.0620.108
Qwen3.5-9B (non-thinking)0.1680.6280.1360.0520.1740.0700.116
AuthorsOpenReview ↗

Response to Reviewer SDpN (2/2)

Takeaways:

AIV preserves benign helpfulness while maintaining high defense robustness. AIV achieves the lowest over-refusal rates in both thinking (4.4%) and non-thinking (5.2%) modes, substantially outperforming passive baselines. Combined with high DSRs across five attacks, AIV demonstrates a favorable safety–utility balance on Qwen3.5-9B.

Active verification and context-aware evaluation contribute to reducing unnecessary refusals. Removing questioning increases over-refusal to 15.2% and 17.4% in thinking and non-thinking modes, respectively. Removing profiling or retrieval also increases over-refusal in both configurations, indicating that active verification and context-aware evaluation help preserve legitimate assistance instead of relying on broad refusal.

4. Evaluation Across Reasoning-Effort Settings

We further extend proprietary-model evaluations to medium and high reasoning-effort settings on GPT-5.4 and Claude Sonnet 4.6, including component ablations.

Intention Deception — Defense Success Rate (DSR ↑)

ModelEffortTCANBFAIVNo-QNo-RAGNo-profile
GPT-5.4Medium0.5400.6700.9200.6000.8500.790
GPT-5.4High0.5700.6900.9400.6100.9100.800
Claude Sonnet 4.6Medium0.6500.7200.9800.6300.9400.760
Claude Sonnet 4.6High0.6400.7500.9800.6800.9500.770

Takeaways:

AIV maintains robust defense performance across medium and high reasoning-effort settings. AIV achieves DSRs of 0.920–0.980 across medium and high reasoning efforts, consistently outperforming passive baselines. Increasing reasoning effort improves GPT-5.4's DSR from 0.920 to 0.940, while Claude Sonnet 4.6 maintains a DSR of 0.980 under both settings, indicating stable defense performance across evaluated reasoning efforts.

Increasing reasoning effort cannot replace active questioning and continuous intention tracking. Under high reasoning effort, removing questioning reduces DSR from 0.940 to 0.610 on GPT-5.4 and from 0.980 to 0.680 on Claude Sonnet 4.6. Removing profiling further reduces DSR to 0.800 and 0.770, respectively, demonstrating that increased reasoning effort alone does not replace active verification and continuous intention tracking.

Frontier-model evaluations remain limited to Intention Deception and 100 tasks per model due to API costs and the limited effectiveness of other evaluated attacks on frontier models. The expanded ActorAttack and Qwen3.5 evaluations provide broader coverage across attacks and model architectures, while reasoning-effort comparisons and component ablations provide additional evidence of AIV's effectiveness across model capabilities.

W2 - Strong assumptions underlying the theoretical guarantee

We thank the reviewer for raising this concern. We agree that aggregate empirical estimates do not establish the uniform validity of Assumption 1 across all ambiguous histories and malicious objectives. Proposition 3 establishes sufficient conditions under which active verification can overcome the evidence bottleneck of passive defenses. The theoretical guarantee is conditional on Assumption 1 and does not claim that probing universally satisfies the required conditions.

The intuition behind Assumption 1 does not require perfectly distinguishing benign and malicious users. Benign users generally seek assistance consistent with their legitimate objectives. Even when an attacker maintains fabricated yet internally consistent benign explanations, active probing encourages the attacker to commit to a benign operational context, progressively narrowing the conversation toward a safe scope. Ambiguous requests can be answered within the established benign context, while requests that deviate toward harmful objectives can trigger further verification or refusal. Low downstream harmful assistance can be achieved even when the attacker's true intent remains concealed.

We provide empirical support for Assumption 1 in Appendix C.3 (Table 6). Across four models, benign coverage reaches 98.4%–99.1%, while only 2.6%–4.7% of admitted malicious responses lead to unsafe assistance. Appendix C.5 (Table 10) further evaluates protocol-aware adaptive attackers, with AIV maintaining a DSR of 0.78–0.94 under the combined adaptive attack. The experimental results support the practical relevance of Assumption 1 under the evaluated attack settings, without claiming uniform validity.

Thank you for providing the additional experimental results. I have raised my rating.

Thank you for taking the time to review our response and additional experimental results. We sincerely appreciate your constructive feedback, which has helped us strengthen the paper, and thank you for reconsidering your rating.

Reviewer_qMMSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Active Intention Verification (AIV), which is an interactive defense against multi-turn jailbreaks. Instead of passively classifying conversation history, AIV maintains a running intent profile and asks verification questions when user intent is ambiguous. The authors formalize the limiattions of passive defenses under ambiguous intent using an information bottleneck framework, and gives conditions under which active verification improves safety. Experiments on six LLMs and four multi-turn attacks give higher defense success rates and lower over-refusal than the considered baselines.

Critical strengths

S1. The paper provides an interesting perspective on multi-turn defense using the information bottleneck. Actively gathering evidence about user intent is also well motivated.

S2. The empirical performance is strong, with consistent gains over baselines across six LLMs and four attacks. The authors consider both defense success and over-refusal.

S3. The paper provides a useful theoretical motivation/limitation of passive defenses, and the ablations in the experiment section indicate that active verification drives much of the improvement.

Critical weaknesses

W1. The analysis in the main paper is insufficient. The experiment section consists mainly of two large tables. There is little discussion of how AIV behaves during actual interactions, what the verification questions reveal, or why particular attacks succeed or fail.

W2. The three probe types (clarification, credential, and trap) are also never characterized individually. Are they equally effective? Do we always need all three of them? Which matter most against which attacks? How often is each triggered? Mechanistic analysis and a broader qualitative study of successful and failed interactions in the main text would be valuable.

W3. AIV assumes verification questions elicit distinguishing evidence, but a knowledgeable attacker can still provide fabricated, yet internally consistent, answers to probing questions. What happens when the attacker dynamically generates deceptive answers to arbitrary probes? Stronger adaptive evaluations are needed.

W4. Evaluation setup and the retrieval module need more details. AdvBench, JailbreakBench, and ORBench are all single-turn benchmarks. The paper should specify how the multi-turn attackers convert them into multi-turn jailbreak datasets. And it is also unclear how BenignPairBench is constructed exactly. In addition, how does Wikipedia evidence retrieved by the retrieval module help to distinguish a benign user from an attacker? Generation configurations and thinking efforts for the proprietary models are also missing.

Questions for authors

See weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer qMMS (1/3)

We thank the reviewer for the thoughtful comments. To address your concerns and make the paper more comprehensive, our response is structured as follows: we will first answer W2, W3, and W4. Then, based on these discussions, we will present our response to W1.

W2 - Discussion on Probes

To better characterize the individual contributions of the three probe types, we conduct additional probe-type ablations and analyze their triggering frequencies across different attacks. We also expand the qualitative analysis of successful and failed interactions to clarify the underlying mechanisms and limitations of AIV.

1. Individual Contributions of Three Probe Types

We conduct leave-one-out ablations under Intention Deception, removing one probe type at a time while keeping the remaining AIV components unchanged.

Probe-type Ablation (DSR ↑)

ModelFull AIVNo-ClarificationNo-CredentialNo-Trap
Qwen2.5-7B0.8980.7560.7240.802
GPT-4o0.9220.6960.7900.784
Claude Sonnet 4.60.9800.6360.8100.802
  • All three probe types contribute to AIV's defense performance. Removing any probe type reduces DSR across all three models, indicating that clarification, credential, and trap probes provide complementary benefits. Full AIV consistently achieves the highest DSR.

  • The relative importance of different probe types varies across models. Removing credential probes causes the largest degradation on Qwen2.5-7B (0.898 to 0.724), while removing clarification probes has the largest impact on GPT-4o (0.922 to 0.696) and Claude Sonnet 4.6 (0.980 to 0.636). No single probe type consistently dominates across models.

2. Probe-Type Triggering Frequency

We further analyze the frequency of each probe type across five multi-turn attacks. All percentages represent the proportion of total conversation turns in which the corresponding probe type is triggered.

AttackClarificationCredentialTrap
Crescendo8.44%6.35%2.17%
Intention Deception7.23%9.13%4.30%
ActorAttack8.31%5.32%2.69%
Opposite-Day9.42%4.58%2.54%
Chain-of-Attack7.60%5.77%2.86%
  • AIV selects different probe types according to the characteristics of each attack. Intention Deception frequently relies on claimed benign identities or professional roles, making credential verification particularly relevant (9.13%). Intention Deception also produces dual-use requests, leading to the highest trap-probe frequency (4.30%) among the evaluated attacks.

  • Clarification probes are especially relevant to attacks involving gradual conversational shifts. Crescendo progressively steers conversations toward harmful objectives, making clarification useful for establishing the intended scope of evolving requests (8.44%). Credential probes (6.35%) can also examine inconsistencies when later role claims or objectives conflict with earlier statements.

3. Qualitative Analysis of Successful and Failed Interactions

We further examine successful and failed interactions to understand how different probes contribute to defense and where AIV remains vulnerable.

Successful interactions. Under Intention Deception, attackers often establish a plausible benign identity to justify dual-use requests. Credential probes examine the consistency of claimed roles and objectives, while trap probes test whether attackers pursue directions that are useful for malicious purposes but irrelevant to their stated benign goals. In some interactions, attackers reveal their underlying objectives when responding to trap probes. Even when attackers maintain consistent benign explanations, clarification and credential probes help establish a safe operational scope that constrains subsequent assistance. Under Crescendo, clarification probes help identify the intended scope as requests gradually evolve, while credential probes can reveal inconsistencies between earlier commitments and later claims.

Failed interactions. Our qualitative analysis also identifies two failure modes. First, persistent intention deception can prevent AIV from detecting malicious objectives when attackers maintain internally consistent benign explanations, allowing harmful requests to remain concealed within a plausible benign context. Second, safe-scope enforcement failures can occur when AIV successfully establishes a benign operational context, but the response generator still provides overly specific or actionable information beyond the established scope. The two failure modes highlight remaining challenges in detecting concealed malicious objectives and consistently enforcing conversational safety constraints during response generation.

AuthorsOpenReview ↗

Response to Reviewer qMMS (2/3)

W3 - Stronger adaptive evaluations

We would like to clarify that we have evaluated AIV against both protocol-aware and online adaptive attacks based on Intention Deception, specifically targeting the verification mechanism.

First, Appendix C.5 evaluates protocol-aware variants of Intention Deception, including (a) preparing internally consistent deceptive answers to anticipated clarification and credential probes, and (b) recognizing and avoiding trap probes. The combined adaptation achieves an average DSR of 0.834 across five models, compared with 0.892 under the original Intention Deception attack. Second, we additionally evaluate an online adaptive variant of Intention Deception using the same evaluation setup as Table 10, where the attacker dynamically generates deceptive answers after observing each actual verification question, without relying on pre-staged responses. AIV achieves an average DSR of 0.826 under online adaptive deception, demonstrating robustness even when attackers adapt their responses to the specific probes generated during the interaction.

AttackQwen2.5-7BGPT-4oLlama-3-8BMistral-7BClaude Sonnet 4.6Average
Online Adaptive Intention Deception0.820.840.780.770.920.826

Importantly, AIV does not require perfectly identifying fabricated answers or recovering the attacker's hidden intent. Even when an attacker maintains internally consistent benign explanations, active probing can progressively constrain the conversation toward a safe operational scope. Ambiguous requests can be answered within the established benign context, while attempts to obtain harmful assistance beyond the established scope can trigger further verification or refusal. Successful defense does not necessarily require successful identification of malicious intent.

W4 - details of evaluation setup

We clarify the multi-turn dataset construction, benign benchmark generation, retrieval mechanism, and proprietary-model configurations below.

  1. Multi-turn Attack Construction. AdvBench and JailbreakBench provide single-turn harmful queries, which we use as the underlying attack goals. Each goal is provided to a multi-turn attacker agent, which generates a sequence of conversational requests according to the corresponding attack strategy (e.g., Crescendo, Intention Deception, Opposite-Day, or Chain-of-Attack). The attacker interacts with the defender across multiple turns, adapting subsequent requests based on the defender's responses. The original single-turn queries serve as attack objectives, while the actual evaluation uses the resulting multi-turn interactions.

  2. BenignPairBench Construction. As described in Section 6.1, we construct BenignPairBench by transforming each harmful goal in AdvBench into a semantically related, safety-oriented benign goal. For example, "Give step-by-step instructions for how to steal someone's identity" is transformed into "Give step-by-step instructions for how to protect someone from identity theft." We then initialize a benign user agent with the transformed goal and instruct the agent to generate multi-turn conversations seeking legitimate, defense-oriented assistance, including realistic ambiguity and underspecification. For ORBench, we similarly initialize a benign user agent based on the topic of each original query to generate a 10-turn, topic-consistent conversation, as described in Appendix C.2.

  3. Role of Wikipedia Retrieval. Wikipedia retrieval serves three purposes in AIV: (1) providing relevant background knowledge to help the defender understand the current topic and operational context, (2) supporting the generation of targeted verification questions, and (3) providing factual evidence for evaluating user responses and identifying potential inconsistencies. As detailed in Appendix B, AIV retrieves relevant Wikipedia articles and selects evidence chunks using semantic similarity. Wikipedia evidence is not treated as direct proof of benign or malicious intent. Instead, external knowledge supports context-aware probing and more informed evaluation of user-provided explanations. The no-RAG ablation in Table 1 further demonstrates the contribution of retrieval to defense performance.

  4. Generation Configurations and Thinking Efforts. We used the following configurations for proprietary models. Temperature and top-p were omitted from API requests, leaving sampling behavior to the corresponding API defaults.

ModelTemperature / Top-pMax Output TokensReasoning Effort
GPT-4oAPI default8,000N/A
GPT-5.4API default20,000Medium
Claude Sonnet 4.6API default20,000Medium
AuthorsOpenReview ↗

Response to Reviewer qMMS (3/3)

We additionally evaluated GPT-5.4 and Claude Sonnet 4.6 under high reasoning effort. Under Intention Deception, AIV achieves DSRs of 0.920 and 0.940 on GPT-5.4 with medium and high effort, respectively, and 0.980 under both settings on Claude Sonnet 4.6. The additional evaluations demonstrate consistent defense performance across the evaluated reasoning-effort settings. W1 - Discussion of how AIV behaves during actual interactions We will incorporate the following discussion in the main text and add corresponding results to the appendix:

How AIV behaves during interactions. Our additional probe-type analysis reveals distinct verification patterns across attacks. Under Intention Deception, attackers frequently claim benign professional identities, leading to more credential probes (9.13%). Dual-use requests also trigger trap probes (4.30%), which can expose malicious objectives when attackers pursue directions inconsistent with their stated benign purposes. Under Crescendo, gradual shifts in conversational objectives lead to frequent clarification probes (8.44%), while credential probes can identify inconsistencies between earlier commitments and subsequent claims.

Why active probing improves defense. Verification questions do not necessarily need to uncover the attacker's true hidden intent. Clarification and credential probes encourage users to establish and maintain a consistent benign operational scope, allowing AIV to provide safe assistance for ambiguous requests. Trap probes can additionally reveal malicious objectives when attackers pursue harmful directions. Our probe-type ablations further confirm the complementary contributions of all three mechanisms, as removing any probe type reduces defense success rates across the evaluated models.

Why some attacks still succeed. Our qualitative analysis identifies two failure modes. First, persistent intention deception can conceal malicious objectives when attackers maintain internally consistent benign explanations, preventing AIV from detecting meaningful contradictions. Second, safe-scope enforcement failures can occur when AIV establishes a benign operational context but the response generator still provides actionable information beyond the established scope. Both failure modes reveal limitations in intention verification and downstream response generation.

To further illustrate the mechanism, we add a qualitative example to the Appendix, starting on page 23, which shows how AIV handles an Intention Deception attack in which the attacker consistently maintains a benign explanation, yet the probing process narrows the conversation toward a safe scope and prevents harmful assistance.

Reviewer_oJGJOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors addressed multi-turn jailbreak attacks and identified an evidence bottleneck in passive defenses. The paper argued that observed conversation history may be insufficient to distinguish benign from malicious intent. They propose Active Intention Verification (AIV), a framework that maintains an evolving intent profile and actively asks clarification when user intent is ambiguous. In addition, the authors provided a theoretical analysis of the active verification. The experiments across different attack strategies and LLMs show the improved defense success rates and lower over-refusal rates when comparing with the baselines.

Critical strengths
  1. The paper identifies an important limitation of passive defenses in multi-turn jailbreak scenarios and proposes active intention verification as an alternative. The paper idea is interesting and potentially useful for improving LLM safety.
  2. The authors evaluate AIV across multiple attack strategies and LLMs. The experimental results look promising in defense success rates while maintaining relatively low over-refusal rates.
Critical weaknesses
  1. This is my main concern when reading through the paper. It does not clearly distinguish between identifying malicious intent and restricting responses based on a user's stated benign purpose. A malicious user could maintain a consistent benign explanation throughout the interaction, making it difficult for AIV to identify the true intention. Although the authors evaluate adaptive attacks, it remains unclear whether the improvements come from genuine intention verification or simply constraining the generated responses.
  2. While the ablation studies demonstrate the importance of active questioning, the paper does not compare AIV against a simpler clarification-prompting baseline. I'd like authors to add ablations to compare with a few simple clarification prompts to understand whether the improvements come from the proposed framework or from the general benefit of asking clarifying questions.
Questions for authors
  1. How does AIV distinguish between genuinely identifying malicious intent and simply restricting responses based on a user's stated benign purpose? I hope that the authors can clarify how AIV handles attackers who consistently maintain a plausible benign explanation throughout the interaction.
  2. I think the ablation demonstrates the importance of active questioning, but how does AIV compare with a simpler baseline that asks clarifying questions before responding? If we need to better understand the value of the full framework, could the authors explain what evidence supports the additional value of the full AIV framework beyond basic clarification prompting?
  3. Proposition 3 relies on Assumption 1, which requires the post-verification acceptance region to maintain both high benign coverage and low unsafe assistance probability. Could the authors clarify how these conditions are justified in practice, particularly when malicious users can provide responses indistinguishable from benign users?
7 comments
AuthorsOpenReview ↗

Response to Reviewer oJGJ (1/2)

We thank the reviewer for the constructive comments. We want to address your concerns as follows:

W1 and Q1 - Clarifying the Role of Intention Verification in Safe Response Generation

First, we would like to emphasize that constraining generated responses is ultimately the goal of any defense mechanism, as we aim to provide responses that are both safe and useful for the user's needs. The purpose of intention verification is not necessarily to accurately determine whether a user's hidden intent is benign or malicious, but to guide the conversation toward a safer and more well-defined scope.

We agree that a sophisticated attacker may consistently maintain a convincing benign explanation throughout the interaction. In fact, Intention Deception is precisely such an attack strategy, which we explicitly evaluate in our experiments. Even if an attacker successfully maintains a benign explanation across all turns, active probing can still be effective. By asking the attacker to clarify and commit to a benign purpose, AIV progressively narrows the scope of the conversation, allowing the model to provide safe information consistent with the stated purpose. In other words, even when AIV cannot uncover the attacker's true malicious intent, it can still steer the interaction toward a safe conversational trajectory.

Regarding the concern that the improvements may come from "simply constraining the generated responses," we consider such constraints an intended effect of active intention verification, not a competing explanation. The key contribution of AIV lies in actively shaping the conversation through probing, so that the resulting responses remain safe while addressing the user's stated needs. To further illustrate the mechanism, we add a qualitative example to the Appendix, starting on page 23, which shows how AIV handles an Intention Deception attack in which the attacker consistently maintains a benign explanation, yet the probing process narrows the conversation toward a safe scope and prevents harmful assistance.

W2 and Q2 - Evaluating the Full AIV Framework Beyond Simple Clarification Prompting

To distinguish the benefits of the full AIV framework from general clarification prompting, we conducted additional experiments with a clarification-only baseline, which asks clarifying questions before responding without AIV's structured intention profiling and verification mechanisms.

Comparison with Clarification-only Baseline (DSR ↑)

Each cell reports AIV / Clarify-only. Higher values indicate better defense performance.

ModelCrescendoIntention DeceptionOpposite-DayChain-of-AttackActorAttack
Qwen2.5-7B0.972 / 0.3420.898 / 0.1920.980 / 0.7760.966 / 0.3360.920 / 0.522
Llama-3-8B0.948 / 0.6640.854 / 0.2060.976 / 0.7860.970 / 0.3540.884 / 0.538
Mistral-7B0.914 / 0.4360.826 / 0.1980.890 / 0.5360.852 / 0.5880.912 / 0.540
GPT-4o0.940 / 0.5080.922 / 0.1860.954 / 0.6920.938 / 0.1920.934 / 0.646
GPT-5.4 (medium)–0.920 / 0.590–––
GPT-5.4 (high)–0.940 / 0.680–––
Claude Sonnet 4.6 (medium)–0.980 / 0.690–––
Claude Sonnet 4.6 (high)–0.980 / 0.710–––
Qwen3.5-9B (thinking)0.910 / 0.5220.924 / 0.5420.940 / 0.6780.962 / 0.4360.942 / 0.524
Qwen3.5-9B (non-thinking)0.918 / 0.4420.906 / 0.3160.902 / 0.7420.922 / 0.5480.874 / 0.532

The results show that AIV outperforms clarification-only across all 34 evaluated model–attack configurations, achieving an average Defense Success Rate (DSR) of 92.6% compared to 50.6%, an improvement of 42.0 percentage points. The improvements are particularly significant under Intention Deception, where AIV achieves 89.8% vs. 19.2% on Qwen2.5-7B and 92.2% vs. 18.6% on GPT-4o.

AuthorsOpenReview ↗

Response to Reviewer oJGJ (2/2)

Why is simple clarification insufficient? The key limitation of clarification-only prompting is the lack of a structured mechanism to evaluate and act on the user's answers. A malicious user can provide a convincing benign explanation, and without continuous intention profiling and probe-answer evaluation, the model may trust the stated purpose and proceed with subsequent requests without sufficiently examining their implications. As a result, asking clarifying questions alone does not reliably constrain the subsequent conversation.

AIV addresses the limitation by integrating probing with continuous intention profiling, risk assessment, and retrieval-grounded verification. User responses are not treated as proof of benign intent. Instead, AIV evaluates their implications, tracks the user's commitments across turns, and uses the established benign context to determine the appropriate scope of assistance. Even when an attacker consistently maintains a benign explanation, ambiguous requests can be answered within the corresponding safe scope. Attempts to obtain harmful information beyond the established scope can trigger further verification or refusal. The full framework enables active questioning to progressively shape and narrow the conversation toward a safe operational region, instead of merely accepting the user's explanations at face value.

Q3 -Practical Justification and Empirical Evidence for Assumption

The key intuition behind Assumption 1 is that benign users generally seek benign assistance, while malicious users must eventually obtain information that advances their harmful objectives. Even when an attacker provides fabricated yet internally consistent answers to verification questions, maintaining a benign explanation constrains the conversation to a corresponding safe operational scope. For ambiguous requests, AIV can provide useful information consistent with the established benign context without providing harmful operational details. If the attacker subsequently requests information beyond the benign scope, the deviation provides additional evidence of unsafe intent and can trigger further verification or refusal.

We also provide empirical support for Assumption 1 in Appendix C.3 (Table 6). Across four models, the post-verification acceptance region achieves 98.4%–99.1% benign coverage, while only 2.6%–4.7% of admitted malicious responses lead to unsafe assistance. Furthermore, Appendix C.5 (Table 10) evaluates protocol-aware adaptive attackers that maintain consistent benign explanations and avoid trap probes, where AIV retains a DSR of 0.78–0.94 under the combined adaptive attack. The results provide empirical support for the practical relevance of Assumption 1, even when malicious users strategically conceal their underlying intent.

Dear Reviewer oJGJ,

Thank you once again for your insightful and constructive feedback. As the discussion period is approaching its end, we would like to kindly follow up on our rebuttal. We hope that our clarifications and additional experimental results have addressed your concerns.

We would greatly appreciate your reassessment of our work in light of our responses and would be happy to address any remaining questions or concerns.

Thank you again for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.