- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Active Intention Verification (AIV), which maintains an intent profile and asks targeted probes before assisting or refusing ambiguous, sensitive requests. It combines profiling, risk scoring, retrieval, and clarification, supported by theoretical analysis and multi-turn attack evaluations.
Critical strengths
-
The distinction between passive classification and active evidence gathering provides a useful perspective on multi-turn defense.
-
Component ablations and protocol-aware attacks provide evidence that probing contributes to the reported gains.
-
Evaluations include multiple model families, benign benchmarks, and computational overhead measurements.
Critical weaknesses
-
Limited attack and reasoning-model coverage. ActorAttack is omitted without explanation. Most model–attack combinations use older open-source models, including Qwen2.5-7B, Llama-3-8B, and Mistral-7B, which have limited multi-turn alignment. Frontier closed-source evaluations cover only Intention Deception and 100 tasks per model. Broader testing on contemporary reasoning models, such as Qwen3.5, across attacks, benign tasks, and reasoning-budget settings is needed to support the general robustness claims.
-
Strong assumptions underlying the theoretical guarantee. Assumption 1 requires high benign acceptance and low downstream harmful assistance after verification. The guarantee is therefore conditional on probing already producing the desired safety properties. Aggregate empirical estimates do not establish the assumption uniformly across ambiguous histories and malicious objectives.
Questions for authors
See Weakness
4 comments
Response to Reviewer SDpN (2/2)
Takeaways:
AIV preserves benign helpfulness while maintaining high defense robustness. AIV achieves the lowest over-refusal rates in both thinking (4.4%) and non-thinking (5.2%) modes, substantially outperforming passive baselines. Combined with high DSRs across five attacks, AIV demonstrates a favorable safety–utility balance on Qwen3.5-9B.
Active verification and context-aware evaluation contribute to reducing unnecessary refusals. Removing questioning increases over-refusal to 15.2% and 17.4% in thinking and non-thinking modes, respectively. Removing profiling or retrieval also increases over-refusal in both configurations, indicating that active verification and context-aware evaluation help preserve legitimate assistance instead of relying on broad refusal.
4. Evaluation Across Reasoning-Effort Settings
We further extend proprietary-model evaluations to medium and high reasoning-effort settings on GPT-5.4 and Claude Sonnet 4.6, including component ablations.
Intention Deception — Defense Success Rate (DSR ↑)
| Model | Effort | TCA | NBF | AIV | No-Q | No-RAG | No-profile |
|---|---|---|---|---|---|---|---|
| GPT-5.4 | Medium | 0.540 | 0.670 | 0.920 | 0.600 | 0.850 | 0.790 |
| GPT-5.4 | High | 0.570 | 0.690 | 0.940 | 0.610 | 0.910 | 0.800 |
| Claude Sonnet 4.6 | Medium | 0.650 | 0.720 | 0.980 | 0.630 | 0.940 | 0.760 |
| Claude Sonnet 4.6 | High | 0.640 | 0.750 | 0.980 | 0.680 | 0.950 | 0.770 |
Takeaways:
AIV maintains robust defense performance across medium and high reasoning-effort settings. AIV achieves DSRs of 0.920–0.980 across medium and high reasoning efforts, consistently outperforming passive baselines. Increasing reasoning effort improves GPT-5.4's DSR from 0.920 to 0.940, while Claude Sonnet 4.6 maintains a DSR of 0.980 under both settings, indicating stable defense performance across evaluated reasoning efforts.
Increasing reasoning effort cannot replace active questioning and continuous intention tracking. Under high reasoning effort, removing questioning reduces DSR from 0.940 to 0.610 on GPT-5.4 and from 0.980 to 0.680 on Claude Sonnet 4.6. Removing profiling further reduces DSR to 0.800 and 0.770, respectively, demonstrating that increased reasoning effort alone does not replace active verification and continuous intention tracking.
Frontier-model evaluations remain limited to Intention Deception and 100 tasks per model due to API costs and the limited effectiveness of other evaluated attacks on frontier models. The expanded ActorAttack and Qwen3.5 evaluations provide broader coverage across attacks and model architectures, while reasoning-effort comparisons and component ablations provide additional evidence of AIV's effectiveness across model capabilities.
W2 - Strong assumptions underlying the theoretical guarantee
We thank the reviewer for raising this concern. We agree that aggregate empirical estimates do not establish the uniform validity of Assumption 1 across all ambiguous histories and malicious objectives. Proposition 3 establishes sufficient conditions under which active verification can overcome the evidence bottleneck of passive defenses. The theoretical guarantee is conditional on Assumption 1 and does not claim that probing universally satisfies the required conditions.
The intuition behind Assumption 1 does not require perfectly distinguishing benign and malicious users. Benign users generally seek assistance consistent with their legitimate objectives. Even when an attacker maintains fabricated yet internally consistent benign explanations, active probing encourages the attacker to commit to a benign operational context, progressively narrowing the conversation toward a safe scope. Ambiguous requests can be answered within the established benign context, while requests that deviate toward harmful objectives can trigger further verification or refusal. Low downstream harmful assistance can be achieved even when the attacker's true intent remains concealed.
We provide empirical support for Assumption 1 in Appendix C.3 (Table 6). Across four models, benign coverage reaches 98.4%–99.1%, while only 2.6%–4.7% of admitted malicious responses lead to unsafe assistance. Appendix C.5 (Table 10) further evaluates protocol-aware adaptive attackers, with AIV maintaining a DSR of 0.78–0.94 under the combined adaptive attack. The experimental results support the practical relevance of Assumption 1 under the evaluated attack settings, without claiming uniform validity.
Thank you for providing the additional experimental results. I have raised my rating.
Thank you for taking the time to review our response and additional experimental results. We sincerely appreciate your constructive feedback, which has helped us strengthen the paper, and thank you for reconsidering your rating.
Response to Reviewer SDpN (1/2)
We thank the reviewer for the detailed comments. We want to address your concerns as follows:
W1 - Limited attack and reasoning-model coverage We have conducted additional experiments covering ActorAttack, Qwen3.5-9B in both thinking and non-thinking modes, and higher reasoning-effort settings for proprietary models. We also include component ablations to examine whether AIV's individual mechanisms remain effective across additional attacks and model configurations.
1. Additional Evaluation on ActorAttack
We extend our evaluation to ActorAttack across four models, including three open-source models and GPT-4o, with component ablations.
Takeaways:
AIV generalizes effectively to ActorAttack. AIV achieves DSRs of 0.884–0.934 across four models, outperforming all passive baselines. ActorAttack evaluation extends attack coverage beyond the four multi-turn attack methods originally evaluated.
Active probing remains the most important component of AIV under ActorAttack. Removing active questioning reduces average DSR from 0.913 to 0.403, while removing intention profiling or retrieval reduces DSR to 0.641 and 0.851, respectively. Active probing contributes the largest improvement, with profiling and retrieval providing complementary benefits under ActorAttack.
2. Evaluation on Contemporary Reasoning Models Across Multiple Attacks
We further evaluate Qwen3.5-9B in both thinking and non-thinking modes across five multi-turn attacks: Crescendo, Intention Deception, Opposite-Day, Chain-of-Attack, and ActorAttack, including component ablations.
Defense Success Rate (DSR ↑)
Takeaways:
AIV maintains strong defense performance across both reasoning configurations and all five attacks. AIV outperforms all passive baselines across all ten configurations, achieving average DSRs of 0.936 in thinking mode and 0.904 in non-thinking mode. Under Intention Deception, AIV achieves 0.924 and 0.906, compared with only 0.164 and 0.146 for the strongest passive baselines, demonstrating robustness against concealed malicious intent in both reasoning configurations.
Model reasoning capability does not eliminate the benefits of active verification and intention profiling. Across both modes and five attacks, removing questioning reduces average DSR from 0.920 to 0.454, while removing profiling and retrieval reduces DSR to 0.574 and 0.880, respectively. The consistent performance degradation demonstrates that model reasoning capability does not eliminate the benefits of active probing and cross-turn intention tracking.
3. Benign Helpfulness on Qwen3.5-9B
We additionally evaluate over-refusal on benign tasks under both Qwen3.5-9B configurations to assess whether improved defense robustness compromises benign helpfulness.
Over-refusal Rate (OR ↓)