acceptodds
Withdrawn from ICLR 2027

Scalable Intent-Aware Safety for Adversarial LLM Dialogue

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

When a user asks a language model something harmful, is it a genuine attack or a misunderstood but well-meaning question? This ambiguity is one of the central challenges of LLM safety. Assuming the worst harms legitimate users; always assuming the best can be exploited. The problem is compounded in multi-turn dialogue, where an attacker's true intent may only reveal itself gradually across many exchanges. Yet, existing safety frameworks apply a contextual bandit treatment without considering the trajectory of the conversation. To that end, we propose Dialogue Critic Guided Sampling (DCGS), a framework that addresses this by inferring user intent at every turn of dialogue. DCGS learns what inferences about the user's intent are probable based on conversational history, and resamples responses accordingly. Formally, we model adversarial dialogue as a Markov Decision Process and learn value and regret-based critics at both the individual token and utterance (full response) levels, scoring candidate responses via an action-value critic. We prove that this inference-time reweighting approximates exponential tilting of the base policy, guaranteeing improvement in expected return for any finite candidate pool, a property that group-relative objectives do not exhibit. Evaluated on CARES-18k, WildJailbreak, Redbench, SafeDialBench, and Harmbench, DCGS outperforms strong robust baselines and frontier models on adversarial dialogue tasks. Once trained, DCGS transfers to frontier models, improving their robustness without fine-tuning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Policy Improvement, Adversarial Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_cuM8OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper study at a very important research in LLM safety, which is that how can a model tell whether someone is asking an innocent question or just want to to get harmful information and guidance on dangerous things. For example, asking how locks work is reasonable at the first galance, but the same user question could becomes quite harmful if earlier users messages focused on breaking into a particular building.

To solve this problem, this paper propose Dialogue Critic-Guided Sampling. DCGS works in two stages. First, the model generates several candidate interpretations of the user’s question/intented. And a trained scoring model helps to select one candidate response by measuring how well the conversation would go if LLM take that interpretation; (ii) Based on the estimation, the model generates several final replies, and another models helps choose a response.

For the experimental results their proposed method show clear improvement to multiple kinds of attacks.

Critical strengths

This paper studies a quite meaningful safety problem. This paper pointed out that user's harmful question can become clear after the conversation with llm goes on after several turns. Based on this good intuition, the authors study that how the dialogue could be a reasoanble way to address both intented jailbreak attacks and unnecessary refusals of legitimate requests that ask an innocent questions.

Besides, the evaluation is comprehensive. Specifically, the users take both usefulness and safety into consideration. This paper measures both attack resistance and completion of benign requests,. Additionally, they also examine for various length of conversations, and tested on different model families. Generally, the evaluation is sound.

Critical weaknesses

My biggest concern is that for the current experimental results, it is not easy to determined whether the proposed approach, i.e., the two-stage intent modeling, actually drives the improvement or not. Specifically, their method combines several components: (i) candidate intent generatio, response reranking, etc. However, this paper haven't conduct abalation study clearly. My suggestion is that the authors could isolate the response spearately. This abalation study could help to decide it is which component that really carries the improvements. To sum up, more abalation is needed.

Besides, I think the theoretical is a bit of overclaimed. To be honest, I didn't fully understand the theorem. From my understanding, the theorectical results does not gurantee that better safety just because critic score is higher.

I also struggle to see why the proposed regret-based formulation should improve robustness to unknown user intent. It adjusts rewards by removing the lowest value obtained across the sampled intent hypotheses. For a given candidate pool, however, this shared offset leaves the ordering of immediate rewards unchanged (right?). I think the paper needs a clearer argument connecting this reward adjustment to its claimed robustness against unseen intend in my opinion.

Questions for authors

Much of the interactive evaluation uses a simulator that gradually reveals a predefined goal, and Figure 2 only shows five turns. Although the paper also includes SafeDialBench, the results leave open how well the method handles much longer conversations or attackers that specifically try to mislead its intent inference.

Reviewer_UF4hOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

N/A

Critical strengths

N/A

Critical weaknesses

I do not think this is a paper that is ready to be submitted, and the required AI use statement is not included in the submission.

Questions for authors

No question.

Reviewer_phm6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

DCGS models multi-turn adversarial dialogue as an MDP with hidden partner intent. At each turn it samples intent hypotheses from a frozen base model and scores them with a TD-trained critic, optionally combined with a regret critic. It picks one hypothesis by softmax resampling and reranks candidate responses with a token-level critic in the same way. The paper proves that this resampling approximates an exponential tilt of the base policy and improves the expected critic score. With an LLM user simulator, DCGS achieves higher average defense success than existing defenses, and critics trained on Zephyr-7B transfer to GPT-5.4-mini.

Critical strengths
  • The problem is relevant. Intent in multi-turn attacks only becomes visible across turns, and judging responses by their long-horizon consequences addresses a real gap.

  • The method needs only samples, so it applies to API-only models. The transfer to GPT-5.4-mini is the most convincing result. It gives a clear robustness gain at little cost in goal completion, and the gain is larger than from enabling the model’s own reasoning.

Critical weaknesses
  • The novelty is limited and the theory is overstated. Eq. 2 is soft best-of-n sampling (Verdun et al., 2025), which already has a stronger convergence result than Lemma 1. Proposition 1 only guarantees a higher score under the learned critic, not a higher true return. Section 4.1.4 claims that Bradley-Terry RLHF cannot be read as sampling from a value-tilted policy. This is incorrect, since the KL-regularized RLHF optimum is exactly such a tilt, which is the basis of DPO. The intent-then-respond design follows Intention Analysis (Zhang et al., COLING 2025), and the paper does not discuss multi-turn defenses such as RED QUEEN.

  • The central claim, that long-horizon value matters, is not tested. There is no myopic ablation (γ = 0, or reranking by the per-turn guard score) and no variant without the critic. The main tables also lack an undefended Zephyr-7B row, and the base models of most baselines are not stated. So the rows differ in base model as well as in defense.

  • The evaluation is in-distribution and the judging is unclear. Critics are trained and tested against the same simulator and personas, so robustness to other multi-turn attackers such as ActorAttack or X-Teaming is not shown. The partner model and the judges for DSR and GCR are not identified. If LlamaGuard-3 is the judge, the critic is evaluated by its own training reward model.

  • The description is inconsistent in places. For example, the critic is linear in Section 4.1.2 but an MLP in Section 5.1, and β denotes two different quantities.

  • Overall, to me the proposed method is best-of-K reranking with learned critics, and the theory restates known results with one incorrect claim. The experiments neither isolate the multi-turn component nor test unseen attackers.

Questions for authors

Please answer the questions raised in my review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.