- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper answers the question on whether the conversation leading up to a request can make an LLM behave unsafely, even when the final request stays exactly the same (fixed final request). The paper evaluates whether the model fails to refuse or whether it refuses when it shouldn't
Critical strengths
The paper uses a variety of recent LLMs from the Gemini, Claude, Muse, GPT and Grok families. All models are used as attackers and targets. Gemini 3.8 Flash is the judge for harmful-response classification. Figure 1 summarizes really well the study and the tradeoffs of model choice in the paired-setting.
LLMs are really no longer used (or barely used) in a single turn, so addressing vulnerabilities and alignment issues in these settings is crucial to improve ai safety and improve user-agent interactions.
Critical weaknesses
This paper, even if not on red teaming, follows a similar principle. You have a single turn task, and you convert it into a multi-turn one, by maintained a fixed goal/end state. https://arxiv.org/pdf/2505.06120. Not the same problem, but related in the sense that models are more vulnerable to fail in a multi-turn setting. Please consider discussing it future versions of the paper. A perfect place is lines 72-74. In addition, there is some recent or not that recent work that could have been cited https://arxiv.org/abs/2606.01738, https://arxiv.org/pdf/2510.15017, https://arxiv.org/pdf/2503.00187
The paper uses 100 harmful and 100 benign prompts, which are then used for the rollouts. The data is limited to conclude but the cost of running these models is not negligible and this should not be a reason to discard the ideas and methodology.
Questions for authors
why only Gemini-3.8 Flash as judge? I don't think I read the justification behind that choice.
3 comments
Author response addressed my questions
I will maintain my score but i am happy to increase it during the discussion period.
Thank you hivW
Thank you for the review, for responding to our rebuttal, and for kindly agreeing to raising your score during the discussion. We will add the additional references and incorporate other feedback from our discussion into the final paper.
Response to reviewer hivW
W1: Additional multi-turn vulnerability and defense literature
We will add the four suggested references as well as what distinguishes our work. LLMs Get Lost in Multi-Turn Conversation studies the reliability loss that arises when tasks are decomposed across conversation turns. THRD, Active Honeypot Guardrail System, and Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks study detection or defense against multi-turn jailbreak trajectories.
Overall, the multi-turn attacking procedure used in our experiments is not new. The key distinction is that our paper asks about the reusability of the artifacts produced by these procedures. When an attacker model discovers a successful history against one source target, does that frozen history reproduce on the source and transfer well enough to be reused in a broad last-turn replay benchmark or post-training dataset?
Compared with a fully adaptive evaluation whose final metric is discovery success over repeated attack attempts, last-turn replay of frozen artifacts is faster, lower variance, and less expensive. Most SFT/RL post-training pipelines likewise use serialized histories followed by a sampled continuation, and adds to why we are explicitly interested in the last-turn replay setup.
W2: Scope of the 100 harmful and 100 benign prompts
This may seem small, but each objective crosses those requests with nine attackers and nine targets, yielding 8,100 attacker–target–request mining observations, or 16,200 across both objectives before baseline filtering. Mining produced approximately ~1k different successful histories, which were subsequently evaluated across nine destination models with three fresh full-history replay generations per (history, destination) pair, resulting in approximately ~28k evaluable generations after destination-specific cold-baseline gating.
All of the final requests come from WildGuard. This is a single dataset and therefore limits population-wide prevalence claims, but WildGuard was designed with strong diversity and balance considerations for the safety domain and supports both mining objectives in our study.
Q1: Why Gemini 3.8 Flash is the sole judge
For the violation mining pipeline, the Gemini 3.8 Flash judge was validated against WildGuard’s human labels. On 1,705 parseable WildGuardTest examples, Gemini 3.8 Flash achieved 0.935 precision and 0.461 recall for harmful-response classification. We prioritized precision so that flagged harmful responses are usually clear positives rather than maximizing the number of discoveries.
The benign-degradation pipeline uses two separate judgments. The helpfulness delta judge compares the cold and post-history responses and was validated by passing a 400-comparison calibration set (all identical-answer comparisons received a zero delta, no judgments were unparseable, the mean order gap was 0.3175 points on the seven-point scale, and order-reversed responses agreed 70.25% of the time). Only maximal raw drops proceed to the full-context adjudicator, which determines whether the loss of helpfulness was justified by the history. WildGuard does not provide human labels for this derived contextual judgment, so we do not claim an independent human validation set for the adjudicator. The revised paper will report that limitation and the exact rubric.
None of the judges receive the model identity of the response generator and our results do not show any Gemini-specific outcome pattern. For example, Gemini 3.5 Flash has the highest harmful-compliance susceptibility, while Gemini 3.8 Flash is in the middle of the harmful target range and near the low end for benign degradation. As an attacker, Gemini 3.8 Flash was the least effective harmful-compliance attacker, largely because it often declines to participate, but is mid-panel for benign degradation. This is not proof that no judge-family effect exists, but it is inconsistent with a simple self-preference-bias explanation.
We will add judge validation details to the Appendix.