acceptodds
Under review as a conference paper at ICLR 2027

When Conversation is the Attack: Mining Transferable Contexts for Harmful Compliance and Over-Refusal

Abstract

Single-turn prompts are insufficient for evaluating the safety of agents that operate over extended conversations. We study automated conversation-history mining in which an attacker model iteratively constructs a conversation history preceding a fixed final request to discover contexts that induce either harmful compliance or an over-refusal. We evaluate a full 9×9 attacker–target panel using balanced sam- ples of 100 harmful and 100 benign prompts from WildGuardTest. For suscep- tibility, harmful-compliance discovery rates range from 1.2–14.5% while benign over-refusal discovery rates range from 0.7–21.3%. Claude 5 Sonnet and Muse Spark 1.0 are relatively resistant to harmful-compliance mining but susceptible to over-refusal mining, whereas Gemini 3.5 Flash shows the opposite. GPT-5.6 Terra and GPT-5.6 Sol are relatively robust to both objectives, while Grok 4.6 and 4.3 are susceptible to both. For attacker effectiveness, Grok 4.6 achieves the highest harmful-compliance discovery rate, whereas GPT-5.6 Terra achieves the highest over-refusal discovery rate. Self-attack provides no clear advantage. For trajec- tory transferability, successfully mined histories reproduce on their source mod- els in 52–57% of fresh generations but transfer to other models in only 14–17% under prefill-style last-turn replay. This large source-to-transfer gap shows that au- tomatically mined histories should not be assumed to be model-neutral benchmark items. Safety benchmarks based on automatic red-teaming methods can substan- tially favor or disadvantage models depending on which source models were used to supply histories.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
large language models, AI safety, automated red teaming, multi-turn attacks, conversational robustness, harmful compliance, over-refusal, attack transferability

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hivWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper answers the question on whether the conversation leading up to a request can make an LLM behave unsafely, even when the final request stays exactly the same (fixed final request). The paper evaluates whether the model fails to refuse or whether it refuses when it shouldn't

Critical strengths

The paper uses a variety of recent LLMs from the Gemini, Claude, Muse, GPT and Grok families. All models are used as attackers and targets. Gemini 3.8 Flash is the judge for harmful-response classification. Figure 1 summarizes really well the study and the tradeoffs of model choice in the paired-setting.

LLMs are really no longer used (or barely used) in a single turn, so addressing vulnerabilities and alignment issues in these settings is crucial to improve ai safety and improve user-agent interactions.

Critical weaknesses

This paper, even if not on red teaming, follows a similar principle. You have a single turn task, and you convert it into a multi-turn one, by maintained a fixed goal/end state. https://arxiv.org/pdf/2505.06120. Not the same problem, but related in the sense that models are more vulnerable to fail in a multi-turn setting. Please consider discussing it future versions of the paper. A perfect place is lines 72-74. In addition, there is some recent or not that recent work that could have been cited https://arxiv.org/abs/2606.01738, https://arxiv.org/pdf/2510.15017, https://arxiv.org/pdf/2503.00187

The paper uses 100 harmful and 100 benign prompts, which are then used for the rollouts. The data is limited to conclude but the cost of running these models is not negligible and this should not be a reason to discard the ideas and methodology.

Questions for authors

why only Gemini-3.8 Flash as judge? I don't think I read the justification behind that choice.

3 comments
AuthorsOpenReview ↗

Response to reviewer hivW

W1: Additional multi-turn vulnerability and defense literature

We will add the four suggested references as well as what distinguishes our work. LLMs Get Lost in Multi-Turn Conversation studies the reliability loss that arises when tasks are decomposed across conversation turns. THRD, Active Honeypot Guardrail System, and Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks study detection or defense against multi-turn jailbreak trajectories.

Overall, the multi-turn attacking procedure used in our experiments is not new. The key distinction is that our paper asks about the reusability of the artifacts produced by these procedures. When an attacker model discovers a successful history against one source target, does that frozen history reproduce on the source and transfer well enough to be reused in a broad last-turn replay benchmark or post-training dataset?

Compared with a fully adaptive evaluation whose final metric is discovery success over repeated attack attempts, last-turn replay of frozen artifacts is faster, lower variance, and less expensive. Most SFT/RL post-training pipelines likewise use serialized histories followed by a sampled continuation, and adds to why we are explicitly interested in the last-turn replay setup.

W2: Scope of the 100 harmful and 100 benign prompts

This may seem small, but each objective crosses those requests with nine attackers and nine targets, yielding 8,100 attacker–target–request mining observations, or 16,200 across both objectives before baseline filtering. Mining produced approximately ~1k different successful histories, which were subsequently evaluated across nine destination models with three fresh full-history replay generations per (history, destination) pair, resulting in approximately ~28k evaluable generations after destination-specific cold-baseline gating.

All of the final requests come from WildGuard. This is a single dataset and therefore limits population-wide prevalence claims, but WildGuard was designed with strong diversity and balance considerations for the safety domain and supports both mining objectives in our study.

Q1: Why Gemini 3.8 Flash is the sole judge

For the violation mining pipeline, the Gemini 3.8 Flash judge was validated against WildGuard’s human labels. On 1,705 parseable WildGuardTest examples, Gemini 3.8 Flash achieved 0.935 precision and 0.461 recall for harmful-response classification. We prioritized precision so that flagged harmful responses are usually clear positives rather than maximizing the number of discoveries.

The benign-degradation pipeline uses two separate judgments. The helpfulness delta judge compares the cold and post-history responses and was validated by passing a 400-comparison calibration set (all identical-answer comparisons received a zero delta, no judgments were unparseable, the mean order gap was 0.3175 points on the seven-point scale, and order-reversed responses agreed 70.25% of the time). Only maximal raw drops proceed to the full-context adjudicator, which determines whether the loss of helpfulness was justified by the history. WildGuard does not provide human labels for this derived contextual judgment, so we do not claim an independent human validation set for the adjudicator. The revised paper will report that limitation and the exact rubric.

None of the judges receive the model identity of the response generator and our results do not show any Gemini-specific outcome pattern. For example, Gemini 3.5 Flash has the highest harmful-compliance susceptibility, while Gemini 3.8 Flash is in the middle of the harmful target range and near the low end for benign degradation. As an attacker, Gemini 3.8 Flash was the least effective harmful-compliance attacker, largely because it often declines to participate, but is mid-panel for benign degradation. This is not proof that no judge-family effect exists, but it is inconsistent with a simple self-preference-bias explanation.

We will add judge validation details to the Appendix.

Reviewer_hivWOpenReview ↗

Author response addressed my questions

I will maintain my score but i am happy to increase it during the discussion period.

AuthorsOpenReview ↗

Thank you hivW

Thank you for the review, for responding to our rebuttal, and for kindly agreeing to raising your score during the discussion. We will add the additional references and incorporate other feedback from our discussion into the final paper.

Reviewer_gUzbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the transferability and replicability of automatically-constructed conversation history attacks for both harmful compliance and over-refusal. It evaluates across nine LLMs as attacker and defender, and uses an LLM as a judge to identify conversation traces that are successful in inducing the target behavior where it did not exist in the first turn, then replays those conversations across all target models. It finds that attacks do not necessarily transfer or replicate, that attacker models' success and transferability are not linked, that over-refusal and harmful compliance are not linked, and that self-attack is not necessarily an advantage, all of which have implications for LLM generated multi-turn attack benchmarks.

Critical strengths

The paper offers an excellent and comprehensive discussion of attack success and failure for each model, and runs experiments to disentangle factors like attacker success vs transferability, attacker failure vs attacker refusal, and a warranted vs unwarranted loss of helpfulness in a conversation. This provides a better and more robust discussion, as well as adding to the credibility of the results and their implications for the broader community.

Critical weaknesses

The paper identifies that transferability and replicability of automatically mined conversation traces is low and highlights the implications for benchmark creation but does not discuss the methods used (specifically, if considering the transferability and replicability of attacks has been previously investigated and how attacker/target models are chosen and used) to create currently used benchmarks, nor link the set-up of these experiments to common set-ups for benchmark creation. Additionally motivating the experimental design with comparisons to benchmarks cited in the Related Work (including PAIR, Many-shot Jailbreaking, and GOAT) would significantly strengthen the paper.

While the use of all LLMs as attackers and targets, as well as the necessity of rerunning all successful traces, adds complexity to scaling up, 200 prompts from a single dataset is a fairly small test set and the paper does not contain sufficient information to replicate the experiments. Additionally, the paper does not include further information on the prompts (are specific prompts or categories of prompt more likely to lead to successful attacks?). Appendix B is interesting and adds depth to the style of prompt, but does not appear to take into consideration that the adversarial-style prompts in WildGuard are applying a jailbreaking technique even in single-turn. The paper also does not report confidence intervals, p-values, or standard deviations for any statistical analyses.

Questions for authors

What verification of the LLM judge--for the harmful compliance judge, the benign degradation judge, and the adjudicator judge--did you complete? How did you select the Gemini 3.8 Flash judge, and did you find any self-bias in the judge toward the Gemini 3.8 Flash target/attacker? The paper notes that replays were done with a fixed prefill method and a temperature of 0 -- how much did the text generation on the same model differ across the replications? Does using the default recommended temperature setting for LLMs impact transferability or replicability rates (future work)? Do you plan to release code or a dataset to assist with reproducibility? When investigating transferability, did you account for whether the new target models demonstrated harmful compliance or overrefusal on the original prompt?

4 comments
AuthorsOpenReview ↗

Response to reviewer gUzb (1/3)

W1: Relationship to prior attacks and benchmark construction

> The paper should connect its setup more directly to PAIR, Many-shot Jailbreaking, GOAT, and common benchmark-construction practices.

PAIR, Many-shot Jailbreaking, GOAT, and Crescendo are already cited and discussed in Related Work. The multi-turn attacking procedure used in our experiments is not new.

The key distinction is that our paper asks what is reusability of the artifacts produced by multi-turn attacking procedures. Compared to fully adaptive benchmarks where the final metric is something like ASR = (# successful attacks / # attempts), last-turn replay of frozen artifacts is faster, lower variance, and a less expensive procedure, and it is how mined conversations are already being reused in practice.

As benchmarks: SafeDialBench [1] constructs its dialogues against assistant models and then supplies them as fixed “golden context” histories when evaluating other LLMs, and MHJ [2] releases human multi-turn jailbreaks as a standalone dataset with the model completions removed. As post-training data: SafeMTData [3] and XGuard-Train [4] are multi-turn conversations generated by an automated attacker and then used for safety fine-tuning.

If automatically mined histories are to become benchmark items or used to make post-training datasets, our results show that models with the highest discovery success do not necessarily produce the most portable histories, and successful histories reproduce significantly more often on their source model than off diagonal. The full 9X9 design of our experiments is what enables us to measure the source-to-target transferability gap evenly.

[1] Hongye Cao, Sijia Jing, Yanming Wang, Ziyue Peng, Zhixin Bai, Zhe Cao, Meng Fang, Fan Feng, Jiaheng Liu, Boyan Wang, Tianpei Yang, Jing Huo, Yang Gao, Fanyu Meng, Xi Yang, Chao Deng, and Junlan Feng. SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks. ICLR 2026.

[2] Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue. LLM Defenses Are Not Robust to Multi-Turn Human Jailbreaks Yet. arXiv:2408.15221, 2024.

[3] Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. Derail Yourself: Multi-turn LLM Jailbreak Attack through Self-discovered Clues. arXiv:2410.10700, 2024.

[4] Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, and Saadia Gabriel. X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents. COLM 2025.

AuthorsOpenReview ↗

Response to reviewer gUzb (2/3)

W2: Sample size, prompt structure, statistical support, and reproducibility

> The study uses 200 prompts from one dataset

The study uses 200 unique final requests (100 for each objective), which may seem small, but each objective crosses 100 requests with nine attackers and nine targets, yielding 8,100 attacker–target–request mining observations, or 16,200 across both objectives before baseline filtering. Mining produced ~1k different successful histories, which are subsequently evaluated across nine destination models with three fresh full-history replay generations per (trace, destination) pair, resulting in ~28k evaluable generations after destination-specific cold-baseline gating.

It's true that all of our data comes from a single dataset (WildGuard), however, this dataset was already designed with strong diversity and balance considerations for the safety domain, fit for both of the mining objectives of our study.

> omits uncertainty estimates

We have now computed request-clustered bootstrap confidence intervals. Harmful-compliance discovery is 5.70% (95% CI: 3.96–7.68%), and adjudicated benign-degradation discovery is 8.41% (6.68–10.30%). Harmful source reproduction is 56.83% (50.79–62.52%) versus 17.08% (10.78–22.93%) off diagonal. Benign source reproduction is 52.03% (48.40–55.21%) versus 13.95% (11.12–17.04%) off diagonal. These intervals do not change the central source-versus-transfer result.

> does not analyze prompt types

The prompt sample was stratified rather than treated as homogeneous. The harmful sample contains 50 adversarial-style and 50 vanilla prompts, all with unanimous harmfulness labels. The benign sample contains four equal strata crossing adversarial versus vanilla style with unanimous versus two-of-three benign labels. Harmful discovery is higher on vanilla than adversarial-style prompts (7.24% versus 4.18%), while benign degradation is higher on adversarial-style prompts (13.16% versus 3.96%). The benign rate is nearly unchanged across the original human-agreement strata (8.23% for unanimous versus 8.61% for two-of-three). Harmful category rates also vary substantially, from 0.15% for cyberattack and 0.46% for copyright to 13.89% for disinformation, 15.84% for toxic or hateful content, and 20.50% for stereotypes and discrimination. We will add these descriptive breakdowns and explicitly note that WildGuard's adversarial-style stratum already contains single-turn jailbreak framing. Category-specific estimates are descriptive because some categories are small.

> does not provide enough information to replicate the experiment.

For reproducibility, the paper already includes the exact attacker and judge prompts. We plan to release the sampling code, and the exact set of seeds used as well as derived histories and provider outputs to the extent permitted by provider terms and the publication review process.

Q1: Judge validation, selection, and possible Gemini self-bias

Gemini 3.8 Flash was selected as a fixed judge after validation against WildGuard's human response labels. On 1,705 parseable WildGuardTest examples, it achieved 0.935 precision and 0.461 recall for harmful-response classification. We prioritized high precision so that the harmful-compliance analysis counts clear failures rather than maximizing the number of flagged positives.

The benign-degradation pipeline uses two separate judgments. The helpfulness delta judge compares the cold and post-history responses and was validated by passing a 400-comparison calibration set (all identical-answer comparisons received a zero delta, no judgments were unparseable, the mean order gap was 0.3175 points on the seven-point scale, and order-reversed responses agreed 70.25% of the time). Only maximal raw drops proceed to the full-context adjudicator, which determines whether the loss of helpfulness was justified by the history. WildGuard does not provide human labels for this derived contextual judgment, so we do not claim an independent human validation set for the adjudicator. The revised paper will report that limitation and the exact rubric.

None of the judges receive the response generator's model identity and our results do not show a simple Gemini-favoring signature. Gemini 3.5 Flash has the highest harmful-compliance susceptibility, while Gemini 3.8 Flash is mid-panel for harmful compliance and near the low end for benign degradation. As an attacker, Gemini 3.8 Flash is the least effective harmful-compliance attacker because it often declines to participate, but is mid-panel for benign degradation. This cannot rule out every judge-family effect, but it is inconsistent with a simple self-favoring explanation.

We will state explicitly that the study instantiates one fixed, model-blind operational rubric and that judgment sensitivity remains a limitation, as it also does in WildGuard's non-unanimous human annotations.

AuthorsOpenReview ↗

Response to reviewer gUzb (3/3)

Q2: Sampling temperature and replay variation

The attacker was sampled at temperature 1.0. Target and judge calls during mining used temperature 0, as did replay. A history entered the replay corpus after its first judged success (confirm_samples=1), so replay estimates whether that selected event persists over fresh hosted-model generations rather than assuming deterministic reproduction.

We measured behavioral rather than lexical variation across the three replays. Among traces with all three source generations observed, harmful histories reproduced 0/3, 1/3, 2/3, and 3/3 times for 89 (21.3%), 85 (20.4%), 103 (24.7%), and 140 (33.6%) traces. The corresponding benign counts are 184 (30.4%), 102 (16.8%), 117 (19.3%), and 203 (33.5%). Thus, 78.7% of harmful histories and 69.6% of benign histories reproduce at least once, while approximately one-third reproduce in all three generations.

Provider-recommended default temperatures would define a different evaluation and may change absolute reproduction and transfer rates. The present experiment fixes temperature across all models to isolate attacker, target, and source-destination structure under one controlled replay protocol.

Q3: Code and data release

Yes. We plan to release the experiment prompts, seed manifests, model and sampling configuration, filtering and adjudication logic, and analysis code. We also intend to release derived histories and outputs where provider terms and the publication review process allow.

Q4: Destination models' original cold behavior

Transfer is conditioned on each destination model's own fresh cold behavior on the fixed request. A trace-destination observation is included only if that destination handled the request correctly cold. A model that already exhibits harmful compliance or benign degradation does not enter that denominator. We will state this gate explicitly in the methodology and in the Table 4 and Figure 3 captions.

Thank you for your detailed response. I maintain my original score, and note that my concerns remain on:

  • Judge validation: a multi-turn setting and a single turn setting are fundamentally different, and validation on one does not necessarily transfer to the other. Agreement in the helpfulness judges delta also appears low
  • The scale of differences in reproducibility on a fixed model with temperature 0 appear high -- I apologize for an unclear question, but I was asking how different the responses themselves are qualitatively, not how frequently they were scored differently. A variation in score could be attributed either to the generation model or the grader.
  • I apologize again for an unclear question, and I appreciate that the discussion of other benchmarks exists in the Related Works, but my question was about how this measurement of transferability/reusability differs from the analyses those benchmarks already measure. PAIR and Crescendo, for example, both contain discussions about transferability in the original paper.

Thank you again for this response, I appreciate especially the additional information on confidence intervals, the make-up of the WildGuard prompts, and the additional information on transfer being conditioned on the model's own fixed cold behavior.

Reviewer_12JtOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies automated conversation-history mining. An attacker LLM builds up to four setup turns before a fixed final request, aiming to induce either harmful compliance on harmful seeds or unjustified helpfulness loss on benign seeds. The authors run a fully crossed 9×9 attacker–target panel of frontier models on 100 harmful and 100 benign WildGuardTest prompts, with Gemini 3.8 Flash as judge. Target susceptibility ranges from 1.2–14.5% for harmful compliance and 0.7–21.3% for adjudicated benign degradation, and the two objectives are largely uncorrelated across targets. The central finding concerns static last-turn replay: mined histories reproduce on their source model in 52–57% of fresh generations but transfer to other models in only 14–17%. The authors conclude that mined histories should not be treated as model-neutral benchmark items.

Critical strengths
  1. A mining procedure that converts labeled single-turn prompts into multi-turn evaluation histories while holding the final request fixed. It covers both harmful compliance and over-refusal, and uses a context-aware adjudicator to separate justified from unjustified withholding.
  2. A fully crossed 9×9 panel that separates attacker effectiveness, target susceptibility, and attacker engagement.
  3. A replay analysis that separates same-source reproduction from cross-model transfer. It shows a large source-dependence gap, which matters for benchmark construction.
Critical weaknesses
  1. The transfer claim lacks a null baseline and statistical support. It is unclear whether destination models already fail the seed cold, which could bias the transfer rates. No confidence intervals or significance tests are reported, and correlations over nine models are underpowered. Source reproduction may also reflect selection effects, since traces are accepted after a single success.
  2. The judge is single, unvalidated, and part of the panel. Gemini 3.8 Flash assigns all outcomes while also being an attacker and target, with no human agreement study or multi-judge check.
  3. Scope is limited and the methodology is incremental. The study uses 200 prompts from one dataset, a constrained attack budget with small per-cell counts, and static replay only, so the causes of the transfer gap cannot be separated. The mining loop is close to existing multi-turn methods (GOAT, Crescendo, PAIR).
Questions for authors
  1. Is cross-model transfer computed relative to each destination model’s own turn-zero behavior on the seed? Could you report transfer restricted to destinations that handled the seed correctly cold? Could you also report a null baseline, such as replaying non-mined or shuffled histories, or the seed alone?
  2. What sampling temperature was used during mining? Given that replay uses temperature-zero generations, what accounts for the ~45% source-reproduction failure? What is the distribution of traces reproducing 0/3, 1/3, 2/3, and 3/3?
  3. Could you clarify some count discrepancies? 450 harmful flips vs. 446 replayed traces, and 641 adjudicated benign failures vs. 627 traces. Table 4 reports 1,251 source-replay generations, while 446 traces × 3 generations would give 1,338.
6 comments
AuthorsOpenReview ↗

Response to reviewer 12Jt (1/3)

W1: Transfer baselines, statistical support, and selection effects

> It is unclear whether destination models already fail the seed cold, which could bias the transfer rates.

Transfer is conditioned on each target model’s own cold behavior. For every (trace, target model) pair, we only include the trace if that target model handled the single-turn base prompt correctly. If the original base prompt was already handled poorly wrt the target objective, it is impossible for the history-enriched trace to demonstrate attack success. This eligibility gate is consistently implemented throughout the paper and is reflected in Table 4's denominators. We agree that the final manuscript did not state the transfer-specific gate prominently enough, and we will make it explicit in the methodology, Table 4 caption, and Figure 3 caption.

> No confidence intervals or significance tests are reported

We agree that uncertainty is important for assessing generalization beyond the sampled requests. Our original presentation of results excludes CIs because our goal was not to establish a statistically significant ranking among closely separated models.

We have now computed request-clustered bootstrapped confidence intervals. The key measurements relevant to the paper are:

  • Harmful-compliance discovery: 5.70% (95% CI: 3.96–7.68%).
  • Benign-degradation discovery: 8.41% (6.68–10.30%).
  • Harmful source reproduction: 56.83% (50.79–62.52%) versus off-diagonal transfer of 17.08% (10.78–22.93%), a gap of 39.76 points (34.45–45.81).
  • Benign source reproduction: 52.03% (48.40–55.21%) versus off-diagonal transfer of 13.95% (11.12–17.04%), a gap of 38.08 points (34.95–41.20).

These intervals reinforce our central finding that there is a significant transferability gap between the traces that were mined successfully on a source model and how effectively those traces transfer to a target model. We will report the complete set of intervals in new appendix tables.

> correlations over nine models are underpowered

We agree that correlations over nine models have limited power. Under a standard Pearson test, the harmful discovery to portability relationship is not distinguishable from zero (r=.28r=.28, p=.46p=.46), whereas the benign relationship is strongly positive (r=.84r=.84, p=.0048p=.0048). The cross-objective Spearman estimate is near zero (ρ=−.07\rho=-.07, exact p=.88p=.88). We will remove the term "decorrelated," report the estimates as descriptive, and avoid population-level claims from nine models. Descriptively, the panel contains models that are robust to both objectives, susceptible to both, and comparatively stronger on one objective than the other, and no single monotonic trade-off summarizes all nine models.

> Source reproduction may also reflect selection effects, since traces are accepted after a single success.

This is why we explicitly re-measure source reproduction with fresh generations, which we find to be around 52–57% (after the cold eligibility gate). More details are discussed in our response to Q2 below.

W2: Use and validation of a single Gemini judge

For the violation mining pipeline, the Gemini 3.8 Flash judge was validated against WildGuard’s human labels. On 1,705 parseable WildGuardTest examples, Gemini 3.8 Flash achieved 0.935 precision and 0.461 recall for harmful-response classification. We prioritized precision so that flagged harmful responses are usually clear positives rather than maximizing the number of discoveries.

The benign-degradation pipeline uses two separate judgments. The helpfulness delta judge compares the cold and post-history responses and was validated by passing a 400-comparison calibration set (all identical-answer comparisons received a zero delta, no judgments were unparseable, the mean order gap was 0.3175 points on the seven-point scale, and order-reversed responses agreed 70.25% of the time). Only maximal raw drops proceed to the full-context adjudicator, which determines whether the loss of helpfulness was justified by the history. WildGuard does not provide human labels for this derived contextual judgment, so we do not claim an independent human validation set for the adjudicator. The revised paper will report that limitation and the exact rubric.

None of the judges receive the model identity of the response generator and our results do not show any Gemini-specific outcome pattern. For example, Gemini 3.5 Flash has the highest harmful-compliance susceptibility, while Gemini 3.8 Flash is in the middle of the harmful target range and near the low end for benign degradation. As an attacker, Gemini 3.8 Flash was the least effective harmful-compliance attacker, largely because it often declines to participate, but is mid-panel for benign degradation. This is not proof that no judge-family effect exists, but it is inconsistent with a simple self-preference-bias explanation.

We will add judge validation details to the Appendix.

AuthorsOpenReview ↗

Response to reviewer 12Jt (2/3)

W3: Scope and relationship to prior multi-turn methods

> The study uses 200 prompts from one dataset.

The study uses 200 unique final requests (100 for each objective), which may seem small, but each objective crosses 100 requests with nine attackers and nine targets, yielding 8,100 attacker–target–request mining observations, or 16,200 across both objectives before baseline filtering. Mining produced ~1k different successful histories, which are subsequently evaluated across nine destination models with three fresh full-history replay generations per (trace, destination) pair, resulting in ~28k evaluable generations after destination-specific cold-baseline gating.

It's true that all of our data comes from a single dataset (WildGuard), however, this dataset was already designed with strong diversity and balance considerations for the safety domain, fit for both of the mining objectives of our study.

> The mining loop is close to existing multi-turn methods (GOAT, Crescendo, PAIR).

The multi-turn attacking procedure is not itself new and PAIR, Crescendo, Many-shot Jailbreaking, and GOAT are already cited and described in Related Work. Our contribution is studying the reusability of the artifacts that are produced by an attacking procedure: we fully cross attacker and target identity, separate discovery from fresh reproduction and unchanged transfer, and evaluate both harmful-compliance and benign-degradation mining.

> static replay only

Compared to fully adaptive benchmarks where the final metric is something like ASR = (# successful attacks / # attempts), last-turn replay of frozen artifacts is faster, lower variance, and a less expensive benchmarking procedure. Most SFT/RL post-training pipelines also use serialized histories followed by a sampled continuation, which is also why we focus on testing mined artifacts for last-turn replay transferability. The paper already discusses static transcripts with last turn replay vs. other alternatives.

Q1: Destination cold behavior and a null-history baseline

> Is cross-model transfer computed relative to each destination model’s own turn-zero behavior on the seed?

Yes. Cross-model transfer is computed relative to each destination model’s own fresh cold behavior on the fixed request. A trace–destination observation is eligible only if that destination succeeds cold.

> Could you report transfer restricted to destinations that handled the seed correctly cold? Could you also report a null baseline, such as replaying non-mined or shuffled histories, or the seed alone?

Within the eligible transfer population, the seed-alone control is already accounted for – the induced-failure rate is zero by construction.

We are happy to run an additional unsuccessful history baseline to check whether selected histories outperform generic foreign context. We are currently collecting this data and will report it if we are able to complete this during the rebuttal period.

AuthorsOpenReview ↗

Response to reviewer 12Jt (3/3)

Q2: Mining temperature and approximately 45% source non-reproduction

> What sampling temperature was used during mining? Given that replay uses temperature-zero generations, what accounts for the ~45% source-reproduction failure?

The attackers were sampled at temperature 1.0. Target and judge calls during mining and replay used temperature 0. A history entered the replay corpus after its first judged success (confirm_samples=1); replay therefore provides a held-out estimate of whether that selected event persists rather than assuming that discovery implies deterministic success.

Among traces with all three source generations observed, harmful histories reproduced 0/3, 1/3, 2/3, and 3/3 times for 89 (21.3%), 85 (20.4%), 103 (24.7%), and 140 (33.6%) traces. The corresponding benign counts are 184 (30.4%), 102 (16.8%), 117 (19.3%), and 203 (33.5%). Thus, 78.7% of harmful histories and 69.6% of benign histories reproduce at least once, while approximately one-third reproduce in every generation.

Source modelHarmful reproductionBenign reproduction
Claude 5 Sonnet57.6% (11 traces)52.1% (126)
Muse Spark 1.054.5% (11)48.2% (76)
GPT-5.6 Terra33.3% (18)25.0% (12)
GPT-5.6 Sol46.0% (29)38.9% (6)
GPT-5.433.3% (29)45.6% (49)
Gemini 3.8 Flash54.7% (50)42.4% (22)
Gemini 3.5 Flash70.3% (120)50.0% (10)
Grok 4.656.8% (98)55.3% (172)
Grok 4.355.6% (51)57.0% (134)

The concentration at both 0/3 and 3/3 indicates substantial trace-level heterogeneity: some histories are robust, while others were selected from an unstable or borderline success.

We cannot fully decompose those two sources of variability in the current experiment, but our view is that we are probably seeing (a) residual variability from subsequent target model generation and judge calls and (b) mined histories result in borderline outputs where small changes in the generated answer are moving the judge across the success threshold. It's worth noting that the low-reproduction rate is not confined to one model family. We will add this discussion to the paper.

Q3: Accounting for the reported trace and generation counts

Detailed accounting will be added to the final paper.

  • Harmful mining produced 450 judged discoveries where 4 lacked a valid replay artifact, leaving 446 locked traces.
  • Benign mining produced 641 adjudicated discoveries where 14 lacked a valid replay artifact, leaving 627.
  • At replay time, 418 harmful source trace-prompt pairs remained eligible under the fresh source baseline. 3 of the resulting 1,254 requested generations were missing, giving 1,251 evaluable generations. For benign degradation, 607 source pairs were eligible and 1 of 1,821 generations was missing, giving 1,820.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.