- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper combines self-play with mosaic attack and defense, using theoretical analysis from the perspective of a finite extensive-form game with imperfect information. It describes the mosaic attacks as multi-turn sequences whose individual fragments are innocuous in isolation yet assemble into a harmful payload. To defend against such attacks, it formalizes a watchman, an online state mechanism that carries safety-relevant information forward. Experiments using GRPO and LoRA show that this method strengthens both roles, helping attackers to elicit more harmful responses and defenders to become more robust across harm categories, including on unseen objectives.
Critical strengths
- Interesting theoretical analysis from the perspective of a finite extensive-form game.
- Transferring self-play to a mosaic attack, concerning the situation of innocuous individual fragments assembling into a harmful payload. This kind of attack can cause a lot of harm in reality.
Critical weaknesses
- The work uses LLM-as-a-Judge but lacks human evaluation, so the trustworthiness is not sure. The prompt used by the judge model is also not clear.
- The WildJailbreak decomposition study shows that 38.8% pass the equivalence test; how about on WildMosaic? How to guarantee that the decomposed requests still carry the harmful payload?
Questions for authors
- The watchman is defined as a discrete finite-state machine, but implemented by using LoRA adapter. How is this fundamental representational gap addressed?
- How to guarantee that the decomposed requests still carry the harmful payload?
- What’s the reliability of the judge? Why prefer to use the same backbone model as the judge (self-judging)?
3 comments
Thank you for the additional information and clarifications. I appreciate the effort to address the reviewers' concerns. However, I still have the following reservations.
- The theory and experiments address different questions: the theory is an existence result about an exactly safety-separating watchman, while the experiments train LoRA adapters that, as the authors note, do not implement it. It would be helpful for readers to see this distinction stated explicitly as a scope limitation.
- Judge reliability is a boundary on the empirical claim. Given the reported disagreement (17.1% vs. 25.0% on Qwen; 65.0% vs. 41.8% on Gemma) and the authors' own caveat that significance cannot be assumed unchanged under the alternative judge, the robustness of the defender gains across judges remains open.
- The equivalence/preservation rates (99%/98%) are computed on already-successful cases, so they do not estimate strict-mosaic validity over all attempts. Since the defender's gains are evaluated on these attacks, the extent to which they reflect mosaic detection rather than rejection of near-direct harmful requests remains unverified.
We appreciate your follow-up and the opportunity to clarify these points. We have conducted additional analyses addressing judge reliability and mosaic validity, and clarify the relationship between our theoretical and empirical contributions.
1. Relationship between theory and experiments.
We agree that the theoretical watchman and the trained LoRA defenders should not be conflated. However, the theory goes beyond establishing the existence of an exactly safety-separating watchman. It characterizes the information needed to defend against sequential mosaic attacks, proves limitations of bounded-history defenders, and establishes state-complexity lower bounds. Our new results in Appendices D.4 and D.5 extend the analysis to weakened assumptions and approximate safety separation.
The experiments address the corresponding behavioral challenge: whether attacker-defender self-play can improve resistance to multi-turn attacks. The defender receives the conversation history and can use information from earlier turns, but we do not claim that its LoRA adapter implements the theoretical watchman or satisfies its formal guarantees. Rather, the theory characterizes the information requirements and limitations of safe defense, while the experiments evaluate whether self-play improves defender behavior against these attacks. We will make this explicit in Section 7.
2. Judge reliability and the source of cross-judge disagreement.
We agree that judge choice affects the reported attack-success rates. To investigate, we performed an additional cross-judge analysis of the vanilla base-attacker/base-defender condition, without either LoRA adapter, on 1,040 conversations per model (520 malicious and 520 benign).
Even without training, cross-judge agreement is 89.4% (Cohen's ) on Qwen conversations and 74.5% () on Gemma conversations, close to the agreement rates of 88.0% and 77.3% reported across the trained evaluation conditions. The differences in attack-success judgments are also present before training: the two judges report 20.8% versus 29.2% success on vanilla Qwen conversations, and 40.4% versus 70.0% on vanilla Gemma conversations. Same-judge reruns yield 93.8% and 91.9% agreement, respectively, while cross-judge agreement on benign over-refusal is 98.7-99.8%.
These results show that the substantial disagreement in attack-success judgments is already present in the untrained models and before LoRA. They also distinguish disagreement about attack success from the higher agreement on benign over-refusal. We will add this to Appendix I.2.
As a final remark, while this does not establish that the magnitude or statistical significance of the defender's improvement is unchanged under another judge, it provides context for interpreting the disagreement: the judges differ substantially even when evaluating identical conversations generated without self-play training.
3. Mosaic validity beyond successful attacks.
Yes, the previously reported 99%/98% objective-preservation rates were measured on attacks already judged successful. We have extended the analysis to unsuccessful attempts, allowing us to examine how the defining properties of mosaic attacks differ between successful and unsuccessful conversations.
Across 1,169 unsuccessful attempts, 72% still preserved the original harmful objective. Thus, objective preservation is common even when the attack fails, and preserving the payload alone does not explain success.
The differences are more pronounced for mosaic structure. In an annotated sample of 137 successful and 90 unsuccessful attacks, 93% of successes satisfied the benign-in-isolation criterion, compared with 70% of failures. More strikingly, 87% of successes exhibited the full mosaic structure, compared with only 36% of failures.
Among attacks explicitly refused by the defender, 60% satisfied benign-in-isolation and only 31% retained the full mosaic structure. In other words, 40% contained at least one turn judged not benign in isolation, and 69% did not exhibit the full mosaic structure.
These findings provide evidence that payload preservation and mosaic structure are distinct properties. The harmful objective is frequently preserved even in failed attacks, whereas successful attacks are substantially more likely to maintain individually benign-looking fragments and the full mosaic structure. The results are consistent with the defender refusing more often when the decomposition breaks down and harmful intent becomes apparent in individual turns.
Although this comparison does not isolate a mosaic-specific detection mechanism, it addresses the concern that our earlier validity analysis considered only successful attacks. We will incorporate these results into Appendix I.3.
We hope these new analyses clarify the scope of the theoretical contribution and provide a complete picture of judge disagreement and the mosaic structure.
Reviewer UN89 — Comment 1/1
We appreciate your questions about the relationship between the theoretical watchman and the trained models, as well as the reliability of our evaluation. We have revised the paper to clarify both points (the PDF, with changes in blue).
We now address all the reviewers' concerns. We identify them by the number each appears in the review (e.g., W1/Q1 means Weakness, point 1, and Question 1).
Q1: Is the LoRA defender a watchman?
The watchman is a theoretical online state that records information relevant to safety decisions across turns. In the experiments, we train separate attacker and defender LoRA adapters on a frozen backbone. The defender receives the conversation history and can therefore use earlier turns when responding, but we do not train a separate finite-state watchman. The experiments test whether self-play improves the defender's behavior against multi-turn attacks, rather than whether the LoRA implements the theoretical state machine. Section 7 now explains this relationship and the distinction between the theory's information requirements and the observed behavioral improvements.
W2/Q2: Objective preservation and fragment benignness
The 38.8% equivalence acceptance rate reported in Section 7 comes from a separate WildJailbreak decomposition study. We also conducted a curation experiment using fixed decompositions of 262 WildMosaic seeds. For the frozen Qwen model, judged success against the original request increased from 3/262 (1.1%) with direct prompts to 102/262 (38.9%) with the decomposed sequences.
These results provide evidence that decomposing a harmful request can preserve enough of its original objective to elicit a successful response. To examine the attacks generated during self-play, we also assessed whether individual fragments were benign in isolation, whether the sequence preserved the original harmful objective, and whether the defender's final response achieved that objective.
We used Claude Opus 5 as an independent model-based annotator to assess selected successful trained-Qwen attacks (Appendix I.3). On 137 AdvBench successes, 93% have turns judged non-refusable in isolation, 99% preserve the seed objective, and 92% meet both conditions. Among 249 successful WildJailbreak attacks, 72% have turns judged non-refusable in isolation, 98% preserve the seed objective, and 71% meet both conditions. The annotator judged the harmful content to be distributed across turns in 91% of the AdvBench successes and 75% of the WildJailbreak successes (Appendix I.3). The new annotation provides direct evidence about the selected successful attacks. Because the sample was selected for prior judged success, and the benign-in-isolation criterion allows borderline or dual-use turns, it does not estimate strict-mosaic validity among all generated attempts. Defender gains on a strictly valid subset require a separate paired comparison.
W1/Q3: Judge configuration and reliability
Appendix I.2 now includes the judge prompt and configuration. For Qwen, the judge is a separately served copy of the frozen Qwen3.6-35B-A3B weights, using temperature zero, reasoning enabled, a 3,072-token budget, and JSON-constrained output. The same judge prompt is used during training and evaluation.
Gemma uses a judge from its own backbone family, while DeepSeek is evaluated using Qwen.
To assess the consequences of judge choice, we also added a cross-family comparison on fixed AdvBench transcripts at checkpoint 65. Gemma rejudged 4,160 Qwen conversations and Qwen rejudged 4,160 Gemma conversations using the same prompt. Agreement was 88.0% (κ=0.81) on Qwen conversations and 77.3% (κ=0.65) on Gemma conversations; temperature-zero self-agreement was 93.8% and 91.9%, respectively.
Disagreement matters particularly for attack success: the two judges label 17.1% versus 25.0% of malicious Qwen conversations successful, and 65.0% versus 41.8% of malicious Gemma conversations. Benign-refusal decisions agree on 97.2% and 95.8% of episodes, although κ is 0.61 and 0.35 in the two directions. This is evidence about consistency, not accuracy; absolute ASR depends on the judge, and paired significance cannot be assumed unchanged without recomputation under the alternative judge. We have not conducted human evaluation.
Ethics
We train and evaluate attackers because a defender trained against a fixed opponent may overfit to that opponent. The observed attacker gains are modest: for Qwen, they are significant at one of six AdvBench checkpoints and none of the three WildJailbreak checkpoints. We include only abbreviated attack excerpts, and the system was not deployed to real users. The revised ethics statement also discusses the dual-use implications of training stronger attackers.