- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Distributional Adversarial Training (DAT), which generates prompts conditioned on harmful responses using a diffusion LLM, filters them based on the target model’s responses, and applies continuous adversarial training. Experiments on three instruction-tuned LLMs show substantial robustness improvements across multiple attacks.
Critical strengths
1.The empirical robustness gains are substantial. DAT reduces worst-case ASR from 0.88 to 0.34 on Llama3 and from 0.98 to 0.18 on Qwen2.5 compared with CAT, and improvements persist after excluding Inpainting (Table 1).
2.The ablations provide useful evidence for the design choices. The diffusion-only and PAIR-DAT controls support the combination of diffusion-generated data and continuous adversarial training, while additional experiments examine prompt filtering and robustness across diffusion attack models (Table 1; Appendices C.2, C.5, and C.7).
Critical weaknesses
-
The sampling design is not well connected to the stated distributional objective. DAT retains the 12 highest-scoring prompts out of 1,000 candidates per behavior, creating a model-dependent training distribution that is not covered by Theorem 4.1. It remains unclear whether the gains reflect better distributional coverage or more effective attack selection (Section 4.6; Table 11).
-
The methodological novelty over the closest prior work appears limited. DAT follows the diffusion-based prompt generation procedure of Lüdke et al., while MixAT already combines generated adversarial prompts with continuous adversarial training (Section 2; Appendix C.1). The main methodological change appears to be the prompt generator within this existing training framework.
-
The broader robustness–utility advantage is not established at matched benign compliance. For example, DAT lowers Qwen2.5’s XSTest score from CAT’s 0.544 to 0.436 (Table 1). The Pareto analysis only covers Inpainting on Llama3, leaving unclear whether the multi-attack robustness advantage persists at comparable compliance across models (Figure 3).
Questions for authors
-
How does selecting the highest-scoring prompts improve the intended distributional approximation? Can the authors distinguish the benefit of broader coverage from that of selecting more effective attacks?
-
What is the substantive methodological contribution of DAT beyond combining the existing diffusion-inpainting generator with CAT? Please clarify the advance over MixAT beyond the choice of prompt generator.
-
Can the authors report WCA–XSTest trade-offs across all three models and compare DAT with CAT at matched XSTest compliance? This would clarify whether the broader robustness advantage persists when benign compliance is controlled.
2 comments
Dear Reviewer wz9u,
We would like to thank you once again for your thoughtful review and for recognizing the substantial robustness gains of DAT and the value of our ablations.
In response to your review, we provided the following clarifications and additional experiments, most of which are now included in the revised paper:
- Disentangled distributional coverage from attack selection: we pointed to our random-vs-top-12 filtering ablation and to PAIR-DAT, which uses the same top-12 selection but fails to generalize to other attack families (App. C.2, now on all 100 JailbreakBench behaviors).
- Added a new experiment showing that with 16 samples per behavior, random sampling without any target-model selection matches top-16 filtering (WCA 0.33 vs. 0.36 on Llama3-8B), in line with the unfiltered estimator of Theorem 4.1 (App. C.7).
- Clarified the contribution beyond MixAT in the general comment: DAT shapes the outer-loop training distribution rather than strengthening the inner-loop adversary, and a single offline diffusion source outperforms the multi-attack MixAT+GCG at roughly 1/8 of its training compute.
- Compared DAT with CAT at matched XSTest compliance on all three models, where DAT lowers WCA from 0.925 to 0.425 (Qwen2.5-14B), 0.900 to 0.350 (Llama3-8B), and 0.900 to 0.300 (Gemma3-12B) (App. C.8, Table 13).
- Added a WCA–XSTest trade-off plot for all three models (Fig. 4).
As we approach the end of the discussion period, we would appreciate hearing whether our rebuttal has addressed your concerns. We would be happy to discuss any remaining questions or provide further clarifications.
Dear Reviewer wz9u, Thank you for the thoughtful review and for recognizing the substantial robustness gains. We answer the novelty concerns (W2 / Q2) and matched-compliance comparison (W3 / Q3) in the general comment.
W1 / Q1: Coverage vs. selection of effective attacks.
In Section 6.5 we have ablated the filtering strategy between 12 highest-scoring and randomly sampled candidates with full table in Appendix C.7. The latter has a WCA of 0.60 on Llama3-8B, worse than top-12 but better than all other baselines.
Broader coverage is needed at same training effort. We have another ablation on the broader coverage in Appendix C.2 with an autoregressive attack generator PAIR-DAT. It applies the same top-12 selection to PAIR prompts under the same training recipe and budget. Due to the filtering, the selected prompts for PAIR-DAT and regular DAT also share equal attack effectiveness. PAIR-DAT is robust against PAIR but not GCG and Inpainting, similar to MixAT. Selecting effective attacks alone therefore does not produce the robustness of DAT.
Selection becomes unnecessary with more samples. Furthermore, we followed the suggestion of Reviewer hwjg and added an extra experiment containing a larger set of training data. We observe that random-16 reaches the same level of robustness as top-16 at the same XSTest level:
Filtering therefore compensates for a small sample, in which a random sample covers fewer of the rare prompts that elicit harmful responses. Theorem 4.1 describes exactly this unfiltered estimator. At 16 samples per behavior, random sampling follows the theorem and removes the dependence of the training set on the target model. We plan to run the same comparison on Qwen2.5-14B and Gemma3-12B and further investigate the role of filtering.
Action: We add the 16-sample comparison to Appendix C.7 and revise Section 6.5 to state that filtering matters only at small sample sizes.
W3 / Q3: WCA–XSTest trade-off at matched compliance.
We trained additional DAT models to match the XSTest compliance of CAT on each model (details in the general comment, Tables R1a–c). On the first 40 JailbreakBench behaviors:
WCA is taken over the three attacks. At matched or higher XSTest compliance, DAT lowers WCA from 0.925 to 0.425 on Qwen2.5-14B, from 0.900 to 0.350 on Llama3-8B, and from 0.900 to 0.300 on Gemma3-12B. The robustness advantage therefore persists when benign compliance is controlled.
Action: We add these matched-compliance comparisons for all three models to the revision.
Thank you again for the constructive feedback. We have added the random-16 versus top-16 comparison, matched-compliance results on Qwen2.5-14B and Llama3-8B, and a clarification of the contribution over MixAT. We hope these revisions address your concerns and are happy to provide further clarification if needed.