- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes REACT, an online attacker–defender co-evolution framework that replaces judge-mediated scalar training signals with dense, role-specific supervision extracted from the interaction itself. REACT contains two components, RGSD and DFID. Experiments use Qwen3.5 models at 2B, 4B, and 9B scales and evaluate harmful refusal and benign compliance across HarmBench, StrongREJECT, XSTest, WildJailbreak, OR-Bench, and SQuAD. REACT reports strong results at 2B and especially 9B, including a 0.5% strict unsafe rate on HarmBench for the 9B model.
Critical strengths
- A novel reframing of attacker–defender co-evolution: REACT treats the attacker rationale and defender response as complementary dense supervision rather than compressing the interaction into an external judge score. This is a meaningful direction because it may provide richer token-level credit while reducing dependence on a separate judge during training.
- Thoughtful task-specific mechanisms: DFID’s query-masked interaction contrast and transformation-token mask directly target plausible shortcuts in feedback-conditioned attacker training.
- Meaningful evaluation: The evaluation spans three model scales and multiple harmful and benign benchmarks, and cross-play is more informative than evaluating only the final paired checkpoint.
Critical weaknesses
-
The paper lacks the component-level ablations required to attribute its gains.
The main results evaluate only the complete REACT system, without comparing no exchange, RGSD-only, DFID-only, and full REACT under a matched training budget. Therefore, the improvements in defender safety, benign compliance, and attacker strength cannot be attributed to either proposed component.
-
For RGSD, the paper does not ablate the Teacher, Self, and None views. Consequently, it remains unclear whether the multi-view construction actually mitigates privilege illusion or whether direct privileged-response distillation would perform similarly. Besides, the safety-token weighting experiment provides some direct evidence, but its effect is small and validates only one subcomponent of RGSD.
-
For DFID, the paper does not compare direct feedback conditioning, the factual first difference $ \ell^{QA}-\ell^{Q0}, $ and the full interaction residual $ (\ell^{QA}-\ell^{Q0})-(\ell^{\bar QA}-\ell^{\bar Q0}). $ Thus, there is no direct evidence that the second difference removes generic response bias. The transformation-mask diagnostics show that copied positions receive zero direct residual weight and that copied tokens otherwise have larger residual magnitudes, but they do not compare the final attack success or intent-preservation performance with and without the mask.
Overall, the paper demonstrates that the complete REACT system performs well and that several mechanisms operate as implemented, but it does not demonstrate that the specific effects claimed for RGSD and DFID are responsible for the reported gains. This missing causal attribution substantially weakens the paper's central methodological claims.
-
-
The strongest empirical and novelty claims need stronger validation.
- RGSD closely follows DAPD's Teacher/Cross, Self, and None-view structure [1]. Moreover, RGSD reverses the direction of DAPD's inference anchor: DAPD trains Self toward detached None, whereas RGSD trains None toward detached Self. This modification may be reasonable, but the paper provides neither a theoretical justification nor a direct comparison.
- Experimentally, the reported 9B HarmBench improvement from 20.1% to 0.5% is unusually large, while the 4B gains are much smaller. Analysis on this discrepancy is needed.
- The high-level idea of bidirectional attacker--defender co-evolution is not itself novel. The more specific distinction of REACT is that it converts role-specific interaction artifacts. However, even this contribution is largely compositional. RGSD substantially builds on DAPD-style privileged multi-view distillation.
[1] DAPD: Dual-Anchored Policy Distillation
Questions for authors
-
Can the authors provide a module-level comparison of no exchange, RGSD-only, DFID-only, and full REACT under the same rollout and optimization budget? This would materially affect my assessment of the main contribution.
-
For DFID, can the authors compare direct feedback conditioning, the factual first difference, and the full DiD residual? A shuffled or mismatched-response control would also test whether DFID extracts query-specific feedback rather than generic refusal style.
-
Figure 4 shows that, from R1 to R3-R5, the fraction of material transformations decreases from approximately 64% to 20-30%, while exact-copy proposals increase from approximately 36% to around 80%. Can this suggest a self-reinforcing degeneration of the attacker?