REACT: Role-aware Exchange for Attacker–defender Co-evolution Training

As adversarial attacks evolve rapidly, online attacker–defender co-evolution has emerged as a promising paradigm for LLM safety alignment. However, existing frameworks typically adopt an adversarial framing, relying on an external judge model to reduce rich interaction trajectories into guided rewards. This compression discards fine-grained contextual signals and creates a strong dependency on the judge's capability. To address this limitation, we propose Role-aware Exchange for Attacker–defender Co-evolution Training(REACT), a framework that preserves adversarial objectives at deployment and complements the co-evolution by enabling a learning-level cooperation during training: the two roles exchange their internal interaction artifacts—the attacker's hidden transformation rationale and the defender's execution feedback—as dense, reciprocal supervision. REACT consists of two complementary mechanisms: (1) Rationale-Grounded Safety Distillation(RGSD), which converts the attacker's hidden transformation intent into response-level supervision for the defender, leveraging multi-view distillation and safety-token weighting to enhance intent sensitivity without causing over-refusal; and (2) Defender-Feedback Interaction Distillation (DFID), which extracts a query-specific interaction residual from the defender's response via a difference-in-differences intervention, guiding the attacker's tactics without altering core intent. Extensive evaluations on standard adversarial benchmarks and benign compliance tasks show that REACT consistently outperforms competitive baselines. Specifically, REACT reduces the strict unsafe rate down to 0.5% on HarmBench while boosting benign compliance on OR-Bench up to 29.9%. Cross-play evaluations further confirm that REACT fosters a genuine co-evolutionary arms race, yielding both a stronger red-teaming adversary and a more resilient defender.