- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies defenses against compromised agents in LLM multi-agent debate. It views existing defenses as projections from transcripts to detection scores, which can make benign and malicious messages overlap. Based on this view, the authors train an RL attacker that uses the target defense's detection score as reward, and greedily combine defenses with complementary blind spots. Experiments on MMLU, GSM8K, and BBH with three models show that the attack lowers group accuracy under four existing defenses, and that the selected compositions improve accuracy over single defenses under fixed attack policies.
Critical strengths
-
The paper applies adaptive attacks to four defenses that rely on different information sources, using a single unified reward interface. This makes it a general stress-testing tool for MAS defenses. It also offers a useful insight: existing defenses evaluated only against fixed attacks may overestimate their robustness.
-
Beyond accuracy or ASR, the paper reports flag rates for both attackers and honest agents, as well as the message-withholding cost in the clean setting. This makes its conclusions more solid.
Critical weaknesses
-
The attack's effectiveness cannot be attributed to the RL-trained LoRA. The paper claims that no hand-designed attack strategy is needed, yet the attacker's prompt already explicitly instructs the model to mirror its peers' style "so that a monitor sees you as one of them." The paper lacks several key ablations: a prompt-only attacker without the LoRA, an attacker trained without the evasion reward, and comparisons with existing attacks.
-
The defense compositions are evaluated only against fixed attackers. Each attacker is trained against a single defense and then used to attack the selected composition, yet in Table 2, 3 of the 4 selected compositions do not even include the defense the attacker was trained against. The gains may therefore simply come from the attacker never having seen the new detectors, rather than showing that the composition actually covers each defense's blind spots.
Questions for authors
-
The Single column in Table 2 matches the attacked numbers in Table 1, whereas Selected and All four use the 128 questions × 2 seeds described in Appendix D. Are these columns evaluated on the same set of questions? If not, what are the results for each on the same test set?
-
The attacked result for no-defense is not provided in Table 1. What is the group accuracy under attack when no defense is applied (e.g., using an attacker trained without the evasion reward)?
-
The paper unifies all defenses into a protocol that hides flagged messages in the current round but retains their final votes. Under this protocol, the votes of the 2 attackers are always counted. How much of the accuracy drop in Table 1 comes from the attack evading detection, and how much from the protocol itself weakening the defenses? Does the attack remain effective under each defense's original intervention?
I would be happy to reconsider my rating if the authors can address these concerns.
4 comments
Response to Reviewer DvBF (2/3):
Q1. Same questions for all configurations. No, they were not, and thank you for checking this. The Single entries in the submitted Table 2 were taken directly from the 100-question Table 1 experiment, so they were not on the same questions or under the same intervention as Selected and All four. We have rerun every configuration, including all four individual defenses, under one protocol: the same 128 questions, the same two seeds and the same intervention. The selector now chooses by group accuracy on an independent validation set and may return a pair, a single defense or none. Each cell gives group accuracy / targeted ASR (%); columns name the attacker's training-target defense, and † marks that defense deployed alone.
| Defense | Confidence attacker | BlindGuard attacker | G-Safeguard attacker | GUARDIAN attacker |
|---|---|---|---|---|
| None | 48.4 / 25.8 | 43.0 / 32.0 | 52.7 / 21.5 | 51.2 / 25.4 |
| Confidence | 46.9 / 30.9 † | 48.4 / 32.8 | 51.2 / 25.0 | 43.0 / 27.0 |
| BlindGuard | 46.9 / 28.1 | 46.5 / 28.5 † | 48.0 / 28.1 | 43.0 / 33.2 |
| G-Safeguard | 47.3 / 32.0 | 48.4 / 26.6 | 50.0 / 28.5 † | 50.0 / 25.8 |
| GUARDIAN | 52.0 / 28.9 | 49.2 / 29.3 | 49.6 / 27.0 | 52.0 / 28.5 † |
| All four | 45.7 / 31.2 | 46.1 / 29.7 | 43.4 / 28.5 | 47.3 / 28.9 |
| Selected | 53.5 / 27.3 | 48.4 / 32.8 | 50.0 / 28.5 | 51.2 / 25.4 |
| (Selected defense) | GUARDIAN+G-Safeguard | Confidence | G-Safeguard | none |
Selected is above All four in every column, by 7.8, 2.3, 6.6 and 3.9 points. We will replace Table 2 and its text with this rerun.
Q2. Accuracy under attack without a defense. Thank you for this suggestion. The attacks remain damaging with every detector disabled: on the same 128 questions and two seeds, clean accuracy is 69.1% (177/256), and the prompt-only attacker lowers it to 53.5%. The four trained attackers give:
| Attacker's training target | Without evasion reward: accuracy (%) | Full attacker: accuracy (%) |
|---|---|---|
| Confidence | 44.9 | 48.4 |
| BlindGuard | 50.0 | 43.0 |
| G-Safeguard | 49.2 | 52.7 |
| GUARDIAN | 48.4 | 51.2 |
The damage therefore does not depend on detector failures, and it holds for attackers trained without the evasion reward, as you suggested.
Response to Reviewer DvBF (3/3)
Q3. Withholding protocol and retained votes. Thank you for this question; it also led us to correct the protocol description. In Table 1, only Confidence withheld messages, while BlindGuard, G-Safeguard and GUARDIAN removed flagged agents together with their votes. Flagged attackers therefore did not keep their votes under the three graph safeguards, and accuracy still fell sharply, for example from 64.0% to 23.0% under BlindGuard on MMLU with Llama-3.1-8B. The composition study and all new runs use withholding with all votes kept. We will correct the description.
Withholding does not weaken the defenses. On the same questions and frozen attackers, agent removal is worse for GUARDIAN in all eight comparisons, each statistically significant:
| Attacker | Accuracy (%), withhold vs remove | Targeted ASR (%), withhold vs remove |
|---|---|---|
| Confidence | 52.0 vs 36.7 | 28.9 vs 46.5 |
| BlindGuard | 49.2 vs 37.5 | 29.3 vs 43.4 |
| G-Safeguard | 49.6 vs 40.6 | 27.0 vs 39.5 |
| GUARDIAN | 52.0 vs 43.4 | 28.5 vs 40.6 |
Only 38% to 42% of removed agents are attackers, close to their 40% share of the group, and in 25% to 33% of debates both removed agents are honest. For BlindGuard, removal also raises targeted ASR, by 14.1 to 36.3 points across the four attackers, each with a 95% CI excluding zero; against its own attacker, accuracy falls from 46.5% under withholding to 23.4% under removal.
Communication also carries a substantial part of the damage even when compromised votes are kept: an oracle that withholds exactly the attackers' messages raises accuracy by 8.3 points (95% CI 5.2 to 11.7) and lowers targeted ASR by 12.2 points (95% CI 8.6 to 15.9), on 128 held-out questions with two seeds, pooled over the four attackers. The attacks therefore remain effective under both interventions, and their damage is not merely an effect of retained votes.
Thank you again for the constructive review. We hope these controls and additions address your concerns, and we would be glad to clarify anything further.
As the discussion period comes to an end, we would like to express our sincere gratitude for your thoughtful review and suggestions. Your questions on existing attacks, on evaluating every defense configuration on the same questions, and on the withholding protocol led to a rerun of all configurations under one protocol and a direct comparison of withholding with agent removal, and they helped us correct the protocol description. We have uploaded a revised version in which the changes made in response to your review are marked in blue and labeled with your reviewer ID.
We truly hope that our answers have addressed your concerns. That being said, if there are any other questions, please feel free to leave us a message. We would be keen to address them even after the discussion period ends.
We greatly appreciate your time and effort, and thank you again for your valuable feedback.
Response to Reviewer DvBF (1/3)
We sincerely thank Reviewer DvBF for the constructive review, and for recognizing the unified attack framework as a general stress-testing tool for multi-agent defenses and the value of reporting both attacker and honest flag rates. Our central contribution is the attack side: one RL framework that adapts to four defenses built on different information and consistently degrades them. Composition is a first use of the blind spots it exposes, not a final defense. We address each point below and will add all of it to the revision.
All new runs use MMLU with Llama-3.1-8B: 128 questions and two seeds, giving 256 debates per cell, with five agents including two compromised, three rounds, round-local withholding with all final votes counted, and targets fixed in advance. Confidence intervals (CIs) are 95% question-paired bootstrap intervals.
W1. What the RL-trained LoRA adds. Thank you for pointing this out; we will make the wording precise. The peer-mirroring prompt is a generic strategy shared by all four attackers, and what is not hand-designed is any defense-specific strategy. The three controls you asked for show that training adds substantial strength beyond this prompt:
The learned attacker gives the lowest accuracy and the highest ASR under the three embedding-level defenses, BlindGuard, G-Safeguard and GUARDIAN, which Benign Wrapper was designed to evade. As in Table 1, the learned row chooses its target after the honest first round.
We will add these controls and state that training drives the damage, while detector feedback adds measured evasion under BlindGuard.
W2. Fixed attackers and complementarity. Thank you for this point. We agree that the composition claims hold only under fixed attack policies, as stated in the paper and in Appendix F, and that swapping in a detector the attacker has not seen can account for part of the gain. What the rerun in Q1 supports is that selecting deployments beats stacking: Selected is above All four in every column, by 2.3 to 7.8 points, while All four withholds 59.2% of honest messages even in clean debates (Appendix E). Against the Confidence attacker, the selected pair reaches 53.5%, above every individual defense.
The rerun also points to a broader finding. Under every attacker, every configuration, including all four defenses combined, stays at least 15.6 points below the 69.1% clean accuracy: an attacker trained against one defense also damages the other three. This suggests that current defenses share blind spots, which limits how much any selection among them can recover (Appendix F). The more promising direction, which our attacker directly enables, is to use it as a source of adaptive attacks for building defenses that close these shared blind spots. We will make this framing explicit in the revision.