acceptodds
Under review as a conference paper at ICLR 2027

Defense as Projection: Exposing and Patching Blind Spots in Multi-Agent LLM Systems

Abstract

Large Language Model (LLM)-based multi-agent systems have demonstrated strong capabilities in solving complex tasks through agent communication and collaboration. However, compromised agents can exploit these interactions to influence other agents, undermining the reliability of multi-agent systems. Existing defenses leverage diverse information sources and detection algorithms to identify malicious behaviors, yet share a common structure: each maps its input information to a detection signal for making a decision. Nonetheless, their design objectives do not explicitly constrain the overlap between benign and malicious distributions in the detection space. To understand this overlap, we introduce Defense as Projection, a unified view that analyzes how information representations and detection algorithms jointly shape benign–malicious separability in the resulting detection space. Building on this view, we develop a unified adaptive attack trained through reinforcement learning, using each defense's detection signal to seek messages that steer the group toward a targeted wrong answer while evading detection. Because an attack missed by one defense may still be detected by another, we use the exposed blind spots to select complementary defense combinations. Extensive experiments show that our attacks consistently degrade group performance across existing defenses, while selected defense combinations can improve group accuracy over original defense under fixed attack policies.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-Agent LLM Systems, AI Security, Adaptive Attacks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DvBFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies defenses against compromised agents in LLM multi-agent debate. It views existing defenses as projections from transcripts to detection scores, which can make benign and malicious messages overlap. Based on this view, the authors train an RL attacker that uses the target defense's detection score as reward, and greedily combine defenses with complementary blind spots. Experiments on MMLU, GSM8K, and BBH with three models show that the attack lowers group accuracy under four existing defenses, and that the selected compositions improve accuracy over single defenses under fixed attack policies.

Critical strengths
  • The paper applies adaptive attacks to four defenses that rely on different information sources, using a single unified reward interface. This makes it a general stress-testing tool for MAS defenses. It also offers a useful insight: existing defenses evaluated only against fixed attacks may overestimate their robustness.

  • Beyond accuracy or ASR, the paper reports flag rates for both attackers and honest agents, as well as the message-withholding cost in the clean setting. This makes its conclusions more solid.

Critical weaknesses
  • The attack's effectiveness cannot be attributed to the RL-trained LoRA. The paper claims that no hand-designed attack strategy is needed, yet the attacker's prompt already explicitly instructs the model to mirror its peers' style "so that a monitor sees you as one of them." The paper lacks several key ablations: a prompt-only attacker without the LoRA, an attacker trained without the evasion reward, and comparisons with existing attacks.

  • The defense compositions are evaluated only against fixed attackers. Each attacker is trained against a single defense and then used to attack the selected composition, yet in Table 2, 3 of the 4 selected compositions do not even include the defense the attacker was trained against. The gains may therefore simply come from the attacker never having seen the new detectors, rather than showing that the composition actually covers each defense's blind spots.

Questions for authors
  • The Single column in Table 2 matches the attacked numbers in Table 1, whereas Selected and All four use the 128 questions × 2 seeds described in Appendix D. Are these columns evaluated on the same set of questions? If not, what are the results for each on the same test set?

  • The attacked result for no-defense is not provided in Table 1. What is the group accuracy under attack when no defense is applied (e.g., using an attacker trained without the evasion reward)?

  • The paper unifies all defenses into a protocol that hides flagged messages in the current round but retains their final votes. Under this protocol, the votes of the 2 attackers are always counted. How much of the accuracy drop in Table 1 comes from the attack evading detection, and how much from the protocol itself weakening the defenses? Does the attack remain effective under each defense's original intervention?

I would be happy to reconsider my rating if the authors can address these concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DvBF (1/3)

We sincerely thank Reviewer DvBF for the constructive review, and for recognizing the unified attack framework as a general stress-testing tool for multi-agent defenses and the value of reporting both attacker and honest flag rates. Our central contribution is the attack side: one RL framework that adapts to four defenses built on different information and consistently degrades them. Composition is a first use of the blind spots it exposes, not a final defense. We address each point below and will add all of it to the revision.

All new runs use MMLU with Llama-3.1-8B: 128 questions and two seeds, giving 256 debates per cell, with five agents including two compromised, three rounds, round-local withholding with all final votes counted, and targets fixed in advance. Confidence intervals (CIs) are 95% question-paired bootstrap intervals.

W1. What the RL-trained LoRA adds. Thank you for pointing this out; we will make the wording precise. The peer-mirroring prompt is a generic strategy shared by all four attackers, and what is not hand-designed is any defense-specific strategy. The three controls you asked for show that training adds substantial strength beyond this prompt:

  • Prompt only, without the LoRA. With the prompt held fixed, adding the adapter raises targeted ASR in all 24 comparisons across six deployed configurations and four attackers, by 5.9 to 16.4 points, each with a 95% CI excluding zero, and lowers group accuracy in all 24.
  • Without the evasion reward. This attacker still exceeds the prompt-only attacker in all eight ASR comparisons, by 7.4 to 18.8 points, so the damage comes from RL training itself. The evasion reward adds evasion where the defense actually catches attackers: under BlindGuard, over three training seeds per arm on 128 held-out questions, it significantly lowers attacker withholding, by 7.2 points. Details are in our reply to Reviewer kLVf.
  • Existing attacks. On the 100-question MMLU / Llama-3.1-8B protocol of Table 1, each cell gives group accuracy / targeted ASR (%). Naive and Blunt are scripted controls, and Benign Wrapper is an existing attack designed to evade embedding-based defenses (Zhang et al., 2026).
AttackConfidenceBlindGuardG-SafeguardGUARDIAN
Naive63 / 1362 / 1758 / 1939 / 41
Blunt56 / 2459 / 1754 / 2249 / 36
Benign Wrapper37 / 4640 / 4349 / 3335 / 54
Learned (ours)52 / 3423 / 7346 / 4133 / 58

The learned attacker gives the lowest accuracy and the highest ASR under the three embedding-level defenses, BlindGuard, G-Safeguard and GUARDIAN, which Benign Wrapper was designed to evade. As in Table 1, the learned row chooses its target after the honest first round.

We will add these controls and state that training drives the damage, while detector feedback adds measured evasion under BlindGuard.

W2. Fixed attackers and complementarity. Thank you for this point. We agree that the composition claims hold only under fixed attack policies, as stated in the paper and in Appendix F, and that swapping in a detector the attacker has not seen can account for part of the gain. What the rerun in Q1 supports is that selecting deployments beats stacking: Selected is above All four in every column, by 2.3 to 7.8 points, while All four withholds 59.2% of honest messages even in clean debates (Appendix E). Against the Confidence attacker, the selected pair reaches 53.5%, above every individual defense.

The rerun also points to a broader finding. Under every attacker, every configuration, including all four defenses combined, stays at least 15.6 points below the 69.1% clean accuracy: an attacker trained against one defense also damages the other three. This suggests that current defenses share blind spots, which limits how much any selection among them can recover (Appendix F). The more promising direction, which our attacker directly enables, is to use it as a source of adaptive attacks for building defenses that close these shared blind spots. We will make this framing explicit in the revision.

AuthorsOpenReview ↗

Response to Reviewer DvBF (2/3):

Q1. Same questions for all configurations. No, they were not, and thank you for checking this. The Single entries in the submitted Table 2 were taken directly from the 100-question Table 1 experiment, so they were not on the same questions or under the same intervention as Selected and All four. We have rerun every configuration, including all four individual defenses, under one protocol: the same 128 questions, the same two seeds and the same intervention. The selector now chooses by group accuracy on an independent validation set and may return a pair, a single defense or none. Each cell gives group accuracy / targeted ASR (%); columns name the attacker's training-target defense, and † marks that defense deployed alone.

DefenseConfidence attackerBlindGuard attackerG-Safeguard attackerGUARDIAN attacker
None48.4 / 25.843.0 / 32.052.7 / 21.551.2 / 25.4
Confidence46.9 / 30.9 †48.4 / 32.851.2 / 25.043.0 / 27.0
BlindGuard46.9 / 28.146.5 / 28.5 †48.0 / 28.143.0 / 33.2
G-Safeguard47.3 / 32.048.4 / 26.650.0 / 28.5 †50.0 / 25.8
GUARDIAN52.0 / 28.949.2 / 29.349.6 / 27.052.0 / 28.5 †
All four45.7 / 31.246.1 / 29.743.4 / 28.547.3 / 28.9
Selected53.5 / 27.348.4 / 32.850.0 / 28.551.2 / 25.4
(Selected defense)GUARDIAN+G-SafeguardConfidenceG-Safeguardnone

Selected is above All four in every column, by 7.8, 2.3, 6.6 and 3.9 points. We will replace Table 2 and its text with this rerun.

Q2. Accuracy under attack without a defense. Thank you for this suggestion. The attacks remain damaging with every detector disabled: on the same 128 questions and two seeds, clean accuracy is 69.1% (177/256), and the prompt-only attacker lowers it to 53.5%. The four trained attackers give:

Attacker's training targetWithout evasion reward: accuracy (%)Full attacker: accuracy (%)
Confidence44.948.4
BlindGuard50.043.0
G-Safeguard49.252.7
GUARDIAN48.451.2

The damage therefore does not depend on detector failures, and it holds for attackers trained without the evasion reward, as you suggested.

AuthorsOpenReview ↗

Response to Reviewer DvBF (3/3)

Q3. Withholding protocol and retained votes. Thank you for this question; it also led us to correct the protocol description. In Table 1, only Confidence withheld messages, while BlindGuard, G-Safeguard and GUARDIAN removed flagged agents together with their votes. Flagged attackers therefore did not keep their votes under the three graph safeguards, and accuracy still fell sharply, for example from 64.0% to 23.0% under BlindGuard on MMLU with Llama-3.1-8B. The composition study and all new runs use withholding with all votes kept. We will correct the description.

Withholding does not weaken the defenses. On the same questions and frozen attackers, agent removal is worse for GUARDIAN in all eight comparisons, each statistically significant:

AttackerAccuracy (%), withhold vs removeTargeted ASR (%), withhold vs remove
Confidence52.0 vs 36.728.9 vs 46.5
BlindGuard49.2 vs 37.529.3 vs 43.4
G-Safeguard49.6 vs 40.627.0 vs 39.5
GUARDIAN52.0 vs 43.428.5 vs 40.6

Only 38% to 42% of removed agents are attackers, close to their 40% share of the group, and in 25% to 33% of debates both removed agents are honest. For BlindGuard, removal also raises targeted ASR, by 14.1 to 36.3 points across the four attackers, each with a 95% CI excluding zero; against its own attacker, accuracy falls from 46.5% under withholding to 23.4% under removal.

Communication also carries a substantial part of the damage even when compromised votes are kept: an oracle that withholds exactly the attackers' messages raises accuracy by 8.3 points (95% CI 5.2 to 11.7) and lowers targeted ASR by 12.2 points (95% CI 8.6 to 15.9), on 128 held-out questions with two seeds, pooled over the four attackers. The attacks therefore remain effective under both interventions, and their damage is not merely an effect of retained votes.

Thank you again for the constructive review. We hope these controls and additions address your concerns, and we would be glad to clarify anything further.

As the discussion period comes to an end, we would like to express our sincere gratitude for your thoughtful review and suggestions. Your questions on existing attacks, on evaluating every defense configuration on the same questions, and on the withholding protocol led to a rerun of all configurations under one protocol and a direct comparison of withholding with agent removal, and they helped us correct the protocol description. We have uploaded a revised version in which the changes made in response to your review are marked in blue and labeled with your reviewer ID.

We truly hope that our answers have addressed your concerns. That being said, if there are any other questions, please feel free to leave us a message. We would be keen to address them even after the discussion period ends.

We greatly appreciate your time and effort, and thank you again for your valuable feedback.

Reviewer_kLVfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper views a multi-agent defense as a projection from interaction records to detection signals, and gives a sufficient local condition under which benign and malicious records overlap in the projected space. It then trains defense-specific attackers using reinforcement learning rewards for a targeted wrong answer, peer adoption, and detector evasion. The resulting attacks are cross-scored to select complementary defenses under false-positive constraints. Experiments evaluate four defenses across three benchmarks and three backbones, with a smaller study of defense composition under fixed attack policies.

Critical strengths
  • The paper evaluates several different defenses using a shared adaptive-attack framework. Reporting both group accuracy and attacker/honest flag rates is useful: reduced accuracy alone would not reveal that a defense may suppress honest communication while allowing malicious messages through.

  • The composition procedure considers conditional detection gains and false-positive costs rather than simply combining every detector. The paper also distinguishes its local overlap result from the existence of realizable attacks, and limits the composition claims to fixed attack policies. These qualifications make the scope of the analysis clearer.

Critical weaknesses
  • The connection between the theoretical overlap result and the empirical attacks remains largely motivational. Proposition 1 concerns overlap induced by a locally smooth projection under density and derivative assumptions, whereas the empirical attacks directly optimize detector feedback and downstream influence. The paper acknowledges that the proposition does not establish attack reachability or downstream effect. Consequently, the experiments do not show that the observed blind spots arise from the projection-induced overlap characterized by the theory, and no experiment tests a prediction specific to that mechanism. Since Defense as Projection is presented as the first contribution and the organizing principle of the paper, the theoretical result currently serves as a conceptual justification for examining information loss rather than an explanation validated by the attack experiments.

  • The attack evaluation does not isolate the contribution of detector feedback. Table 1 compares clean debates with debates containing learned attackers, but does not compare against the same attacker prompt without training or the same RL objective without the evasion term. Thus, it is unclear how much of the degradation comes from adaptive evasion rather than ordinary wrong-answer persuasion and the compromised agents' retained votes. Reporting targeted attack success, alongside accuracy and flag rates, would better establish whether the intended attack objective is achieved.

Questions for authors
  • Were the Single entries in Table 2 obtained from new evaluations on the same 128 questions and two seeds used for Selected and All four, or were they copied from the 100-question Table 1 experiment? If the latter, please report Single, Selected, All four, and the other individual defenses on the same questions and seeds.
  • How does the full attacker compare with (i) the same attacker prompt without RL tuning and (ii) an equally trained attacker with the detector-evasion term removed from the reward? In addition to group accuracy, could you report targeted attack success and deployed-defense acceptance for these variants? This would isolate whether defense feedback provides an advantage beyond learning to promote the target answer and influence peers.
4 comments
AuthorsOpenReview ↗

Response to Reviewer kLVf (1/2)

We sincerely thank Reviewer kLVf for the careful review, and for recognizing the shared adaptive-attack framework and the value of reporting both attacker and honest flag rates. Our central contribution is the attack side: one RL framework that adapts to four defenses built on different information and consistently degrades them. Composition is a first use of the blind spots it exposes, not a final defense. We address each point below and will add all of it to the revision.

W1. Connection between the theory and the attacks. Thank you for this precise reading. We agree that Proposition 1 is a sufficient local condition and that the attacks test reachability rather than the mechanism itself, and we will state this explicitly. The theory serves two purposes in the paper:

  • Which information is lost matters, not dimension. The counterexample in Appendix A shows that reducing dimension alone does not imply overlap, so the question for each defense is which distinction between benign and malicious behavior its projection discards.
  • Why retuning one defense is not enough. Corollary 1 shows that, when the proposition's overlap condition holds, no rule on the same projection, including a retuned threshold, can avoid both missed attacks and false rejections. This motivates combining defenses that use different information rather than retuning one.

The measured detection outcomes are consistent with this picture: in 13 of 16 single-defense comparisons, attacker messages are withheld less often than honest messages, and under BlindGuard, evasion shifts rejection from attackers to honest agents (W2).

W2 / Q2. Contribution of detector feedback. Thank you for suggesting these controls. All new runs use MMLU with Llama-3.1-8B: 128 questions and two seeds, giving 256 debates per cell, with five agents including two compromised, three rounds, round-local withholding with all final votes counted, and targets fixed in advance. Confidence intervals (CIs) are 95% question-paired bootstrap intervals.

(i) Same prompt without RL tuning. Against each attacker's own target defense, group accuracy / targeted ASR (%):

Target defenseSame prompt, no adapterFull RL attacker
Confidence55.9 / 17.646.9 / 30.9
BlindGuard52.0 / 19.546.5 / 28.5
G-Safeguard54.3 / 18.450.0 / 28.5
GUARDIAN54.3 / 17.252.0 / 28.5

Across all 24 comparisons, six deployed configurations by four attackers, the adapter raises targeted ASR by 5.9 to 16.4 points, each with a 95% CI excluding zero, and lowers group accuracy in every case.

(ii) RL without the evasion term. This variant also exceeds the prompt-only attacker in all eight ASR comparisons, by 7.4 to 18.8 points, so the damage comes from RL training itself. For the two defenses whose RL variants share the same training configuration, against each attacker's own defense:

Target defenseAttacker variantGroup accuracy (%)Targeted ASR (%)Attacker-message acceptance (%)
ConfidenceRL without evasion46.528.599.5
ConfidenceFull RL46.930.997.9
BlindGuardRL without evasion43.833.665.4
BlindGuardFull RL46.528.570.9
AuthorsOpenReview ↗

Response to Reviewer kLVf (2/2)

Accuracy and targeted ASR do not differ significantly between the two RL variants; the G-Safeguard and GUARDIAN variants differ in more than the evasion term, so we do not compare them. The evasion term adds evasion where the defense actually catches attackers: Confidence accepts nearly all attacker messages either way, while BlindGuard's acceptance rises from 65.4% to 70.9%. Over three training seeds per arm on 128 held-out questions, the term significantly lowers BlindGuard's attacker withholding, by 7.2 points, and since BlindGuard flags a fixed quota, honest withholding rises significantly, by 4.8 points.

Persuasion versus retained votes. An oracle that withholds exactly the attackers' messages while keeping their votes raises accuracy by 8.3 points (95% CI 5.2 to 11.7) and lowers targeted ASR by 12.2 points (95% CI 8.6 to 15.9), on 128 held-out questions with two seeds, pooled over the four attackers. A substantial part of the damage therefore travels through messages, not votes alone.

Targeted success. With fixed targets, training concentrates errors on the target: under each attacker's own defense, the share of incorrect debates that end at the target is 38% to 41% for the prompt alone and 53% to 59% for the full attacker. Targeted ASR for all 36 Table 1 cells, with the target setting of each cell, is in our reply to Reviewer BKaU; across cells, a median 58% of incorrect debates end at the target.

We will add these controls and state that training drives the damage, while detector feedback adds measured evasion under BlindGuard.

Q1. Same-question comparison of defenses. Yes, you are right, and thank you for checking this. The Single entries in the submitted Table 2 were taken directly from the 100-question Table 1 experiment, so they were not on the same questions or under the same intervention as Selected and All four. We have rerun every configuration, including all four individual defenses, under one protocol: the same 128 questions, the same two seeds and the same intervention. The selector now chooses by group accuracy on an independent validation set and may return a pair, a single defense or none. Each cell gives group accuracy / targeted ASR (%); columns name the attacker's training-target defense, and † marks that defense deployed alone.

DefenseConfidence attackerBlindGuard attackerG-Safeguard attackerGUARDIAN attacker
None48.4 / 25.843.0 / 32.052.7 / 21.551.2 / 25.4
Confidence46.9 / 30.9 †48.4 / 32.851.2 / 25.043.0 / 27.0
BlindGuard46.9 / 28.146.5 / 28.5 †48.0 / 28.143.0 / 33.2
G-Safeguard47.3 / 32.048.4 / 26.650.0 / 28.5 †50.0 / 25.8
GUARDIAN52.0 / 28.949.2 / 29.349.6 / 27.052.0 / 28.5 †
All four45.7 / 31.246.1 / 29.743.4 / 28.547.3 / 28.9
Selected53.5 / 27.348.4 / 32.850.0 / 28.551.2 / 25.4
(Selected defense)GUARDIAN+G-SafeguardConfidenceG-Safeguardnone

Selected is above All four in every column, by 7.8, 2.3, 6.6 and 3.9 points; against the Confidence attacker, the selected pair reaches 53.5%, above every individual defense.

The rerun also points to a broader finding. Under every attacker, every configuration, including all four defenses combined, stays at least 15.6 points below the 69.1% clean accuracy: an attacker trained against one defense also damages the other three. This suggests that current defenses share blind spots, which limits how much any selection among them can recover (Appendix F). The more promising direction, which our attacker directly enables, is to use it as a source of adaptive attacks for building defenses that close these shared blind spots. We will replace Table 2 with this rerun and make this framing explicit.

Thank you again for the constructive review. We hope these controls and additions address your concerns, and we would be glad to clarify anything further.

Thank you for the detailed rebuttal and the new experiments.

The rebuttal confirms that the projection theory is motivational, not the mechanism. The finding that attackers are withheld less often than honest agents in 13 of 16 comparisons is consistent with overlap. It would also follow from many other causes, such as a weak detector or a miscalibrated threshold, so it does not test a prediction specific to the projection mechanism.

The ablations are informative. RL training clearly drives the attack. However, removing the evasion term does not significantly change accuracy or targeted ASR. Under BlindGuard, the evasion term does produce significant detection-level evasion, shifting withholding from attackers to honest agents. However, this does not translate into lower accuracy or higher targeted ASR, and the comparison covers only one of the four defenses. The evidence therefore shows that detector feedback can change who is flagged, but not that it drives the attack's damage, which comes mainly from RL-trained persuasion.

The rerunning raises new concerns. Table 1 and the rerun disagree. On MMLU/LLaMA-3.1-8B with the BlindGuard attacker against BlindGuard, Table 1 reports 23.0% attacked accuracy, while the rebuttal's rerun reports 46.5%. The GUARDIAN numbers are 33.0% vs. 52.0%. The selector changed, and the composition gains are small, which weakens the composition contribution.

Overall, I think the paper requries substaintial reframing and explanations of the results. I therefore keep my rating.

Thank you for the follow-up and for reading the new results so carefully. We respond briefly to each point.

Theory. We agree that the 13 of 16 result alone does not isolate the projection mechanism from alternatives such as a weak detector or a miscalibrated threshold. In the revision, Proposition 1 is a sufficient local condition that motivates the attack design, our attacks test reachability, and Appendix G.2 states that this detection pattern is not a test of the mechanism.

What drives the damage. We agree, and the revision states it this way: training drives the damage, while detector feedback adds measured evasion under BlindGuard, the defense where this can be tested, since Confidence accepts nearly all attacker messages either way. The oracle control confirms that the damage travels mainly through RL-trained persuasion in the messages: withholding exactly the attackers' messages while keeping their votes raises accuracy by 8.3 points and lowers targeted ASR by 12.2 points. This learned persuasion attack, which adapts to four defenses built on different information, is the central contribution of the paper.

Table 1 versus the rerun. The two use different interventions. In Table 1, BlindGuard, G-Safeguard and GUARDIAN removed flagged agents together with their votes, while the rerun withholds flagged messages and keeps all votes (our reply to Reviewer DvBF, Q3). On the rerun's questions with the same frozen attackers, switching BlindGuard to removal gives 23.4% against its own attacker, matching the 23.0% in Table 1, and switching GUARDIAN to removal lowers accuracy against its own attacker from 52.0% to 43.4%; the two experiments also use different question sets (100 vs. 128 questions). The revision states the intervention of every Table 1 row and reports both interventions side by side.

Selector and composition. The rerun uses a selector that chooses by group accuracy on an independent validation set. The selected deployment is above stacking all four defenses in every column and avoids most of their honest-message cost, and against the Confidence attacker it is above every individual defense (53.5%). We present composition as a first use of the blind spots the attacker exposes. The broader finding is that every configuration stays at least 15.6 points below clean accuracy: current defenses share blind spots, and our attacker can serve as a source of adaptive attacks for building defenses that close them.

The revision adopts the framing you suggest: the attack is the central contribution, Proposition 1 motivates it, and composition is a first use of the blind spots it exposes. Thank you again for the careful reading.

Reviewer_BKaUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The experiments establish substantial vulnerability, but the contributions of defense-aware attack training and complementary defense composition are not sufficiently distinguished from simpler alternatives.

Critical strengths
  • The projection view provides a coherent way to discuss heterogeneous defenses. Section 3.1 distinguishes information loss from benign–malicious overlap and carefully limits what the local result establishes. The selection objective in Section 3.3 then connects this view to additional attack detection and honest-message rejection. This is a useful organizing framework, even though the theoretical argument alone does not establish exploitable failures.

  • The attack results are consistently damaging. Table 1 shows accuracy decreases across all 36 defense–dataset–backbone combinations. Figure 3 adds a relevant diagnostic: low attacker detection can coexist with substantial rejection of honest participants. These findings provide useful evidence about the evaluated safeguards beyond a single successful attack example.

Critical weaknesses
  • The composition comparison does not isolate the benefit of combining defenses. In Table 2, “Single” is the defense targeted during attacker training, rather than a singleton selected under the same calibration constraints as the proposed combination. A different individual detector might already recover much of the reported accuracy. The natural control is the initial feasible singleton selected by Algorithm 1, evaluated against the same frozen attack. Without that comparison, I cannot determine how much the result supports complementarity rather than replacing a poorly matched defense.

  • The contribution of defense-aware optimization is unclear. Section 4.2 primarily compares clean operation with operation under the trained attacker. This establishes disruption, but does not show what incorporating the defense’s feedback adds over an otherwise comparable attack. A matched attack without the evasion reward would help distinguish the proposed optimization from successful persuasion more generally.

  • Targeted success is defined but not quantified at the final endpoint. Section 2.1 defines success as the group producing the assigned wrong answer. Table 1 instead reports correct-answer accuracy, while Figure 3 reports detection rates. These outcomes cannot distinguish steering toward the specified target from causing other mistakes. The evidence for damage is clear; the evidence for targeted control is less complete.

  • The selected configurations’ communication cost remains unresolved. The cached false-positive constraint is explicit in Section 3.3, but Table 2 does not report clean accuracy or honest-message withholding for the exact selected configurations. Since controlling this cost is part of the selection objective, attacked accuracy alone does not establish the intended protection–communication trade-off.

Questions for authors
  • Evaluation check: How do the final combinations compare with Algorithm 1’s initial selected singleton on the same held-out debates and under the same false-positive budget? What clean accuracy and honest-message withholding accompany those configurations?

  • Attack-contribution check: Are there matched results with the evasion reward disabled, or another comparable attacker without defense feedback? How much additional damage or evasion comes from that feedback?

  • Outcome clarification: Among the complete debates evaluated for Table 1, what fraction ended at the assigned wrong target, rather than another incorrect answer?

3 comments
AuthorsOpenReview ↗

Response to Reviewer BKaU (1/2)

We sincerely thank Reviewer BKaU for the careful review, and for finding the projection view a coherent way to discuss heterogeneous defenses and the attack results consistently damaging. Our central contribution is the attack side: one RL framework that adapts to four defenses built on different information and consistently degrades them. Composition is a first use of the blind spots it exposes, not a final defense. We address each point below with new controls and will add all of them to the revision.

All new runs use MMLU with Llama-3.1-8B: 128 questions and two seeds, giving 256 debates per cell, with five agents including two compromised, three rounds, round-local withholding with all final votes counted, and targets fixed in advance. Confidence intervals (CIs) are 95% question-paired bootstrap intervals.

W1. Benefit of composition over a singleton control. Thank you for this important point; we agree that the singleton control is needed. The Single entries in the submitted Table 2 were also taken directly from the 100-question Table 1 experiment, so they were not matched to the other rows. We reran every configuration, including each individual defense, under one protocol, with the same questions, seeds, intervention and frozen attacks, so any singleton can be compared directly with the selected deployment. The selector now chooses by group accuracy on an independent validation set and may return a pair, a single defense or none; this replaces, rather than reproduces, Algorithm 1's comparison with its initial singleton under the cached false-positive budget. Each cell gives group accuracy / targeted ASR (%); columns name the attacker's training-target defense, and † marks that defense deployed alone.

DefenseConfidence attackerBlindGuard attackerG-Safeguard attackerGUARDIAN attacker
None48.4 / 25.843.0 / 32.052.7 / 21.551.2 / 25.4
Confidence46.9 / 30.9 †48.4 / 32.851.2 / 25.043.0 / 27.0
BlindGuard46.9 / 28.146.5 / 28.5 †48.0 / 28.143.0 / 33.2
G-Safeguard47.3 / 32.048.4 / 26.650.0 / 28.5 †50.0 / 25.8
GUARDIAN52.0 / 28.949.2 / 29.349.6 / 27.052.0 / 28.5 †
All four45.7 / 31.246.1 / 29.743.4 / 28.547.3 / 28.9
Selected53.5 / 27.348.4 / 32.850.0 / 28.551.2 / 25.4
(Selected defense)GUARDIAN+G-SafeguardConfidenceG-Safeguardnone

Selected is above All four in every column, by 7.8, 2.3, 6.6 and 3.9 points. Against the Confidence attacker it pairs GUARDIAN with G-Safeguard and reaches 53.5%, above every individual defense; against the other attackers it keeps a single defense or none and avoids the honest-message cost of stacking (W4).

The rerun also points to a broader finding. Under every attacker, every configuration, including all four defenses combined, stays at least 15.6 points below the 69.1% clean accuracy: an attacker trained against one defense also damages the other three. This suggests that current defenses share blind spots, which limits how much any selection among them can recover (Appendix F). The more promising direction, which our attacker directly enables, is to use it as a source of adaptive attacks for building defenses that close these shared blind spots. We will replace Table 2 with this rerun and make this framing explicit.

W2. Contribution of defense-aware optimization. Thank you for suggesting these controls. We compared the full attacker with two matched variants: the same prompt without the trained adapter, and the same RL training without the evasion reward.

  • Same prompt, no adapter. Adding the trained adapter raises targeted ASR in all 24 comparisons across six deployed configurations and four attackers, by 5.9 to 16.4 points, each with a 95% CI excluding zero, and lowers group accuracy in all 24.
  • RL without the evasion reward. This variant also exceeds the prompt-only attacker in all eight ASR comparisons, by 7.4 to 18.8 points, so the damage comes from RL training itself. The evasion reward adds evasion where the defense actually catches attackers. Under BlindGuard, which withholds the most attacker messages, it lowers attacker withholding from 34.6% to 29.1%, and over three training seeds per arm on 128 held-out questions, it lowers it significantly, by 7.2 points. Because BlindGuard flags a fixed quota, honest withholding rises significantly, by 4.8 points: evading attackers push rejection onto honest agents, the pattern of Figure 3 you highlighted. Confidence accepts over 97% of attacker messages with or without the term, so there is little to evade.

Details are in our reply to Reviewer kLVf. We will add these controls and state that training drives the damage, while detector feedback adds measured evasion under BlindGuard.

AuthorsOpenReview ↗

Response to Reviewer BKaU (2/2)

W3. Targeted success at the final endpoint. Thank you for this suggestion; we now report targeted ASR (%) for all 36 defended cells of Table 1. In 22 of the 36 cells, at least half of the incorrect debates end at the attack target, with a median of 58%.

SettingConf.BlindGuardG-SafeguardGUARDIAN
Llama, MMLU*34734158
Llama, GSM8K*16361341
Llama, BBH*55.5745471.5
Qwen, MMLU1961*1840
Qwen, GSM8K548416
Qwen, BBH31652936
Mistral, MMLU29402533
Mistral, GSM8K61385820
Mistral, BBH34473332

In the 13 starred cells, the target can be updated after the honest first round to the most frequent wrong answer among honest agents; the other cells and all new runs use a target fixed in advance. With fixed targets, training concentrates errors on the target: under each attacker's own defense, the share of incorrect debates that end at the target is 38% to 41% with the prompt alone and 53% to 59% with the trained adapter. We will add targeted ASR to Table 1 and state both target settings.

W4. Clean accuracy and honest-message withholding. Thank you for raising the communication cost. Three of the four selected deployments are a single defense or none, and their clean costs are measured without an attacker in Appendix E:

Selected deployment (attacker column)Clean accuracy (%)Honest-message withholding (%)
Confidence (BlindGuard attacker)66.811.6
G-Safeguard (G-Safeguard attacker)66.08.3
None (GUARDIAN attacker)69.10.0
All four, for reference68.459.2

Selection thus avoids most of the honest-message cost of stacking all four defenses, which is the trade-off the selection objective targets. We will report these costs next to Table 2.

Thank you again for the constructive review. We hope these controls and additions address your concerns, and we would be glad to clarify anything further.

As the discussion period comes to an end, we would like to express our sincere gratitude for your thoughtful review and suggestions. Your questions on the singleton control for composition, the contribution of detector feedback, and targeted success at the final endpoint led to new matched controls and a full rerun of the composition study, and they helped us state the paper's claims more precisely. We have uploaded a revised version in which the changes made in response to your review are marked in blue and labeled with your reviewer ID.

We truly hope that our answers have addressed your concerns. That being said, if there are any other questions, please feel free to leave us a message. We would be keen to address them even after the discussion period ends.

We greatly appreciate your time and effort, and thank you again for your valuable feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Discussion for the Area Chair and Reviewers

We sincerely thank the Area Chair and Reviewers BKaU, kLVf and DvBF for their time and for a constructive discussion. Below we summarize our contribution, our clarifications, the new experiments, and the status of each reviewer's concerns.

1. Contribution. Our central contribution is the attack side: one reinforcement learning framework that adapts, through a single reward interface, to four multi-agent defenses built on different information (Confidence, BlindGuard, G-Safeguard and GUARDIAN).

  • A general stress test. The attack lowers group accuracy in all 36 defense, dataset and backbone cells of Table 1, and in 22 of the 36 cells at least half of the incorrect debates end at the attack target.
  • Measurements beyond accuracy. Attacker and honest flag rates show that a defense can suppress honest communication while letting malicious messages through.
  • A finding about current defenses. Under every attacker, every deployed configuration, including all four defenses combined, stays at least 15.6 points below clean accuracy: an attacker trained against one defense also damages the other three. Composition is a first use of these shared blind spots.

2. Clarifications.

  • Role of the theory (kLVf W1 and follow-up). Proposition 1 is a sufficient local condition that motivates the attack design; the attacks test reachability (Section 3.1, Appendix G.2).
  • Intervention protocol (DvBF Q3; kLVf follow-up). In Table 1, Confidence withholds messages while the three graph safeguards remove flagged agents with their votes; the rerun and the composition study withhold messages and keep all votes. This explains most of the difference between Table 1 and the rerun: with removal on the rerun's questions, BlindGuard gives 23.4% against its own attacker, matching 23.0% in Table 1. The revision states each row's intervention.
  • Fixed attackers (DvBF W2). The composition results are stated for fixed attack policies, as in the submission.

3. New experiments.

  • What RL training adds (BKaU W2; kLVf W2, Q2; DvBF W1). Against the same prompt without training, the trained attacker raises targeted ASR by 5.9 to 16.4 points in all 24 comparisons, each with a 95% CI excluding zero. An attacker trained without the evasion term also exceeds the prompt in all eight ASR comparisons, and under BlindGuard the evasion term lowers attacker withholding by 7.2 points.
  • Messages versus votes (DvBF Q3). Withholding exactly the attackers' messages while keeping their votes raises accuracy by 8.3 points, so the damage travels mainly through learned persuasion in the messages.
  • Existing attacks and no defense (DvBF W1, Q2). The learned attacker gives the lowest accuracy and the highest targeted ASR under the three embedding-level defenses, and the attacks remain damaging with every detector disabled.
  • Targeted success (BKaU W3). Targeted ASR for all 36 cells of Table 1; a median of 58% of incorrect debates end at the target.
  • Same-protocol rerun of all defense configurations (BKaU W1, W4; kLVf Q1; DvBF Q1). The selected deployment is above stacking all four defenses in every column, by 2.3 to 7.8 points, and avoids most of their honest-message cost (59.2% withholding for all four in clean debates).

4. Status of each reviewer's concerns.

  • Reviewer kLVf found the ablations informative and agreed that RL training clearly drives the attack. The reviewer keeps the rating and asked for a framing centered on this finding and for an explanation of the difference between Table 1 and the rerun. The revision adopts this framing, and we explained the difference by the change of intervention.
  • Reviewers BKaU and DvBF have not posted further comments. We answered every question with the controls they asked for: the singleton control, the matched attackers, targeted ASR and clean costs for BKaU; the prompt-only and evasion-free attackers, existing attacks, the same-question rerun and both interventions for DvBF.

We are grateful for the careful reading and suggestions, which are reflected in the revised paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.