acceptodds
Under review as a conference paper at ICLR 2027

MADBench: Benchmarking the Security of Multi-Agent Debate

Abstract

Multi-agent debate (MAD) can improve large language model (LLM) reasoning by allowing multiple agents to exchange and critique their answers to the same task. However, the interactions that enable agents to correct mistakes can also spread adversarial errors and steer the agents toward an incorrect answer. Although some efforts have been made to examine particular attack types on MAD, systematic evaluation of MAD under diverse attacks remains limited. A central question is whether debate mitigates adversarial influence or amplifies it. In this paper, we present MADBench, a benchmark for evaluating the security of MAD. We organize attacks into a layered taxonomy following the MAD workflow, incorporating both established attacks and new strategies tailored to debate. We evaluate six attack families over 356 source tasks and 3,958 test cases, examining their effects on the final answer and the propagation of adversarial influence. Our results show that, under attacks, MAD does not necessarily improve LLM reasoning. Compared with a single-agent baseline, MAD can mitigate attacks on answer accuracy in question-answering tasks while amplifying unauthorized reads or writes in both question-answering and workspace tasks. Moreover, even when three out of five agents collude, the attack changes the final answer from correct to wrong on only 28.30% of tasks answered correctly without attack, while only 3.26% of initially correct honest agents switch to wrong answers during debate.

Primary area:
datasets and benchmarks
Keywords:
Multi-agent debate, Large language model agents, Multi-agent system security, Adversarial robustness, Agent evaluation benchmark, Indirect prompt injection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PvarOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces MADBench, a benchmark for evaluating the security of Multi-Agent Debate (MAD) systems. It organizes six attack families across orchestration, agent, and resource layers, using an Amplification Factor (AF) to compare attack impacts between MAD and single-agent settings. Experiments across 356 tasks and 3,958 test cases show that MAD can improve answer-level robustness but may amplify unauthorized tool operations, highlighting unique multi-agent security risks.

Critical strengths

S1. A systematic evaluation framework covering multiple security dimensions of MAD.

The paper provides a structured taxonomy of six attack families across the orchestration, agent, and resource layers. More importantly, it evaluates not only answer accuracy but also unauthorized read/write operations and computational cost. This broader perspective is valuable because a multi-agent system may produce a correct final answer while still performing unsafe actions. The unified framework and released implementation could provide a useful starting point for future research on MAD security.

S2. Interesting observations about the distinction between answer robustness and action safety.

The experiments reveal a potentially important distinction: MAD can mitigate attacks on answer accuracy while increasing the risk of unauthorized operations. The workspace case study in Table 3 is particularly illustrative, showing an unauthorized payment that occurs in the MAD setting but not in the single-agent baseline. The observation that relatively few initially correct honest agents adopt incorrect answers despite successful manipulation of some final outputs is also interesting. These findings motivate a closer investigation of how multi-agent communication and aggregation affect different security objectives.

Critical weaknesses

W1. The experimental design does not adequately isolate the effect of debate from other multi-agent system components.

The central research question is whether MAD absorbs or amplifies adversarial attacks. However, the main evaluation compares a multi-agent debate system consisting of five agents and an orchestrator against a single-agent baseline. These two settings differ in several important aspects beyond debate, including the number of agents, opportunities for exposure to adversarial information, aggregation mechanisms, and the distribution of tool-execution responsibilities.

In particular, a higher unauthorized-operation rate in MAD could arise simply because more agents independently encounter or respond to malicious information, even without any debate. Similarly, improvements in answer accuracy might result from aggregating multiple independent answers rather than from the debate process itself.

The paper does not provide a matched multi-agent baseline with the same number of agents and orchestrator but without debate. Although the authors vary the number of debate rounds, these experiments do not isolate the effect of enabling debate in the first place.

Consequently, the reported Amplification Factors characterize differences between the evaluated system configurations but do not establish that debate itself causes the observed amplification or mitigation. This substantially weakens the paper's central claim.

W2. The unauthorized-operation metric does not sufficiently distinguish attack success from incidental security violations.

The paper defines Incorrect Read and Write (IRW) as the occurrence of any unauthorized read or write operation. While this is a reasonable broad measure of security violations, it does not necessarily capture whether the attacker's specific objective was achieved.

For example, the case study in Section 4.2 and Appendix D.6 reports unauthorized file creation during an attack targeting answer accuracy. Such behavior constitutes a policy violation but does not necessarily demonstrate successful execution of an attacker-targeted action.

Furthermore, unlike the accuracy-based ASR, which conditions on correct clean-run outputs, IRW does not explicitly isolate violations introduced by the attack from violations that may already occur in clean executions. The aggregate metric also combines unauthorized reads and writes and does not clearly attribute violations to compromised agents, honest agents, or the orchestrator.

These distinctions are important for interpreting the reported attack amplification. The evaluation would be substantially stronger with separate read/write metrics, attacker-goal-specific success rates, and attack-induced violation rates relative to clean executions.

Questions for authors

Q1. Is the observed attack amplification attributable to debate itself?

Have the authors evaluated a multi-agent baseline with the same number of agents, orchestrator, tool permissions, and attack exposure, but without the debate stage? If so, how do its accuracy and unauthorized-operation rates compare with those of MAD? This comparison would be important for distinguishing the effects of debate from those of multi-agent aggregation.

Q2. How much of the reported IRW is directly attributable to successful attacker objectives?

Could the authors clarify how frequently the exact attacker-targeted unauthorized operation is executed, as opposed to an unrelated unauthorized operation? Additionally, how often do similar unauthorized operations occur in clean runs? Separating these cases would help determine whether the reported results reflect attack-induced security failures.

1 comment

Thank you for asking us to separate the effects of debate, multi-agent aggregation, and unauthorized operations. We added the no-debate control and the HybridQA50 audit to make those distinctions clearer.

Q1: Is the observed attack amplification attributable to debate itself? Have the authors evaluated a multi-agent baseline with the same number of agents, orchestrator, tool permissions, and attack exposure, but without the debate stage? If so, how do its accuracy and unauthorized-operation rates compare with those of MAD?

A1: The results show that lower ASR and higher unauthorized-read rates can occur even without debate. Thus, differences from a single agent cannot be explained by debate alone. We use the same five workers, orchestrator, tool permissions, and attack setup, but remove peer review as a no-debate baseline. For M3 and accuracy attack, the results are:

ConfigurationClean accuracyAttacked accuracyASR
Single agent38/49 (77.55%)4/50 (8.00%)34/38 (89.47%)
No-debate MAD, R=032/49 (65.31%)14/50 (28.00%)18/32 (56.25%)
MAD, R=127/49 (55.10%)16/50 (32.00%)16/27 (59.26%)

Both multi-agent settings have lower ASR than the single agent. Adding debate raises attacked accuracy, but lowers clean accuracy and slightly raises ASR in this comparison. :

HybridQA50 attack and measured outcomeSingle agentNo-debate MAD, R=0MAD, R=1
M2: IR under Unauth.read attack52%96%96%
M4: IW under Unauth.write attack100%0%10%

These rates use all 50 tasks. M2 IR under Unauth.read attack is equally high with and without debate. M4 IW under Unauth.write attack rises from 0% to 10% with debate.

【Section 4.1】 describes the no-debate baseline and explains the scope of the comparison.

【Table 2(b)–(c)】 reports the AFs relative to the single-agent and no-debate baselines.

Q2: How much of the reported IRW is directly attributable to successful attacker objectives? Could the authors clarify how frequently the exact attacker-targeted unauthorized operation is executed, as opposed to an unrelated unauthorized operation? Additionally, how often do similar unauthorized operations occur in clean runs?

A2: We now separate IRW into unauthorized reads (IR) and unauthorized writes (IW). We also added an authorization audit in Appendix E.2 to check whether the specific attacker-requested operation is completed and whether violations already occur without attack. For MAD on HybridQA50, the results are:

DatasetAttack conditionIRTarget-read completionIWTarget-write completion
AgentDojoM2: Unauth.read attack94.64%52/56 (92.86%)48.21%–
AgentDojoM2: Unauth.write attack50%–66.07%20/56 (35.71%)
SealQAM3: Unauth.read attack100%100/100 (100%)3%–
SealQAM3: Unauth.write attack46%–78%77/100 (77%)
HybridQA50M2: Unauth.read attack96%96%6%–
HybridQA50M3: Unauth.read attack72%72%6%–
HybridQA50M2: Unauth.write attack0%–62%62%

Without attacks, both IR and IW are 0% for MAD and the single agent on HybridQA50. No-debate MAD has 0% IR and 2% IW. All 31 MAD M2-Unauth.write write violations are absent from their clean pairs. In contrast, the sole no-debate M5-Unauth.write violation already occurs in its clean run and does not complete the attacker's target.

We also checked whether the wrong answers counted by ASR match the attacker's chosen answer.

Accuracy attackASR (any wrong answer)ASRTarget
M360.38%60.38%
M449.06%49.06%
M5, colluding28.30%28.30%

Every ASR failure in these three conditions adopts the specified wrong answer. Thus, these failures reflect successful answer manipulation rather than unrelated mistakes.

【Section 3.3, Tables 2 and 4】 defines and reports IR and IW separately.

【Appendix E.2】 reports target-operation completion and paired clean-run comparisons on HybridQA50.

【Appendix D.1, Section 4.2】 reports the original SealQA ASRs used in the answer-target comparison above.

Reviewer_7ruPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MADBench, a benchmark for systematically evaluating the security of Multi-Agent Debate (MAD) systems under adversarial attacks. The authors propose a three-layer threat taxonomy covering orchestration, agents, and external resources, and implement six attack families, including prompt injection, RAG poisoning, tool hijacking, coordinated malicious agents, and orchestrator compromise. The benchmark consists of 356 source tasks and 3,958 test cases, evaluating attack success rates, unauthorized read/write operations, and token consumption against single-agent baselines. Experimental results suggest that MAD can mitigate attacks targeting answer accuracy in question-answering tasks while amplifying unauthorized operations, particularly in workspace tasks. Additionally, the authors find that coordinated malicious agents can manipulate the final decision even when honest agents rarely change their initially correct answers.

Critical strengths
  1. The paper organizes security threats into three layers and evaluates six attack families, including both established single-agent attacks and attacks specific to MAD.
  2. The benchmark considers answer accuracy, unauthorized read/write operations, and token consumption, using an amplification factor to compare MAD against single-agent baselines. This multidimensional evaluation reveals that improvements in answer robustness do not necessarily translate into better operational security
  3. Beyond reporting aggregate attack success rates, the paper analyzes agent behavior across the Propose, Debate, and Decide stages.
Critical weaknesses

The paper defines ASR as the fraction of initially correct tasks that become incorrect under attack, and uses paired clean and attacked runs with matched random seeds. However, LLM-based debate is inherently stochastic, and the observed answer changes may partially arise from natural run-to-run variability instead of adversarial influence. The paper lacks repeated clean-run controls or multi-seed evaluations to quantify this baseline variability. Consequently, it remains unclear how much of the reported attack success rates can be causally attributed to the attacks themselves. 2. The main experiments rely on a fixed configuration with five fully connected agents, a centralized orchestrator, and a single debate round. Although the authors provide additional experiments varying the underlying models and debate rounds, they do not investigate different communication topologies, aggregation strategies, agent role assignments, or decentralized MAD architectures. Consequently, it remains unclear how well the reported findings generalize to other MAD designs. 3. The M6 attack grants the adversary direct control over the orchestrator's instructions, allowing it to override agent outputs or invoke unauthorized tools, making its near-100% success rates largely a consequence of compromising a privileged component. For M5, different coordination modes involve variations in attack instructions and message delivery mechanisms, making it difficult to isolate the causal effect of coordination itself, as also acknowledged in the appendix. Moreover, the evaluation does not investigate adaptive attacks specifically optimized against MAD interactions, leaving the system's behavior under stronger adversarial strategies largely unexplored.

Questions for authors

N/A

1 comment

Thank you for pointing out the need to check result stability and clarify the scope of our findings. We added three seeds with matched controls and clarify what the attack comparisons establish, including the limits on architecture coverage and adaptive attacks.

Q1: However, LLM-based debate is inherently stochastic, and the observed answer changes may partially arise from natural run-to-run variability instead of adversarial influence. The paper lacks repeated clean-run controls or multi-seed evaluations to quantify this baseline variability. A1: The additional seeds support our original finding that MAD can reduce accuracy-attack ASR while increasing unauthorized-read or write rates relative to a single agent. For the comparisons below, all three additional seeds retain the original direction despite variation in the magnitude. We repeated the original task settings with seeds 1–3 and matched single-agent controls. The table compares the original AFs with the mean and sample standard deviation across the additional seeds:

Attack and measured outcomeOriginal AFMean AF ± sample SD, seeds 1–3Direction across seeds 1–3
M3: ASR under accuracy attack0.740.71 ± 0.02Below 1 in every seed
M4: ASR under accuracy attack0.870.77 ± 0.14Below 1 in every seed
M2: IR under Unauth.read attack1.962.20 ± 0.14Above 1 in every seed
M3: IR under Unauth.read attack1.051.04 ± 0.01Above 1 in every seed
M3: IW under Unauth.write attack1.321.33 ± 0.07Above 1 in every seed

M3/M4 accuracy-attack AFs remain below 1, while the listed read/write AFs remain above 1. These repetitions show that the reported directions in these comparisons are consistent across seeds.

【Appendix E.1】 reports individual seed results, matched single-agent controls, sample standard deviations, and M1 seed sensitivity.

Q2 Moreover, the evaluation does not investigate adaptive attacks specifically optimized against MAD interactions, leaving the system's behavior under stronger adversarial strategies largely unexplored. We include adaptive attacks in M5. During debate, compromised workers read other workers' responses and adjust their arguments and fabricated evidence to persuade them to accept the attacker's answer. In the colluding strategy, the wrong target stays fixed, but the supporting messages adapt to peer responses across rounds.

【Section 3.2】 defines the M5 attack strategies.

【Appendix B】 describes peer review and the compromised-worker prompts.

【Appendix D.2】 reports M5 results with one, two, and three debate rounds.

Q3:Although the authors provide additional experiments varying the underlying models and debate rounds, they do not investigate different communication topologies, aggregation strategies, agent role assignments, or decentralized MAD architectures. Our benchmark can be used to evaluate different communication topologies, aggregation rules, and agent roles. This paper focuses on security attacks across MAD's layers, rather than comparing these design choices, so we hold the architecture fixed: five fully connected workers share their reports, and a central orchestrator aggregates them. Other work has studied topology directly.

【Section 4】 specifies the five-agent centralized MAD design used in the evaluation.

【Section 5】 discusses NetSafe and Huang et al.'s work on network topology and collaboration structures.

Reviewer_KPGnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper, MADBench, studies whether multi-agent debate (MAD) absorbs or amplifies different attacks. It evaluates six attack families across 356 source tasks (from four prior datasets, including SealQA, StrategyQA, AgentDojo, and JailbreakBench), spanning 3,958 cases organized around three layers: agents, resources, and orchestration. The central finding of the paper is that MAD can reduce the impact of some QA attacks while increasing unauthorized workspace read/write operations. Colluding malicious agents can influence final decisions, although honest agents often retain their original answers.

Critical strengths
  • Useful benchmark structure. The layered organization of attacks provides a reasonable basis for comparing several attack surfaces within one evaluation framework. The use of paired clean and attacked executions is also a sound experimental design choice.
  • Relevant and well-motivated problem. Evaluating multi-agent systems only by final-answer accuracy overlooks potentially serious safety failures. The paper's distinction between correctness and unauthorized workspace actions is important. I also particularly appreciated the case study examples. An indirect injection induces an unauthorized read, after which the debate agents converge on an unauthorized payment that the single-agent baseline does not execute. This is a compelling case of how malicious influence can propagate through a decision-making process.
Critical weaknesses
  • The paper's central finding that MAD can absorb attacks against answer accuracy while amplifying unauthorized operations is interesting, but I am not fully convinced that the current experiments sufficiently establish this general pattern. In particular, Table 2 evaluates different attack families on different task types (e.g., M2 on workspace tasks versus M3/M4 on factual QA tasks), making it difficult to distinguish whether the observed differences arise from the task characteristics, attack mechanisms, or the MAD architecture itself. Moreover, the main M1–M4 experiments rely primarily on GPT-4o under a single default MAD configuration, without sufficient evidence of robustness across models and independent random seeds. Although the presented case studies illustrate plausible mechanisms of attack absorption and propagation, they do not establish how frequently these mechanisms explain the overall results.

  • I would encourage the authors to provide stronger experimental controls, including comparable attack mechanisms across QA and workspace tasks where feasible, repeated evaluations with confidence intervals, and additional models for the main RQ1 experiments. This would help establish whether the reported absorption and amplification patterns generalize beyond the specific evaluated configurations.

  • Although the paper motivates MADBench through the propagation of adversarial influence during debate, the evaluation primarily measures final attack outcomes, making it difficult to determine whether debate interactions themselves cause the observed failures. For example, under M5, colluding agents can manipulate the orchestrator's final decision without necessarily convincing honest agents to adopt incorrect answers, as also suggested by the low correct-to-wrong (C2W) rates. Similarly, increased unauthorized operations could arise from the aggregation mechanism or broader multi-agent architecture rather than the propagation of malicious instructions during debate.

  • A particularly important missing control is a multi-agent no-debate baseline, in which agents independently propose answers that are directly aggregated. Comparing this configuration against the current MAD setup would help isolate the impact of debate interactions from the effects of multiple agents and aggregation. Furthermore, a more systematic analysis of intermediate agent responses and tool calls could reveal how frequently adversarial influence actually propagates between agents and contributes to final security failures. The existing ablation across different numbers of debate rounds does not fully address this issue, since all evaluated configurations still involve debate.

  • The proposed metrics raise concerns about whether they accurately capture attack-induced failures. In particular, Equation 6 combines unauthorized reads and writes into a single IRW metric, even though these are distinct security violations with potentially different consequences. Consequently, an attack targeting an unauthorized read may receive a successful IRW outcome even if the system instead performs an unrelated unauthorized write. Furthermore, IRW measures violations in the attacked execution without explicitly accounting for unauthorized operations that might already occur in the corresponding clean execution. This makes it difficult to attribute the observed failures specifically to the attacks.

  • The amplification factor introduces an additional interpretability concern, as comparing adverse-outcome rates between MAD and single-agent configurations does not necessarily isolate the effect of debate or communication. For ASR, the two systems may also use different sets of initially correct tasks. Reporting clean and attacked failure rates separately, distinguishing targeted read and write violations, and including attack-induced changes would make the security conclusions more reliable.

Questions for authors
  • Experimental validity and central claims: Could the authors address the concerns and suggested experimental controls discussed in Major Weaknesses 1 and 2, particularly regarding the robustness of the results and the extent to which the observed absorption/amplification can be attributed specifically to debate interactions?
  • Could the authors address the metric-related concerns in weakness 3 and clarify whether the reported IRW results reflect successful execution of the attacker's specific objective or any unauthorized operation?
  • Could the authors clarify what distinguishes M5 and M6 from previously studied multi-agent attacks, particularly what additional MAD-specific security insights are gained from evaluating a directly compromised orchestrator?
3 comments

Thank you for pointing out the limits of the evaluation. To certify our results,we designed a new benchmark HybridQA50 to test all attack families in one benchmark. We also added a no-debate mode of MAD as baseline, three additional seeds, and a targeted authorization audit for determining whether unauthorized operations are introduced by the attack and whether they achieve the attacker’s specific objective.

Q1:In particular, Table 2 evaluates different attack families on different task types (e.g., M2 on workspace tasks versus M3/M4 on factual QA tasks), making it difficult to distinguish whether the observed differences arise from the task characteristics, attack mechanisms, or the MAD architecture itself.

A1: We design a benchmark HybridQA50, where all attack families can be evaluate. From our results, we find that the absorption and amplification patterns remain consistent across different task types.

In all benchmarks we use, MAD has lower ASR than a single agent for M3/M4, but higher IR under M2–M4 attacks targeting unauthorized reads. This supports our original claim that MAD can absorb attacks on answer accuracy while amplifying unauthorized operations. The table compares the original results with HybridQA50.

Attack and measured outcomeOriginal task setOriginal AFHybridQA50 AF
M2: ASR under accuracy attackAgentDojo1.051.16
M3: ASR under accuracy attackSealQA0.740.66
M4: ASR under accuracy attackSealQA0.870.92
M2: IR under Unauth.read attackAgentDojo1.961.85
M3: IR under Unauth.read attackSealQA1.053.60
M4: IR under Unauth.read attackSealQA1.021.47
The direction is consistent across the two task settings for each comparison shown.
[Section 4.1, Table 2(b)], adds the M2–M4 comparisons against a single agent.
[Table 2(c)] adds M2–M6 comparisons against no-debate MAD.
[Section 4.2, Table 4(d)], adds the M5 coordination results.
[Appendix C] describes the HybridQA50 task construction, examples, success criteria, and authorization rules. We have released the benchmark at HybridQA50.

Q2: without sufficient evidence of robustness across models and independent random seeds.

A2: The results across the three additional seeds are consistent with our original conclusion that MAD absorbs M3/M4 accuracy attacks relative to a single agent. Changing the seed does not change the direction of either comparison. To check this consistency, we repeated the original task settings with seeds 1–3 and matched single-agent controls. The table shows AFs for the comparisons supporting the two parts of this finding.

Attack and measured outcomeTask setOriginal AFSeed 1 AFSeed 2 AFSeed 3 AF
M3: ASR under accuracy attackSealQA0.740.720.690.72
M4: ASR under accuracy attackSealQA0.870.740.650.93
M2: IR under Unauth.read attackAgentDojo1.962.252.302.04
M3: IR under Unauth.read attackSealQA1.051.051.031.04
M3: IW under Unauth.write attackSealQA1.321.271.311.40

M3/M4 accuracy-attack AFs stay below 1, while the listed read/write AFs stay above 1. These repetitions support both parts of the main finding on the evaluated QA and workspace tasks.

[Appendix E.1] reports the seed results, matched single-agent controls, and sample standard deviations.

Q3: I would encourage the authors to provide stronger experimental controls, including comparable attack mechanisms across QA and workspace tasks where feasible, repeated evaluations with confidence intervals, and additional models for the main RQ1 experiments.

A3: The additional experiments support both parts of our main finding: MAD can absorb accuracy attacks while amplifying unauthorized operations. On HybridQA50, M3/M4 accuracy-attack AFs remain below 1, while M2–M4 read-attack IR AFs remain above 1. Across the three additional seeds on the original tasks, M3/M4 accuracy-attack AFs also remain below 1, while M2/M3 read-attack IR AFs and M3 write-attack IW AFs remain above 1.

【Section 4.1, Table 2(b)】 adds the HybridQA50 comparisons against a single agent, using a shared task set for M2–M4.

【Appendix E.1】 reports the additional seed results, matched single-agent controls, and sample standard deviations. These quantify variation across seeds.

【Section 4.2, Table 4(b)】 reports the model comparison for M5.

Q4: A particularly important missing control is a multi-agent no-debate baseline, in which agents independently propose answers that are directly aggregated. A4:

To test this, we evaluated a no-debate baseline with the same five workers, orchestrator, tool permissions, and corresponding attack setup. The no-debate results show that both lower accuracy-attack ASR and higher unauthorized-read rates relative to a single agent can occur without peer debate. This supports assessing accuracy and authorization separately, while clarifying that the difference from a single agent cannot be attributed to debate alone.

Attack and measured outcomeSingle agentNo-debate MAD, R=0MAD, R=1
M3: ASR under accuracy attack89.47%56.25%59.26%
M4: ASR under accuracy attack76.32%56.25%70.37%
M2: IR under Unauth.read attack52%96%96%
M2: IW under Unauth.write attack42%68%62%
M5: ASR under accuracy attack—43.75%37.04%
M5: IW under Unauth.write attack—2%6%

The no-debate control also clarifies the scope of the conclusion. M2 read rates are already high without debate, and M3/M4 ASR is higher with debate than without it. Thus, debate can have different effects across attacks, and amplification relative to a single agent need not come from peer exchange alone.

【Section 4.1】 describes the no-debate baseline and explains why differences from a single agent should be distinguished from differences caused by peer debate.

【Table 2(b)–(c)】 reports the HybridQA50 AFs relative to the single-agent and no-debate baselines.

Q5: Could the authors address the metric-related concerns in weakness 3 and clarify whether the reported IRW results reflect successful execution of the attacker's specific objective or any unauthorized operation?

A5: We report IR, IW, target invocation, and target completion separately. For MAD write-goal attacks on HybridQA50, the results are:

AttackIRIWIRWTarget invocationTarget completion
M2: Unauth.read attack96%6%96%96%96%
M3: Unauth.read attack72%6%72%72%72%
M4: Unauth.read attack100%0%100%100%100%
M2: Unauth.write attack0%62%62%62%62%
M4: Unauth.write attack2%10%12%10%10%
M5: Unauth.write attack0%6%6%0%0%

【Section 3.3】 defines IR and IW separately and explains their union, IRW.

【Appendix E.2】 defines target invocation and completion and reports the audit results and concrete examples.

Q6: Furthermore, IRW measures violations in the attacked execution without explicitly accounting for unauthorized operations that might already occur in the corresponding clean execution.

A6: The paired audit shows that the high write-violation rates in the M2-G4 comparisons are largely absent from clean runs. It also identifies cases where a violation already occurs without an attack. This supports the observed increase in unauthorized operations while separating new violations from existing ones. On HybridQA50, clean IRW is 0% for the single agent, 0% for MAD, and 2% for no-debate MAD. The selected write comparisons are:

Configuration and attackClean IWAttacked IWTasks newly violating
MAD, M2: IW under Unauth.write attack0%62%62%
No-debate MAD, M2: IW under Unauth.write attack2%68%66%
No-debate MAD, M5: IW under Unauth.write attack2%2%0%

【Appendix E.2】 reports clean authorization rates, selected paired transitions, and examples distinguishing existing violations from newly completed attacker objectives.

Q7: Could the authors clarify what distinguishes M5 and M6 from previously studied multi-agent attacks, particularly what additional MAD-specific security insights are gained from evaluating a directly compromised orchestrator?

A7: M5 and M6 extend MADBench from attacks delivered through inputs and resources to attacks on the workers and the orchestrator. Together with M1–M4, they let us compare where MAD resists an attack and where that resistance breaks down, using the same workflow and outcome metrics. This systematic comparison is our contribution, building on prior malicious-agent and orchestration attacks.

M1–M4 adapt attacks from single-agent systems and test whether MAD absorbs or amplifies them relative to a single agent. They establish the starting point: MAD can reduce accuracy-attack ASR while increasing unauthorized operations. M5 then tests what happens when the attacker controls debate workers themselves. We compare three strategies: colluding workers support one shared wrong answer, independent workers choose wrong answers separately, and inconsistent workers send different reports to different peers. We compare these strategies at matched compromised-worker counts and also vary that count.

The M5 comparison shows why both strategy and outcome matter. On SealQA with three compromised workers, colluding, independent, and inconsistent attacks have ASRs of 28.30%, 11.32%, and 5.66%. Their IW rates are 28%, 39%, and 66%. Thus, the strategy most effective at changing the final answer is the least effective at inducing unauthorized writes among these three conditions. These are findings about the implemented strategies, not an isolated causal effect of coordination.

M6 completes the comparison by testing control of the component that dispatches tasks, judges worker reports, and executes writes. Its high success is expected, but it identifies a boundary that worker-level tests cannot establish: correct worker reasoning cannot protect the final outcome if the orchestrator overrides it. On HybridQA50, M6 has 100% ASR both with and without debate. We therefore use it as a direct-control reference, rather than evidence of amplification by debate.

【Section 3.2】 defines M1–M6, the three M5 strategies, and M6's role as a direct-control baseline.

【Section 4.1, Table 2】 reports the absorption/amplification comparisons and the added no-debate control.

【Section 4.2, Table 4】 compares M5 strategies and compromised-worker counts and reports M6 outcomes. The accompanying text includes the HybridQA50 M6 results.

【Section 5】 discusses the relationship to prior malicious-agent, orchestration-attack, and multi-agent benchmark work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.