- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
To address the vulnerability in multi-agent systems where individual messages may appear benign but become dangerous when combined across sessions, the authors introduce ErosionBench. Focusing on enterprise business scenarios, this framework constructs test cases—incorporating either contaminated historical data/documents or standard, clean data—which are then processed by a multi-agent system consisting of four fixed roles. The success of an attack is determined by the actual tools invoked, while execution logs are used to evaluate the system's defensive detection capabilities and false-positive rates. Furthermore, the authors propose Meta-Auditor, an independent, authoritative auditing mechanism that combines rule-based checks with large language model analysis of execution logs to determine whether a multi-agent system has been manipulated into an attack or has deviated from its authorized scope. The authors conducted extensive experiments and provide valuable insights.
Critical strengths
-
The benchmark highlights a significant issue: in multi-agent scenarios, as a conversation progresses, a gradual divergence emerges between the user’s initial intent—as expressed through their authorization—and the final actions actually executed.
-
The benchmark covers 4 enterprise domains, 4 types of degradation mechanisms, and 1,728 scenarios, each with a paired benign control group.
-
Based on actual tool invocations and parameter validation violations, the benchmark provides clear evidence for successful attacks, ensuring that every state change can be traced back to a known trigger.
-
The benchmark provides an additional 198 image-based attacks, requiring defense mechanisms to be capable of withstanding the same attacks across any input channel—offering a more comprehensive evaluation.
Critical weaknesses
-
The figure is not formatted correctly. On the far right of Figure 1, under "verdict:allow/block," there are anomalous special characters. In the rectangular area at the bottom left of Figure 1, "Long-context" is misspelled as "Long-contest." Additionally, in the "State 1.1" section of Figure 1, the term "Flamed Value" appears; this is not a standard term and is likely a typo.
-
There is an inconsistency in the description: in section 3.1, the author defines four processes, yet in Figure 1, only three processes—stage 1.1 through stage 1.3—are shown under stage 1.
-
The authors focus on scenarios involving multi-agent collaboration and cross-session propagation; however, their primary conclusions are validated only within a fixed, four-role structure. In real-world enterprise business contexts, by contrast, topological structures are often diverse, with each structure requiring distinct approaches to scheduling and shared-memory mechanisms.
Questions for authors
-
The core subject of this paper is collaboration among multi-agent systems. Given that collaborative architectures, memory mechanisms, and authorization processes are all significant variables, how well can Meta-Audiotr adapt to such variations, and what is its level of transferability?
-
Did the authors take into account the model's "reasoning intensity" settings during the experiments—for instance, the "low," "medium," and "high" options available for GPT-5.1? This does not appear to be disclosed in the paper. If these settings were considered, what was the basis and rationale for their use?
-
What is the computational cost of a standard LLM judge in an evaluation task? And if such a judge were granted a second inference opportunity—akin to a "gray-area" arbitration process—could its performance become comparable to that of Meta-Auditor?
4 comments
(2/4) - Q1 (Transferability)
We varied each of the three factors mentioned by the reviewer, one at a time, with the Meta-Auditor frozen, on the same stratified 192-scenario slice (96 attacks / 96 benign controls) mentioned in the general comment:
- Architecture: the 3- and 5-role meshes above.
- Memory: agents see only the current shared state. The cross-session memory normally surfaced to them (prior approvals on file, value histories across sessions) is withheld. 64/96 attacks rely on this cross-session context.
- Authorization process: mentioned in paragraph below table.
| Configuration | Mesh ASR | Benign over-refusal (mesh) | Auditor recall | Auditor FPR | Undetected |
|---|---|---|---|---|---|
| 4-role, default memory, deliberative review (paper) | 44.8% | 42.7% | 97.9% | 2.1% | 2.1% |
| Architecture: 3-role | 100.0% | 0.0% | 97.9% | 1.0% | 2.1% |
| Architecture: 5-role | 29.2% | 43.8% | 99.0% | 5.2% | 1.0% |
| Memory: latest state only | 34.4% (p = .08)† | 41.7% | 99.0% | 2.1% | 1.0% |
Same 192-trajectory slice, paired exact McNemar vs. the paper configuration. †A second, independent mesh draw does not replicate the memory ASR drop (42.7% vs. 45.8%, p = .65).
What transfers. Across these configurations, recall (97.9–99.0%), FPR (1.0–5.2%) and the undetected-compromise rate (≤ 2.1%) all stay stable.
Authorization. We also replaced deliberative review with context-free change-ticket approval: the approver sees only the request and the proposed tool call. This cuts attack success from 44.8% to 16.7%, but only by rejecting 88.5% of legitimate work, the same over-blocking pattern as the perimeter defenses in our ladder. Auditor recall is unchanged (97.9%) and its ranking holds (AUC 0.98 vs. 0.998), but FPR rises to 19.8%, so the calibrated operating point is specific to the authorization process. We report this in the appendix and the revised Limitations.
Scope. All variants are sequential collaboration graphs over one shared ledger. Parallel/fan-out scheduling and fully private per-agent stores remain future work, and we state this in the revised Limitations. Together with transfer across four auditor models (auditor-panel table), three attacker-mesh models (mesh-sweep table), two externally authored corpora (AgentDojo, ASB) and an unseen fifth domain (healthcare: 100% recall, 0% FPR), this supports transfer of detection across the variables the reviewer raises, with the calibration caveat above.
(3/4) Q2 (Reasoning Effect)
Thank you for bringing this up, we can disclose this here: Every model ran at its provider-default reasoning effort.
We never set reasoning_effort. Our rationale is that the benchmark measures agents as deployed by
default, and that a shared default keeps the cross-auditor comparison from mixing per-vendor tuning
choices. The revised appendix adds the per-model settings:
| Model | Role | Reasoning effort | Temperature |
|---|---|---|---|
| gpt-5.5 | mesh agents (all ladder rungs) | provider default (≈ medium, see below) | not accepted |
| gpt-5 | data generation | provider default | not accepted |
| gpt-5.1 | cross-auditor | provider default (= no reasoning) | not accepted |
| gpt-4.1 | firewall LLM judge | n/a | 0.0 |
| claude-opus-4-8 | Meta-Auditor (headline), LLM-judge baseline | no extended thinking | not accepted |
| claude-sonnet-4-6 | cross-auditor | no extended thinking | 0.0 |
| gemini-3.5-flash | cross-auditor | provider default thinking | 0.0 |
Reasoning models receive a fixed 4,000-token completion headroom so hidden reasoning cannot starve the visible answer.
To test whether the setting matters, we swept it on both sides of the system (same 192 scenarios, 96 attacks and 96 benign controls; paired exact McNemar vs. the paper setting). We found that more reasoning makes the attacked agents harder to compromise, but never immune. On the defense side, the calibrated score is robust to it without re-fitting; the grey-zone arbiter is the one stage that is not. The tables below show the effect on the gpt-5.5 mesh and on the gpt-5.1 auditor.
(a) Attacked mesh (gpt-5.5, undefended):
| Mesh effort | Attack success [95% CI] | Δ vs. paper (p) | Benign over-refusal | Reasoning tok/call |
|---|---|---|---|---|
| none | 58.3 [48.3, 67.7] | +12.5 (p = .008) | 38.5% | 0 |
| low | 45.8 [36.2, 55.8] | +0.0 (p = 1.0) | 39.6% | 61 |
| paper default | 45.8 [36.2, 55.8] | — | 42.7% | 141 |
| medium | 42.7 [33.3, 52.7] | −3.1 (p = .65) | 40.6% | 144 |
| high | 35.4 [26.6, 45.4] | −10.4 (p = .053) | 42.7% | 262 |
Attack success falls monotonically with effort but stays at 35.4% even at high, and benign over-refusal is flat throughout. The paper's default uses the same reasoning per call as medium (141 vs. 144 tokens), so the reported numbers correspond to medium effort.
(b) Auditor (gpt-5.1). The calibration (score-to-probability map and decision threshold) was fit once at default effort and reused unchanged at medium and high:
| Auditor effort | Calibrated score + counterfactual (recall / FPR) | Full Meta-Auditor (recall / FPR) | Auditor tok/scenario |
|---|---|---|---|
| paper default | 88.5 / 6.2 | 80.2 / 0.0 | 6,065 |
| medium | 85.4 / 5.2 | 59.4 / 3.1 | 8,823 |
| high | 83.3 / 2.1 | 59.4 / 2.1 | 21,092 |
Without re-fitting, the calibrated score loses only 3–5 recall points across effort levels while its FPR falls. Both effects have the same cause: with more reasoning, the auditor scores every trajectory as less suspicious. Its mean raw compromise score falls from 0.70 to 0.48 on attacks and from 0.11 to 0.02 on benign controls. The threshold was fit on the default-effort scores, so this shift pushes a few borderline attacks below it (lower recall) and clears more benign runs (lower FPR). The ranking of attacks above benign runs stays largely intact (AUC 0.945 → 0.891), so the loss is mainly a shift relative to a frozen threshold, which is what per-configuration calibration is for. However, it is to be noted that the tier-3 grey-zone arbiter, a direct LLM verdict on borderline cases, is sensitive to effort (80.2 → 59.4% recall) ; the revision describes it as a precision stage to be validated per configuration. Both sweeps are added to the appendix.
Taken together, the sweeps support the paper's central claim: extra reasoning, whether in the attacked agents or in the auditor, does not substitute for a calibrated trajectory-level check.
(4/4) - Q3 (LLM Judge Cost)
We tested this directly. On the same 192-trajectory slice used above, the same-telemetry opus-4-8
judge was given:
(i) two independent samples (self-consistency)
(ii) a second pass using the Meta-Auditor's own grey-zone arbiter prompt. The arbiter's deterministic metrics were withheld from
it, since they are the component in question.
| Defense | Recall [95% CI] | FPR [95% CI] | Tokens / trajectory | Median latency (s) | Second pass on |
|---|---|---|---|---|---|
| LLM judge, single call | 100.0 [96.2, 100] | 66.7 [56.8, 75.3] | 3,429 | 2.7 | — |
| Judge, 2 samples (AND) | 100.0 [96.2, 100] | 65.6 [55.7, 74.4] | 6,880 | 5.5 | — |
| Judge, 2 samples (OR) | 100.0 [96.2, 100] | 75.0 [65.5, 82.6] | 6,880 | 5.5 | — |
| Judge + arbiter, grey zone only | 100.0 [96.2, 100] | 47.9 [38.2, 57.8] | 9,428 | 10.4 | 59% |
| Judge + arbiter, all trajectories | 100.0 [96.2, 100] | 47.9 [38.2, 57.8] | 11,266 | 11.9 | 100% |
| Meta-Auditor (same trajectories) | 97.9 [92.7, 99.4] | 2.1 [0.6, 7.3] | ~10,400 | 6.7 | 48% |
- Cost. A standard single-call judge costs about 3.4k tokens and 2.7 s per trajectory, roughly a third of the Meta-Auditor's tokens. It operates at a 66.7% false-positive rate, which is unusable in deployment.
- Second pass. Self-consistency alone gives no improvement. A second pass with our arbiter lowers the judge's FPR from 66.7% to 47.9%. Even at a higher budget than the Meta-Auditor (11.3k vs. 10.4k tokens; 11.9 s vs. 6.7 s), it remains about 23× worse on FPR on the same trajectories (47.9% vs. 2.1%).
- Where the gap comes from. Extra inference narrows the gap but does not close it. Our component ablation (General Response, row 2) locates the remainder in the calibrated trajectory-level intent-divergence score. Calibration alone moves the uncalibrated semantic monitor from 20.8% to 2.0% FPR. A second LLM call does not account for it.
We have added this row group to the judge comparison in the revised appendix.
(1/4) - Weaknesses (1, 2, 3)
We thank the reviewer for the close reading of Figure 1 and for pushing on transferability and on reasoning settings. Each point is addressed below.
Weakness 1 & 2. Figure 1 formatting and §3.1 consistency
Thank you for catching these inconsistencies in the figure. All four issues are fixed in the revised Figure 1 (see the updated PDF):
We also made the figure easier to read:
Weakness 3. Conclusions validated only on a fixed four-role structure
We agree this was a real limitation, so we tested it directly. We ran the frozen headline Meta-Auditor (claude-opus-4.8 same calibration artifact and threshold, no re-fitting) on two structurally different meshes:
Both runs use the same stratified slice of 192 trajectories (96 attacks / 96 matched benign controls) and are paired against the paper's 4-role run on the same trajectories.
Mesh ASR and auditor columns: L5 run on the slice. Benign over-refusal: a separate L2 mesh draw on the same ids. Paired exact McNemar vs. 4-role. Auditor recall/FPR differences vs. 4-role are not statistically detectable.
Why the 3-role mesh reaches 100%. The mesh follows separation of duties: one agent holds approval authority per scenario (the Policy reviewer in 93% of attacks, the intake Gateway in 7%). The other agents are the channel the attack travels through: shared state written in earlier sessions, the Orchestrator's arguments, and, in 12% of attacks, peers that attest to the poisoned state. Removing the approver therefore leaves a single point of failure. This is not single-agent behaviour: an isolated agent (L1) admits 0% of attacks. Adding a second approver who reads the same shared state still lets 29.2% through, so redundancy does not restore independent judgment. This is also where an external auditor matters most: with no reviewer in the mesh, the frozen Meta-Auditor still flags 97.9% of attacks at 1.0% FPR.
We have added this table to the appendix and state the 4-role mesh as a scoped design choice in §3. Further variation of memory and authorization is described in Q1 below.