- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies whether static source centrality reliably predicts misinformation risk in finite-horizon LLM multi-agent systems. It proposes CORA-GG, which combines position-independent verification quotas, assertion-level risk prioritization, semantic lineage tracking, and verification-gated downstream influence.
Critical strengths
- The paper addresses a meaningful problem: structurally peripheral agents can introduce consequential misinformation and may receive insufficient verification under centrality-based allocation.
- The controlled interventions on verification allocation are useful for examining how allocation bias affects residual risk. The additional stress tests also explore relevant limitations, including evidence poisoning and constrained verification budgets.
Critical weaknesses
-
The reported ASR and accuracy values appear inconsistent with the stated evaluation definition. Appendix C.1.2 defines a targeted attack success as a terminal task failure attributable to the injected misinformation. Under the same evaluation population and denominator, targeted ASR therefore cannot exceed the task error rate. However, Table 20 reports 93.3% ASR alongside 86.7% attacked-task accuracy for undefended MMLU under Hub attacks, and 96.7% ASR alongside 93.3% accuracy under Nonhub attacks. Table 1 similarly reports 82.2% mixed ASR and 32.8% attacked-task accuracy for the undefended system. These discrepancies require clarification of the denominators, evaluation populations, or metric implementation. As presented, they undermine confidence in the quantitative conclusions.
-
The model coverage is limited. The main comparisons use Qwen2.5-7B-Instruct for both agents and verification. Although Table 24 includes Llama-3.1-8B and a Mistral-7B verifier, these are limited transfer checks for CORA-GG without corresponding baseline comparisons. The evaluated models also remain within the 7–8B scale. Consequently, the experiments do not establish whether the relative advantages persist across substantially different agent and verifier capabilities.
-
The evaluation lacks direct comparison with closely related misinformation-specific defenses. In particular, ARGUS (Goal-aware identification and rectification of misinformation in multi-agent systems, ICLR 2026) is neither cited nor evaluated. ARGUS directly addresses misinformation localization and correction, combining topology with semantic relevance and dynamic feedback. Comparing against a static Hub-preferential policy does not adequately represent this alternative. A direct comparison under matched attack and resource settings would help establish represent CORA-GG’s advantage over existing misinformation defenses.
-
The experimental presentation relies too heavily on tables and provides limited visibility into the underlying mechanisms. Although Figure 1 illustrates the architecture, the experimental findings are presented almost entirely as tabulated aggregates.
If authors could address these concerns, I will consider raising my grade.
Questions for authors
See weaknesses
5 comments
Thanks for the response and the new experiments. The revision addresses several of my concerns. I have a few follow-up questions about the reported results and the main insight.
-
The original metric mismatch is fixed, but a new one appears in Table 17. The corrected ASR and accuracy values resolve the contradiction I raised. However, page 27 contains another mismatch. The text says ARGUS has 66.4% clean accuracy and leaves its SCR blank because the supplied 68.6% is inconsistent. Table 17 instead reports 68.4% clean accuracy and 64.2% SCR. The paragraph below again uses 66.4%. Which values are correct? Is the text outdated, or does the table need correction? Please clarify where the final values come from.
-
The ARGUS comparison helps, but what does it tell us about position-decoupled allocation? The new ARGUS results largely address my missing-baseline concern. I also appreciate the clearer discussion of its adaptive localization and goal-aware rectification. My remaining question is about the main insight. ARGUS already uses semantic information and dynamic feedback, so it does not simply assume that central sources deserve more protection. The full-system comparison shows that CORA-GG performs better in the reported setting. But CORA-GG also changes verification, lineage handling, and downstream influence. What evidence connects this advantage over ARGUS specifically to position-decoupled allocation? The routing controls support an advantage over the tested allocation rules. How far do the authors think that conclusion extends to adaptive, content-aware allocation?
-
The link between the allocation theory and the implemented gate is still unclear to me. The revision clarifies that Eq. (3.5) is a simplified allocation-level model. It also states that unverified assertions have zero direct factual weight. Under this rule, checking fewer assertions does not by itself explain why more false information reaches the final answer. Appendix G.6 lists possible explanations, but does not identify which occurred in the reported failures. What do the existing failed runs actually show? Did the verifier approve a false assertion, did the lineage tracking fail, or did the attack act through another stage? This matters for the paper’s main explanation. Does the allocation model capture the failure mechanism of CORA-GG, or does it mainly motivate the allocation rule?
The new comparisons and visualization are helpful. My remaining concerns are the unresolved reporting mismatch and the connection between the allocation insight and the observed gains. Clarifying these points would help me update my assessment.
Dear Reviewer,
Thank you for the follow-up. We answer each point directly.
1. Which ARGUS clean-task values are correct?
Table 17 is correct: ARGUS Clean Accuracy is 68.4% and SCR is 64.2%. The original default-configuration clean-execution logs confirm these values; 66.4% Clean Accuracy, 68.6% SCR, and the blank-SCR explanation came from outdated prose. We apologize and have corrected Appendix E (tab:clean_utility). Both metrics use the same clean-task executions and denominator: Clean Acc = correct / N_clean; SCR = (safe AND correct) / N_clean. The corrected CORA-GG vs. ARGUS comparison is 72.8% vs. 68.4% Clean Acc and 70.9% vs. 64.2% SCR.
2. Does the ARGUS advantage specifically come from position decoupling?
The ARGUS comparison alone cannot establish that. ARGUS uses adaptive localization and goal-aware rectification, while CORA-GG also changes verification, lineage handling, and gating. Hence ARGUS's 5.4% mixed/8.6% worst-source ASR versus CORA-GG's 0.0% observed ASR (tab:main_comparison) is a whole-system comparison, not an allocation-only causal estimate.
The allocation-specific evidence is the controlled routing intervention, with other CORA-GG components fixed (Section 4.4; Appendix D): Hub-preferential, Uniform, global Risk-only, and Position-Decoupled routing yield 6.7%, 6.0%, 4.8%, and 0.0% mixed ASR, respectively (tab:routing_control). Risk-only is content-aware but lacks per-source base quotas. Changing only the centrality preference shifts Hub/Nonhub ASR from 4.5%/0.1% (γ=−2) to 0.0%/2.2% (γ=+2), versus 0.0%/0.0% (γ=0) (tab:gamma_dose). Risk shifts to the deprioritized source. Without genealogy or gating, the minimal router-plus-verifier control reduces mixed ASR 81.9%→17.4% (tab:minimal_system); routing × hard-gating controls show 6.7%→0.0% with gating on and 13.4%→0.6% with gating off (tab:routing_gate_factorial).
Scope: These interventions isolate a routing benefit among tested policies, not the full ARGUS gap or universal superiority. A calibrated adaptive allocator might perform as well or better. Our claim: without calibrated source hazards, static position should not set base verification quotas; content risk still ranks assertions within quotas. Theorem 2 applies only to this uncalibrated routing-level surrogate.
3. What actually caused the failures, and what does Eq. (3.5) explain?
Fewer checks do not directly let YELLOW/RED assertions through the GREEN-only gate. Appendix G.6 now reports the audit of the 60 successful executions under the 180-call, 8× assertion-volume condition (tab:capacity_failure_attribution): 25 Hub + 35 Nonhub successes out of 1,000 executions each, or 3.0% mixed ASR.
56/60 (93.3%) were primarily capacity-related cross-stage failures: verifier-budget saturation delayed checks of attack-relevant assertions, and attack-affected intermediate workflow states influenced later processing and terminal decisions. This is indirect cross-stage influence, not direct YELLOW/RED admission at the fixed-stage gate. Not every subsequent step was localized. 4/60 (6.7%) were false verifier approvals: false assertions received SUPPORTED verdicts and became GREEN, making them eligible for factual aggregation. These are verifier errors, not promotion-rule bypasses. No separate genealogy error was identified as a primary failure category; secondary contributors are not excluded.
The capacity intervention supports this diagnosis: at 180 calls, increasing volume 1×→8× decreased verified share 30.1%→4.2% and increased mixed ASR 0.0%→3.0%; at 360 calls/8×, mixed ASR fell to 0.5% (tab:global_budget_full). These trends are not per-run counterfactual proofs. Proposition A5 guarantees only stage-local zero direct factual weight for YELLOW/RED, not cross-stage non-influence or verifier correctness.
Eq. (3.5), q_i^D=q_i^0(1−ν_iη), mainly motivates the allocation rule. It abstracts coverage-dependent residual hazard; Theory motivates allocation; routing controls isolate its tested effect; Appendix G.6 identifies observed failure pathways.
Thank you.
Response to Reviewer xxoJ: Corrected Metrics, Cross-Backbone Comparisons, Resource-Matched ARGUS Evaluation, and Assertion-Genealogy Visualization
Dear Reviewer xxol,
Thank you for your review. We corrected the reporting error and added cross-backbone, ARGUS, and genealogy evidence.
1. ASR–Accuracy Consistency
We sincerely apologize for the accuracy-reporting error and concerns about evaluation reliability. Its cause was an incorrect field reference: accuracy used a pre-attack prediction, not the attacked terminal answer. ASR already used attacked terminal outcomes and semantic attribution, so ASR remains unchanged. We recalculated accuracy from terminal answers using task-specific criteria.
Appendix C.1.2 (
app:targeted_attribution) defines correctness Ce, attribution Ae, and Se=(1−Ce)Ae. Both metrics use identical attacked outputs and denominators: ASR=∑Se/∣E∣, AttackAcc=∑Ce/∣E∣, so ASR + AttackAcc ≤ 1. All runs count. Appendix C.1.3–C.1.4 (app:run_accounting,app:statistics): 3,000 runs per task-family/position (200 tasks × 3 frameworks × 5 repeats), 9,000/position, 18,000/method; mixed ASR is a 1:1 Hub/Nonhub aggregate, not another sample; 600 task IDs are inference clusters.Appendix F.1.2 (
tab:task_subgroups) corrects undefended MMLU Hub ASR/accuracy: 93.3%/6.0%; Nonhub: 96.7%/2.9% (sums 99.3%/99.6%). Section 4.2 (tab:main_comparison) corrects pooled undefended accuracy to 4.25% (82.2% mixed ASR), Reflection to 20.6%; Appendix D (tab:minimal_system) corrects Minimal Hub-preferential/Uniform to 13.2%/28.5%. Pooled accuracy: (10.0+3.3+3.3+0.0+6.0+2.9)/6=4.25%. Zero observed ASR is not zero risk: the one-sided exact 95% bound is 0.50% on task-level any-failure probability over the 600-task grid, not per-run risk (Section 4.2; Appendix F.2.2,tab:zero_bounds).2. Cross-Backbone Baseline Comparison
Standalone transfer tests were insufficient. Sections 4.1/4.5 and Appendix G.3 (tab:paired_backbone) compare None, Balanced, G-Safeguard, ARGUS, CORA-GG on Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, DeepSeek-R1-Distill-Qwen-1.5B, and DeepSeek-R1-Distill-Llama-8B agent backbones. Each of the three newly run backbones uses matched tasks, attacks and seeds across methods (200 tasks × 2 positions × 5 repeats = 2,000 runs/method; 30,000 new runs total). Qwen2.5-7B reuses Exp-M aggregates (600 IDs; 18,000 runs/method), not an independent sample.
ARGUS→CORA-GG mixed ASR: Qwen-3B 7.9%→0.8%; Qwen-7B 5.4%→0.0%; DeepSeek-Distill-Qwen-1.5B 9.3%→1.5%; DeepSeek-Distill-Llama-8B 6.1%→0.6%. CORA-GG worst-source ASRs: 1.1%, 0.0%, 2.1%, 0.9%, respectively—lowest among the five methods throughout.
3. Direct and Resource-Matched ARGUS
ARGUS adaptively localizes propagation channels and rectifies goals; it is not static Hub prioritization (Sections 2, 4.2; Appendix E.1). Exp-M ARGUS Hub/Nonhub/mixed/worst ASR: 2.2%/8.6%/5.4%/8.6%, versus CORA-GG 0.0% observed. CORA-GG vs. ARGUS: attacked accuracy 41.3%/40.1%, FPR 2.3%/2.6%, tokens 29.8k/46.5k, latency 171.6s/270.4s.
Appendix E.1 (
tab:constrained_summary,tab:constrained_asr,tab:constrained_aux) validation-tunes/fixes ARGUS operating points on a held-out slice (200 clusters; 2,000 runs/method/constraint). ARGUS mixed/worst ASR: iso-token 10.0%/14.5% (29.9k vs. 29.8k tokens); iso-FPR 5.9%/9.2% (2.3%); iso-clean-utility 4.9%/7.8% (72.8%). CORA-GG remains 0.0% observed. ARGUS leads baseline mixed ASR under all three; G-Safeguard beats ARGUS on iso-token worst (13.5% vs. 14.5%).4. Log-Verified Assertion Genealogy
Section 4.4/Appendix D.3 (
app:assertion_lineage_case,fig:assertion_lineage_trace) add a log-verified observed Nonhub trace. Retriever injects false 48×25=1400, inherited by Solver as L-A0. Balanced adopts it without a timely verdict; CORA-GG returnsCONTRADICTED, makes L-A0 RED with zero factual weight, and aggregates independently supported 48×25=1200 through GREEN L-E1. Assertion identities, lineage edges, verdicts, and terminal outcomes were checked against paired execution logs.Controlled dose–response (Section 4.4,
tab:gamma_dose): γ=−2: 4.5%/0.1%; γ=+2: 0.0%/2.2%; γ=0: 0.0%/0.0%, Hub/Nonhub ASR, with equal verification shares at zero.Conclusion
These changes improve metric consistency, cross-backbone and resource-matched comparisons, and log-grounded interpretation. We appreciate your reconsideration.