acceptodds
Withdrawn from ICLR 2027

Search Apart, Err Together: Exploiting False Corroboration in Multi-Agent Web Search

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-agent Web retrieval systems increasingly rely on independent search paths for factual reliability. Yet independent retrieval does not guarantee independent evidence: a single factual origin may propagate across publishers, domains, and content forms, allowing agents to retrieve superficially distinct but provenance-dependent evidence. We term this failure mode cross-agent corroboration amplification. In this paper, we introduce SourceSplit, a Web-level source expansion mechanism that transforms a single factual origin into heterogeneous descendant pages while preserving their shared evidential basis. By altering only its Web exposure, SourceSplit increases its reach across independent agent retrieval paths without introducing additional factual support. We construct a controlled Web retrieval benchmark with 100 paired counterfactual cases. Across cases, SourceSplit increases the Coordinator's selection of the target-origin answer from 6.6% to 34.0%, showing that changing how the same origin is represented and exposed—without adding independent evidential support—can substantially influence the final multi-agent decision. To mitigate this effect, we develop a candidate selection strategy based on cross-agent evidential dependence, reducing the attack-induced gain from 27.4 to 7.2 percentage points. These results show that retrieval diversity does not imply evidence independence and highlight the need to account for provenance and evidential dependence in multi-agent Web retrieval.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multi-Agent Web Search, Evidence Provenance, False Corroboration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Fya4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The work explores a false-corroboration phenomenon that may arise in multi-agent web retrieval systems, where evidence appearing to come from different sources may trace back to the same underlying factual origin. Therefore, they give disproportionate support to the same claim. The paper highlights that this phenomenon can potentially be exploited by an adversary and proposes SOURCESPLIT, a method for constructing web evidence configurations that induce such false corroboration. Furthermore, the authors propose Cross-Agent Relay Quota as a defense mechanism and evaluate the effectiveness.

Critical strengths
  1. The paper includes a comprehensive analysis of the core phenomenon, including retrieval exposure, agent coverage, fixed-evidence control, agent scaling, and controlled cross-agent evidence allocation.

  2. The defense mechanism has low computational overhead. When detector results are cached, the selector incurs a small cost while increasing the defense success rate on the synthetic benchmark.

Critical weaknesses
  1. The defense's generality is unclear. The real-page replay in Appendix E shows increased exposure and citations, but no changes in final answers. The defense is not evaluated in this setting. The released detector uses keyword and text-position cues to identify independent acquisition, then groups all detected relays for a task together. This could merge reports based on different independent origins. Its strong ECHO-WEB results therefore leave open whether it captures evidence dependence or regularities of the synthetic documents. The limited description of data generation and validation makes this harder to assess, reducing confidence that the defense would preserve legitimate corroboration outside the synthetic benchmark.

  2. The contribution over closely related work requires further clarification. Polymorphic Sybil [1] poisoning and epistemic Sybil resistance [2] already address diverse apparent support and shared-evidence inflation. It would be helpful to include a matched-budget comparison with existing multi-document poisoning works. The defense comparisons also show limited advantage over stronger dependence-aware alternative methods.

  • [1] A Failure-Mode Benchmark for Polymorphic Sybil Poisoning in RAG (Lee et al, 2026)

  • [2] Epistemic Sybil Resistance: Multiplying AI Agents Without Multiplying Evidence (Bara, 2026)

Questions for authors
  1. How were the ECHO-WEB cases and documents generated and validated? Were the detector rules fixed before evaluation?

  2. How does relay grouping handle pages that report different independent sources but lack acquisition cues? Has its reliability been evaluated outside ECHO-WEB?

  3. What additional insight does SOURCESPLIT provide over polymorphic Sybil poisoning and epistemic Sybil resistance? Comparisons with relevant attacks and defenses under matched budgets, including a single-agent system with the same total retrieval budget, would help clarify this contribution.

Reviewer_nxPYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a multi-agent Web retrieval setup with three search agents and a coordinating model that aggregates their reports. The authors point out that independent search paths do not imply independent evidence, since a single upstream origin can be republished across many domains, and they build SOURCESPLIT, which turns one evidence record into six descendants that differ in framing (exact value, rounded value, margin, direction) and in surface identity (publisher, domain, genre, audience), while sharing the same origin and threshold relation.

Critical strengths
  1. The ablation study is convincing. First, replication alone buys nothing: six verbatim copies occupy 5.49 of the eight displayed slots yet reach only 5.6% target rate, versus 34.0% for the full construction. Second, surface identity is the operative axis. The variant with view diversity but no surface diversity has the highest retrieval exposure of any condition (5.96 slots, best rank 1.20) and the lowest outcome (3.8%), so retrieval quantity is clearly not the mechanism. Third, the fixed-evidence control removes retrieval entirely and still shows the effect (85.2% for one item, 90.4% for three copies, 97.4% for three reformulations). The cross-agent allocation study is the cleanest of the set: holding the system-wide evidence multiset fixed at three same-origin pages and varying only their distribution across agents gives 2.0% under (3, 0, 0) and 15.0% under (1, 1, 1), with a paired interval of [6.0, 20.0]. Together these isolate the phenomenon to the model's treatment of source identity diversity as evidence independence, which is a sharper claim than "repetition is persuasive".
Critical weaknesses
  1. Appendix A states that for every one of the 100 cases the target-origin answer disagrees with the ground truth. So the correct response is always to reject the target evidence, and any mechanism that suppresses repeated same-origin material is rewarded with no downside anywhere in the benchmark. That is exactly why the accuracy column in Table 2 moves in the right direction in both conditions (78.2% → 86.8% under C1, 52.6% → 75.8% under D6). Those numbers establish that the quota helps against a false relay; they say nothing about what it does to a true claim carried by several pages, or to a legitimate relay of correct information. The 8.0% misgrouping rate hints at the cost but cannot price it, because those competing pages are independent sources whose removal only ever hurts accuracy here — the benchmark cannot reward the defense for keeping a page. Add a mirrored case family in which the shared-origin evidence agrees with the ground truth, and report accuracy and target rate there. Without that cell, "reduces the attack gain from 27.4 to 7.2 points" is a one-sided measurement.

  2. The corpus gives the attacker far more room than the open Web, and the headline number is conditioned on it. Each case contains 19 documents, six of which are target descendants. In the D6 condition the target block is therefore 31.6% of the corpus and supplies 4.89 of the eight displayed slots per agent. Retrieval is closed-corpus BM25 (and BGE-M3 and reciprocal rank fusion in robustness checks) with no authority, recency, or spam signal, and the descendants compete with a ten-page background set rather than a long tail. The 27.4-point gain is best read as an upper bound in a permissive regime. Section 5.1 should state the 6-of-19 composition next to the headline number, and a dilution experiment would settle it: keep the six descendants fixed and grow the non-target corpus, or hold the target fraction constant while varying the count. The real-page replay points the same way — exposure and citation metrics move (mean agent coverage 1.45 → 2.09, coordinator citations 6/11 → 9/11) but every proposition was already answered YES under C1, so the decision effect on naturally collected pages is untested.

Questions for authors
  1. Does retrieval run over a per-case corpus of 19 documents, or over the pooled 1,900? Appendix A implies the former and Section 5.1 says "the closed Web corpus"; this changes how the exposure numbers should be read.

  2. Can you add the mirrored case family from Weakness 1 and report the defense's false-negative side?

  3. On the fixed-evidence control: a single false item shown in context already yields 85.2%, while a single page retrieved under C1 yields 6.6%. The gap presumably comes from whether the claim is opened at all, which Table 8 supports (F1 = 0%, F2 = 14.9%). Should the paper's framing be that the vulnerability is exposure and restatement rather than aggregator-level corroboration?

Reviewer_djkJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a problem in multi-agent search systems. Different agents may find different web pages that all trace back to the same original source. The system may then treat agreement between the agents as independent confirmation, even though they are all relying on the same underlying evidence.

The authors introduce an attack called SOURCESPLIT. It takes one source and turns it into several different-looking web pages. These pages can use different wording, numerical representations, publication styles, domains, and publisher identities, but they still support the same claim and come from the same origin.

The authors evaluate the attack using ECHO-WEB, a benchmark they created. They compare a setting where the target source appears on one web page with another setting where the same source appears on six different pages. They report that the multi-page setting makes the system more likely to choose the answer supported by the target source.

The authors also propose the Cross-Agent Relay Quota as a defense. It uses deterministic rules to identify pages that appear to repeat the same underlying evidence, then limits how many agents can receive those pages. The defense performs better than exact and semantic deduplication on ECHO-WEB. However, the strongest results come from the authors’ synthetic benchmark. The natural-web experiment provides weaker evidence and does not show the same final-answer changes.

Critical strengths
  1. The controlled comparison is well designed for the narrow question being studied. The paper compares one page with several different pages derived from the same source while keeping the underlying claim and competing evidence fixed. This makes it easier to show that the change in the system’s answer comes from the increased exposure and presentation of that source rather than from adding genuinely new evidence.

  2. The paper identifies a useful multi-agent failure mode. Independent agents do not necessarily produce independent evidence. Different agents can follow different search paths and still end up relying on the same original source. This is a relevant issue for systems that use agreement between agents as a sign of reliability.

Critical weaknesses
  1. The benchmark represents a much simpler problem than real adversarial amplification. The attack takes one simple factual comparison and presents it in different ways. For example, the same value may be shown exactly, rounded, expressed as a difference from a threshold, or described as being above or below that threshold.

Real information campaigns are more complex. They may selectively present facts, change the framing of an event, assign blame, promote different causal explanations, attack the credibility of other sources, or make different predictions about future effects. Reports may also share one source for the basic event while adding their own interviews, analysis, or contextual information.

The paper does not model these mixed forms of dependence. It shows that repeated versions of one simple claim can affect retrieval and decisions, but it does not show that its approach can handle realistic narrative amplification.

  1. The benchmark and defense are too closely matched. ECHO-WEB is created by the authors specifically around the attack they define. The repeated target source is wrong, better independent evidence is available, and the generated target pages have the same properties that the defense is designed to detect.

This makes the defense likely to perform well in the benchmark. Removing the repeated wrong pages allows the better evidence to take their place. However, the paper does not adequately test harder cases, such as:

the repeated source is correct; the repeated source is the most authoritative source; there is no alternative source; several independent sources use similar wording; or downstream reports add useful original information.

The paper also does not provide enough evaluation of how accurately its deterministic rules identify shared origins on natural web pages. Without precision, recall, and false-positive results on manually labelled real-world examples, it is difficult to know whether the defense is finding actual dependence or simply filtering pages that make similar claims.

Finally, the basic defensive idea is not especially surprising. Grouping related evidence and preventing repeated material from being counted several times is closely related to deduplication, provenance tracking, source clustering, and diversity-aware retrieval. The cross-agent application may still be useful, but the paper does not clearly establish how much it adds beyond these existing ideas.

Questions for authors
  1. How accurately does the Relay Quota identify shared origins on real web pages? Can the authors report precision, recall, and false-positive rates using naturally occurring pages with manually labelled source relationships? In particular, how often does the method incorrectly group independent reports or suppress reports that add original information?

  2. Does the attack and defense generalize beyond ECHO-WEB? Can the authors evaluate SOURCESPLIT and the Relay Quota on an external fact-verification, retrieval, or RAG-poisoning benchmark? This would help show that the findings are not specific to the benchmark created for this paper.

  3. Why suppress the pages instead of adjusting their evidence weight? Have the authors compared their defense with an approach that keeps all relevant pages available but counts same-origin pages as one unit of factual support? Such an approach might prevent false corroboration while preserving useful context or analysis from downstream reports.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.