- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies what search agents do after reading fabricated promotional content about products. The authors build a closed Chinese-language web environment with about 72k real pages, 8 product categories and 154 made-up brands. For each made-up brand and false claim, they write poisoned pages at three levels: a bare advertisement, the claim embedded in a review, and supporting “evidence” spread across several pages that appear to back each other up. Search returns only titles, and the agent has to open a page to read it, so the authors can log which poisoned pages each agent read, whether it tried to verify the brand, and whether it recommended it in the end. They run 10 agents on 240 queries under a standard prompt and a cautionary prompt, and use 64 queries on real but little-known brands to measure false rejection. They find that recognition gets worse as the poisoning looks more corroborated, and that agents rarely go back to verify a fake brand once they have accepted it. Extra search rounds reduce endorsement without improving recognition, while explicit reasoning improves both. The cautionary prompt also raises false rejection of real brands from 11.4% to 18.0%.
Critical strengths
-
The design holds the brand and the false claim fixed and changes only how the claim is presented, so the three levels can be compared directly. Even with page type and length matched (App. A.10), L3 still raises the fooled rate by about 13 points over L1, which I find fairly convincing.
-
Page text is only visible after Scrape, so the authors can separate “never saw the poisoned page” from “saw it and wasn’t persuaded.” This is what lets them show that extra search rounds lower the fooled rate without making agents any better at spotting fake evidence (Sec. 5.3).
-
The control set of real, little-known brands shows what the defense costs: with the cautionary prompt, false rejection rises from 11.4% to 18.0%, and it rises for every model. Most work in this area reports only the drop in attack success.
Critical weaknesses
-
Whether an agent counts as “fooled” depends on whether the brand was made up, not on whether the evidence it saw was enough to tell. If an L3 fake brand looks as well supported as a real one, recommending it is a reasonable call. The real brands in the control were also chosen because they have several independent sources, so FR and FRR are measured on problems of very different difficulty. I would like to see a human baseline with the same tools, or real brands wrapped in the same L1–L3 promotional pages.
-
The fake brands have no trace at all in the clean corpus, and it is not clear whether the environment contains anything like business-registration or trademark records. The low verification yield may therefore say more about the environment than about the agents. The claim that agents rarely re-examine after adoption also rests on only 57 trajectories.
-
L3 changes several things at once, including cross-page corroboration and how official individual pages look. It is not yet clear that corroboration is what drives the effect the paper calls the “corroboration trap.” A simple test would be to put the same content on a single page and compare it with spreading the content across several pages.
Questions for authors
see weaknesses
Thank you for the thoughtful response and the additional experiments. However, my core concern remains: it is still unclear whether the fabricated evidence is distinguishable from genuine evidence in this setting, and thus whether the results reflect a real weakness of the agents. I will therefore keep my current score.