- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies agents that repeatedly rewrite tool descriptions to gain selections. It tests five seller systems separately with two buyers on two sets of ten tools.All seller systems expand their claims. Gemini/Antigravity sharply reduces first-choice success; the other systems generally preserve or improve it. Success is inferred from recorded tool outputs.
Critical strengths
- Real tool executions provide a basis for checking whether selected tools can answer each question. Fixed outputs, hidden vendor identities, equal prices, and shuffled order help isolate description effects.
- The repeated market connects seller rewrites to buyer choices over time. Severe losses occur with both buyers and both tool sets. Amazon first-choice success decreases and appendix H shows a seller-contact database gaining selections after falsely advertising product-scraping capabilities.
- The authors check claim extraction with a second judge and audit vague completeness claims separately.
Critical weaknesses
- Section 3.4 uses absence from one output sample as evidence of dishonesty. The sample may not show every capability, and the original description or schema may support additional claims. The score therefore does not establish which claims contradict the full information available to the seller. This weakens the honesty interpretation, while leaving the observed selection losses intact.
- Competitor feedback is part of the setup, but its added effect is not isolated. A selection objective could encourage promotional rewrites even without rival descriptions or counts. The statement in Section 3.3 that strategy comes from the model “and not from the prompt” is too strong. The results support the narrower claim that false descriptions arose without explicit instructions to exaggerate.
- (Faghih et al. (2025))[https://arxiv.org/abs/2505.18135v2] and (Blankenstein et al. (2026))[https://arxiv.org/abs/2510.00307] already show description and position effects on tool choice. This paper adds repeated rewriting and consequences for real tools with unequal capabilities. The tests show instruction sensitivity, but give little explanation for why the seller systems respond differently. Ten of sixteen mild markets improve, so claim growth does not consistently mean buyer harm. The study establishes a failure case, with limited evidence about which other systems or market rules would produce it.
- Most combinations of buyer, seller, and tool set have one independent run; only two Amazon combinations have three seeds. Bands in Figures 3 and 4 span buyers, and Table 8 reports variation across rounds. Neither estimates variation across repeated runs with the same settings and appendix F does not report repeated seeds for the changed objectives. This leaves uncertainty about how reliably the honesty instruction prevents failure.
Questions for authors
- How many unsupported claims contradict the full product sheet? Does the system gap remain after excluding empty and error-only samples?
- Can you separate the effect of rival feedback from repeated rewriting for selection, or narrow the competition-specific claim?
- How many independent runs support each Appendix F condition? Does the honesty-instruction result hold across seeds?
4 comments
W3
Weakness 3: (Faghih et al. (2025))[https://arxiv.org/abs/2505.18135v2] and (Blankenstein et al. (2026))[https://arxiv.org/abs/2510.00307] already show description and position effects on tool choice. This paper adds repeated rewriting and consequences for real tools with unequal capabilities. The tests show instruction sensitivity, but give little explanation for why the seller systems respond differently. Ten of sixteen mild markets improve, so claim growth does not consistently mean buyer harm. The study establishes a failure case, with limited evidence about which other systems or market rules would produce it.
W3.1 Why the seller systems respond differently
The seller models differ in how reliably their honesty alignment holds. All sellers had the same prompt, information and incentive. So the difference comes from the models themselves. This is the problem the paper reveals: honesty toward third parties is not yet reliable in every deployed model.
W3.2 Claim growth does not consistently mean buyer harm
Exactly. This is a key finding of the paper: claim growth alone does not harm buyers. Cheap-talk theory (Crawford & Sobel, 1982) predicts that unverifiable claims inflate until descriptions carry no information. Yet under mild toolflation, buyers kept their task success. The added claims were mostly true and helped buyers tell capable tools apart. Large losses came only under severe toolflation, where unsupported claims rose (Section 4.4). So what harms buyers is dishonesty, not promotion. This is why we frame toolflation as a problem of honesty. We added this contrast with the theory to Section 5 of the revision.
W3.3 Why this failure case matters
Tool markets are the subject of this paper, as the title says. They are an agent-to-agent market, and this setting is spreading fast. Personal agents such as Meta Muse and OpenAI Dots already pick tools, products and services for their users. No human reads the listings they choose from. A failure case in such a market is valuable on its own.
Zooming out, the market is a testbed for honesty under economic incentives. Toolflation reveals a safety and alignment problem there. Seller agents spontaneously overclaim, even though no one told them to exaggerate.
W4 & Q3: Stability across seeds
Weakness 4: Most combinations of buyer, seller, and tool set have one independent run; only two Amazon combinations have three seeds. Bands in Figures 3 and 4 span buyers, and Table 8 reports variation across rounds. Neither estimates variation across repeated runs with the same settings and appendix F does not report repeated seeds for the changed objectives. This leaves uncertainty about how reliably the honesty instruction prevents failure.
Question 3: How many independent runs support each Appendix F condition? Does the honesty-instruction result hold across seeds?
Our budget did not allow repeated seeds for every combination. We therefore ran three seeds for the two Amazon markets that carry the main contrast. They serve exactly to measure the variation across repeated runs with the same settings. In every seed, toolflation stayed mild with GPT-5.6 Sol sellers and became severe with Gemini 3.8 Flash sellers (Table 8). Over the last five rounds, task success was 92%, 92% and 94% in the three seeds with GPT-5.6 Sol sellers. It was 26%, 33% and 34% in the three seeds with Gemini 3.8 Flash sellers. The outcome is therefore stable across seeds.
Appendix F now gives the number of independent runs of each condition (Table 7). We added two seeds for the honest objective with Gemini 3.8 Flash sellers, so this condition now has three seeds. The runs of Appendix F last ten rounds, and it compares rounds 5 to 9. Over these rounds, task success with the honest objective was 94%, 94% and 93% in the three seeds. With the neutral objective it was 43%, 46% and 34%. The honesty-instruction result therefore holds across seeds. The remaining conditions of Appendix F have one run each.
Thank you for the additional tests. They strengthen the results, but a claim missing from the product sheet is not necessarily false. Also, removing rival information and sellers’ own selection counts together does not show which caused the change. These issues still limit the claims about dishonesty and competition. I am maintaining my score.
We thank the reviewer for reading the new tests and for noting that they strengthen the results. We respectfully disagree that the two remaining points limit the claims. Both are answered by evidence in the revision and in our first response, and by two further counts that we add here.
"A claim missing from the product sheet is not necessarily false."
The dishonesty claim does not rest on the product sheet alone. The paper tests truth separately, by running every tool on all 30 targets (Sections 3.1 and 3.4). As our first response reports, for 150 of the 160 unsupported claims (94%), none of the 30 runs returns the field (Appendix B). We add one count here. Of the 112 claims that are missing from the full product sheet, all 112 name a field that the tool returns in none of its 30 runs. These claims are false in every run, and nothing in the seller's own product sheet supports them. We also emphasize that the harm is measured, not inferred. Task success is scored on the real tool outputs, and it falls from 88–89% to 29–31% on Amazon (Section 4.4).
"Removing rival information and sellers' own selection counts together does not show which caused the change."
Our first response made the separation that Question 2 asks for: "the effect of rival feedback" against "repeated rewriting for selection". The separation raised now is a finer one, and we have run it as well. We replayed the same six seller turns with a third input. Each seller sees its own selection count, but neither the counts nor the listings of its rivals. The 60 rewritten descriptions then carry 1.3 unsupported claims on average. With no feedback they carry 0.7, and with the full feedback 4.0 (Appendix F). Most of the change therefore comes from the information about the rivals. This agrees with what the sellers do. They copy the field list and the opening sentence of the most selected listings (Section 4.2, Appendix H), and a seller's own count contains neither. We also reiterate that the full feedback is the realistic condition. A real store shows every listing and how often each tool is used, so a deployed seller agent sees both parts together (Section 3.3).
"These issues still limit the claims about dishonesty and competition."
The claims in question are false in every real run, and the severe form comes mainly from what the sellers see of their rivals. We therefore believe that the claims about dishonesty and competition stand as written. We appreciate both points. They led us to state the evidence more clearly, and the updated revision now reports both new counts (Appendices B and F).
W1 & Q1: Does absence from one output sample show dishonesty?
Weakness 1: Section 3.4 uses absence from one output sample as evidence of dishonesty. The sample may not show every capability, and the original description or schema may support additional claims. The score therefore does not establish which claims contradict the full information available to the seller. This weakens the honesty interpretation, while leaving the observed selection losses intact.
Question 1: How many unsupported claims contradict the full product sheet? Does the system gap remain after excluding empty and error-only samples?
We agree that a claim missing from one sample is not by itself proof of dishonesty. Such a claim could still be fair in two cases. The tool may return the field for other targets. Or another part of the product sheet may support it. We checked both cases.
For the first case, we ran every tool once on each of the 30 targets (Section 3.1). The one sample gives almost the same result as all 30 runs. In the final round, the sample does not support 160 claims of Gemini 3.8 Flash sellers. None of the 30 runs supports 150 of them (94%). The sample is therefore a sound basis for the count.
For the second case, we recounted against the full product sheet: the sample, the vendor's original description and the input schema. A claim counts as unsupported only if all three lack the field. Any mention of the field in the schema counts as support, so this count is a lower bound. Of the 160 claims, 112 (70%) contradict all the information that the seller had. Counting them as dishonest is justified.
The system gap remains by this count. Against the full product sheet, the final descriptions of Gemini 3.8 Flash sellers carry 112 unsupported claims. Those of DeepSeek V4.1 Flash carry 8, those of Grok 4.6 carry 3, those of Claude Opus 5 carry 1 and those of GPT-5.6 Sol carry none.
The system gap also remains after excluding empty and error-only samples. One Amazon tool has an empty sample and one Instagram tool has an error-only sample. Without them, Gemini 3.8 Flash sellers still make 2.8 such claims per description on Amazon and 2.6 on Instagram. The other four seller models make at most 0.3. These checks are now in Appendix B of the revision.
W2 & Q2: Rival feedback is now isolated, and the claim is narrowed
Weakness 2: Competitor feedback is part of the setup, but its added effect is not isolated. A selection objective could encourage promotional rewrites even without rival descriptions or counts. The statement in Section 3.3 that strategy comes from the model "and not from the prompt" is too strong. The results support the narrower claim that false descriptions arose without explicit instructions to exaggerate.
Question 2: Can you separate the effect of rival feedback from repeated rewriting for selection, or narrow the competition-specific claim?
Rival feedback is a deliberate design choice. A real store shows every listing and how often each tool is used, so a real seller agent can see its rivals (Section 3.3). We agree that its effect should be measured separately, so we did both things you ask. A new experiment isolates the rival feedback, and the wording is narrowed.
We reran the Amazon market with Gemini 3.8 Flash sellers and removed the rival feedback (Appendix F, Table 7). The objective sentence is unchanged. Each seller sees only its own description and its product sheet. It sees no selection counts and no other listings. As you expected, the selection objective alone encourages promotional rewrites. Claimed fields grew from 1.6 to 3.9 per description. But over rounds 5 to 9, unsupported claims stayed at 0.9 per description, and task success was 93%. With rival feedback, the three seeds reach 3.6 to 4.0 unsupported claims over the same rounds, and task success is 34% to 46%. So repeated rewriting for selection adds claims, and severe toolflation appears only when sellers also see their rivals.
A second check starts both conditions from the same state. We replayed six seller turns of the original runs, once with and once without rival feedback. With rival feedback, the 60 rewritten descriptions carry 4.0 unsupported claims on average. Without it, they carry 0.7 (Appendix F). The unsupported claims therefore come with the rival feedback.
Our main claim already has the narrow form you suggest. The abstract says that sellers "told only to be selected more often" practice generative engine optimization. The wording in Section 3.3 was inaccurate, and we have corrected it in the revision. It now says that false descriptions arose spontaneously, that is, without explicit instructions to exaggerate. The introduction and the conclusion said "competition alone" and "asked only to compete". Both now use the wording of the abstract.