- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper asks whether LLM bidders reproduce the ordering of auction formats observed in human experiments, even when their bids differ from human bids. It compares five models across seven auction settings using human benchmarks reconstructed from earlier studies. The main finding is that some models preserve several orderings despite differences in the size and direction of their deviations from theory.
Critical strengths
The paper addresses an interesting question: whether LLMs can help compare auction formats even when they do not accurately reproduce human bids. Establishing this would give researchers a useful, more limited alternative to treating LLMs as substitutes for human participants.
Critical weaknesses
The submission shows insufficient care in editing and proofreading. The main text appears to have been carelessly edited before submission, possibly due to AI assistance. For example, the OpenReview abstract contains strikethrough text. More seriously, the introduction claims that clock framing increases truthful bidding for Claude, while Table 4 shows that the share of bids close to the true value decreases. A lower average error does not mean more truthful bidding...
Important details are left to the appendices, and the main text does not clearly and consistently describe the measurements. In the common-value settings, it is unclear how the descriptions of errors in bids relate to the descriptions of errors in profits. Because the main claim depends on comparing deviations across auction formats, this affects how the results are interpreted.
These problems go beyond writing quality. They make it difficult to trust the connection between the measurements, results, and conclusions.
Questions for authors
How does the claim about increased truthful bidding for Claude agree with Table 4? What exactly is measured in each auction setting, and are those measurements comparable?
We decided to withdraw this submission. We thank the reviewers for their comments and will address them in a revised version of the paper. But we would like to raise two disagreements with the reviews .
Reviewer RvRo pointed to a strikethrough left in the OpenReview abstract as a sign of careless editing and speculated that the paper was written with AI assistance. The strikethrough was a proofreading mistake on our part. We do not think it should weigh in the evaluation of the paper, and the speculation about how the manuscript was prepared is unsupported.
The human benchmarks come from five published auction experiments: Kagel and Levin (1993) for first- and second-price sealed-bid auctions with independent private values; Li (2017) for sealed second-price and open ascending clock auctions with affiliated private values; Breitmoser and Schweighofer-Kodritsch (2022) for the ascending clock without dropout information; Kagel and Levin (1986) for first-price common-value auctions; and Levin, Kagel, and Richard (1996) for English common-value auctions. The appendix lists each source study and explains how we extracted the reported statistics and reconstructed the benchmarks from them. Questions about specific values, such as the mixture weight attributed to Kagel and Levin, are fair, and the revision will answer them precisely. These questions are about how we used existing evidence. The request for a new human experiment is a different question: it asks whether the findings hold in an auction design that has not yet been tested with human subjects. We would have liked the reviews to keep the two apart.