acceptodds
Under review as a conference paper at ICLR 2027

Quote or Invent? Fabricated Evidence in LLM-Written Peer Reviews

Abstract

Estimates indicate that language models now draft a visible share of peer reviews. Reviews can back their comments with quotations from the paper, i.e., words in quotation marks. The other reviewers and the authors trust a quotation in a review to provide the paper’s own words. When quotations turn out to be reworded or invented, that trust is lost, and with it also the trust in the quality of peer review. We measure how often this happens. Fourteen models from Anthropic, OpenAI, Google, DeepSeek, Moonshot and Meta review 100 accepted ICLR 2026 papers under a prompt that requires a verbatim quotation for every strength and weakness. In one condition the model receives the paper. In the other, the same paper with numbered sentences, and every quotation must cite the sentence it comes from. A deterministic checker locates every quotation in the text the model read, and three LLM judges from vendors other than the reviewing model’s grade every quotation the checker cannot confirm. 30% of the 2,759 usable reviews contain at least one quotation that is not the paper’s wording, but the models are far apart. With the paper alone, the rate runs from 2 and 3% for Gemini 3.1 Pro and GPT-6 Astra to 70% for gpt-oss-120b, and the share of reviews with a quotation whose meaning differs from the paper from 0 to 16 %. Numbering the sentences and asking for the number lowers the rate by 3.7 points. The quote-back check of a deployed review assistant would pass 73% of the invented quotations. The fuzzy matcher of earlier work accepts every changed number, negation and changed digit.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large language models, peer review, LLM-generated reviews, hallucination, quotation fidelity, faithfulness evaluation, LLM-as-a-judge

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_EuPCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper propose a study on how much commercial and open-source LLMs hallucinates findings when reviewing papers, distorting the real claims presented in the original manuscript with minimal perturbations (i.e., the generated text does not coincides with the original one when quoted). The study draws 100 papers from submissions to ICLR 2026 and instructs models to review papers and include quotes from them to justify the statements. These are then evaluated by commercial LLMs to understand whether the quotes are correct or not.

Critical strengths
  • The paper analyses the faults of models when reviewing papers, showing that the justification for a specific judgment is not aligned with real text present in the paper, since models report them into the reviews with mistakes;
  • the LLM-as-a-Judge validation uses an odd number of models to ensure that majority voting is fair;
Critical weaknesses
  1. The contribution of this paper is minimal, since it showing that models that generates tokens are not generating the same content received in input. While the motivation of the paper is interesting, it is not clear how this study extends the current state of knowledge of LLMs used as textual aggregators. These are known to hallucinate, and for this reason RAG are usually deployed to produce (mostly) factual sentences. Moreover, also human-written review can induce these errors, and I think that studying the amount of errors caused by reviewers in general would have been a much more interesting topic (which has been investigated).

  2. The paper, which has a super simple experimental design, is incredibly complicated to read. The narration should have been something like "LLMs are used as reviewers or automatic writers -> LLMs incorrectly quote portions of the paper -> we experiment with models". But currently, this abstract structure is lost in the first paragraphs where technical details are already presented. Moreover, I understand it is difficult to extract text from PDFs, but there is no clear definition on how this is done in the paper (or probably yes, but it is obfuscated inside the text). Experimental results are even more obscure: there are no figures supporting the methodology, there are no algorithms to show processes, there are three tables which are super complicated to understand (many concepts and manipulations are just quoted without specifying what they are stating). I do not advocate to add content for the sake of having figures or other visual representations, but I advocate for manuscript to be readable by humans (which is the whole point of this paper, that models are quoting erroneously manuscripts).

  3. Moreover, the results of this manuscript can not be replicated. There is no code, no configurations, no prompts: it is impossible to judge whether these results are sound or not.

Questions for authors
  1. Can you better clarify the contributions? From what I can see, this paper is just looking at quantifying the difference between a generated text and its original counterpart.
Reviewer_P1GVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Authors measure whether LLM-generated reviews contain reworded or invented quotations, which undermines trust in reviews. To check this, authors use fourteen frontier models to review 100 accepted iclr 2026 papers under a prompt that requires a verbatim quotation for every strength and weakness. The result shows that 30% of the usable reviews contain at least one distorted quotation, but it highly depends on model choice. Numbering sentences can lower this rate.

Critical strengths
  • The paper is generally well written and the study design and experimental setup clearly shows what authors did.
  • Authors tackle an interesting problem, checking if LLMs invent quotations or not, and use that as a proxy for reliability of LLM-generated reviews.
Critical weaknesses

The most critical downside of this paper is that the tackled problem is too narrow so that it is not interesting for ML community. Of course, inventing quotations is highly problematic and should be mitigated, but the experimental setup shows that the prompt enforces the model to write quotations for every weakness and strength, which is unrealistic. Human reviewers do not usually use quotation for every review items. Also, there are lots of scenarios that can reduce reliability of LLM-generated reviews, but the authors only consider this invented quotation problem. I suggest authors to look at broader scenarios or submit this work to another venue. I'm sure that this does not fit as an ICLR paper.

Questions for authors

There are RQs in Section 5, but they are not clearly described. I suggest to clarify this and/or put it in the first paragraph of the section or in the Introduction if authors want to emphasize this.

Reviewer_DDjwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work checks how often different LLM's reviews contain false quotations. Each LMM is instructed to include a quote from the paper in its review and the quote is checked. The results show wide gaps between the percentages of invented quotes for different models. When each sentence in the paper is numbered and the LLM is also to asked to provide the number of the sentence from which the quotation came from the results show that the percentage of fabricated quotations only drops by a small amount. The works tests 100 ICLR 2026 paper with 36,386 quotations, and 14 LLMs, and expectedly finds that lower tier models invent more than state of the art models. The work proposes that checking quotations should complement checking claims in evaluating reviews.

Critical strengths
  1. The study design is detailed and takes into account the required nuances in checking and evaluation.

  2. Comparing the LLM results with humans is worthwhile and provides insight into LLM and human errors.

  3. The study encompasses 14 different LLMs, and checks the effects to different prompts.

Critical weaknesses
  1. The results demonstrate that state of the art frontier models rarely fabricate quotes. It is unclear what effort level is used with Claude Code and GPT-6. I suspect that effort level ulracode with Claude Code and Ultra with Astra would make this work obsolete, by improving quote verification in the generated review to begin with, and by improving checking quality.

  2. The conclusion that people disagree with one another as often as with the judges is not novel.

Questions for authors

What effort level was used for Claude and GPT-6? Was effort ultracode and Ultra used? Do those effort levels make fabricated quotes obsolete?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.