- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper propose a study on how much commercial and open-source LLMs hallucinates findings when reviewing papers, distorting the real claims presented in the original manuscript with minimal perturbations (i.e., the generated text does not coincides with the original one when quoted). The study draws 100 papers from submissions to ICLR 2026 and instructs models to review papers and include quotes from them to justify the statements. These are then evaluated by commercial LLMs to understand whether the quotes are correct or not.
Critical strengths
- The paper analyses the faults of models when reviewing papers, showing that the justification for a specific judgment is not aligned with real text present in the paper, since models report them into the reviews with mistakes;
- the LLM-as-a-Judge validation uses an odd number of models to ensure that majority voting is fair;
Critical weaknesses
-
The contribution of this paper is minimal, since it showing that models that generates tokens are not generating the same content received in input. While the motivation of the paper is interesting, it is not clear how this study extends the current state of knowledge of LLMs used as textual aggregators. These are known to hallucinate, and for this reason RAG are usually deployed to produce (mostly) factual sentences. Moreover, also human-written review can induce these errors, and I think that studying the amount of errors caused by reviewers in general would have been a much more interesting topic (which has been investigated).
-
The paper, which has a super simple experimental design, is incredibly complicated to read. The narration should have been something like "LLMs are used as reviewers or automatic writers -> LLMs incorrectly quote portions of the paper -> we experiment with models". But currently, this abstract structure is lost in the first paragraphs where technical details are already presented. Moreover, I understand it is difficult to extract text from PDFs, but there is no clear definition on how this is done in the paper (or probably yes, but it is obfuscated inside the text). Experimental results are even more obscure: there are no figures supporting the methodology, there are no algorithms to show processes, there are three tables which are super complicated to understand (many concepts and manipulations are just quoted without specifying what they are stating). I do not advocate to add content for the sake of having figures or other visual representations, but I advocate for manuscript to be readable by humans (which is the whole point of this paper, that models are quoting erroneously manuscripts).
-
Moreover, the results of this manuscript can not be replicated. There is no code, no configurations, no prompts: it is impossible to judge whether these results are sound or not.
Questions for authors
- Can you better clarify the contributions? From what I can see, this paper is just looking at quantifying the difference between a generated text and its original counterpart.