- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ARD-Review to reduce generic, unsuited, nitpicky, and invalid criticism in AI-generated peer reviews. The pipeline first uses a harsh critic to generate candidate comments, then applies a merger to verify and filter unreasonable feedback, and finally retrieves reference papers with human reviews and scores to calibrate the target paper’s score. The authors also introduce ARD-Bench, consisting of 393 ICLR 2026 papers, to evaluate scoring accuracy, review validity, and cross-paper review overlap. The results show improved agreement with human scores and a high win rate in review-validity comparisons conducted by an LLM judge. Experiments with different backbone models and ablations provide additional evidence for the pipeline’s effectiveness.
Critical strengths
-
The paper addresses a practically relevant problem in AI reviewing.
It focuses on generic, inappropriate, trivial, and incorrect criticism, and separates broad identification of potential issues from verification of whether those criticisms are justified. This design addresses an important reviewing need and could reduce unsupported or insufficiently specific feedback. -
Score calibration has clear empirical support.
With the same DeepSeek backbone, the full method improves Pearson correlation from 0.5512 to 0.6137 and reduces MAE from 1.7358 to 1.3335 compared with No-Calibration. These results support the usefulness of calibrating scores against historical human reviews. Results across different backbones provide further evidence for the pipeline’s applicability.
Critical weaknesses
-
Insufficient validation on established benchmarks. The authors develop both ARD-Review and its evaluation framework, with the method and evaluation criteria closely aligned around reducing GUNI content. This makes it difficult to determine whether the reported gains represent broadly applicable improvements in review quality. Evaluation on existing, publicly available, widely used benchmarks is needed to provide independent validation and make the results more comparable with prior work.
-
Limited methodological novelty. The method mainly combines candidate generation by a harsh critic, filtering by a merger, and retrieval-based score calibration. These components form a sensible pipeline, but the contribution primarily lies in workflow organization and prompt design. The paper should more clearly establish what methodological advance this combination introduces and why that advance is consequential beyond the reported empirical gains.
-
Incomplete baseline comparisons. Comparisons with additional strong baselines are missing, leaving the empirical assessment incomplete. While the reported results demonstrate improvements over the included systems, they do not sufficiently establish the method’s competitiveness against a broader set of strong existing approaches. More comprehensive comparisons under consistent experimental settings would strengthen the evidence for the proposed pipeline’s advantage.
Questions for authors
See Critical Weaknesses.
Dear Reviewer,
As the discussion period is ending soon, we wanted to kindly check whether our response has addressed your concerns. We would be grateful for any further feedback, and we are happy to clarify anything else in the remaining time.
Thank you for your time!
Best regards, The Authors