DynReviewer: A Domain-Aware AI Reviewer for Each Paper from Past Human Reviews

AI reviewers are entering peer review, where papers from different domains go to different experts, who judge them by the standards of their domain. Yet most AI reviewers apply one fixed configuration to every paper. They are also evaluated mainly on ICLR, where a single average hides performance differences across domains. We propose DynReviewer, which reads the paper under review first, then dynamically configures a dedicated AI reviewer for it. DynReviewer rewrites general checks distilled from past human reviews, along with reviews of related papers, into a paper-specific rubric, then searches the paper for evidence on each item. The whole process requires no model training and no human-written rubrics. We also build ReviewAtlas-40K+, which collects 45,551 papers and their official reviews from six venues. Reviews up to 2024 serve as historical experience, and papers from 2025 form the benchmark ReviewAtlas-Bench, covering four venues and four ICLR research domains. Across three LLM backbones, DynReviewer outperforms all compared methods in every venue and domain at covering human reviewers' concerns with criticisms grounded in the paper. It leads the strongest same-backbone baseline by 5.6 to 19.9 percentage points. Its reviews also have the highest content quality in most settings, and its ICLR acceptance decisions agree best with actual outcomes. Analyses show that general checks distilled from human reviews clearly outperform LLM-written ones, and that rewriting experience into a paper-specific rubric works much better than using it directly. Past human reviews thus let an AI reviewer adapt to each paper, as human peer review does.