acceptodds
Withdrawn from ICLR 2027

SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autonomous AI research agents aim to speed up scientific discovery by automating the research pipeline from hypothesis generation to peer review. However, existing benchmarks rarely test a fundamental bottleneck whether LLMs can judge the methodological viability of a research idea investing time and computational resources. Without a reliable “first-gate” filter, autonomous agents risk scaling flawed methodology rather than accelerating meaningful science. We introduce , a curated benchmark of 1,099 real research proposals, with source verified extraction, reviewer-derived proxy labels, and proposal-only expert judgments on a subset of 327 proposals. Across 12 frontier LLMs, our evaluation reveals a pervasive : standard prompting yields a mean false-positive rate. Aggressive prompting reduces false approvals but substaintially increases false rejections. Additional analyses show that optimism persists under controls for public corpus contamination, and surface features, as well as in literature-grounded agent evaluation. These results suggest a limitation in judging scientific soundness, indicating that current LLMs are not yet reliable as standalone gatekeepers for scientific rigor.

Primary area:
datasets and benchmarks
Keywords:
Benchmarking LLMs, Autonomous AI agents, Scientific reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mj3KOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces SoundnessBench, a benchmark designed to evaluate whether LLMs can assess the methodological viability of research proposals before experimental execution. The authors construct a dataset of 1,099 research proposals extracted from existing ICLR submissions, using reviewer-derived soundness scores as proxy labels and additional expert annotations on 327 proposals. The benchmark evaluates 12 frontier LLMs under different prompting strategies and reports a significant optimism bias, where models frequently classify low-soundness proposals as high-soundness. The authors conclude that current LLMs are not sufficiently reliable for pre-execution scientific judgment.

Critical strengths

The paper addresses an important and timely research question regarding the ability of LLMs to evaluate scientific ideas before committing substantial computational and human resources. Such a capability would be particularly valuable for autonomous research agents, where early identification of flawed research designs could significantly improve research efficiency. The authors also make an effort to construct a relatively large benchmark covering multiple research areas and evaluate a broad range of frontier LLMs. The inclusion of different prompting strategies, additional expert annotations, and robustness analyses provides useful observations about the sensitivity of LLM judgments. Overall, I appreciate the motivation and the effort devoted to collecting and evaluating this dataset.

Critical weaknesses
  1. Fundamental retrospective bias in the benchmark construction. My primary concern is that the proposed benchmark is fundamentally biased by reconstructing research proposals from completed papers. The abstract, introduction, related work, hypothesis, and experimental design of a completed manuscript have already been organized and refined based on the conducted research and its outcomes. Therefore, extracting these components and removing explicit results does not recover the original research proposal before execution. This introduces substantial hindsight bias, making the reconstructed proposals fundamentally different from the actual pre-execution research ideas that the benchmark claims to evaluate.

  2. Mismatch between the benchmark labels and the research question. The paper aims to determine whether LLMs can identify methodologically flawed ideas before experimentation, yet its primary labels originate from reviewer assessments of completed research papers. These judgments may depend on experimental results, implementation quality, and evidence unavailable at the proposal stage. Although the authors introduce proposal-only expert annotations, these are still based on retrospectively reconstructed proposals and do not eliminate the underlying concern. Consequently, the benchmark does not convincingly establish that its labels represent the methodological viability of the original research ideas before execution.

  3. Insufficient evidence to support the claimed conclusions. The reported optimism bias is measured against labels whose relationship to genuine pre-execution methodological soundness remains uncertain. Therefore, disagreements between LLM predictions and benchmark labels cannot necessarily be interpreted as failures of scientific judgment. The additional prompting experiments and robustness analyses are conducted on the same retrospectively constructed benchmark and do not resolve this fundamental issue. As a result, I believe the paper primarily contributes a dataset and descriptive evaluation results, without sufficient methodological evidence to support its broader conclusions about LLM research-evaluation capabilities.

Questions for authors
  1. How can the authors justify reconstructing pre-execution research proposals from completed papers, given that the hypotheses, experimental designs, and related-work discussions may already have been refined based on experimental outcomes? How is this retrospective bias addressed beyond removing explicit results?

  2. How can reviewer-derived soundness scores from completed papers serve as reliable ground truth for assessing methodological viability before execution? How do the authors distinguish flaws identifiable at the proposal stage from weaknesses revealed only through experimentation?

  3. Have the authors considered evaluating LLMs on genuine research proposals collected before experiments are conducted, with independent expert assessments at that stage? Such an evaluation would appear more directly aligned with the paper's central research question.

Reviewer_X8WqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce SoundnessBench, which is a bench for judging whether AI can give appropriate soundness score on research proposals before conducting experiments. Data is extracted from ICLR submissions and labels are derived from reviewer soundness scores. Across 12 LLMs, the authors find that standard prompting frequently approves proposals labeled as low soundness (74.0% mean false-positive rate). A stricter prompt reduces false approvals but also substantially reduces recall on high-soundness proposals. The paper further examines literature-grounded evaluation and several possible dataset confounders.

Critical strengths
  1. The paper targets a consequential gap in research-agent evaluation.
  2. The evaluation exposes a useful error trade-off. Across 12 models, standard prompting frequently approves low-soundness proposals, while stricter prompting substantially increases rejection of high-soundness ones.
Critical weaknesses
  1. The soundness labels are an uncertain reference standard. Soundness scores are subjective, and reviewers saw the full papers while the models saw only proposals. Each proposal in the expert subset was also rated by just one person.

  2. The task feels narrow. Methodological soundness matters, but the paper does not clearly show what this benchmark adds beyond existing evaluations of research ideas and agents.

  3. The baselines need updating. I would like to see GPT-6 and Opus-5 models, as well as research or review agents such as the AI Scientist (Lu et al., 2024) and DeepReviewer 2.0 (Weng et al., 2026). Without them, the conclusions about current autonomous research agents feel too broad.

Questions for authors

See weakness.

Reviewer_GZnbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SoundnessBench, which evaluates whether language models can assess a research proposal's methodology before seeing its results. It constructs 1099 proposals from ICLR papers, assigns proxy labels from reviewer soundness scores, and adds proposal expert judgements for 327 examples. Across 12 models, standard prompting tends to approve proposals labeled unsound, while aggressive prompting reduces approvals at the cost of rejecting many proposals labeled sound.

Critical strengths
  1. The proposal-only expert evaluation addresses an important source of bias. The authors recognize that a review of a completed paper can depend on results unavailable at the proposal stage. Evaluating the same 327 proposals against both label sources, while keeping model predictions fixed, is a useful control.

  2. The analysis examines both kinds of decision error. Reporting recall for both classes makes the cost of aggressive prompting clear: fewer false approvals can come with many more false rejections. The controlled injection of severe hypothesis experiment mismatches is also informative. GPT 5.4's approval rate falls from 77% to 1%, showing that the observed approval tendency does not mean that the model ignores methodological content altogether.

Critical weaknesses
  1. The benchmark does not establish how reliably its target can be judged from a proposal. The main result counts disagreement with the reference label as a model error. This is difficult to interpret when the large scale labels come from full papers and the proposal-only subset assigns each proposal to just one expert. The later one is a substantial improvement in input matching, but it provides no estimate of agreement between experts given the same proposal.

  2. The evaluation prompt asks for a different judgment from the stated task. The task definition explicitly excludes novelty and asks whether the experiment can test the hypothesis rigorously. The standard prompt instead defines the classes in terms of the strength of the scientific contribution and includes a trivial hypothesis as a reason for a low label. A modest or unsurprising hypothesis can still be tested soundly.

  3. The experiments do not separate poor calibration from poor discrimination. The aggressive prompt explicitly tells the model to default to a low label. Its shift toward rejection is consequently evidence of prompt sensitivity, but does not by itself show that the model cannot distinguish sound from unsound proposals at a better operating point. The ScholarEval threshold analysis is useful, yet it concerns a separate evaluator and does not establish this distinction for the 12 directly prompted models.

Questions for authors
  1. Does the extraction preserve details that determine soundness?

  2. Which apparent model errors survive independent adjudication?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.