acceptodds
Under review as a conference paper at ICLR 2027

DeepQuest: From Knowledge Gaps to Scientific Discovery in Alzheimer's Disease

20 views

Abstract

Automating experiments does not necessarily advance scientific understanding. Research agents must identify meaningful knowledge gaps, determine what available data can resolve, and assess whether their findings advance existing knowledge. We introduce DeepQuest, a knowledge-gap-driven AI scientist framework for Alzheimer’s disease research. DeepQuest transforms unresolved questions from domain reviews into traceable, testable hypotheses and prioritizes them through data feasibility screening and groupwise Elo ranking. It couples iterative experimentation with two-level auditing that separates experimental reliability from knowledge contribution relative to a frozen literature corpus. On OASIS-3, DeepQuest produces 42 feasible hypotheses and outperforms the strongest baselines in 14 of 15 evaluator–dimension comparisons across three language-model reviewers. The full framework achieves an audit-assessed scientific finding rate of 95.24%, versus 78.57% for Codex and 76.19% for Claude Code on the same hypotheses. Representative findings concern neuropsychiatric symptoms and dementia risk, longitudinal amyloid spatial stability, and neocortical tau-associated clinical impairment, with key associations supported in independent ADNI and AIBL cohorts. These results show how aligning knowledge gaps, data feasibility, and evidence auditing can turn automated research into traceable, evidence-supported scientific discovery. Our source code is available at https://anonymous.4open.science/r/DeepQuest-3066.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI Scientists, Knowledge-Gap-Driven Research, Automated Scientific Discovery

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Pv2eOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DeepQuest, a knowledge-gap-driven AI scientist framework for Alzheimer's disease (AD) research that automates the workflow from literature review to empirical discovery. It transforms unresolved limitations from domain reviews into traceable, testable hypotheses, and then select and prioritize tasks under data feasibility constraints. A two-level auditing mechanism separates experimental execution reliability from knowledge contribution relative to a frozen 200-paper corpus. On OASIS-3, DeepQuest generates 42 feasible hypotheses, achieves a 95.24% Scientific Finding Rate, and validates key clinical findings across independent ADNI and AIBL cohorts as well as external finding reproduction tasks.

Critical strengths
  • Principled Coupling of Data Feasibility and Two-Level Auditing: By grounding hypothesis selection in structured multimodal data summaries and auditing both sandbox code execution and literature novelty, DeepQuest achieves a 95.24% Scientific Finding Rate on OASIS-3 while drastically reducing execution skips and uncertain outcomes
  • Rigorous Cross-Cohort Biomedical Validation and Reproduction: Beyond LLM-based metrics, the framework produces clinically meaningful findings in OASIS-3 that replicate in independent ADNI and AIBL cohorts (covering NPS-related dementia risk, longitudinal amyloid topographic stability, and neocortical tau progression), and independently recovers core patterns from two recent 2026 clinical studies across different tracers and outcome definitions.
Critical weaknesses
  • Reliance on Automated Evaluators and a Narrow Frozen Corpus Without Expert Calibration: In Section 4.2, the three LLM evaluators overlap with the baseline generators (potentially introducing self-preference bias), and in Section 3.6, assessing novelty solely against 200 papers published after February 16, 2026 may overstate the rate of "Candidate New Discoveries" by missing earlier foundational AD literature, with no blind evaluation by human clinical experts.
  • Lack of presentation of the criteria of selecting the 200 clinical papers.
  • The findings in Sec 4.4, unveiled by DeepQuest, are somehow established in the area. They may also stem from the model's parametric knowledge, not just the auto research process. An isolation between them can make more solid the claim that the model is able to give novel findings.
  • Incomplete Component Ablations : As noted in Sections 4.2–4.3, the nested Top-10 evaluation and end-to-end execution comparison do not isolate the individual contributions of groupwise Elo ranking, within-round execution auditing, or cross-round planning updates.
Questions for authors

Clarifications:

  • Why was the evidence bank restricted to 200 clinical papers published after February 16, 2026, and how many of the 20 "Candidate New Discoveries" would be reclassified as "Rediscovery" or "Extension" if audited against a broader historical PubMed corpus? What is the criteria of selecting the 200 clinical papers.
  • Did the authors check for self-preference bias when GPT-5.6-sol, Claude Opus 4.8, and DeepSeek-v4-pro-0813 evaluated their own generated hypotheses, and how does the 95.24% SFR change if cross-round optimization or execution auditing is ablated on DeepQuest-H?

Suggestions:

  • Adding a small-scale blind review by domain neurologists on the Top-10 hypotheses and audited claims would significantly strengthen the validation of clinical significance and novelty.
Reviewer_SGX3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

They propose an agentic pipeline for Alzheimer's disease research, DeepQuest. It works by extracting research limitations from review articles, turning them into hypotheses, filtering them for data feasibility, and then ranking them. Selected hypotheses go through up to three rounds of experiments with an execution auditor. A final stage compares each claim to a set of existing papers and labels it as Rediscovery, Extension, Candidate New Discovery or Uncertain.
 They report that LLM judges score DeepQuest's hypotheses above five LLM baselines.

Critical strengths

I like the idea of separating whether an experiment is reliable from whether its result actually adds knowledge, this seems important

Every claim is traceable back through the experiment and hypothesis to the source limitation.


It looks like the case study analyses use sensible methods

Critical weaknesses

The paper's central claim is that DeepQuest produces findings that advance existing knowledge, and the main evidence for that is the SFR and the contribution labels. I don't think either of these actually measures discovery particularly well though. 


I'm not sure what the frozen corpus is meant to be. Section 3.6 treats it as "existing knowledge", but it only contains papers published after Feb 2026, so a result that's been in the literature for ten years can still come back as Candidate New. If the corpus is meant as a temporal holdout, i.e. ground truth the system couldn't have seen, then Rediscovery should be the success case, and the interesting number would be how many held-out findings DeepQuest recovers. As it stands, zero Rediscoveries is presented as a good thing. In the case studies, Claude tells me that the core associations (neocortical tau and faster decline, neuropsychiatric symptoms and later dementia) are well established, and while the specific refinements might fairly count as Extensions, the paper doesn't say which label each case study received.

Questions for authors

Why does the evidence corpus only include papers published after Feb 16, 2026? Is it intended as a temporal holdout, and if so, why isn't Rediscovery treated as the success outcome?

Which contribution label did each of the three case studies receive? How would they be labelled if the contribution assessment searched the broader AD literature (e.g. PubMed scale retrieval) instead of the 200 paper corpus?

Did the model baselines run with the same auditor, revision limit, rounds and compute budget etc as DeepQuest?

Is anything in the pipeline specific to Alzheimer's, beyond the choice of papers, data and imaging adapters? Do you see this pipeline as something that generalises?

Which model is used for DeepQuest's backbone? Could there be self-preference effects in the evaluation?

Could you perform ablations so we can tell which component (gap extraction, feasibility, ranking, auditor, cross-round revision) is responsible for the performance?

SFR counts Rediscovery, Extension, Candidate New and supported null effects as findings – does this mean the only failures are Uncertain and Skip? This seems more like a completion rate, i.e. a 95% SFR tells me the pipeline completes, not that it found anything new – is that correct?

Reviewer_9WQSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DeepQuest develops a useful automated research workflow. It turns knowledge gaps from Alzheimer's disease (AD) reviews into testable hypotheses, filters and ranks them using available data, and conducts iterative experiments with separate execution and novelty audits. Evaluation covers 42 OASIS-3 hypotheses, a reported 95.24% audit-assessed scientific finding rate, and three selected case studies with external checks in ADNI/AIBL cohorts.

Critical strengths
  • Traceability is a useful design contribution. Linking hypotheses to documented knowledge gaps, while separating experimental reliability from knowledge contribution, makes the research process easier to inspect and challenge.
  • External cohort validation provides evidence beyond LLM judgments. The selected case studies test findings in independent ADNI or AIBL cohorts, with support for several reported associations. This strengthens the empirical demonstration, although not every association is clearly confirmed across cohorts.
Critical weaknesses
  • Scientific novelty and evaluator reliability are insufficiently validated. The knowledge audit covers only 200 studies published after February 16, 2026, leaving earlier precedents outside its scope. Hypothesis quality is assessed by LLM judges, but no expert calibration is reported for these scores or the discovery labels. This weakens the evidence that the reported gains reflect genuine scientific advancement
  • The baseline comparisons are not sufficiently controlled and omit dedicated scientific agents. In the hypothesis quality evaluation, DeepQuest contributes 42 screened hypotheses and uses detailed data summaries, whereas each language-model baseline generates 20 hypotheses with the cohort overview as data context. Execution comparisons use Codex and Claude Code, but do not include a dedicated AI scientist baseline. These comparisons leave unclear whether the gains persist under matched information and selection budgets, and whether DeepQuest improves on task-compatible scientific agent workflows rather than general-purpose execution agents.
Questions for authors
  1. How were the 200 reference papers and the February 16, 2026 cutoff selected? For the representative case studies, what specific proposition is new relative to the closest prior work, including earlier literature, and what supports the Candidate New Discovery or Extension classification?
  2. Why did each baseline generate 20 hypotheses in the hypothesis quality evaluation, and how many candidates did DeepQuest generate before screening? Is there evidence that the gains persist when baselines receive the same structured data summaries and equivalent generation and feasibility screening budgets?
  3. Which model performs the knowledge audit? Were hypothesis scores and discovery labels independently checked by domain experts blinded to the generating system? If so, how closely did the automated assessments agree with expert judgments?
  4. Why were dedicated scientific agents, particularly the biomedical systems discussed in Related Work, excluded from the evaluation? Is there a comparison with a task-compatible research agent under matched literature, data access, and resource budgets?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.