- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper presents CellScientist, an autonomous agent framework for single-cell and spatial omics research. It combines Scientific Semantic Alignment, which enforces consistency among research questions, estimands, experimental designs, and conclusions, with Biological Prior Injection, which incorporates literature, domain knowledge, and analysis expertise to guide hypothesis generation. The system further performs iterative experiment search, validation, falsification, and manuscript generation.
CellScientist is evaluated on 20 real omics research topics against AI Scientist and OpenScientist. It achieves higher LLM-judged manuscript scores, particularly in reproducibility, research design coherence, and data–conclusion traceability. Ablation studies support the contributions of the two main components, and several biological case studies are provided to illustrate the generated findings.
Critical strengths
A key strength is the paper’s formulation of an important dilemma for autonomous scientific discovery: an AI scientist must avoid both scientifically invalid but seemingly interesting studies and methodologically sound but biologically trivial ones. This framing is meaningful because it goes beyond simply automating the research workflow and highlights two distinct requirements for useful scientific discovery: rigor and significance. The proposed separation between Scientific Semantic Alignment and Biological Prior Injection is a reasonable and intuitive response to this challenge.
Critical weaknesses
-
The main evaluation does not directly validate the paper’s central claim. The paper is motivated by a dilemma between scientifically invalid but seemingly interesting studies and scientifically valid but biologically trivial ones. However, the primary evaluation reduces research quality to scores assigned by another LLM to AI-generated manuscripts. Because the evaluation rubric itself emphasizes properties such as rigor, reproducibility, coherence, and traceability—properties explicitly built into CellScientist—the reported gains are difficult to disentangle from alignment with the judging prompt. More importantly, aggregate scores do not show whether the framework actually prevents the concrete scientific failures it is designed to address.
-
The paper lacks convincing case-level evidence demonstrating how the proposed mechanisms improve scientific discovery. The reported biological case studies mainly present final statistical findings, but do not clearly show how baseline agents fail, where Scientific Semantic Alignment or Biological Prior Injection intervenes, and how the resulting research question, analysis, or conclusion is materially improved. A small number of carefully analyzed paired examples would provide much stronger evidence for the claimed resolution of the rigor–significance dilemma than the current manuscript-level score comparisons.
Questions for authors
Clarification requests
-
How sensitive are the reported manuscript-level scores to the exact evaluation prompt and rubric? In particular, have you tested alternative judge prompts, different evaluators, or human expert assessment to determine whether the observed gains persist beyond the current LLM-based scoring setup?
-
Can you provide concrete paired examples in which a baseline system produces either a scientifically invalid study or a biologically trivial one, and show specifically how Scientific Semantic Alignment or Biological Prior Injection changes the research question, analysis, or conclusion? Such examples would materially strengthen the claim that the framework resolves the dilemma motivating the paper.
-
For the biological case studies, how do you ensure that the reported findings are genuine discoveries rather than rediscoveries guided by retrieved literature? In particular, are the original source papers or closely related downstream studies excluded from the retrieval corpus during hypothesis generation?