Can Scientific Agents Make Research State Transitions as Evidence Evolves?
Abstract
Scientific discovery unfolds through evolving research states: researchers reason from the available evidence context, identify what remains unresolved, and update conclusions as new evidence arrives. However, scientific agents are often evaluated on isolated tasks or with the full research context available at once, leaving unclear whether they can construct, advance, carry and even revise a state along a real research trajectory. To make these capabilities measurable, we collect recent biomedical studies as source-grounded discovery graphs that record what was known at each point, what evidence supported each claim, and which analyses followed. This structure enables us to derive evidence boundaries and research checkpoints naturally from each study’s own trajectory, rather than impose them manually. We instantiate three settings: Initial Discovery from the data available at the outset; Continued Investigation from an source-grounded checkpoint, with only subsequent evidence revealed; and Full-Trajectory, where agents continue from the state they constructed themselves. We evaluate seven frontier agentic systems with graph-grounded rubrics that separately assess scientific setup, analysis design, and empirical endpoint, while allowing equivalent alternatives to the authors’ exact implementations. Agents often identify relevant analyses but rarely carry them through to supported conclusions. Supplying a grounded research state substantially reduces stable cross-system differences, indicating that state construction is a major source of capability variation. Strikingly, on matched downstream tasks, performance drops from 61.1% with the source-grounded checkpoint to 35.2% when agents continue from their own state. The loss is concentrated in analyses missed at earlier stages, which agents rarely revisit and repair. Our work shifts evaluation from whether an agent can rediscover scientific findings to whether it can advance a scientifically valid research state as evidence evolves.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.