acceptodds
Under review as a conference paper at ICLR 2027

SCIONBench: Evaluating Evidence-Preserving Research Trajectories from Motivation to Conclusion in Scientific Discovery

Abstract

A correct scientific conclusion can conceal an incomplete or unsupported research trajectory. Existing evaluations emphasize task outcomes, leaving the continuity of evidence across research stages insufficiently characterized. In order to address the problem, we introduce SCIONBench, a benchmark for evidence-preserving reconstruction of scientific research trajectories. SCIONBench represents 53 papers spanning 15 scientific domains and 131 research cases as source-grounded graphs linking motivation, method, experimental design, experiment, and conclusion. Four controlled disclosure modes progressively expose upstream information and assess reconstruction of held-out target nodes. Frozen evidence packages and explicit applicability decisions distinguish unavailable resources from execution failures, enabling stage-specific assessment under controlled information access. Node-level scoring assesses scientific correctness, experimental adequacy, evidence grounding, and conclusion scope. Across six model-agent configurations, agents recover conclusions more reliably when reference evidence is supplied than they reconstruct the investigation from motivation alone. On matched conclusion targets across 53 papers, evidence disclosure yields domain-macro gains of 2.68-2.88 points on a 0-10 scale for Claude Code configurations. However, providing a correct experimental design does not improve every independently executed continuation. These findings distinguish evidence-conditioned synthesis from reconstructing the investigation that supports a conclusion. SCIONBench provides a diagnostic framework for evaluating how scientific agents preserve the evidential dependencies that connect research stages.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.