When Statistical Discoveries Become State: Reliability in Continual Research Agents
Abstract
When scientific discovery agents operate continually, statistical discoveries can become operational state that guides subsequent search rather than remain terminal outputs. This creates a Decision-to-State Gap: guarantees for local admission decisions do not characterize the reliability of the persistent state that drives future search. We study this gap in a controlled benchmark with exact ground truth, 128 hypotheses, and a frozen stateful proposer, distinguishing currently usable discoveries (CURRENT) from immutable admission history (EVER). Across three preregistered positive-dependence stress conditions, permanent append yielded terminal Global-Null active-state FDRs of 0.410–0.422, compared with 0.044–0.064 under cumulative recertification; the approximate 95% simultaneous bands for all three paired gaps lay entirely above zero. Controlled interventions on proposal-visible state changed the next candidate in 20% of selected planted-signal snapshots, while equal-size states with different membership yielded different next candidates in 37.5% of planted pairs. A frozen real-LLM transport pilot further found state-conditioned proposal-distribution shifts beyond same-state repeat instability (, 95% stability interval [0.158, 0.413]). An exploratory downstream-error signal motivated an independently seeded, preregistered 320-pair confirmation, but its primary hypothesis, an increase in distinct future false admissions, was not confirmed. Together, these results identify reusable statistical state as a distinct reliability object and motivate a governance principle: preserve admission history while separately governing future action capability.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.