- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces an Audited Discovery framework that separates the exploration process of AI scientific agents from statistical reporting.
Critical strengths
-
The paper accurately identifies a fundamental weakness of current AI scientists: their tendency to produce false discoveries when evaluating their own findings. Introducing independent statistical auditing is a particularly insightful and elegant solution.
-
The authors conduct extensive experiments across synthetic and semi-synthetic (plasmode) settings, complemented by blinded expert evaluations using real-world neuroscience data. This thorough evaluation provides compelling evidence for the effectiveness of the proposed approach.
Critical weaknesses
-
The current framework only supports statistical hypotheses expressed in predefined structured templates, such as correlations and group differences. This limits its ability to handle free-form scientific claims or more complex scientific concepts.
-
The improved reliability comes at the cost of missing some genuine scientific discoveries. Most agents experience a 6–35% drop in recall after auditing, highlighting a trade-off between statistical rigor and discovery yield.
Questions for authors
How do the authors plan to extend the auditing framework to more complex, non-standardized scientific claims that lack well-defined statistical testing procedures?
7 comments
Response to Reviewer nzeC (2/3)
W1 & Q1 (continued)
Why each claim type has a fixed test. We agree that fixed claim types are a limitation. We chose them because FDR control holds only when every claim's test is valid, and agents do not reliably write valid tests. Our experiments show three ways in which agent-written tests fail.
- Tests that overstate the evidence. We reran POPPER with its default settings and found that 80% of its false acceptances rested on a single agent-written test with a very small -value, although our fixed test found no significant effect on the same holdout
[new]. - Running multiple tests until one passes. POPPER's agent writes up to ten tests per candidate and continues until their combined evidence crosses a threshold. We found that 95% of its false acceptances came after two or more tests in the paper's run
[new]. - Ignoring dependence between rows. When rows are related, for example repeated measurements of the same person, a valid test should account for this dependence. We ran POPPER with Claude Sonnet 5 as its backbone on such data, and none of the 191 tests its agent wrote accounted for the dependence
[new]. When we reran these tests on new draws of the data, they found an effect for false claims 12.4% of the time at a stated rate of 5%, while the audit reported none of these false claims.
When we changed only the test, holding Opus's candidates, the holdout and the e-BH reporting rule fixed, FDR fell from 13.7% with POPPER's tests to 0.4% with our fixed tests (App. E.4). We therefore assign each claim type one prespecified test, a standard permutation or resampling test (Chung & Romano, 2013; DiCiccio & Romano, 2017; Freedman & Lane, 1983; Winkler et al., 2014; Wu, 1986), and check its validity once for the whole type (App. A.5).
Claims without a well-defined test [new]. For a claim type without a well-defined testing procedure, the literature or an agent can propose candidate procedures. The audit accepts a candidate only if its false-positive rate stays at the nominal level on simulated data with no true effect. Finding a valid test therefore becomes a program search with a verifiable objective, and the audit completes this search before reading the holdout.
As a proof of concept, we ran the search for two claim types outside our set, an indirect effect ( affects through ) and bimodality (a variable has two modes), with 10 and 6 candidate procedures. The search rejected several procedures that look reasonable. For example, one permutation test for the indirect effect found an effect in up to 84% of datasets that had none. With the procedures that passed, the audit reported 85% of the true new claims as discoveries and almost none of the false ones, and FDR over all reported claims stayed at 0.5%. The audit can therefore take on new claim types without weakening its control.
Free-form claims. The same search would support free-form claims. The agent writes each claim together with the code that tests it, using only the exploration split. The audit runs this code on simulated data with no true effect and returns the false-positive rate to the agent, which revises the code until it passes. The audit then freezes the code and runs it once on the holdout, under the same FDR control as every other claim. We will describe this extension in the revised paper.
References
- Chung, E. & Romano, J. P. (2013). Exact and asymptotically robust permutation tests. The Annals of Statistics, 41(2), 484–507.
- DiCiccio, C. J. & Romano, J. P. (2017). Robust permutation tests for correlation and regression coefficients. Journal of the American Statistical Association, 112(519), 1211–1220.
- Freedman, D. & Lane, D. (1983). A nonstochastic interpretation of reported significance levels. Journal of Business & Economic Statistics, 1(4), 292–298.
- Winkler, A. M., Ridgway, G. R., Webster, M. A., Smith, S. M. & Nichols, T. E. (2014). Permutation inference for the general linear model. NeuroImage, 92, 381–397.
- Wu, C. F. J. (1986). Jackknife, bootstrap and other resampling methods in regression analysis. The Annals of Statistics, 14(4), 1261–1295.
Response to Reviewer nzeC (3/3)
W2. Trade-off between reliability and recall of true discoveries
"The improved reliability comes at the cost of missing some genuine scientific discoveries. Most agents experience a 6–35% drop in recall after auditing ..."
We agree that the guarantee costs recall, and every procedure that controls false discoveries pays this cost. With a fixed amount of data, a weak true effect and no effect can produce the same evidence, so a procedure that admits fewer false claims may also miss more true ones. Genovese & Wasserman (2002) characterize this trade-off between false and missed discoveries, and Rabinovich et al. (2020) derive the best trade-off that any procedure can achieve in a Gaussian model. In return for the 6–35% loss in recall, the audit cuts the share of false claims from 18–66% in the agents' own reports to at most 1.3%.
At equal FDR, the audit finds the most true discoveries. At a 5% FDR matched in hindsight, the audit finds 84.8 true discoveries per run for Opus, against 69.0 for correction on exploration evidence and 31.3 for Opus's own ranking (Fig. 4a). No other agent's own ranking reaches 5% FDR at all. The recall loss therefore comes from the lower FDR that the audit delivers.
Different FDR procedures trade recall for FDR differently. A procedure that makes fewer assumptions about the tests is more cautious and finds fewer true discoveries. Bonferroni is the most cautious. BY and e-BH stay valid however the tests depend on each other. BH finds the most true discoveries but assumes the tests are independent or positively related. We compared these procedures on the same holdout evidence as a practical guide for users. For Claude Code (Opus):
| Procedure | FDR (target 5%) | # True | FDR (target 10%) [new] | # True |
|---|---|---|---|---|
| Agent's own report | 17.9% | 77.7 | 17.9% | 77.7 |
| Bonferroni | 0.3% | 61.0 | 0.9% | 64.4 |
| e-BH | 0.6% | 69.2 | 1.0% | 72.0 |
| BY (our default) | 1.0% | 73.4 | 1.5% | 76.3 |
| BH | 3.3% | 81.7 | 6.7% | 85.9 |
BH finds more true discoveries than Opus's own report (81.7 against 77.7 per run) at a fifth of its FDR (Table 15). The revised paper will include this comparison for every procedure and agent.
The missed claims are weak effects [new]. The recall the audit gives up comes mostly from weak effects, which a holdout of limited size cannot reliably confirm. Our semi-synthetic environments plant true relationships at three strengths, so we can measure which ones the audit misses.
| Effect strength | Strong | Medium | Weak |
|---|---|---|---|
| Percentage recovered | 81% | 70% | 49% |
The audit's power rises with the size of the holdout, from 75% of true claims at 500 rows to 93% at 2,000 on the health survey data.
With FDR controlled, agents can explore more. The audit controls FDR over every candidate an agent registers, however many there are, so the agent's own cutoff no longer limits what is reported. ShinkaEvolve, for example, reports only a third of the true candidates it finds, and auditing its full list raises its true discoveries by 63% (Sec. 5.3).
We also tested whether a larger exploration budget pays off under the audit [new]. With 1,000 candidates in place of 300 and twice the time, Claude Code (Opus) doubled its audited true discoveries on the superconductor data, which contain many true relationships, and the audit kept FDR at or below 1.0%. At the same FDR, the audit found 2.6 times as many true discoveries as Opus's own ranking. The gain depends on the agent, and Sonnet and ShinkaEvolve did not gain consistently from the larger budget. Whatever the budget, the audit kept FDR controlled, so agents can explore further without reporting more false claims.
We hope these answers address your questions. Overall, our paper systematically measures how often discovery agents report false claims, identifies why their own checks fall short, and introduces an audit that controls the false discovery rate of existing agents without changing how they explore. We would be happy to continue the discussion. If these answers resolve your concerns, we would be grateful if you would consider raising your score.
References
- Genovese, C. & Wasserman, L. (2002). Operating characteristics and extensions of the false discovery rate procedure. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(3), 499–517.
- Rabinovich, M., Ramdas, A., Jordan, M. I. & Wainwright, M. J. (2020). Optimal rates and trade-offs in multiple testing. Statistica Sinica, 30(2), 741–762.
Response to Reviewer nzeC (1/3)
Thank you for your thoughtful and constructive review. Your comments helped us strengthen the paper. We are glad you found the audit "a particularly insightful and elegant solution" and the evaluation thorough and compelling. Below we answer both concerns with new results.
W1 & Q1. Support for free-form and more complex scientific claims
Our supported claim types go well beyond correlations and group differences. They include effects that differ between groups or depend on a third variable, nonlinear effects such as thresholds, associations after controlling for other variables, and trends and delayed effects over time. These types follow the standard two-variable and adjusted analyses in two widely used test-selection guides (UCLA Statistical Consulting Group, n.d.; Motulsky, 2017), extended to data that follow the same units over time. Claims can also be restricted to a subgroup defined by up to two conditions and adjusted for up to three covariates (App. A.2), so on a table with 50 columns the claim types can express about 1013 distinct claims
[new].What each claim needs. The audit needs a valid test for each claim, one that gives false positives no more often than its nominal level (e.g., at most 5% of the time at level 0.05) when there is no relationship. A new test can therefore be added to the audit once it shows this property on simulated data with no true relationship. This check uses only simulated data, so it is done before the test ever touches the holdout. Tests for non-standard claim types and for free-form claims can then be added with the same FDR control.
Complex hypotheses in practice. Agents in our runs proposed hypotheses that combine several of these features at once, such as the three below.
References