acceptodds
Under review as a conference paper at ICLR 2027

Scoring How, Not Whether: A Protocol for Evaluating Discovery in Scientific Agents

Abstract

Scientific research combines prior knowledge with evidence, and a discovery is usually made where the prior is absent or wrong. An agent can therefore contribute to discovery only if it derives claims from the evidence and lets that evidence override what it already knows or has been told. A correct answer alone cannot show which happened: reasoning from the evidence, recall from a prior, or acceptance of an answer it was handed. We therefore propose a protocol that evaluates the process separately from the outcome, combining controlled disclosure of the target identity, a planted false identity, and three trace measures: where each claim came from, whether it is backed by computed evidence, and whether a supplied identity was tested against the agent’s own. The measures are read from the trace by judge models from outside the evaluated families, so that no model grades its own, and each is paired with a control condition or an input ablation. We validate the protocol on cancer subtype discovery with GPT-5.5, Sonnet 5, Opus 5 and Gemini 2.5 Pro. Four findings emerge. Withholding the cancer identity does not keep it hidden: GPT-5.5 and Sonnet 5 infer it from the gene symbols, Opus 5 names it before any gene is visible, most often from the sample count and clinical profile, and Gemini 2.5 Pro most often names none; the agent then confirms the cancer rather than considering alternatives, and the outcome score records none of this. The agents are better interpreters than analysts: all four propose well-supported mechanisms, yet once the identity is withheld three recover the published subtypes no better than a scripted k-means, and only Opus 5 beats it in every condition. A prescribed procedure improves the outcome score with no detectable reduction in adoption of a planted false identity. When a supplied identity conflicts with the agent’s own evidence, Sonnet 5 reads its evidence as supporting the label in every episode it tests, GPT-5.5 records the contradiction and defers, Gemini rarely tests the label, and Opus 5 follows its evidence while accepting every true label; what separates them is which side prevails when a check fails. Human annotators agree with the judges on which side prevailed, less so on whether the label was tested. The results indicate that current agents are not yet reliable enough to be left unsupervised on a discovery problem: their answers can be recalled or accepted rather than derived, and a correct one does not show which. The protocol provides the means to examine this from the agent’s log, and can be applied where no answer is available to check against.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
scientific discovery agents, LLM agents, evaluation protocol, process evaluation, benchmark contamination

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_6hq2OpenReview ↗
Rating
2(−1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a protocol for evaluating how scientific agents reach their conclusions rather than only checking final answers. It uses controlled disclosure of cancer identities, planted false identities, and LLM judges to evaluate evidence provenance, support, and premise testing. The authors evaluate four agent models on 760 episodes across seven TCGA cancer cohorts. The results show that agents often recognize cancer identities from prior knowledge or available clues. Three models frequently adopt the planted false identity, while Opus 5 is more resistant. The paper also compares agents with a scripted k-means baseline and analyzes several evaluation artifacts.

Critical strengths
  1. Generally speaking, the paper is trying to address a meaningfull and important question in the agent scientific evaluation. It is true that the correct answer does not always mean an agent discovered it from the data. Also, the paper's method separates outcome quality from evidence provenance. Additionally, it test how agents with the false information.

  2. By and large, the experiment is comprehensive. This paper's evaluation consists of judge sensitivity analysis and comparisons across four models. Also itself identify several problems in the evaluation, I appreciate that the negative results are reported.

Critical weaknesses

Concerns about the process evaluation reliability:The reliability of the process evaluation remains questionable. LLM judges show large differences in D2 explore counts (ranging from 102 to 401) and fail to reach a majority consensus in 153 out of 760 episodes. Additionally, neither provenance instrument has undergone human validation. A 20-trace human audit shows low agreement with the LLM judges on whether a label was tested (κ = 0.41 and 0.10), and the underlying model of one judge, laguna, remains undisclosed.

Confounded Evidence-Based Resistance: It is difficult to isolate evidence-based resistance. While Opus 5 avoids the false identity in a portion of G3 episodes, it also identifies cancers early in G2 before gene-symbol release, making it unclear whether its robust performance relies on active evidence checking or simply prior knowledge. Furthermore, experimental inconsistencies—such as G0 providing the true identity upfront versus G3 introducing a false identity later, alongside varying prompt structures—weaken the control of the procedure comparison.

Questions for authors
  1. Could you provide the human validation of the provenance labels and tell me about the model of laguna?

  2. I wonder for opus 5, will it still be resistent to the false labels when it has not identified the correct cancer before? To what extent do the different disclosure timelines between G0 and G3 impact this conclusion?

Post-rebuttal update: lowered from 3 to 2, see comment

3 comments

We thank the reviewer. All changes in the revised PDF are marked in blue.

Q1. Human validation of the provenance labels, and the laguna model.

Two new annotators reviewed 32 traces for identity provenance (Appendix A.12). Against the panel, they agreed on the D2 strategy in 12/25 and 12/26 traces (κ=0.08\kappa = 0.08 and 0.040.04). They confirmed the panel's exploit calls (11/11, 9/12) but rarely its explore calls (1/10, 3/10). Both put G0G_0 exploration at 0%; for G2G_2 they gave 7% and 40%, against the panel's 46%. We now treat these labels as exploratory and have narrowed the claims that rely on them (Section 4.1, Appendix A.9 and Limitations). Analytic provenance remains unaudited by humans.

For premise handling (Appendix A.11), the two new annotators agreed with the panel on which side prevailed, the supplied label or the agent's own evidence, in 19/20 and 18/19 assessable cases, and on whether the agent tested the label in 17/20 and 17/19.

The "laguna" judge is Poolside's Laguna S 2.1 (model card), served through our institutional gateway; the paper now names it in Section 3 and Appendix A.8. We chose the three judges (nemotron-3-super, Laguna S 2.1, Qwen3.6-27B) for diversity: they come from three different developers, none belongs to an evaluated agent family, so no model grades its own, and each has a context window long enough to read a full episode trace.

Q2. Does Opus 5 resist the false label when it has not identified the cancer first, and how does disclosure timing affect this?

We added experiments to address this (Appendix A.13). We ran a 2×22\times 2 design crossing label truth with label timing. Every cell delivers the label through the same channel, with 16 episodes per cell and per model.

LabelTimingOpus 5 ends on the true cancerSonnet 5 ends on the true cancer
falsestart11/160/16
false3rd observation (G3aG_{3a})7/160/16
true3rd observation15/1616/16
truestart16/1616/16

Yes, Opus 5 resists without a prior identification. With the false label given before any data, it rejects the label in 11/16 episodes. Neither model rejects a true label at either timing, and Sonnet 5 adopts the false label at both timings. (The false-label cell at the third observation is the 16 lean G3aG_{3a} episodes, re-scored.)

Across the original G3G_3 episodes, we split Opus 5 by whether it had named the true cancer before the label:

Before the labelResisted95% CI
Had identified the true cancer28/43 (65%)50–78
Had not identified it6/8 (75%)41–93

Resistance is similar either way (p=0.70p = 0.70). Sonnet 5 rejected 0/29 without a prior identification and 0/13 with one. A prior identification is neither necessary nor sufficient for resistance.

On timing, the original G3G_3 already varies when the label arrives (third vs fifth observation; later reveals would leave the agent nothing to do before the label). With the new start cell this gives three timings:

Label arrivesHad identified firstResisted
Start0%11/16
3rd observation (G3aG_{3a})75%21/32
5th observation (G3bG_{3b})100%13/19

The later the false label arrives, the more often Opus 5 has already identified the true cancer: 0%, 75%, 100%. Its rejection of the false label stays at 66–69% (p=0.97p = 0.97), whereas the prior-identification account predicts it would rise. We detect no timing effect, though the cells are small (16–32 episodes) and cannot exclude one. Sonnet 5 adopts the false label at every timing.

New episodes were scored by a single LLM judge (Qwen3.8-Flash), because the original panel endpoints were retired. On the 32 episodes scored both ways, it agrees with the paper's panel on 30. The G3G_3 splits above use the paper's panel. The design and decision rules were fixed before the runs.

Reviewer_6hq2OpenReview ↗

Response to rebuttal

The new experiments address part of the issues. However, some new concerns arise and I decide to lower the rating from 3 to 2.

  1. Human validation shows the core provenance instrument is not reliable: When κ = 0.08 and 0.04, Annotators agree with the panel’s D2 strategy label. Annotators then set the 7% and 40%. This is aganist the 46% of the panel.

  2. author only copy the number that looks closest to the final answer. They agree on the winner. But they do not agree on whether the test actually happened (κ 0.10–0.61). And the numbers swing wildly depending on the judge (+8.3 to +71.4 pt). So it seems that there's barely any proof for how the scoring works.

3.Question 2 has not been completely solved yet. Opus 5 could still recognizs the dataset when the label is revealed. for the 11/16 start-label rejections. we couldn't know whether the Opus 5 name the true cancer after the symbol is released or not.

Therefore, the paper is moe like a case study not a validated evaluation protocol

We thank the reviewer for the careful follow-up.

We agree that the provenance labels are not reproduced by humans, and we present the explore rates as exploratory. The disagreement is largely explained by the judges: they tend to accept the agent's own account of how it reached the identity (e.g., "inferred from KLK3/KLK2") even when the trace shows the cancer was named before any gene name was visible. In the premise test, where the judge is told when the label arrived, agreement is high. In future work we will give the judges the harness's event timing and replace the panel with human annotation where it disagrees substantially with humans. In the present paper, the supported claims rest on the interventions (withholding the identity, planting a false label, exposure-conditioned denominators), not on trace labels.

On point 2: we agree that which side prevailed is read from the final identity, and that the judges' readings of the trace are weakly validated. However, the "how" comes mainly from the interventions rather than from these judge readings. On its own, a final identity does not show how it was reached. When we control what the agent is told, it does: an agent that ends on the true cancer after being given a false one did not take its answer from the label. No model rejects a true label (Appendix A.13, Table 13), so a rejection is not a habit of contradicting the harness. With a false label given before any data, Opus 5 rejects it in 11/16 episodes and Sonnet 5 in 0/16 (Fisher p<0.001p < 0.001). The two models receive the same input; Sonnet 5 takes the label, and Opus 5 ends on the true cancer citing markers it has computed (see Q2). Opus 5's answer therefore came from its own evidence, not from the label, and this needs no judge (see also Section 5, lines 465-466). Identity provenance at G2 also needs no judge, since it is read from the harness timestamps: 100 of 168 withheld identities are named only after the gene symbols are released (Section 4.1, line 309), so withholding the identity did not make most agents derive it; and Opus 5 names a cancer before any gene name is visible in 24 of 42 episodes (Appendix A.14, Table 15, line 1306). The outcome score cannot separate these routes: within each model it is the same across them, and its gate fires in 5 of 168 G2 episodes (Section 4.1, lines 319-322). The low human agreement in A.12 does not affect these readings.

On Q2: all cells of the 2×22\times 2 are scored by the same judge; on five Opus 5 start-label episodes the opening instruction omits "show your reasoning", and every reading holds without them (Appendix A.13, lines 1250-1254). The matched contrast is 11/16 rejections with the label at the start versus 7/16 at the third observation (Fisher p=0.29p = 0.29), not significant and opposite to what prior identification predicts: by the third observation Opus 5 has usually named the cancer first (Table 14), yet it rejects less. In 10 of the 11 start-label rejections, Opus 5 first names the true cancer only after the gene symbols are released, in 9 while citing lineage markers it has just computed (e.g., TP63/SOX2/KRT5 high and NKX2-1/NAPSA low for LUSC; PAX8/WT1/MUC16 high for OV). None of the 5 adopting episodes names the true cancer. The shape cues (sample count, deaths, age, stage) are the same in all eight episodes of a cohort, yet with 11 rejections over two cohorts of eight, at least one cohort gives both outcomes. The rejections therefore follow marker evidence rather than shape cues, although a regex over the reasoning text cannot fully separate derivation from recognition (Section 5, lines 463–465; Appendix A.13, lines 1274-1275). The final identity and the naming time are read from the submission and the timestamps, not from the judge.

We accept that the findings are a case study of one task. On this task, the parts of the protocol that read the log without a judge are validated; the judges' trace labels are not. We found their error by checking them against the harness timing, which is the point of Appendix A.21 (line 1611, "Instruments require known-value arms").

Reviewer_3pTQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper argues that a correct answer cannot show whether an agent derived it from evidence, recalled it from prior knowledge, or accepted it from the harness, and proposes a protocol that scores the process separately from the outcome. It controls the channels through which an answer could arrive by holding the target identity and releasing identifying symbols after the agent commits. The paper experiments on blinded TCGA subtype discovery.

Critical strengths

None

Critical weaknesses
  • Unqualified general claims from a single task: The title, abstract, and conclusion mention findings about scientific agents without specifying the scope. However, the experiments are conducted only for TCGA, and the limitations section mentions that the validation is conducted on one task in one domain without generalization. The claim should not be generalized without sufficient testing.
  • The instrument is validated on too little human data: The human evaluation covers 20 traces and effectively 2 annotators. This is thin for a paper whose contribution is the process measure itself.
Questions for authors

None

2 comments

We thank the reviewer. All changes in the revised PDF are marked in blue.

W1. Unqualified general claims from a single task.

We agree. Our findings come from one task, and the paper now says so wherever a finding is stated:

  • Title. ICLR does not permit title changes during the review period, so the title is unchanged. If it can be changed for the camera-ready version, we will add the subtitle "A Case Study in Cancer Genomics".
  • Abstract and conclusion. Every finding now names its setting: subtype discovery on seven TCGA cohorts, four model families, two false-label pairs. "Not yet reliable enough to be left unsupervised" now ends "…on cancer subtype discovery".
  • Conclusion. We removed the general claim that reliability is a property of the model rather than of the prescribed procedure (Section 6), and Appendix A.22 now limits the corresponding statement to cancer subtype discovery.
  • Generality. The protocol itself is task-agnostic: a known-source arm, a blinded arm, a planted false premise, and exposure-conditioned denominators. Appendix A.22 (Applying the protocol to other tasks) shows how. We present it as a method validated on one task, not as a general finding about scientific agents.

W2. The instrument is validated on too little human data.

We agree. To strengthen the validation, we added two new human annotators to the premise audit and a new provenance audit.

  • Premise audit (Appendix A.11). The audit now has four annotators. They match the panel on which side prevailed in 17–19 of 19–20 traces, and on whether the label was tested in 8–17.
  • Provenance audit (Appendix A.12). This tests whether humans agree with the judges on recall versus derivation. Both new annotators put G0G_0 exploration at 0% and G2G_2 exploration at 7% and 40%, against the panel's 46%: they confirm the direction of the effect but not its size, so we now treat the explore rates as exploratory (Section 4.1, Limitations).

Thank you for the response. Scoping the claims does not make a single cancer-subtype study sufficient for a paper whose contribution is the protocol itself. The added annotators do not resolve the second concern either: with 4 annotators over 20 traces, agreement is too low and the number of annotators is still insufficient. I maintain my rating.

Reviewer_99uZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a protocol for evaluating scientific agents that separates answer quality from how conclusions are reached. Through identity disclosure, delayed disclosure, and planted false identities, the protocol assesses claim provenance, evidential support, and premise-testing behavior. The authors evaluate four models across seven TCGA cancer cohort. The results suggest that correct answers may arise from dataset recognition or acceptance of supplied labels, and that higher outcome scores do not necessarily indicate more reliable use of evidence.

Critical strengths
  1. This paper addresses an important question that a correct answer does not show whether an agent actually used the evidence. Separating answer quality from evidence use is useful for evaluating scientific agents.

  2. The experimental checks are careful. The authors audit identity leakage, correct exposure denominators, test sensitivity to different judges, and openly report negative results.

Critical weaknesses
  1. The contribution is not focus and clear to me. ALCE and L-CiteEval already study citation support and context fidelity, while there are also several recent works examine whether cited evidence was actually used. The paper does not discuss related literature in enough depth or provide comparisons showing what its protocol adds beyond existing evaluations.

.2 The core “derivation vs. recall” metric is not sufficiently validated. Process labels vary substantially across LLM judges, and provenance labels have no human validation. The human audit covers only 20 traces, with low agreement on whether a premise was tested. This leaves the central process claims weakly supported.

  1. The conclusions about scaffolding and discovery reliability are too strong. Not detecting a scaffold effect does not establish that reliability depends on the model rather than the procedure, especially when the prompts also differ in whether they request an identity. Results from one task and two false-label pairs should support narrower conclusions about the tested setting.
Questions for authors

n/a

1 comment

We thank the reviewer. All changes in the revised PDF are marked in blue.

W1. What the protocol adds beyond ALCE, L-CiteEval and attribution work.

Our contribution is a protocol that intervenes on what the agent is told, withholding the true identity or planting a false one, to determine whether an agent derived its answer from evidence it computed, recalled it from prior knowledge, or accepted it from a supplied premise.

ALCE and L-CiteEval test whether a fixed context supports a claim, and recent attribution and faithfulness work tests whether that context was actually used by intervening on the context (Cohen-Wang et al., 2024; Wallat et al., 2025). In our setting, the agent computes its own evidence, and prior knowledge can produce the same answer without it. A support check cannot separate the two, and our experiments show this:

  • Our support judge rates recalled and derived identities as "grounded" at similar rates: 71% vs 69%.
  • A judge-free marker rule gives the same picture, passing 66% vs 75%. It also passes 66% of episodes where the cancer was simply given.

Evidence support, even when checked without a judge, therefore does not show that an identity was derived from the data. What the protocol adds are the controls that make derivation measurable: a known-source arm, a blinded arm, a planted false identity, and exposure-conditioned denominators. Related Work (Section 2) now covers ALCE, L-CiteEval and attribution and faithfulness work, and Appendix A.9 reports this comparison.

W2. The derivation-vs-recall metric is not sufficiently validated.

We agree. We added two new human annotators, who labelled both the original premise audit (Appendix A.11) and a new provenance audit (Appendix A.12). Both put G0G_0 exploration at 0% and G2G_2 exploration higher (7% and 40%, against the panel's 46%), so humans confirm the direction of the G2G_2 effect but not its size. Overall, they agree with the panel's strategy label in 12/25 and 12/26 traces (κ=0.08\kappa = 0.08 and 0.040.04); most disagreements fall between "derived" and "recognized first", the same boundary on which the judges split. We now treat the explore rates as exploratory (Section 4.1, Appendix A.9, Limitations).

W3. The conclusions about scaffolding and reliability are too strong.

We agree that the scaffold conclusion was too broad. The staged prompt increased the outcome score by 0.043, and we detected no reduction in false-label adoption. The prompts also differed in whether they explicitly asked for the cancer identity, so the comparison cannot isolate the effect of scaffolding. We narrowed the claims as follows:

  • Conclusion (Section 6): removed "reliability is a property of the model rather than of the prescribed procedure"; the conclusion is now limited to the seven TCGA cohorts and the two false-label pairs tested, and agents are "not yet reliable enough to run unsupervised on cancer subtype discovery".
  • Abstract and Introduction: the procedure result now reads "no detectable reduction/change in false-premise adoption", and the findings name their setting (seven TCGA cohorts, two source–label pairs, "on cancer subtype discovery").
  • Appendix A.22 (Applying the protocol to other tasks): the general statement about what moved the balance is replaced by a task-specific one that notes the prompt confound.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.