acceptodds
Under review as a conference paper at ICLR 2027

Are Research Agents Fooled by Clever Hans? A Benchmark for Proactive Pitfall Detection in Medical Autonomous Research

Abstract

Research agents built on language models now design experiments, run code, and draft claims across long research workflows. Along the way they can introduce methodological errors that make a result look strong for the wrong reason, and in medical AI such a result can travel into a deployment claim with clinical consequences. It remains unclear whether these agents catch such errors on their own or, like the audience of Clever Hans, accept an answer because it looks right. We introduce MAP-Eval, a benchmark for proactive pitfall detection in medical autonomous research. Each of its 189 medical-AI workspaces contains one methodology pitfall drawn from a 33-type taxonomy, and the agent is never told that a flaw exists. MAP-Eval holds each pitfall fixed across nine harnesses, from single-turn review and a tool-using ReAct loop to review states captured from three autonomous-research frameworks. Across ten frontier models, static review reaches 66–70% mean identification rate, the six framework states reach 41–50%, and a shared review checkpoint after data preparation holds every model between 21% and 34%. Tool access raises some models by up to 26 points and lowers others by up to 37. Further analyses suggest that the bottleneck is applying methodological knowledge inside a workflow, not acquiring it. The same models that name a pitfall when handed the whole workspace overlook it when the evidence is spread across files, when they must find it with tools, or when the current stage asks them to write the result up as a claim. MAP-Eval provides a controlled testbed for measuring whether agents apply methodological knowledge where the workflow needs it, and a foundation for autonomous research that catches its own errors before they reach a clinical claim.

Primary area:
datasets and benchmarks
Keywords:
research agents, medical AI, methodological pitfalls, proactive error detection, benchmark

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SrCEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MAP-Eval, a benchmark for testing whether research agents can detect methodological flaws in medical AI projects without being told that a flaw exists. The authors construct 189 workspaces by injecting one flaw into each project and have human experts verify the flaws and their consequences. They evaluate ten models across nine harness settings, including full-context review, tool-based investigation, and review at different research stages. Models generally perform better with the full workspace provided, while tool use helps some models but hurts others.

Critical strengths
  1. The paper contributes a benchmark for detecting methodological flaws in medical AI research. Human experts check the constructed instances, and human annotations are used to validate the automated judge.

  2. The evaluation covers ten models and nine harness settings, allowing comparisons across models, tool use, and different stages of the research workflow.

Critical weaknesses
  1. Although the study focuses on medical AI, prior work has reported similar findings about methodological failures in autonomous research, the limitations of evaluating papers without checking supporting artifacts, and models’ difficulty detecting scientific errors [1,2,3]. The authors should compare these findings with their own results and discuss the similarities, differences, and possible reasons behind them. This would help clarify what the medical setting adds and provide a deeper analysis beyond reporting performance differences.

  2. Each model–instance–harness combination is evaluated using a single low-temperature generation, without repeated runs or uncertainty estimates. The authors should at least repeat evaluations on a representative subset to demonstrate that the observed performance differences are robust to sampling variability.

[1] Zhang et al. How Far Are We From True Auto-Research? 2026.

[2] Luo et al. The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems 2025.

[3] Xi et al. FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers 2025.

Questions for authors

See Weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer SrCE (Part 1/2)

We thank you for your insightful comments on the relationship to prior work and statistical robustness. We provide detailed responses below.

Q1: Prior findings and the medical setting

“The authors should compare these findings with their own results and discuss the similarities, differences, and possible reasons behind them. This would help clarify what the medical setting adds and provide a deeper analysis beyond reporting performance differences.”

A1: MAP-EVAL places these questions in medical autonomous research workflows, testing whether agents proactively identify problems without an error cue and judge whether the results support clinical deployment.

Prior studies show that code, data, and logs help identify errors in papers. Our experiments further hold the same methodological flaw fixed to observe whether agents actively seek and use this evidence at different research stages. Sections 2.5, 7 and Table 1 now include the following comparisons.

StudyConnection to our findingsWhat MAP-EVAL adds
ResearchArena, Zhang et al. [1]Artifact-aware review catches setup flaws, fabricated metrics, and code/paper mismatches missed by manuscript-only review.Identification of the same flaw varies across research stages: approximately 30% at Pre-Training Review (PTR), 48–50% at Post-Result Analysis (PRA), and 41–47% at Pre-Claim Drafting (PCD).
Luo et al. [2]Providing code and trace logs improves external detection of problems in AI-scientist outputs.Finding evidence through tools affects models differently: +25.9 points for Qwen and −37.0 points for Kimi.
FLAWS, Xi et al. [3]Controlled error injection tests whether explicitly prompted models can locate errors that invalidate a paper's claims.Runnable clinical workspaces test detection without disclosing that a flaw exists, together with false accusations and unflagged unsafe deployment.

The difference from Luo et al. can be understood through evidence acquisition. Their reviewer directly receives code, traces, and issue-specific detection rules; in L2, agents must locate relevant files through tools. Appendix H shows traces where agents accept successful execution or high metrics without examining the evidence containing the methodological problem. How evidence is provided and how agents investigate it both affect detection.

The medical setting requires clinical reasoning across files and a judgment about deployment. Checking patient-level independence requires connecting split code to repeated encounters, radiographic views, or ECG segments. Detecting scanner shortcuts requires relating acquisition metadata to labels; checking prevalence mismatch requires comparing the evaluation cohort with the target deployment population. These problems can coexist with successful execution and consistent logs.

Deployment decisions also provide information beyond identification rates. Claude Sonnet-4.6 identifies more flaws than Gemini-3.1-Pro (62% vs. 56%), yet has higher unflagged unsafe deployment (19% vs. 6%). How many problems an agent detects and whether it approves unreliable clinical claims therefore need separate evaluation. Sections 2.5, 3.3, 5.2 and 7 explain these requirements of the medical setting.

AuthorsOpenReview ↗

Response to Reviewer SrCE (Part 2/2)

Q2: Single-run evaluation and statistical uncertainty

“Each model–instance–harness combination is evaluated using a single low-temperature generation, without repeated runs or uncertainty estimates. The authors should at least repeat evaluations on a representative subset to demonstrate that the observed performance differences are robust to sampling variability.”

A2: The original experiments cover ten models, nine conditions, and 189 instances under uniform settings to evaluate proactive detection in medical research workflows and compare whether the same flaw is found under different conditions. The full results show consistent identification gaps across research stages and opposite tool effects for different models.

Your suggested repeated runs further test whether these patterns are sensitive to sampling and tool-trajectory variation. Thank you for this suggestion. We sampled two instances from each of the 33 pitfall types and ran three trials of full-workspace review, tool investigation, and PRA for three models, retaining the original decoding settings.

Consistent with the original observations, Qwen and Kimi retain the same directions of tool effects in every run, while Claude and Kimi consistently score lower in PRA than in full-workspace review. For NanoResearch's PRA stage (Nano-PRA), the gap from L1 is 32.3 points for Claude (95% paired bootstrap CI [18.7, 46.0]) and 23.2 points for Kimi (CI [10.1, 36.9]), both p < 0.001. Qwen's corresponding gap is 2.5 points (CI [−8.1, 13.1], p = 0.71).

We also ran paired bootstrap analyses on all 189 instances. Across ten models, L1 exceeds the six-framework-state mean by 22.7 points (95% CI [19.2, 26.2]) and PTR by 39.7 points (CI [36.0, 43.3]).

Repeated runs directly assess decoding and tool-trajectory variation; the full-instance analysis quantifies uncertainty in aggregate differences. The General Response summarizes the repeated-run table, with full results in Sections 3.2, 4.2–4.3 and Appendix M, Table 17.

References

[1] Zhang et al. How Far Are We From True Auto-Research? 2026.

[2] Luo et al. The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems. 2025.

[3] Xi et al. FLAWS: A Benchmark for Error Identification and Localization in Scientific Papers. 2025.

AuthorsOpenReview ↗

Following up on our responses and additional experiments

Dear Reviewer SrCE:

We sincerely appreciate your valuable time devoted to reviewing our manuscript. We would like to gently remind you of the approaching deadline for the discussion phase. We have diligently addressed the issues you raised in your feedback, providing detailed explanations and statistical analyses. For instance, we conducted three independent repeated runs across all 33 pitfall types and paired bootstrap analyses over all 189 instances, confirming that both the workflow-stage drops (a 22.7-point gap between full-workspace review and workflow states) and the model-specific tool effects are highly consistent against sampling variability (run-to-run SDs of 0.9%–2.3%). Moreover, we expanded Sections 2.5 and 7 and Table 1 to compare our findings with Zhang et al., Luo et al., and Xi et al., elucidating how tool-based evidence acquisition, cross-file clinical reasoning, and unsafe deployment decisions distinguish proactive pitfall detection in medical research. Would you kindly take a moment to look at it?

We are very enthusiastic about engaging in more in-depth discussions with you.

Reviewer_vW1fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a benchmark for evaluating research agents in medical research to see if it can identify planted errors.

Critical strengths
  • It addresses an important problem as agents are increasingly used for research.
Critical weaknesses
  • The models used are a bit outdated, but this is a difficult issue in the current state of AI.
  • It would be useful to know how much mitigation strategies such as adding skills can address the issues.
  • It would be useful to know how well Claude Code/Codex handle these issues.
Questions for authors

Minor suggestions: HookA, HookB are not meaningful terminologies. It would be useful to define them more clearly.

Other related work includes: Veritas: Towards a General-Purpose Replication Tool for Scientific Research by Liu et al. ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers? by Ye et al.

5 comments
AuthorsOpenReview ↗

Response to Reviewer vW1f (Part 1/2)

Thank you for the suggestions on coding agents, methodology skills, and related benchmarks. We provide detailed responses below.

Q1: Coding agents and skill-based mitigation

“It would be useful to know how much mitigation strategies such as adding skills can address the issues. […] It would be useful to know how well Claude Code/Codex handle these issues.”

A1: The original experiments hold the same methodological flaw fixed across three autonomous-research frameworks to test whether agents proactively detect problems at different stages. The results already show frequent missed flaws or approval of unsupported deployment claims even when diagnostic evidence is available.

Your suggestion extends this evaluation to coding agents used in practice and tests an intervention that can be directly adopted. Thank you for this suggestion. We evaluated Claude Code and Codex on the same 66 instances, with and without a clinical methodology skill.

The skill raises Claude Code's identification from 71.2% to 81.8% and Codex's from 65.2% to 77.3%. Unflagged unsafe deployment falls from 15.2% to 7.6% and from 12.1% to 6.1%, respectively, halving for both agents. This is consistent with the checklist directing checks toward patient grouping, metrics, and evidence across files.

Claude Code gains 10.6 points (95% paired bootstrap CI [3.0, 19.7], McNemar p = 0.039); Codex gains 12.1 points (CI [3.0, 21.2], p = 0.021). Raw extra-claim rates rise from 87.9% to 93.9% and from 83.3% to 90.9%; Table 20 reports these changes alongside identification and deployment decisions.

We used Claude Code v2.1.322 with Claude Sonnet-4.6 and Codex CLI v0.167.2 with GPT-5.4 (medium reasoning). These match the backbones in the L2 comparison, which scores 86.4% and 77.3% on the same instances.

The skill loads the same CLAIM/TRIPOD+AI checklist through each tool's project-skill mechanism, covering data splits, clinical confounders, metrics, comparators, and log evidence without revealing the planted flaw. Section 5.3 and Appendix P, Table 20 report the results; Appendix P provides the full skill text and execution settings.

Q2: Newer frontier models

“The models used are a bit outdated, but this is a difficult issue in the current state of AI.”

A2: We added Claude Opus 5.5 and GPT-6.1 Sol, and both newer models still miss many identifiable methodological problems within medical research workflows.

The evaluation was conducted in November 2026 on the same 66 stratified instances, covering full-workspace review, tool investigation, and three research stages: Pre-Training Review (PTR), and NanoResearch's Post-Result Analysis (PRA) and Pre-Claim Drafting (PCD).

ModelFull-workspace review (L1)Tool investigation (L2)Nano-PRANano-PCDPTR
Claude Opus 5.592.4%90.9%63.6%57.6%39.4%
GPT-6.1 Sol87.9%86.4%59.1%54.5%37.9%

Both models perform strongly in full-workspace review but still drop by approximately 29–35 points in PRA and PCD and remain below 40% in PTR. Proactive detection in medical research workflows therefore remains important to evaluate as models improve. These results are included in Sections 3.2, 4.2 and Appendix P.

AuthorsOpenReview ↗

Response to Reviewer vW1f (Part 2/2)

Q3: Stage terminology and replication benchmarks

“HookA, HookB are not meaningful terminologies. It would be useful to define them more clearly. […] Other related work includes: Veritas: Towards a General-Purpose Replication Tool for Scientific Research by Liu et al. ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers? by Ye et al.”

A3: We use stage names that directly describe their position and task:

  • PTR (originally HookA): a shared methodological review checkpoint inserted after data preparation and before model training.
  • PRA (originally HookB): analysis after the first experimental metrics become available.
  • PCD (originally HookC): judgment before drafting the manuscript or deployment claim.

We define these abbreviations at first mention and use them consistently in the text, figures, tables, and responses.

We also added comparisons with VERITAS [1] and ReplicationBench [2]. VERITAS checks whether code and data reproduce reported results; ReplicationBench evaluates end-to-end replication of published astrophysics analyses. Both connect closely to MAP-EVAL's checks of consistency among result tables, logs, and scientific claims.

Medical research also requires judging whether the experimental design supports clinical use. A patient-overlap split or scanner shortcut can consistently reproduce a high AUROC while failing to support the corresponding deployment claim. Detecting these problems requires connecting the cohort, code, and target population. Sections 2.5, 7 and Table 1 explain these connections.

References

[1] Liu et al. VERITAS: Towards a General-Purpose Replication Tool for Scientific Research. 2026.

[2] Ye et al. ReplicationBench: Can AI Agents Replicate Astrophysics Research Papers? 2025.

AuthorsOpenReview ↗

Following up on our rebuttal

Dear Reviewer vW1f:

We sincerely appreciate your valuable time devoted to reviewing our manuscript. We would like to gently remind you of the approaching deadline for the discussion phase. We have diligently addressed the issues you raised in your feedback, providing detailed explanations and new experiments. For instance, we evaluated Claude Code and Codex with and without a clinical methodology skill (CLAIM/TRIPOD+AI), showing that the skill boosts identification from 71.2% to 81.8% for Claude Code and from 65.2% to 77.3% for Codex while cutting unflagged unsafe deployment in half for both agents. Moreover, we evaluated newer frontier models (Claude Opus 5.5 and GPT-6.1 Sol)—which reach 92.4% and 87.9% in full-workspace review but still drop by 29–35 points within research workflow stages—and replaced HookA/B/C with descriptive stage names (PTR, PRA, PCD) alongside discussions of VERITAS and ReplicationBench. Would you kindly take a moment to look at it?

We are very enthusiastic about engaging in more in-depth discussions with you.

Reviewer_h9XLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether an LLM research agent notices a methodological flaw in work it inherits, when nobody tells it to look. It introduces MAP-EVAL, 189 medical-AI workspaces — code, cohort manifest, result table, run logs, and a PI memo — each containing exactly one planted pitfall drawn from a 33-type taxonomy spanning methodological validity, reporting integrity, and scientific honesty. Instances are built on five clinical datasets covering imaging, tabular EHR, and ECG.

The central design choice is that the agent is never told a flaw exists, and that the clean workspaces are held to contain no other pitfall types, so the planted flaw is the only true positive and every other asserted pitfall counts as a false positive. The same pitfall is then held fixed across nine harnesses: single-turn review of the whole workspace (L1), a 15-turn ReAct loop that must locate evidence with tools (L2), six state snapshots captured from three autonomous-research frameworks at the after-first-result and before-writing stages, and HookA, a skeptical-review checkpoint the authors insert after data preparation.

Ten frontier models are evaluated with a GPT-5.4 judge, validated against two clinical-ML annotators (κ = 0.82 against their consensus) and re-run with a second judge (91.6% agreement; micro-average 49.7% → 49.2%). Headline results: static review reaches 70% and 66% mean identification, the six framework states 41–50%, and HookA holds every model between 21% and 34%. Tool access raises one model by 26 points and lowers others by 27 and 37. Extra-claim rates run 67–95% across all models, and the best identifier also has the highest. The authors conclude the bottleneck is applying methodological knowledge inside a workflow rather than possessing it.

Critical strengths

The single-pitfall design turns false positives into a measurable quantity, and that pays off in the paper's most useful result (§2.3, §5.1, Table 3). Because each instance contains exactly one planted flaw and the clean workspace is held to contain no others, every additional asserted pitfall is a scored false positive, and each instance acts as its own negative control for the other 32 types. This is what lets the paper show that extra-claim rates sit at 67–95% for every model and that the strongest identifier (Claude Sonnet-4.6, 62% identification) also has the highest extra-claim rate (95%), with no model in the high-identification/low-false-positive corner. Most critique benchmarks report only detection, which rewards indiscriminate flagging; this design makes the tradeoff visible and is the main reason the paper's numbers mean more than a detection rate alone.

Holding the pitfall fixed while varying only the harness isolates the question the paper actually wants to answer (§3.1, Table 2). Because the same flaw is diagnosable from what the model sees in every condition, the spread from 70% (L1) down to 30% (HookA) is attributable to how the evidence is presented and what the stage asks the model to do, rather than to differences in task difficulty or in earlier agent steps. HookA is the sharpest instance: every one of the ten models lands between 21% and 34%, a tight band that holds across closed and open models and across the whole quality ranking. That is a genuine headroom result and a more informative claim than a leaderboard.

Critical weaknesses

Every cell in the main results is a single generation, including the finding the paper makes most of (§3.2, §4.3). Each model generates once per instance at low temperature, so Table 2 carries no variance estimate. This matters most for the "tool paradox": investigation raises Qwen3.5-397B by 26 points but lowers Kimi-K2.6 by 37 and Gemini-3-Flash by 27, and this is the paper's most striking result. A 15-turn tool loop is where run-to-run variation is plausibly largest, and the paper's own trace analysis (§4.3, App. H) attributes the losses partly to whether the agent opens the file that holds the pitfall, which can differ between runs of the same model. Without repeated runs on at least the L1/L2 subset, a swing of this size cannot be separated from trajectory noise, and the model ordering in Table 3 inherits the same uncertainty. This is fixable, since repeated runs on a subset with dispersion reported would settle it, but as submitted the headline comparative claims rest on single draws.

The false-positive measurement rests on an assumption about the clean workspaces that is stated but not verified (§2.2, §2.3, §3.3). An "extra claim" is any specific methodology flaw a model asserts other than the planted one. The judge lists those assertions and counts each as a false positive, without checking whether the assertion is true of the workspace. That is only sound if the workspace contains no other flaw. The paper describes curating clean workspaces against reporting guidelines, and it has two clinical-ML researchers plus a senior adjudicator review all 189 instances. But the stated purpose of that review is to confirm the planted pitfall invalidates the deployment claim and to check the consequence; the leak audit checks that the label is not exposed. Neither is described as checking that no second flaw is present. Realistic 1–3k-row clinical workspaces, one third of them built from synthetic SYNTHEA data, are not obviously free of all 32 other pitfall types, and a correct observation about the cohort or the baseline would be scored as a false positive. With extra-claim rates of 67–95% for every model, the §5.1 conclusion that no model reaches the high-identification, low-false-positive corner depends on this. It is either evidence of over-flagging or partly an artifact of how the negatives were built. Independent annotation of a sample of clean workspaces for non-planted flaws, and a precision estimate for extra claims, would settle it.

Questions for authors
  1. Clarification. Was any sample of clean workspaces independently annotated for flaws other than the planted one? If a sample of extra claims were adjudicated against the workspace, what fraction were correct observations rather than false positives?
  2. Clarification. Were any instances run more than once? Repeated L1 and L2 runs for two or three models, with the spread reported, would show whether the tool-paradox swings exceed trajectory variance.
  3. Clarification. HookA uses one shared prompt adapted from NanoResearch, whereas the other six states keep each framework's native prompt. How much of HookA's 30% reflects the stage rather than that prompt? Running HookB or HookC with the shared prompt would separate the two.
  4. Clarification. Two closely related works are not cited. Luo, Kasirzadeh and Shah (2025, arXiv:2509.08713) study data leakage and metric misuse in AI-scientist systems and find that trace logs and code enable far more effective detection than the final paper alone. Li et al. (EMNLP 2025 Findings, arXiv:2505.23715) formalize proactive versus explicitly prompted critique of flawed premises and find that most models rely on explicit prompts. How do the "no prior evaluation" statement in §2.5 and the framing of proactive detection relate to them?
  5. Clarification. How were the clean workspaces produced (human-authored, templated, or LLM-generated), and is there any comparison between behavior on injected pitfalls and on real flawed projects?
5 comments
AuthorsOpenReview ↗

Response to Reviewer h9XL (Part 1/3)

We thank you for your insightful comments. Detailed responses are provided below.

Q1: Independent workspace verification and extra claims

“Was any sample of clean workspaces independently annotated for flaws other than the planted one? If a sample of extra claims were adjudicated against the workspace, what fraction were correct observations rather than false positives?”

A1: We added independent checks of the base workspaces and human adjudication of extra claims. After verification, 71.1% of responses still contain false accusations.

Two clinical-ML researchers independently checked 35 pre-injection workspaces under blinded conditions, covering imaging, EHR, and ECG. They verified 33/35 (94.3%) as free of additional taxonomy flaws. Two bases contained secondary issues affecting 7/189 instances. Excluding these instances changes the ten-model mean raw extra-claim rate from 84.2% to 83.8%, a 0.4-point decrease.

For extra claims, we sampled two complete responses from distinct instances in each model–harness cell, totaling 180 responses, stratified across pitfall families and datasets. Two researchers adjudicated all 432 extra claims against the workspace files:

  • 337 claims (78.0%) were model false accusations, such as alleging patient leakage despite patient-grouped splits or claiming that an existing comparator was missing.
  • 63 claims (14.6%) were judge over-extractions of generic suggestions, such as recommending prospective validation.
  • 32 claims (7.4%) were accurate secondary observations: five identified the additional workspace issues above, and 27 noted disclosed study conditions outside the pitfall taxonomy.

The same sample has a raw extra-claim rate of 152/180 = 84.4% and a human-verified false-accusation rate of 128/180 = 71.1% (95% Wilson CI [64.1%, 77.2%]). Model-level verified rates remain positively correlated with identification (r = 0.75, ρ = 0.66).

Human verification therefore retains the paper's observation that stronger identifiers can also frequently assert problems that are absent from the workspace. The revision distinguishes raw extra claims from human-verified false accusations and reports the checks and full results in Sections 2.3, 3.3, 5.1 and Appendix N, Table 18.

Q2: Do tool effects exceed run-to-run variation?

“Were any instances run more than once? Repeated L1 and L2 runs for two or three models, with the spread reported, would show whether the tool-paradox swings exceed trajectory variance.”

A2: Yes. The tool effects in repeated runs substantially exceed run-to-run variation, and their directions persist in every run.

We ran three independent trials for Qwen, Claude, and Kimi on 66 instances covering all 33 pitfall types, retaining the original decoding settings and L2's 15-turn tool budget. L1 provides the full workspace directly; L2 requires the model to find relevant evidence through tools.

The L2−L1 difference is +24.7 ± 3.2 points for Qwen (95% paired bootstrap CI [+11.6, +38.4]) and −37.4 ± 3.8 points for Kimi (CI [−50.5, −23.7]), both p < 0.001. Claude's difference is −1.0 ± 1.7 points (CI [−6.6, +5.1], p = 0.78).

Across the three runs, Qwen's difference stays between +21.2 and +27.3 points and Kimi's between −33.3 and −40.9 points, compared with L2 run SDs of 1.5–2.3 points. Agreement across all three runs on whether the pitfall-bearing file was opened is 83.3%–92.4%.

The repeated-run table in the General Response gives the means and SDs by condition. Sections 3.2, 4.3 and Appendix M, Table 17 report run counts, paired differences, and confidence intervals.

AuthorsOpenReview ↗

Response to Reviewer h9XL (Part 2/3)

Q3: Workflow stage or prompt wording?

“HookA uses one shared prompt adapted from NanoResearch, whereas the other six states keep each framework’s native prompt. How much of HookA’s 30% reflects the stage rather than that prompt? Running HookB or HookC with the shared prompt would separate the two.”

A3: The original experiments retain each framework's native instructions at the Post-Result Analysis (PRA) and Pre-Claim Drafting (PCD) stages across the three research frameworks, to test whether agents proactively identify methodological problems at these points in the workflow; Pre-Training Review (PTR) is an additional shared checkpoint. Holding the same flaw fixed, identification still changes substantially across workflow states, motivating evaluation at different stages of the research process.

Your suggested shared-prompt comparison further separates wording from workspace state and available evidence. Thank you for this suggestion. We applied the exact PTR prompt to all 189 NanoResearch PRA and PCD snapshots (Nano-PRA and Nano-PCD) for four models.

Consistent with the original results, identification still rises by 22.5 points from PTR to PRA under the same prompt, from 30.8% to 53.3% (95% CI [17.5, 27.9]). Holding the PCD workspace fixed and replacing its native prompt with the shared review prompt raises identification by 4.5 points on average (CI [1.5, 7.7]), including an 11-point gain for Qwen.

Substantial workflow-state differences persist under identical wording, and the fixed-state comparison shows that prompting also affects proactive checks. Sections 4.4, 5.3 and Appendix O, Table 19 report both comparisons and the model-level results.

Q4: Relation to Luo et al. and Li et al.

“How do the ‘no prior evaluation’ statement in §2.5 and the framing of proactive detection relate to them?”

A4: We added both studies and explain their connection to MAP-EVAL in Section 2.5 and Table 1, replacing the broad ‘no prior evaluation’ statement.

Luo et al. [1] study benchmark selection, data leakage, metric misuse, and post-hoc selection bias in AI-scientist systems, and find that providing code and trace logs improves external detection. Li et al. [2] study whether models proactively challenge flawed conversational premises and find that explicit prompting often plays an important role in eliciting critique.

MAP-EVAL places proactive detection in runnable medical research workspaces: agents must connect code, cohort information, results, and logs to identify problems without being told that a flaw exists. Holding the same problem fixed across workflows lets us observe when agents seek and use diagnostic evidence.

This also explains the connection to Luo et al. Providing reviewers with code and logs helps detection; in our tool-investigation condition, agents must also locate the relevant files themselves. On the same workspaces, this condition raises Qwen's identification by 25.9 points and lowers Kimi's by 37.0 points. Appendix H gives traces where models accept successful execution or high metrics without checking the methodological evidence.

The medical setting also requires agents to judge whether the problems affect clinical deployment. Patient overlap compromises evaluation independence, while scanner shortcuts can prevent high metrics from supporting use in the target population. We therefore measure identification, false accusations, and approval of deployment without detecting the flaw. Sections 2.5, 7 and Table 1 explain these connections and distinctions.

AuthorsOpenReview ↗

Response to Reviewer h9XL (Part 3/3)

Q5: Workspace provenance and naturally flawed projects

“How were the clean workspaces produced (human-authored, templated, or LLM-generated), and is there any comparison between behavior on injected pitfalls and on real flawed projects?”

A5: The workspaces originate from 19 human-authored, runnable clinical-ML pipelines across five datasets, parameterized into 54 base workspaces. They include patient/encounter-grouped splits, training scripts, clinical metrics, comparator specifications, and execution logs from 1–3k-row cohorts. LLMs drafted narrative text in the research-goal and deployment-memo files, which human annotators checked against the code and logs.

The evaluation uses controlled injected flaws; a separate quantitative comparison with naturally flawed repositories was not conducted. Controlled injection keeps the same methodological problem fixed across research frameworks, allowing comparison of how evidence presentation, tool investigation, and workflow stage affect detection. The injected problems instantiate failures documented in clinical-ML studies [3–6].

These workspaces support evaluation in NanoResearch, AI-Scientist-v2, EvoScientist, Claude Code, and Codex. Section 2.2 and Appendices C and N describe their construction and independent verification.

References

[1] Luo et al. The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems. 2025.

[2] Li et al. Don't Take the Premise for Granted: Evaluating the Premise Critique Ability of Large Language Models. EMNLP Findings, 2025.

[3] Roberts et al. Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans. Nature Machine Intelligence, 2021.

[4] Varoquaux & Cheplygina. Machine learning for medical imaging: methodological failures and recommendations for the future. npj Digital Medicine, 2022.

[5] Wynants et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ, 2020.

[6] Kapoor & Narayanan. Leakage and the reproducibility crisis in machine-learning-based science. Patterns, 2023.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their careful reading and for recognizing the importance of proactive pitfall detection, the single-pitfall design, and comparisons across research workflows.

MAP-EVAL asks whether research agents can proactively identify methodological problems in medical autonomous research workflows. We construct 189 runnable medical research workspaces and evaluate whether agents find the problems, assess the evidence, and decide whether the results support clinical deployment, without telling them that a flaw exists. Holding the same flaw fixed across conditions lets us compare how evidence presentation, tool use, and workflow stage affect proactive detection.

The reviewers' suggestions strengthen our checks of repeatability, the validity of extra claims, and practical use. The added experiments and explanations are included in the revised manuscript and marked in blue.

1. Repeated runs retain the main findings (h9XL Q2; SrCE Q2)

We sampled two instances from each of the 33 pitfall types and ran three independent trials for three models under three conditions, retaining the original decoding settings and tool budget. L1 gives the model the full workspace for review; L2 allows up to 15 tool-use turns to find evidence; Nano-PRA evaluates the model at NanoResearch's Post-Result Analysis (PRA) stage, after its first experimental results. Identification rates are three-run mean ± SD.

ModelFull-workspace review (L1)Tool investigation (L2)Post-result analysis (Nano-PRA)
Qwen3.5-397B48.5% ± 1.5%73.2% ± 2.3%46.0% ± 0.9%
Claude Sonnet-4.688.9% ± 0.9%87.9% ± 1.5%56.6% ± 0.9%
Kimi-K2.678.8% ± 1.5%41.4% ± 2.3%55.6% ± 0.9%

Every run reproduces the positive tool effect for Qwen and the negative effect for Kimi, with changes substantially larger than run-to-run variation. Claude and Kimi also consistently score lower in post-result analysis than in full-workspace review.

Across all 189 instances and ten models, L1 averages 69.6% and the six framework states average 47.0%, a 22.7-point gap (95% paired bootstrap CI [19.2, 26.2]). The gap between L1 and Pre-Training Review (PTR) is 39.7 points (CI [36.0, 43.3]). Full results appear in Sections 3.2, 4.2–4.3 and Appendix M, Table 17.

2. False accusations remain common after human verification (h9XL Q1)

Two researchers independently checked 35 of the 54 base workspaces and adjudicated all 432 extra claims from 180 complete responses. Of these claims, 78.0% were model false accusations, 14.6% were generic suggestions over-extracted by the judge, and 7.4% were accurate secondary observations.

After human verification, 71.1% of responses still contain false accusations (95% Wilson CI [64.1%, 77.2%]). Model-level rates range from 55.6% to 83.3% and correlate positively with identification, supporting the need to evaluate both detection and false accusations. The revision distinguishes raw extra-claim rates from human-verified false-accusation rates (Sections 2.3, 3.3, 5.1; Appendix N, Table 18).

3. Workflow-state differences persist under the same prompt (h9XL Q3)

We applied the exact PTR prompt to all 189 NanoResearch PRA and Pre-Claim Drafting (PCD) snapshots for four models, adding 1,512 evaluations.

With the prompt fixed, identification rises from 30.8% before training to 53.3% after the first results, a 22.5-point increase (95% CI [17.5, 27.9]). Holding the pre-claim workspace fixed and changing only the prompt raises identification from 47.8% to 52.3%, a 4.5-point increase (CI [1.5, 7.7]). These comparisons test workflow-state differences under identical wording and prompt effects within a fixed state (Sections 4.4, 5.3; Appendix O, Table 19).

4. Coding agents, methodology skills, and newer models (vW1f Q1–Q2)

On the same 66 instances, we evaluated Claude Code and Codex with a CLAIM/TRIPOD+AI methodology skill that directs checks of patient grouping, clinical confounders, metrics, comparators, and log evidence.

AgentIdentification: default → skillUnflagged unsafe deployment: default → skill
Claude Code71.2% → 81.8%15.2% → 7.6%
Codex65.2% → 77.3%12.1% → 6.1%

The skill improves identification by 10.6 and 12.1 points and halves unflagged unsafe deployment for both agents. Raw extra-claim rates also rise by 6.1 and 7.6 points (Section 5.3; Appendix P, Table 20).

We also evaluated Claude Opus 5.5 and GPT-6.1 Sol. They reach 92.4% and 87.9% in full-workspace review, but drop by approximately 29–35 points in post-result or pre-claim states and remain below 40% before training. Strong detection ability does not automatically translate into proactive checks within research workflows.

The revision also compares related research and replication benchmarks and explains workspace provenance and the workflow stages.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.