- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces BRIDGE-Bench, a 38-question expert-curated benchmark evaluating LLM agents on realistic drug discovery and development tasks across the therapeutic pipeline (e.g., ADME, QSP, retrosynthesis, ADC design, CMC). Evaluating 13 harness–model configurations (including Claude Code and Codex) on answer quality and trajectory execution, the authors identify key agent bottlenecks in biological reasoning, workflow execution, evidence integration, and reproducibility.
Critical strengths
- Practical, pipeline-wide coverage
- Rich failure analysis
Critical weaknesses
W1. Independent tasks rather than pipeline integration: Tasks are standalone stage-specific problems rather than interconnected multi-stage workflows where decisions propagate downstream, undermining claims of true pipeline-level evaluation.
W2. Small scale and imbalanced distribution: With only 38 questions, distribution is heavily skewed (e.g., drug combination screening comprises ~26%, while several domains have only 1 question), making aggregate metrics unreliable across sparse categories.
W3. Unvalidated LLM judge: Relies on gpt-5.6-sol for rubric grading without proving alignment with human expert evaluations or controlling for self-evaluator bias (since gpt-5.6-sol is also a subject model).
W4. Single-run stochasticity: Evaluates each configuration only once per question, omitting confidence intervals or significance tests despite close score gaps and high agent execution variance.
W5. Lack of benchmark accessibility and reproducibility.
The paper does not provide a publicly accessible repository or a clear data release plan for BRIDGE-Bench. Although Appendix A summarizes the 38 benchmark tasks, the complete task instructions, input artifacts, reference answers, grading rubrics, and evaluation scripts are not made available in the manuscript. Without these resources, independent researchers cannot reproduce the reported evaluations or use the benchmark to assess new agent systems. The authors should clarify the benchmark release plan and ideally provide the complete evaluation resources.
Questions for authors
Provide above