acceptodds
Desk rejected at ICLR 2027

DORA: Discovery-Oriented Agent for Drug Repurposing

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Scientific discovery requires connecting fragmented evidence to construct and verify hypotheses that are not explicitly documented. We tackle this challenge through a case study on drug repurposing: identifying new therapeutic uses for existing drugs by connecting evidence across molecular targets, pathways, phenotypes, biological processes and disease mechanisms. We construct a comprehensive benchmark from approximately 34k clinical trials and 60k drug–disease pairs. We introduce DORA, a Discovery-Oriented Agent for drug repurposing, guided by an agentic environment-driven paradigm for scientific exploration. We construct a comprehensive biomedical knowledge network consisted of 273K nodes and 16.22M edges from a massive amount of heterogeneous databases, literature, and scientific tools, and design a novel algorithm, Meta-Pathfinder, to uncover non-trivial, implicit mechanistic paths connecting drugs to diseases. Rather than executing a predefined workflow, an LLM operates through a general- purpose coding-agent harness to interact dynamically with this environment: invoking Meta-Pathfinder and scientific tools, traversing evidence, identifying missing links, challenging competing hypotheses, and iteratively refining therapeutic hypotheses. Across seven frontier models, DORA improves five LLM-only baselines, with gains of 2.45–28.09 normalized-score points and 1.00–16.00 percentage points in categorical accuracy; two backbones do not improve. Notably, smaller models equipped with our agentic environment can outperform substantially larger models without it. To our knowledge, this is the first systematic framework for agentic drug repurposing, moving beyond retrieving and reasoning over existing knowledge to connecting fragmented evidence, uncovering latent mechanisms, and generating novel therapeutic hypotheses. We will publicly release our platform, data, and resources to support reproducible research.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Agentic AI, Drug Repurposing, AI for Science

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_7FvBOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce DORA - an agent for drug repurposing supplied with a biomedical knowledge-graph, Meta-Pathfinder, an algorithm that performs graph search to find nontrivial drug-disease paths, and scientific tools such as docking, protein design tools etc. LLM-only approach is compared with DORA across 7 LLMs using continuous clinical trial success-based scores and categorical accuracy. DORA improves five of seven LLMs and surpasses biomedical AI agent Biomini. The authors demonstrate 3 positive examples of DORA results, discuss the observed failure modes and suggest that the knowledge-rich environment itself can be a key scaling axis for scientific discovery rather than scaling models.

Critical strengths
  1. The idea is practically important. The authors try to build a system that produces verifiable hypotheses through a transparent pipeline.
  2. The paper is of high quality, it is easy to follow, the experimentation is rigorous.
Critical weaknesses
  1. The paper claims that DORA discovers previously unrecognized drug–disease mechanisms. The three qualitative examples in Figure 5 illustrate biologically plausible connections, but do not establish that these mechanisms were previously unknown and that they are actually potent and useful. Probably, a small human expert study across several experts could strengthen this claim.
  2. The proposed metric uses asymmetric losses for ongoing and completed trials. For ongoing trials, predictions above the currently achieved phase incur no loss. This potentially rewards overly optimistic predictions, especially given the large proportion of ongoing trials in the dataset. For example, an ongoing Phase II trial can receive the maximum score even when the agent predicts eventual approval without sufficient evidence.
  3. The biomedical knowledge graph, presented as a central component of DORA, has a negative estimated contribution according to ablation studies. This requires more clarification because it undermines the conclusion that knowledge-rich environment itself can be a key scaling axis for scientific discovery rather than scaling models.
Questions for authors
  1. How do the authors distinguish genuinely novel discoveries from connections already documented in the literature? Could the authors conduct a small-scale evaluation involving several independent biomedical experts to assess the novelty, mechanistic plausibility, and potential therapeutic utility of the generated hypotheses?
  2. How do the authors explain the result that knowledge graph has negative contribution to DORA?
  3. How do the authors justify the proposed scoring scheme, and to what extent might it inflate the reported performance?
4 comments
AuthorsOpenReview ↗

Thank you and our response

Thank you for recognizing the practical importance of DORA, the transparency of its pipeline, and the rigorousness of our evaluation. We appreciate your questions about mechanism discovery, asymmetric labels, and the Biomedical KG analysis.

Weakness 1 and Question 1: Mechanism discovery and expert evaluation. DORA is a system that provides evidence for any pair of drug candidates and diseases. Limited by available ground truth data, we are only able to evaluate based on existing trials and results. That said, the system can be deployed for any open pair of drug candidates and diseases. We have also been actively collaborating with users to discover new drug candidates that have never been discovered by humans, even though their evaluation will take much longer and rely on real-world experiments. In Section 4.3, we provide a few examples of such efforts, and Appendix D presents one complete pairwise DORA report with its force-directed Meta-Pathfinder visualization.

We appreciate the advice on letting human experts evaluate the insights and hypotheses provided by DORA. We asked two human evaluators to complete human evaluation of 19 DORA reports. For each drug–disease pair, evaluators first independently searched literature available on or before July 10, 2026 and froze a literature-based baseline. They then read the DORA report and assessed its strongest incremental insight for novelty beyond the baseline, biological plausibility, and therapeutic utility. On average, they spent 33.2 minutes manually searching the literature and identified 2.1 relevant literature or registry sources per pair. 14 of the 19 hypotheses were rated at least “plausible with gaps,” including nine rated “coherent and supported” or “supported by a strong convergent rationale.” Among them, 3 were also rated as containing a “specific new insight” or a “clearly distinct and consequential insight”: delgocitinib for vitiligo, through a candidate melanocyte-intrinsic mechanism involving inhibition of laminin-332-induced melanocyte dedifferentiation; linsitinib for thyroid eye disease, by connecting clinical response with IR-beta cross-inhibition and hyperglycemia; and ravulizumab for IgA nephropathy, through a candidate C5–EDNRA connection extending the complement-centered mechanism. All three also received high therapeutic-utility ratings and were considered worthy of further exploration or experimental verification. These results provide promising signals for DORA on this extremely challenging problem and illustrate how the system may inspire researchers to explore mechanistic gaps that could otherwise be overlooked within the vast volume of biomedical knowledge. We are continuing working on a larger human evaluation throughout the system deployment process, and might also update the results.

Weakness 2 and Question 3: Asymmetric scoring and optimistic predictions. We appreciate the question and the chance to clarify our metric design. In our evaluation set, whether a question has an exact label or a lower-bound label is hidden from every model, so a model cannot cheat by selectively over-reporting scores only on lower-bound labels, but not on exact label queries. Additionally, we provide a factor that upcurves the overconfident branch in exact label loss, so that in expectation, a blind model cannot gain more by overreporting scores in an uninformed way. We provided more details in Section 2.

Weakness 3 and Question 2: Biomedical KG coefficient. We appreciate the question and the chance to clarify the observation better. Knowledge graphs (KGs) are included in the system to provide more long-established, historically recognized human knowledge of the involved entities. They are usually built from older sources than rapidly updated literature, so they might provide information that is less up-to-date. This is reflected as a non-strictly-positive effect in the quantitative evaluation when the KG is used alone to predict the latest clinical trials. In the corrected additive Bayesian model, the KG effect is +1.43+1.43 normalized-score points (95% CrI [−6.36,9.21][-6.36, 9.21], P(β>0)=0.64P(\beta>0)=0.64). We keep this factor for multiple reasons: First, we envision that future versions of the system could potentially address this and reconcile the trade-off between knowledge credibility and recency; Second, knowledge graphs provide a network to which the latest literature can be anchored, providing more comprehensive and better-connected pathfinding results. Our human experts in the team still appreciate the value of knowledge graphs, so from the perspective of producing the most useful product for this challenging quest, we don’t want to remove that component at an early stage of the evolving system solely due to the negative effect in preliminary quantitative results.

Thank you again for the detailed questions. We would appreciate your follow-up comments and, if your concerns are addressed, an updated score.

Thank you for the clarifications! I am pleased with the answers. I have one additional question. Why is comparison with baselines limited to Biomini? Did you consider including other baselines that are mentioned in Related Work section?

AuthorsOpenReview ↗

Author response

We thank the reviewer for this helpful question. The baseline with the closest setting to our problem is K-Paths, which retrieves up to top-10 3-hop paths from a Hetionet graph augmented with training interactions, and primarily supports relation classification between entities represented in that graph. DORA instead accepts arbitrary drug–disease pairs and searches across independently maintained knowledge and literature providers with provenance-preserving integration. We implement K-Paths with its original graph, relation semantics, and retrieval settings. However, on our public-100 benchmark, K-Paths retrieved 0/100 paths covering both entities (drug candidates and diseases). If relaxed to one-endpoint coverage, it resolves for 32/100 pairs (28 drug and 4 disease endpoints). Running this model would be theoretically equivalent to the LLM-only baseline. This shows the challenge of our problem setting and a unique advantage of our method.

We hope this addresses your concersn and look forward to your final version of scores.

Thank you. I will increase my rating.

Reviewer_b43UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DORA equips an LLM coding agent with biomedical knowledge graphs, a literature Wiki, cross-provider path search, and scientific tools for drug repurposing. On a 100-pair development set derived from clinical trials, it improves outcome prediction over LLM-only baselines for five of seven backbones.

Critical strengths
  1. Useful and traceable biomedical infrastructure. The environment integrates heterogeneous biomedical databases and literature through shared entity resolution and retrieval interfaces, while retaining source provenance (§3.1). This provides a practical basis for inspecting the evidence behind an agent's assessment.

  2. Clinically grounded evaluation across multiple models. Labels are derived from clinical trial records and reviewed by a human expert (§4.1). The main experiments evaluate seven backbones over three runs, including models that do not improve (Table 4). This helps assess the environment's utility and how its benefits vary across models.

Critical weaknesses
  1. Limited methodological contribution to drug repurposing. The motivation in §3—reducing the effort of locating and connecting evidence—is generic to tool-using agents. Rephetio [1] already uses typed graph paths, K-Paths [2] supplies retrieved paths to LLMs, and DrugAgent [3] integrates predictive models, knowledge graphs, and literature. DORA's cross-provider interfaces primarily improve access to biomedical evidence. What remains unclear is which specific limitation of prior repurposing methods this design overcomes. The experiments report overall prediction gains, but do not link these gains to overcoming an identified limitation of existing methods.

  2. Discovery claims are not matched by the evaluation. The abstract claims generation of novel therapeutic hypotheses, and the introduction claims previously unrecognized drug–disease connections, explicitly referring to §4.3 (lines 093–094). However, the quantitative task predicts clinical progress for supplied pairs already investigated in clinical trials. Figure 5 also revisits associations with pre-cutoff clinical precedents: a randomized trial of dupilumab for allergic rhinitis [4]; FDA approval of pembrolizumab plus chemotherapy for advanced HER2-negative gastric/GEJ adenocarcinoma [5]; and a phase II trial of ravulizumab for IgA nephropathy [6]. These precedents leave open the possibility of new mechanistic explanations, but §4.3 does not identify what is newly inferred beyond them. The evidence for generating new therapeutic hypotheses therefore remains insufficient.

[1] Himmelstein, D. S., et al. Systematic integration of biomedical knowledge prioritizes drugs for repurposing. eLife, 6:e26726, 2017. doi:10.7554/eLife.26726.

[2] Abdullahi, T., et al. K-Paths: Reasoning over Graph Paths for Drug Repurposing and Drug Interaction Prediction. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, pp. 5–16, 2025. doi:10.1145/3711896.3737011.

[3] Inoue, Y., Song, T., and Fu, T. DrugAgent: Explainable Drug Repurposing Agent with Large Language Model-based Reasoning. arXiv:2408.13378v1, 2024.

[4] Corren, J., et al. Short-Term Subcutaneous Allergy Immunotherapy and Dupilumab are Well Tolerated in Allergic Rhinitis: A Randomized Trial. Journal of Asthma and Allergy, 14:1045–1063, 2021. doi:10.2147/JAA.S318892.

[5] U.S. Food and Drug Administration. FDA approves pembrolizumab with chemotherapy for HER2-negative gastric or gastroesophageal junction adenocarcinoma. Drug approval announcement, November 16, 2023.

[6] Lafayette, R., et al. Efficacy and Safety of Ravulizumab in IgA Nephropathy: A Phase 2 Randomized Double-Blind Placebo-Controlled Trial. Journal of the American Society of Nephrology, 36(4):645–656, 2025. doi:10.1681/ASN.0000000534.

Questions for authors
  1. Which concrete limitation of existing drug-repurposing methods does the proposed design address, and what experiment demonstrates its advantage on that limitation?
  2. In lines 093–094, does novelty refer to a new graph route, a new therapeutic hypothesis, or a new biological mechanism? Which example demonstrates that novelty relative to prior literature?
2 comments
AuthorsOpenReview ↗

Thank you and our response

Thank you for recognizing DORA's traceable biomedical infrastructure, source provenance, and evaluation across seven backbones. We appreciate your questions and address them as follows:

Weakness 1 and Question 1: Contribution relative to Rephetio, K-Paths, and DrugAgent. Thanks for listing these related literature. Rephetio provides typed paths over an integrated graph, K-Paths supplies graph paths to an LLM, and DrugAgent combines predictive models, knowledge graphs, and literature. Different from existing work, DORA advances this line of work with cross-provider search over independently maintained resources that use different identifiers, schemas, and evidence semantics. More specifically, DORA searches these providers at query time without first collapsing them into one graph, and preserves each hop's provider, identifier, evidence type, and source, allowing the model and users to inspect cross-provider paths and incomplete coverage. Additionally, DORA extracts and integrates claims from raw literature into the graph, connecting entities linked by recent studies and expanding the knowledge boundary beyond static, established knowledge graphs. We have added discussions comparing this to prior literature in Section 5.3.

Weakness 2 and Question 2: Repurposing evaluation and Figure 5. DORA is a system that provides evidence for any pair of drug candidates and diseases. Limited by available data, we are only able to evaluate based on existing trials and results. That said, the system can be deployed for any open pair of drug candidates and diseases. We have also been actively collaborating with users to discover new drug candidates that have never been discovered by humans, even though their evaluation will take much longer and rely on real-world experiments. In Section 4.3, we provide a few examples of such efforts, and Appendix D presents one complete pairwise DORA report with its force-directed Meta-Pathfinder visualization.

To better answer this question, we asked two human evaluators to complete human evaluation of 19 DORA reports. For each drug–disease pair, evaluators first independently searched literature available on or before July 10, 2026 and froze a literature-based baseline. They then read the DORA report and assessed its strongest incremental insight for novelty beyond the baseline, biological plausibility, and therapeutic utility. On average, they spent 33.2 minutes manually searching the literature and identified 2.1 relevant literature or registry sources per pair. 14 of the 19 hypotheses were rated at least “plausible with gaps,” including nine rated “coherent and supported” or “supported by a strong convergent rationale.” Among them, 3 were also rated as containing a “specific new insight” or a “clearly distinct and consequential insight”: delgocitinib for vitiligo, through a candidate melanocyte-intrinsic mechanism involving inhibition of laminin-332-induced melanocyte dedifferentiation; linsitinib for thyroid eye disease, by connecting clinical response with IR-beta cross-inhibition and hyperglycemia; and ravulizumab for IgA nephropathy, through a candidate C5–EDNRA connection extending the complement-centered mechanism. All three also received high therapeutic-utility ratings and were considered worthy of further exploration or experimental verification. These results provide promising signals for DORA on this extremely challenging problem and illustrate how the system may inspire researchers to explore mechanistic gaps that could otherwise be overlooked within the vast volume of biomedical knowledge. We are continuing working on a larger human evaluation throughout the system deployment process, and might also update the results.

Thank you again for the detailed questions. We would appreciate an updated score.

Thank you for the rebuttal. I appreciate the clarification of the design and the additional human evaluation. However, the advantage of DORA’s cross-provider search over alternative retrieval or path-search strategies using the same evidence resources remains unclear. My concern about the methodological contribution therefore remains, and I maintain my original score.

Reviewer_2vjcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces DORA, a drug repurposing agent that combines a heterogeneous biomedical knowledge network, literature, and a path-search method called Meta-Pathfinder. Given a drug–disease pair, the agent investigates mechanistic evidence and predicts clinical trial progression. On a 100-pair development set, DORA improves performance over an LLM-only baseline for five of seven models, while performance declines for two. The paper also presents ablations, qualitative paths, and evidence-verification analyses.

Critical strengths
  • Investigating mechanisms, conflicting evidence, and clinical suitability is more useful than predicting a drug–disease link alone.

  • The framework combines multiple evidence sources while retaining provenance, and treats retrieved paths as hypotheses to examine rather than established mechanisms.

  • The paper reports results for all seven models, including regressions, and acknowledges the negative ablation coefficient for the Biomedical KG.

Critical weaknesses
  1. The evaluated drug–disease pairs have already entered clinical trials. The main task therefore measures outcome forecasting more directly than discovery of previously untested indications. The three paths in Figure 5 do not establish, through a systematic independent evaluation, that DORA discovered novel therapeutic opportunities.

  2. Only 30 of the 100 pairs have exact labels; the others have lower-bound labels, for which predicting a later phase can still count as correct. Under the stated rules, an always-1.00 predictor would achieve at least 74% categorical accuracy from the 70 lower-bound cases and four approval cases alone. Results on exact-label cases, stage-stratified results, and simple constant baselines are needed to interpret the reported accuracy.

  3. Appendix B.3 acknowledges that several models have training cutoffs after January 1, 2026. It also states that a pair can enter the benchmark because of a post-cutoff record even when an earlier record determines its label. The paper does not quantify how many final labels were already knowable before the cutoff or how much public outcome information the models may have encountered.

  4. Meta-Pathfinder has a positive coefficient in Table 6, but the Biomedical KG has a negative one, and several reduced configurations in Table 10 outperform the full configuration. Ten component recipes may not adequately identify interactions under an additive model. The evidence that each central component is necessary is therefore limited.

  5. Appendix A.7 says that verification checks run on the harness side in benchmark mode and are not exposed to the agent as callable tools. Evidence-support recalibration on 50 real trajectories also does not improve outcome accuracy. The paper needs clearer trajectory-level evidence for the claim that the agent verifies and revises its hypotheses during the reported runs.

Questions for authors
  1. For how many of the 100 pairs did the information determining the final label first become public before versus after the cutoff? Can the authors report results on a subset whose outcomes were strictly unknown at the cutoff?

  2. Can the authors report performance on the 30 exact-label pairs, errors and calibration by trial stage and status, and simple baselines such as always predicting 1.00?

  3. What explains the negative Biomedical KG ablation coefficient? Can the authors provide a direct comparison with other settings fixed and test its interaction with Meta-Pathfinder?

  4. How are cross-source edge costs, the A* heuristic, path deduplication, and evidence filtering defined in Meta-Pathfinder? These details are needed to reproduce the method and assess its algorithmic novelty.

  5. How many experts assessed the “novel” paths in Figure 5, using what blinded criteria? What was their agreement, and how were previously known drug–indication links excluded?

2 comments
AuthorsOpenReview ↗

Author Response (1/2)

Thank you for the detailed review and for recognizing DORA's provenance, conflicting-evidence analysis, and evaluation across seven backbones. We address each point below.

Weakness 1 and Question 5: Repurposing evaluation and expert assessment.

DORA is a system that provides evidence for any pair of drug candidates and diseases. Limited by available data with ground truth, we are only able to evaluate based on existing trials and results. That said, the system can be deployed for any open pair of drug candidates and diseases. We will make our system publicly accessible for any pair of drug candidates and diseases after the paper’s acceptance. We have also been actively collaborating with users to discover new drug candidates that have never been discovered by humans, even though their evaluation will take much longer and rely on real-world lab experiments. In Section 4.3, we provide a few examples of such efforts, and Appendix D presents one complete pairwise DORA report with its force-directed Meta-Pathfinder visualization.

To better evaluate the insights and hypotheses provided by DORA, we asked two human evaluators to complete human evaluation of 19 DORA reports. For each drug–disease pair, evaluators first independently searched literature available on or before July 10, 2026 and froze a literature-based baseline. They then read the DORA report and assessed its strongest incremental insight for novelty beyond the baseline, biological plausibility, and therapeutic utility. On average, they spent 33.2 minutes manually searching the literature and identified 2.1 relevant literature or registry sources per pair. 14 of the 19 hypotheses were rated at least “plausible with gaps,” including nine rated “coherent and supported” or “supported by a strong convergent rationale.” Among them, 3 were also rated as containing a “specific new insight” or a “clearly distinct and consequential insight”: delgocitinib for vitiligo, through a candidate melanocyte-intrinsic mechanism involving inhibition of laminin-332-induced melanocyte dedifferentiation; linsitinib for thyroid eye disease, by connecting clinical response with IR-beta cross-inhibition and hyperglycemia; and ravulizumab for IgA nephropathy, through a candidate C5–EDNRA connection extending the complement-centered mechanism. All three also received high therapeutic-utility ratings and were considered worthy of further exploration or experimental verification. These results provide promising signals for DORA on this extremely challenging problem and illustrate how the system may inspire researchers to explore mechanistic gaps that could otherwise be overlooked within the vast volume of biomedical knowledge. We are continuing working on a larger human evaluation throughout the system deployment process, and might also update the results.

Weakness 2 and Question 2: Asymmetric scoring and exact-label analysis. The corrected split is 15 exact-label and 85 lower-bound cases. We appreciate the question and the chance to clarify our metric design. In our evaluation set, whether a question has an exact label or a lower-bound label is hidden from every model, so a model cannot cheat by selectively over-reporting scores only on lower-bound labels, but not on exact label queries. Additionally, we provide a factor that upcurves the overconfident branch in exact label loss, so that in expectation, a blind model cannot gain more by overreporting scores in an uninformed way. We will provide more details in Section 2.

Weakness 3 and Question 1: Cutoff knowability. We have examined all the data in the dev set and found that a number of pairs do have results before the cutoff date. That said, it does not create a data leakage problem in our evaluation because we also have run a data cleaning phase before these entries are entered in the development set to make sure that no models know the answers without searching online. For each datum, we ran Claude-opus-5 and GPT-5.6-sol, and made sure that none of the models reproduce or memorize the results of the clinical trial.

AuthorsOpenReview ↗

Author Response (2/2)

Weakness 4 and Question 3: Biomedical KG and Meta-Pathfinder interaction. The negative coefficient in Table 6 comes from the original additive model treating the plus-Pathfinder recipe as independent, although that recipe already includes the Biomedical KG. We therefore follow the suggestion and run a matched nested ablation on the same 100 pairs with three seeds. Biomedical KG alone, KG+literature, and full DORA obtain normalized scores of 65.91 ±\pm 2.62, 63.19 ±\pm 0.71, and 58.65 ±\pm 1.47, with categorical accuracies of 88.67%, 88.00%, and 84.00%. As more evidence is added, mean promisingness decreases from 0.574 to 0.566 and 0.564, producing lower scores under the lower-bound labels. Full DORA also constructs 58 cross-provider paths with complete trace and edge provenance. The point-wise effects of KG, Pathfinder, and their interaction are −1.99-1.99 (95% CI [−5.75,1.76][-5.75, 1.76]), −3.80-3.80 ([−10.17,1.90][-10.17, 1.90]), and −1.48-1.48 ([−7.78,4.49][-7.78, 4.49]) normalized points, respectively; none is statistically significant due to the small size of data. To see a more statistical significant trend, we also correct the additive Bayesian model to encode the KG dependency of the plus-Pathfinder recipe, and the KG and Pathfinder effects are +1.43+1.43 (95% CrI [−6.36,9.21][-6.36, 9.21], P(β>0)=0.64P(\beta>0)=0.64) and −1.58-1.58 ([−9.77,6.61][-9.77, 6.61], P(β>0)=0.35P(\beta>0)=0.35), respectively. We replace the original coefficient analysis with these matched component results in Section 4.2 and Appendix C.

Question 4: Meta-Pathfinder reproducibility. Meta-Pathfinder uses bounded A* over an implicit provider graph: g(n) accumulates edge cost and provider ranking supplies h_p(n) for successor expansion. Section 3.2 gives the search logic; Appendix E gives the queue, budgets, path selection, and provenance-preserving merge.

Weakness 5: Verification and trajectory evidence. We appreciate the suggestion on enriching more details in the effect of the verification step. We run an outcome-blind paired evaluation over all 100 pairs with three seeds. An independent GPT-5.5 verifier checks each GPT-5.6-sol answer against its retrieved evidence, the same GPT-5.6-sol backbone revises it, and a second audit checks the revision. Unsupported claims decrease from 18.44% to 1.05%, citation entailment increases from 68.73% to 95.63%, and 270/300 answers are substantively revised. All 278 flagged claims and all six contradicted claims are removed or independently cleared. Categorical accuracy changes from 84.00% to 72.67%. On the 15 exact-label cases, Brier improves from 0.0906 to 0.0539. Appendix A.6 reports these evidence-quality results and representative before/after trajectories.

Thank you again for the detailed questions. We would appreciate your follow-up comments and, if your concerns are addressed, an updated score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to Reviews and Comments

We thank the reviewer for the constructive comments and for the encouraging assessment, including that “The idea is practically important” and that “The paper is of high quality, it is easy to follow, the experimentation is rigorous,” providing "Useful and traceable biomedical infrastructure" for "Clinically grounded evaluation across multiple models". We have made all efforts to address the questions regarding

  1. hypothesis novelty and expert evaluation
  2. label construction and cutoff validity,
  3. the Biomedical KG and Meta-Pathfinder ablations,
  4. Meta-Pathfinder reproducibility, verification behavior, and
  5. comparisons with related work and additional baselines.

Our open-pair drug–disease repurposing setting presents a unique evaluation challenge because outcomes for previously unexplored hypotheses are rarely available at scale. We therefore use a retrospective benchmark as an evaluation surrogate: DORA accesses evidence available before a fixed cutoff, and are required to predict independently recorded trial updates. Although this is an exceptionally challenging problem with practical importance but limited prior work in the same setting, we hope DORA provides a useful first step using the best currently available biomedical resources and evaluation data.

We sincerely thank the reviewer and the Area Chair for their time and effort. We really appreciate the opportunity to continue this thread of work and inspire more future efforts into this challenging quest.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.