acceptodds
Under review as a conference paper at ICLR 2027

EpiGraph: Building Generalists for Evidence-Intensive Epilepsy Reasoning in the Wild

Abstract

Epilepsy diagnosis and treatment require evidence-intensive reasoning across heterogeneous clinical knowledge, including biosignal patterns, genetic mechanisms, pharmacogenomics, treatment strategies, and patient outcomes. In this work, we present EpiGraph, a large-scale epilepsy knowledge graph and benchmark for evaluating knowledge-augmented clinical reasoning. EpiGraph integrates 48,166 peer-reviewed papers and seven clinical resources into a heterogeneous graph containing 24,324 entities and 32,009 evidence-grounded triplets across five clinical layers. Built upon this graph, EpiBench defines five clinically motivated tasks spanning clinical decision-making, EEG report generation, pharmacogenomic precision medicine, treatment recommendation, and deep research planning. We evaluate six LLMs under both standard and Graph-RAG settings. Results show that integrating EpiGraph consistently improves performance across all tasks, with the largest gains observed in pharmacogenomic reasoning (+30–41%). Our findings demonstrate that structured epilepsy knowledge substantially enhances evidence-grounded clinical reasoning and provides a practical benchmark framework for evaluating knowledge-augmented LLMs in real-world neurological settings.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Epilepsy, knowledge graph, large language models, clinical reasoning, pharmacogenomics

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Tof4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper has two main contributions: 1) It introduces EpiKG, an epilepsy knowledge graph built from seven ontology sources and 48,166 PubMed papers; 2) It also introduces the EpiBench benchmark, which includes five tasks (T1 clinical QA, T2 clinical report generation from EEG and patient information, T3 biomarker-driven precision medicine, T4 treatment recommendation and T5 deep research planning). The Graph-RAG retriever, which leverages the EpiKG, is compared with several baselines and shows improved performance, as compared to the baselines, on most tasks.

Critical strengths

The knowledge graph constructed addresses a critical gap in the epilepsy literature: existing epilepsy resources (EpSO, Epilepsy Ontology) are annotation vocabularies and do not support chains such as gene → mechanism → drug → contraindication.

Schema grounded in authoritative sources (ILAE, OMIM, HPO, ChEBI, AES, CPIC).

Broad benchmark for extrinsic evaluation: QA, generation, precision medicine, treatment, and research planning in one package, including a workflow-realistic EEG-to-impression task on real neurologist notes.

Hyperparameters seem to be "selected on T1 MCQ validation performance," but Table 7 lists every split as test-only.

Critical weaknesses

The most significant limitation is the lack of validation of the EpiKG quality. 70% of triplets are LLM-extracted, yet there is no precision estimate, no sampled audit, no inter-annotator agreement for the expert review of flagged candidates.

T3 gold comes from CPIC/ILAE; the KG is seeded from some of those guidelines. Consequently, the Graph-RAG gains may measure answer lookup/graph adjacency rather than reasoning. There is no retrieval-only (no-LLM) baseline, and no per-category T3 breakdown. Those would be useful to show.

Baselines such as MedRAG/DR.KNOWS/AMG-RAG are under-reported and their corresponding backbone LLM, task, and metric are unspecified.

Questions for authors

What is the measured precision of LLM-extracted triplets on a random audited sample? How many candidates reached expert review and what fraction survived?

On which split were the Fig. 3 hyperparameters chosen?

Reviewer_yQawOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study proposes to develop EPIGRAPH, a knowledge graph and benchmark for evidence-intensive reasoning in epilepsy. The authors argue that current gaps exist in that there are no dedicated knowledge graph to epistasis. Thus, they build such a knowledge graph extracting from Pubmed, together with a benchmark dataset and evaluation of EPIGRAGH on such a dataset.

Critical strengths

The major contribution of this study is providing a knowledge graph dedicated to epilepsy, together with a benchmark dataset and evaluations of the constructed knowledge graph.

Critical weaknesses

The major concern I have is about the novelty in the proposed work. Since the construction of knowledge graphs from mining PubMed and related literature has been widely explored in general and dedicated disease domains, it is unclear how those technologies will be transferable to this study and what are the innovations of the proposed work from technological perspectives. The proposed methods are rather traditional and evaluations are primarily empirical. Another concern is about how the conflicts, heterogeneity, and inconsistency of knowledge extracted from Pubmed will be taken care of.

Questions for authors

How would you expect the derived knowledge graph will be enhanced if more data is included such as figures, tables, supplementary materials of the pubmed articles.

Reviewer_65tFOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces EPIGRAPH, a large-scale epilepsy knowledge graph. It also includes a benchmark for evaluating knowledge-augmented clinical reasoning. EPIGRAPH brings together data from over 48,000 research papers and seven clinical resources into a knowledge graph. The subsequent graph was used to test across a range of tasks in epilepsy with multiple LLM families.

Critical strengths

A critical strength is the the development of the GNN across multiple clinically validated epilepsy resources. This was a huge effort to curate and develop such an instrument which could then be used along with Graph-RAG.

Critical weaknesses

-The majority of the benchmark is pulled from publicly available data, thus, this contribution seems less of an impact that a novel dataset. Yes, the Q&A construction is a new "dataset", but this is still pulled from readily available public data, which makes the novel contribution of EPIGRAPH to performance difficult to assess (is it just helping point towards information already memorized in training?). -To the above, Figure 4 makes me particularly concerned that there is some memorization that has occurred. It seems uncanny that the report after EPIGRAPH and the gold standard would be pretty much identical. -I would have loved some novel tasks within epilepsy care with ecological validity to have been tested with and without EPIGRAPH. This would give a better assessment of how well EPIGRAPH performs.

Questions for authors

Though the EEG dataset is behind credentialing, is there any concern of leakage? How do you explain how EpiGRAPH seems to lead to almost identical results with the gold impression?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.