- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes an ontology-constrained knowledge graph for occupational metal exposure and its associated health conditions. The knowledge graph is constructed through a 4 pipeline comprising data mining and document chunking; ontology-guided triplet extraction across document chunks, supported by an expert-validated ontology and an LLM; integration of retrieved knowledge; normalization of concepts and relations; and evidence linking. The resulting knowledge graph has potential applications in occupational health knowledge retrieval, knowledge discovery, and the identification of insights that may inform strategies for improving occupational health outcomes.
Critical strengths
- The authors propose an ontology-constrained knowledge graph with evidence support to characterize metal exposure and its associated health effects. The key novelty of this work lies in its approach to grounding knowledge across the literature, particularly through the use of an ontology schema validated by domain experts to improve the consistency and reliability of knowledge extraction. In addition, the authors provide a dataset to support the proposed approach and facilitate further research.
Critical weaknesses
-
The main weakness of this paper is that it contains no evaluation of the constructed ontology-constrained knowledge graph. The authors cannot simply describe a schema without any evaluation on specific tasks for a dataset and benchmark type of contribution. A comparison with existing biomedical knowledge graphs (e.g Semnet, Semnet-2, OptimusKG,..) must be evaluated to enforce their claim about the necessity of the proposed MHKG graph. Under the current state, the following contribution falls short for an ICLR conference.
-
Overall the authors should revise the presentation of their framework and the wording, it is easy for a reader to get confused within section 3. Also, a lot of Appendix content is mentioned in section 3 but given that the whole manuscript is 8 pages long, authors could use that space more wisely and bring forward prompts or figures to ease the reading. “LLM” is mentioned multiple times in the pipeline for candidate resolution (stage 3) and triplet extraction (stage 2) yet no precision about the nature of the LLM nor the analysis of results/error analysis are given in the main study. In section, 3.2, some errors are occurring :
-
(l.191-192) “Figure 2 shows the part of the ontology that we define.”
-
(l.279-282) sentence repeated twice.
The following claim is made in the conclusion “In order to guarantee the quality of the graph, we construct an automated process covering pattern-constrained knowledge extraction, structure and evidence auditing, entity fusion and normalization, as well as relationship reconstruction.” With no concrete evaluation and error analysis, it is not demonstrated yet.
-
Questions for authors
Can the authors revise figure 2 ? (with the current state it adds little information compared to figure 3 for instance). Is it possible to provide an analysis of the LLM employed in stage 1 and stage 2 ? Any comparative study with multiple LLMs ? Any downstream task to evaluate your knowledge graph ?