- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper presents KGMRT, a framework intended to combine imaging, clinical text, and tabular data for clinical decision support. The proposed approach uses UMLS-based supervision to align modality representations, a cross-modal transformer to retrieve candidate evidence, and a selection objective incorporating concept coverage, redundancy, and ontological consistency. The authors claim a approximation guarantee for greedy evidence selection and report evaluations on several dataset, claiming improvements in classification performance, calibration, evidence-set size, and inference latency.
Critical strengths
None
Critical weaknesses
-
Contradictory training descriptions and inconsistent experimental results. Section 3.5 states that Stage 2 jointly optimizes the classifier, encoders, and cross-modal transformer, whereas Appendix A states that the encoders and transformer are frozen after Stage 1. The reported results also conflict: Table 1 gives MIMIC-IV/CXR macro AUCs of 0.871 for AdaCoMed and 0.934 for KGMRT, while Table 3 reports 0.843 and 0.917 for the same methods without explaining a different evaluation setting. Furthermore, averaging the per-class values in Table 3 yields approximately 0.845 and 0.918, which are inconsistent with its own reported macro averages beyond ordinary rounding. Section 5.1 also names FusionNet as the strongest ADNI baseline, although it does not appear in Table 1. These discrepancies make both the training protocol and the reported performance unreliable as presented.
-
Incomplete presentation and extensive citation errors. Appendix B contains an unresolved “Table ??” reference to TCGA results, but the corresponding table is missing. Table 3 is labeled as a MIMIC-CXR[1] evaluation while listing the NIH ChestX-ray14[2] disease categories, with no explanation of an alternative labeling procedure. The references for arXiv:2605.13277 and arXiv:2607.09982 contain incorrect authors and years, and several bibliography entries list “others” as the author. Multiple references also retain “UNVERIFIED” or “UNVERIFIED peer review” annotations. These unresolved verification markers, together with confirmed bibliographic errors, indicate that the references were not adequately checked before submission. Collectively, these problems substantially undermine the manuscript’s completeness and credibility.
[1] https://physionet.org/content/mimic-cxr-ext-mimic-cxr-db/1.0/
[2] Baltruschat IM, Nickisch H, Grass M, Knopp T, Saalbach A. Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification. Sci Rep. 2019 Apr 23;9(1):6381. doi: 10.1038/s41598-019-42294-8. PMID: 31011155; PMCID: PMC6476887.
Questions for authors
- Stage 2 training: Section 3.5 states that the classifier, encoders, and cross-modal transformer are jointly optimized during Stage 2, whereas Appendix A states that the encoders and transformer are frozen. Which description matches the implementation used to obtain the reported results?
- Conflicting AUC results: Why do the MIMIC-IV/CXR macro AUCs in Table 1 (0.871 for AdaCoMed and 0.934 for KGMRT) differ from those in Table 3 (0.843 and 0.917)? Were different cohorts, splits, or evaluation protocols used?
- Within-table numerical inconsistencies: Averaging the per-class AUCs in Table 3 yields approximately 0.845 for AdaCoMed and 0.918 for KGMRT, rather than the reported macro averages. How were these averages calculated?
- Dataset labels: Why does Table 3 list the NIH ChestX-ray14 disease categories while identifying the benchmark as MIMIC-CXR? Were these labels independently constructed for MIMIC-CXR? If so, please explain the annotation procedure and provide the corresponding evaluation code.
- Missing results: Appendix B refers to TCGA per-cancer-type results through “Table ??”, but the table is absent. Were these experiments conducted, and can the authors provide the referenced results?
- Reference verification: How do the authors account for the incorrect authors and years associated with arXiv:2605.13277 and arXiv:2607.09982, the entries listing “others” as the author, and the retained “UNVERIFIED” annotations? Please verify the affected references and clarify whether the cited works support the associated claims.
I urge the authors to take responsibility for the accuracy and completeness of their manuscript and to verify it carefully before submission. Peer review should assess a rigorously prepared study, rather than serve as a chance-based opportunity for an unchecked manuscript to be accepted.