- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a new Transformer-based model to identify the primary site of cancer from longitudinal electronic health records. Specifically, the authors concentrate on two aspects: 1) classification of 5 different cancers and 2) similar patient retrieval. The main contribution of the paper is the new architecture which perform better than examined baselines. The authors evaluate their proposed model on an internal set and more importantly an external Unidentified Mass Clinic cohort encompassing 53 patients.
Critical strengths
The main strength of this paper lies in its research topic, which is timely and interesting.
Critical weaknesses
-
Several recent studies have investigated temporal EHR modelling for patient similarity retrieval, while primary tumour-site prediction has also received increasing attention. Therefore, comparisons with more recent approaches are necessary to establish the novelty and competitiveness of the proposed framework. Most of the methods discussed or cited in the manuscript were published before 2024. Moreover, only a limited number of baseline models are included in the experimental evaluation. As a result, it is difficult to determine whether the proposed framework offers meaningful improvements over existing approaches, which weakens the authors' claims regarding its effectiveness.
-
The authors claim several advantages of the proposed model and present its components as key contributions. However, the experimental results do not clearly demonstrate the benefits of individual components, such as the diagnosis-relative temporal Transformer and the cluster-specific sparse attention mechanism. This concern is further reinforced by the relatively poor performance on the external UMC cohort, which raises questions about the practical effectiveness and generalizability of these components. Although some configurations outperform others, the results do not provide sufficiently and statistically convincing evidence that the proposed architectural innovations offer meaningful advantages over existing approaches.
Questions for authors
-
Does Contribution (1) not already encompass Contributions (2), (3), and (4)? These contributions appear to address different aspects of the same overarching idea rather than presenting clearly distinguishable innovations.
-
Contribution (5) does not appear to represent a "real contribution", as it primarily describes the evaluation of the proposed framework. The authors should reconsider whether this should be listed as a standalone contribution.
-
Several recent studies have investigated temporal EHR modeling for patient similarity retrieval, while primary tumor-site prediction has also received increasing attention. Therefore, comparisons with more recent approaches are necessary to establish the novelty and competitiveness of the proposed framework. Most of the methods discussed or cited in the manuscript were published before 2024. Given the rapid developments in this field over the past two years, the related work should be updated to include more recent studies and clearly distinguish the proposed approach from existing methods.
-
The mathematical formulations are difficult to follow because several variables are introduced without clear definitions or dimensional information. Furthermore, the notation is inconsistent, with some variables written in boldface and others in regular font without an apparent distinction.
-
The dataset description is rather vague. Important information concerning data characteristics, preprocessing procedures, missing-value handling and statistical properties is not adequately presented. Furthermore, the external evaluation involves very few patients for several cancer types, making it difficult to assess the reliability and generalizability of the reported findings.
-
The experimental design is not explained in sufficient detail. In particular, the authors should clearly state the purpose of each experiment, which research objective or methodological contribution it is intended to evaluate and why the selected baselines are appropriate.
-
The number of baseline methods is relatively limited, making it difficult to assess the competitiveness of the proposed approach.
-
The evaluation metrics are not adequately introduced or explained.
-
The manuscript presents primary cancer-site identification as a broad research objective, particularly in the context of diagnostically uncertain patients. However, the experiments are restricted to five predefined cancer categories. While these experiments demonstrate the framework's ability to. distinguish among the selected cancer types, they do not fully establish its effectiveness for primary-site identification in more general clinical settings, particularly for patients with cancer of unknown primary.