- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes G3I-Net for distinguishing between early-stage and advanced-stage T-stages in non-small cell lung cancer. The model integrates a frozen Swin-UNETR CT encoder, a radiomics-based MLP, and a gene expression graph network; it employs a CT-conditioned hypernetwork to generate patient-specific edge weights, informed by a fixed protein-protein interaction (PPI) support set.
Critical strengths
1.Rather than interpreting a higher AUROC directly as successful patient-level graph generation, the authors examine the parametric constraints of the graph generation process, inter-patient variability, static surrogates, and modality ablations. This approach helps uncover redundant mechanisms within complex multimodal models that might otherwise be obscured by the final performance metrics.
2.The paper explicitly acknowledges that the gating improvements did not significantly outperform the baseline on the primary endpoint, that the gene branches failed to provide incremental predictive value, that the router is prone to degradation, and that model selection across 52 configurations tends to yield overly optimistic point estimates.
Critical weaknesses
1.Section 3.4.1 reports an effective rank of 1.72, with the first principal component accounting for 84.9% of the variance; based on this, the authors characterize the patient plots as approximating a single, fixed pattern. However, the paper fails to clarify whether the effective rank was calculated using modulation codes, node scores, or a normalized adjacency matrix; it also omits details regarding whether data were centered across patients and which spectral normalization method was applied. Similarly, the claim in the introduction—that the data are "73% from the prior"—lacks an explicit definition of the distance formula and the normalization procedure used.
2.Section 3.3.2 describes a process involving initial ranking via LASSO followed by feature-count selection through cross-validation; however, it remains unclear whether these steps are fully nested within the outer training folds. Furthermore, the specific independent validation set used for early stopping is not specified. Section 5 acknowledges that the final configuration was selected from among 52 candidates; consequently, the reported out-of-fold (OOF) point estimates and bootstrap-based confidence intervals for patients do not inherently account for or eliminate model selection bias.
Questions for authors
-
What data are used for each of the following steps in ASSO: ranking, feature selection, normalization, early stopping, and final configuration selection? Please provide the complete nested workflow, and ensure that all primary baselines employ a consistent, patient-level out-of-fold (OOF) aggregation method. If the baselines are provided with the same frozen CT features, does the performance gap still persist?
-
How exactly are the effective rank, 84.9% explained variance, and 73% prior distance calculated? Are the statistics computed using the training patients or the held-out patients? Please provide the centering method, the full spectrum, and the distribution of graph distances between patients; furthermore, please distinguish between the dimensionality of the latent codes, the local degrees of freedom, and the linear effective rank across patients.
-
When the trained weights are held fixed, how do replacing patient-specific scans—either through random permutation, substitution with the training set’s mean scan, or use of a globally learned static scan—affect the predicted probabilities, loss, and AUROC? Can these results provide evidence for a "lack of patient specificity," and help distinguish between potential explanations such as CT scan compression, parameter saturation, or the absence of incremental genetic signals relevant to the task?
-
Please explain how the threshold for matching specificity was determined, and provide the experimental basis for the lower bound of approximately 0.0125. Furthermore, in Table 2, both models show a specificity of 0.944 among the 107 negative cases; their respective accuracy values of 0.875 and 0.883 correspond to approximately 11 and 12 true positives, implying an incremental sensitivity of about . However, Figure 4 reports an increment of 0.095 for the "mixture-of-gates" approach. Were different thresholds or aggregation rules used in these instances? Please clarify and ensure consistent reporting.
3 comments
Thank you for your rebuttal; it addresses some of my concerns. However, the paper still has some issues: the training set includes only 128 patients, and this dataset may contain some noise, which could skew the results. Furthermore, the paper does not include an appendix, which leads me to believe that the research is not yet thorough. I will unchange my score.
Thank you for the follow-up. We agree that this is a real limitation. The cohort is small (128 patients, 21 with advanced T-stage), comes from one public collection without external validation, and has the input-pipeline problems disclosed in our general response. The revision states this in the Limitations (§5) and makes no accuracy claim.
We use NSCLC Radiogenomics because our question, whether a graph generated from a patient's CT is patient-specific and used, needs CT, tumour segmentation, gene expression and clinical labels for the same patients, which this public collection provides (Bakr et al., 2018).
Within it, we tried to make the most of the data:
- Fewer parameters to fit: a frozen CT trunk pretrained without labels, 89 node scores per patient instead of 7,921 adjacency entries, a fixed protein-interaction prior, and 10 radiomic descriptors.
- Paired evaluation of every patient: five-fold cross-validation over five seeds on splits shared by all arms, with a patient-level bootstrap. Pairing narrows the comparisons: static − dynamic is +0.014 [−0.012, +0.046], against [0.547, 0.791] for the AUC itself.
- No held-out label in training: feature selection inside each training fold and a fixed schedule. For the full model, the shortcuts this removes inflated AUC by +0.221 on this cohort.
- A positive control on the same patients: histology, where genes are predictive (logistic regression 0.905), shows that our tests detect graph and gene effects at this size (e.g. −0.106 when the graph is replaced by the prior).
- Training variation measured by repeating complete trainings.
Hyperparameters were still chosen on this cohort, so some optimism remains. Testing generality needs a larger, external cohort; the tests can be rerun on one unchanged.
Reply to Reviewer eCJL
Thank you. Two of your questions exposed errors; see the general response. W/Q number your weaknesses and questions in order.
W2/Q1. The complete workflow.
What the submitted pipeline did:
The first three are replaced: selection and normalisation inside each outer training fold, a fixed schedule, final weights. The configuration (e.g. width 64) was still fixed on this cohort, which our CIs do not cover; the revision states this.
Baselines are now aggregated like the model: patient-level, from seed-averaged out-of-fold predictions. With the same frozen CT features (Swin-embedding PCA, fit within each fold), logistic regression on CT alone reaches 0.52–0.57, in line with the deep CT-only arm (0.509). Under corrected evaluation, no gap remains.
Q2/W1. Effective rank, 84.9% and 73%. All values use held-out patients, for each of the 25 trained models f, reported as mean and range.
Representations:
Effective rank:
Corrected models (untrained network on the same inputs in brackets):
Distance from the prior: S(p)=∥Adyn(p)−P∥F/∥P∥F.
Between-patient distances (same normalisation):
The three quantities you distinguish:
Q3. Fixed-weight replacements (table in our reply to PieR Q2).
Separating the three causes:
Q4. Matched specificity. You found a real bug.