acceptodds
Withdrawn from ICLR 2027

RA-HMFNet: Region-Aware Hierarchical Multimodal Fusion for Preoperative Two-Year Recurrence Prediction in NSCLC

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Preoperative prediction of postoperative recurrence in non-small cell lung cancer (NSCLC) requires integrating heterogeneous information from the tumor, surrounding lung tissue, metabolic imaging, and clinical variables. However, conventional global feature aggregation may dilute scale-specific peritumoral information and limit the effectiveness of multiscale multimodal fusion. We propose RA-HMFNet, a region-aware hierarchical multimodal fusion framework for predicting recurrence within two years after surgery. Preoperative CT is partitioned into the intratumoral region and three non-overlapping peritumoral rings at 0–3 mm, 3–6 mm, and 6–9 mm. RA-HMFNet performs ROI-aware pooling directly on deep feature maps using the corresponding ring masks, thereby preserving spatially localized multiscale representations. These representations are subsequently integrated with PET and clinical features through hierarchical multimodal fusion, with an optional patient-specific scale-weighting mechanism. We evaluate the framework on 107 patients with complete multimodal data, including 25 patients with recurrence within two years, using 5-repeat 5-fold nested cross-validation and patient-level averaged out-of-fold predictions. Under the same ROI-aware representation, equal, global, and patient-specific scale fusion achieved AUROCs of 0.6439, 0.6512, and 0.6639, respectively. The patient-specific model also achieved an AUPRC of 0.3304 and a Brier score of 0.2127. Its AUROC was numerically higher than clinical logistic regression (0.6380), radiomics fusion (0.6434), and multimodal concatenation (0.6000). However, paired bootstrap intervals for patient-specific versus equal or global weighting included zero, indicating that the incremental value of adaptive weighting remains uncertain. These results identify ROI-aware representation as a central component of the framework and suggest that preserving localized peritumoral structure can improve multiscale multimodal recurrence prediction. External validation is required to establish generalizability.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Non-small cell lung cancer, recurrence prediction, multimodal learning, region-aware pooling, peritumoral imaging, patient-specific attention

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_WhDwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors proposed Region-Aware Hierarchical Multimodal Fusion (RA-HMF) model for estimation of post-operative recurrence from pre-operative data for lung cancer using computed tomography (CT), positron emission tomography (PET) and clinical infomation. Region-Aware module is built based on peritumoral region extracted from segmentation mask by expanding it using dilation (morphological operation) by 3, 6, 9 mm. then region-specific features can be extracted by multiplying mask with feature map. Those region-aware features are later aggregated with attention mechanism and further ‘enhanced’ patient-level PET and clinical information.

The contributions according to the authors are: region-aware pooling for different peritumoral regions; PET and clinical information aggregation mechanism, and mechanistic ablation.

Evaluation of the proposed method is done on 107 patients (25 positive cases, 82 negative cases) from publicly available data repository. The proposed method is compared against the baselines and the shown improvement seems to modest.

Critical strengths

The paper is relatively easy to understand.

Critical weaknesses

Some of the key concepts claimed in this submission as main contributions have been studies in the literature e.g. region-aware concept with distinguishing 3, 6, 9 mm peritumoral regions for features extraction was reported e.g. https://doi.org/10.1371/journal.pone.0206108 https://doi.org/10.3390/diagnostics12061313 (as cited for radiomics studies)

The other proposed concepts (masked pooling, PET&clinical information attention aggregation) while not proposed earlier in the context of the recurrence lung cancer prediction show rather modest improvement in the overall pipeline (as evidenced in Table 2).

See this work on tumor centric attention https://doi.org/10.1007/s40846-024-00884-6 See cross-modality attention mechanism for multimodal fusion used in this paper: https://doi.org/10.1117/12.3006036 The paper requires substantial revision of the background literature and adjustment of the conclusions accordingly.

Questions for authors

Evaluation of the proposed framework is done on 107 patients, while this is not a reason to reject the paper but the evaluation itself is limited to very few baselines and possibly missing relevant approaches.

The most important one is to show how classic radiomics approach (e.g. https://doi.org/10.3390/diagnostics12061313) could compare against the proposed method. Radiomics could be extracted from each peritumoral region (03, 36, 69) and fed into the logistic regression (similarly as it was done for Clinicar LR – see Table 1).

Evaluation also assume (implicitly though) that the given segmentation is perfect, this raises a question how stable is the presented method for feature extraction when the segmentation quality is perturbed – some studies performed such analysis for radiomics approaches (though for different tumor location): e.g https://doi.org/10.1109/ISBI56570.2024.10635413

For Evaluation – no std is provided.

Recent meta-analysis paper to better frame the presented work: https://doi.org/10.2196/84223

Reviewer_PJBoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RA-HMFNet, a multimodal framework for predicting two-year postoperative recurrence in non-small-cell lung cancer (NSCLC) using preoperative CT, PET, and clinical data. The method divides the peritumoral region into three non-overlapping distance-based regions and extracts region-specific features from a shared CT encoder. PET and clinical features are then used to generate patient-specific attention weights for these regions, followed by hierarchical multimodal fusion for recurrence prediction. The method is evaluated on 107 patients from the NSCLC-Radiogenomics dataset using repeated nested cross-validation, demonstrating improvement over the reported baselines.

Critical strengths
  1. The paper addresses a clinically meaningful problem of predicting postoperative recurrence from preoperative data.

  2. The proposed framework is reasonablely motivated, as preserving spatial information from different peritumoral regions may help capture local tumor characteristics relevant to recurrence prediction.

Critical weaknesses
  1. The main technical contributions are limited where it borrows largely from prior works, although the application to NSCLC is different. ROI-based pooling, attention-based weighting, and hierarchical fusion are established techniques.

  2. The ablation results do not support the benefits of distance-specific decomposition and patient-specific scale weighting. As an example in Table 2, using three peritumoral regions with fixed equal weights performs worse than using a single 0–9 mm ROI (AUROC 0.6390 vs. 0.6478). The experiments are also conducted on a single dataset with only 25 recurrence cases.

Questions for authors
  1. What evidence supports the need for the proposed hierarchical adaptive fusion instead of a simpler single-ROI model?

  2. Can the authors provide confidence intervals for the AUROC and AUPRC differences between RA-HMFNet and the strongest baselines, as well as the ROI Global and P09 variants? This would help assess the reliability of the reported improvements, especially given the small number of recurrence cases.

  3. The learned patient-specific attention weights are reported to be close to uniform and have limited repeatability. Provide justification of the benefit of patient-specific weighting over fixed or globally learned weights?

Reviewer_TgfDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  1. This manuscript investigates whether recurrence within 730 days after surgery can be predicted in patients with resectable non-small cell lung cancer (NSCLC) using preoperative CT, ^18F-FDG PET, and clinical variables. The authors propose RA-HMFNet, a region-aware multimodal deep-learning framework with three main components: (1) peritumoral CT feature pooling across three concentric regions (0–3 mm, 3–6 mm, and 6–9 mm); (2) PET- and clinical-conditioned weighting of peritumoral scales; and (3) hierarchical fusion of intratumoral CT, peritumoral CT, PET, and clinical representations.

  2. The study uses the publicly available NSCLC-Radiogenomics dataset, comprising 107 patients and 25 recurrence events. Model evaluation employs five repetitions of five-fold outer cross-validation with three-fold inner cross-validation for model selection. The proposed model achieves an AUROC of 0.6634, an AUPRC of 0.3861, a Brier score of 0.1683, and a log loss of 0.5146.

  3. The ablation experiments suggest that explicit peritumoral ROI localization contributes more to predictive performance than patient-specific scale attention. The latter yields only a small improvement over globally learnable scale weights.

  4. The study addresses a clinically relevant problem and presents a potentially useful multimodal modeling strategy. However, the limited cohort size, modest performance differences, absence of quantified performance uncertainty, and lack of independent external validation prevent strong conclusions about generalizability or clinical utility.

Critical strengths
  • Clinically relevant prediction target and appropriate preoperative input modalities. The study addresses postoperative recurrence risk in NSCLC using CT, PET, and clinical information available before surgery, rather than relying on postoperative pathological findings as model inputs. This makes the prediction task potentially relevant to preoperative risk stratification and subsequent clinical management, provided that the model's performance and clinical value can be established through further validation.

  • Ablation experiments provide useful evidence about the contribution of peritumoral ROI localization. The authors compare multiple configurations, including NoROI, P09, ROI Equal, ROI Global, and the full RA-HMFNet. These comparisons help distinguish the contribution of explicit peritumoral localization from that of multiscale decomposition and patient-specific attention. The results also support a relatively cautious interpretation of the attention mechanism, given its near-uniform weights and limited stability across repetitions.

Critical weaknesses
  • The small number of recurrence events and lack of statistical uncertainty quantification undermine the reliability of the performance claims. The cohort contains only 107 patients and 25 recurrence events, whereas the proposed model has approximately 4.4 million trainable parameters. Although repeated nested cross-validation is a reasonable attempt to mitigate model-selection bias, it cannot compensate for the limited number of independent patients and events. The reported AUROC improvement of the full model over ROI Global is only 0.0068, and the improvement over P09 is only 0.0156. Without patient-level confidence intervals, performance distributions across repetitions, and appropriate paired comparisons, it is unclear whether these differences reflect reproducible improvements or sampling variability. This limitation directly affects the credibility of the claimed advantages of the proposed architecture.

  • The evidence is insufficient to establish the model's incremental clinical value or generalizability. The principal empirical contribution appears to be peritumoral ROI localization rather than patient-specific scale attention, whose incremental performance gain is minimal. Moreover, the manuscript lacks independent external validation and does not establish superiority over a sufficiently specified clinical risk model or demonstrate clinical net benefit. Consequently, the current results support, at most, exploratory predictive performance under internal validation; they do not establish robust cross-cohort generalization or clinical utility.

Questions for authors

A. Clarifications that could materially affect the assessment

    1. Performance uncertainty and statistical comparisons: Could the authors report performance distributions across all outer folds and repetitions, together with patient-level 95% confidence intervals for AUROC, AUPRC, Brier score, and log loss for each model? Could they also provide appropriate paired comparisons between RA-HMFNet and the principal baselines, particularly ROI Global and P09, to establish whether the observed improvements are statistically and practically meaningful?
    1. Cohort selection and potential selection bias: Could the authors provide a detailed patient-selection flowchart showing the number of patients excluded at each stage because of missing CT segmentation, unavailable PET, registration failure, missing clinical variables, or insufficient follow-up? Were the included and excluded patients systematically different in age, disease stage, tumor characteristics, or recurrence outcomes? Such differences could affect the representativeness of the final cohort and the interpretation of the reported performance.

B. Recommendations for future work

    1. Evaluation of model complexity: Compare RA-HMFNet with simpler alternatives, including a single P09 ROI model, a lightweight multimodal fusion model, and conventional radiomics-based approaches. These comparisons would help determine whether the added architectural complexity yields a reproducible and meaningful performance advantage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.