RA-HMFNet: Region-Aware Hierarchical Multimodal Fusion for Preoperative Two-Year Recurrence Prediction in NSCLC
Abstract
Preoperative prediction of postoperative recurrence in non-small cell lung cancer (NSCLC) requires integrating heterogeneous information from the tumor, surrounding lung tissue, metabolic imaging, and clinical variables. However, conventional global feature aggregation may dilute scale-specific peritumoral information and limit the effectiveness of multiscale multimodal fusion. We propose RA-HMFNet, a region-aware hierarchical multimodal fusion framework for predicting recurrence within two years after surgery. Preoperative CT is partitioned into the intratumoral region and three non-overlapping peritumoral rings at 0–3 mm, 3–6 mm, and 6–9 mm. RA-HMFNet performs ROI-aware pooling directly on deep feature maps using the corresponding ring masks, thereby preserving spatially localized multiscale representations. These representations are subsequently integrated with PET and clinical features through hierarchical multimodal fusion, with an optional patient-specific scale-weighting mechanism. We evaluate the framework on 107 patients with complete multimodal data, including 25 patients with recurrence within two years, using 5-repeat 5-fold nested cross-validation and patient-level averaged out-of-fold predictions. Under the same ROI-aware representation, equal, global, and patient-specific scale fusion achieved AUROCs of 0.6439, 0.6512, and 0.6639, respectively. The patient-specific model also achieved an AUPRC of 0.3304 and a Brier score of 0.2127. Its AUROC was numerically higher than clinical logistic regression (0.6380), radiomics fusion (0.6434), and multimodal concatenation (0.6000). However, paired bootstrap intervals for patient-specific versus equal or global weighting included zero, indicating that the incremental value of adaptive weighting remains uncertain. These results identify ROI-aware representation as a central component of the framework and suggest that preserving localized peritumoral structure can improve multiscale multimodal recurrence prediction. External validation is required to establish generalizability.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.