acceptodds
Under review as a conference paper at ICLR 2027

FMD-IMHF: Foundation Model Distillation and Hypergraph Fusion for Neurodegenerative Disease Diagnosis with Incomplete Multimodal Data

Abstract

Accurate diagnosis of neurodegenerative diseases such as Alzheimer's disease (AD) and Parkinson's disease (PD) requires integrating complementary information from multiple modalities, including structural MRI (sMRI), PET, and clinical data. However, a limited number of labeled samples and frequently missing modalities in clinical data pose challenges to representation learning and multimodal fusion. Medical foundation models offer powerful pretrained representations for modality-specific learning, but jointly using independently pretrained models incurs substantial computational costs and leads to misaligned representation spaces. Meanwhile, existing incomplete multimodal learning approaches remain limited in exploiting complementary information from available modalities to compensate for missing information. To address these challenges, we propose FMD-IMHF, a framework that distills modality-specific foundation models into lightweight encoders and aligns their representations in a shared diagnostic semantic space. A patient-modality hypergraph further models cross-modal complementarity and inter-patient similarity, enabling incomplete multimodal fusion without reconstructing missing modalities. Experiments on ADNI demonstrate the effectiveness of FMD-IMHF for AD diagnosis and MCI progression prediction, as well as its robustness under varying missing-modality rates. Evaluations on AIBL, SCAN, and PPMI further demonstrate its generalizability. Distillation reduces parameter counts, inference time, and peak GPU memory, while ablation studies validate the key components of FMD-IMHF.

Primary area:
applications to neuroscience & cognitive science
Keywords:
multimodal diagnosis, Artificial intelligence for science, Alzheimer's disease, foundation models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_hkbhOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates neurodegenerative disease diagnosis with incomplete multimodal data. Instead of directly deploying multiple large foundation models or reconstructing missing modalities, the paper proposes to distill modality-specific foundation models into lightweight encoders while aligning their representations in a shared semantic space. It then constructs a patient-modality hypergraph to integrate information from available modalities and similar patients. To this end, it introduces FMD-IMHF, a framework that combines foundation model distillation, cross-modal alignment, and hypergraph fusion. The method is evaluated on ADNI, AIBL, SCAN, and PPMI for AD diagnosis and disease progression prediction. The results demonstrate improvements in diagnostic performance under missing-modality settings while reducing computational costs.

Critical strengths
  1. This paper addresses an important problem in multimodal medical diagnosis, where missing modalities are common in real clinical data. I find the combination of foundation model distillation and hypergraph fusion reasonable, as it reduces the computational overhead of multiple foundation models while leveraging information from similar patients to compensate for missing modalities.

  2. The evaluation covers multiple datasets and different missing-modality settings. The authors evaluate the method on AD diagnosis and disease progression prediction, and conduct ablation studies to investigate the contributions of different components. The results provide empirical support for the effectiveness of the proposed framework.

Critical weaknesses
  1. I have some concerns about the comparison with existing multimodal methods. Several multimodal baselines perform substantially worse than PET-only, even when all modalities are available. For MCI progression prediction, PET-only achieves a bACC of 0.761, while the proposed method achieves only 0.750. Moreover, the unimodal baselines use distilled student encoders, but it is unclear whether the other multimodal baselines benefit from the same pretrained representations. This raises concerns about the fairness of the comparison, making it difficult to determine whether the reported improvements come from the proposed fusion method or differences in feature extraction.

  2. The evaluation mainly focuses on binary classification tasks, such as AD vs. NC and sMCI vs. pMCI, without considering more challenging multiclass diagnosis, such as NC vs. MCI vs. AD. Furthermore, the external validation cohorts are relatively small, with only 23 subjects in AIBL and 90 subjects in SCAN. These limitations make it difficult to assess whether the proposed method can generalize to more complex clinical settings and larger independent cohorts.

Questions for authors
  • Why do several multimodal baselines perform worse than PET-only? Are all compared methods evaluated using the same pretrained modality encoders?

  • Why does the evaluation focus only on binary classification tasks? Can the proposed method achieve similar improvements on more challenging three-class diagnosis, such as NC vs. MCI vs. AD?

  • What are the exact clinical features used as model inputs? Although CDR-SB is explicitly excluded, is global CDR also excluded? Are all clinical features collected at baseline rather than during follow-up?

11 comments
AuthorsOpenReview ↗

The Response to Weakness 1(Part I)

We thank the reviewer for raising this concern.

(1) Regarding the Lower Performance of Multimodal Methods Compared with PET-only

The superior performance of some unimodal baselines over multimodal methods was mainly attributable to two issues in our original experimental design. First, unimodal baselines used all available subjects for each modality, whereas complete multimodal methods were evaluated only on subjects with all three modalities (545 for AD vs. NC and 615 for sMCI vs. pMCI), resulting in larger cohorts for the unimodal baselines. Second, the unimodal baselines employed modality-specific student encoders distilled from foundation models, providing stronger feature representations, whereas the other multimodal baselines did not consistently use the same distilled encoders. To address these issues, we revised the experimental design and conducted additional controlled experiments to ensure fair comparisons.

We repeated the experiments for all three unimodal baselines using the same complete-case subjects as the corresponding multimodal methods, with consistent data splits and evaluation protocols. The results before and after revision are presented below:

Table: Performance of unimodal methods on AD vs. NC.

(1) Before Revision

MethodAUCbACCACCSENSPE
MRI-only0.936±0.0150.868±0.0250.868±0.0210.870±0.0450.867±0.023
PET-only0.968±0.0160.901±0.0240.901±0.0230.911±0.0410.892±0.037
Clinical-only0.806±0.0280.721±0.0400.741±0.0370.670±0.0660.771±0.045
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

(2) After Revision

MethodAUCbACCACCSENSPE
MRI-only0.885±0.028**0.788±0.036**0.791±0.036**0.765±0.078**0.810±0.068**
PET-only0.949±0.022**0.878±0.040**0.884±0.035**0.833±0.093*0.923±0.046
Clinical-only0.800±0.052**0.724±0.030**0.719±0.043**0.756±0.093**0.691±0.138**
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

Note: Asterisks indicate statistically significant differences from Ours based on two-sided paired t-tests (*p < 0.05; **p < 0.01).

Table: Performance of unimodal methods on sMCI vs. pMCI.

(1) Before Revision

MethodAUCbACCACCSENSPE
MRI-only0.774±0.0530.722±0.0570.718±0.0490.733±0.0740.711±0.047
PET-only0.830±0.0220.761±0.0380.776±0.0600.734±0.0830.788±0.089
Clinical-only0.717±0.0420.633±0.0350.635±0.0750.628±0.1550.638±0.172
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

(2) After Revision

MethodAUCbACCACCSENSPE
MRI-only0.723±0.024**0.661±0.028*0.704±0.0430.578±0.1410.744±0.097
PET-only0.795±0.035*0.722±0.0590.737±0.0490.694±0.0770.750±0.041
Clinical-only0.647±0.037**0.599±0.049**0.646±0.075*0.510±0.163*0.688±0.135
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

Note: Asterisks indicate statistically significant differences from Ours based on two-sided paired t-tests (*p < 0.05; **p < 0.01).

The revised results reveal substantial changes in several performance metrics. Accordingly, we have updated Table 1(a) in the main manuscript and Table A4 in the Appendix to reflect these changes. We thank the reviewer again for pointing out this issue, which has helped us improve the fairness and reliability of our experimental evaluation.

AuthorsOpenReview ↗

The Response to Weakness 1 (Part II)

(2) Regarding the Performance of Multimodal Baselines and the Fairness of Comparisons

Among the compared methods, only Q-Former (ICLR 2026) and our method are designed to incorporate pretrained foundation models as modality-specific feature encoders. Therefore, pretrained foundation models are used only in these two methods, while the other methods follow their respective original implementations.

This comparison setting was adopted because cross-modal knowledge distillation from foundation models is one of the core contributions of this work. Since the other compared methods, except Q-Former, do not incorporate pretrained foundation models, their original implementations were retained. However, this setup did not adequately control for the potential performance gains from pretrained representations. Consequently, their contributions cannot be fully disentangled from those of the proposed fusion method, potentially affecting the fairness of the comparison.

To further disentangle the effects of foundation-model representations from those of fusion methods, an additional set of controlled experiments was conducted. All compared methods used the same distilled student models as modality-specific feature encoders while retaining their respective downstream fusion and prediction modules. Specifically, it employs BrainMVP (UniFormer-S) for MRI, SegAnyPET (ViT-B) for PET, and Qwen2.5-7B for clinical feature extraction, followed by knowledge distillation to obtain lightweight student encoders identical to those used in our method. The Flex-MoE method retained its original feature extraction and fusion pipeline because its fusion mechanism requires spatially structured 3D patch features, whereas the pretrained representations are 512-dimensional global vectors.

The specific experimental results are presented in our comment entitled “The Response to Weakness 1 (Part III)”.

Under the unified feature encoder setting, our method performs well on AD versus NC classification. It outperforms all competing methods across metrics with complete modalities and maintains strong performance with naturally missing modalities, indicating good robustness to modality incompleteness.

For sMCI versus pMCI classification, the advantage is more modest. With complete modalities, our method achieves the highest AUC and sensitivity, while bACC, ACC, and specificity remain below the best competing results. With naturally missing modalities, it achieves the highest AUC, bACC, ACC, and sensitivity; its specificity is slightly lower than that of Q-Former. Overall, our method performs consistently on AD versus NC classification and shows strong discriminative ability on sMCI versus pMCI, particularly under naturally missing modalities. The significance test results further support its overall performance advantage.

In addition, we have added a hypergraph-based comparison method, which also employs a unified feature encoder; the related results and analysis are provided in our comment entitled “The Response to Weakness 1 (Part IV)”.

These experimental results have been added to Sections C.2 (Complete Comparison Results on ADNI) and C.3 (Complete Comparison Results on ADNI Without Unified Feature Encoders) of the Appendix. In addition, the comparison experiments in Table 1 of the manuscript have also been revised accordingly.

AuthorsOpenReview ↗

The Response to Weakness 1 (Part III)

Below are the comparison experimental results obtained using the unified distilled student encoder, excluding the newly added hypergraph-based comparison method.

Table: Comparison experiment of AD vs. NC for unified feature encoder

(1) Complete multimodal setting

MethodAUCbACCACCSENSPE
TMC0.893±0.057*0.819±0.054**0.819±0.059**0.827±0.087**0.812±0.094*
MMIN0.950±0.015**0.874±0.026**0.870±0.028**0.907±0.041*0.840±0.031**
ShaSpec0.943±0.021**0.856±0.046**0.852±0.047**0.896±0.054*0.817±0.081*
DFTD0.940±0.029*0.866±0.037**0.860±0.037**0.897±0.055*0.836±0.054**
3MT0.925±0.039*0.841±0.056**0.841±0.050**0.854±0.095*0.829±0.045**
DrFuse0.922±0.044*0.848±0.045**0.844±0.047**0.875±0.042**0.822±0.052**
HEALNet0.948±0.020*0.881±0.021**0.875±0.020**0.914±0.047*0.847±0.036**
Flex-MoE0.775±0.062**0.702±0.073**0.695±0.086**0.755±0.134*0.649±0.200*
IMDL0.921±0.042*0.858±0.037**0.854±0.042**0.883±0.040**0.833±0.072**
Q-Former0.969±0.015*0.886±0.041*0.888±0.039*0.868±0.089*0.904±0.062
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

(2) Naturally incomplete multimodal setting

MethodAUCbACCACCSENSPE
TMC0.889±0.025**0.802±0.023**0.818±0.027**0.763±0.069**0.841±0.055*
MMIN0.940±0.012**0.870±0.027**0.875±0.024**0.857±0.046**0.883±0.028**
ShaSpec0.939±0.010**0.865±0.020**0.873±0.020**0.846±0.045**0.885±0.033**
DFTD0.939±0.012**0.868±0.021**0.879±0.024**0.840±0.040**0.896±0.037*
3MT0.965±0.009**0.907±0.022*0.913±0.022*0.894±0.039**0.921±0.029
DrFuse0.929±0.018**0.859±0.019**0.873±0.013**0.824±0.055**0.894±0.029*
HEALNet0.935±0.014**0.872±0.017**0.876±0.011**0.863±0.043**0.881±0.019**
Flex-MoE0.806±0.026**0.717±0.032**0.748±0.026**0.640±0.091**0.794±0.053**
IMDL0.917±0.026**0.861±0.025**0.870±0.022**0.840±0.038**0.883±0.022**
Q-Former0.956±0.011**0.885±0.031*0.883±0.039*0.889±0.030**0.881±0.053*
Ours0.990±0.0060.966±0.0130.966±0.0160.965±0.0120.966±0.022

Table: Comparison experiment of sMCIvs. pMCI for unified feature encoder

(1) Complete multimodal setting

MethodAUCbACCACCSENSPE
TMC0.740±0.061*0.649±0.056*0.675±0.0640.599±0.042*0.699±0.071
MMIN0.750±0.039*0.650±0.064*0.676±0.0580.599±0.077*0.701±0.052
ShaSpec0.726±0.064*0.632±0.0720.653±0.1500.580±0.1740.684±0.226
DFTD0.726±0.056*0.640±0.050*0.676±0.1550.555±0.1780.725±0.242
3MT0.833±0.0450.755±0.0290.772±0.0730.729±0.0940.782±0.116
DrFuse0.691±0.063**0.618±0.068*0.680±0.0430.495±0.2100.741±0.100
HEALNet0.658±0.049**0.586±0.047**0.611±0.1020.531±0.2690.641±0.213
Flex-MoE0.659±0.037**0.596±0.047**0.633±0.059*0.524±0.195*0.667±0.130
IMDL0.762±0.035**0.691±0.035**0.701±0.0390.674±0.0280.709±0.042
Q-Former0.839±0.0400.752±0.0390.779±0.0570.700±0.0890.803±0.090
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

(2) Naturally incomplete multimodal setting

MethodAUCbACCACCSENSPE
TMC0.684±0.066**0.633±0.033**0.643±0.070*0.609±0.2050.658±0.190
MMIN0.731±0.048**0.668±0.045**0.665±0.051*0.676±0.1320.660±0.108
ShaSpec0.762±0.046*0.666±0.062*0.649±0.1090.715±0.1260.617±0.207
DFTD0.758±0.042**0.674±0.058*0.673±0.1170.679±0.1350.670±0.230
3MT0.843±0.0450.752±0.041*0.750±0.0470.759±0.1040.746±0.091
DrFuse0.745±0.070**0.671±0.054**0.665±0.042*0.687±0.1440.655±0.081
HEALNet0.715±0.039**0.630±0.026**0.606±0.068**0.700±0.1640.561±0.171
Flex-MoE0.718±0.065**0.651±0.051**0.635±0.031**0.696±0.1310.606±0.052*
IMDL0.740±0.050*0.675±0.034**0.683±0.0390.660±0.0250.690±0.043
Q-Former0.854±0.0320.761±0.0480.768±0.0300.743±0.1710.780±0.095
Ours0.863±0.0330.793±0.0260.778±0.0410.834±0.0320.752±0.070
Reviewer_yWf9OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FMD-IMHF addresses neurodegenerative disease diagnosis and progression prediction with incomplete multimodal data. The framework adapts modality-specific foudation models, distills them into lightweight student encoders, and aligns their representations in a shared diagnostic space. It then constructs a patient–modality hypergraph with two types of hyperedges: connections among a patient‘s available modalities and connections among similar patients within each modality. Hypergarph propagation aggregates complementary information without explicitly reconstructing missing modalities. Experiments cover AD diagnosis and MCI progression prediction on ADNI, external evaluation on AIBL and SCAN, and retraining for PD progression prediction on PPMI.

Critical strengths
  • Addresses clinically relevant modality missingness while reducing encoder costs.
  • Cross-patient aggregation provides a meaningful distinction from prior within-patient foundation-model interaction.
  • Ablations support the utility of semantic alignment and patient-similarity connections.
Critical weaknesses
  • The released snapshot lacks the core Q-Former baseline implementation, preventing verification of faithful reproduction.
  • Encoder efficiency does not establish end-to-end efficiency; dense graph operations and full-bank processing raise scalability concerns.
  • External cohorts are small, and improvements are not consistent across complete-modality settings.
Questions for authors
  • How much benefit comes from neighboring features versus predictive probabilities?
  • Please provide matched end-to-end runtime and memory measurements across reference-bank sizes, alongside paired uncertainty estimates for the reported gains.
5 comments
AuthorsOpenReview ↗

The Response to Weakness 1

We thank the reviewer for pointing out this issue. The Q-Former baseline is based on the method proposed by Gu et al. in their ICLR 2026 paper, Joint Adaptation of Uni-modal Foundation Models for Multi-modal Alzheimer's Disease Diagnosis. This work introduces a modality-anchored interaction framework that facilitates feature interactions among unimodal foundation models through Q-Former.

We have updated the anonymized code repository: https://anonymous.4open.science/r/ICLR_code-B819/. The implementation, training and evaluation scripts, and configuration files are provided in comparison/jointfm_qformer/, covering the core Q-Former query alignment, anchor rotation, and prediction aggregation procedures.

To ensure a fair comparison, we adopted the same modality-specific foundation models for MRI, PET, and Clinical as those used in our method and implemented and trained the Q-Former baseline under identical data splits and evaluation protocols. This design helps control for differences in foundation models and data conditions, allowing the comparison to primarily reflect the differences in multimodal interaction and fusion strategies. Since our study involves only MRI, PET, and Clinical, we adapted its modality-anchored interaction mechanism to our three-modality setting. Specifically, one modality and its corresponding foundation model are selected as the anchor, while the remaining modalities serve as auxiliary modalities. The auxiliary features are mapped into the feature space of the anchor modality through modality-aware Q-Formers and then jointly modeled with the anchor features. By rotating the three modalities as anchors and aggregating the predictions from different anchor configurations, the final classification results are obtained.

AuthorsOpenReview ↗

The Response to Question 1

We thank the reviewer for raising this question. For patient i and modality m, the initial node representation is defined as: xim=[sim;pim]x_{i}^{m}=[s_{i}^{m};p_{i}^{m}]. Here, sims_{i}^{m} is the 512-dimensional student representation, which retains rich modality-specific information, and pimp_{i}^{m} is the modality-specific predictive probability vector, which provides a compact, task-relevant summary of the model’s class prediction. In the original model, messages passed through intra-modality patient-similarity hyperedges contain both neighbors’ representations and their predictive probabilities.

In Appendix C.1, we performed a hyperedge ablation by separately removing the intra-patient cross-modal hyperedges and the intra-modality patient-similarity hyperedges. Removing the patient-similarity hyperedges blocks the cross-patient propagation of both neighbor representations and neighbor probabilities. Thus, this ablation measures the overall contribution of patient-similarity hyperedges, but does not separate the contributions of these two types of information. To separately assess the contributions of neighbor representations and neighbor predictive probabilities, we added a dedicated ablation study in the revised manuscript.

We evaluate four settings by independently enabling or disabling the propagation of neighboring features and predictive probabilities through intra-modality patient-similarity hyperedges. All settings use the same data splits, features, training setup, and kNN graphs (k = 8; cosine similarity; training subjects only). We report the mean ± standard deviation of AUC, ACC, bACC, SEN, and SPE over five-fold stratified cross-validation for AD vs. NC and sMCI vs. pMCI.

Table G: Ablation Study of Neighbor Information Components under Naturally Incomplete Multimodal Settings.

(a) AD vs. NC

SettingAUCACCbACCSENSPE
Full neighbor information (Ours)0.990±0.0060.966±0.0160.966±0.0130.965±0.0120.966±0.022
Neighbor features only0.987±0.0040.953±0.0050.949±0.006*0.937±0.009**0.960±0.005
Neighbor probabilities only0.984±0.003*0.946±0.005*0.940±0.005**0.924±0.011**0.956±0.008
Without neighbor information0.983±0.007**0.938±0.016**0.931±0.013**0.913±0.011**0.948±0.022**

(b) sMCI vs. pMCI

SettingAUCACCbACCSENSPE
Full neighbor information (Ours)0.863±0.0330.778±0.0410.793±0.0260.834±0.0320.752±0.070
Neighbor features only0.857±0.0280.771±0.039**0.782±0.025**0.814±0.032**0.750±0.067
Neighbor probabilities only0.840±0.023*0.758±0.039**0.762±0.029**0.775±0.027**0.750±0.058
Without neighbor information0.834±0.028**0.750±0.037**0.751±0.036**0.755±0.044*0.748±0.043

As shown in Table G, incorporating both types of neighbor information achieves the best overall performance on both AD vs. NC and sMCI vs. pMCI tasks. Using neighbor features sims_{i}^{m}alone generally outperforms using neighbor predictive probabilities pimp_{i}^{m} alone, indicating that feature-level information provides stronger complementary cues. Removing both components leads to the largest performance degradation, particularly in AUC, balanced accuracy, and sensitivity. These results demonstrate that sims_{i}^{m} and pimp_{i}^{m} provide complementary information, and their joint utilization enhances the effectiveness of patient--modality relational modeling.

We have also added this analysis to the manuscript in Appendix C.5, Ablation Study of Neighbor Information.

AuthorsOpenReview ↗

End-to-End Computational Performance

Under consistent hardware, batch size, and inference settings, we evaluate the end-to-end inference time and GPU memory consumption across different reference-bank sizes. For the AD vs. NC task, there are 1,545 samples, which are divided into 988 training samples, 248 validation samples, and 309 test samples. To prevent information leakage, the reference bank must exclude all test samples; otherwise, test subjects would appear in both the reference bank and the test set. Therefore, the maximum permissible reference-bank size in this fold is 1,236 (= 988 + 248). Accordingly, we evaluate six reference-bank sizes: 250, 500, 750, 1,000, 988, and 1,236. For the sMCI vs. pMCI task, the same principle is applied to determine the reference-bank sizes. We evaluate six configurations: 125, 250, 375, 500, 504 (the training-set reference-bank size following the protocol adopted in the original paper), and 631.

Table H: End-to-End Computational Performance under Different Reference Bank Sizes

(1) AD vs. NC

Reference Bank SizeNumber of NodesGraph Construction Time (ms)Hypergraph Processing Time (ms)End-to-End Inference Time (ms)Graph GPU Memory (MiB)End-to-End GPU Memory (MiB)
25013184852.0360.8139.1407.7
50019065052.3461.1157.6426.2
75025026082.8861.6578.6447.2
988 (reference )30416973.3862.15101.8470.4
100030796973.3762.15104.0472.5
1236 (max)36357383.7562.53131.5500.0

(2) sMCI vs. pMCI

Reference Bank SizeNumber of NodesGraph Construction Time (ms)Hypergraph Processing Time (ms)End-to-End Inference Time (ms)Graph GPU Memory (MiB)End-to-End GPU Memory (MiB)
1257833983.0161.7824.6393.2
25011385163.9762.7533.1401.7
37514865824.6163.3943.6412.2
50018305854.8563.6252.9421.5
504(reference)18366064.9863.7653.0421.6
631 (max)21936165.2864.0564.7433.3

To assess the statistical significance of performance differences across reference-bank sizes, we perform 2,000 paired bootstrap resamples using AUC and balanced accuracy (bACC) as evaluation metrics. The 95% confidence intervals (CIs) are estimated from the 2.5th and 97.5th percentiles, and two-sided p-values are calculated from the bootstrap distributions of paired performance differences.

Table I: Performance and Statistical Significance under Different Reference Bank Sizes

(1) AD vs. NC

Reference Bank SizeAUCbACCΔAUC vs 988 [95%CI]p_AUCΔbACC vs 988 [95% CI]p_bACC
2500.95600.8950−0.0287 [−0.0380, −0.0194]<0.001**−0.0493 [−0.0650, −0.0336]<0.001**
5000.96600.9100−0.0187 [−0.0270, −0.0104]<0.001**−0.0343 [−0.0660, −0.0026]0.030*
7500.98200.9410−0.0027 [−0.0052, −0.0002]0.034*−0.0033 [−0.0140, +0.0074]0.55
988 (reference )0.98470.9443————
10000.98520.9452+0.0005 [−0.0042, +0.0052]0.83+0.0009 [−0.0090, +0.0108]0.86
1236 (max)0.98600.9460+0.0013 [−0.0035, +0.0061]0.60+0.0017 [−0.0080, +0.0114]0.73

(2) sMCI vs. pMCI

Reference Bank SizeAUCbACCΔAUC vs 504 [95%CI]p_AUCΔbACC vs 504 [95% CI]p_bACC
1250.84000.7300−0.0361 [−0.0520, −0.0202]<0.001**−0.0654 [−0.0990, −0.0318]<0.001**
2500.85000.7500−0.0261 [−0.0440, −0.0082]0.004*−0.0454 [−0.0770, −0.0138]0.005**
3750.86200.7750−0.0141 [−0.0270, −0.0012]0.032*−0.0204 [−0.0380, −0.0028]0.024*
5000.87800.7980+0.0019 [−0.0065, +0.0103]0.66+0.0026 [−0.0174, +0.0226]0.80
504 (reference )0.87610.7954————
631(max)0.87550.7945−0.0006 [−0.0070, +0.0058]0.86−0.0009 [−0.0190, +0.0172]0.92
Reviewer_cuBtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FMD-IMHF for neurodegenerative disease diagnosis with incomplete multimodal data. The method distills modality-specific foundation models into lightweight encoders and uses cross-modal alignment to learn shared representations. A patient-modality hypergraph is then introduced to capture relationships across patients and modalities. Experiments on ADNI, AIBL, SCAN, and PPMI evaluate diagnostic performance, robustness to missing modalities, and computational efficiency.

Critical strengths
  1. The paper addresses a practical problem in multimodal medical diagnosis. Combining model distillation with incomplete-modality fusion is meaningful, particularly when computational resources and data availability are limited.
  2. The experiments cover different missing-modality settings and multiple cohorts. The ablation studies also provide evidence that cross-modal alignment and hypergraph fusion contribute to the final performance.
Critical weaknesses
  1. The methodological novelty is limited. Knowledge distillation, cross-modal alignment, and hypergraph-based multimodal fusion have been extensively studied. In particular, Liu et al. (2017) [1] already investigated hypergraph learning for AD diagnosis with incomplete multimodal data. The paper lacks sufficient comparisons with existing hypergraph-based methods to establish the benefit of the proposed design.
  2. The experimental improvements are not consistently convincing. On the complete multimodal MCI task, the proposed method performs slightly worse than the PET-only baseline in bACC (0.750 vs. 0.761). The external validation is also limited, particularly on AIBL with only 23 subjects. Moreover, the improvements over strong baselines are sometimes small, without statistical significance analysis.

[1] Liu M, Zhang J, Yap PT, Shen D. View-aligned hypergraph learning for Alzheimer’s disease diagnosis with incomplete multi-modality data. Medical image analysis. 2017 Feb 1;36:123-34.

Questions for authors
  1. Could the authors provide comparisons with existing hypergraph-based methods under the same experimental setting? This would help clarify the novelty and effectiveness of the proposed fusion strategy.
  2. Were the compared multimodal methods evaluated using the same pretrained feature extractors? How much of the improvement comes from foundation-model representations rather than the proposed fusion method?
  3. Could the authors provide confidence intervals or statistical significance tests for the improvements over the strongest baselines, particularly on the external cohorts?
9 comments
AuthorsOpenReview ↗

The Response to Weakness 1

We thank the reviewer for highlighting the concerns regarding the methodological novelty and the lack of comparisons with existing hypergraph-based methods.

Although knowledge distillation, cross-modal alignment, and hypergraph-based multimodal fusion have been explored in prior studies, their effective application to neurodegenerative disease diagnosis under limited-data and missing-modality settings remains challenging. To address these challenges, we propose the FMD-IMHF framework with two methodological contributions: (1) jointly leveraging modality-specific foundation models and designing a cross-modal knowledge distillation mechanism to align modality-specific feature representations, thereby enhancing feature learning under limited-data settings while reducing computational overhead; and (2) constructing a patient–modality hypergraph to jointly capture intra-patient cross-modal complementarity and inter-patient similarity within each modality, enabling information propagation across patients and modalities to compensate for missing modalities.

To better clarify these contributions, we have added a brief discussion of graph- and hypergraph-based methods in the Introduction (Lines 68–72), focusing on their ability to model relationships between patients and modalities. In Section 2.2 (Lines 120–129), we have expanded our discussion of existing hypergraph-based methods, including VAHL (MedIA 2017), MHL (JBHI 2018), MUSE (ICLR 2024) and PHGNN (PR 2026), to clarify their respective relationship modeling strategies and highlight the differences from our unified patient-modality hypergraph.

Furthermore, we have added comparative experiments with representative hypergraph-based methods under consistent experimental settings to evaluate the effectiveness and advantages of our proposed hypergraph design. To ensure a fair comparison of multimodal fusion methods and exclude performance gains from pretrained encoders, all compared methods use the frozen student models as feature extractors. The newly added hypergraph-based methods include:

(1) VAHL (Medical Image Analysis, 2017): Constructs view-specific hypergraphs based on modality availability and aligns different views through a regularizer for multi-view label fusion [1].

(2) MHL (IEEE JBHI, 2018): Constructs modality-specific hypergraphs and adaptively fuses available modality representations using learnable softmax weights [2].

(3) HGCN (IEEE TMI, 2023): Combines high-order hypergraph propagation with pairwise graph propagation via clique expansion to capture complementary structural relationships [3].

(4) MUSE (ICLR, 2024): Models patient–modality associations using a bipartite graph and learns patient representations through Edge GraphSAGE and contrastive learning[4].

(5) HDPL (IEEE JBHI, 2026): Generates attribute-based top-k hyperedges from projected clinical embeddings and derives patient-specific prompts from individual embeddings[5].

(6) PHGNN (Pattern Recognition, 2026): Aggregates modality nodes into patient representations and constructs a patient-level k-nearest-neighbor hypergraph using training-fold samples, augmented with learnable prompts[6].

[1] Liu, M., Zhang, J., Yap, P.-T., & Shen, D. (2017). View-aligned hypergraph learning for Alzheimer's disease diagnosis with incomplete multi-modality data. Medical Image Analysis, 36, 123–134.

[2] Liu, M., Gao, Y., Yap, P.-T., & Shen, D. (2018). Multi-hypergraph learning for incomplete multi-modality data. IEEE Journal of Biomedical and Health Informatics, 22(4), 1197–1208.

[3] W. Hou, C. Lin, L. Yu, J. Qin, R. Yu and L. Wang, "Hybrid Graph Convolutional Network With Online Masked Autoencoder for Robust Multimodal Cancer Survival Prediction," in IEEE Transactions on Medical Imaging, vol. 42, no. 8, pp. 2462-2473, Aug. 2023, doi: 10.1109/TMI.2023.3253760.

[4] Wu, Z., Dadu, A., Tustison, N., Avants, B., Nalls, M., Sun, J., & Faghri, F. (2024). Multimodal patient representation learning with missing modalities and labels. International Conference on Learning Representations (ICLR).

[5] Zhou, X., Huang, G., Zhao, Q., He, J., Yuan, X., Li, M., Pun, C.-M., Guo, L., Lei, B., & Yang, Q. (2026). HDPL: Hypergraph-based dynamic prompting learning for incomplete multimodal medical learning. IEEE Journal of Biomedical and Health Informatics, 30(10), 8417–8430.

[6] Liu, C., Cosmo, L., & Rossi, L. (2026). Diagnosing Alzheimer's disease using hypergraph neural networks with prompt tuning. Pattern Recognition, 176, 113290.

We have updated the corresponding results in Section 4.3 (Lines 382–386), Table 1, and Appendix C.2 (Complete Comparison Results on ADNI) of the revised manuscript.

AuthorsOpenReview ↗

The experimental results of hypergraph-based methods

The results of the newly added hypergraph-based baselines are presented below. Tables A and B report the performance of different methods on the AD vs. NC and sMCI vs. pMCI tasks, respectively, under both complete and naturally incomplete multimodal settings. Statistical significance was assessed using a paired t-test. * and ** indicate p < 0.05 and p < 0.01.

Table A: Comparison of Hypergraph-Based Methods for AD vs. NC Classification

(a) Complete multimodal setting

MethodAUCbACCACCSENSPE
VAHL0.934±0.0470.866±0.061*0.862±0.070*0.887±0.025**0.844±0.105
MHL0.962±0.017*0.902±0.025**0.904±0.027**0.896±0.021**0.908±0.039*
HGCN0.958±0.020*0.904±0.033*0.904±0.034*0.912±0.033*0.896±0.045*
MUSE0.967±0.016*0.920±0.024*0.920±0.024*0.918±0.042*0.921±0.036
HDPL0.947±0.0370.886±0.052*0.884±0.062*0.897±0.026**0.875±0.105
PHGNN0.875±0.079*0.778±0.070**0.774±0.060**0.824±0.1470.731±0.060**
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

(b) Naturally incomplete multimodal setting

MethodAUCbACCACCSENSPE
VAHL0.938±0.021*0.867±0.031**0.880±0.028**0.835±0.059*0.899±0.039*
MHL0.958±0.014**0.883±0.031**0.900±0.023*0.842±0.028**0.924±0.029
HGCN0.956±0.013**0.885±0.023**0.894±0.025**0.861±0.045*0.908±0.040
MUSE0.968±0.010**0.905±0.020**0.917±0.011*0.874±0.054*0.935±0.021
HDPL0.954±0.017*0.874±0.019**0.892±0.025*0.829±0.036**0.919±0.041
PHGNN0.918±0.025**0.837±0.032**0.843±0.012**0.820±0.100*0.853±0.040**
Ours0.990±0.0060.966±0.0130.966±0.0160.965±0.0120.966±0.022

Table B: Comparison of Hypergraph-Based Methods for sMCI vs. pMCI Classification

(a) Complete multimodal setting

MethodAUCbACCACCSENSPE
VAHL0.815±0.029**0.710±0.035*0.715±0.080*0.700±0.0820.720±0.128
MHL0.846±0.0160.755±0.0430.751±0.0850.761±0.0650.748±0.128
HGCN0.847±0.0320.734±0.0360.738±0.0870.728±0.0900.741±0.140
MUSE0.756±0.0350.632±0.051**0.642±0.093**0.612±0.055*0.652±0.135**
HDPL0.826±0.0370.697±0.039*0.706±0.087*0.680±0.072*0.713±0.136
PHGNN0.786±0.034*0.676±0.034**0.678±0.079*0.673±0.079*0.679±0.126*
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

(b) Naturally incomplete multimodal setting

MethodAUCbACCACCSENSPE
VAHL0.851±0.031**0.772±0.025**0.749±0.0400.838±0.0430.707±0.072
MHL0.838±0.0240.759±0.026**0.746±0.041**0.795±0.032**0.724±0.070**
HGCN0.813±0.035**0.722±0.023**0.710±0.035**0.755±0.022**0.689±0.058*
MUSE0.807±0.025*0.711±0.031**0.702±0.043**0.735±0.018**0.687±0.065**
HDPL0.825±0.0470.740±0.051*0.726±0.0450.782±0.1620.699±0.107
PHGNN0.832±0.036**0.767±0.0380.737±0.0600.850±0.0890.685±0.115
Ours0.863±0.0330.793±0.0260.778±0.0410.834±0.0320.752±0.070

For AD vs. NC classification, our method achieves the highest AUC, bACC, ACC, SEN, and SPE under both complete and naturally incomplete modality settings, with several differences reaching statistical significance. These results support the performance advantage of our fusion framework over the evaluated hypergraph methods and its ability to use available multimodal information effectively despite missing modalities.

For sMCI vs. pMCI classification, the advantage is more modest. Under complete modalities, our method achieves the highest AUC and ACC, while some competing methods perform comparably or better on other metrics. Under naturally incomplete modalities, it achieves the highest AUC, bACC, ACC, and SPE, although its SEN is slightly lower than that of PHGNN. Compared with the complete multimodal setting, the naturally incomplete multimodal setting achieves better performance across most evaluation metrics. We attribute this behavior mainly to the joint modeling of complementary information through within-modality and within-patient hyperedges.

AuthorsOpenReview ↗

The Response to Weakness 2

We thank the reviewer for pointing out these weaknesses in our experimental design and analysis. In response, we re-examined the experiments and made the following revisions.

(1) Regarding the Lower Performance of Multimodal Methods Compared with PET-only

The superior performance of some unimodal baselines over multimodal methods was mainly attributable to two issues in our original experimental design. First, unimodal baselines used all available subjects for each modality, whereas complete multimodal methods were evaluated only on subjects with all three modalities (545 for AD vs. NC and 615 for sMCI vs. pMCI), resulting in larger cohorts for the unimodal baselines. Second, the unimodal baselines employed modality-specific student encoders distilled from foundation models, providing stronger feature representations, whereas the other multimodal baselines did not consistently use the same distilled encoders. To address these issues, we revised the experimental design and conducted additional controlled experiments to ensure fair comparisons.

We repeated the experiments for all three unimodal baselines using the same complete-case subjects as the corresponding multimodal methods, with consistent data splits and evaluation protocols. The results before and after revision are presented below:

Table C: Performance of unimodal methods on AD vs. NC.

(a) Before Revision

MethodAUCbACCACCSENSPE
MRI-only0.936±0.0150.868±0.0250.868±0.0210.870±0.0450.867±0.023
PET-only0.968±0.0160.901±0.0240.901±0.0230.911±0.0410.892±0.037
Clinical-only0.806±0.0280.721±0.0400.741±0.0370.670±0.0660.771±0.045
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

(b) After Revision

MethodAUCbACCACCSENSPE
MRI-only0.885±0.028**0.788±0.036**0.791±0.036**0.765±0.078**0.810±0.068**
PET-only0.949±0.022**0.878±0.040**0.884±0.035**0.833±0.093*0.923±0.046
Clinical-only0.800±0.052**0.724±0.030**0.719±0.043**0.756±0.093**0.691±0.138**
Ours0.996±0.0070.976±0.0110.974±0.0120.991±0.0120.961±0.018

Table D: Performance of unimodal methods on sMCI vs. pMCI.

(a) Before Revision

MethodAUCbACCACCSENSPE
MRI-only0.774±0.0530.722±0.0570.718±0.0490.733±0.0740.711±0.047
PET-only0.830±0.0220.761±0.0380.776±0.0600.734±0.0830.788±0.089
Clinical-only0.717±0.0420.633±0.0350.635±0.0750.628±0.1550.638±0.172
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

(b) After Revision

MethodAUCbACCACCSENSPE
MRI-only0.723±0.024**0.661±0.028*0.704±0.0430.578±0.1410.744±0.097
PET-only0.795±0.035*0.722±0.0590.737±0.0490.694±0.0770.750±0.041
Clinical-only0.647±0.037**0.599±0.049**0.646±0.075*0.510±0.163*0.688±0.135
Ours0.850±0.0310.750±0.0440.751±0.0890.748±0.0820.752±0.137

Note: Asterisks indicate statistically significant differences from Ours based on two-sided paired t-tests (*p < 0.05; **p < 0.01).

The revised results reveal substantial changes in several performance metrics. Accordingly, we have updated Table 1(a) in the main manuscript and Table A4 in the Appendix to reflect these changes.

In our comment titled "The Response to Question 2", we additionally evaluated the multimodal methods using unified modality-specific student encoders distilled from foundation models.

We thank the reviewer again for pointing out this issue, which has helped us improve the fairness and reliability of our experimental evaluation.

(2) Regarding the limitations of external validation.

In the AIBL dataset, 144 subjects were diagnosed with MCI at baseline. However, distinguishing sMCI from pMCI requires sufficient longitudinal follow-up information. Therefore, subjects with only baseline diagnoses or insufficient follow-up data to determine their MCI conversion status were excluded, leaving 23 eligible subjects. Given the limited number of eligible subjects in AIBL, we also conducted external validation using the SCAN cohort. The SCAN cohort comprises 90 subjects, including 62 with sMCI and 28 with pMCI. This larger independent cohort provides additional evidence for evaluating the generalization ability of our model.

In addition, cross-disease generalization experiments on the PPMI dataset further demonstrate the applicability of our method to other disease scenarios.

(3) Regarding the Lack of Statistical Significance Analysis

We have added statistical significance tests and described them in detail in our comment titled “The Response to Question 3”.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.