acceptodds
Under review as a conference paper at ICLR 2027

RPMF: Reliability-Aware Progressive Multimodal Fusion Framework for Multimodal Brain Disease Diagnosis

Abstract

Multimodal learning has emerged as a promising paradigm for brain disease diagnosis by integrating complementary information from heterogeneous clinical modalities. However, the diagnostic reliability of different modalities varies substantially across individual subjects, suggesting that multimodal data should be integrated adaptively according to their reliability. Nevertheless, many existing multimodal methods estimate modality importance only implicitly and directly fuse all modalities in a single step, limiting their ability to prioritize reliable modalities while suppressing noisy or conflicting information. To address these limitations, we propose a Reliability-aware Progressive Multimodal Fusion (RPMF) framework for brain disease diagnosis. Specifically, RPMF first learns modality-specific representations and explicitly estimates subject-specific modality reliability using an entropy-based uncertainty formulation, allowing the model to quantify the diagnostic confidence of each modality. To characterize heterogeneous disease patterns, each modality is further represented by a set of latent prototypes, from which cross-modal compatibility is evaluated through prototype-level relation modeling. Guided by both reliability and compatibility, multimodal information is progressively integrated in a reliability-aware manner, where the most reliable modality serves as the initial anchor and subsequent modalities are sequentially incorporated according to the reliability and compatibility with the current fused representation. This progressive strategy enables the model to adaptively exploit useful modalities that are compatible with the current multimodal representation while suppressing noisy or conflicting information. Extensive experiments on multiple brain disease datasets demonstrate that RPMF consistently outperforms state-of-the-art methods.

Primary area:
applications to neuroscience & cognitive science
Keywords:
Multimodal learning, brain disease diagnosis, reliability-aware multimodal fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6VQmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

RPMF estimates subject-specific modality reliability from predictive entropy and progressively fuses modalities using reliability and prototype-level compatibility. Experiments on TADPOLE and ABIDE report improved classification performance, with ablations and analyses of reliability and modality selection.

Critical strengths
  1. The framework provides a clear mechanism for using both individual modality confidence and compatibility with the current fused representation.
  2. The TADPOLE results are strong. Evaluation across two disease datasets, together with component ablations and reliability analyses, provides useful evidence of the framework’s effectiveness.
Critical weaknesses
  1. The methodological novelty is limited. RPMF combines predictive entropy, multiple latent projections, set-based distance, cross-attention, and gated residual fusion, all of which are established techniques. Its main distinction is the reliability- and relation-guided fusion order, but this contribution is not isolated through fixed-order, random-order, or reliability-only comparisons under the same architecture. The work therefore appears closer to an engineering combination than a clearly established methodological advance.
  2. The proposed reliability score primarily measures predictive confidence rather than correctness. The historical-loss regularizer does not provide a theoretical guarantee of correct reliability ordering; in particular, Eq. (7) permits equal scores and some reversed rankings without penalty. The reported agreement and correlation results show only a moderate empirical association and do not establish calibration or robustness to high-confidence errors.
  3. The role and interpretation of the latent prototypes remain unclear. They are sample-specific projections generated from the same global feature, without diversity constraints, prototype supervision, or semantic analysis. The ablation shows that these additional latent vectors improve performance, but does not demonstrate that they capture distinct disease patterns or that the gain comes from prototype modeling rather than increased capacity and a different fusion representation.
  4. Changing the fusion order does not by itself resolve unreliable evidence. All modalities are eventually incorporated, and the method neither removes corrupted information nor identifies reliable components within a modality. A low-reliability modality can therefore still perturb the fused representation. Without controlled corruption or conflict experiments, the claim that RPMF suppresses noisy or conflicting evidence is insufficiently supported.
Questions for authors
  1. What is the main methodological advance beyond combining existing confidence estimation, set matching, and progressive cross-attention components? How does RPMF compare with fixed-order, random-order, and reliability-only ordering using the same fusion architecture?
  2. Why should predictive entropy be interpreted as reliability rather than confidence? Is Eq. (7) intended to allow equal or partially reversed reliability rankings? Please provide calibration results, high-confidence error analysis, and comparison with always selecting the strongest fixed modality.
  3. What do the latent prototypes represent, and are their roles consistent across subjects? Can the authors provide diversity or semantic analyses and a capacity-matched control to distinguish prototype learning from additional nonlinear projections?
  4. How does RPMF behave when one modality is corrupted or confidently conflicts with the others? Would explicitly dropping a low-reliability modality outperform merely changing its fusion order, and how do the predictions and gate values change at each fusion step?
Reviewer_mFcPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Reliability-aware Progressive Multimodal Fusion (RPMF) framework for brain disease diagnosis. The method estimates the reliability of each modality using prediction entropy, uses prototype-based representations to measure the relationships between different modalities. Based on reliability and cross-modal relationships, it fuses modalities through cross-attention. Experiments on TADPOLE and ABIDE show that RPMF achieves better performance.

Critical strengths
  1. The paper addresses an important problem in multimodal brain disease diagnosis.

  2. The proposed method achieves good results on two benchmark datasets.

Critical weaknesses
  1. The paper uses prediction entropy to estimate modality reliability. However, high confidence does not always mean high reliability, since a model can make incorrect predictions with high confidence. In addition, different unimodal classifiers may have different levels of confidence. It is unclear whether their reliability scores can be directly compared.

  2. The ranking loss in Eq. (7) seems inconsistent with its goal. The authors aim to assign higher reliability scores to samples with lower historical prediction loss. However, the current loss does not enforce this ordering. For example, the loss can be zero even when two samples with different prediction difficulties have the same reliability score.

  3. The paper claims that progressive fusion is better than one-shot fusion. However, the current experiments do not clearly show whether the performance gain comes from the fusion order or from the additional cross-attention operations. The authors only compare the proposed method with weighted fusion. Could the authors provide comparisons with fixed, random, and reversed modality orders while keeping the other parts of the model unchanged?

  4. A main motivation of the paper is to reduce the effect of noisy or conflicting modalities. However, the experiments do not directly test this claim. Also, the proposed method eventually includes all modalities, even those with low reliability. It is therefore unclear whether the method can effectively reduce the influence of unreliable information.

  5. Although the proposed method achieves the best reported results, the improvements on ABIDE are relatively small (1.27% in ACC and 0.39% in AUC). The following issues should be further addressed: 1) The paper lacks statistical significance tests to show whether these improvements are reliable. 2) Since reliability estimation is a key part of the method, the authors should report calibration metrics, such as Expected Calibration Error (ECE), to evaluate the quality of the estimated confidence. 3) The comparison with existing methods is not sufficient. It would be helpful to include more uncertainty-aware baselines, especially EDL-based multi-view or multimodal fusion methods. 4) The experiments are limited to two relatively small datasets. More experiments on larger datasets or across different clinical sites would help evaluate the generalization ability of the method. 5) Since the method aims to reduce the effect of unreliable modalities, experiments under noisy or conflicting modality settings are needed to verify its robustness. 6) It is suggested to compare RPMF with each individual modality under the same experimental settings.

Questions for authors

See Weaknesses.

Reviewer_9NoLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a reliability-aware progressive multimodal fusion framework called RPMF for the diagnosis of multimodal brain diseases. The method first learns modality-specific representations and estimates individual-specific modality reliability using prediction entropy. Furthermore, it utilizes historical single-modality prediction loss and regularizes the reliability estimate using a pairwise ranking objective function. Each modality representation is projected onto multiple latent prototypes, and cross-modal compatibility is measured through bidirectional prototype matching. RPMF starts with the modality with the highest reliability estimate and progressively fuses the remaining modalities based on joint reliability relationship scores, performing the fusion using a prototype-level cross-attention mechanism.

Critical strengths

1、This paper explores an important practical problem in multimodal medical learning: the diagnostic utility of different modalities may vary from person to person, therefore treating all modalities equally may not be optimal. The individualized fusion strategy proposed in this paper is reasonable and clearly explained. 2、On the TADPOLE and ABIDE datasets, the RPMF algorithm achieves continuous improvement across a relatively wide range of multimodal baseline models, and the authors provide component-level ablation analysis, rather than relying solely on the final comparison results.

Critical weaknesses
  1. Compared to existing multimodal fusion methods with perceived uncertainty (especially QMF), this method offers limited methodological innovation. The reliability estimation mechanism is closely related to existing methods that utilize prediction uncertainty and historical single-modal loss to estimate modality quality. RPMF employs a similar strategy, with its main add-ons being prototype-based relational modeling and progressive fusion. However, this paper does not adequately elucidate the methodological differences or demonstrate the extent to which these add-ons contribute to existing methods. Furthermore, the proposed prototypes are generated from the same global modality embeddings using independent MLPs, lacking explicit diversity or separation constraints, making it difficult to interpret them as different disease-related modalities.

  2. The core argument for suppressing unreliable or conflicting modalities is not adequately validated. Experiments were conducted under standard fixed-modality settings without controlled modality corruption or missing modality assessments. Low prediction entropy does not necessarily imply reliable predictions, as the model may still produce highly confident but erroneous outputs. Moreover, the progressive fusion strategy ultimately integrates all available modalities, and the positive soft-addition fusion strength does not explicitly exclude unreliable modalities. Based on current experiments, it cannot be directly proven whether RPMF can effectively identify and suppress real noise or conflict information.

Questions for authors
  1. Please further elaborate on the differences between the proposed method and QMF. The paper's method seems similar to QMF in terms of prediction uncertainty and historical single-modal loss. A more direct comparison is suggested for differentiation.
  2. Please supplement the experiments under controlled modal corruption. Does the proposed progressive fusion provide a sustained improvement in robustness compared to existing dynamic fusion methods? 3.Since the prototypes are independently generated from the same global modal representation using multiple MLPs, there is a risk of redundant representations or convergence to similar modes. Can the authors provide quantitative evidence of prototype diversity and demonstrate whether prototype-level matching offers additional advantages compared to parameter-matched multi-branch projection baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.