acceptodds
Under review as a conference paper at ICLR 2027

CARE-Fuse: Reliability-Aware Multimodal Fusion Under Missing and Conflicting Clinical Evidence

Abstract

Multimodal fusion must remain effective when evidence is missing or expert predictions conflict. We introduce CARE-Fuse, a task-adaptive late-fusion framework that incorporates evidence availability and cross-expert disagreement into validation-trained gates. A reliability gate is trained with label-independent validation perturbations to improve robustness to missing and conflicting inputs. We evaluate CARE-Fuse on clinical prediction tasks by combining first-24-hour vital signs and laboratory features, temporal representations, report embeddings, and language-model scores. On a locked test cohort of 14,637 MIMIC ICU stays, CARE-Fuse achieves AUROC/AUPRC of 0.8674/0.5344 for in-hospital mortality and 0.8310/0.6878 for stays longer than three days. Relative to the strongest single expert, paired bootstrap analysis with 5,000 draws yields gains of 0.0428 AUROC and 0.0671 AUPRC for mortality, and 0.0386 AUROC and 0.0960 AUPRC for prolonged stay; all 95% confidence intervals exclude zero. Under missing temporal or report evidence and conflicting language-model outputs, the perturbation-trained reliability gate reduces degradation compared with an unaugmented gate, while a clinical-only stack remains competitive on clean data. Calibration and selective-prediction analyses further characterize evidence coverage, expert disagreement, and case-review behavior. These findings support perturbation-trained fusion with explicit availability and disagreement features as a practical approach to robustness in multimodal clinical prediction.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Multimodal learning, missing-modality robustness, clinical risk prediction, reliability-aware fusion, expert disagreement, selective prediction

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.20 over 5 reviews, above 2% of the others.
Reviewer_eyHkOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces CARE-Fuse, a task-adaptive late-fusion framework that incorporates evidence availability and cross-expert disagreement into validation-trained gates, with a reliability gate trained with label-independent validation perturbations to improve robustness to missing and conflicting inputs

Critical strengths

The topic is interesting.

Critical weaknesses

The paper claims Reliability-Aware Multimodal Fusion Under Missing and Conflicting Clinical Evidence, but I didn't see any fusion methods or any missing/conflicting evidence-related contributions. Please be careful and responsible with the paper you submitted.

The authors claim that a reliability gate is trained with label-independent validation perturbations to improve robustness to missing and conflicting inputs, but I didn't see any description of the reliability gate.

Questions for authors

It's not clear to me how the fusion method you proposed addresses the missing and conflicting evidence problem.

Reviewer_EweuOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

While the paper claims in the abstract to introduce the CAREFuse framework, a reading of the Methods section shows that it is not a new method but primarily an evaluation analysis, as the authors’ own introduction also indicates. In fact the work reads more like an exploratory evaluation of existing methods (undergrad level student project) than a substantial scientific contribution. Therefore I do not believe the paper currently meets the standard expected for acceptance at a leading AI conference or journal.

Critical strengths

None

Critical weaknesses

See above summary

Questions for authors

NA

Reviewer_uSsGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary
  • This paper tackles data missingness and conflicting evidence within a late fusion framework referred to as CARE-Fuse. It has four fusion variants: clinical stack, simple five-score fusion, engineered fusion, and perturbation variant.
  • The model combines the first 24-hour vital signs and labs, temporal representations, report embeddings, and language-model scores.
  • The model is tested on in-hospital mortality prediction and long length of stay (longer than three days).
Critical strengths
  • The study uses the MIMIC dataset, so the results are on a publicly available dataset which would allow reproducibility.
  • The study investigates an important question related to evidence availability in fusion frameworks.
Critical weaknesses
  • I was unable to find a link to the code in the paper.
  • The language is extremely difficult to understand and seems over complicated. Some terms seem to be referring to simpler things but are using complex language to describe it. For example, what is a "scalar-score interface", "analyze frozen results", or "outcomes describe the whole stay"?
  • The introduction begins to describe predictive systems ordering cases, displaying probability, or selecting records for further processing, and then connects this to multimodal learning. It later describes how to select models and classifiers. This feels a bit disconnected from the main contributions of the paper.
  • The contributions are a bit narrow as they are stated, focusing only on specific results.
  • The related work section is limited (one paragraph).
  • In Table 1, the results seem similar across AUROC and no statistical significance testing is performed.
  • The paper is missing fusion baselines.
Questions for authors
  • What do you mean by Luna auxiliary channel?
  • Can you make it clear what the main objective of the study is?
Reviewer_fZ6jOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes CARE-Fuse, a late-fusion framework that incorporates modality availability and disagreement among expert predictions into a learned fusion gate. Its reliability variant is trained with perturbations to validation-set expert outputs to improve robustness to missing and conflicting evidence. Experiments on clinical prediction tasks using MIMIC data show that the perturbation-trained gate reduces performance degradation relative to its unaugmented counterpart.

Critical strengths

Handling conflicting expert predictions is a meaningful problem, particularly when multimodal language models can produce unreliable or hallucinated outputs that conflict with predictions from specialized models. Explicitly considering availability and disagreement is a useful direction.

Critical weaknesses
  1. The presentation makes the framework and its contributions difficult to follow. As a multimodal learning researcher, I found a lot of the descriptions in the paper difficult to follow. For example, lines 049–051 state: "CARE-Fuse examines frozen multimodal clinical outputs through four fusion variants: a clinical stack, simple five-score fusion (S), engineered fusion with availability and disagreement features (U), and a perturbation-trained variant (R). Three fixed rankings are then applied to U.” The phrase "clinical stack" and "three fixed rankings" are introduced without sufficient context. The contribution statements are similarly difficult to understand: "Coverage and output diagnostics distinguishing referenced from readable images, and continuous ranking changes from hard-decision changes under image replacement." and "An error-objective analysis connecting validation threshold selection to fixed-budget ranking. Validation balanced accuracy implies FN/FP weights of approximately 7.66 for mortality and 2.17 for LOS, above the respective 10% selection crossovers of 6.118 and 1.571. Selected and retained confusion counts explain the policy reversal." These statements present specialized terminology and numerical results before explaining the underlying ideas or what new insight the paper contributes. The framework figure (Figure 1) provides limited help in resolving this confusion.

  2. Methodological details and design justifications are insufficiently clear. For example, how are the "structured vital-sign/demographic/admission scores (p_v)" and "laboratory-summary scores (p_l)" generated? For the "GRU-representation scores (p_g)" and "RadBERT-representation scores (p_r)," what exactly are the inputs to the GRU and RadBERT, and how are their representations converted into prediction scores? What model does "a Luna auxiliary channel" refer to? Its identity and the relevant reference should be provided when it is first introduced. Likewise, the statement "Images reach fusion through the auxiliary channel; Sol is a separate comparator" leaves the input pipeline unclear: how are images supplied to the auxiliary model, and what model does “Sol” refer to?

Questions for authors
  1. How does CARE-Fuse compare with other late-fusion methods, such as probability averaging and learned weighted averaging?
  2. Could the method be evaluated on another dataset, such as ADNI, eICU, or MC-MED, where compatible modalities and tasks are available?
  3. Why are the specific 15 features in Equation (1) selected? How does performance change with fewer or additional features?
  4. For "engineered fusion with availability and disagreement features (U)," (Line 050) could the authors evaluate availability-only, disagreement-only, and combined variants?
  5. Section 4.2 states that "Vitals and laboratory scores remain intact." Why are these branches excluded from perturbation? It also states: "Conflict sets pa=1−mean⁡(pv,pl,pg,pr)p_a=1-\operatorname{mean}(p_v,p_l,p_g,p_r), preserving candidate-score availability." why is candidate-score availability preserved?
  6. "All 76 floating-to-integer checks agree within (10^{-12})." (Line 185) Appendix F.5 connects these checks to reconstructed confusion counts, but which quantities are checked, how is 76 obtained, and what do these checks establish?
  7. "Costs are sensitivity parameters, not estimated clinical utilities." (Line 194), What does the "sensitivity parameters" and "estimated clinical utilities" mean here?
  8. "U and R have different fitting seeds and separately selected hyperparameters." (Line 164) Could the authors report comparisons with matched fitting seeds and cross-validation folds across multiple runs, using the same tuning protocol, to better isolate the effect of perturbation training?
Reviewer_FGjkOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents CARE-Fuse, a reliability-aware late-fusion framework that combines clinical expert predictions with evidence availability and disagreement features to improve robustness under missing and conflicting modalities. The authors introduce perturbation-trained fusion and analyze calibration, selective prediction, and cost-sensitive case-review strategies. Experiments on 73,181 MIMIC-IV ICU stays evaluate in-hospital mortality and prolonged ICU stay prediction. Results show comparable performance across fusion methods on clean data, while perturbation training improves robustness under simulated missingness and expert conflicts. The study further demonstrates how different false-positive and false-negative costs influence case-selection strategies.

Critical strengths

The paper provides a systematic evaluation of multimodal clinical prediction under missing and conflicting evidence, highlighting the importance of robustness beyond standard discrimination metrics. Its cost-sensitive selective prediction analysis is particularly valuable, demonstrating how different false-positive and false-negative costs can change the preferred case-review strategy and offering practical insights for clinical decision support.

Critical weaknesses
  1. The methodological novelty is limited, as numerous existing frameworks have already addressed similar problems. The proposed approach does not demonstrate a particularly distinctive or substantial methodological contribution.

  2. Could the performance degradation of U and the Clinical stack under modality removal be driven by learned associations between availability masks and outcomes, rather than true robustness limitations? Similarly, could the degradation of S and U under auxiliary conflict result from shifting Luna’s highly concentrated logits (around −11.5-11.5) outside the training distribution? Additional mask ablations and more realistic perturbation experiments would help clarify these effects.

  3. On clean data, the AUROC differences among the four fusion approaches are below 0.001. Under stress conditions, CARE-Fuse-R improves over the simple scores-only baseline (S) by only approximately 0.002, suggesting limited incremental benefits from the engineered features and perturbation training.

  4. Training and testing employ the same perturbation families, with perturbations applied only to prediction scores and availability masks rather than raw images or clinical text. Therefore, the results do not establish robustness to realistic distribution shifts.

  5. The dataset is split at the ICU-stay level, allowing the same patient to appear across training and test sets. Furthermore, a consistent 24-hour prediction landmark is not enforced, and laboratory and report availability windows are not strictly constrained. The authors acknowledge that these limitations cannot be retrospectively resolved.

Questions for authors
  1. Could the authors clarify the methodological novelty of CARE-Fuse, given its limited improvement over the simple scores-only baseline (S)? Could the degradation under modality removal and auxiliary conflict be driven by learned mask–outcome associations and out-of-distribution shifts in Luna's logits rather than genuine robustness limitations?

  2. How do the authors address potential patient-level and temporal data leakage from ICU-stay-level splitting and inconsistent 24-hour observation windows? Additionally, why does the abstract report results from Appendix A rather than the main results in Table 1?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.