- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies label-correlated modality availability, which may encourage models to exploit missingness patterns as predictive shortcuts. Motivated by the argument that such shortcut learning induces representation inconsistency, it proposes AURA to encourage consistent representations across modality-availability patterns, together with a controlled evaluation protocol for missingness-mechanism shifts. Experiments on MIMIC-IV and ADNI assess predictive performance, while additional MNAR-shift experiments show improved robustness over the compared baselines.
Critical strengths
The paper addresses the important problem of model failures under shifts in modality missingness mechanisms and introduces a controlled evaluation protocol to assess robustness to such shifts.
Critical weaknesses
- The connection between missingness shortcuts and representation consistency is insufficiently explained. Although the bound motivates stability across missingness patterns, representation changes may also reflect the loss of task-relevant information caused by missing modalities. More direct evidence is needed to establish that the proposed objective reduces reliance on missingness shortcuts.
- The baseline comparison does not sufficiently cover recent methods specifically designed for missing-modality learning. MASAM and AMC primarily address modality imbalance rather than missing modalities. Comparisons with recent dedicated missing-modality methods are needed.
Questions for authors
The new controlled protocol has only been validated in the classification of strokes. Can the main conclusions be replicated in another task or dataset?
5 comments
Author Response to Reviewer RdHi (Part 2/3)
(Continued from Part 1)
W2. Comparison with recent dedicated missing-modality methods
The baseline comparison does not sufficiently cover recent methods specifically designed for missing-modality learning. MASAM and AMC primarily address modality imbalance rather than missing modalities.
We added two recent dedicated missing-modality methods, DyMo (Du et al., ICLR 2026) and HAD (Liu et al., NeurIPS 2025), and evaluated them on all four prediction tasks under the Standard setting and stroke type prediction (STP) under Controlled MNAR Shift Evaluation protocol.
Unlike MASAM and AMC, which primarily address modality imbalance, these methods directly target incomplete multimodal learning. DyMo dynamically selects reliable recovered modalities at inference time based on task-relevant information, while HAD combines hierarchical feature extraction, Hilbert curve-based Mamba, and mutual-information learning to integrate heterogeneous incomplete modalities while preserving modality-specific information.
1. Standard setting: AURA outperforms recent dedicated missing-modality methods across all four tasks.
Table 1. Standard-setting results on stroke type prediction (STP) and length-of-stay (LOS) prediction. Mean ± standard deviation over five folds. Best results are shown in bold.
| Method | STP BAcc (%) | STP F1 (%) | STP κ | LOS MSE | LOS MAE | LOS R² |
|---|---|---|---|---|---|---|
| DyMo | 69.34 ± 1.18 | 69.86 ± 0.98 | 0.5489 ± 0.0179 | 0.4276 ± 0.0287 | 0.4880 ± 0.0136 | 0.4015 ± 0.0402 |
| HAD | 66.65 ± 1.12 | 67.36 ± 1.07 | 0.5174 ± 0.0171 | 0.3911 ± 0.0168 | 0.4853 ± 0.0098 | 0.4527 ± 0.0219 |
| AURA | 73.49 ± 0.33 | 73.77 ± 0.75 | 0.6046 ± 0.0057 | 0.3216 ± 0.0060 | 0.4400 ± 0.0022 | 0.5498 ± 0.0105 |
Table 2. Standard-setting results on 100-day mortality prediction and ADNI disease-stage prediction. Best results are shown in bold.
| Method | Mort. BAcc (%) | AUROC (%) | AUPRC (%) | ADNI BAcc (%) | F1 (%) | κ |
|---|---|---|---|---|---|---|
| DyMo | 69.39 ± 3.50 | 85.90 ± 0.59 | 67.03 ± 1.51 | 58.01 ± 1.29 | 54.84 ± 2.99 | 0.3353 ± 0.0488 |
| HAD | 71.88 ± 2.27 | 84.14 ± 0.78 | 63.43 ± 3.05 | 55.41 ± 2.02 | 55.06 ± 2.24 | 0.3108 ± 0.0339 |
| AURA | 74.69 ± 1.67 | 86.86 ± 0.69 | 69.90 ± 1.19 | 61.65 ± 1.70 | 60.84 ± 1.21 | 0.3901 ± 0.0204 |
AURA achieves the best performance across all reported metrics on all four tasks (Tables 1–2), The improvements are particularly pronounced on LOS regression, where AURA reduces MSE from 0.3911–0.4276 to 0.3216 and increases to 0.5498, and on ADNI, where Cohen's improves from 0.3108–0.3353 to 0.3901. These results support AURA's ability to exploit complementary cross-modal information while maintaining strong predictive performance under incomplete inputs.
2. Controlled MNAR Shift Evaluation protocol: AURA remains more robust than dedicated missing-modality methods.
We further compared AURA with DyMo and HAD under Controlled MNAR Shift Evaluation protocol on STP at , where the label–missingness relationship changes between TEST-SAME and TEST-SHIFT.
Table 3. Robustness on STP under Controlled MNAR Shift Evaluation protocol (). Parentheses indicate relative drops from TEST-SAME to TEST-SHIFT; bold denotes the best TEST-SHIFT performance.
| Method | Setting | BAcc (%) | F1 (%) | Cohen's κ |
|---|---|---|---|---|
| DyMo | TEST-SAME | 67.23 ± 2.68 | 67.79 ± 2.28 | 0.5159 ± 0.0239 |
| TEST-SHIFT | 41.01 ± 1.08 (−39.0%) | 41.33 ± 0.89 (−39.0%) | 0.1613 ± 0.0185 (−68.7%) | |
| HAD | TEST-SAME | 80.65 ± 1.25 | 81.42 ± 0.99 | 0.7056 ± 0.0163 |
| TEST-SHIFT | 45.08 ± 2.91 (−44.1%) | 45.32 ± 2.79 (−44.3%) | 0.2219 ± 0.0421 (−68.6%) | |
| AURA | TEST-SAME | 80.83 ± 0.69 | 81.80 ± 0.73 | 0.7130 ± 0.0111 |
| TEST-SHIFT | 59.29 ± 1.89 (−26.6%) | 59.25 ± 1.86 (−27.6%) | 0.4236 ± 0.0125 (−40.6%) |
Table 3 demonstrates robustness over recent dedicated missing-modality methods under Controlled MNAR Shift Evaluation protocol. HAD achieves nearly identical TEST-SAME BAcc to AURA (80.65% vs. 80.83%), its performance drops sharply to 45.08% under TEST-SHIFT, compared with 59.29% for AURA. The advantage is even more pronounced in Cohen's : HAD and DyMo achieve only 0.2219 and 0.1613, respectively, whereas AURA maintains 0.4236. These results demonstrate that AURA is substantially more robust than both DyMo and HAD to changes in the label–missingness relationship.
References
- Siyi Du, Xinzhe Luo, Declan P. O'Regan, and Chen Qin. Inference-Time Dynamic Modality Selection for Incomplete Multimodal Classification. In ICLR, 2026.
- Chengliang Liu, Yuanxi Que, Qihao Xu, Yabo Liu, Jie Wen, Jinghua Wang, and Xiaoling Luo. Hierarchical Information Aggregation for Incomplete Multimodal Alzheimer's Disease Diagnosis. In NeurIPS, 2025.
Author Response to Reviewer RdHi (Part 3/3)
(Continued from Part 2)
Q1. Generalization of Controlled MNAR Shift beyond stroke classification
The Controlled MNAR Shift protocol has only been validated on stroke classification. Can the main conclusions be replicated on another task or dataset?
To examine whether the robustness advantage observed on STP (Table 3) generalizes beyond stroke classification, we extended the Controlled MNAR Shift Evaluation protocol to 100-day mortality prediction. We evaluated AURA against DyMo, HAD, and the five original baselines under a stricter setting that matches both per-modality artificial-removal rates and normalized label–missingness mutual information between TEST-SAME and TEST-SHIFT at .
The experiment includes 7,446 admissions, each with three available clinical modalities (clinical notes, ICD codes, and laboratory measurements). We added two label-independent Gaussian dummy modalities sampled from and changed only their availability. Since all clinical modalities remain available, this setup isolates sensitivity to missingness patterns from the loss of clinical evidence.
Table 4. Robustness on 100-day mortality prediction under dummy-modality Controlled MNAR Shift Evaluation protocol (), with matched removal rates and MNAR severity. Mean ± standard deviation over five folds. Parentheses indicate relative drops from TEST-SAME to TEST-SHIFT; bold denotes the best TEST-SHIFT performance.
| Method | Setting | BAcc (%) | F1 (%) | AUROC (%) | AUPRC (%) |
|---|---|---|---|---|---|
| HEALNet | TEST-SAME | 84.01 ± 1.56 | 86.54 ± 1.19 | 93.91 ± 0.44 | 88.85 ± 0.78 |
| TEST-SHIFT | 44.15 ± 0.57 (−47.4%) | 43.47 ± 0.65 (−49.8%) | 30.59 ± 2.72 (−67.4%) | 16.71 ± 0.59 (−81.2%) | |
| Flex-MoE | TEST-SAME | 84.46 ± 1.06 | 84.11 ± 1.90 | 93.33 ± 0.87 | 87.84 ± 1.25 |
| TEST-SHIFT | 44.67 ± 1.56 (−47.1%) | 44.64 ± 1.43 (−46.9%) | 28.13 ± 2.22 (−69.9%) | 16.21 ± 0.58 (−81.5%) | |
| MultiModN | TEST-SAME | 83.38 ± 0.68 | 78.48 ± 1.22 | 91.87 ± 1.03 | 77.89 ± 4.17 |
| TEST-SHIFT | 49.95 ± 1.27 (−40.1%) | 49.83 ± 1.27 (−36.5%) | 33.51 ± 2.71 (−63.5%) | 25.52 ± 2.18 (−67.2%) | |
| MASAM | TEST-SAME | 83.86 ± 1.97 | 86.83 ± 1.46 | 94.13 ± 0.49 | 89.02 ± 0.94 |
| TEST-SHIFT | 44.12 ± 1.49 (−47.4%) | 43.23 ± 1.98 (−50.2%) | 31.42 ± 2.27 (−66.6%) | 16.85 ± 0.53 (−81.1%) | |
| AMC | TEST-SAME | 84.14 ± 1.61 | 82.61 ± 1.97 | 92.88 ± 1.26 | 87.11 ± 1.83 |
| TEST-SHIFT | 44.39 ± 0.92 (−47.2%) | 44.43 ± 0.94 (−46.2%) | 27.36 ± 1.95 (−70.5%) | 15.95 ± 0.31 (−81.7%) | |
| DyMo | TEST-SAME | 80.08 ± 2.72 | 82.51 ± 2.55 | 87.68 ± 1.61 | 79.39 ± 3.66 |
| TEST-SHIFT | 53.39 ± 3.48 (−33.3%) | 53.06 ± 4.22 (−35.7%) | 58.95 ± 5.10 (−32.8%) | 32.22 ± 6.48 (−59.4%) | |
| HAD | TEST-SAME | 76.75 ± 0.96 | 78.74 ± 0.35 | 90.15 ± 0.72 | 79.04 ± 1.42 |
| TEST-SHIFT | 51.42 ± 1.28 (−33.0%) | 49.03 ± 0.69 (−37.7%) | 61.37 ± 5.61 (−31.9%) | 30.72 ± 2.34 (−61.1%) | |
| AURA | TEST-SAME | 74.53 ± 3.67 | 77.39 ± 3.45 | 89.93 ± 1.18 | 77.30 ± 3.82 |
| TEST-SHIFT | 57.16 ± 2.70 (−23.3%) | 57.30 ± 3.95 (−26.0%) | 74.56 ± 1.57 (−17.1%) | 42.07 ± 3.27 (−45.6%) |
The mortality results replicate AURA's robustness on a distinct binary prediction task (Table 4). AURA achieves the highest TEST-SHIFT performance across all four metrics, including 74.56% AUROC, compared with 61.37% for HAD and 58.95% for DyMo. Its relative AUROC decrease is only 17.1%, versus 31.9–32.8% for HAD/DyMo and 63.5–70.5% for the five original baselines.
Importantly, the two tasks provide complementary evidence. STP evaluates shifts that can remove clinically informative modalities, whereas mortality evaluates availability shifts while retaining all clinical evidence. The consistent findings across both tasks demonstrate that AURA's robustness advantage extends beyond the original stroke benchmark and further support the role of representation consistency in reducing missingness-shortcut reliance.
Overall, extending Controlled MNAR Shift Evaluation protocol to mortality prediction confirms that AURA's robustness advantage generalizes across distinct clinical prediction tasks. The results further demonstrate that representation consistency is essential for resisting changes in the label–missingness relationship, beyond simply handling incomplete modality inputs.
Author Response to Reviewer RdHi (Part 1/3)
We thank the Reviewer for recognizing the importance of robustness to missingness-mechanism shifts and the novelty of our Controlled MNAR Shift Evaluation protocol.
We would like to emphasize that our work makes three closely connected contributions: (1) Representation Consistency, a principle supported by a theoretical bound on missingness-shift sensitivity; (2) AURA, a framework designed to promote this property while preserving predictive information; and (3) Controlled MNAR Shift Evaluation protocol, a protocol for systematically evaluating robustness to changes in the label–missingness relationship. The additional analyses and experiments below directly validate the first two contributions and extend the third to a new prediction task.
W1. Distinguishing missingness shortcuts from task-information loss
Representation inconsistency δ in Eq. (6) can arise from both reliance on missingness-pattern shortcuts and the loss of task-relevant information when modalities are removed. To distinguish these effects and verify that AURA reduces shortcut reliance while preserving predictive information, we conducted three analyses (detailed results in our responses to Reviewers PArf and 3en7).
1. Missingness-pattern probe (STP). We trained a probe to predict the modality-availability pattern R from the final learned representation Z. AURA substantially reduced the recoverability of missingness patterns: probe F1 decreased from 88.95% without modality dropout and 80.02% without alignment to 58.54% with AURA. This provides direct representation-level evidence that AURA encodes less information about modality availability, supporting its intended effect of discouraging missingness-based shortcuts (see Response to Reviewer PArf, W1.1, Table 1).
2. Controlled MNAR Shift Evaluation protocol with dummy modalities. To isolate shortcut reliance from task-information loss, we constructed a controlled experiment with 7,446 admissions, retaining all three clinical modalities (clinical notes, ICD codes, and laboratory measurements) and adding two label-independent Gaussian dummy modalities sampled from N(0,1). The Controlled MNAR Shift Evaluation protocol changes only the availability of these dummy modalities, leaving all clinical evidence intact.
With the same modality-dropout procedure, adding AURA's alignment reduced dummy-modality representation inconsistency from 0.6991 to 0.2223. Crucially, TEST-SHIFT AUROC improved from 66.19% to 74.56%, while AUPRC increased from 34.52% to 42.07%. Since no clinical information is removed, these results provide strong evidence that alignment reduces sensitivity to missingness-pattern shortcuts rather than merely compensating for lost modality information (see Response to Reviewer PArf, W1.1, Table 2).
3. Connecting final-representation consistency to robustness and information preservation. We further measured the worst-case Euclidean (ℓ2) discrepancy of the final representation Z, as defined in Eq. (6), together with the TEST-SAME–TEST-SHIFT binary cross-entropy gap ΔBCE. Across w/o dropout → w/o alignment → AURA → full alignment, ℓ2 inconsistency decreased monotonically (16.74 → 11.64 → 7.64 → 5.31), accompanied by a corresponding reduction in ΔBCE (1.3199 → 0.3743 → 0.2433 → 0.1733). This empirically supports the relationship between representation consistency and shift sensitivity established in Eq. (7) (see Response to Reviewer 3en7, W1, Table 1).
Importantly, AURA selectively aligns only artificially masked modality slots (m∈D), rather than forcing all modality representations to become identical. This helps preserve information in retained modalities that remains useful for prediction. On STP, selective alignment outperformed full alignment in all eight evaluated availability patterns containing at least three modalities, where richer cross-modal information is available (see Response to Reviewer 3en7, W4, Table 4). This explains how AURA balances representation consistency with the preservation of task-relevant information, rather than indiscriminately minimizing representation differences.
Together, these complementary analyses demonstrate that AURA reduces missingness-pattern information in learned representations, improves robustness even when clinical information loss is excluded, and preserves useful task information through selective alignment.