acceptodds
Desk rejected at ICLR 2027

Bridge the Gaps: Gap-Aware Preservation and Expansion for Continual Missing Modality Learning

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Continual Missing Modality Learning (CMML) requires multimodal models to learn new classes sequentially while retaining old knowledge under incomplete visual or textual inputs. Existing methods mainly recover missing information or reduce interference through efficient adaptation and knowledge preservation, but overlook how missingness alters cross-modal relations before and after fusion. Our stage-wise analysis reveals two coupled challenges: compensated features obscure degradation of the real image-text geometry before fusion, while shared fusion under unequal modality reliability induces misleading modality dependencies after fusion. To address them, we propose GAPE, a Gap-Aware Preservation and Expansion framework. GAPE separately monitors real geometry degradation and compensation-induced shifts to select a safe adaptation stage, and combines counterfactual anchoring, confidence, and modality availability to calibrate fused decisions toward reliable evidence. Experiments on two CMML benchmarks show consistent state-of-the-art performance across diverse missing patterns and ratios.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual Learning, Multimodal Learning, Missing Modality

Related papers

Discussion (0)

Sign in to comment.