Beyond One-Geometry-Fits-All: State-Conditioned Dual-Path Reconstruction for Missing-Modality PEFT
Abstract
Parameter-efficient fine-tuning (PEFT) for multimodal missing-modality adaptation typically reuses a single low-rank adaptation structure across all availability states. However, independently optimized adapters exhibit systematically larger cross-state than same-state, cross-seed subspace separation. Assigning independent adapters per state enhances state-specific expressiveness but scales linearly with the number of states and may weaken cross-state knowledge transfer through parameter isolation; a fixed shared structure is more efficient, yet its limited rank budget cannot simultaneously cover all state-preferred update directions. We formalize this Capacity-Transfer Tension geometrically, establishing a constructive representational separation between CODAR and fixed shared rank-r geometry via subspace coverage and reconstruction analyses. To address it, we propose Conditional Dual-Path Adapter Reconstruction (CODAR), which reconstructs both low-rank factors from reusable dictionaries while conditioning their composition on modality availability. A modality-specific pathway captures state-dependent variation, whereas a transferable pathway reuses shared input-side directions across towers. Across MM-IMDb, UPMC-Food101, and Hateful Memes, CODAR consistently improves over representative PEFT baselines with 101,116 trainable adapter parameters. Controlled comparisons and component ablations further show that the gains cannot be explained by availability conditioning or parameter count alone, while geometric analyses show that CODAR realizes state-dependent update subspaces while retaining shared primitives. Code is available at https://anonymous.4open.science/r/anonymous_code_-48D6/.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.