ModalShare: Contribution-Aware Budget Allocation for Multimodal Split Learning

Multimodal models combine several input streams, yet training them where the data originates, such as on edge devices carrying cameras, microphones, and other sensors, is impractical when each modality requires its own heavy encoder. Split Learning (SL) makes such training feasible by keeping only the first layers on the device and offloading the rest to a server, but the resulting communication must carry intermediate activations for every modality at every step. Communication-efficient SL reduces that cost by compressing the activations, keeping only a fraction of each; however existing schemes hand every modality the same keep-ratio, so the shared budget is split in proportion to activation dimension, a quantity unrelated to how much each modality contributes to the fused prediction. We make that split an explicit decision, namely inter-modality allocation: under a fixed budget, every policy transmits the same expected payload and differs only in how that payload is divided across modalities. Our allocator, ModalShare, sets each modality's keep-ratio from a Shapley contribution score computed by the server over coalitions of activations it has already received, so measurement adds no transmission overhead, no client-side computation, and assumes no prior knowledge of which stream is which. ModalShare improves accuracy over equal keep-ratios by $15.4$ and $12.4$ percentage points on CREMA-D and MVSA at matched payload under $5\times$ compression, with consistent gains across three compressors, three datasets, and four budgets. Existing compressors leave these gains unclaimed in multimodal settings; contribution-aware allocation recovers them.