Align What Is Observed: Availability-Constrained Unbalanced Optimal Transport for Multimodal Survival Prediction
Abstract
Multimodal models often assume that every input token is observed and should participate in cross-modal alignment. This assumption is overly strict in light of clinical data with multiple modalities, where some clinical variables may be absent, invalid, or only weakly related to a particular image region. Balanced optimal transport (OT) compounds the problem by forcing every token to send and receive prescribed mass, whereas conventional unbalanced OT (UOT) relaxes the marginals but does not by itself exclude unavailable observations. In light of the limitation, we introduce Masked-UOT, a differentiable alignment module that combines an availability-constrained transport support, patient-specific learned marginal priors, and KL-relaxed mass conservation. The resulting plan is computed with generalized Sinkhorn iterations and used to transfer information bidirectionally between local 3D CT tokens and structured clinical tokens. A marginal-prior regularizer prevents degenerate concentration, while an optional patient-level contrastive objective complements the local transport loss. In a retrospective study of 429 CT sessions from 330 patients, Masked-UOT with contrastive learning achieved an AUC of 0.7176, outperforming its balanced-OT counterpart and single-modality models using the same backbone.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.