acceptodds
Under review as a conference paper at ICLR 2027

Align What Is Observed: Availability-Constrained Unbalanced Optimal Transport for Multimodal Survival Prediction

Abstract

Multimodal models often assume that every input token is observed and should participate in cross-modal alignment. This assumption is overly strict in light of clinical data with multiple modalities, where some clinical variables may be absent, invalid, or only weakly related to a particular image region. Balanced optimal transport (OT) compounds the problem by forcing every token to send and receive prescribed mass, whereas conventional unbalanced OT (UOT) relaxes the marginals but does not by itself exclude unavailable observations. In light of the limitation, we introduce Masked-UOT, a differentiable alignment module that combines an availability-constrained transport support, patient-specific learned marginal priors, and KL-relaxed mass conservation. The resulting plan is computed with generalized Sinkhorn iterations and used to transfer information bidirectionally between local 3D CT tokens and structured clinical tokens. A marginal-prior regularizer prevents degenerate concentration, while an optional patient-level contrastive objective complements the local transport loss. In a retrospective study of 429 CT sessions from 330 patients, Masked-UOT with contrastive learning achieved an AUC of 0.7176, outperforming its balanced-OT counterpart and single-modality models using the same backbone.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Masked Unbalanced Optimal Transport, Multimodal Alignment, Lung Cancer, Multimodal Survival Prediction

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_mdiEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes masked UOT for fusing CT and clinical features with partially missing tokens. Unavailable tokens are excluded from the transport support, and patient-specific marginal priors are generated through masked softmax. The method uses entropy-regularized UOT with KL marginal penalties, solved by generalized Sinkhorn iterations, for bidirectional feature transfer. The learned priors are also used for final pooling, with an optional patient-level symmetric InfoNCE loss. Experiments on institutional and LUNG1 cohorts evaluate binary classification of recorded alive/deceased status, rather than censored time-to-event survival prediction.

Critical strengths
  1. The method is technically well developed.

  2. The experiments cover several aspects of the method.

  3. The limitations are clearly acknowledged.

Critical weaknesses
  1. Under the same encoders, fusion structure, pooling, masks, and training budget, how does the method compare with masked balanced OT, unmasked UOT, uniform-marginal masked UOT, and learned pooling without UOT?

  2. How were the 25%, 50%, and 75% availability levels in Figure 2 generated? In the institutional cohort, 35.8% of tumors are classified as “non-lung primary,” including sarcoma and melanoma. What are the exact inclusion criteria? Could the authors also report results for a lung-only subgroup?

  3. Are labels in the institutional cohort defined at the patient level or the session level? If a patient has multiple sessions, does this patient contribute repeatedly to the session-level metrics? Please also report results aggregated at the patient level.

  4. In LUNG1, does the positive class correspond to deceased patients? Given the prevalence of approximately 88.2%, how much does the reported Average Precision (AP) improve over the no-skill baseline? The AUC of MASKED-UOT on LUNG1 is only around 0.60, and contrastive learning appears to reduce performance with both backbones. Should contrastive learning be considered an essential component, or an optional module whose benefit depends on the configuration?

  5. Could the authors provide mask-only, learned-prior-only, UOT-only, and no-transport ablations to separate the contributions of availability masking, learned pooling, and UOT marginal relaxation?

Questions for authors

1.Can the authors clarify the individual contributions of masking, learned priors, and UOT?

2.Are the results stable across different random masks?

3.Can the authors clarify the cohort inclusion criteria and provide patient-level and lung-only results?

4.How should the LUNG1 results and the role of contrastive learning be interpreted?

Please refer to the Critical Weaknesses for details.

Reviewer_VG6vOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes MASKED-UOT for combining local 3D CT features with structured clinical-variable tokens. It integrates availability masks, learned marginal priors, KL-relaxed transport, and bidirectional feature transfer, with an optional patient-level contrastive objective. The institutional evaluation reports an AUC of 0.7176 for the MC3-18 configuration with contrastive learning. The evaluated outcome is recorded as alive/deceased status rather than survival at a specified time horizon.

Critical strengths

The method addresses a relevant practical problem: combining imaging with partially observed clinical records without reconstructing unavailable values. Its explicit availability handling and differentiable bidirectional fusion form a coherent design, and the positive institutional results with MC3-18 suggest that the complete pipeline may be useful.

Critical weaknesses

The central technical distinction from standard UOT is not established. The motivation emphasizes that conventional UOT does not exclude unavailable observations. However, Equation 7 defines a support that is simply the Cartesian product of the observed token sets. Applying ordinary UOT after removing unavailable rows and columns therefore gives the same active-support problem. The same exclusion can be expressed through standard full-vector generalized KL penalties with zero prior mass for unavailable tokens. Under the convention stated in Equation 9, positive transported mass against a zero prior incurs infinite divergence, so any finite-objective solution must assign zero mass to those rows or columns. Explicit masking can be useful computationally, but this property alone does not establish a distinct transport formulation. The generalized Sinkhorn updates are also presented as an application of existing scaling methods. The remaining contribution is therefore primarily the integration of learned priors, transport, and fusion. The paper needs to establish what this integration adds beyond the equivalent standard UOT formulation. As written, the central motivation overstates the technical gap being addressed.

The experiments do not establish the independent value of the remaining design. Section 4.1 states that BOT already uses clinical availability masks, whereas MASKED-UOT additionally changes CT masking, transport priors, and prediction pooling. Equation 21 directly uses the learned priors for readout, so improved pooling remains an alternative explanation for the reported gain. Sharing encoder architectures does not control these differences. Table 2 evaluates auxiliary losses while retaining transport fusion in every configuration, so it does not isolate the necessity of transport or the proposed components. The decisive comparisons should hold CT and clinical masks and prediction readout fixed while independently varying marginal relaxation and learned transport priors, and include a matched learned-pooling model without transport. These controls could change whether the claimed contribution remains supported. Given the limited technical distinction above, the current predictive comparisons are insufficient to establish a substantial methodological contribution.

Questions for authors

What capability does the proposed support constraint provide beyond standard KL-UOT on the observed token sets, or standard full-vector KL-UOT with zero prior mass on unavailable tokens?

Does the advantage persist when masks and prediction readout are identical across transport models? How does a matched learned-pooling model without transport perform?

Reviewer_YzJsOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Entropic unbalanced OT between 160 local 3D-CT tokens and nine structured clinical-variable tokens. A 0/1 availability mask restricts the transport support, patient-specific marginal priors are learned by masked softmax, KL relaxation is solved with unrolled generalised Sinkhorn, and the plan drives bidirectional residual fusion followed by prior-weighted pooling for binary vital-status classification. Evaluated on an institutional cohort (429 CT sessions / 330 patients, 57-session test set) and LUNG1 (416 patients, 62-patient test set) against clinical-only, CT-only, a reimplemented TGA baseline and balanced OT, plus an evaluation-time variable-masking sweep and a loss-weight ablation.

Critical strengths
  1. Separating hard availability from soft transport mass is a reasonable framing, and Sections 3.2–3.4 are explicit about what is and is not enforced. The released code matches the equations and the anonymisation is careful.
  2. The authors are candid in the results and conclusion that the direction and size of the gains depend on encoder, cohort and metric.
Critical weaknesses
  1. The evidence cannot carry the claims. The institutional test set has 57 sessions (~19 deceased); the ± is the spread of four fold models on that one set, not test uncertainty, and a 95% interval for an AUC of 0.70 at this n is roughly ±0.15. The headline gap is +0.031, and only +0.009 over clinical-only; in the same Table 1, MedicalNet CT-only beats every MedicalNet multimodal row. On LUNG1 (Appendix E) every configuration is between 0.48 and 0.60 AUC on about 7 alive test patients, and the “MedicalNet benefit” quoted in the main text is reversed under MC3-18. The two-cohort claim reduces to one cohort with a weak signal and one at chance.
  2. None of the three proposed components is ablated. Table 2 varies only loss weights with the transport fusion active in every row; there is no uniform-vs-learned prior, masked-vs-unmasked, or balanced-vs-unbalanced comparison with everything else fixed, and the conclusion lists these as future work. The BOT baseline differs from Masked-UOT in four respects at once (balance, marginals, CT masking, pooling), so Table 1 cannot attribute anything. Section 4.5 also shows CL-only with λ_CL×10 reaching 0.7227, above the headline configuration.
Questions for authors
  1. Please add three controls with everything else fixed: uniform priors; no availability mask (missing tokens zeroed but kept in the support); balanced vs unbalanced. Which component accounts for the gain over BOT?
Reviewer_hLk6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses cross-modal alignment for multimodal survival prediction. The authors argue that existing methods typically assume all input tokens are available and participate in cross-modal alignment, whereas in practice clinical data often contain missing, invalid, or weakly image-relevant variables, so forcing alignment over all tokens introduces noise. To address this, they propose MASKED-UOT, an availability-constrained unbalanced optimal transport module with three core components: (1) availability-constrained transport support, which excludes unavailable tokens from the transport support via binary masks; (2) patient-specific learned marginal priors, which assign prior weights to available tokens through a lightweight scoring network; and (3) a KL-relaxed mass conservation constraint, which unbalances the OT by allowing the transported mass to deviate from the prior. The transport plan is solved via a generalized Sinkhorn iteration and used for bidirectional information exchange between 3D CT tokens and structured clinical tokens. Effectiveness is validated on an in-house institutional cohort and the public LUNG1 cohort.

Critical strengths

Principled problem decomposition. Separating availability (hard support masks), patient-specific priors, and KL-relaxed mass is a clean, mathematically sound formulation of a real clinical problem.

Critical weaknesses

No ablation of the three core components. Only loss terms (BCE/OT/CL) are ablated; the support mask, learned priors, and KL relaxation are never individually removed, so their necessity is unverified.

Questions for authors

The paper is framed as survival prediction, but the task is binary vital-status classification with a plain BCE loss — no time-to-event outcome or censoring appears anywhere in the model or training. Was follow-up time/censoring available in either cohort and simply discarded? If not, the title and abstract overstate the task.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.