acceptodds
Withdrawn from ICLR 2027

Adaptive Survival Prediction via Multimodal Case-Level Evidence Reliability

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal survival models combine complementary evidence from computed tomography (CT), radiomics, and clinical records, but the trustworthiness of that evidence varies across patients. Standard fusion rules infer modality weights from content alone and can therefore conflate semantic relevance with data quality, missingness, and prediction-derived certainty. We introduce Case-Adaptive Evidence Reliability (CAER), a modular framework that separates these two roles. Each modality produces an independently supervised discrete-time survival distribution. A reliability channel—which does not access the evaluated patient’s outcome at inference—combines modality availability, an explicit observable-quality proxy, and distributional certainty. Reliability then acts as a bounded residual to the content gate; exact masking assigns zero weight to unavailable evidence and preserves a patient-level audit trail. On the locked split of the 422-patient NSCLC-Radiomics cohort, CAER obtains the best values among six statistical, tree-based, and neural survival baselines for Harrell and Uno concordance, 1/2/3-year time-dependent AUC, and integrated Brier score. Controlled ablations favor the complete relevance–reliability decomposition over content-only fusion and simpler reliability controls. The results support explicit case-level reliability as a useful inductive bias for multimodal survival prediction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal learning, survival analysis, evidence reliability, missing modalities, medical imaging

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NjChOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The author separates the supportive and reliability of data in the surivival prediction model.

Critical strengths

The authors demonstrate the performance improvement of their method on one dataset.

Critical weaknesses
  1. Survival curve in Patient-level predicted in Fig. 1 consists of five discontinuous line segments, which suggests that the figure is illustrative rather than generated from actual numerical results. This undermines the credibility of the paper.
  2. The marks between the encoder and survival head are masked by arrows.
  3. The paper does not examine how sensitive the results are to the mixing weights in Eq. (16) or the auxiliary loss weights in Eq. (22). The experiments are also limited to a single dataset. It is unclear whether the proposed method generalizes to other datasets.
  4. The readability of the paper needs improvement.
  5. Titile is not consistent.
Questions for authors

It would be clearer if the authors included EsurvFusion [1] and NADMSurv [2] in the related work discussion.

[1] Huang, L., Xing, Y., Lin, Q., Duan, J., Ruan, S. and Feng, M., 2025. Esurvfusion: An evidential multimodal survival fusion model based on epistemic random fuzzy sets. IEEE Transactions on Fuzzy Systems. [2] Ming, W., Ye, W., Liu, M., Chen, D., Jiao, Y., Xu, J. and Wang, X., 2026, September. Noise-Aware Importance–Uncertainty Disentangled Multimodal Learning for Robust Cancer Survival Prediction. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 481-490). Cham: Springer Nature Switzerland.

Reviewer_edqJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper argues that multimodal survival models usually let one learned weight decide two different things: how relevant a modality is to the prediction, and how much its evidence for a given patient can be trusted. To separate these, the authors propose CAER for overall-survival prediction in non-small-cell lung cancer, using CT, radiomic and clinical data.

Each modality is first processed by its own branch, which is trained on its own to produce a complete discrete-time survival curve, so every branch is meaningful in isolation. The branches are then combined by a gate. A learned content score expresses how relevant each modality is for the patient. Added to it is a bounded reliability adjustment that does not use the patient's outcome at inference.

The reliability score mixes two ingredients, with observable quality taking almost all the weight and prediction certainty a small remainder:

(i) For the two imaging modalities, it is the fraction of the tumour delineation that fits inside a fixed crop around the tumour, lowered if the tumour touches the crop border. For clinical data, it is the share of seven key clinical fields that are recorded. (ii) This is a small correction based on how concentrated each branch's predicted event-time distribution is.

A modality that is absent receives exactly zero weight, so missing data is never treated as low-quality data.

The authors prove that the gate behaves predictably. Raising one modality's reliability can only increase its own weight, at the expense of the others. The total influence of reliability is capped by a fixed scale, missing modalities have no effect on how the remaining ones are weighted, and small changes in reliability produce proportionally small changes in the prediction. They also present the stored branch outputs, quality values, reliability scores and fusion weights as a patient-level audit trail.

Validation uses the public NSCLC-Radiomics cohort of 422 patients, split once into 269 training, 68 validation and 85 test patients. CAER is compared with six established survival models (CoxNet, random survival forest, DeepSurv, MTLR, gradient-boosted survival and DeepHit), each given the same concatenated features. CAER reports the best concordance, 1-, 2- and 3-year AUC, and integrated Brier score. Four ablations replace the reliability mechanism with simpler alternatives, and they favour the full design.

The authors conclude that explicit case-level reliability is a useful and auditable inductive bias for multimodal survival prediction. They acknowledge that the evidence comes from a single cohort with a small test set and simple quality proxies.core for modality mm and patient ii, while RimR_i^m captures the dynamic reliability residual.The reliability score is further decomposed into:Rim=aim(0.95 Qim+0.05 Pim)R_i^m = a_i^m(0.95\,Q_i^m + 0.05\,P_i^m) This combines an observable quality proxy (QQ) with a distribution concentration statistic (PP). For CT and radiomics, QQ measures the fraction of the tumour mask retained within a fixed 64364^3 crop, applying a penalty if the mask touches the crop boundary. For clinical data, QQ represents the fraction of seven essential clinical fields that are present. Concurrently, PP averages three concentration statistics derived from the branch's own predicted event-time distribution. Moreover, When modalities are missing, CAER handles them exactly through a masked softmax that filters out absent data without disrupting the remaining features. The theoretical properties of this fusion gate are rigorously established in Appendix C, which proves four core attributes: a monotone response to reliability, a bounded change in pairwise log-odds, exact data masking, and a Lipschitz bound.

Critical strengths

The paper's main strength is that its experimental pipeline is built carefully to avoid information leakage, and its fusion gate behaves predictably and can be checked. All preprocessing is fitted on the training patients only. The CT encoder is pretrained on a separate dataset (LIDC-IDRI) without ever seeing LUNG1 survival outcomes. Availability indicators are created before missing values are filled in, so imputed entries are never confused with real measurements. The test set is kept untouched until every design choice has been fixed. Many clinical machine-learning papers are careless on exactly these points, and here they have clearly been thought through.

The gate itself is also well designed. Because reliability enters as a bounded correction rather than replacing the learned content score, content stays the main signal. Reliability can only nudge the balance between modalities by a limited, known amount. A missing modality receives exactly zero weight and does not distort how the remaining modalities are weighted. I checked the derivations in the appendix that establish these properties, and they are correct. The framework is reproducible and modular, and the reliability component could easily be tested with better quality measures or on new cohorts. However, careful engineering does not by itself show that the reliability mechanism is responsible for the reported gains. For that reason it does not outweigh the weaknesses described below.

Critical weaknesses
  1. The reliability signal may largely measure tumour size, and the robustness claim that motivates it is never tested.

My most serious concern is what the "observable quality" score actually captures. For both CT and radiomics, quality is the fraction of the tumour delineation that fits inside a fixed 64 crop, lowered further if the tumour touches the crop border. LUNG1 is a cohort of locally advanced tumours, and many of them are larger than 64. In practice, a patient will usually get a low "quality" score not because the scan is poor, but because the tumour is big. Tumour size and volume are among the strongest known predictors of survival in this cohort, as shown in Aerts et al. (2014), which the authors themselves cite.

As a result, the channel described as outcome-independent reliability is very likely feeding prognostic information into the gate. That directly undercuts the paper's central claim that relevance and reliability are kept separate. The loss of "quality" is also created by the authors' own choice of crop size, not by acquisition artefacts or segmentation errors, which are what the introduction uses to motivate the method. And because CT and radiomics share the same quality score, the reliability term can never shift weight between those two modalities, only between imaging and clinical data.

There is also reason to think the mechanism does very little for most patients. The paper itself notes that the gate is unaffected by a shift applied equally to all modalities. So for a patient whose tumour fits fully in the crop and whose clinical record is complete, reliability can change the gate only through the small certainty term, which is worth at most about 0.05 in logit units. The paper never shows how quality, reliability or fusion weights are distributed across patients, nor how often a modality is actually missing in LUNG1. Without that, it is possible that the missing-modality handling is never exercised by the real data, and that any effect of reliability is concentrated in the large-tumour subgroup described above.

Finally, the robustness claim that motivates the paper is never tested directly. There is no controlled experiment that degrades the inputs (adding noise, eroding masks, corrupting radiomic features, or deleting clinical fields) and shows that CAER holds up better than plain late fusion. There is no evaluation with modalities removed at test time, and no external cohort. LUNG1 comes from a single institution (MAASTRO), so its acquisition variability is limited to begin with.

The falsifiability argument in Section 4.6 does not fill this gap. Holding the content scores fixed, the claimed directional behaviour holds by construction, so no data could contradict it. The section also cites Eq. 21 where Eq. 20 is meant. Likewise, the auditability contribution is asserted but never demonstrated: there are no case studies and no analysis of how the fusion weights behave.

  1. The reported gains are within the range of sampling and training noise, and the comparison set does not test the fusion claim.

All of the evidence rests on a single test split of 85 patients. There are no confidence intervals, no paired significance tests, and no variance across seeds, because predictions from the five seeds are averaged before evaluation. At this sample size and with LUNG1's event rate, the standard deviation of Harrell's C is also could not found. Moreover, 1-year AUC of 0.5768 is close to chance and rests on a few dozen events. The IBS differences in the ablation, around 0.002, are negligible. Reporting results to four decimal places suggests a precision the data cannot support.

The paper's own ablation table makes the noise problem concrete. The "Equal residual" variant adds the same amount to every available modality's gate score. Since the gate ignores such a shift, this variant computes the same function as plain late fusion. Yet the two are reported at 0.5846 and 0.5745 Harrell C. That gap of about 0.010 between models that should be equivalent is a direct measure of run-to-run variation. It is about half the 0.022 gain credited to adding the certainty term on top of quality alone. By the authors' own bounds, that term can change the balance between any two modalities by only a very small amount, so attributing a 0.022 gain in concordance to it is not convincing.

The baselines also do not test the claim the paper makes. The paper is about how to fuse modalities, yet none of the six comparison methods is a multimodal fusion model; each simply receives one concatenated feature vector. Missing comparisons include:

a. QMF adapted to survival (cited by the authors but not compared); b. evidential or trusted fusion (e.g., Han et al., ICLR 2021); c. mixture-of-experts gating; d. simple averaging of the branch predictions; e. a clinical-only Cox model on stage and age, which is the reference clinicians use.

Baseline tuning is not documented. DeepHit's IBS of 0.337 and the 3-year AUCs of about 0.51 for MTLR and DeepHit suggest these baselines were under-tuned rather than inherently weak. CAER is also effectively a five-model ensemble, while CoxNet is a single deterministic model. we observed that plain late fusion without any reliability term already beats every baseline. About 85% of the IBS improvement over the best baseline comes from the per-branch architecture rather than from reliability. This contradicts the discussion's suggestion that the gains come from how evidence is combined rather than from model capacity.

Questions for authors

I think paper lacks meaningful strength and it requires regerous study to geniounly publishable and i think author will not be abe to make experiments with external validation on another NSCLC cohort, for example NSCLC-Radiogenomics on TCIA. However, I have few quick questions for authors, which may affect my rating:

  1. Are the radiomic features computed on the full delineation or on the cropped 64364^3 region? If on the full mask, why is QRad=QCTQ^{Rad} = Q^{CT}?
  2. Is “Equal residual” exactly as defined in Table 5? If so, how do you explain its difference from Late fusion, given that the softmax ignores a common shift?
  3. Is PP computed with a stop-gradient? If not, branches can steer their own gate weight through PP.
  4. What fraction of test patients have any aim=0a_i^m = 0, and what fraction have any Qim<1Q_i^m < 1?
  5. How many configurations were evaluated on the 68 validation patients to choose the 0.95/0.05 mixture, the loss weights, the 0.25 boundary penalty, and κ\kappa? How were the baselines tuned?
  6. How were the 64 radiomic features chosen? (Sec. 3.2, Appendix A.2 describe them differently.)

Additional analyses needed for this submission:

  1. Correlations of QCTQ^{CT} with GTV volume and T-stage, and the univariate C-index of QCTQ^{CT} as a risk score. Also CAER’s results with a crop large enough (or tumour-adaptive) that ri≈1r_i \approx 1 for all patients.
  2. Bootstrap 95% confidence intervals and paired tests for Tables 1–2; the SD across seeds of individual-seed models; and repeated stratified cross-validation over all 422 patients.
  3. Unimodal performance for each branch, and a controlled degradation / test-time modality-dropout experiment comparing CAER against late fusion.
  4. Fusion baselines (QMF adapted to survival, evidential fusion, branch averaging) and a clinical-only Cox model.
Reviewer_bfPKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper is in the area of multimodal fusion with the aim of separating the contributions from content versus reliability and applying it to the downstream task of survival prediction. Among the problems facing fusion are how to handle missing, spurious and unreliable data. This paper presents one such approach that combines independently supervised CT, radiomic, and clinical survival branches through a reliability-adjusted late-fusion gate. It distinguishes learned content relevance from an explicit reliability score based on availability, observable-quality proxies, and prediction-derived certainty. On a fixed NSCLC-Radiomics split, the method reports improved discrimination and integrated Brier score over conventional survival baselines and internal fusion ablations. Its main contribution is an inspectable reliability correction to late fusion; whether this establishes a meaningful separation between content and reliability, or offers advantages over interaction-based fusion, remains insufficiently demonstrated.

Critical strengths

The method is simple and exposes modality-level predictions, quality indicators, and fusion weights, allowing reconstruction and inspection of the fusion decision. It distinguishes complete modality absence from partially observed or lower-quality evidence and ensures that unavailable modalities receive zero weight. Independent branch supervision supports unimodal fallback. The reported protocol includes patient-level separation, training-only preprocessing, validation-based model selection, and multiple training seeds. Evaluating both discrimination and probabilistic accuracy is also appropriate. It falls under the general realm of late fusion and post-process type of methods of fusion.

Critical weaknesses

The architecture separates content and reliability computationally, but does not establish their semantic identifiability: missingness, ROI truncation, and clinical completeness can affect both the encoded content and prognosis. The reliability channel may therefore supply additional predictive metadata rather than isolate evidence trustworthiness.

Moreover, combining independent survival predictions restricts cross-modal feature interactions compared with graph-based fusion or transformer cross-attention, neither of which is directly evaluated. So comparison with these approaches maybe an improvement for the future.

The paper does not establish whether the reliability mechanism improves stronger interaction-based architectures. Evidence is further limited by a single 85-patient test split without confidence intervals or controlled degradation experiments.

Finally, the equal-residual ablation requires clarification: adding an identical residual to every available modality should leave softmax weights unchanged, yet different performance is reported.

Questions for authors

How does CAER compare with graph-based fusion and transformer cross-attention using the same modality features, patient splits, survival objective, and tuning budget?

Why does the equal-residual ablation differ from content-only fusion despite softmax invariance to a shared additive shift?

How are fusion weights validated as explanations of modality contribution, beyond allowing reconstruction of the gate’s computation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.