acceptodds
Under review as a conference paper at ICLR 2027

Modality Value Is a Paired Comparison: Censoring- and Acquisition-Aware Estimation of Omics Modality Gain

Abstract

Multimodal survival research typically measures an omics modality's value by the performance gain from adding it, a metric that shapes model design, modality selection, and testing resource allocation. In practice, however, omics testing covers only a subset of patients, its availability can depend on patient characteristics, disease status, and clinical workflow, and survival outcomes are further subject to right censoring—making the modality's true gain in the target population hard to obtain from the observed subset. Existing weighting methods typically estimate performance with and without the modality separately, turning a paired target into a difference of separately normalized estimates, which can inflate variance and turn “not estimable” into “no value.” We define modality value as the paired performance difference of a fixed predictor under controlled-input ablation, derive identification and non-identifiability conditions, and propose a model-agnostic framework: joint weighting corrects for selective acquisition and censoring, paired increments preserve score correspondence, support diagnostics flag unreliable cases, and rank discordance guides adaptive selection between estimators—without retraining or imputation. Across 10 TCGA cohorts, 3 model classes, 5 acquisition mechanisms, SurvBoard's natural missingness, and 10 cross-cohort transfers, naive estimation shows substantially higher error and flips the sign of the estimated gain in 26.0% of signal-present cases. A selection rule learned on semi-synthetic cohorts and frozen before external transfer improves over either fixed estimator.

Primary area:
datasets and benchmarks
Keywords:
multimodal learning, survival analysis, multi-omics, missing modalities, selective acquisition, paired comparison

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_U4XjOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper aims to estimate the modality value of omics data in survival analysis. Here, the authors identify to major challenges: (i) distribution shift between patients where omics data is observed vs unobserved, and (ii) right censoring of survival outcome. For his, the authors argue that current methods use naive comparison of predictions between the groups which leads to biased estimates. From this, the authors derive conditions when the modality gain is actually identifiable and propose a paired increment estimator with IPW and IPCW. They evaluate their method on TCGA datasets against baselines and ablations.

Critical strengths
  • The overall problem setting of distribution shift in observed omics data (“acquisition”) and right censoring in survival data for estimating the modality value of omics data seems plausible, and defining conditions for identifiability and using ideas from IPW and IPCW seems in general like an useful approach.
Critical weaknesses

The main problem of this submission is that the paper is very hard to follow and the overall presentation is poor which also makes it hard to judge the soundness of the paper:

  • The language uses terms that are unnecessarily complex for the concepts they describe and that are also not defined before using and are used inconsistently throughout the paper. The paper loses itself in details in the theory and experiments section, where however, the motivation of the choices for all these implementation details is incomprehensible or completely missing.
  • The overall structure is also not clear. As one general example, the contributions part in the introduction does unstructured cross-referencing to also partially overlapping and not connected and not ordered sections, in the contributions arbitrary results are already stated for where it is not clear what setups and experiments they refer to, etc. This is just one representative example of similar style throughout the paper.
  • The same holds for Sec. 3, which tightly compresses setup, objective and theory into one section, but where the motivation and interpretation of the different components is missing or unclear for most of the paragraphs here. Also here, empirical results are already referenced for which the experiment setup is not described (yet) at all. Overall, this makes it additionally challenging to correctly evaluate the soundness of the paper.
  • Further, the comparison to related work is rather short, making it hard to judge the contribution of the paper properly.
  • The experimental setup in Sec. 5 also does not provide any intuition on the motivation for dataset, modelling, and evaluation choices and reads more like an automatically generated short summary. This makes it hard to judge the value of the proposed experimental setup and the results of Sec. 6 which are then also hard to follow.

General formatting concern: It seems like this paper uses an incorrect citation style for ICLR submissions (i.e., it uses numbered citations instead of the “authors, year” style). It should be checked whether this itself is a major violation which is ground for rejection.

Questions for authors

I do not have any concrete questions whose answers would change my assessment. I think the major issue is the presentation which needs to be fixed first to judge the soundness better.

Reviewer_y9JgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to estimate an omics modality’s contribution to survival discrimination when modality acquisition is selective and outcomes are right-censored. The estimand is the concordance difference between a frozen predictor evaluated with the modality and the same predictor evaluated with that modality replaced by a reference value. The proposed framework combines acquisition and censoring weights, a paired-increment estimator, an empirical support check, and a discordance-based rule for choosing between estimators.

Validation includes semi-synthetic acquisition masks across TCGA cohorts and three predictor families, cross-cohort transfer experiments, and a natural-missingness audit. The results demonstrate substantial errors in naive evaluation and suggest that the preferred estimator depends on the frequency of order-changing pairs. A threshold learned on the semi-synthetic experiments and frozen before transfer provides a modest reduction in average transfer error.

Critical strengths

1. The paper addresses an important evaluation problem.

I found the premise compelling: before using modality-contribution estimates to compare models or prioritize data collection, we should establish that those estimates represent the intended population. Separating selective acquisition from censoring, and distinguishing insufficient support from zero contribution, gives the paper practical relevance beyond proposing another survival predictor.

2. The evaluation includes informative negative results.

The authors openly report parity with complete-case estimation, attenuation of the HT estimator, and deterioration of pairing at high discordance. Freezing the selection threshold before transfer is also a positive design choice. These aspects make the empirical investigation more informative, although they do not resolve the central methodological concerns below.

Critical weaknesses

1. The methodological novelty is not established against central prior work on the estimation problem.

Given the paper’s motivation, I expected close engagement with the statistical literature on survival predictiveness and incremental biomarker value. I was therefore surprised that several directly relevant methodological connections are absent from both the discussion and the evaluation:

These works do not all estimate the same quantity as fixed-predictor reference-value ablation. Those differences should be explicitly analyzed, and suitable estimation principles should be compared under aligned targets and assumptions. Their omission leaves the reader without the comparisons most relevant to judging the claimed methodological advance.

I do not regard the simplicity of the paired-increment idea as a weakness by itself. A simple correction can be valuable. However, the central identity follows directly from the common population denominator, while pairing, acquisition weighting, and censoring weighting are established principles. I was looking for convincing evidence that this particular construction offers something beyond an appropriate application or adaptation of those principles.

A particularly important missing comparator is a contrast-level augmented estimator targeting d=h+−h−d=h^{+}-h^{-}. The manuscript attributes the attenuation of T5 to modeling concordance levels, but does not evaluate whether augmenting the contrast directly resolves that disadvantage. This makes it difficult to distinguish a benefit specific to the proposed procedure from a limitation of the chosen augmented baseline.

ICYM²I is included, so the issue is not a complete absence of relevant comparisons. Nevertheless, the observed-training transplant in Appendix A.7 uses evaluation-side IPW without a fully weighted Cox training loss. The manuscript acknowledges this restriction; the experiment therefore cannot establish inferiority of the original training-and-evaluation correction framework.

This weakness is consequential because the main contribution is an estimation method. Adding references would improve the positioning, but establishing novelty and comparative value also requires an appropriate estimand-aligned augmented or efficient comparator.

2. The experiments do not isolate whether the paired construction causes the reported improvements.

Table 1 was where I expected the methodological argument to become most convincing. Instead, the parity between paired Hájek and complete-case IPCW stood out: both obtain error 0.080 in the signal-present stratum. The paper acknowledges this result, and the methods need not target the same population quantity. Nevertheless, this makes it especially important to isolate where the proposed construction provides a distinctive benefit.

The main estimator ladder changes stabilization settings together with the estimator formulation. Section 5 and Appendix A.4 use propensity clipping to [0.05,1][0.05,1] and a 99th-percentile cap on stabilized subject weights for subject-level estimators. The paired estimator instead uses clipping to [0.1,1][0.1,1] and caps acquisition pair weights at 10. In a paper whose explanation centers on weight-induced variance, these are consequential differences. Is the improvement due to pairing, stronger regularization of extreme weights, or both? Table 3 isolates D1, but an equally controlled D2 comparison is not established.

The distinction also matters algebraically. With identical weights, pair sets, and normalization,

∑i,jwijhij+∑i,jwij−∑i,jwijhij−∑i,jwij=∑i,jwij(hij+−hij−)∑i,jwij\frac{\sum_{i,j} w_{ij} h_{ij}^{+}}{\sum_{i,j} w_{ij}} - \frac{\sum_{i,j} w_{ij} h_{ij}^{-}}{\sum_{i,j} w_{ij}} = \frac{\sum_{i,j} w_{ij}\left(h_{ij}^{+}-h_{ij}^{-}\right)}{\sum_{i,j} w_{ij}}

Computing increments first is therefore not, by itself, a different estimator. The substantive changes involve the observations, weights, and normalization used for the ablated-score term. Their effects need to be separated.

The remaining evidence does not resolve this attribution problem. Paired HT has lower error but a calibration slope of 0.45. The frozen transfer rule improves mean error from 0.0378 and 0.0390 to 0.0340, but Table 4 reports uncertainty for individual methods rather than for their paired differences. Table 5 shows selection accuracy of 0.685 versus 0.683 for complete-case, and evaluates DR on a different set of valid decisions.

I was particularly interested in the natural-missingness audit as a test of practical relevance. It was disappointing that Table 6, the estimand-aligned audit, omits both the paired estimator and the final adaptive rule. Its MAE reference is also insufficiently specified. Moreover, the main complete-fold reference retains the original outcome censoring: recovering that reference does not by itself demonstrate recovery of the uncensored population functional.

The theoretical and reproducibility limitations reinforce this concern. Proposition 2 analyzes a pair-kernel component rather than the full estimator variance, and shared dependence on ZZ alone does not establish the direction of covariance inflation asserted in Corollary 2.4. The non-identifiability example also removes censoring support beyond a fixed follow-up time, so it does not isolate the additional role of latent-outcome-dependent acquisition. Finally, the provided PDF refers to a release package without an identifiable access link and does not fully specify the experimental pipeline.

The authors acknowledge several limitations, including the empirical nature of the threshold. My concern remains that the evidence does not adequately separate the proposed mechanism from stabilization choices, baseline construction, and reference-estimation effects.

Questions for authors

1. Methodological comparison and evidence.

How does the proposed estimator differ from the relevant survival-predictiveness and incremental-value estimation frameworks above once estimands and assumptions are aligned? In particular, what advantage remains against a contrast-level augmented estimator targeting h+−h−h^{+}-h^{-}? An explicit methodological comparison and corresponding results could materially change my assessment of novelty.

2. Controlled evidence for D2.

Can the authors provide a paired-versus-separately-normalized comparison using identical predictors, nuisance estimates, evaluation samples, and pair-weight clipping policies? Does the advantage persist across shared stabilization settings, and what are the paired confidence intervals for the resulting error differences?

3. Clarification of validation targets and missing results.

What exactly is the reference used to calculate MAE in Table 6, and how is it obtained under natural missingness? Why are the paired estimator and adaptive rule absent from this aligned audit, and what are their results on the same valid evaluation units? Please also distinguish recovery of the censored complete-fold reference from recovery of the population estimand.

These questions concern evidence needed to substantiate the current claims, rather than optional extensions to future work. Clarifying that the estimand is fixed-predictor contribution, that the threshold is empirical, or that complete-case targets a selected population would not resolve the two main concerns, since those points are already acknowledged.

Reviewer_jV2oOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a paired estimator for omics modality gain under selective acquisition and censoring. Experiments show reduced estimation error.

Critical strengths
  1. The paper addresses an important problem in modality evaluation under selective acquisition and censoring.
  2. The proposed method does not require model retraining or missing modality imputation.
  3. The experiments cover 17 cohorts, 3 model families, and multiple acquisition mechanisms.
Critical weaknesses
  1. Different estimators use different weight clipping strategies, making the comparison potentially unfair.
  2. The adaptive threshold is evaluated on only 20 transfer units, with limited improvement (0.034 vs. 0.038). Its generalization remains unclear.
  3. The simulated acquisition mechanisms use variables that are also included in the propensity model. It is unclear whether the method generalizes to more real-world missingness mechanisms.
Questions for authors

Can the authors evaluate their method under natural missingness to demonstrate their effectiveness beyond simulated acquisition mechanisms?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.