acceptodds
Under review as a conference paper at ICLR 2027

SCOPE: Spectral-Conditioned Omic-Pathology Encoder for Interpretable Multimodal Cancer Survival Analysis

Abstract

Predicting patient survival from cancer biopsies is a multimodal learning problem in which whole-slide images (WSI) and gene expression must be fused effectively. The state of the art relies almost exclusively on flattened bag of patch tokens, discarding the 2D structure of tissue and yielding fusion mechanisms that are difficult to interpret biologically. We introduce SCOPE (Spectral-Conditioned Omic-Pathology Encoder), the first spectral-conditioned multimodal architecture for joint pathology-genomics survival modeling. The core primitive is the Gene-Modulated Convolution: a global 2D convolution whose kernel is generated with FiLM modulations driven by pathway-level gene embeddings. Stacking these modules across a multi-scale Haar wavelet pyramid delivers a global receptive field at each spatial-frequency band at log-linear cost. A mask-aware differentiable Top- predictor then selects the most prognostic spatial positions for the discrete-time negative-log-likelihood survival head. On 10 TCGA cohorts, SCOPE raises the mean C-index to ( relative) over the strongest baseline while using lower peak memory and running faster, and it remains the top method in both the pathology-only and missing-omics settings. SCOPE further exposes three complementary axes of intrinsic interpretability (Top- spatial saliency, wavelet sub-band allocation, and pathway attention) that bridge molecular biology and observable tissue architecture. Our code/models will be publicly available.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Cancer survival analysis, whole-slide images, gene expression, intrinsic interpretability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_8gLROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a model named SCOPE (Spectral-Conditioned Omic-Pathology Encoder). The model combines two modalities, WSI (Whole-Slide Image) and genes, to predict the five-year survival probabilities of cancer patients. GM-Conv (Gene-Modulated Convolution) is proposed to deeply integrate the gene modality to the original WSI processing network, which is realized by inserting a gene-conditioned normalization method named Gene-FiLM. The model also integrates Haar DWT to capture the energy distribution across frequency bands. The model possesses strong interpretability, including which WSI regions the model fouces on, whether it focused on coarse-grained structure or fine-grained details, and the influence of omics. Overall, this paper solves a traditional problem, while the experiments are sufficient and the methodology has novel aspects.

Critical strengths
  • Gene-FiLM, an ingenious method to modulate model parameters. The module influences model parameters in a normalization form, which can be written as: output=(1+α)⊙Norm(input)+β\mathrm{output} = (1 + \alpha) \odot \mathrm{Norm}(\mathrm{input}) + \beta, where input\mathrm{input} and output\mathrm{output} are the model parameters before and after modulation (e.g. the weights of a convolution), and α\alpha and β\beta are scaling and bias factors conditioned by genes. I consider this design ingenious for two reasons. First, by initializing �0��0� and �1��1� to zero, the model starts from an unmodulated state, using only the WSI modality for prediction. Second, by randomly dropping out �2��2� and �3��3� during training, the model learns to make predictions even when the gene modality is missing. This module makes the incorporation of the gene modality has almost no damage to the original model performance.
  • Spectral interpretability. The paper's contribution to interpretability is impressive. Especially, the model integrates Haar DWT (Haar Discrete Wavelet Transform). By analyzing the energy distribution across frequency bands, I can observe whether the model is observing coarse structures or fine textures when processing WSI patches. In contrast, attention-based models can only reveal which patches the model has viewed.
Critical weaknesses
  • Present when the model observes coarse structures and when it observes fine textures more clearly. Appendix~F (line 1126-1133) analyzes which gene cohorts lead the model to observe coarse structures, which lead to fine textures, and which lead to neither. However, the writing is too vague and lacks focus. More importantly, this is one of the most interesting findings of the paper, while it is in the appendix. I suggest providing a simplified version in the main text, as not everyone reads the appendix. Furthermore, I am interested in why certain disease gene cohorts drive the model to focus on coarse-grained structures, others on fine-grained textures, and still others on neither. The reason for the observation differences could be an interesting discussion.
  • There are insufficient ablation experiments. The proposed SCOPE framework consists of heavily‑nested components: GM‑Conv incorporates GeneFiLM; the SCOPE Operator stacks multiple GM‑Conv blocks; each SCOPE Layer contains several SCOPE Operators; and the overall SCOPE Encoder further repeats multiple modules equipped with GeneFiLM and SCOPE Layers. Given such deep nesting, it is difficult to pinpoint which specific design choices contribute to the final survival‑prediction performance gain. In the main text, the authors only conduct ablation over wavelet decomposition depth J within the SCOPE Operator. The authors are requested to explicitly clarify which components represent their original innovations and supply comprehensive ablation studies to quantify the individual contribution of each building block. Recommended ablations include: ablations for different insertion positions of Gene‑FiLM, residual block depth L, the differentiable Top‑K operator, and the grid size of patch features.
  • Moreover, more baselines should be added for the Pathology‑only setting, such as DSML [1] and WiKG [2].

[1] Li, Bin, Yin Li, and Kevin W. Eliceiri. "Dual‑stream multiple instance learning network for whole slide image classification with self‑supervised contrastive learning." 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021. [2] Li, Jiawen, et al. "Dynamic graph representation with knowledge‑aware attention for histopathology whole slide image analysis." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

Questions for authors

In Table~1, I observe a phenomenon. For all gene cohorts, the scores of the SCOPE model follow the order Pathology + Transcriptome >> Missing-Omics >> Pathology-Only. This aligns with intuition, as incorporating gene information helps the model better leverage WSI features for survival prediction. While, the LUAD and LUSC cohorts (still for the SCOPE model) are different. LUAD is Pathology + Transcriptome >> Pathology-Only >> Missing-Omics, and LUSC is Missing-Omics >> Pathology + Transcriptome >> Pathology-Only. More coincidentally, the two abnormal cohorts are both lung cancer cohorts, and the SlotSPE baseline behaviors normally. I am curious whether there are underlying reasons for this.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8gLR (part I)

We thank you for the positive assessment and for the encouraging comments. We respond to each comment below and have updated the paper accordingly, with all revisions highlighted in purple.

W1: Coarse vs. fine findings across cancer cohorts

Thank you for this insightful suggestion. As analyzed in Appendix F, the Pearson correlation between a patient's coarse-band (LL) energy fraction and predicted risk, rLLr_{\text{LL}}, groups the ten cohorts into three regimes, each consistent with known scale-specific prognostic morphology:

  • Coarse-risk cohorts (BRCA, STAD, COADREAD; rLLr_{\text{LL}} = +0.155, +0.147, +0.112). These glandular adenocarcinomas are graded by the loss of gland formation [1]. As differentiation is lost, the alternation of lumina, epithelium and stroma gives way to solid sheets or diffusely infiltrating cells, neighboring patches become more alike, and energy shifts to the coarse band.
  • Detail-risk cohorts (BLCA, LUSC, UCEC, SKCM; rLLr_{\text{LL}} = -0.259, -0.227, -0.126, -0.103). In these cancers the composition and spatial organization of the tumor-immune microenvironment carry prognostic information [2]. Interleaved tumor, immune, stromal and necrotic compartments create strong contrasts between neighboring patches, which shift energy to the detail bands.
  • Mixed / non-significant cohorts (KIRC, LUAD, HNSC; rLLr_{\text{LL}} = +0.046, +0.018, -0.056). Prognostic morphology here mixes scales or is largely molecular. LUAD grading combines the predominant pattern with high-grade solid, micropapillary and complex glandular components [3]. HNSC comprises HPV-positive and HPV-negative tumors with markedly different survival. KIRC aggressiveness is tied to molecular alterations such as a metabolic shift. Opposite scale trends within a cohort can then cancel in the pooled coefficient.

Following your advice, we have promoted a concise summary of these three regimes into the main text (Sec. 4.4, Line479-485) and expanded the biological and molecular discussion in Appendix F (Line1386-1457).

[1] Nagtegaal, I. D., et al. "The 2019 WHO classification of tumours of the digestive system." Histopathology, 76(2):182-188, 2020.

[2] Thorsson, V., et al. "The Immune Landscape of Cancer." Immunity, 48(4):812-830, 2018.

[3] Moreira, A. L., et al. "A Grading System for Invasive Pulmonary Adenocarcinoma: A Proposal From the International Association for the Study of Lung Cancer Pathology Committee." Journal of Thoracic Oncology, 15(10):1599-1610, 2020.

AuthorsOpenReview ↗

Response to Reviewer 8gLR (part II)

W2: Original components and component-wise ablations

Thank you for the suggestion. Our contributions lie in how omics conditions the spatial filtering:

  1. GM-Conv (Eq. 3). c\mathbf{c} enters the implicit kernel generator through Gene-FiLM, so every patient gets its own global 2D kernel, whereas the kernels of prior implicit-filter models do not depend on the input.
  2. The SCOPE Operator (Eqs. 6-7). Each wavelet detail band and the deepest approximation have their own GM-Conv, recombined by learned cross-scale mixing.
  3. Gene-FiLM as the only entry point of omics (Eqs. 2, 4). It is zero-initialized. With c=0\mathbf{c}=\mathbf{0} it reduces to a parameter-free standardization, which gives the missing-omics path without auxiliary losses.

We conducted component-wise ablations that change one factor of SCOPE at a time.

ComponentVariantBLCABRCACOADREADHNSCKIRCLUADLUSCSKCMSTADUCECMean
—SCOPE (full)0.7470.7780.7540.6900.8450.7090.6410.7220.7450.8090.744
Gene-FiLM positionPost-norm FiLM0.7350.7720.7480.6740.8450.6960.6280.7180.7370.7970.735
Residual block depthL=4L{=}40.7500.7630.7530.6770.8470.7040.6490.7230.7380.8120.742
Residual block depthL=6L{=}60.7450.7610.7390.6710.8390.7160.6370.7270.7230.8110.737
Top-KK gateMean pooling0.7420.7660.7500.6750.8320.6980.6290.7210.7310.7960.734
Top-KK gateGated attention pooling0.7440.7690.7480.6800.8420.7000.6290.7160.7270.8050.736
Grid size32232^20.7030.6890.7130.6330.8090.6760.5650.7000.6600.7850.693
Grid size1282128^20.7540.7700.7590.6930.8540.7120.6450.7280.7410.8140.747
  • Gene-FiLM position. Applying Gene-FiLM after each sub-layer instead of as pre-norm lowers the mean by 0.009.
  • Residual depth LL. Over L∈[2(default),4,6]L\in[2 (\text{default}),4,6], deeper stacks slightly hurt, consistent with the modest cohort sizes.
  • Differentiable Top-KK. Replacing the gate with mean or gated attention pooling lowers the mean by 0.010 and 0.008.
  • Grid size. A 32232^2 grid loses 0.051. A 1282128^2 grid gains only 0.003 at 4×4\times the tokens.

We have added these ablations to Sec. 4.3 (Line423-449) and Appendix E.2 (per-cohort results, Line1240-1253).

W3: More pathology-only baselines

Thank you for the suggestion. We added DSMIL [1] and WiKG [2] to the Pathology-Only block of Table 1. Both were proposed for slide classification, so we keep their official architectures and replace only the classification output with the same discrete-time survival head and NLL loss as SCOPE. All methods share the same patch features, 5-fold splits and DSS labels.

MethodBLCABRCACOADREADHNSCKIRCLUADLUSCSKCMSTADUCECMean
DSMIL0.6560.6870.7010.5960.7770.6300.5720.6020.6370.7350.659
WiKG0.6640.7100.7090.6290.7730.6550.5820.6290.6690.7640.678
SCOPE0.6600.7420.7450.6660.7990.6680.6120.6680.6780.7860.702

SCOPE outperforms DSMIL on all 10 cohorts and WiKG on 9 of 10. Both gains are significant under the paired tt-test over the ten cohorts (DSMIL: Δ‾=+0.0431\overline{\Delta}=+0.0431, p=6.7×10−5p=6.7\times10^{-5}; WiKG: Δ‾=+0.0240\overline{\Delta}=+0.0240, p=4.3×10−4p=4.3\times10^{-4}).

We have revised Table 1 (Line 342-343), the significance table in Appendix B (Line 1046-1047), and the baselines paragraph in Sec. 4.1 (Line356) to add these methods.

[1] Li, B., et al. "Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning." CVPR, 2021.

[2] Li, J., et al. "Dynamic graph representation with knowledge-aware attention for histopathology whole slide image analysis." CVPR, 2024.

AuthorsOpenReview ↗

Response to Reviewer 8gLR (part III)

Q: Ordering of the three settings on LUAD / LUSC

Thank you for this insightful observation. We attribute the behavior on the two non-small cell lung cancer cohorts (LUAD and LUSC) to both biological and architectural factors:

  • Biological perspective. LUAD prognosis is largely determined by histological growth patterns: the IASLC grading system [1] stratifies patients by the predominant pattern and the proportion of high-grade (solid, micropapillary, complex glandular) components, i.e., tissue architecture that is directly visible in the WSI. LUSC is histologically more homogeneous, and its bulk transcriptome mixes tumor, stromal and immune compartments in a smoking-driven, highly mutated background, so its transcriptomic prognostic signal is weaker and more heterogeneous across patients.
  • Architectural perspective.
    • SCOPE. Because c\mathbf{c} modulates every GM-Conv kernel, the reliability of the transcriptome directly shapes how the slide is filtered. On LUSC, patient-specific noise in a weakly informative transcriptome may propagate into the spatial filtering of all positions, whereas withholding c\mathbf{c} falls back to unmodulated kernels that retain the morphology representation learned with omics as training-time guidance and are regularized by conditional dropout, which may explain why Missing-Omics exceeds both Pathology + Transcriptome and Pathology-Only. On LUAD, joint training may route part of the prognostic signal through c\mathbf{c}, so the unmodulated kernels may invest less in morphological cues than a dedicated WSI-only model, which may explain why Pathology-Only exceeds Missing-Omics.
    • SlotSPE. SlotSPE compresses the slide and the omics into separate sets of slots and fuses them by cross-attention between the two sets, so its slide encoding does not depend on omics. When omics is missing, it imputes the omics features from histology through a cross-modal reconstruction head trained jointly with the survival objective. Its missing-omics path thus substitutes a histology-derived estimate, which may make its ordering less sensitive to the cohort-specific reliability of the transcriptome. SCOPE nevertheless remains above SlotSPE in all three settings on both cohorts.

[1] Moreira, A. L., et al. "A Grading System for Invasive Pulmonary Adenocarcinoma: A Proposal From the International Association for the Study of Lung Cancer Pathology Committee." Journal of Thoracic Oncology, 15(10):1599-1610, 2020.

Summary

We appreciate your thoughtful suggestions, which helped us bring the cross-cohort scale findings into the main text, quantify the contribution of each component, and broaden the pathology-only comparison. We believe the paper is much stronger as a result. We hope the revision merits a more favorable evaluation, and we would be glad to answer any further questions promptly.

Reviewer_Ngs9OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SCOPE is a multimodal model for cancer survival prediction from whole-slide images and bulk RNA-seq. It rasterizes frozen patch embeddings into a spatial feature grid and pools pathway embeddings into a patient-level conditioning vector. This vector modulates an implicit convolution-kernel generator through Gene-FiLM. The resulting global convolutions operate on multiple Haar-wavelet subbands and are evaluated using FFTs. A differentiable Top-K predictor aggregates spatial features, followed by inter-slide aggregation and a discrete-time survival head. Conditional dropout supports inference without omics. The paper evaluates ten TCGA cohorts using patient-level five-fold cross-validation. It reports a mean multimodal C-index of 0.744 versus 0.721 for SlotSPE, with additional pathology-only and missing-omics results, wavelet-depth and basis ablations, efficiency measurements, and spatial, spectral, and pathway visualizations.

Critical strengths

S1. A useful architecture for retaining spatial structure during multimodal fusion. The combination of a 2D feature grid, multiscale wavelet processing, and pathway-conditioned implicit filters is a coherent approach to incorporating molecular information into spatial tissue processing. Conditioning the filters themselves provides an interesting alternative to token-based cross-attention. Table 2 offers meaningful evidence for the multiscale component: the cohort-mean C-index increases from approximately 0.690 with a single full-resolution convolution to 0.744 at J=3. Appendix E.1 also shows similar performance across several wavelet bases, suggesting that the benefit is not specific to Haar filters. This is the paper’s most informative architectural finding.

S2. Broad and encouraging empirical results across cancer types and modality settings. The evaluation covers ten TCGA cohorts using established patient-level splits and a common frozen CTransPath encoder. SCOPE improves the mean multimodal C-index by 0.0234 over SlotSPE and wins on seven cohorts. The pathology-only and missing-omics evaluations are valuable complements: the reported means are approximately 0.702 and 0.729, respectively. Appendix B provides cohort-level paired comparisons, and the risk-stratification analysis goes beyond a single ranking metric. The reported training memory and runtime are also promising, although the profiling setup needs further specification. Together, these results make the method worth investigating despite the concerns below.

Critical weaknesses

W1. Substantive inconsistencies in the mathematical specification weaken the soundness of the survival objective and the claimed architectural advantages.

First, Section 3.2 states that a softmax produces the survival probabilities, while Sections 3.1 and 3.4 use (C=4C=4) event-time bins and the censored likelihood (P(Y>y)P(Y>y)). If the softmax has exactly four event-time categories, (P(Y>4)=0P(Y>4)=0), giving an infinite loss for a censored observation assigned to the final bin. An additional tail category, a hazard-based parameterization, or an explicitly different treatment of terminal-bin censoring could resolve this, but none is specified. This is an ambiguity in the written method, rather than evidence that the implementation necessarily has this problem.

Second, Proposition 5 incorrectly concludes that attention cannot give the constant spatial mode a gain different from oscillatory modes. For example,

\qquad 0<a<1,$$ is strictly positive and row-stochastic, and is exactly $\operatorname{softmax}(\log P)$. It satisfies ($P\mathbf{1}=\mathbf{1}$) but ($Pv=av$) for every zero-mean vector ($v$). Thus, row-stochasticity does not imply the claimed equality of spatial-mode gains. The proof also states that changing the skip gain ($\rho$) leaves zero-mean inputs unchanged, whereas changing ($\rho$) changes the output by ($\Delta\rho\,v$). These are explicit mathematical errors in an argument used to motivate superiority over attention. They do not invalidate the reported C-indices, but the corresponding proposition and its use in Sections 4.3 and A.5 need correction. **W2. The evidence does not sufficiently establish the distinctive contribution of gene-conditioned filtering or the claimed intrinsic interpretability.** The depth and wavelet-basis ablations support multiscale processing, but they do not isolate patient-specific kernel conditioning. The pathology-only experiment removes access to omics altogether, so its gap from the multimodal model does not establish that injecting omics into the kernel is better than a simpler fusion mechanism. A matched comparison against an unconditional wavelet encoder with late omics fusion, and against an otherwise matched spatial attention model, is needed to attribute the gains to the proposed fusion mechanism. **W3. The spatial interpretation exceeds what the architecture guarantees.** Top-K selection occurs after global convolutions, so a selected position can contain information aggregated from unselected tissue regions. It identifies the contextualized features read by the prediction head, but does not by itself identify the original tissue regions responsible for the prediction. Similarly, the pathway attention weights in Equation 1 describe pooling into a conditioning vector, rather than the downstream effect of each pathway on risk. The qualitative examples and modest spectral-risk correlations, including the additional cohorts in Appendix F, are useful descriptive diagnostics, but do not establish attribution faithfulness. Since interpretable cross-modal modeling is a central contribution, input-level region perturbations and pathway interventions would materially strengthen the paper.
Questions for authors

Please refer to the strengths and weaknesses above.

7 comments
AuthorsOpenReview ↗

Response to Reviewer Ngs9 (part I)

We are grateful for the careful and technically precise review. Our point-by-point responses follow, and the corresponding revisions are marked in purple in the updated manuscript.

W1: Substantive inconsistencies in the mathematical specification

Thank you for the careful check.

Survival head with C=4C=4 bins. The softmax sentence in Sec. 3.2 was a writing error: the actual implementation uses the discrete-time hazard parameterization. Each of the C=4C=4 patient logits gives a hazard hi,c=σ(ℓi,c)h_{i,c}=\sigma(\ell_{i,c}), and Si(c)=∏k≤c(1−hi,k)S_i(c)=\prod_{k\le c}(1-h_{i,k}) with Si(0)=1S_i(0)=1, so that P(Yi=yi)=hi,yiSi(yi−1)P(Y_i=y_i)=h_{i,y_i}S_i(y_i-1) and P(Yi>yi)=Si(yi)P(Y_i>y_i)=S_i(y_i). Since every hi,k<1h_{i,k}<1, Si(C)>0S_i(C)>0, and the loss of a patient censored in the last bin is finite. To correct this, we have replaced the softmax sentence in Sec. 3.2 with the hazard parameterization (Line251-258), and written out both probabilities below Eq. (8) in Sec. 3.4 (Line300-302).

Proposition 5. The original Proposition 5 drew wrong conclusions regarding oscillatory gains and ρd\rho_d. We have corrected it in the revised text to state that the structural difference between attention and SCOPE lies in spatial filter diversity across channels:

  1. Attention: An HheadH_{\mathrm{head}}-head residual self-attention sublayer has Kronecker rank at most Hhead+1H_{\mathrm{head}}+1.
  2. SCOPE: It attains Kronecker rank ≥min⁡(D,RH,W)\ge \min(D, R_{H,W}) for almost every parameter value, where RH,WR_{H,W} is the number of distinct distances from the grid center.

The revised Proposition 5 shows that, whenever Hhead+1<min⁡(D,RH,W)H_{\mathrm{head}}+1 < \min(D, R_{H,W}) (a condition easily satisfied), the SCOPE Operator generically provides more linearly independent, channel-specific spatial filters under omics conditioning than multi-head self-attention without incurring the quadratic cost. Appendix A.3 (Line852-963) and A.5 (Line990-991) are updated accordingly.

W2: Contribution of gene-conditioned filtering

Thank you for the suggestion. We conducted two strictly matched comparisons, each changing only the target mechanism while keeping the rest of the SCOPE architecture identical:

  • Unconditional wavelet encoder with late omics fusion: keeps the exact same wavelet global-convolution encoder, disables Gene-FiLM (c=0\mathbf{c}{=}\mathbf{0}), and fuses the pathway embedding c\mathbf{c} with the pooled slide representation zs\mathbf{z}_s via concatenation before the survival MLP.
  • Matched spatial attention model: keeps the exact same pre-norm Gene-FiLM conditioning, pathway encoder and Top-KK head, and replaces only the SCOPELayer with a multi-head spatial self-attention layer over the 2D grid.
ModelBLCABRCACOADREADHNSCKIRCLUADLUSCSKCMSTADUCECMean
SCOPE0.7470.7780.7540.6900.8450.7090.6410.7220.7450.8090.744
Late omics fusion0.7260.7510.7510.6640.8340.6980.6280.7160.7250.8070.730
Spatial self-attention0.6950.6950.7120.6260.7940.6490.5720.6740.5970.7970.681
  • Gene-conditioned filtering vs. late fusion. Replacing kernel conditioning with late fusion drops the mean C-index from 0.744 to 0.730 (with paired tt-test p=9.2×10−4p=9.2\times 10^{-4}, dz=1.54d_z=1.54). Notably, late fusion with full test-time omics (0.730) barely exceeds SCOPE's Missing-Omics setting without test-time omics (0.729). This confirms that test-time transcriptomics is unlocked specifically by modulating the spatial filters.
  • SCOPE vs. matched spatial attention. Under the identical conditioning and prediction pipeline, SCOPE outperforms the matched spatial self-attention model on all ten cohorts (+0.063, p=1.1×10−4p=1.1\times 10^{-4}), confirming the advantage of channel-specific multi-scale spectral filtering over channel-shared spatial attention.

We have added the late-fusion comparison to Table 2 (Sec. 4.3 / Line431-439, per-cohort results in Appendix E.2 / Line1241-1253), and the self-attention control was reported in Appendix E.1 (Table 6, Line1215-1237).

AuthorsOpenReview ↗

Response to Reviewer Ngs9 (part II)

W3: Input-level region perturbations and pathway interventions

Thank you for this precise point. We conducted the two experiments you suggested.

1. Input-level region perturbation. On each slide we delete (set to zero) or insert (keep only) the 25% of tissue patches ranked highest by the Top-KK score, or a random 25% (3 draws). We measure the change in predicted risk (∣Δr∣|\Delta r|, in units of the cohort risk SD) and the C-index. Both tables report the ten-cohort mean ±\pm std.

RankDeletion ∣Δr∣\lvert\Delta r\rvert ↑Insertion ∣Δr∣\lvert\Delta r\rvert ↓Deletion C-index ↓
Top-KK score0.235 ± 0.0460.204 ± 0.0560.626 ± 0.016
Random0.145 ± 0.0410.354 ± 0.0970.698 ± 0.014

Deleting the top-scored patches perturbs the prediction more than random deletion, and keeping only those patches preserves it better than a random subset.

2. Pathway interventions. For each patient we replace the top-10 pathways ranked by the attention weights with their training-set means, and compare against 100 random sets of 10 pathways.

MetricTop-10 pathwaysRandom 10 pathways
Risk change ∣Δr∣\lvert\Delta r\rvert0.574 ± 0.2180.030 ± 0.005
Top-KK overlap with original (Jaccard)0.814 ± 0.0240.968 ± 0.005
Relative kernel change0.222 ± 0.0590.015 ± 0.002
High-frequency energy share change ∣Δlog⁡10∣\lvert\Delta\log_{10}\rvert0.100 ± 0.0260.010 ± 0.002

Intervening on the top-weighted pathways changes risk 19$\times$ more than random sets and exceeds the 95th percentile of the random distribution for 82% of patients. The intervention also changes the generated kernels, the energy allocated to high-frequency sub-bands and the selected regions, showing that the pathways change how the slide is processed.

We have added these experiments to Sec. 4.4 (Line463-468, 513-517) and Appendix F (Line1459-1495).

S2: Profiling setup

Thank you for the positive remark. SCOPE and all compared baselines are profiled under the exact same protocol on a single NVIDIA A800-SXM4-80GB GPU in PyTorch (fp32), processing one patient per step (batch size 1) in the full multi-modal setting on the same patient inputs. For training, after 5 warm-up steps we time 30 forward-backward-optimizer steps, calling torch.cuda.synchronize() before each timer read, and report the mean wall-clock time per step; peak memory is recorded via torch.cuda.max_memory_allocated() after resetting peak-memory statistics (torch.cuda.reset_peak_memory_stats()). For inference, we run in eval() mode under torch.no_grad() with 10 warm-up and 50 timed forward passes per patient. We have added this specification to Appendix D (Line1075-1107).

Summary

Thank you once more for the rigorous reading. Correcting the survival-head description and Proposition 5, and adding the input-level perturbation and pathway-intervention analyses, have made the paper considerably more sound. If our responses have resolved your concerns, we would sincerely appreciate it if you would re-evaluate this work. We remain available to discuss any further questions.

Reviewer_Ngs9OpenReview ↗

Official Comment by Reviewer Ngs9

Thank you for the detailed response and the additional experiments. My concerns on W1 (the survival head and Proposition 5) and W3 (the region-perturbation and pathway-intervention analyses) have been resolved, and the profiling details answer my earlier question. For W2, the late-fusion comparison supports kernel-level conditioning over concatenation within the same encoder (0.744 vs. 0.730). One question about the spatial self-attention control remains.

Specifically, the attention control (0.681) scores below even the single-scale convolution variant in Table 2 (≈0.690). This suggests that the +0.063 gap comes largely from multiscale processing rather than from the convolution itself. Could the authors compare the attention control with the single-scale (J=0) SCOPE variant, to separate these two effects?

Reviewer_3ER6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SCOPE, a multimodal survival prediction framework that uses a multi-level Haar wavelet pyramid to model the 2D spatial structure of whole-slide images (WSIs) under gene-expression conditioning. The authors first encode pathway-level gene-expression signatures and aggregate them into a patient-level conditioning vector. They then introduce a 2D Gene-Modulated Convolution (GM-Conv), in which an implicit kernel generator is modulated through Gene-FiLM based on the pathway-level representation. In this way, gene-expression information directly conditions the extraction of pathological features. GM-Conv is further incorporated into a three-level 2D Haar wavelet pyramid to capture tissue morphology across different spatial-frequency scales. Finally, a differentiable Top-KK predictor selects prognostically relevant spatial regions within each slide. The selected features are aggregated into slide-level predictions, which are subsequently combined to produce a patient-level survival prediction. The method is evaluated against a range of baselines on ten public TCGA cohorts, and the proposed components are examined through ablation studies.

Critical strengths
  1. The proposed method is supported by a relatively extensive theoretical analysis. In particular, the authors analyze the reconstruction property, computational complexity, initialization behavior, and structural characteristics of the SCOPE operator. These analyses help explain several of the architectural choices made in the paper.

  2. The empirical results are examined from several complementary perspectives. In addition to the C-index, the authors report Kaplan--Meier curves, log-rank tests, restricted mean survival time statistics, computational efficiency measurements, and several ablation and interpretability analyses. Together, these results provide a relatively broad empirical characterization of SCOPE.

Critical weaknesses
  1. The motivation for introducing a spectral-conditioned operator is not sufficiently established. The manuscript identifies the insufficient use of 2D tissue structure, the limited interpretability of multimodal fusion, and the lack of robustness to missing modalities as limitations of existing methods. However, these problems are relatively broad and have already been considered in prior work through spatially structured WSI modeling, positional encoding, interpretable multimodal interaction, and missing-modality learning. The manuscript does not clearly explain which limitations remain unresolved by these existing approaches or why a spectral-domain formulation is particularly suitable for addressing them.

  2. The method description in the main text is difficult to follow. The formulations of GM-Conv and the SCOPE operator are not presented with sufficient clarity, and several components and symbols shown in Figure 1 are not explicitly defined or consistently connected to the equations in the main text. Although additional details are provided in the appendix, the main method section is not sufficiently self-contained. This substantially affects the readability and flow of the paper.

  3. The experimental evaluation does not yet provide sufficiently targeted evidence for several central design motivations. Broader and more carefully matched baseline comparisons are needed, particularly for spatially aware WSI modeling and missing-modality learning. Moreover, additional controlled analyses are needed to isolate the respective contributions of the explicit two-dimensional grid, gene conditioning, global convolution, wavelet decomposition, and Top-KK predictor.

Questions for authors
  1. Could the authors clarify and further substantiate the motivation for the proposed spectral-conditioned architecture? Existing studies have already considered 2D tissue structure in pathology--genomics learning [1], and positional encoding in Transformer-based WSI models can also incorporate spatial information [2]. Interpretability and learning under incomplete or limited multimodal observations have likewise been explored in related work [3,4]. The manuscript should therefore clarify which specific limitations of these approaches cannot be addressed effectively without spectral-domain modeling.

    In addition, the current interpretability analysis separately visualizes selected WSI regions, wavelet sub-band energy, and pathway attention, but it does not directly examine how gene-expression information changes the processing of particular pathological regions or spatial-frequency components. The missing-omics experiments demonstrate strong predictive performance, but the current analysis does not sufficiently isolate why this performance is maintained when gene-expression data are unavailable. Without more direct analyses of these two aspects, it remains difficult to establish SCOPE's specific advantages in cross-modal interpretability and missing-modality robustness.

  2. Could the authors provide the missing definitions and clarify the notation in the method section? For GM-Conv, the computational flow illustrated in Figure 1 is difficult to reconcile with the formulation in the main text. For example, Pos. Enc. and Modulation are not explicitly defined, while the symbols KK, FF, and f′f' in the figure are not clearly connected to the notation used in the equations. Similarly, the roles of ShortConv & Split, q1,…,qNq_1,\ldots,q_N, and the gated element-wise products in the SCOPE layer are not formally described. The notation used for the wavelet decomposition and reconstruction is also difficult to follow. Since these components constitute the main technical contribution, the method section should provide a self-contained and consistent formulation.

  3. SCOPE uses the same globally aggregated gene-conditioning vector to modulate GM-Conv operators at different wavelet scales, although the GM-Conv parameters are scale-specific. Is a single global conditioning representation sufficiently expressive to model scale-specific visual--molecular associations? Could the authors explain how the architecture accommodates potential semantic differences between fine-grained pathological features and coarse tissue structures? An ablation comparing the current shared conditioning representation with scale-specific pathway aggregation or conditioning would help clarify this design choice.

  4. Could the authors expand and clarify the baseline comparisons, particularly with respect to multimodal methods designed to operate under missing or incomplete modalities [3,4]? It would also be helpful to explain whether all missing-omics baselines are trained and evaluated under the same modality-availability protocol as SCOPE. Since different methods may assume different patterns of modality availability during training, a matched experimental protocol is necessary for a fair comparison.

  5. The interpretability analysis would benefit from more direct qualitative and quantitative validation. For spatial interpretability, could the Top-KK heatmaps be compared with annotations from pathologists or with relevant histological regions? The faithfulness of the selected regions could also be evaluated through region masking, deletion, or insertion experiments. For multimodal interpretability, pathway perturbation or counterfactual analysis could be used to examine how specific pathways affect the learned convolution kernels, spectral allocation, selected WSI regions, and final risk prediction. It would also be useful to report the stability of these explanations across folds and random seeds.

References

[1] Zheng, Y., et al. "Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival." IEEE Transactions on Medical Imaging, 43(9):3085--3097, 2024.

[2] Zheng, Y., et al. "Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis." IEEE Transactions on Medical Imaging, 42(9):2726--2739, 2023.

[3] Xu, Y., et al. "A multimodal knowledge-enhanced whole-slide pathology foundation model." Nature Communications, 16:11406, 2025.

[4] Wu, K., et al. "Histopathology-Genomics Multi-modal Structural Representation Learning for Data-Efficient Precision Oncology." International Conference on Learning Representations, 2026.

9 comments
AuthorsOpenReview ↗

Response to Reviewer 3ER6 (part I)

We sincerely thank you for the detailed and constructive review. Below we address each weakness and question point by point. We have revised the paper accordingly, and all changes in the revised PDF are highlighted in purple.

W1 & Q1(a): Motivation for the spectral-conditioned architecture

Thank you for your insightful suggestions. We clarify below which specific limitations of existing approaches cannot be addressed effectively without a spectral-conditioned architecture.

  1. No lossless separation of tissue scales. Existing multi-scale omic-pathology models (e.g., MSMR [1]) re-tile the slide at several magnifications and encode each bag separately, so the scales are redundant and fused only afterwards. Only an orthogonal wavelet decomposition splits a single feature grid into spatially localized sub-bands that are both non-redundant and exactly invertible. Fourier bases lack locality, Laplacian pyramids are redundant, and pooling is lossy.
  2. Omics cannot shape how the slide is processed. In existing works, omics interacts with the slide only after or independently of slide encoding (co-attention [2], pre-training [4], and late concatenation [5]), so the visual pathway is identical for every patient and the regulatory coupling between molecular programs and morphology is not exposed. Making the visual pathway patient-specific requires a conditional operator generated from each patient's profile.
  3. Global 2D context vs. cost. In the spatial domain, a global receptive field over NN patches costs Θ(N2)\Theta(N^2) (self-attention). [2] and [3] restrict mixing to an 8-neighbor patch graph or fixed Gaussian masks, i.e., local or hand-shaped context, and raster-scanned SSMs reach Θ(N)\Theta(N) only by imposing an axis asymmetry on the 2D grid. Among these token mixers, only the spectral-domain operator is simultaneously global, isotropic in 2D and sub-quadratic.

The existing experiments, together with those added in the revision, substantiate each of the three points:

  1. Tissue scales. Removing the wavelet path (J=0J{=}0) lowers the mean C-index from 0.744 to 0.690, and the wavelet setting outperforms both non-wavelet controls on all ten cohorts (Appendix E.1). The sub-band energy is also associated with risk in a cancer-type-specific way (Sec. 4.4).
  2. Gene-conditioned visual processing. Replacing Gene-FiLM with late fusion drops the mean C-index from 0.744 to 0.730 (Table 2), which barely exceeds SCOPE's Missing-Omics setting without test-time omics (0.729). Resetting a patient's top-10 attended pathways changes the generated kernels by 22.2%, vs. 1.5% for random pathways (Appendix F).
  3. Global 2D context vs. cost. Treating patches as 1D tokens or restricting the kernel to a local 3×33{\times}3 lowers the mean C-index by 0.042 and 0.036 (Table 2). The JJ-level pyramid costs two global convolutions for every J≥1J\ge1 (Prop. 2), and SCOPE trains with 222 MB and 0.025 s per step vs. 429 MB and 0.041 s for SlotSPE (Appendix D).

To clarify the motivation, we have rewritten the Abstract (Line014-017), Sec. 1 (Line043-067) and Sec. 2 (Line112-120) in the revised text, and added the related works [1-5] to the discussion.

[1] Hou, Q., et al. "Multi-scale Multimodal Representation for Enhanced Survival Prediction in Computational Pathology." Interdisciplinary Sciences: Computational Life Sciences, 2026.

[2] Zheng, Y., et al. "Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival." IEEE Transactions on Medical Imaging, 43(9):3085-3097, 2024.

[3] Zheng, Y., et al. "Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis." IEEE Transactions on Medical Imaging, 42(9):2726-2739, 2023.

[4] Xu, Y., et al. "A multimodal knowledge-enhanced whole-slide pathology foundation model." Nature Communications, 16:11406, 2025.

[5] Wu, K., et al. "Histopathology-Genomics Multi-modal Structural Representation Learning for Data-Efficient Precision Oncology." International Conference on Learning Representations, 2026.

W2 & Q2: Clarity of the method description

Thank you for the careful reading. To make the method self-contained, we have rewritten Sec. 3.1 (Line160-161), Sec. 3.2 (Line216-258) and Sec. 3.3 (Line262-281) in the revised text, and relabeled Fig. 1 to match these equations.

AuthorsOpenReview ↗

Response to Reviewer 3ER6 (part II)

W3(a) & Q4: Matched baseline comparisons and modality-availability protocol

Thank you for the suggestion. We have added four baselines to Table 1: the spatially aware GAT-Fusion [1] (Pathology + Transcriptome) and KAT [2] (Pathology-Only), and the missing-modality methods mSTAR [3] and MSRL [4] (Missing-Omics). All methods use the same evaluation settings as SCOPE.

Modality-availability protocol. In the Missing-Omics setting, SCOPE and all compared baselines follow a strictly matched protocol: models are trained on complete paired data (WSI + omics) and evaluated at test time using WSI alone, where each baseline handles the absent transcriptomics via its native inference mechanism.

SettingMethodBLCABRCACOADREADHNSCKIRCLUADLUSCSKCMSTADUCECMean
Pathology + TranscriptomeGAT-Fusion [1]0.6360.6900.6790.5860.7700.6230.5700.6640.6370.7800.664
Pathology + TranscriptomeSCOPE0.7470.7780.7540.6900.8450.7090.6410.7220.7450.8090.744
Pathology-OnlyKAT [2]0.6270.6230.6580.5920.7790.6240.6020.5920.6470.7240.647
Pathology-OnlySCOPE0.6600.7420.7450.6660.7990.6680.6120.6680.6780.7860.702
Missing-OmicsMSRL [4]0.5620.6510.6390.6180.7270.6030.5500.6570.6350.6600.630
Missing-OmicsmSTAR [3]0.6550.7190.7290.6490.7970.6460.6160.6510.6400.7950.690
Missing-OmicsSCOPE0.7080.7670.7530.6900.8380.6650.6470.6760.7430.8070.729

In each setting, SCOPE outperforms the new baselines on all 10 cohorts, and all gains are significant under the paired tt-test over the ten cohorts (GAT-Fusion: Δ‾=+0.0805\overline{\Delta}=+0.0805, p=3.1×10−6p=3.1\times10^{-6}; KAT: Δ‾=+0.0556\overline{\Delta}=+0.0556, p=5.9×10−4p=5.9\times10^{-4}; MSRL: Δ‾=+0.0992\overline{\Delta}=+0.0992, p=2.2×10−5p=2.2\times10^{-5}; mSTAR: Δ‾=+0.0397\overline{\Delta}=+0.0397, p=8.9×10−4p=8.9\times10^{-4}).

We have added these compared methods to Table 1 (Line325-354) and the significance table in Appendix B (Line1027-1056), and updated the baselines paragraph in Sec. 4.1 (Line323-359).

[1] Zheng, Y., et al. "Graph attention-based fusion of pathology images and gene expression for prediction of cancer survival." IEEE Transactions on Medical Imaging, 43(9):3085-3097, 2024.

[2] Zheng, Y., et al. "Kernel attention transformer for histopathology whole slide image analysis and assistant cancer diagnosis." IEEE Transactions on Medical Imaging, 42(9):2726-2739, 2023.

[3] Xu, Y., et al. "A multimodal knowledge-enhanced whole-slide pathology foundation model." Nature Communications, 16:11406, 2025.

[4] Wu, K., et al. "Histopathology-Genomics Multi-modal Structural Representation Learning for Data-Efficient Precision Oncology." International Conference on Learning Representations, 2026.

AuthorsOpenReview ↗

Response to Reviewer 3ER6 (part III)

W3(b): Controlled ablations of the five components

Thank you for the suggestion. We conducted component ablations that change one factor of SCOPE at a time. In the table below, removing or replacing any of the five components degrades performance on all ten cohorts:

  • 2D grid. Flattening the 2D tissue grid into a 1D sequence of patch tokens drops the mean C-index (0.744 →\to 0.702), confirming that preserving the 2D spatial layout of tissue is essential.
  • Gene conditioning. With the same wavelet encoder, letting omics modulate the spatial filtering outperforms fusing them after encoding (0.744 vs. 0.730).
  • Global convolution. Restricting the kernel to a local 3×33{\times}3 loses 0.036 and replacing the operator with global self-attention loses 0.063, confirming that the gain comes from global, channel-specific spectral filtering.
  • Wavelet decomposition. Disabling the multi-level wavelet pyramid (J=0J{=}0) causes the largest single-component drop (0.744 →\to 0.690), showing that multi-scale spatial-frequency decomposition is critical.
  • Top-KK predictor. The gain is smaller but consistent, and the selected regions are faithful to the prediction (see the deletion/insertion test in Q1(b) & Q5).
ComponentVariantBLCABRCACOADREADHNSCKIRCLUADLUSCSKCMSTADUCECMean
—SCOPE (full)0.7470.7780.7540.6900.8450.7090.6410.7220.7450.8090.744
2D grid1D bag of tokens0.6880.7120.7300.6390.8030.6750.5800.7060.6920.7920.702
Gene conditioningLate omics fusion0.7260.7510.7510.6640.8340.6980.6280.7160.7250.8070.730
Global convolutionLocal 3×33{\times}3 kernel0.7160.7290.7240.6460.8180.6860.5900.6860.7040.7810.708
Global convolutionSelf-attention0.6950.6950.7120.6260.7940.6490.5720.6740.5970.7970.681
Wavelet decompositionNo wavelet path0.7240.6960.6980.6230.7800.6780.5980.6930.6270.7820.690
Top-KK predictorMean pooling0.7420.7660.7500.6750.8320.6980.6290.7210.7310.7960.734
Top-KK predictorGated attention pooling0.7440.7690.7480.6800.8420.7000.6290.7160.7270.8050.736

We have added these ablations to Sec. 4.3 (Line423-449) and Appendix E.2 (per-cohort results, Line1240-1253).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers and the AC for their time and constructive feedback. All revisions are highlighted in purple in the updated paper.

Strengths noted by the reviewers

  • Design. Reviewers found that conditioning the filters on omics is "a useful architecture for retaining spatial structure during multimodal fusion" and "an interesting alternative to token-based cross-attention" (Ngs9), and described Gene-FiLM as "an ingenious method to modulate model parameters" (8gLR).
  • Interpretability. Reviewers found "the paper's contribution to interpretability is impressive". "By analyzing the energy distribution across frequency bands", SCOPE shows "whether the model is observing coarse structures or fine textures", while "attention-based models can only reveal which patches the model has viewed" (8gLR).
  • Theory and evaluation. Reviewers noted that the method is supported by "a relatively extensive theoretical analysis" and that the results give "a relatively broad empirical characterization of SCOPE" (3ER6), with "broad and encouraging empirical results across cancer types and modality settings" (Ngs9).

Main revisions

  1. Motivation and method. We clarified the motivation (Abs. Line 014-017; Sec. 1 Line 043-067; Sec. 2 Line 112-120), rewrote Sec. 3 to be self-contained (Line 223-287), corrected the survival head (Sec. 3.4 Line 305-307) and Proposition 5 (Appendix A.3 Line 852-963), and added pseudo-code and a notation table (Appendix G Line 1508-1673).
  2. Baselines. We added GAT-Fusion, KAT, mSTAR, MSRL, DSMIL and WiKG under a matched protocol (Sec. 4.1 Line 358-363). SCOPE outperforms all of them (Line 336-352), with significant gains under the paired tt-test (Line 1040-1054).
  3. Ablations. One-factor-at-a-time ablations of the 2D grid, gene conditioning, global convolution, wavelet decomposition, Top-KK predictor, Gene-FiLM position, depth, grid size and scale-specific conditioning (Sec. 4.3 Line 425-450; Appendix E.2 Line 1241-1253), plus a single-scale (J=0J{=}0) vs. self-attention comparison (Appendix E.1 Line 1228-1233).
  4. Interpretability. Comparison with pathologist annotations (Appendix F Line 1502-1506), region deletion/insertion, pathway interventions and counterfactual conditioning (Sec. 4.4 Line 464-468, 513-518; Appendix F Line 1483-1500).
  5. Cross-cohort findings. The coarse- vs. fine-scale regimes are moved to the main text with biological explanations (Sec. 4.4 Line 479-485; Appendix F Line 1392-1481).

Status of the discussion

  • 3ER6 stated that we have "largely addressed" the concerns on motivation, experimental design, ablations and interpretability, and raised the rating from 2 to 3 (Weak acceptance) after we added the requested pseudo-code and notation table in Appendix G.
  • Ngs9 confirmed that "this resolves my main concerns". Ngs9 raised the rating from 2 to 3 (Weak acceptance), citing "the architectural contribution and the empirical results across cancer cohorts and modality settings".
  • 8gLR confirmed that our "replies address my concerns" and maintained a rating of 3 (Weak acceptance).

We are grateful for these discussions, which have substantially strengthened the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.