- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a model named SCOPE (Spectral-Conditioned Omic-Pathology Encoder). The model combines two modalities, WSI (Whole-Slide Image) and genes, to predict the five-year survival probabilities of cancer patients. GM-Conv (Gene-Modulated Convolution) is proposed to deeply integrate the gene modality to the original WSI processing network, which is realized by inserting a gene-conditioned normalization method named Gene-FiLM. The model also integrates Haar DWT to capture the energy distribution across frequency bands. The model possesses strong interpretability, including which WSI regions the model fouces on, whether it focused on coarse-grained structure or fine-grained details, and the influence of omics. Overall, this paper solves a traditional problem, while the experiments are sufficient and the methodology has novel aspects.
Critical strengths
- Gene-FiLM, an ingenious method to modulate model parameters. The module influences model parameters in a normalization form, which can be written as: , where and are the model parameters before and after modulation (e.g. the weights of a convolution), and and are scaling and bias factors conditioned by genes. I consider this design ingenious for two reasons. First, by initializing and to zero, the model starts from an unmodulated state, using only the WSI modality for prediction. Second, by randomly dropping out and during training, the model learns to make predictions even when the gene modality is missing. This module makes the incorporation of the gene modality has almost no damage to the original model performance.
- Spectral interpretability. The paper's contribution to interpretability is impressive. Especially, the model integrates Haar DWT (Haar Discrete Wavelet Transform). By analyzing the energy distribution across frequency bands, I can observe whether the model is observing coarse structures or fine textures when processing WSI patches. In contrast, attention-based models can only reveal which patches the model has viewed.
Critical weaknesses
- Present when the model observes coarse structures and when it observes fine textures more clearly. Appendix~F (line 1126-1133) analyzes which gene cohorts lead the model to observe coarse structures, which lead to fine textures, and which lead to neither. However, the writing is too vague and lacks focus. More importantly, this is one of the most interesting findings of the paper, while it is in the appendix. I suggest providing a simplified version in the main text, as not everyone reads the appendix. Furthermore, I am interested in why certain disease gene cohorts drive the model to focus on coarse-grained structures, others on fine-grained textures, and still others on neither. The reason for the observation differences could be an interesting discussion.
- There are insufficient ablation experiments. The proposed SCOPE framework consists of heavily‑nested components: GM‑Conv incorporates GeneFiLM; the SCOPE Operator stacks multiple GM‑Conv blocks; each SCOPE Layer contains several SCOPE Operators; and the overall SCOPE Encoder further repeats multiple modules equipped with GeneFiLM and SCOPE Layers. Given such deep nesting, it is difficult to pinpoint which specific design choices contribute to the final survival‑prediction performance gain. In the main text, the authors only conduct ablation over wavelet decomposition depth J within the SCOPE Operator. The authors are requested to explicitly clarify which components represent their original innovations and supply comprehensive ablation studies to quantify the individual contribution of each building block. Recommended ablations include: ablations for different insertion positions of Gene‑FiLM, residual block depth L, the differentiable Top‑K operator, and the grid size of patch features.
- Moreover, more baselines should be added for the Pathology‑only setting, such as DSML [1] and WiKG [2].
[1] Li, Bin, Yin Li, and Kevin W. Eliceiri. "Dual‑stream multiple instance learning network for whole slide image classification with self‑supervised contrastive learning." 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2021. [2] Li, Jiawen, et al. "Dynamic graph representation with knowledge‑aware attention for histopathology whole slide image analysis." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
Questions for authors
In Table~1, I observe a phenomenon. For all gene cohorts, the scores of the SCOPE model follow the order Pathology + Transcriptome Missing-Omics Pathology-Only. This aligns with intuition, as incorporating gene information helps the model better leverage WSI features for survival prediction. While, the LUAD and LUSC cohorts (still for the SCOPE model) are different. LUAD is Pathology + Transcriptome Pathology-Only Missing-Omics, and LUSC is Missing-Omics Pathology + Transcriptome Pathology-Only. More coincidentally, the two abnormal cohorts are both lung cancer cohorts, and the SlotSPE baseline behaviors normally. I am curious whether there are underlying reasons for this.
6 comments
Response to Reviewer 8gLR (part II)
W2: Original components and component-wise ablations
Thank you for the suggestion. Our contributions lie in how omics conditions the spatial filtering:
- GM-Conv (Eq. 3). enters the implicit kernel generator through Gene-FiLM, so every patient gets its own global 2D kernel, whereas the kernels of prior implicit-filter models do not depend on the input.
- The SCOPE Operator (Eqs. 6-7). Each wavelet detail band and the deepest approximation have their own GM-Conv, recombined by learned cross-scale mixing.
- Gene-FiLM as the only entry point of omics (Eqs. 2, 4). It is zero-initialized. With it reduces to a parameter-free standardization, which gives the missing-omics path without auxiliary losses.
We conducted component-wise ablations that change one factor of SCOPE at a time.
| Component | Variant | BLCA | BRCA | COADREAD | HNSC | KIRC | LUAD | LUSC | SKCM | STAD | UCEC | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| — | SCOPE (full) | 0.747 | 0.778 | 0.754 | 0.690 | 0.845 | 0.709 | 0.641 | 0.722 | 0.745 | 0.809 | 0.744 |
| Gene-FiLM position | Post-norm FiLM | 0.735 | 0.772 | 0.748 | 0.674 | 0.845 | 0.696 | 0.628 | 0.718 | 0.737 | 0.797 | 0.735 |
| Residual block depth | 0.750 | 0.763 | 0.753 | 0.677 | 0.847 | 0.704 | 0.649 | 0.723 | 0.738 | 0.812 | 0.742 | |
| Residual block depth | 0.745 | 0.761 | 0.739 | 0.671 | 0.839 | 0.716 | 0.637 | 0.727 | 0.723 | 0.811 | 0.737 | |
| Top- gate | Mean pooling | 0.742 | 0.766 | 0.750 | 0.675 | 0.832 | 0.698 | 0.629 | 0.721 | 0.731 | 0.796 | 0.734 |
| Top- gate | Gated attention pooling | 0.744 | 0.769 | 0.748 | 0.680 | 0.842 | 0.700 | 0.629 | 0.716 | 0.727 | 0.805 | 0.736 |
| Grid size | 0.703 | 0.689 | 0.713 | 0.633 | 0.809 | 0.676 | 0.565 | 0.700 | 0.660 | 0.785 | 0.693 | |
| Grid size | 0.754 | 0.770 | 0.759 | 0.693 | 0.854 | 0.712 | 0.645 | 0.728 | 0.741 | 0.814 | 0.747 |
- Gene-FiLM position. Applying Gene-FiLM after each sub-layer instead of as pre-norm lowers the mean by 0.009.
- Residual depth . Over , deeper stacks slightly hurt, consistent with the modest cohort sizes.
- Differentiable Top-. Replacing the gate with mean or gated attention pooling lowers the mean by 0.010 and 0.008.
- Grid size. A grid loses 0.051. A grid gains only 0.003 at the tokens.
We have added these ablations to Sec. 4.3 (Line423-449) and Appendix E.2 (per-cohort results, Line1240-1253).
W3: More pathology-only baselines
Thank you for the suggestion. We added DSMIL [1] and WiKG [2] to the Pathology-Only block of Table 1. Both were proposed for slide classification, so we keep their official architectures and replace only the classification output with the same discrete-time survival head and NLL loss as SCOPE. All methods share the same patch features, 5-fold splits and DSS labels.
| Method | BLCA | BRCA | COADREAD | HNSC | KIRC | LUAD | LUSC | SKCM | STAD | UCEC | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DSMIL | 0.656 | 0.687 | 0.701 | 0.596 | 0.777 | 0.630 | 0.572 | 0.602 | 0.637 | 0.735 | 0.659 |
| WiKG | 0.664 | 0.710 | 0.709 | 0.629 | 0.773 | 0.655 | 0.582 | 0.629 | 0.669 | 0.764 | 0.678 |
| SCOPE | 0.660 | 0.742 | 0.745 | 0.666 | 0.799 | 0.668 | 0.612 | 0.668 | 0.678 | 0.786 | 0.702 |
SCOPE outperforms DSMIL on all 10 cohorts and WiKG on 9 of 10. Both gains are significant under the paired -test over the ten cohorts (DSMIL: , ; WiKG: , ).
We have revised Table 1 (Line 342-343), the significance table in Appendix B (Line 1046-1047), and the baselines paragraph in Sec. 4.1 (Line356) to add these methods.
[1] Li, B., et al. "Dual-stream multiple instance learning network for whole slide image classification with self-supervised contrastive learning." CVPR, 2021.
[2] Li, J., et al. "Dynamic graph representation with knowledge-aware attention for histopathology whole slide image analysis." CVPR, 2024.
Response to Reviewer 8gLR (part III)
Q: Ordering of the three settings on LUAD / LUSC
Thank you for this insightful observation. We attribute the behavior on the two non-small cell lung cancer cohorts (LUAD and LUSC) to both biological and architectural factors:
- Biological perspective. LUAD prognosis is largely determined by histological growth patterns: the IASLC grading system [1] stratifies patients by the predominant pattern and the proportion of high-grade (solid, micropapillary, complex glandular) components, i.e., tissue architecture that is directly visible in the WSI. LUSC is histologically more homogeneous, and its bulk transcriptome mixes tumor, stromal and immune compartments in a smoking-driven, highly mutated background, so its transcriptomic prognostic signal is weaker and more heterogeneous across patients.
- Architectural perspective.
- SCOPE. Because modulates every GM-Conv kernel, the reliability of the transcriptome directly shapes how the slide is filtered. On LUSC, patient-specific noise in a weakly informative transcriptome may propagate into the spatial filtering of all positions, whereas withholding falls back to unmodulated kernels that retain the morphology representation learned with omics as training-time guidance and are regularized by conditional dropout, which may explain why Missing-Omics exceeds both Pathology + Transcriptome and Pathology-Only. On LUAD, joint training may route part of the prognostic signal through , so the unmodulated kernels may invest less in morphological cues than a dedicated WSI-only model, which may explain why Pathology-Only exceeds Missing-Omics.
- SlotSPE. SlotSPE compresses the slide and the omics into separate sets of slots and fuses them by cross-attention between the two sets, so its slide encoding does not depend on omics. When omics is missing, it imputes the omics features from histology through a cross-modal reconstruction head trained jointly with the survival objective. Its missing-omics path thus substitutes a histology-derived estimate, which may make its ordering less sensitive to the cohort-specific reliability of the transcriptome. SCOPE nevertheless remains above SlotSPE in all three settings on both cohorts.
[1] Moreira, A. L., et al. "A Grading System for Invasive Pulmonary Adenocarcinoma: A Proposal From the International Association for the Study of Lung Cancer Pathology Committee." Journal of Thoracic Oncology, 15(10):1599-1610, 2020.
Summary
We appreciate your thoughtful suggestions, which helped us bring the cross-cohort scale findings into the main text, quantify the contribution of each component, and broaden the pathology-only comparison. We believe the paper is much stronger as a result. We hope the revision merits a more favorable evaluation, and we would be glad to answer any further questions promptly.
Response to Reviewer 8gLR (part I)
We thank you for the positive assessment and for the encouraging comments. We respond to each comment below and have updated the paper accordingly, with all revisions highlighted in purple.
W1: Coarse vs. fine findings across cancer cohorts
Thank you for this insightful suggestion. As analyzed in Appendix F, the Pearson correlation between a patient's coarse-band (LL) energy fraction and predicted risk, rLL, groups the ten cohorts into three regimes, each consistent with known scale-specific prognostic morphology:
Following your advice, we have promoted a concise summary of these three regimes into the main text (Sec. 4.4, Line479-485) and expanded the biological and molecular discussion in Appendix F (Line1386-1457).
[1] Nagtegaal, I. D., et al. "The 2019 WHO classification of tumours of the digestive system." Histopathology, 76(2):182-188, 2020.
[2] Thorsson, V., et al. "The Immune Landscape of Cancer." Immunity, 48(4):812-830, 2018.
[3] Moreira, A. L., et al. "A Grading System for Invasive Pulmonary Adenocarcinoma: A Proposal From the International Association for the Study of Lung Cancer Pathology Committee." Journal of Thoracic Oncology, 15(10):1599-1610, 2020.