- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ViO-Align, which transfers single-cell molecular supervision from Xenium spatial transcriptomics to H&E-only in situ cell phenotyping. Stage I performs cross-modal contrastive alignment via morphology-conditioned gene gating (MGG), and Stage II bridges the molecular supervision to raw-patch localization and phenotyping through registered cell identities. ViO-Align gains 4.67 macro-F1 points on average over four phenotyping benchmarks and attains the highest AUROC on all five molecular cell-state tasks.
Critical strengths
-
Clear motivation and well-positioned framework. The paper distills existing gaps into supervision-, context-, and inference-level mismatches (Figure 1). Its cell-level alignment → raw-patch transfer → H&E-only inference design connects them sensibly, with practical value.
-
Complete method with thorough ablations. MGG, CFL, MEA, and cross-stage alignment each play a distinct role, and ablations cover all key components. Removing MGG drops Stage-I Top-1 retrieval from 92.88% to 33.14% (Table 4).
-
Rigorous evaluation protocol. The strictly test-disjoint principle is enforced, excluding baselines whose weights overlap the test set. HoVer-Net and CellViT++ are re-evaluated under identical splits and evaluator (Section 4.1).
-
Comprehensive tasks with consistent gains. Beyond the four benchmarks (average macro-F1 +4.67), five molecularly defined cell-state tasks directly test molecularly guided representations. ViO-Align attains the highest AUROC on all five (Table 3).
-
Compact model. ViO-Align uses only 19.6M parameters, far smaller than baselines such as HoVer-Net (54.7M). This is an advantage in the accuracy–efficiency trade-off.
Critical weaknesses
-
(Primary weakness) The core idea is straightforward, but the writing is over-packaged and hard to read. Stripped of stacked self-coined terms and module names, the paper is essentially cross-modal pretraining on Xenium pairs plus identity-bridged transfer to raw patches. Key details are pushed to the appendix, leaving the training procedure unreconstructable from the main text.
-
The critical no-transcriptomic-supervision control is missing. Stage-I alignment is never replaced by visual self-supervision or ImageNet initialization under an identical budget. The alternative explanation—gains mainly from large-scale upstream pretraining—thus cannot be ruled out.
-
Upstream–downstream tissue overlap is an undiscussed confound. The 15 upstream Xenium datasets include breast cancer samples, while downstream BreaKHis and three molecular-state tasks are also breast cancer. The paper never discusses or removes this confound.
Questions for authors
-
Most critical: please provide the no-transcriptomic control—identical pipeline and budget, with Stage-I alignment replaced by visual self-supervision or ImageNet initialization—and report the downstream results.
-
What is downstream performance after excluding breast-cancer or same-organ upstream data, and how large is the contribution of tissue overlap?
-
Is the anomalous CellViT++ result on MoNuSAC (Fd = 46.15) due to class mapping or the evaluation pipeline? Do the 300 epochs in Table 10 and the "at most 100 epochs" in the text denote different phases, and what are the missing λ_center/λ_count values?
8 comments
W2 — "The critical no-transcriptomic-supervision control is missing"
We thank you for requesting the direct control needed to distinguish the contribution of transcriptomic supervision from that of additional upstream visual pretraining. We agree that the relevant comparison must preserve the two-stage architecture and training schedules while changing the source of Stage-I teacher supervision. To this end, we have trained a matched-epoch H&E-only self-supervised teacher and compared it with the molecularly supervised teacher under identical Stage-II and downstream procedures. The control design and its downstream results are detailed below.
(1) Exactly what is changed. Stage I uses the same Xenium H&E cell crops, data splits, randomly initialized visual encoder, augmentations, optimizer, batch size, and 50 epochs, but learns only from two-view image NT-Xent. We remove the RNA encoder, MGG, RNA-view objective, and H&E–RNA contrastive objective. This produces a purely visual Stage-I teacher. Both variants then train a newly initialized Stage II with the same uncropped patches, localization targets, identity-matched NCE objective, and 300-epoch schedule, followed by identical supervised downstream fine-tuning. Thus the control changes the source of teacher supervision, not the transfer architecture; matched epochs do not necessarily imply identical total compute.
(2) Downstream results ( / phenotype , %; merged main Table 5).
| Dataset | H&E-only | Full | H&E-only | Full | ||
|---|---|---|---|---|---|---|
| CoNSeP | 74.16 | 76.18 | +2.02 | 57.83 | 64.22 | +6.39 |
| GLySAC | 84.25 | 87.07 | +2.82 | 58.16 | 68.68 | +10.52 |
| MoNuSAC | 82.15 | 83.96 | +1.81 | 67.68 | 73.68 | +6.00 |
| PanNuke | 80.16 | 82.22 | +2.06 | 51.86 | 53.61 | +1.75 |
| Average | 80.18 | 82.36 | +2.18 | 58.88 | 65.05 | +6.17 |
Under matched Stage-I and Stage-II training schedules, transcriptomic supervision yields +2.18 pp average detection F1 and +6.17 pp average phenotype macro-F1. The larger phenotyping gains provide direct evidence that the molecular signal contributes beyond additional H&E-only pretraining.
Revision in the manuscript: The controlled experiment and its results appear in the merged Table 5 (Section 4.3), with exact image-only NT-Xent, encoder, and training-protocol details in Appendix "Transcriptomics-Free H&E-only Control."
W3 — "Upstream–downstream tissue overlap is an undiscussed confound"
We thank you for highlighting the possibility that shared tissue types between upstream pretraining and downstream evaluation could confound the interpretation of transfer performance. We understand that this requires distinguishing overlap in biological samples from overlap at the organ level, and then testing whether same-organ pretraining materially influences the relevant downstream results. We therefore clarify the sample-disjoint data setting and compare full, breast-only, and leave-breast-out pretraining on four breast molecular-state tasks. The scope and findings of this analysis are described below.
(1) Sample overlap versus organ overlap. The Xenium pretraining cohort and downstream datasets contain distinct biological samples. Some tasks share the tissue type, which is different from sharing patients or slides. We now state this distinction explicitly.
(2) Breast-exclusion check. We compared Full, Breast-only, and Leave-Breast-Out pretraining on four breast molecular-state tasks:
| Pretraining sources | Four-task mean AUROC |
|---|---|
| Full | 0.6742 |
| Breast-only | 0.6624 |
| Leave-Breast-Out | 0.6702 |
Removing all breast-derived pretraining data changes the average AUROC by −0.0040, compared with Full. These results show limited sensitivity of these four molecular-state evaluations to same-organ pretraining. They do not refer to a separate BreaKHis tissue-exclusion run.
Revision in the manuscript: The Appendix "Impact of Breast-Derived Upstream Pretraining Data" includes this controlled comparison; the corresponding discussion distinguishes same-organ from sample-level overlap.
We thank you for the detailed and constructive feedback, including the recognition of our framework's motivation and practical H&E-only inference setting. We understand that the key concerns are whether the two-stage procedure is sufficiently clear and reproducible, whether its gains truly come from transcriptomic supervision rather than additional image pretraining, whether same-organ pretraining affects downstream results, and whether the CellViT++ comparison and training specifications are correct. We have therefore reorganized the method description, added a matched-schedule H&E-only control and breast-exclusion analyses, corrected the CellViT++ evaluation, and made the training schedules and loss weights explicit. We address these points individually below.
W1 — "Overly verbose and difficult to read" / "Key details are relegated to the appendix"
We thank you for acknowledging the motivation of the framework while pointing out that excessive terminology and appendix-dependent implementation details obscure the actual training procedure. We understand that the central concern is whether a reader can reconstruct the sequence from single-cell molecular alignment to raw-patch transfer and H&E-only inference without piecing together scattered descriptions. We have therefore simplified the manuscript's organization, brought essential training specifications into the main text, and aligned the ablation discussion with the two supervision-transfer steps. To make the sequence reconstructable at a glance, we first summarize the inputs, supervision, and learned role of each training phase below, followed by the corresponding manuscript revisions.
Overview of the three training phases, including their inputs, supervision, and learned roles.
(1) Simplified organization. The revised Introduction focuses on cell-crop molecular supervision versus raw-patch in situ inference. The Method follows the actual execution order—framework overview (Section 3), Stage-I molecular representation learning (3.1), identity-matched Stage-II transfer (3.2), and H&E-only downstream phenotyping (3.3)—instead of introducing the framework through an expanding set of module labels. MGG, CFL, and MEA remain only where necessary to specify mechanisms and ablations.
(2) Key details moved into the main text. Section 3 now explains the Stage-I/II inputs, objectives, fixed teacher references, physical-cell-ID positives, and the transition from registered to predicted centers at inference. Section 4.1 separately specifies the 50 / 300 / up-to-100-epoch schedules, optimizers, and complete Stage-II loss weights (5, 20, 20, 0.5, 0.1). Target-construction algorithms remain in the Appendix.
(3) Experiments now follow the two transfers. Section 4.3 / Table 5 combines two independently defined controls that test (i) the RNA-derived Stage-I teacher and (ii) the Stage-II alignment objective. These directly correspond to the two mechanisms described in Section 3.
Revision in the manuscript: Sections 1, 3, 4.1, and 4.3 have been rewritten/expanded for clarity and reproducibility; Figures 1–2 are retained, not claimed as newly redesigned.