- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper focuses on the problems of multi-omics understanding. The main discovery is the study of missing mIF protein signals in a target tissue section. Instead of just using or predicting from HE morphology alone, this work targets the cross-encoding mappings. It proposes a third paradigm: transferring experimentally measured mIF signals from a reference section, usually a physically adjacent serial section of the tissue block. The method NeighborMIF first registers the target and reference HE WSIs, then builds cell-level morphological features by aggregating visual tokens of cell masks. Then it extracts per-cell measured mIF vectors from the reference section and transfers them to target cells via a learnable query key matcher.
Critical strengths
- This paper proposes a new well-motivated problem formulation that contains pathology images, mIF, and other information.
- Instead of just natively ensembling the information, the paper proposes a valid method to better encode different modality data.
- The experiment and evaluation design are well handled and managed. The reference bank for the cross-sample setting is restricted to the training split without leakage.
Critical weaknesses
- The evaluation set is a very small private dataset, which limits the effectiveness and generalizability of the paper.
- How much of the method's effectiveness comes from the neighbors' signal? The comparison is not very clear; an ablation study might be needed to justify the performance gain.
- The experimental setting and comparison design need more clarity. Are the external baselines trained on the private data? if so, more clarification and controls are needed for the baseline comparison.
Questions for authors
- As mentioned above, what if a naive baseline performs on the consecutive section benchmark, like directly transferring the registered reference mIF to the target cells?
- How does a simple k-NN regressor compared to the learned matcher?
- How were those baselines retrained and compared?
2 comments
Reply to Reviewer xZYR (Part2/2)
[Q1] What if a naive baseline performs on the consecutive section benchmark, like directly transferring the registered reference mIF to the target cells?
[A4] : Thank you for this question. We added three types of baselines:
(1) Registered-mIF Copy: Directly assigns the Reference mIF at the registered spatial location to each Target cell.
(2) Spatial-kNN: Selects the nearest Reference cells based on the spatial distance of cell centers after registration, and averages their mIF signals.
(3) Morphology-5NN-Mean: Selects the five most morphologically similar Reference cells based on cosine similarity between frozen H-optimus cell features and averages their measured mIF signals.
All these baselines directly transfer measured Reference mIF without NeighborMIF's learnable matching or subsequent molecular feature reconstruction and fusion.
| Method | AFP | CK7 | ARG1 | CDX2 | SATB2 | Mean |
|---|---|---|---|---|---|---|
| Registered-mIF Copy | 48.98 | 56.84 | 61.45 | 53.90 | 55.34 | 55.30 |
| Spatial-1NN | 55.12 | 58.92 | 57.20 | 58.15 | 57.60 | 57.40 |
| Spatial-3NN-Mean | 58.35 | 62.10 | 65.84 | 61.42 | 60.88 | 61.72 |
| Spatial-5NN-Mean | 59.62 | 64.85 | 67.90 | 64.18 | 63.45 | 64.00 |
| Morphology-5NN-Mean | 60.48 | 67.42 | 77.52 | 85.46 | 81.29 | 74.43 |
| NeighborMIF | 73.29 | 76.58 | 85.94 | 91.27 | 87.05 | 82.83 |
The results demonstrate that direct registration only achieves 55.30, while spatial neighbor transfer and morphological neighbor transfer improve to 64.00 and 74.43, respectively. Meanwhile, NeighborMIF reaches 82.83, outperforming the strongest simple transfer baseline by 8.40 percentage points. This indicates that although Reference mIF provides valuable molecular information, relying solely on registration or neighborhood averaging remains insufficient; NeighborMIF utilizes Reference molecular information more effectively through learnable cell matching and subsequent molecular feature reconstruction and fusion.
[Q2] How does a simple k-NN regressor compared to the learned matcher?
[A5] : Thank you for this question. We conducted a controlled comparison by replacing NeighborMIF's learnable Q/K Matcher with k-NN matching based on frozen H-optimus cell features, using . For each Target cell, the top Reference donor cells in the feature space were selected and fed into the same downstream network for prediction.
| Method | AFP | CK7 | ARG1 | CDX2 | SATB2 | Mean |
|---|---|---|---|---|---|---|
| NeighborMIF (Morphology-1NN) | 70.09 | 73.87 | 81.28 | 87.51 | 86.86 | 79.92 |
| NeighborMIF (Morphology-3NN) | 70.44 | 75.55 | 84.60 | 86.82 | 87.22 | 80.93 |
| NeighborMIF (Morphology-5NN) | 70.10 | 76.42 | 85.91 | 87.30 | 86.97 | 81.34 |
| NeighborMIF (Learned Matcher) | 73.29 | 76.58 | 85.94 | 91.27 | 87.05 | 82.83 |
The results show that NeighborMIF's Learned Matcher achieves an average Cell Pearson of 82.83 by learning task-relevant cell matching relationships, which is an improvement of 1.49 percentage points over the best frozen-feature k-NN, demonstrating that learnable matching can select donor cells more effectively.
Reply to Reviewer xZYR (Part1/2)
Thank you very much for your thoughtful feedback. We sincerely appreciate your positive acknowledgment of the contribution to our work, and we would like to address your concerns as follows:
[W1] The evaluation set is a very small private dataset, which limits the effectiveness and generalizability of the paper.
[A1] : We acknowledge that our dataset is not very large. Nevertheless, we have taken several measures to mitigate the impact of the limited sample size as much as possible. First, all experiments adopt mutually exclusive, case-level data splits to prevent data leakage, combined with bidirectional transfer and leave-one-case-out evaluation protocols for validation. Furthermore, we have added cross-fold standard deviations in the revised manuscript to reflect performance variations across different case splits (see the UNCERTAINTY ANALYSIS subsection in the Appendix). We agree that further validation on larger, multi-center cohorts with paired adjacent-section H&E–mIF measurements will be necessary in the future to thoroughly evaluate the broader generalizability of NeighborMIF. We will pursue this line of research in our future work.
[W2] How much of the method's effectiveness comes from the neighbors' signal?
[A2] : Reference mIF improves the average Cell Pearson by 4.37 percentage points over the matched H&E-only baseline (82.83 vs. 78.46; You may also refer to Appendix Table 5 for details). To further distinguish the contribution of Reference mIF itself from the additional benefit of using a physically adjacent section, we conducted an additional cross-sample Reference retrieval experiment (Retrieved Cross-sample Ref). Specifically, instead of using the physically adjacent section, we retrieved morphologically similar Reference patches from other samples based on the Target H&E and fed their corresponding measured mIF signals into the same transfer and fusion backend as NeighborMIF. The results are as follows:
The results show that NeighborMIF consistently outperforms H&E-only across all five proteins, achieving an average improvement of 4.37 percentage points. In comparison, cross-sample Reference retrieval provides only a modest improvement of 0.81 percentage points. This indicates that Reference mIF can provide additional molecular information for Target H&E, and physical adjacent sections provide more effective reference information compared to cross-sample references obtained via morphological retrieval.
[W3&Q3] How were those baselines retrained and compared?
[A3] : To ensure a fair comparison, all methods were evaluated on the same held-out test cases, cell instances, and ground-truth annotations as NeighborMIF, using the same Cell Pearson metric. Considering the differences between the two datasets and the availability of pretrained models, we adopted dataset-specific training strategies for the external baselines.
(1) In-house Serial-section Dataset
Since the protein panels in our in-house dataset differ from those used to train the original baseline models, we retrained or fine-tuned all external baselines using the same case-level fold splits as NeighborMIF. Specifically, MIPHEI-ViT was retrained using its publicly available implementation; ROSIE was fine-tuned from publicly available pretrained weights; HEMIT retained its original panel-joint training strategy; MUPAD was reproduced and trained following the original paper; and GigaTIME was adapted and fine-tuned using its public code and pretrained weights.
(2) ORION CRC Dataset
MIPHEI-ViT, ROSIE, and HEMIT have publicly released checkpoints trained using the complete ORION training data. Therefore, we directly used these checkpoints for inference on the same ORION test set, without additional retraining. MUPAD was reproduced and trained according to its original paper, while GigaTIME was adapted and fine-tuned on the official ORION training split using its public code and pretrained weights.