acceptodds
Under review as a conference paper at ICLR 2027

NeighborMIF: Reference-Guided Cross-Section Molecular Transfer for Multiplex Immunofluorescence Recovery

Abstract

In spatial multi-omics studies, different molecular modalities are often measured on separate tissue sections. Existing approaches mainly infer molecular expression from H&E morphology or learn cross-omics mappings, but their performance remains limited for protein markers that are weakly associated with morphology or other omics features. To address this, we propose NeighborMIF, a reference-guided framework for cross-section molecular information transfer. NeighborMIF performs cell-level matching to transfer experimentally measured protein signals from reference sections, reconstructs the transferred molecular representations in the spatial feature space of the target section, and integrates them with H&E features for missing mIF recovery. On the consecutive-section dataset, NeighborMIF achieves an average cell-level Pearson correlation of 82.83% (r × 100), outperforming the best external method by 6.11 percentage points. On ORION CRC under the cross-sample non-contiguous-reference setting, reference molecular information consistently improves the same-architecture H&E-only baseline across all 15 proteins. These results demonstrate that experimentally measured reference molecular signals provide effective complementary evidence beyond H&E morphology, offering a new computational pathway for cross-section molecular information transfer and spatial multi-omics integration.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
multiplex immunofluorescence, spatial multi-omics, cross-section molecular transfer, reference-guided prediction, computational pathology, cell-level matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_xZYROpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper focuses on the problems of multi-omics understanding. The main discovery is the study of missing mIF protein signals in a target tissue section. Instead of just using or predicting from HE morphology alone, this work targets the cross-encoding mappings. It proposes a third paradigm: transferring experimentally measured mIF signals from a reference section, usually a physically adjacent serial section of the tissue block. The method NeighborMIF first registers the target and reference HE WSIs, then builds cell-level morphological features by aggregating visual tokens of cell masks. Then it extracts per-cell measured mIF vectors from the reference section and transfers them to target cells via a learnable query key matcher.

Critical strengths
  1. This paper proposes a new well-motivated problem formulation that contains pathology images, mIF, and other information.
  2. Instead of just natively ensembling the information, the paper proposes a valid method to better encode different modality data.
  3. The experiment and evaluation design are well handled and managed. The reference bank for the cross-sample setting is restricted to the training split without leakage.
Critical weaknesses
  1. The evaluation set is a very small private dataset, which limits the effectiveness and generalizability of the paper.
  2. How much of the method's effectiveness comes from the neighbors' signal? The comparison is not very clear; an ablation study might be needed to justify the performance gain.
  3. The experimental setting and comparison design need more clarity. Are the external baselines trained on the private data? if so, more clarification and controls are needed for the baseline comparison.
Questions for authors
  1. As mentioned above, what if a naive baseline performs on the consecutive section benchmark, like directly transferring the registered reference mIF to the target cells?
  2. How does a simple k-NN regressor compared to the learned matcher?
  3. How were those baselines retrained and compared?
2 comments
AuthorsOpenReview ↗

Reply to Reviewer xZYR (Part1/2)

Thank you very much for your thoughtful feedback. We sincerely appreciate your positive acknowledgment of the contribution to our work, and we would like to address your concerns as follows:

[W1] The evaluation set is a very small private dataset, which limits the effectiveness and generalizability of the paper.

[A1] : We acknowledge that our dataset is not very large. Nevertheless, we have taken several measures to mitigate the impact of the limited sample size as much as possible. First, all experiments adopt mutually exclusive, case-level data splits to prevent data leakage, combined with bidirectional transfer and leave-one-case-out evaluation protocols for validation. Furthermore, we have added cross-fold standard deviations in the revised manuscript to reflect performance variations across different case splits (see the UNCERTAINTY ANALYSIS subsection in the Appendix). We agree that further validation on larger, multi-center cohorts with paired adjacent-section H&E–mIF measurements will be necessary in the future to thoroughly evaluate the broader generalizability of NeighborMIF. We will pursue this line of research in our future work.

[W2] How much of the method's effectiveness comes from the neighbors' signal?

[A2] : Reference mIF improves the average Cell Pearson by 4.37 percentage points over the matched H&E-only baseline (82.83 vs. 78.46; You may also refer to Appendix Table 5 for details). To further distinguish the contribution of Reference mIF itself from the additional benefit of using a physically adjacent section, we conducted an additional cross-sample Reference retrieval experiment (Retrieved Cross-sample Ref). Specifically, instead of using the physically adjacent section, we retrieved morphologically similar Reference patches from other samples based on the Target H&E and fed their corresponding measured mIF signals into the same transfer and fusion backend as NeighborMIF. The results are as follows:

MethodAFPCK7ARG1CDX2SATB2Mean
H&E-only65.1970.8980.0190.3185.8878.46
NeighborMIF (Retrieved Cross-sample Ref)68.2371.1580.6590.3485.9679.27
NeighborMIF (Physical Neighbor Ref)73.2976.5885.9491.2787.0582.83
Δ\Delta Cross-sample Ref − H&E-only+3.04+0.26+0.64+0.03+0.08+0.81
Δ\Delta Physical Neighbor Ref − Cross-sample Ref+5.06+5.43+5.29+0.93+1.09+3.56

The results show that NeighborMIF consistently outperforms H&E-only across all five proteins, achieving an average improvement of 4.37 percentage points. In comparison, cross-sample Reference retrieval provides only a modest improvement of 0.81 percentage points. This indicates that Reference mIF can provide additional molecular information for Target H&E, and physical adjacent sections provide more effective reference information compared to cross-sample references obtained via morphological retrieval.

[W3&Q3] How were those baselines retrained and compared?

[A3] : To ensure a fair comparison, all methods were evaluated on the same held-out test cases, cell instances, and ground-truth annotations as NeighborMIF, using the same Cell Pearson metric. Considering the differences between the two datasets and the availability of pretrained models, we adopted dataset-specific training strategies for the external baselines.

(1) In-house Serial-section Dataset

Since the protein panels in our in-house dataset differ from those used to train the original baseline models, we retrained or fine-tuned all external baselines using the same case-level fold splits as NeighborMIF. Specifically, MIPHEI-ViT was retrained using its publicly available implementation; ROSIE was fine-tuned from publicly available pretrained weights; HEMIT retained its original panel-joint training strategy; MUPAD was reproduced and trained following the original paper; and GigaTIME was adapted and fine-tuned using its public code and pretrained weights.

(2) ORION CRC Dataset

MIPHEI-ViT, ROSIE, and HEMIT have publicly released checkpoints trained using the complete ORION training data. Therefore, we directly used these checkpoints for inference on the same ORION test set, without additional retraining. MUPAD was reproduced and trained according to its original paper, while GigaTIME was adapted and fine-tuned on the official ORION training split using its public code and pretrained weights.

AuthorsOpenReview ↗

Reply to Reviewer xZYR (Part2/2)

[Q1] What if a naive baseline performs on the consecutive section benchmark, like directly transferring the registered reference mIF to the target cells?

[A4] : Thank you for this question. We added three types of baselines:

(1) Registered-mIF Copy: Directly assigns the Reference mIF at the registered spatial location to each Target cell.

(2) Spatial-kNN: Selects the nearest K=1,3,5K=1, 3, 5 Reference cells based on the spatial distance of cell centers after registration, and averages their mIF signals.

(3) Morphology-5NN-Mean: Selects the five most morphologically similar Reference cells based on cosine similarity between frozen H-optimus cell features and averages their measured mIF signals.

All these baselines directly transfer measured Reference mIF without NeighborMIF's learnable matching or subsequent molecular feature reconstruction and fusion.

MethodAFPCK7ARG1CDX2SATB2Mean
Registered-mIF Copy48.9856.8461.4553.9055.3455.30
Spatial-1NN55.1258.9257.2058.1557.6057.40
Spatial-3NN-Mean58.3562.1065.8461.4260.8861.72
Spatial-5NN-Mean59.6264.8567.9064.1863.4564.00
Morphology-5NN-Mean60.4867.4277.5285.4681.2974.43
NeighborMIF73.2976.5885.9491.2787.0582.83

The results demonstrate that direct registration only achieves 55.30, while spatial neighbor transfer and morphological neighbor transfer improve to 64.00 and 74.43, respectively. Meanwhile, NeighborMIF reaches 82.83, outperforming the strongest simple transfer baseline by 8.40 percentage points. This indicates that although Reference mIF provides valuable molecular information, relying solely on registration or neighborhood averaging remains insufficient; NeighborMIF utilizes Reference molecular information more effectively through learnable cell matching and subsequent molecular feature reconstruction and fusion.

[Q2] How does a simple k-NN regressor compared to the learned matcher?

[A5] : Thank you for this question. We conducted a controlled comparison by replacing NeighborMIF's learnable Q/K Matcher with k-NN matching based on frozen H-optimus cell features, using K=1,3,5K=1,3,5. For each Target cell, the top KK Reference donor cells in the feature space were selected and fed into the same downstream network for prediction.

MethodAFPCK7ARG1CDX2SATB2Mean
NeighborMIF (Morphology-1NN)70.0973.8781.2887.5186.8679.92
NeighborMIF (Morphology-3NN)70.4475.5584.6086.8287.2280.93
NeighborMIF (Morphology-5NN)70.1076.4285.9187.3086.9781.34
NeighborMIF (Learned Matcher)73.2976.5885.9491.2787.0582.83

The results show that NeighborMIF's Learned Matcher achieves an average Cell Pearson of 82.83 by learning task-relevant cell matching relationships, which is an improvement of 1.49 percentage points over the best frozen-feature k-NN, demonstrating that learnable matching can select donor cells more effectively.

Reviewer_dABdOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Biologists are interested in measuring multiple modalities on the same tissue section, but some modalities have incompatible acquisition protocols, hence they are measured on different sections of the tissue. This paper proposes a novel method to infer one expensive modality (multiplex immunofluorescence, mIF) from a cheap and common one (hematoxylin and eosin stain, H&E) by leveraging a neighboring section where both have been measured. The authors' motivation is that mIF values could then be recovered for sections that are also used for other modalities incompatible with mIF. The proposed method computes a weighted average of the mIF values for cells in the target H&E image, where weights result from cross-attention between features of cells from the reference and the target H&E images. A UNet then computes an estimation of the target mIF image from the previous result and features of the reference H&E image. The authors report state-of-the-art quantitative results and good qualitative results on an in-house dataset (kept private). They also adapted their method to replace neighboring references with a retrieval pipeline so that they could evaluate on a public dataset on which they obtained strong performance.

Neither the code nor a reproducibility statement is provided. The mandatory AI usage statement is also missing.

Critical strengths

The main idea is sound: an experimentally measured reference logically provides additional evidence that cannot be inferred from the H&E image alone. I find their model architecture relevant to the task: they propose a fusion module added as a correction on top of an H&E-only model. Additionally, building an in-house dataset with adjacent sections to test the exact setting of the method is valuable work. The authors compare with recent baselines on this private dataset, but also on a public dataset with a consistent adaptation of their method.

Critical weaknesses

Framing of the contribution. The introduction (L052) categorizes existing methods into (a) H&E-to-mIF prediction, where the target modality is learned in the weights, and (b) (H&E + another modality)-to-mIF prediction, where the other modality can be measured on the same section. The proposed method is a third option (c): (H&E + mIF)-to-mIF prediction, but with the mIF modality measured on another section. The authors criticize (b) because it "may introduce additional biases" (L071), but (c) brings its own bias too: adjacent sections do not contain the same cells. Likewise, the argument that "shared tissue architecture and local cellular environments provide valuable structural constraints" (L079) applies even more to (b), where the other modality can be measured on the same section.

I think the introduction would be more faithful if it were formulated as a trade-off between two axes: (b) has the same section but an indirect modality, while (c) gets the right modality but an adjacent section. Which bias is preferable depends on the data: (c) should work well where the local context is redundant, while (b) may be better for rare / isolated cells absent from the adjacent section. I think such a formulation does not diminish the authors' work, but rather positions it more precisely in the literature. Additionally, (b) and (c) are orthogonal, and thus could be combined rather than opposed (Q1).

Missing baselines. All five baselines in Tables 1 and 2 are type (a) methods that only see the target H&E. By contrast, the authors' method receives more information through the additional mIF input, which gives it a significant edge. I believe additional comparisons with type (b) methods should be added, to highlight the key benefit brought by mIF rather than by any alternative modality. It would also allow testing the modality vs section trade-off discussed above. Regarding type (b) methods, the authors say (L072) "spatial transcriptomics combined with histological images has been used to predict spatial protein expression" but no such method is cited. I expect the authors to add the missing references and run the missing comparisons. (Q2)

Unsupported architecture choices. The method combines many modules, but few choices are supported by experiments. For example, the candidate pool Ai\mathcal{A}_i is composed of spatially close cells, plus similar ones in a larger region. I found this choice only lightly motivated ("the latter compensates for missed candidates", L247), and it is not ablated (Q3). More generally, the matcher proposed by the authors combines a learned cross-attention layer with a hand-crafted candidate pool. It sits between two simpler, more natural alternatives: first, a fully learned architecture, where attention is performed over all cells, without Ai\mathcal{A}_i, and second, a non-learned pooling, for example the mean of the mIF features after a k-NN over the feature vectors for cells in Ai\mathcal{A}_i. Showing that the authors' hybrid design beats both would justify its complexity (Q4).

Nature of the reference frame. The consecutive-section and non-contiguous-section settings (respectively Table 1 and Table 2) are evaluated on different datasets (respectively the in-house dataset and ORION), so the benefit of adjacency itself is never isolated. Running the non-contiguous-section setting (i.e. via cross-sample retrieval) on the in-house dataset would show whether an adjacent reference really matters, or whether references from a more general bank are enough (Q5). Both outcomes would be informative, and better performance with an adjacent reference would strongly support the authors' thesis.

Missing result uncertainties. The authors do not report uncertainty for their results in any table of the paper. Given that the in-house dataset used a leave-one-out protocol, it could directly provide standard deviations across folds. For ORION, several training seeds could be used. Right now, it is unclear whether gains (such as +0.80 over MIPHEI-ViT on CDX2 in Table 1) or even rankings (NeighborMIF vs MIPHEI-ViT in Table 2) are due to noise (Q6).

Minor comments

  • Table 5 contains a result that proves the paper's main thesis: adding the mIF modality as an additional input improves the results. I almost missed it when reading the paper for the first time. It should be part of the main paper, maybe part of Table 1 if space is an issue.
  • Figure 2: it took me a while to understand that YmIFrY^r_\text{mIF} enters at the bottom right and flows from right to left, while the output flows the other way at the top right. Please make it explicit or reorganize the figure. The violet cross symbol is also obscure and should be defined.
  • Figure 3 could be bigger: details are hard to see on a printed version, and there is free space on both sides.
  • Do the authors plan to release the code and the in-house dataset? Please add a reproducibility statement.
  • The related work could be expanded, especially for type (b) and retrieval-based methods.
  • The mandatory AI usage statement is missing: the authors are requested to add it.

Conclusion. Overall, I believe the idea is sound and the architecture coherent, but the experiments do not yet support each design choice. Additionally, Table 5 shows that an extra mIF input complements the H&E modality, but not that a neighboring section is preferable to any other. Based on the submitted version, I am leaning toward acceptance, although I will adjust my score up or down depending on the authors' answers to my questions and on how they address the requested experiments.

Questions for authors
  • Q1. Could you discuss the trade-off between (b) and (c), and whether they could be combined?
  • Q2. Which type (b) methods are you referring to in the introduction? Could you compare your method with one of them?
  • Q3. Could you report performance for different candidate pools Ai\mathcal{A}_i ? Spatial-only pool, similarity-only pool, and no restriction (i.e. global cross-attention) would be informative.
  • Q4. Could you compare the learned matcher to a non-learned alternative, for example k-NN on frozen features?
  • Q5. Could you run the cross-sample retrieval evaluation on the in-house dataset?
  • Q6. Could you report standard deviations across folds (in-house) and confidence intervals (ORION)?
  • Q7. How does the model behave when no good match exists in the reference?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer dABd (Part1/3)

We sincerely thank you for the time and care you devoted to reading our manuscript and for your insightful feedback. We have carefully considered your comments and respond to each point below.

[Q1] Could you discuss the trade-off between (b) and (c), and whether they could be combined?

[A1] : Thank you for raising this valuable suggestion. We agree that (b) and (c) each have their own strengths and are suitable for different research scenarios. The categorization in the original manuscript is intended to distinguish the molecular modalities used by different methods and their measurement locations, rather than implying that the two categories are mutually exclusive.

Both types of methods have distinct characteristics: (c) is more advantageous for regions where local tissue architecture and cellular microenvironments are relatively consistent across adjacent sections; whereas (b) may be more suitable for rare or isolated cells that are absent in adjacent sections. Our choice of (c) also stems from practical application considerations: compared with high-throughput spatial molecular measurement technologies such as spatial transcriptomics, mIF offers relatively lower cost and greater accessibility in many experimental settings.

We also agree that the two approaches can be combined: jointly utilizing the H&E of the Target section, same-section non-mIF modality information, and measured mIF from the adjacent Reference section allows same-section molecular information and adjacent-section direct protein evidence to complement each other. This complementary fusion represents a promising research direction for NeighborMIF to explore further.

[Q2] Which type (b) methods are you referring to in the introduction? Could you compare your method with one of them?

[A2-1] : Thank you for this question. We have added citations to representative methods such as SPRINT in the revised manuscript (see Line 72 of the paper).

[A2-2] : Regarding the experimental comparison, we sincerely apologize that we are unable to provide a direct comparison with SPRINT and other related methods in this revision. Both our in-house dataset and ORION CRC lack the spatial transcriptomics data required by these methods. Substituting alternative features for the missing inputs would change their original input configurations and experimental settings, making a fair and direct comparison infeasible. We regret this limitation and appreciate the reviewer's understanding.

[Q3] Could you report performance for different candidate pools ? Spatial-only pool, similarity-only pool, and no restriction (i.e. global cross-attention) would be informative.

[A3] : We are more than happy to provide this evaluation. We have added three candidate-pool ablations:

(1) NeighborMIF (Spatial-only): Candidates are selected based on spatial proximity.

(2) NeighborMIF (Similarity-only): Candidates are selected based on morphological similarity using frozen H-optimus features.

(3) NeighborMIF (Global): An unrestricted candidate pool that includes all valid Reference cells for matching;

The full NeighborMIF (Spatial ∪ Similarity) combines spatially nearby candidates with morphologically similar candidates. All configurations use the same Learned Q/K Matcher to select the Top-5 donors, while keeping the remaining network architecture and training settings unchanged.

MethodAFPCK7ARG1CDX2SATB2Average
NeighborMIF (Spatial-only)68.2775.6482.5285.4081.9778.76
NeighborMIF (Similarity-only)72.5176.0985.4887.1687.2081.69
NeighborMIF (Global)73.0476.5086.7590.8187.2482.87
NeighborMIF (Spatial ∪ Similarity)73.2976.5885.9491.2787.0582.83

The results demonstrate clear complementarity between spatial proximity and morphological similarity. The complete NeighborMIF (Spatial ∪ Similarity) achieves an average Cell Pearson of 82.83, outperforming Spatial-only and Similarity-only by 4.07 and 1.14 percentage points, respectively, and achieving the best results on AFP, CK7, and CDX2. Its average performance is nearly identical to that of NeighborMIF (Global) (82.83 vs. 82.87). Compared to the unrestricted candidate pool, the combined candidate pool significantly reduces matching computation with virtually no loss in prediction accuracy, thereby accelerating cell matching—a speed advantage that becomes increasingly pronounced as the Reference region expands and the number of cells grows.

AuthorsOpenReview ↗

Reply to Reviewer dABd (Part2/3)

[Q4] Could you compare the learned matcher to a non-learned alternative, for example k-NN on frozen features?

[A4] : We would be glad to do so. We conducted a controlled comparison by replacing NeighborMIF's learnable Q/K Matcher with k-NN matching based on frozen H-optimus cell features, using K=1,3,5K=1,3,5. For each Target cell, the top KK Reference donor cells in the feature space were selected and fed into the same downstream network for prediction.

MethodAFPCK7ARG1CDX2SATB2Mean
NeighborMIF (Morphology-1NN)70.0973.8781.2887.5186.8679.92
NeighborMIF (Morphology-3NN)70.4475.5584.6086.8287.2280.93
NeighborMIF (Morphology-5NN)70.1076.4285.9187.3086.9781.34
NeighborMIF (Learned Matcher)73.2976.5885.9491.2787.0582.83

The results show that NeighborMIF's Learned Matcher achieves an average Cell Pearson of 82.83 by learning task-relevant cell matching relationships, which is an improvement of 1.49 percentage points over the best frozen-feature k-NN, demonstrating that learnable matching can select donor cells more effectively.

[Q5] Could you run the cross-sample retrieval evaluation on the in-house dataset?

[A5] :That is a great point, and we have now run this evaluation on the in-house dataset . Specifically, we replaced the physical adjacent-section Reference with a Reference retrieved from other samples, and compared it against both H&E-only and the complete NeighborMIF.

MethodAFPCK7ARG1CDX2SATB2Average
H&E-only65.1970.8980.0190.3185.8878.46
NeighborMIF (Retrieved Cross-sample Ref)68.2371.1580.6590.3485.9679.27
NeighborMIF (Physical Neighbor Ref)73.2976.5885.9491.2787.0582.83
Δ\Delta Cross-sample Ref − H&E-only+3.04+0.26+0.64+0.03+0.08+0.81
Δ\Delta Physical Neighbor − Cross-sample Ref+5.06+5.43+5.29+0.93+1.09+3.56

Cross-sample Reference retrieval provides an average improvement of only 0.81 percentage points, whereas using a physically adjacent Reference section yields a further improvement of 3.56 percentage points, reaching an average Cell Pearson of 82.83. These results indicate that although morphology-based cross-sample retrieval can provide some useful molecular information, physically adjacent sections offer more informative Reference evidence.

[Q6] Could you report standard deviations across folds (in-house) and confidence intervals (ORION)?

[A6] : Thank you for this suggestion. On the in-house dataset, all methods were evaluated using three case-disjoint folds. NeighborMIF achieved a five-protein mean Cell Pearson of 83.67 ± 2.91, compared with 78.50 ± 2.95 for the strongest external baseline, MIPHEI-ViT. On ORION CRC, NeighborMIF and GigaTIME were independently trained using three random seeds under the same patient-level split. Their 15-protein mean Cell Pearson values were 49.12 ± 1.50 with a two-sided 95% confidence interval of [45.40, 52.85], and 44.09 ± 0.26 [43.44, 44.73], respectively. MIPHEI-ViT was evaluated using its publicly released ORION checkpoint and was therefore reported as a single fixed result rather than a multi-seed estimate. Detailed fold-wise, seed-wise, and per-protein results are provided in the Uncertainty Analysis subsection of the revised Appendix.

[Q7] How does the model behave when no good match exists in the reference?

[A7] : Thank you for raising this question. Currently, NeighborMIF cannot explicitly identify or reject cases where none of the Reference candidate cells match. In such scenarios, the model still selects the Top-5 candidate donors and aggregates their molecular signals. We would like to clarify that NeighborMIF is designed for information transfer across adjacent tissue sections, where the Reference originates from the spatially corresponding region on an adjacent section of the Query, rather than arbitrarily selected, unrelated patches. Moreover, NeighborMIF does not require strict one-to-one cellular correspondence; Instead, it retrieves and aggregates molecular information from multiple donor cells via the Learned Matcher. Explicitly handling cases where no reliable match exists represents an important limitation of the current framework. In future work, we plan to explore confidence-based rejection mechanisms—such as introducing similarity thresholds—and incorporate missing or mismatched Reference scenarios during training to further improve model robustness.

AuthorsOpenReview ↗

Reply to Reviewer dABd (Part3/3)

Minor Comments

Thank you for taking the time to carefully review our manuscript and provide these helpful minor comments. We address each point below.

1. Moving Table 5 to the main text.

To prevent large-scale restructuring at this point, we currently maintain the existing table structure, while ensuring that Table 5 and the in-house cross-sample retrieval results are integrated into the main text in the final manuscript.

2. Clarification of data-flow directions and the purple plus sign in Figure 2.

To make the framework easier to understand, the caption of Figure 2 has been expanded to explain the directions of the two data flows and the meaning of the purple cross (see Page 4 of the revised manuscript). The added text reads: In the figure, the measured Reference mIF enters from the bottom right and flows from right to left for cell-level molecular transfer, whereas the Target H&E prediction stream proceeds from left to right and produces the predicted Target mIF at the top right. The purple cross denotes cell-mask-guided mean pooling of the measured Reference mIF signal to obtain the protein-expression vector of each Reference cell.

3. Enlarging Figure 3.

We have cropped the blank margins on both sides of the figure and enlarged the figure and its text to improve the readability of its details (see Page 7 of the revised manuscript).

4. Release the code and the in-house dataset? Adding a reproducibility statement.

Yes, upon publication, we will publicly release the NeighborMIF source code and the de-identified in-house H&E–mIF dataset. The release will include experimental configurations, case-level data splits, and complete preprocessing, training, inference, and evaluation scripts to facilitate the reproduction of all experiments reported in this paper. The modeling principles and architectural components of NeighborMIF are described in detail in the Methods section, with comprehensive quantitative results and ablation studies provided to support the verification of our findings. We have also added a reproducibility statement to the revised manuscript (see Page 10).

5. Expanding the Related Work section.

We have added representative type (b) methods to the Related Work section (see the blue-highlighted text on Page 3 of the revised manuscript).

6. Adding an AI use statement.

We have added an AI Use Statement to the revised manuscript (see Page 10 of the revised manuscript).

I thank the authors for their substantive rebuttal, which answers almost all the points I raised. I'm surprised to see that relatively few new experiments made it into the revised version: I believe the paper would benefit from them, mainly Q3 and Q4. I also expect the authors to discuss the trade-off raised in Q1 and the limitation raised in Q7 in the main text of the camera-ready version, should the paper be accepted. I am happy to raise my score.

Reviewer_rxBqOpenReview ↗
Rating
1(−1)
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

NeighborMIF is a reference-guided framework that predicts missing protein expression in target tissue sections by transferring measured mIF signals from reference sections through cell-level morphological matching and molecular feature fusion. NeighborMIF is evaluated on serial-section and non-neighboring-reference settings.

Critical strengths
  1. The broader biological problem of spatial multi-omics integration across tissue sections is important and promising, with significant potential for advancing our understanding of tissue organization and molecular heterogeneity.
Critical weaknesses
  1. Oversimplified problem formulation and overstated practical significance: The paper is motivated by the challenge of spatial multi-omics integration across tissue sections, but the actual implementation is limited to predicting protein expression in a target section using the same proteins already measured in an adjacent reference section. This represents a substantial simplification of the original biological problem, without demonstrating integration across different molecular modalities. Moreover, since the target proteins have already been measured in a neighboring section, the practical value of reconstructing their expression at corresponding spatial locations remains unclear. The authors do not demonstrate what additional biological insights or downstream benefits this reconstruction provides compared with directly registering and analyzing the measured reference mIF. Therefore, the broader claims regarding spatial multi-omics integration and practical utility appear overstated.

  2. Unfair baseline comparison and insufficient evaluation against reference-based methods: The main comparisons in Tables 1–2 are against H&E-based protein prediction methods, but NeighborMIF additionally uses measured reference mIF at inference. This is particularly important in the consecutive-section setting, where the reference comes from the same tissue block. So the reported improvements over external baselines partly reflect its access to additional molecular information rather than the proposed transfer mechanism. Although Tables 5–6 show gains over the matched H&E-only architecture, they do not show superiority over alternative methods using the same reference information. I recommend that the paper include reference-based baselines, such as registration-based mIF transfer, spatial or morphology-based nearest-neighbor aggregation, and simpler molecular fusion strategies. Comparisons with applicable spatial multi-omics integration or missing-modality methods, e.g., SpatialEx, would also strengthen the evaluation.

Questions for authors
  1. Limited methodological novelty and unclear necessity of the proposed architecture. The proposed framework primarily combines existing components, including H-optimus-0, cell mask pooling, Query–Key matching, Top-K weighted aggregation, residual fusion, and UNet3+. The paper does not demonstrate whether the complex learned matching and molecular feature reconstruction offer meaningful advantages over simpler approaches such as reference interpolation or cell-level regression. Furthermore, in the ORION setting, reference molecular information is retrieved from labeled training samples, making the task more closely resemble retrieval-augmented prediction rather than cross-section molecular transfer. Overall, the methodological novelty appears limited, and the necessity of the proposed architecture remains insufficiently justified.

  2. Insufficient validation of the proposed cell-level molecular transfer mechanism. Cell-level matching is arguably the most important methodological contribution of this work, yet the matching module is trained solely through the final mIF prediction loss without direct correspondence supervision. The paper does not evaluate whether the selected donor cells represent biologically meaningful matches, whether prediction performance depends on accurate spatial correspondence, or whether the model captures protein-state variation within the same cell type. Comparisons with simple regional averaging or cell-type-conditioned protein expression are also missing. Moreover, the model first averages mIF signals from five donor cells and then aggregates transferred signals into a coarse bottleneck feature grid, potentially losing cell-specific molecular information. Therefore, high Pearson correlations alone do not establish that the model has learned meaningful cell-level molecular transfer.

  3. Lack of direct evidence for morphology-independent protein prediction. A central motivation of the paper is that certain protein markers cannot be reliably predicted from H&E morphology alone. However, the current evaluation does not directly demonstrate that NeighborMIF addresses this challenge. High overall Pearson correlations could largely reflect differences in protein expression across cell types, which may already be distinguishable from H&E images, rather than the recovery of molecular variation within the same cell type. The paper lacks within-cell-type evaluation or analysis of cell-type-independent protein expression variation, making it unclear whether the proposed method truly improves the prediction of molecular states that are not readily inferred from morphology.

9 comments
AuthorsOpenReview ↗

Reply to Reviewer rxBq (Part1/5)

Thank you for taking the time to provide such detailed and helpful feedback. We have carefully considered your feedback and address your five key points below.

[W1]

We clarify the scope of this work, the motivation for reconstructing target-section mIF, and the empirical improvements of our method over direct registration and simple reference-transfer approaches from the following three perspectives.

1. Scope of the study.

[A1-1] : We apologize for any confusion caused by the way the scope and broader implications of our work were presented in the manuscript. We appreciate the opportunity to further clarify this point. NeighborMIF is primarily developed as a reference-guided framework for cross-section molecular information transfer. Specifically, it leverages experimentally measured molecular information from a Reference section to recover missing protein expression signals in a Target section. This central objective is described in the Introduction (page 2, lines 84–89 of the revised manuscript). More broadly, we envision that this reference-guided transfer framework could be extended to support spatial multi-omics integration across different molecular modalities. This potential direction is reflected in the Abstract (page 1, lines 26–28) and Conclusion (page 9, lines 485–487). The extension to broader spatial omics modalities is also identified as an important direction for future research (Conclusion, lines 488–490). We hope this distinction helps clarify the relationship between the present contribution and its broader potential: this work establishes a reference-guided approach to cross-section molecular information transfer, which may serve as a foundation for future research on more comprehensive spatial multi-omics integration.

2. Why is mIF reconstruction in the Target section necessary?

[A1-2] : One of our motivations is that, in practice, incompatible experimental workflows often prevent measuring multiple modalities on the same section, so researchers measure different modalities on serial sections and integrate them computationally (Introduction, lines 44–50). NeighborMIF transfers measured protein information into the cellular and spatial coordinate system of the Target section, enabling subsequent cross-modality integration.

3. Comparison with direct registration and simple Reference-transfer methods.

[A1-3] : To evaluate whether direct registration and simple Reference transfer are sufficient, we introduced three baseline methods:

(1) Registered-mIF Copy : Directly assigns the Reference mIF at the registered spatial location to each Target cell.

(2) Spatial-kNN : Selects the nearest K=1,3,5K=1, 3, 5 Reference cells based on the spatial distance of cell centers after registration, and averages their mIF signals.

(3) Morphology-5NN-Mean : Selects the five most morphologically similar Reference cells based on cosine similarity between frozen H-optimus cell features and averages their measured mIF signals.

All these baselines directly transfer measured Reference mIF without NeighborMIF's learnable matching or subsequent molecular feature reconstruction and fusion.

MethodAFPCK7ARG1CDX2SATB2Average
Registered-mIF Copy48.9856.8461.4553.9055.3455.30
Spatial-1NN55.1258.9257.2058.1557.6057.40
Spatial-3NN-Mean58.3562.1065.8461.4260.8861.72
Spatial-5NN-Mean59.6264.8567.9064.1863.4564.00
Morphology-5NN-Mean60.4867.4277.5285.4681.2974.43
NeighborMIF73.2976.5885.9487.5587.0582.08

Registered-mIF Copy achieves a mean Cell Pearson of 55.30, while the strongest spatial and morphology-based transfer baselines achieve 64.00 and 74.43, respectively. NeighborMIF reaches 82.08, outperforming the strongest simple-transfer baseline by 7.65 percentage points. These results show that direct registration or nearest-neighbor averaging cannot fully exploit the Reference mIF, whereas learned matching and subsequent molecular reconstruction and fusion provide substantial additional gains.

AuthorsOpenReview ↗

Reply to Reviewer rxBq (Part2/5)

[W2]

1. Reference-based baseline comparisons.

[A2-1] : Regarding the direct registration, spatial neighbor, and morphological neighbor transfer experiments , we have provided the full results and analysis in Part 3 of W1 above.

2. Comparison with simple molecular fusion strategies.

[A2-2] : The original manuscript already compares DirectFusion with the full ResidualFusion strategy (as shown in Table 3 of the manuscript). Following the reviewer's suggestion, we further introduced two simpler fusion baselines:

(1) AddFusion (new): Directly adds the channel-aligned H&E features (H) and reconstructed molecular features (M) element-wise, without introducing any learnable fusion layer.

(2) Linear-1×1 Fusion (new): Concatenates H and M along the channel dimension and applies a single 1×1 convolution for linear projection, without nonlinear fusion blocks or residual connections.

Both new experiments use the exact same model architecture as DirectFusion and ResidualFusion, differing only in the fusion mechanism between H&E and mIF features.

MethodAFPCK7ARG1CDX2SATB2Average
AddFusion (new)69.8571.9084.5684.1579.4077.97
Linear-1×1 Fusion (new)71.2574.2084.1686.6882.8079.82
DirectFusion73.3573.7486.0087.5481.6580.46
ResidualFusion (NeighborMIF)73.2976.5885.9487.5587.0582.08

The results show that NeighborMIF achieves a mean Cell Pearson of 82.08, outperforming AddFusion, Linear-1×1 Fusion, and DirectFusion by 4.11, 2.26, and 1.62 percentage points, respectively. These findings demonstrate that residual fusion provides a more effective way to integrate Reference molecular information.

3. Comparison with SpatialEx.

[A2-3] : We gladly accept this suggestion and have additionally conducted experiments with SpatialEx. The core SpatialEx model uses H&E-derived cellular morphological features and spatial relationships to predict cell-level omics expression. Since it does not require RNA or Reference mIF at inference time, it can be adapted to our H&E-to-mIF prediction task. We did not use SpatialEx+, which requires additional omics information. The results are shown below:

MethodAFPCK7ARG1CDX2SATB2Average
SpatialEx32.1164.9761.9584.2679.7564.61
H&E-only65.1970.8980.0190.3185.8878.46
NeighborMIF73.2976.5885.9487.5587.0582.08

NeighborMIF achieves a mean Cell Pearson of 82.08, outperforming SpatialEx (64.61) and the matched H&E-only model (77.86).

AuthorsOpenReview ↗

Reply to Reviewer rxBq (Part3/5)

[Q1]

We thank the reviewer for questioning the necessity of the proposed architecture. Within the limited rebuttal period, we prioritized additional ablation experiments targeting the two core components explicitly highlighted by the reviewer: learnable cell matching and subsequent molecular feature reconstruction and fusion. We further compared NeighborMIF with reference interpolation and simple cell-level regression baselines.

1. Necessity of the Learned Matcher.

[A3-1] : We replaced the learnable Q/K Matcher in NeighborMIF with a k-NN matching approach based on frozen H-optimus cell features, setting K=1,3,5K=1, 3, 5 respectively. For each Target cell, the nearest KK Reference donor cells in the feature space were selected and fed into the same downstream network for prediction.

MatcherAFPCK7ARG1CDX2SATB2Average
NeighborMIF (Morphology-1NN)70.0973.8781.2887.5186.8679.92
NeighborMIF (Morphology-3NN)70.4475.5584.6086.8287.2280.93
NeighborMIF (Morphology-5NN)70.1076.4285.9187.3086.9781.34
NeighborMIF (Learned Matcher)73.2976.5885.9487.5587.0582.08

The results show that the Learned Matcher achieves a mean Cell Pearson of 82.08, outperforming the best frozen-feature morphology k-NN alternative by 0.74 percentage points.This indicates that learnable matching can more effectively select donor cells by optimizing cell-level retrieval relationships according to the downstream molecular prediction objective.

2. Necessity of molecular feature reconstruction and fusion.

[A3-2] : We fixed the Top-5 Reference donor cells selected by the Learned Matcher and altered only the subsequent processing pipeline: Direct mIF Mean takes an unweighted average of the donors' measured mIF signals directly as the Target cell prediction, whereas Full Backend employs NeighborMIF's complete pipeline, including weighted molecular aggregation, molecular feature map reconstruction, ResidualFusion, and UNet3+ decoding. The results are shown below:

MethodAFPCK7ARG1CDX2SATB2Average
Learned Matcher + Direct mIF Mean61.8269.2278.5186.1482.8975.72
Learned Matcher + Full Backend (NeighborMIF)73.2976.5885.9487.5587.0582.08

Direct donor-mIF averaging achieves a mean Cell Pearson of 75.72, whereas the complete NeighborMIF reaches 82.08, providing an additional improvement of 6.36 percentage points. This indicates that donor retrieval and direct mIF transfer alone are insufficient; further molecular feature reconstruction, combined with Target H&E fusion and decoding, is essential for significantly boosting prediction performance.

3. Comparison with reference interpolation and simple cell-level regression.

[A3-3] : We compared two reference interpolation methods: Registered-mIF Copy directly assigns the Reference mIF at the registered position to the Target cell, while Spatial-5NN-Mean selects the 5 spatially nearest Reference cells after registration and averages their mIF signals equally. Their average Cell Pearson scores are 55.30 and 64.00, respectively (see Part 3 of W1). We further constructed two H&E-only regression baselines, both taking the Target cell's H-optimus descriptor as input to predict mIF expression protein-by-protein: Cell-Ridge-H&E-only uses Ridge linear regression, and Cell-MLP-H&E-only uses a single-hidden-layer MLP. Neither method utilizes Reference mIF or subsequent molecular feature reconstruction and fusion modules.

MethodAFPCK7ARG1CDX2SATB2Average
Cell-Ridge-H&E-only8.2262.3767.8674.8972.2557.12
Cell-MLP-H&E-only13.6664.9068.4673.4272.9558.68
NeighborMIF73.2976.5885.9487.5587.0582.08

Results show that NeighborMIF achieves an average Cell Pearson of 82.83, outperforming Spatial-5NN-Mean and Cell-MLP by 18.08 and 23.40 points, respectively. This demonstrates that naive spatial interpolation or H&E-only regression is insufficient, whereas NeighborMIF successfully recovers target protein expression via learnable donor retrieval, feature map reconstruction, and residual fusion.

4. Task setting of the ORION experiments.

[A3-4] : Our experiments on the ORION dataset are designed to further validate NeighborMIF's reference-guided mechanism by testing how well measured mIF information retrieved from other samples can assist target protein prediction when paired adjacent sections are unavailable. Thus, the ORION experiments evaluate NeighborMIF's retrieval-augmented protein prediction capability under non-adjacent conditions, complementing the physical adjacent-section molecular transfer evaluated on our in-house dataset.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.