acceptodds
Under review as a conference paper at ICLR 2027

Fine-Grained and Reliable Cell Classification from Pathology Images

15 views

Abstract

Molecular assays such as spatial transcriptomics facilitate fine-grained identification of cell types for analyzing tumor microenvironments. However, they are expensive to acquire in routine clinical practice compared to hematoxylin and eosin (H&E) images. With molecular guidance, such images can distinguish cell lineages, yet often fail to resolve closely related subtypes. To overcome this, we introduce **CellBridge**, a multi-modal contrastive learning framework pretrained on 16M cells with paired H&E and spatial profiles, for transferring molecular information into morphological representations. CellBridge further incorporates a Cell-Ontology-aware conformal prediction module that yields, for each cell, leaf-level labels where morphology suffices and coarser ancestor labels where molecular data is needed. Across in-domain and out-of-domain benchmarks, CellBridge outperforms leading pathology foundation models in macro F1 by up to 16 percentage points, with the largest gains on the most fine-grained label spaces. It consistently achieves smaller prediction sets, while reducing class-wise coverage gaps. We envision CellBridge as a step toward extracting cellular information that currently requires costly molecular assays from routinely acquired pathology images, while explicitly identifying where such assays remain necessary.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
histopathology, spatial transcriptomics, multi-modal learning, conformal prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_5wkROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CellBridge, which uses paired H&E images and spatial transcriptomic profiles to learn cell-level image representations. It combines conformal prediction with the Cell Ontology to return fine-grained labels when supported by morphology and broader labels otherwise.

Critical strengths

The method performs consistently well across multiple external datasets, and its Cell-Ontology-Fallback design provides a practical way to return broader cell labels when H&E morphology does not support a fine-grained prediction.

Critical weaknesses

The claim that low reliability scores identify incorrect predictions is supported mainly by qualitative case studies. A quantitative analysis across the test sets would strengthen this claim.

Questions for authors

Can the authors quantitatively evaluate whether low reliability scores consistently identify incorrect predictions across the test set?

3 comments
AuthorsOpenReview ↗

Our Rebuttal to Reviewer 5wkR (W1 & Q1)

Thank you for reviewing the paper. We appreciate that you find the empirical performance strong and our COF module practical. We address your remaining comments below, which we hope will help clarify and improve your evaluation and confidence in our paper’s contributions.

W1 & Q1: Quantitative evaluation of reliability scores

We thank the reviewer for this comment. In addition to the case studies wherein misclassified cells receive low reliability scores across eight expert-reviewed regions (all five errors in Figure 8 receive near-zero reliability), we conducted two quantitative analyses:

  • Correlation of per-phenotype reliability and error: For our external datasets, we plot reliability scores (per cell type) against classification errors in Figure 5b of the revised manuscript. We observe a clear negative trend, which we demonstrate with the Pearson correlation coefficient in the table below.
DatasetPearson r
NuCLS−0.745
PanNuke−0.951
Lizard−0.825
PhenoCell−0.613
  • Classification with rejection: We progressively rejected cells in ascending order of reliability and measured classification error among the retained cells. Figure 5c in the revised manuscript, summarized in the table below, shows that error decreases monotonically as more cells are rejected across all datasets. This indicates that higher reliability scores are associated with a lower likelihood of misclassification.
Dataset100% retained80% retained60% retained40% retained20% retained
HEST27.5624.5720.2913.459.35
NuCLS21.8919.5912.173.212.88
PanNuke16.7610.598.608.608.60
Lizard14.709.193.113.113.11
PhenoCell49.1346.6743.1436.8421.17

This is consistent with underlying biology. For example, separating discrete labels such as T vs. B vs. NK cells based on their small round shapes identified from H&E is less reliable in the absence of corresponding molecular markers such as CD4+ and CD8+ [1]. That is when COF returns the most specific supported ancestor, such as lymphocyte.

References

[1] Sheibani, K., et al. (1988), Variability in interpretation of immunohistologic findings in lymphoproliferative disorders by hematopathologists. A comprehensive statistical analysis of interobserver performance. Cancer, 62: 657-664.

We thank Reviewer 5wkR once again for the time and effort they put into our work. As the discussion period approaches its end, we would kindly like to ask if there are any further comments or concerns that we could address.

Thank you for the authors' response. I decide to keep my original score.

Reviewer_DESdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CellBridge, a framework for cell phenotyping in pathology H&E images. The method first pre-trains a contrastive CLIP model with paired H&E image + spatial single-cell transcriptomics (ST). The H&E encoder uses UNI2-h FM, a LoRA adapter and a projection head, while the ST encoder uses TERRA FM. The second part of the method provides fallback broader classifications when the classifier is uncertain. It uses a Cell Ontology and calibrated conformal prediction. The experimental part compares CellBridge to 6 SOTA FM on HEST-1k (14 held-out tissue sections) and 4 external datasets. Results show strong improvements over SOTA (avg 7 pct points).

Critical strengths
  • fine-grained phenotyping from routine H&E sections is an important problem and the basis for many downstream tasks. Yet it remains very challenging and this paper addresses some of these challenges.
  • the method efficiently trains a CLIP model on a large cell-level corpus of 16M pairs (H&E cell crops + cell transcriptomics), leveraging pre-trained foundation models (UNI2-h, TERRA) and LoRA fine-tuning.
  • the results are impressive with significant deltas over SOTA (e.g. +16pts on NUCLS)
  • in addition to linear probing (training a linear probe), knn probing is also used to evaluate the learned representation, showing superior clustering of CellBridge embeddings compared to SOTA
  • experimental breadth is strong: includes held-out HEST sections, multiple external datasets, six foundation-model baselines. The paper also reports both balanced accuracy and AUROC.
  • using the Cell Ontology to provide a fallback to common cell-line ancestor is biologically meaningful, providing more interpretable results.
Critical weaknesses
  • the novelty is somewhat limited, as the approach is a combination of existing methods and foundation models. The authors should better stress what are the method's original contributions.
  • one key contribution is the use of single-cell paired H&E + transcriptomics data. I would like to see an ablation that strongly supports the claim that the molecular information makes a better representation and that the improvements are not just coming from additional training of the 16M cell-centered images. It is true that the different performance of TERRA and NOVAE molecular encoders already show that molecular info affects performance, but it would be informative to see where an image-only contrastive loss (using image augmentation) with similar infoNCE objective would place. This ablation would more clearly quantify how much the molecular info contributes to the improved performance of CellBridge. Another interesting ablation could be to randomly permute the image-omics pairs so that they are no longer matching. The permutation could be either global or within-tissue-sections.
  • A central claim in the paper is 'reliability' but this claim is not quantitatively established. It is only qualitatively explained and illustrated on figure 4. It is also not clear how reliability varies across phenotypes. A simple graph showing a per-phenotype classification error vs reliability score would be a good start. Additionally, it would be helpful to demonstatrate that rejecting low-reliability cells meaningfully reduces classification errors compared to regular confidence-based rejection.
  • Another contribution is COF (Cell-Ontology-Fallback). Claims are made on table 4 that it improves CovGap compared to other standard conformal prediction methods. However the Govgap measure alone is not sufficient, since it measures under-coverage and over-coverage the same way (a trivial eager predictor returning all types would have a 10% covgap). Hence we don't really know whether COF's improvement over GRAPH from 16 to 4 is mostly due to enlarging the prediction set at the expense of specificity. Adding a measure of the prediction set size on table 4 would be informative. Additionally i would like to see the coverage for each phenotype, particularly rare ones, to determine whether some cell types remain substantially undercovered despite the improved average coverage gaps reported in Table 4.
  • It should be noted that the probes used to obtain the numbers on table 2 are trained within each tissue section (using a 80%/20% split). Therefore the reported numbers do not reflect true generalization to unseen tissue sections.
Questions for authors

see weaknesses.

4 comments
AuthorsOpenReview ↗

Our Rebuttal to Reviewer DESd (W1 & W2)

Thank you for reviewing the paper. We are glad that you appreciate the importance of the problem and the strong empirical performance of our approach. We address your comments below, which we hope will help and improve your evaluation.

W1: Clarification of novelty

Thank you for the opportunity to clarify our original contributions. We respectfully argue that CellBridge goes beyond combining existing methods by introducing a new problem formulation to predict both cellular identity and the finest level of biological specificity that inexpensive, routinely available H&E images can reliably support. This is challenging for two reasons: (a) absence of molecular information necessary for fine-grained annotation, and (b) complex hierarchical cell type lineages. Current pathology foundation models are fundamentally limited for this application given their unimodal representation learning paradigms, even when certain cell types are morphologically indistinguishable, and flat, fixed label spaces.

To address this, CellBridge first aligns pathology and molecular foundation models to adequately transfer molecular identity at the single-cell level to extend what morphology alone can determine. Second, ontology-aware conformal prediction calibrates the specificity of these predictions, returning fine-grained identities when supported by the evidence and broader, biologically meaningful categories otherwise.

Our empirical results quantitatively validate both components of CellBridge with increasing underlying task difficulty. As labels become finer, we observe +10 to +16 percentage points improvement on NuCLS (Figure 3). When labels are determined from molecular information rather than morphology, CellBridge outperforms the strongest baseline by +12 percentage points on PhenoCell (Table 2). When there is strong class imbalance due to rare cell types, this leads to a fivefold lower coverage gap on Lizard (Table 4). We believe this represents an important conceptual and practical advance toward extracting more detailed cellular information from routine pathology images without requiring expensive molecular assays at inference.

W2: Ablation on pretraining components

We acknowledge the detailed suggestion to isolate how much molecular information itself aids with better representation. We conducted two additional tests:

  • To separate the contribution of molecular information from that of additional training on the 16M cell-centred crops, we pretrained an image-only variant by replacing expression embedding with two augmented views of the same crop (rotations, flips, mild color jitter) [1]. Molecular alignment outperforms image-only pretraining on all six tasks, by 1.3–11.6 pp, with the largest gains on Lizard (+11.6) and HEST (+6.0).
  • To separate the effect of cell-level alignment from exposure to expression data, we randomly aligned the expression profile to an unrelated cell across the entire training set. This resulted in the performance falling close to or below UNI2-h on all six tasks (e.g., HEST 28.0 vs. 31.3) and well below image-only pretraining.
ModelHEST-1kNuCLSPanNukeLizardPhenoCell—allPhenoCell—immune
UNI2-h31.3 ± 1.823.7 ± 1.472.5 ± 1.435.7 ± 1.325.1 ± 2.137.3 ± 4.4
Image-only + LoRA46.3 ± 2.337.0 ± 2.177.8 ± 1.253.8 ± 1.332.3 ± 2.536.9 ± 3.3
Globally shuffled pairs + LoRA28.0 ± 1.825.3 ± 2.271.0 ± 1.434.8 ± 1.423.2 ± 1.933.0 ± 3.9
CellBridge52.3 ± 2.641.1 ± 2.879.1 ± 1.165.4 ± 1.437.1 ± 3.241.2 ± 3.3

We have included this ablation as a new paragraph in Section 3.4 in the revised manuscript.

AuthorsOpenReview ↗

Our Rebuttal to Reviewer DESd (W3 & W4 & W5)

W3: Quantitative evaluation of reliability scores

We evaluated the reliability score (defined in Equation 10) through expert-reviewed case studies (Figure 4, Appendix Figures 7–13). We agree that quantitative analysis can strengthen the validity of CellBridge. Per your comment, we measured per-phenotype classification error with respect to reliability score, and included this plot in Figure 5b of the revised manuscript. We observe that all four datasets exhibit a negative trend, indicating that phenotypes with lower mean reliability scores tend to have higher classification errors. We quantified this association using Pearson's correlation coefficient (table below).

DatasetPearson r
NuCLS−0.745
PanNuke−0.951
Lizard−0.825
PhenoCell−0.613

We also progressively rejected the least reliable cells causing error to reduce monotonically on all datasets. The table below represents Figure 5c in the revised manuscript and shows that highly reliable cells contain a smaller fraction of errors.

Dataset100% retained80% retained60% retained40% retained20% retained
HEST27.5624.5720.2913.459.35
NuCLS21.8919.5912.173.212.88
PanNuke16.7610.598.608.608.60
Lizard14.709.193.113.113.11
PhenoCell49.1346.6743.1436.8421.17

W4: Analysis on CovGap and mean set size trade-off

Thank you for this important suggestion. Below we provide CovGap and mean set size for Graph-CP and COF following the conventions in Table 4. We indeed observe a trade-off between CovGap and mean set size, which supports COF’s value. The additional set size (0.77 more labels on average) accompanies substantially reduced coverage gaps (5.05 percentage points less on average).

DatasetGraph-CP CovGapCOF CovGapGraph-CP Mean Set SizeCOF Mean Set Size
HEST13.36 ± 1.488.63 ± 0.893.40 ± 0.375.05 ± 0.43
NuCLS10.13 ± 1.724.81 ± 0.651.97 ± 0.122.72 ± 0.12
PanNuke3.34 ± 0.311.45 ± 0.191.39 ± 0.011.56 ± 0.01
Lizard16.81 ± 0.994.62 ± 0.361.39 ± 0.032.45 ± 0.10
PhenoCell all7.71 ± 1.445.38 ± 1.116.92 ± 0.297.70 ± 0.40
PhenoCell immune7.95 ± 2.264.11 ± 1.094.41 ± 0.184.64 ± 0.28

Moreover, in order to analyze per-phenotype coverage, we focus on Lizard, which includes rare cell types such as eosinophils and neutrophils, and report the coverage of Standard-CP, Graph-CP, and COF below. We observe that the benefit of COF is more pronounced for rare cell types, where standard and graph CP methods suffer from drastic undercoverage.

PhenotypeCalibration cellsTest cellsStandard-CPGraph-CPCOF
Eosinophil1,23391846.0861.9894.55
Neutrophil1,9911,64937.3658.1095.57
Plasma cell8,7669,22158.4469.7189.97
Connective tissue cell36,05836,63285.0580.7991.29
Lymphocyte36,73731,83287.2887.4596.59
Epithelial cell82,43882,12298.9698.9199.67

W5: Generalization to unseen tissues

Thank you for the careful read of our experimental setup. Indeed, we leverage certain parts of the tissue for validation/calibration/test and assess the end performance with kNN and linear probing for each annotated HEST sample. We would like to emphasize that we followed the approach of [2] in the 80/20 spatial split and never included any cell during pretraining from the validation and test tissues. In addition, our external datasets were never used during pretraining.

References

[1] Nakhli et al., "VOLTA: an enVironment-aware cOntrastive ceLl represenTation leArning for histopathology." Nature Communications. 2024.

[2] Hizmi et al., Cross-Modal Knowledge Distillation from Spatial Transcriptomics to Histology, arXiv 2026.

We thank Reviewer DESd once again for the time and effort they put into our work. As the discussion period approaches its end, we would kindly like to ask if there are any further comments or concerns that we could address.

Reviewer_DESdOpenReview ↗

Update after rebuttal

I find that the authors' detailed rebuttal and additional experiments strengthen the scientific contribution of the paper. In particular, the image-only adaptation and shuffled-pair controls convincingly show that molecular supervision contributes to the improved representations. The quantitative reliability analysis also address my concerns about the practical usefulness of the proposed reliability score.

The additional conformal prediction results clarify the tradeoff between class-wise coverage and prediction-set size. Even though COF improves coverage at the expense of larger prediction sets, the larger improvements for rare phenotypes justify the tradeoff.

Overall my main remaining reservation is with methodological novelty. The proposed framework combines established techniques for multimodal contrastive learning and ontology-aware conformal prediction. While the formulation is interesting and the empirical results are strong, the broader methodological contribution to representation learning remains somewhat limited.

Nevertheless, the additional experiments substantially strengthen the empirical contribution and address several of my original concerns. So i will revise my assessment upward to weak accept, while maintaining my reservations regarding methodological novelty.

Reviewer_oqeZOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Ontology-Aware Reliability for Fine-Grained Cell Phenotyping from H&E Images: Strengths and Outstanding Concerns.

Critical strengths

A clinically meaningful problem formulation: ontology-aware reliability for cell phenotyping. The paper's central insight is that H&E-based cell classification should not always be forced into a flat label space. Instead, it should identify the finest level of the cell-type hierarchy that morphology can reliably support. This is compelling and addresses an important unmet need. Existing multimodal pathology-transcriptomics approaches, such as PAST and SpatialWhisperer, primarily focus on prediction performance, whereas CellBridge explicitly addresses the limits of morphology-based prediction through an ontology-aware fallback mechanism with branch-conditional conformal calibration. This formulation has potential translational value: by identifying cases in which fine-grained morphological predictions are insufficiently reliable, the approach could help prioritize supplementary molecular assays rather than requiring them routinely. The clinical motivation and the explicit treatment of modality-specific predictability are important strengths, although the extent to which the proposed method advances beyond prior alignment and uncertainty-quantification approaches requires further clarification.

Critical weaknesses
  1. The technical contribution is incremental relative to closely related work, and the paper does not establish the necessity of the proposed design. The pretraining recipe, which combines a frozen pathology foundation model, LoRA adaptation, and InfoNCE-based alignment with embeddings from a frozen single-cell foundation model, follows a now-standard paradigm in this area. Likewise, COF combines existing components: the nested ontology-based nonconformity scores build on Corbetta et al. (2026), while calibrating thresholds within disjoint groups follows the established Mondrian or group-conditional conformal prediction framework (Vovk et al., 2003). The most distinctive elements appear to be the use of Cell Ontology branches as calibration groups and the subsequent closure step. Although both choices are reasonable, their conceptual novelty is limited. This matters because ICLR expects a clear methodological contribution. As presented, the work is closer to a careful systems-level integration of existing techniques. The authors therefore need to demonstrate that the specific conformal design, rather than simply using finer-grained calibration groups, is responsible for the reported reliability gains. This key evidence is currently missing.

  2. The conformal prediction evaluation omits its most directly relevant competing baseline, leaving the paper's central reliability claim insufficiently supported. The ablation study compares only standard-split and graph-based conformal prediction (CP), yet the stated motivation for branch-conditional calibration is that class-conditional calibration can be overly conservative or conceal undercoverage. Because CovGap is measured at the class level, a natural baseline would be Mondrian CP calibrated per phenotype or per node of the same ontology DAG, using identical representations. However, this baseline is not reported. Without this comparison, it is unclear whether the approximately fivefold reduction in CovGap results from exploiting the ontology structure or simply from using finer-grained calibration groups.

Furthermore, (R(I) = 1 - |C(I)| / |Y|) is, by construction, a monotonic function of prediction-set size. Thus, the claim that low reliability scores "localize biologically ambiguous cells" is not adequately supported by the metric alone: cells with larger prediction sets necessarily receive lower scores, but this does not establish that the scores identify biological ambiguity beyond what is captured by prediction-set size or the base classifier's own uncertainty estimates. This limitation weakens the paper's central interpretability claim, and the paper provides no analysis demonstrating that (R) carries additional information beyond these simpler measures.

Questions for authors

1.Can you provide quantitative evidence that low reliability scores identify genuine biological ambiguity, rather than merely reflecting generic model uncertainty or larger prediction sets? 2.How much of the reported 16-percentage-point improvement can be attributed specifically to single-cell alignment, as opposed to the additional 16 million training pairs or the LoRA adaptation strategy? Can you disentangle these contributions through controlled ablation experiments? 3.Given that a batch of 16 cells sampled from the same tissue section may contain many same-type cell pairs that InfoNCE treats as negatives, how do you assess the impact of these potential false negatives on representation learning and downstream performance?

5 comments
AuthorsOpenReview ↗

Our Rebuttal to Reviewer oqeZ (W1a & W1b)

Thank you for your detailed review. We appreciate your positive feedback on the compelling translational value of our proposed approach. We address your comments below, which we hope will clarify and improve your evaluation of the paper.

W1a: Clarification of novelty

Thank you for raising this nuanced point. We would like to clarify that, adhering to ICLR’s call for applications to science, our primary contributions are two-fold:

  • a novel problem formulation addressing a fundamental challenge in computational pathology: enabling inexpensive, routinely acquired H&E images to support fine-grained, molecularly informed cell phenotyping, while recognizing that not all molecularly defined cell identities can be reliably distinguished from morphology alone, and
  • application-specific integration of individual alignment and conformal prediction techniques validated with extensive empirical evidence to demonstrate CellBridge’s advantage with increasing task difficulty.

Thus, CellBridge’s pretraining is aimed at transferring transcriptional information from molecular foundation models into pathology representations at the single-cell level to predict fine-grained phenotypes beyond what morphology can determine. At inference, our Cell-Ontology-Fallback (COF) module incorporates biological relationships into conformal prediction, allowing deferrals to meaningful ancestor categories in cases of ambiguity while maintaining lineage-conditional coverage guarantees.

Existing pathological models are unsuited to this task. Quantitatively, this translates to CellBridge’s outperformance over the strongest baselines when labels become finer (+10 to +16 percentage points on NuCLS, Figure 3), when labels are determined from molecular information rather than morphology (+12 percentage points on PhenoCell, Table 2), and when there is strong class imbalance due to rare cell types (fivefold lower coverage gap on Lizard, Table 4).

W1b: Ablation on calibration groups

Moreover, we isolate the contribution of our conformal design, namely, nested ontology-based nonconformity scores in graph CP versus one-minus probability-based ones in standard CP, and branch-based calibration groups in COF versus global calibration in graph CP (Table 4). This already accounts for ~50% of the total CovGap reduction on average across all datasets.

Following your comment, to assess whether biologically defined calibration groups offer benefits beyond group granularity, we evaluated 100 randomly assigned phenotypes to calibration groups while preserving the number of groups and phenotypes per group fixed (same as COF). The table below compares the results obtained using random assignments (left of the arrows) with those obtained using our COF module (right of the arrows). We observe that ontology-defined calibration groups improve prediction specificity in the most complex label taxonomies (HEST and PhenoCell). Performance remains comparable between the two approaches for NuCLS and Lizard. PanNuke shows no change as its calibration groups consist entirely of singletons in both cases.

DatasetCoverage (%)Mean Set Size (↓)Singleton Rate (↑)
HEST95.34 ± 0.31 → 94.165.25 ± 0.18 → 5.0531.63 ± 2.39 → 40.52
NuCLS95.03 ± 0.85 → 95.322.69 ± 0.10 → 2.7238.47 ± 1.28 → 39.52
PanNuke90.83 ± 0.00 → 90.831.56 ± 0.00 → 1.5675.40 ± 0.00 → 75.40
Lizard95.83 ± 0.97 → 96.552.36 ± 0.12 → 2.4566.36 ± 2.05 → 65.82
PhenoCell all93.87 ± 1.31 → 91.628.44 ± 0.57 → 7.7011.62 ± 4.04 → 20.68
PhenoCell immune94.88 ± 1.86 → 93.334.82 ± 0.32 → 4.647.73 ± 5.68 → 18.67

We have updated our introduction to sharpen the nature of our contributions and have included this ablation in Appendix A.5.

AuthorsOpenReview ↗

Our Rebuttal to Reviewer oqeZ (W2 & W3 & Q1)

W2: Ablation on class-conditional CP

Our branch-based calibration expands prediction sets to shared ancestors of rare cells, enabling better coverage. Per your suggestion to sharpen this argument with a quantitative benchmark, we added an ablation with Mondrian CP calibrated per phenotype, called class-CP, which keeps the ontology score and closure of COF fixed using the same CellBridge representations, classifier probabilities, and splits. We observed that COF achieves similar or lower CovGap and equal or smaller prediction sets than class-CP on five of the six datasets. On HEST, Class-CP’s lower CovGap, however, comes at the cost of 16% larger prediction sets (5.85 vs 5.05). On PanNuke, each branch consists of a single phenotype and so both methods are identical. This ablation, along with Table 4, clearly attributes the strong performance of COF to its exploitation of the ontology structure.

DatasetCovGap : Class-CPCovGap : COFMean Set Size : Class-CPMean Set Size : COF
HEST6.2 ± 0.428.6 ± 0.895.85 ± 0.565.05 ± 0.43
NuCLS5.2 ± 0.634.8 ± 0.652.75 ± 0.122.72 ± 0.12
PanNuke1.5 ± 0.191.5 ± 0.191.56 ± 0.011.56 ± 0.01
Lizard5.4 ± 0.364.6 ± 0.362.48 ± 0.102.45 ± 0.10
PhenoCell all8.0 ± 0.825.4 ± 1.119.15 ± 0.377.70 ± 0.40
PhenoCell immune8.2 ± 0.594.1 ± 1.095.42 ± 0.234.64 ± 0.28

We have included this class-CP CovGap ablation in Table 4 and provided details of the experiment designs in Appendix A.5 of the revised manuscript.

W3 & Q1: Quantitative evidence on biological ambiguity

We appreciate this precise observation and agree that quantitative measurements consistent with underlying biology, such as macrophages infiltrating tumor nests (Figure 4), and within-lineage cases such as T versus B cells, which COF returns as lymphocytes (Figure 1), would strengthen the result. Accordingly, we conducted two distinct quantitative analyses. First, we used the single-cell data in HEST to compute molecular ambiguity. For each held-out cell, we identified its 30 nearest neighbours by cosine distance in the TERRA embedding space and calculated the fraction of neighbour pairs with different phenotype annotations. We observe that retaining 50% of cells with the highest reliability (and rejecting the rest) reduces mean molecular ambiguity from 19.45% to 14.54%, supporting a negative association as shown in Figure 5a in the updated manuscript. Second, we examine the relationship between phenotype-level reliability and classification errors across the external datasets. As shown in Figure 5b of the updated manuscript, all four datasets exhibit a negative trend, indicating that phenotypes with lower mean reliability scores tend to have higher classification errors. We quantified this association using Pearson's correlation coefficient (table below).

DatasetPearson r
NuCLS−0.745
PanNuke−0.951
Lizard−0.825
PhenoCell−0.613
AuthorsOpenReview ↗

Our Rebuttal to Reviewer oqeZ (Q2 & Q3)

Q2: Ablation on pretraining components

To disentangle the contributions of the three factors, we held the pretraining setup in Appendix A.3.2 fixed and conducted the following ablation (table below):

  • Additional training pairs: image-only pretraining via InfoNCE between two augmented views, rotated, and horizontal and vertical flips with mild color jitter [1] on the same 16M cell-centred crops.
  • Single-cell alignment: image–expression pairs randomly permuted across the training set, so that they no longer match.
  • LoRA adaptation: LoRA disabled, so that alignment is learned only by the projection head, which is retained at inference.
ModelHEST-1kNuCLSPanNukeLizardPhenoCell—allPhenoCell—immune
UNI2-h31.3 ± 1.823.7 ± 1.472.5 ± 1.435.7 ± 1.325.1 ± 2.137.3 ± 4.4
Image-only + LoRA46.3 ± 2.337.0 ± 2.177.8 ± 1.253.8 ± 1.332.3 ± 2.536.9 ± 3.3
Globally shuffled pairs + LoRA28.0 ± 1.825.3 ± 2.271.0 ± 1.434.8 ± 1.423.2 ± 1.933.0 ± 3.9
Projection head-only, no LoRA32.3 ± 1.927.2 ± 2.272.1 ± 1.437.8 ± 1.325.2 ± 2.133.2 ± 3.7
CellBridge52.3 ± 2.641.1 ± 2.879.1 ± 1.165.4 ± 1.437.1 ± 3.241.2 ± 3.3

On NuCLS, additional training on the cell-centred crops accounts for 11.9 percentage points (37.0 vs. 25.1) of the total 16.0 percentage points improvement over the strongest baseline. Across all six tasks, single-cell alignment adds 1.3–11.6 pp over image-only pretraining, with the largest gains on Lizard (+11.6) and HEST (+6.0). With permuted pairs, performance falls to around the UNI2-h level on every task (HEST 28.0 vs. 31.3). The signal must also reach the encoder: without LoRA, performance remains close to UNI2-h (HEST 32.3), compared with 52.3 for CellBridge.

We have included this ablation as a new paragraph in Section 3.4 in the revised manuscript.

Q3: Ablation on potential false negatives

We would like to clarify that sampling batches within a tissue section is deliberate because negatives from other sections can be separated by section-level differences in staining and scanning rather than by cell features. Cells of the same type receive distinct embeddings since TERRA models a cell’s expression together with its spatial neighbourhood (Appendix A.3.1).

To isolate the impact of potential false negatives, we masked in-batch negatives that share the anchor’s molecular cluster, removing them from the InfoNCE denominator, and kept everything else fixed. We observed that masking gives no consistent improvement (table below). Results on HEST, PanNuke, Lizard, and both PhenoCell tasks change by at most 0.6 pp, while NuCLS drops by 5.1 pp (41.1 → 36.0).

ModelHEST-1kNuCLSPanNukeLizardPhenoCell—allPhenoCell—immune
CellBridge52.34 ± 2.5641.14 ± 2.8279.15 ± 1.1165.41 ± 1.3637.05 ± 3.2441.22 ± 3.31
Molecular cluster masking + LoRA52.31 ± 2.5536.00 ± 2.9679.27 ± 1.1165.72 ± 1.5537.01 ± 3.0840.62 ± 3.39

We have included this sensitivity analysis in Appendix A.5 of the revised manuscript.

References

[1] Nakhli et al., "VOLTA: an enVironment-aware cOntrastive ceLl represenTation leArning for histopathology." Nature Communications. 2024.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

We thank all the reviewers for their detailed reviews. We are glad they found our work on fine-grained cell phenotyping from routine H&E images biologically meaningful with impressive results. Their questions centered on controlled experiments to disentangle the building blocks of CellBridge. We summarize the salient points below:

  • How does CellBridge extend prior research on multimodal alignment and conformal prediction (oqeZ, DESd)?

CellBridge addresses the scientific challenges in reliably predicting cell phenotypes from H&E alone for fine ontology levels, molecularly defined identity, and class imbalance. In service of this novel problem formulation, our method delivers significant gains over state-of-the-art foundation models.

  • Does performance gain arise from molecular alignment or additional data (oqeZ, DESd)?

Molecular alignment outperforms image-only pretraining on the same 16M crops on all six tasks by up to 11.6 percentage points. Shuffling the image-expression pairs or removing LoRA reduces performance to the UNI2-h level.

  • Does ontology structure matter for calibration (oqeZ)?

Compared to per-phenotype Mondrian style calibration, COF yields equal or smaller prediction sets and equal or lower CovGap on five of six datasets. Random calibration groups of the same size lower singleton rates by up to 11 percentage points.

  • Does reliability score identify unreliable predictions (oqeZ, DESd, 5wkR)?

Reliability per-phenotype correlates negatively with classification error across heldout datasets (r = -0.61 to -0.95). Rejecting cells with low-reliability scores consistently reduces error. On HEST, the 50% most reliable cells show lower molecular ambiguity than all cells (14.5% vs 19.5%).

Together, these additions clarify the contributions of CellBridge and provide quantitative support for each of its design choices. We are grateful to the reviewers for their feedback, which has substantially strengthened our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.