acceptodds
Under review as a conference paper at ICLR 2027

Post-hoc Robustification of Pathology Foundation Models with a Vision-Language Judge

Abstract

Pathology foundation models are meant to describe tissue, yet their embeddings also record where and how a slide was acquired. Institution, scanner and staining leave batch effects strong enough that tiles can group by domain instead of by biology, which weakens transfer to new sites. Standard mitigation methods require large amounts of labeled training data, while label-free methods often struggle to remove domain-specific bias while preserving genuine biological signal. Multimodal large language models (MLLMs) can make robust judgements on biological similarity in images, but their utility in computational pathology is under-explored. To address the robustness issue in pathology foundation models, we propose RObustify PAthology models with a vision-Language judge (ROPAL), a label-free protocol in which MLLMs supervise their adaptation toward biological similarity. ROPAL asks an MLLM which tissue tiles are biologically alike across institutions and uses these judgements to fine-tune the foundation model, so that its embeddings group tissue by biology rather than by the domain it came from. We demonstrate the effectiveness of ROPAL through extensive empirical analysis across a diverse range of datasets, diseases, tasks, base models and robustification baselines. We release the code, training corpus and fine-tuned checkpoints.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal models, robustness, computational pathology, domain adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QQ5pOpenReview ↗
Rating
3(+1)
Confidence
4(+2)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes ROPAL, a label-free adaptation framework intended to improve the acquisition robustness of pathology foundation models. The central idea is to use a multimodal language model as a biological similarity judge. The authors construct triplets containing two patches from the same slide and one patch from another institution. A triplet is retained when the judge identifies one of the same-slide patches as the odd one out, thereby providing a cross-domain similarity preference.

The selected preferences supervise parameter-efficient adaptation through a Bradley–Terry ranking objective, complemented by a regularization term intended to preserve within-slide representation structure.

The method is evaluated on three pathology foundation models across multiple benchmarks involving cross-institution retrieval, scanner and staining invariance, spatial gene-expression prediction, mitotic figure classification, and prostate grading. The authors report substantial improvements in several cross-domain similarity and retrieval tasks, along with more heterogeneous improvements on downstream classification tasks.

Critical strengths
  1. A relevant robustness problem and a practical adaptation strategy.

The paper addresses acquisition-related confounding in pathology foundation model embeddings, an important challenge for cross-institution generalization. Using a multimodal model to obtain relative similarity preferences without manual diagnostic labels is a potentially useful approach.

The method is parameter-efficient, and the adapted encoder can be used independently at inference time without further calls to the multimodal judge.

  1. Encouraging cross-domain results across multiple evaluation settings.

The experiments cover three foundation models and several benchmarks evaluating different sources of domain shift. In particular, the improvements in MOSAIC, SCORPION, and PLISM indicate that the proposed adaptation can meaningfully alter embedding relationships across institutions and acquisition conditions.

The inclusion of judge-selection controls, reverse-confounding analysis, and several comparisons with established adaptation approaches provides useful empirical context.

Critical weaknesses
  1. Insufficient validation of biological similarity supervision. The reliance on a multimodal language model for biological similarity supervision (with ~79% average accuracy and high variance) lacks systematic expert pathologist verification for the 60k+ retained triplets. Furthermore, the main MOSAIC benchmark closely mirrors the preference training objective, whereas downstream gains on independent predictive tasks (e.g., MIDOG++, cross-site PANDA) are heterogeneous and base-encoder dependent. The authors must distinguish improvements in cross-domain preference geometry from true generalizable clinical utility.

  2. Flawed baseline optimization and unverified loss components. Main comparisons rely on compromised baselines, retaining runs where Pathology DoRA collapsed (rank ~1.0 on Virchow2, ~4.8 on GenBio-PathFM) under a severely truncated training schedule (1,100 steps, ~0.09 epochs), which fails to reflect properly tuned adaptation approaches. Additionally, the necessity of the rank-preservation objective is unproven, as Table 6 shows the Bradley–Terry-only variant performing comparably or better with no statistically significant advantage for the full objective.

  3. Flawed statistical sampling and uncertainty estimation. Uncertainty estimates rely on patch- or query-level bootstrapping, ignoring non-independent acquisition structures (e.g., repeated tissue locations across scanners in SCORPION). Evaluating via hierarchical bootstrapping at the patient, slide, or site level, along with reporting multi-run variance across seeds, is essential to establish reliable comparative claims.

Questions for authors
  1. Can the authors clarify the independent contribution of biological judgment quality?

How do the results change when judge predictions are corrupted, when low-confidence judgments are removed, or when triplets are selected using more conventional morphological similarity measures under matched training budgets?

  1. Can the collapsed baseline configurations be rerun with stable training settings?

The reported Pathology DoRA configurations collapse for two foundation models. Could the authors provide comparisons using a stable, appropriately tuned implementation and a more comparable optimization budget?

It would also be useful to distinguish results from successfully trained baselines when reporting aggregate method rankings.

  1. Is the rank-preservation regularizer necessary?

Table 6 indicates that the Bradley–Terry-only objective is competitive with the full method and performs better on several metrics. Could the authors clarify the specific failure mode that the additional regularization is intended to address, and provide quantitative evidence that it preserves biologically relevant within-domain structure?

6 comments
AuthorsOpenReview ↗

Response to Reviewer QQ5p, Part 1

Response to Reviewer QQ5p, Part 1

We thank the reviewer for finding the robustness problem relevant and the adaptation strategy practical, and for noting the judge-selection controls and reverse-confound analysis. Below is our point-by-point response (this response also addresses Shared.1 in QQ5p.3 and QQ5p.7, Shared.2 in QQ5p.1 and QQ5p.6, Shared.3 in QQ5p.4 and QQ5p.8, Shared.4 in QQ5p.5 and Shared.5 in QQ5p.2).

[QQ5p.1] The biological similarity supervision lacks expert pathologist verification.

"The reliance on a multimodal language model for biological similarity supervision (with ~79% average accuracy and high variance) lacks systematic expert pathologist verification for the 60k+ retained triplets."

Thank you for this comment. We treat the judge's verdicts as a training signal rather than as ground truth, in the same way self-supervised pretraining treats its teacher. The base models we adapt (UNI2-h, Virchow2, GenBio-PathFM) were themselves trained by teacher-student self-distillation, whose targets (prototype assignments, masked-token predictions) are never verified by pathologists. Their value is established by the usefulness of the resulting embeddings. The judge plays the same role, as it supplies relative similarity preferences, and what matters is whether training on them improves the embeddings, which we evaluate on eleven datasets.

In the revised manuscript, we added the following sentences to the Method to clarify this (Section 3.1).

"We treat the judge's verdicts as a training signal, not as ground truth, as self-supervised pretraining treats the targets of its teacher. Their value is established by the performance of the adapted embeddings (Section 4), and the method does not require every verdict to be correct."

AuthorsOpenReview ↗

Response to Reviewer QQ5p, Part 2

Response to Reviewer QQ5p, Part 2

[QQ5p.2] MOSAIC mirrors the training objective, and downstream gains are heterogeneous.

"the main MOSAIC benchmark closely mirrors the preference training objective, whereas downstream gains on independent predictive tasks (e.g., MIDOG++, cross-site PANDA) are heterogeneous and base-encoder dependent. The authors must distinguish improvements in cross-domain preference geometry from true generalizable clinical utility."

Thank you for this comment. We agree that MOSAIC and our training objective both directly target the prioritization of biological signals over domain attributes. Like PathoROB, MOSAIC is included because it isolates the evaluation of robustness from other characteristics of a model that may contribute to its performance on downstream tasks. However, MOSAIC is only one of the many benchmarks we performed. In total, we evaluate four cross-domain generalization tasks (PathoROB, MOSAIC, SCORPION, PLISM) and five downstream clinical tasks (cross-institution retrieval and classification, HEST, MIDOG++, PANDA), and ROPAL is the best method on average on all 9 of them (Figure 2).

Following your suggestion, to better distinguish improvements on the cross-domain generalization tasks from those on the downstream clinical tasks, the revised Figure 2 marks the category of each experiment, and the revised manuscript defines the two categories and reports them separately (Appendix Figure 15).

"The experiments fall into two categories. The cross-domain generalization tasks (PathoROB, MOSAIC, SCORPION and PLISM) test directly whether embeddings follow biology across institutions, scanners and stainings. The downstream clinical tasks (cross-institution retrieval and classification, HEST, MIDOG++ and PANDA) test whether this robustness carries over to clinical prediction and retrieval tasks. On the cross-domain generalization tasks, ROPAL or one of its variants ranks first in 34 of the 36 result rows. On the downstream clinical tasks, ROPAL or one of its variants ranks first in 66 of the 81 result rows."

The new Appendix Figure 15 also splits each base model into the two categories, and ROPAL is the best method on all six combinations of base model and category.

Regarding performance on downstream tasks, on MIDOG++, averaged over the three base models, ROPAL is the only one of the seven methods that improves on the frozen encoder when tumor types are held out, and when scanners are held out its gain over the frozen encoder is about twice that of the best baseline. A new Figure 6 in the revised manuscript shows these gains for every method (Section 4.4.3). On cross-site PANDA, ROPAL is the only method that improves on the frozen encoder for all three base models, and averaged over them its gain is about twice that of the best baseline. This is shown in the new Average row of Table 4 and in a new Appendix Figure 16 (Section 4.4.4).

On the other downstream tasks, the lead of ROPAL is clearer. In cross-institution classification under institution-class confounding, ROPAL has the highest accuracy on all 6 dataset and base pairs, with a mean gain of 18.7 points over the frozen encoder (Figure 5). In cross-institution retrieval, it has the highest mAP on all 3 datasets when averaged over the base models (Table 2), and on HEST, which predicts gene expression from H&E images, it gives the highest correlation on all three base models, each significantly under the cluster bootstrap (Table 3).

[QQ5p.3] Collapsed Pathology DoRA runs under a severely truncated schedule.

"Main comparisons rely on compromised baselines, retaining runs where Pathology DoRA collapsed (rank ~1.0 on Virchow2, ~4.8 on GenBio-PathFM) under a severely truncated training schedule (1,100 steps, ~0.09 epochs), which fails to reflect properly tuned adaptation approaches."

Thank you for this point. We fixed both issues by tuning DoRA's learning rate so that no run collapses and by matching its training budget to ROPAL's (td9B.1, Appendix A.12, Appendix Table 53). All the DoRA results in the revised manuscript have been updated, and DoRA is now the leading baseline, while the ranking of ROPAL is largely unchanged (td9B.1).

AuthorsOpenReview ↗

Response to Reviewer QQ5p, Part 3

Response to Reviewer QQ5p, Part 3

[QQ5p.4] The necessity of the rank-preservation objective is unproven.

"the necessity of the rank-preservation objective is unproven, as Table 6 shows the Bradley–Terry-only variant performing comparably or better with no statistically significant advantage for the full objective."

Thank you for this comment. A major purpose of the rank preservation term is to regularize training by keeping the within-slide structure of the base model, which the BT loss does not constrain. In a new experiment that trains the model with and without the rank term for an increasing number of epochs, the model degrades less with the rank term than without it as training continues (WBsH.2, Appendix Figure 9 and Table 9).

The Method of the revised manuscript now states this purpose (Section 3.2).

"The Bradley-Terry term constrains only cross-domain preferences. The rank term is a regularizer that keeps the within-slide structure of the base encoder, so that the model degrades less when training runs longer (Appendix A.3.1)."

[QQ5p.5] Patch- or query-level bootstrap ignores the acquisition structure, and training seeds.

"Uncertainty estimates rely on patch- or query-level bootstrapping, ignoring non-independent acquisition structures (e.g., repeated tissue locations across scanners in SCORPION). Evaluating via hierarchical bootstrapping at the patient, slide, or site level, along with reporting multi-run variance across seeds, is essential to establish reliable comparative claims."

Thank you for this suggestion. We agree that the bootstrap should respect the acquisition structure, and the revision replaces the bootstrap over individual samples by a cluster bootstrap. From the saved per-sample outputs, without rerunning any model, we resample whole clusters, namely tissue locations for SCORPION, slides for PLISM, PathoROB and HEST, reference slides for MOSAIC, query slides for the cross-institution tasks, cases for MIDOG++ and biopsies for PANDA. Every SD in the SD Appendix, every error bar in the figures and every significance mark now comes from this cluster bootstrap, and each caption names its cluster unit (Appendix A.7).

A new Appendix Table 35 compares the number of significant leads of ROPAL before and after this change, on the same results. Of the 51 leads significant under the bootstrap of the submission, 46 (90%) remain significant under the cluster bootstrap, and the total changes from 51 to 59. The Appendix now lists the significant leads of each experiment (Appendix A.7), and training-seed variance is reported separately (td9B.7).

[QQ5p.6] (Q1) Independent contribution of judgement quality.

"How do the results change when judge predictions are corrupted, when low-confidence judgments are removed, or when triplets are selected using more conventional morphological similarity measures under matched training budgets?"

Thank you for these suggestions. We added experiments for all three, each in the matched setting of the other ablations.

  • Corrupted judge predictions. Noisy judge (p%) replaces p% of the triplets selected by the judge with judged triplets that it did not select. Performance degrades gradually as the noise rate grows, on both base models, consistent with judge quality mattering (new Appendix Table 10, Appendix A.3.1).
  • Low-confidence judgements removed. High-confidence judge keeps only the selections that a second MLLM judge agrees with. Removing low-confidence judgements gives a moderate gain on most tasks (new Appendix Table 10).
  • Conventional similarity selection. MNN and Odd-one-out score, which select triplets by frozen-encoder similarity, and CONCH image similarity all fall below the judge (Table 5, Appendix Table 21).

[QQ5p.7] (Q2) Stable DoRA with a comparable budget, and rankings without collapsed baselines.

"Could the authors provide comparisons using a stable, appropriately tuned implementation and a more comparable optimization budget? It would also be useful to distinguish results from successfully trained baselines when reporting aggregate method rankings."

Thank you for these suggestions. We further tuned both baselines. For Pathology DoRA, we tuned the learning rate and matched its optimization budget to that of our method, and for AdvInv, we changed the training schedule to the full schedule. No run collapses any more, and the results of both baselines are updated in the revised manuscript. The ranking of our method remains largely unchanged (td9B.1, td9B.2, Figure 2).

Reviewer_td9BOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses acquisition-related biases in pathology foundation models (PFMs), where embeddings may encode institution, scanner, staining, and other domain-specific characteristics strongly enough to interfere with biologically meaningful similarity and cross-site generalization. The authors propose ROPAL, a label-free, post-hoc robustification framework that uses a multimodal large language model (MLLM) as a role-blind judge of biological similarity. ROPAL constructs triplets comprising two tiles from the same slide and one tile from a different institution. Without being informed of these roles, the MLLM identifies the biological odd-one-out. Triplets are retained when a same-slide tile is selected as the odd sample, thereby generating a relative preference indicating that the cross-domain tile should be closer to the remaining same-slide tile. These preferences are used to train a LoRA adapter with a Bradley–Terry ranking objective together with a within-slide rank-preservation term. Importantly, the MLLM is required only for constructing the training preferences and is not used at inference. The method is evaluated on three pathology foundation models—UNI2-h, Virchow2, and GenBio-PathFM—across a broad collection of robustness, retrieval, and downstream benchmarks, including PathoROB, MOSAIC, SCORPION, PLISM, HEST, MIDOG++, PANDA, and cross-institution retrieval/classification tasks. Comparisons include stain normalization, adversarial invariance, concept-erasure, and self-supervised adaptation approaches. Overall, ROPAL achieves the strongest aggregate ranking and ranks first in 88 of 117 reported settings, with particularly substantial improvements under institution-class confounding. Ablations further indicate that MLLM-based triplet selection, rather than the triplet-training structure alone, is responsible for much of the improvement. However, performance gains are considerably smaller on some independent downstream tasks, and simple Reinhard normalization remains superior in several scanner-only PLISM settings.

Critical strengths
  1. Novel and well-motivated label-free supervision for preserving biology while reducing domain bias. The strongest contribution is the conversion of role-blind MLLM biological-similarity judgments into relative distance preferences for post-hoc representation adaptation. This provides an appealing alternative to approaches that either require task-specific annotations or explicitly remove domain information at the risk of also suppressing biologically relevant features. The design is particularly compelling because only slide/institution metadata are required to construct candidate triplets, while the MLLM is removed entirely at inference. The selection ablations substantially strengthen the causal interpretation: random cross-domain triplets can degrade performance, whereas MLLM-selected triplets consistently improve it, indicating that the biological selection signal—not merely cross-domain pulling or the optimization objective—is responsible for the observed gains.

  2. Broad and unusually systematic empirical validation. The evaluation spans three modern pathology foundation models and a diverse set of datasets covering domain robustness, stain/scanner invariance, retrieval, nearest-neighbor classification, gene-expression prediction, mitosis classification, and prostate grading. ROPAL achieves the strongest aggregate ranking and ranks first in 88 of 117 reported settings. Particularly valuable are the controlled selection ablations and reverse-confound analysis, which provide evidence that the model is not simply learning a generic preference for cross-domain samples. This breadth and consistency make the robustness improvements substantially more credible than they would be from evaluation on a single model or benchmark.

Critical weaknesses
  1. The comparative evidence is weakened by imperfectly optimized baselines and the absence of a simpler biological-supervision control. Several important baseline comparisons do not appear fully conclusive. Pathology DoRA reportedly collapses on Virchow2 and GenBio-PathFM under the transferred configuration, meaning those results characterize a failed optimization run rather than the strongest attainable performance of the baseline. AdvInv is also evaluated with a screened weight and shortened training schedule, while other potentially informative comparisons are excluded because of protocol constraints. More importantly, the experiments do not establish whether a general-purpose MLLM is actually necessary for constructing the useful triplets. A lower-cost pathology-specific VLM or zero-shot tissue classifier could potentially provide sufficiently accurate coarse biological similarity judgments. Evaluating such a control under the identical triplet-selection rule, training objective, and comparison budget would materially clarify whether ROPAL's advantage derives specifically from MLLM reasoning or more generally from access to a weak biological labeler. Together, these issues weaken the strength of the comparative claim despite the otherwise extensive evaluation.

  2. The practical significance and robustness of the improvements are uneven across evaluation settings. The largest improvements occur on robustness and confounding-sensitive benchmarks closely aligned with the training objective, whereas gains on several independent downstream evaluations are substantially smaller. MIDOG++ improvements are approximately 0.007–0.018, some PANDA differences are comparable to the reported evaluation variability, and TumSeg results are mixed. ROPAL also does not consistently dominate simple stain normalization under pure scanner shifts: Reinhard normalization performs better in four of six PLISM settings. Moreover, the reported bootstrap deviations primarily quantify evaluation-sample variability; if each adapted checkpoint was trained only once, optimization/seed variability remains unknown. Consequently, the current evidence strongly supports improved embedding robustness under acquisition confounding, but is less decisive regarding the magnitude and consistency of practical downstream benefit across independent tasks and acquisition shifts.

Questions for authors
  1. Training stability and variance: How many independent training seeds were used for each ROPAL checkpoint? Please report across-seed variability for representative robustness and downstream benchmarks, particularly MOSAIC, PathoROB, and MIDOG++. The currently reported bootstrap variation appears to characterize evaluation variability rather than optimization variability.

  2. Baseline optimization: For the Pathology DoRA runs that collapsed on Virchow2 and GenBio-PathFM, were reasonable hyperparameter adjustments—such as learning-rate tuning, gradient clipping, or other stabilization measures—attempted? Similarly, can the authors clarify the rationale for the shortened AdvInv schedule and whether conclusions change under a fully optimized configuration?

  3. Necessity of the MLLM judge: How much of ROPAL's advantage depends specifically on using an MLLM rather than a simpler pathology-specific biological classifier? A particularly informative experiment would construct triplets using zero-shot tissue predictions or similarity scores from a pathology VLM, while keeping the selection rule, number of triplets, optimization objective, and training budget identical. This comparison would determine whether the proposed MLLM-based supervision provides information beyond that available from a substantially cheaper weak biological labeler.

  4. Judge reliability and selection behavior: What proportion of candidate triplets pass the MLLM selection criterion, and how does this rate vary across tissues, cohorts, or institutions? If expert/pathologist annotations or biological labels are available for a representative subset, what is the agreement between the MLLM's odd-one-out judgments and those references? An analysis relating judge accuracy or selection rate to downstream improvement would help establish how sensitive ROPAL is to errors in the supervisory signal.

Combining ROPAL with complementary stain/scanner normalization may be worth investigating, particularly given Reinhard's advantage on several PLISM scanner-only settings. Likewise, validating the adaptation procedure using a non-TCGA multi-institutional training corpus would help determine whether the learned robustification generalizes beyond the batch-effect structure represented in TCGA.

7 comments
AuthorsOpenReview ↗

Response to Reviewer td9B, Part 1

Response to Reviewer td9B, Part 1

We thank the reviewer for finding the role-blind supervision novel and well motivated, and the validation broad and systematic. In particular, we thank the reviewer for the suggestion of combining our method with classical normalization methods, which we implemented and found to further increase our method's lead over the baselines. Below is our point-by-point response (this response also addresses Shared.1 in td9B.1, td9B.2 and td9B.8, Shared.2 in td9B.3, td9B.9 and td9B.10, Shared.4 in td9B.6 and td9B.7, Shared.5 in td9B.4, td9B.5 and td9B.11 and Shared.6 in td9B.5 and td9B.11).

[td9B.1] Pathology DoRA collapses on Virchow2 and GenBio-PathFM.

"Pathology DoRA reportedly collapses on Virchow2 and GenBio-PathFM under the transferred configuration, meaning those results characterize a failed optimization run rather than the strongest attainable performance of the baseline."

Thank you for this point. We retrained DoRA with an optimized learning rate and an increased number of training steps, so that no run collapses (Appendix A.12, Appendix Table 53). All the DoRA results in the manuscript have been updated, and DoRA is now among the leading baselines, but the leads of ROPAL remain largely unchanged (leads on 88 of 117 subtasks before and on 83 of 117 subtasks after).

In the revised manuscript, the DoRA column of every result table and figure shows the retrained runs, and Appendix Table 53 reports the collapse diagnostics of the submitted and the retrained runs.

[td9B.2] AdvInv screened weight and shortened schedule, and excluded comparisons.

"AdvInv is also evaluated with a screened weight and shortened training schedule, while other potentially informative comparisons are excluded because of protocol constraints."

Thank you for this comment. We changed the training schedule of AdvInv to the full schedule. All AdvInv results are updated in the revised manuscript (Appendix Table 52).

[td9B.3] Is a general-purpose MLLM necessary, or would a cheaper pathology VLM suffice?

"the experiments do not establish whether a general-purpose MLLM is actually necessary for constructing the useful triplets. A lower-cost pathology-specific VLM or zero-shot tissue classifier could potentially provide sufficiently accurate coarse biological similarity judgments."

Thank you for proposing this control, which we added. We used CONCH as the judge in two ways, by its image similarity, which names the odd tile of each triplet from CONCH image embeddings, and by its labels, which names it from CONCH zero-shot tissue labels over text-defined classes. Both selections fall below the MLLM judge on every task and base.

In the revised manuscript, the two CONCH selections are new rows of Table 5 and new columns of the per-base Appendix Table 21.

AuthorsOpenReview ↗

Response to Reviewer td9B, Part 2

Response to Reviewer td9B, Part 2

[td9B.4] Downstream gains are smaller and uneven (MIDOG++, PANDA, TumSeg).

"The largest improvements occur on robustness and confounding-sensitive benchmarks closely aligned with the training objective, whereas gains on several independent downstream evaluations are substantially smaller. MIDOG++ improvements are approximately 0.007–0.018, some PANDA differences are comparable to the reported evaluation variability, and TumSeg results are mixed."

Thank you for this comment. We agree that our training objective targets cross-site generalizability and robustness toward confounding. We would like to clarify that the robustness and confounding benchmarks are themselves important clinical benchmarks that reflect real-world challenges. At deployment, the image distribution of a new site differs from that of the training sites, and when site and label are confounded, models learn site signatures as shortcuts, which degrades safety-critical clinical predictions. This is well documented in the literature, where site-specific signatures bias model accuracy [8], the acquisition site can be predicted from TCGA images [9], hidden acquisition variables are learnable from slides [10], models trained at one laboratory or scanner often underperform at another [11, 12], pathology foundation models retain medical-center signatures that dominate their feature space [6, 7], and the PathoROB study finds robustness deficits in all 20 foundation models it evaluates [13]. Cross-institution retrieval and classification, with and without institution-class confounding, test exactly these scenarios, and they are where ROPAL gains most (Section 4.4.1).

In the revised manuscript, we added sentences to highlight this, and Section 4.2 now opens with the following sentences.

"The robustness benchmarks reflect challenges met at deployment: the images of a new site differ from those of the training sites, and models trained at one laboratory or scanner often underperform at another [...]. Pathology foundation models retain medical-center signatures in their embeddings [...], so these benchmarks measure whether an embedding follows the tissue or the site."

Section 4.4.1 now states the following.

"Cross-institution retrieval and classification test whether a tile is matched with tiles of the same tissue class from other institutions on Camelyon16, BEETLE and TumSeg [...], also when institution and class are confounded, a setting in which a model can learn the site signature as a shortcut [...]."

In addition to the robustness-focused benchmarks, we also evaluated the proposed method on different downstream tasks. For MIDOG++, we agree that the absolute improvement of the evaluation metrics is smaller than on the other tasks. This is because the task is challenging for every robustification method, and the advantage of our method is clearer when its improvement is compared with the distribution of the improvements of the baselines. In the revised manuscript, a new Figure 6 shows the gain of each method over the frozen encoder, averaged over the three base models, and Section 4.4.3 now states the following.

"With tumor types held out, ROPAL is the only method above the frozen encoder, with all 6 baselines at or below it. With scanners held out, its gain over the frozen encoder is about twice that of the best baseline."

References

[6] de Jong et al. Current pathology foundation models are unrobust to medical center differences. arXiv:2501.18055, 2025.
[7] Kömen et al. Do histopathological foundation models eliminate batch effects? A comparative study. arXiv:2411.05489 (NeurIPS 2024 AIM-FM workshop), 2024.
[8] Howard et al. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nature Communications, 2021.
[9] Dehkharghanian et al. Biased data, biased AI: deep networks predict the acquisition site of TCGA images. Diagnostic Pathology, 2023.
[10] Schmitt et al. Hidden variables in deep learning digital pathology and their potential to cause batch effects: prediction model study. Journal of Medical Internet Research, 2021.
[11] Tellez et al. Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology. Medical Image Analysis, 2019.
[12] Aubreville et al. Quantifying the scanner-induced domain gap in mitosis detection. MIDL, 2021.
[13] Kömen et al. Towards robust foundation models for digital pathology. arXiv:2507.17845, 2025.

AuthorsOpenReview ↗

Response to Reviewer td9B, Part 3

Response to Reviewer td9B, Part 3

[td9B.4] (continued)

On PANDA, ROPAL is the only method that improves on the frozen encoder for all three base models. In the revised manuscript, Table 4 has a new Average row, a new Appendix Figure 16 shows the gain of each method over the frozen encoder for each base model and on average, and Appendix A.2.8 now states the following.

"ROPAL is the only method above the frozen encoder on every base model. In the Average row, ROPAL is 0.049 above the frozen encoder, about twice the gain of the best baseline."

For tasks involving the TumSeg dataset, ROPAL gives the highest retrieval mAP under institution-class confounding averaged over the three base models, significantly above every baseline under the cluster bootstrap (Table 2).

Finally, our method achieves the highest performance on all the base models on HEST (Section 4.4.2), the widely used benchmark for predicting molecular data from H&E images, which is an active research area [14-21] and the basis of multiple commercial products [22-26].

[td9B.5] Reinhard normalization is better in four of six PLISM scanner-only settings.

"ROPAL also does not consistently dominate simple stain normalization under pure scanner shifts: Reinhard normalization performs better in four of six PLISM settings."

Thank you for this comment. We agree that ROPAL has limited performance compared to Reinhard on PLISM scanner-only retrieval and presented this as a limitation case in the submitted manuscript. Scans of the same slide from different scanners differ mainly in global attributes that change uniformly over the image (for example lightness, colour and contrast), which is aligned with the simple normalization procedure of Reinhard (Appendix A.5.7).

Following your suggestion in td9B.11 (Q5), we combined Reinhard normalization with ROPAL. This variant of our method, ROPAL (Reinhard), is now the best method on all six scanner-only pairs of PLISM, and it outperforms every baseline on all 18 scanner and stain subtasks (SCORPION, PLISM staining and PLISM scanner-only, Section 4.3, Figure 4). This suggests that our method and classical normalization methods have complementary strengths.

References

[14] Coudray et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nature Medicine, 2018.
[15] Kather et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nature Medicine, 2019.
[16] Fu et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nature Cancer, 2020.
[17] Schmauch et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nature Communications, 2020.
[18] He et al. Integrating spatial gene expression and breast tumour morphology via deep learning. Nature Biomedical Engineering, 2020.
[19] Wagner et al. Transformer-based biomarker prediction from colorectal cancer histology: a large-scale multicentric study. Cancer Cell, 2023.
[20] El Nahhas et al. Regression-based deep-learning predicts molecular biomarkers from pathology slides. Nature Communications, 2024.
[21] Jaume et al. HEST-1k: a dataset for spatial transcriptomics and histology image analysis. NeurIPS Datasets and Benchmarks, 2024.
[22] Wang et al. Screen Them All: High-throughput pan-cancer genetic and phenotypic biomarker screening from H&E whole slide images. arXiv:2408.09554, 2024.
[23] Saillard et al. Validation of MSIntuit as an AI-based pre-screening tool for MSI detection from colorectal cancer histology slides. Nature Communications, 2023.
[24] Arslan et al. Evaluation of a predictive method for the H&E-based molecular profiling of breast cancer with deep learning. bioRxiv, 2022.
[25] Mayer et al. Direct identification of ALK and ROS1 fusions in non-small cell lung cancer from hematoxylin and eosin-stained slides using deep learning algorithms. Modern Pathology, 2022.
[26] Park et al. Deep learning predicts EGFR mutation status from histology images in non-small cell lung cancer. Cancer Research Communications, 2025.

Reviewer_WBsHOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROPAL, a method that uses MLLMs to generate weak supervision signals to improve the cross-domain generalization of pathology foundation models. Specifically, the authors first construct triplets containing two image patches from the same slide and one from a different institution. MLLMs are used to evaluate the biological similarity among these patches and select triplets where a cross-domain patch is more similar to one of the same-slide patches than the other. Based on these similarity judgments, the authors introduce a ranking loss and a rank preservation loss to fine-tune existing pathology foundation models using LoRA. This encourages the models to learn embeddings that focus more on biological features rather than domain-specific information. The authors evaluate their method on multiple foundation models, robustness benchmarks, and downstream tasks. The experimental results show that ROPAL improves cross-domain generalization.

Critical strengths
  1. The paper addresses domain bias in pathology foundation models, which is an important challenge for cross-institution generalization. The research problem is well-motivated and has practical relevance.
  2. The authors evaluate ROPAL across multiple foundation models, datasets, and downstream tasks. The extensive comparisons with existing robustification methods provide empirical support for its effectiveness.
  3. The proposed method reduces the reliance on manual annotations and paired multi-scanner data, making it more practical for adapting existing pathology foundation models.
Critical weaknesses
  1. The novelty of this work is limited. Compared with supervised learning methods that rely on manually annotated labels, the authors use MLLMs to assess biological similarity between pathology images and convert these judgments into weak supervision signals for fine-tuning existing pathology foundation models. Moreover, the authors' own ablation results show that the rank preservation loss does not provide a statistically significant improvement over the BT loss alone. Therefore, the core idea of this work is primarily to replace manual annotations with MLLM-generated similarity supervision.
  2. The proposed loss functions may not fully address the problem identified in the Introduction. The authors aim to reduce domain-specific bias while preserving biological information. However, the two loss functions mainly optimize relative similarity rankings and preserve the distance relationships of the original embeddings. Simply strengthening biological features may improve cross-domain similarity rankings without actually reducing domain-specific information in the embeddings. Therefore, the authors should further clarify how these two objectives address the central problem and whether the observed improvements result from reduced domain bias or stronger biological discrimination.
  3. Since the main improvements of the proposed method rely on the supervision signals provided by MLLMs, it is important to understand the impact of their quality. If the similarity judgments were fully correct, would the model achieve better performance? The authors should provide a comparison using oracle triplets constructed from ground-truth labels to evaluate the performance gap between MLLM-generated supervision and ideal supervision.
  4. The Introduction reports experiments on 11 datasets, while Section 4 mentions 12 datasets. The authors should clarify this inconsistency and ensure consistent reporting throughout the paper.
Questions for authors

Please see the Critical Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer WBsH, Part 1

Response to Reviewer WBsH, Part 1

We thank the reviewer for finding the problem well motivated and practically relevant, and for noting the breadth of the comparisons and the reduced reliance on annotations and paired scans. Below is our point-by-point response (this response also addresses Shared.2 in WBsH.1 and WBsH.4, Shared.3 in WBsH.2 and Shared.5 in WBsH.3).

[WBsH.1] Novelty: the work replaces manual annotations with MLLM-generated similarity supervision.

"The novelty of this work is limited. Compared with supervised learning methods that rely on manually annotated labels, the authors use MLLMs to assess biological similarity between pathology images and convert these judgments into weak supervision signals ... Therefore, the core idea of this work is primarily to replace manual annotations with MLLM-generated similarity supervision."

Thank you for this comment. Moving from class labels to relative similarity as the training signal is a non-trivial shift. In computational pathology, manual labels are accurate but expensive, while labels annotated by LLMs are cheap but have limited accuracy [1-4]. This leaves open how LLMs can be used for pathology tasks, a direction with high potential given their promising cross-domain generalizability [5]. To our knowledge, our work is the first to leverage LLMs for a widely recognized problem of pathology foundation models, their limited robustness across institutions. Our key novelty is that the method does not use the LLM's judgement of tile class labels. Instead, it uses the LLM to compare the similarity of tiles. A further contribution is that the training objective is structured by domain, so that it directly targets the strength of biological attributes relative to domain attributes in the embedding and addresses cross-domain generalization directly.

To illustrate the difference between labelling and comparing, we added an experiment in which the LLM directly predicts the class label of each tile, and the odd tile of a triplet is then determined from the predicted labels. On the six labelled benchmark datasets of this paper, the accuracy of the LLM when it compares the tiles exceeds its labelling accuracy by 3 to 34 percentage points and its accuracy when the odd tile is determined from its labels by 15 to 38 percentage points (new Appendix Table 30).

In the revised manuscript, we added the following contribution to the Introduction to highlight this (Section 1).

"We use general-purpose multimodal large language models to compare tiles, not to label them, and turn their judgements of biological similarity into preferences between tiles from different institutions that adapt a pathology foundation model, without tissue labels or paired scanner data."

[WBsH.2] The rank-preservation loss is not significantly better than the BT loss alone.

"the authors' own ablation results show that the rank preservation loss does not provide a statistically significant improvement over the BT loss alone."

Thank you for this comment. A major purpose of the rank preservation term is to regularize training. The BT loss constrains only the cross-domain preferences, so with longer training the adapter keeps moving away from the within-slide structure of the base model, whereas the rank term keeps this structure.

To show this, the revision adds an experiment in which the model is trained with and without the rank term for an increasing number of epochs (Appendix Figure 9 and Table 9). As training continues, the model degrades less with the rank term than without it, so the rank term makes the method less sensitive to the training length.

In the revised manuscript, the Method now states the purpose of the rank term (Section 3.2).

"The Bradley-Terry term constrains only cross-domain preferences. The rank term is a regularizer that keeps the within-slide structure of the base encoder, so that the model degrades less when training runs longer (Appendix A.3.1)."

References

[1] Sun et al. PathMMU: A massive multimodal expert-level benchmark for understanding and reasoning in pathology. ECCV, 2024.
[2] Ferber et al. In-context learning enables multimodal large language models to classify cancer pathology images. Nature Communications, 2024.
[3] Ono et al. Evaluating the efficacy of few-shot learning for GPT-4Vision in neurodegenerative disease histopathology: a comparative analysis with convolutional neural network model. Neuropathology and Applied Neurobiology, 2024.
[4] Lu et al. A multimodal generative AI copilot for human pathology. Nature, 2024.
[5] Zhang et al. Multimodal LLMs outperform pathology foundation models in cross-domain histological similarity. NeurIPS, 2026.

AuthorsOpenReview ↗

Response to Reviewer WBsH, Part 2

Response to Reviewer WBsH, Part 2

[WBsH.3] Do the losses reduce domain-specific information, or only strengthen biological features?

"Simply strengthening biological features may improve cross-domain similarity rankings without actually reducing domain-specific information in the embeddings. Therefore, the authors should further clarify how these two objectives address the central problem and whether the observed improvements result from reduced domain bias or stronger biological discrimination."

Thank you for this question. The key quantity is the strength of the biological signal relative to the strength of the domain signal in the embedding. The lack of robustness of current pathology foundation models comes from their prioritization of domain attributes over biological attributes, as their embeddings are organized more strongly by medical center than by tissue or cancer type, so a tile from the same center is often closer than a tile of the same biology from another center [6, 7]. The BT loss on the domain-structured triplets targets this ratio directly. In each kept triplet the judge says that the tile from the other institution is biologically closer than a tile from the same slide, so the loss magnifies biology-driven variations while it reduces domain-driven variations. In the revised manuscript, we added a paragraph that derives this (Appendix A.1.1, Eq. 8).

"The acquisition gap between the slide and the other institution enters every kept triplet with the same sign, so the BT gradient always shrinks its share of cross-domain distances, while the biological terms move in the direction set by the judge's verdict. [...] ROPAL thus reduces domain-driven similarity without removing it entirely."

Two measurements in the submitted paper reflect how well ROPAL adjusts this ratio. The PathoROB robustness index, the ratio of same-biology other-center to other-biology same-center neighbours, rises on every base (Table 1). In the reverse-confound test, the same-domain bias of the frozen encoders ranges from 0.201 to 0.241 across the three bases, and ROPAL lowers it to 0.050 to 0.086 (Section 4.2.2, Appendix Table 28).

[WBsH.4] Oracle triplets from ground-truth labels.

"If the similarity judgments were fully correct, would the model achieve better performance? The authors should provide a comparison using oracle triplets constructed from ground-truth labels to evaluate the performance gap between MLLM-generated supervision and ideal supervision."

Thank you for suggesting this experiment, which we ran. Training triplets are selected from an expert-labelled dataset either by its labels (oracle) or by the judge, with the same number of triplets, the same objective and the same training schedule. With BEETLE as the training dataset and TIGER as the test dataset, two breast datasets, MOSAIC cross-domain accuracy rises from 0.386 for the frozen encoder to 0.594 with the judge's selections and 0.622 with the oracle's, so MLLM-generated supervision recovers most of the gain of ideal supervision.

[WBsH.5] Eleven versus twelve datasets.

"The Introduction reports experiments on 11 datasets, while Section 4 mentions 12 datasets. The authors should clarify this inconsistency and ensure consistent reporting throughout the paper."

Thank you for the careful reading. The number of datasets is now consistent throughout the paper, and Section 4 reads as follows.

"ROPAL has the best mean rank on all 9 experiments, on all 11 datasets and on all 3 base models"

References

[6] de Jong et al. Current pathology foundation models are unrobust to medical center differences. arXiv:2501.18055, 2025.
[7] Kömen et al. Do histopathological foundation models eliminate batch effects? A comparative study. arXiv:2411.05489 (NeurIPS 2024 AIM-FM workshop), 2024.

I appreciate the authors for addressing my question about supervision quality. Yet, I still see the triplet construction as a modest advance in terms of technique design, and the lower same-center preference could reflect stronger biological discrimination alone. I am keeping my score

Thank you for the follow-up and for the careful reading of our response.

I still see the triplet construction as a modest advance in terms of technique design

We agree that the triplet construction is intentionally simple. The contribution lies in the supervision. With the same triplet rule and training budget, selecting triplets at random, by mutual nearest neighbours or with a pathology vision-language model (CONCH) falls below the MLLM judge on every metric (Table 5), and the judge recovers most of the gain of triplets built from expert labels, as reported in our earlier response (WBsH.4). Our contribution is therefore the finding that role-blind odd-one-out judgments from a general multimodal LLM, which uses no labels, are sufficient supervision to make pathology foundation models more robust.

the lower same-center preference could reflect stronger biological discrimination alone.

We agree that a lower same-center preference by itself cannot separate reduced domain bias from stronger biological discrimination. To examine this, we added two analyses to the revised manuscript (Appendix A.14, Tables 54 and 55). The tables below show ROPAL (base) against the frozen encoder, averaged over the three base models, and the manuscript tables include all baselines.

1. Neighbours of the same class. Among the 20 nearest tiles of the same biological class from other slides, we measured the share from the tile's own institution above chance. The own-institution preference falls on every dataset and base model, although part of it remains. Since all neighbours share the class of the tile, a better separation of the classes does not account for this drop, which is consistent with reduced domain bias.

Same-center preference (excess above chance, in percentage points) within the same biological class

DatasetFrozenROPAL (base)Change
Camelyon1629.724.3−5.4
BEETLE34.625.2−9.4
TumSeg15.912.0−3.9
PathoROB Camelyon44.830.4−14.4
Tolkach ESCA27.215.3−11.9

2. Corresponding tissue locations under different scanners or stainings. SCORPION and PLISM image corresponding tissue locations on different scanners, and PLISM also under different stainings of separate sections. The distance between such copies mostly reflects the acquisition effect. ROPAL brings the copies much closer, while distances between different locations change far less, so the ratio of the two falls on every dataset and base model. This pattern supports reduced sensitivity to acquisition, beyond a better separation of tissue classes.

Change in embedding distance from the frozen encoder to ROPAL (base)

DatasetDistance between copies of a location (gap)Distance between different locations (spread)Gap / spread
SCORPION (5 scanners)−72%−16%−67%
PLISM (across scanners)−63%−2%−62%
PLISM (across stainings)−52%−2%−50%

Taken together, the two analyses indicate that reduced domain bias contributes to the lower same-center preference, alongside any gain in biological discrimination. They do not show that domain information is removed, and part of the same-center preference remains.

Comments on the paper

1 comment

We thank the reviewers for their careful reading and constructive comments. We appreciate that the reviewers find the problem important and well motivated, the label-free, role-blind supervision novel and practical, and the evaluation broad and systematic. The suggestions in the reviews have further improved our work. Following the reviewers' suggestions, we added a total of 13 experiments and 18 tables and figures. We also implemented a feature suggested by one reviewer on our method, which further increased our method's lead over the baselines. Below, we summarize the points shared among the reviewers, followed by a point-by-point response to each individual reviewer.

  • Shared.1: Baselines must be properly trained (the collapsed Pathology DoRA runs, its short budget, the AdvInv settings).
    • Addressed in: td9B.1, td9B.2, td9B.8, QQ5p.3, QQ5p.7
    • How it is addressed: We tuned the learning rate of Pathology DoRA to prevent collapse and increased its training budget to match ROPAL's, so no collapsed run remains in any table or ranking. DoRA is now the strongest baseline, and the paper states where it beats ROPAL. AdvInv is now trained with the full training schedule, and all its results are updated.
  • Shared.2: Is the MLLM supervision reliable, and is an MLLM needed at all?
    • Addressed in: WBsH.1, WBsH.4, td9B.3, td9B.9, td9B.10, QQ5p.1, QQ5p.6
    • How it is addressed: New matched controls with the same rule, counts, objective and budget, namely CONCH as the judge (image similarity and zero-shot labels), a noisy judge and a second-judge filter, and an oracle built from expert labels, which we report in this rebuttal. The MLLM judge is best on every task and base, it recovers most of the gain of the oracle, and performance degrades gradually with injected errors.
  • Shared.3: Is the rank-preservation term necessary?
    • Addressed in: WBsH.2, QQ5p.4, QQ5p.8
    • How it is addressed: The term is a training regularizer that keeps the within-slide structure of the base model. A new training-length experiment shows that the model degrades less with it as training continues.
  • Shared.4: Uncertainty from training seeds and from clustered evaluation units.
    • Addressed in: td9B.6, td9B.7, QQ5p.5
    • How it is addressed: Every SD and significance mark now comes from a bootstrap over clusters (tissue locations, slides, cases, biopsies). Training seeds are reported for the matched setting (Appendix A.5.5) and for the full-corpus checkpoints of the main tables (Appendix Table 46).
  • Shared.5: Robustness gains versus downstream utility.
    • Addressed in: WBsH.3, td9B.4, td9B.5, td9B.11, QQ5p.2
    • How it is addressed: The robustness and confounding benchmarks are themselves important clinical benchmarks that reflect real-world deployment. ROPAL is the best method on average on all nine evaluation tasks, and the new variant ROPAL (Reinhard) is the best method on every PLISM scanner-only pair, the limitation case of the submission.
  • Shared.6: Two new variants of ROPAL.
    • Addressed in: td9B.5, td9B.11
    • How it is addressed: Following a suggestion of Reviewer td9B, we implemented two variants of our method, ROPAL (Reinhard), which embeds Reinhard-normalized tiles with the ROPAL checkpoint, and ROPAL (DoRA), which trains ROPAL on top of the DoRA-adapted encoder. The original method is now named ROPAL (base). The variants are evaluated on all tasks and reported in the result tables and figures throughout the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.