- Rating
- 3(+1)
- Confidence
- 4(+2)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes ROPAL, a label-free adaptation framework intended to improve the acquisition robustness of pathology foundation models. The central idea is to use a multimodal language model as a biological similarity judge. The authors construct triplets containing two patches from the same slide and one patch from another institution. A triplet is retained when the judge identifies one of the same-slide patches as the odd one out, thereby providing a cross-domain similarity preference.
The selected preferences supervise parameter-efficient adaptation through a Bradley–Terry ranking objective, complemented by a regularization term intended to preserve within-slide representation structure.
The method is evaluated on three pathology foundation models across multiple benchmarks involving cross-institution retrieval, scanner and staining invariance, spatial gene-expression prediction, mitotic figure classification, and prostate grading. The authors report substantial improvements in several cross-domain similarity and retrieval tasks, along with more heterogeneous improvements on downstream classification tasks.
Critical strengths
- A relevant robustness problem and a practical adaptation strategy.
The paper addresses acquisition-related confounding in pathology foundation model embeddings, an important challenge for cross-institution generalization. Using a multimodal model to obtain relative similarity preferences without manual diagnostic labels is a potentially useful approach.
The method is parameter-efficient, and the adapted encoder can be used independently at inference time without further calls to the multimodal judge.
- Encouraging cross-domain results across multiple evaluation settings.
The experiments cover three foundation models and several benchmarks evaluating different sources of domain shift. In particular, the improvements in MOSAIC, SCORPION, and PLISM indicate that the proposed adaptation can meaningfully alter embedding relationships across institutions and acquisition conditions.
The inclusion of judge-selection controls, reverse-confounding analysis, and several comparisons with established adaptation approaches provides useful empirical context.
Critical weaknesses
-
Insufficient validation of biological similarity supervision. The reliance on a multimodal language model for biological similarity supervision (with ~79% average accuracy and high variance) lacks systematic expert pathologist verification for the 60k+ retained triplets. Furthermore, the main MOSAIC benchmark closely mirrors the preference training objective, whereas downstream gains on independent predictive tasks (e.g., MIDOG++, cross-site PANDA) are heterogeneous and base-encoder dependent. The authors must distinguish improvements in cross-domain preference geometry from true generalizable clinical utility.
-
Flawed baseline optimization and unverified loss components. Main comparisons rely on compromised baselines, retaining runs where Pathology DoRA collapsed (rank ~1.0 on Virchow2, ~4.8 on GenBio-PathFM) under a severely truncated training schedule (1,100 steps, ~0.09 epochs), which fails to reflect properly tuned adaptation approaches. Additionally, the necessity of the rank-preservation objective is unproven, as Table 6 shows the Bradley–Terry-only variant performing comparably or better with no statistically significant advantage for the full objective.
-
Flawed statistical sampling and uncertainty estimation. Uncertainty estimates rely on patch- or query-level bootstrapping, ignoring non-independent acquisition structures (e.g., repeated tissue locations across scanners in SCORPION). Evaluating via hierarchical bootstrapping at the patient, slide, or site level, along with reporting multi-run variance across seeds, is essential to establish reliable comparative claims.
Questions for authors
- Can the authors clarify the independent contribution of biological judgment quality?
How do the results change when judge predictions are corrupted, when low-confidence judgments are removed, or when triplets are selected using more conventional morphological similarity measures under matched training budgets?
- Can the collapsed baseline configurations be rerun with stable training settings?
The reported Pathology DoRA configurations collapse for two foundation models. Could the authors provide comparisons using a stable, appropriately tuned implementation and a more comparable optimization budget?
It would also be useful to distinguish results from successfully trained baselines when reporting aggregate method rankings.
- Is the rank-preservation regularizer necessary?
Table 6 indicates that the Bradley–Terry-only objective is competitive with the full method and performs better on several metrics. Could the authors clarify the specific failure mode that the additional regularization is intended to address, and provide quantitative evidence that it preserves biologically relevant within-domain structure?
6 comments
Response to Reviewer QQ5p, Part 2
Response to Reviewer QQ5p, Part 2
[QQ5p.2] MOSAIC mirrors the training objective, and downstream gains are heterogeneous.
"the main MOSAIC benchmark closely mirrors the preference training objective, whereas downstream gains on independent predictive tasks (e.g., MIDOG++, cross-site PANDA) are heterogeneous and base-encoder dependent. The authors must distinguish improvements in cross-domain preference geometry from true generalizable clinical utility."
Thank you for this comment. We agree that MOSAIC and our training objective both directly target the prioritization of biological signals over domain attributes. Like PathoROB, MOSAIC is included because it isolates the evaluation of robustness from other characteristics of a model that may contribute to its performance on downstream tasks. However, MOSAIC is only one of the many benchmarks we performed. In total, we evaluate four cross-domain generalization tasks (PathoROB, MOSAIC, SCORPION, PLISM) and five downstream clinical tasks (cross-institution retrieval and classification, HEST, MIDOG++, PANDA), and ROPAL is the best method on average on all 9 of them (Figure 2).
Following your suggestion, to better distinguish improvements on the cross-domain generalization tasks from those on the downstream clinical tasks, the revised Figure 2 marks the category of each experiment, and the revised manuscript defines the two categories and reports them separately (Appendix Figure 15).
"The experiments fall into two categories. The cross-domain generalization tasks (PathoROB, MOSAIC, SCORPION and PLISM) test directly whether embeddings follow biology across institutions, scanners and stainings. The downstream clinical tasks (cross-institution retrieval and classification, HEST, MIDOG++ and PANDA) test whether this robustness carries over to clinical prediction and retrieval tasks. On the cross-domain generalization tasks, ROPAL or one of its variants ranks first in 34 of the 36 result rows. On the downstream clinical tasks, ROPAL or one of its variants ranks first in 66 of the 81 result rows."
The new Appendix Figure 15 also splits each base model into the two categories, and ROPAL is the best method on all six combinations of base model and category.
Regarding performance on downstream tasks, on MIDOG++, averaged over the three base models, ROPAL is the only one of the seven methods that improves on the frozen encoder when tumor types are held out, and when scanners are held out its gain over the frozen encoder is about twice that of the best baseline. A new Figure 6 in the revised manuscript shows these gains for every method (Section 4.4.3). On cross-site PANDA, ROPAL is the only method that improves on the frozen encoder for all three base models, and averaged over them its gain is about twice that of the best baseline. This is shown in the new Average row of Table 4 and in a new Appendix Figure 16 (Section 4.4.4).
On the other downstream tasks, the lead of ROPAL is clearer. In cross-institution classification under institution-class confounding, ROPAL has the highest accuracy on all 6 dataset and base pairs, with a mean gain of 18.7 points over the frozen encoder (Figure 5). In cross-institution retrieval, it has the highest mAP on all 3 datasets when averaged over the base models (Table 2), and on HEST, which predicts gene expression from H&E images, it gives the highest correlation on all three base models, each significantly under the cluster bootstrap (Table 3).
[QQ5p.3] Collapsed Pathology DoRA runs under a severely truncated schedule.
"Main comparisons rely on compromised baselines, retaining runs where Pathology DoRA collapsed (rank ~1.0 on Virchow2, ~4.8 on GenBio-PathFM) under a severely truncated training schedule (1,100 steps, ~0.09 epochs), which fails to reflect properly tuned adaptation approaches."
Thank you for this point. We fixed both issues by tuning DoRA's learning rate so that no run collapses and by matching its training budget to ROPAL's (td9B.1, Appendix A.12, Appendix Table 53). All the DoRA results in the revised manuscript have been updated, and DoRA is now the leading baseline, while the ranking of ROPAL is largely unchanged (td9B.1).
Response to Reviewer QQ5p, Part 3
Response to Reviewer QQ5p, Part 3
[QQ5p.4] The necessity of the rank-preservation objective is unproven.
"the necessity of the rank-preservation objective is unproven, as Table 6 shows the Bradley–Terry-only variant performing comparably or better with no statistically significant advantage for the full objective."
Thank you for this comment. A major purpose of the rank preservation term is to regularize training by keeping the within-slide structure of the base model, which the BT loss does not constrain. In a new experiment that trains the model with and without the rank term for an increasing number of epochs, the model degrades less with the rank term than without it as training continues (WBsH.2, Appendix Figure 9 and Table 9).
The Method of the revised manuscript now states this purpose (Section 3.2).
"The Bradley-Terry term constrains only cross-domain preferences. The rank term is a regularizer that keeps the within-slide structure of the base encoder, so that the model degrades less when training runs longer (Appendix A.3.1)."
[QQ5p.5] Patch- or query-level bootstrap ignores the acquisition structure, and training seeds.
"Uncertainty estimates rely on patch- or query-level bootstrapping, ignoring non-independent acquisition structures (e.g., repeated tissue locations across scanners in SCORPION). Evaluating via hierarchical bootstrapping at the patient, slide, or site level, along with reporting multi-run variance across seeds, is essential to establish reliable comparative claims."
Thank you for this suggestion. We agree that the bootstrap should respect the acquisition structure, and the revision replaces the bootstrap over individual samples by a cluster bootstrap. From the saved per-sample outputs, without rerunning any model, we resample whole clusters, namely tissue locations for SCORPION, slides for PLISM, PathoROB and HEST, reference slides for MOSAIC, query slides for the cross-institution tasks, cases for MIDOG++ and biopsies for PANDA. Every SD in the SD Appendix, every error bar in the figures and every significance mark now comes from this cluster bootstrap, and each caption names its cluster unit (Appendix A.7).
A new Appendix Table 35 compares the number of significant leads of ROPAL before and after this change, on the same results. Of the 51 leads significant under the bootstrap of the submission, 46 (90%) remain significant under the cluster bootstrap, and the total changes from 51 to 59. The Appendix now lists the significant leads of each experiment (Appendix A.7), and training-seed variance is reported separately (td9B.7).
[QQ5p.6] (Q1) Independent contribution of judgement quality.
"How do the results change when judge predictions are corrupted, when low-confidence judgments are removed, or when triplets are selected using more conventional morphological similarity measures under matched training budgets?"
Thank you for these suggestions. We added experiments for all three, each in the matched setting of the other ablations.
- Corrupted judge predictions. Noisy judge (p%) replaces p% of the triplets selected by the judge with judged triplets that it did not select. Performance degrades gradually as the noise rate grows, on both base models, consistent with judge quality mattering (new Appendix Table 10, Appendix A.3.1).
- Low-confidence judgements removed. High-confidence judge keeps only the selections that a second MLLM judge agrees with. Removing low-confidence judgements gives a moderate gain on most tasks (new Appendix Table 10).
- Conventional similarity selection. MNN and Odd-one-out score, which select triplets by frozen-encoder similarity, and CONCH image similarity all fall below the judge (Table 5, Appendix Table 21).
[QQ5p.7] (Q2) Stable DoRA with a comparable budget, and rankings without collapsed baselines.
"Could the authors provide comparisons using a stable, appropriately tuned implementation and a more comparable optimization budget? It would also be useful to distinguish results from successfully trained baselines when reporting aggregate method rankings."
Thank you for these suggestions. We further tuned both baselines. For Pathology DoRA, we tuned the learning rate and matched its optimization budget to that of our method, and for AdvInv, we changed the training schedule to the full schedule. No run collapses any more, and the results of both baselines are updated in the revised manuscript. The ranking of our method remains largely unchanged (td9B.1, td9B.2, Figure 2).
Response to Reviewer QQ5p, Part 1
Response to Reviewer QQ5p, Part 1
We thank the reviewer for finding the robustness problem relevant and the adaptation strategy practical, and for noting the judge-selection controls and reverse-confound analysis. Below is our point-by-point response (this response also addresses Shared.1 in QQ5p.3 and QQ5p.7, Shared.2 in QQ5p.1 and QQ5p.6, Shared.3 in QQ5p.4 and QQ5p.8, Shared.4 in QQ5p.5 and Shared.5 in QQ5p.2).
[QQ5p.1] The biological similarity supervision lacks expert pathologist verification.
Thank you for this comment. We treat the judge's verdicts as a training signal rather than as ground truth, in the same way self-supervised pretraining treats its teacher. The base models we adapt (UNI2-h, Virchow2, GenBio-PathFM) were themselves trained by teacher-student self-distillation, whose targets (prototype assignments, masked-token predictions) are never verified by pathologists. Their value is established by the usefulness of the resulting embeddings. The judge plays the same role, as it supplies relative similarity preferences, and what matters is whether training on them improves the embeddings, which we evaluate on eleven datasets.
In the revised manuscript, we added the following sentences to the Method to clarify this (Section 3.1).