acceptodds
Withdrawn from ICLR 2027

Contamination, Prior, or Evidence? Decomposing and Training Evidence Use in Whole-Slide Vision–Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pathology vision-language models (VLMs) are conventionally evaluated by accuracy, but accuracy alone does not measure evidence use: it may conflate dataset contamination, prior knowledge, and image evidence. In a motivating study of lymph-node metastasis prediction, we found that most public pathology VLMs showed minimal differences when changing from feeding the models with whole-slide images, an annotated lesion, or no image at all. To better understand the specific features leveraged by these models, this paper presents two contributions aimed at disentangling these factors. First, we present CleanSlide, a TCGA-based VQA benchmark designed to eliminate image- and question-side contamination. It contains 149K audited multiple-choice questions over 9,985 slides, with patient- and tissue-source-disjoint splits. Every question is audited for option shortcuts, stem leakage, cross-split duplication, and blind solvability. Second, we propose Pair-DPO, a preference loss over counterfactual slide pairs from the same question and source. By controlling for shared confounding factors, Pair-DPO cancels out the question-attributable signal and leaves image evidence as the source of preference. Specifically, each pair consists of two real slides with opposite, verified findings, introducing neither editing artifacts nor unverified labels for diffuse or graded features such as invasion, necrosis, and tumor grade. Experiments show that our method gains 15.29% from image evidence on the CleanSlide, compared with 2.81% for the best published model. On the external CPTAC and BCNB cohorts, our method achieves accuracies of 57.6% and 59.0%, outperforming all other evaluated models by 9.7% and 3.4%, respectively. We will release the benchmark and code.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
computational pathology, vision-language models, benchmark contamination, counterfactual evaluation, preference optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_prNyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper finds that most pathology VLMs respond the same way whether given a blank image, the whole slide, or just the lesion. It introduces a TCGA benchmark that uses patient- and site-disjoint splits, question audits, and extra evaluation steps to filter out questions that a text-only model could answer. The authors also present Pair-DPO, a method that compares two real slides from the same cancer type and site but with opposite findings for the same answer, and uses this approach to train a slide-level VLM.

Critical strengths
  • the blank vs slide vs lesion test is simple and interesitng
  • site and patient disjoint splits, blind scores next to every accuracy
  • using real same-site slide pairs is neat
  • good controls, step-matched ablations and SlideChat same on the same data
  • lambda picked on validation
Critical weaknesses
  • fixing the answer and using contrasting images within DPO is called V-DPO (Findings of EMNLP 2024, pp. 13258-73). What does your approach add beyond using real slides?
  • MediConfusion (ICLR 2025) asks the same question on two images with opposite answers. You might consider reporting pair accuracy as they do.
  • PathBench (TMI 2025;44(10):4087-97) already showed that pathology LMMs can answer MCQs without looking at the image. You list this in Figure 5 but do not discuss it.
  • for site-disjoint splits, you should cite Howard et al. (Nat Commun 2021;12:4423).
  • The Layer-C filter uses blind runs of SlideChat and Patho-R1, and then you evaluate those same models on it. SlideChat scores below chance. This introduces bias.
  • the 15.29% gain from image evidence in the abstract refers to the EWA score, not an actual gain. The raw gain is much lower.
  • Table 2, the EWA confidence intervals for Pair-DPO and CE overlap, and only Sens separates them. You mention using three seeds, but where is the spread shown?
  • Differences between the slides, such as tumour fraction, size, and stage, still correlate with the finding and separate the pair.
  • for the pathologist review, how many reviewers were involved and what was their agreement rate? Only abou 80% of cases were kept?
  • 25% option shortcut is chance performance for four options.

Xie Y, Li G, Xu X, Kan MY. V-dpo: Mitigating hallucination in large vision language models via vision-guided direct preference optimization. InFindings of the Association for Computational Linguistics: EMNLP 2024 2024 Nov (pp. 13258-13273).

Sepehri MS, Fabian Z, Soltanolkotabi M, Soltanolkotabi M. Mediconfusion: Can you trust your ai radiologist? probing the reliability of multimodal medical foundation models. InInternational Conference on Learning Representations 2025 May 1 (Vol. 2025, pp. 72145-72169).

Sun Y, Wu H, Zhu C, Si Y, Chen Q, Zhang Y, Zhang K, Li J, Cai J, Wang Y, Sun L. Pathbench: Advancing the benchmark of large multimodal models for pathology image understanding at patch and whole slide level. IEEE transactions on medical imaging. 2025 Jul 2;44(10):4087-97.

Howard FM, Dolezal J, Kochanny S, Schulte J, Chen H, Heij L, Huo D, Nanda R, Olopade OI, Kather JN, Cipriani N. The impact of site-specific digital histology signatures on deep learning model accuracy and bias. Nature communications. 2021 Jul 20;12(1):4423.

Questions for authors

see above

3 comments

Thank you for your detailed comments. Before the point-by-point responses, we would like to briefly clarify why pathology differs from natural images. Methods that reduce reliance on language priors (e.g., mDPO, V-DPO) assume that (1) the image can be edited to create a counterfactual, (2) the prior is a nuisance to remove, and (3) the question concerns the presence of an object. None of these holds in pathology: a finding such as necrosis or a higher grade cannot be faithfully added or flipped, clinical priors reflect real disease epidemiology, and questions concern graded findings such as invasion, necrosis, and grade. Our paper therefore measures the prior with a blind score, and trains on real, report-verified counterfactual slide pairs.

W1. Relation to V-DPO

V-DPO (Xie et al., 2024) and mDPO (Wang et al., 2024a) both keep the response fixed and contrast two images, and Sec. 4.2 states that our loss does so “as in mDPO.” We now cite V-DPO next to it. The difference is where the contrasting image comes from. V-DPO builds its image-contrast pairs by editing images with Stable Diffusion, and mDPO by randomly cropping the image. However, this is not available for reporting findings in pathology because they cannot be faithfully inserted or removed by editing a slide. Pair-DPO therefore uses a real slide from another patient with the same cancer type and tissue source site and an opposite, report-verified finding. In Table 2, the synthetic variant (mDPO, which removes pathology-related tiles) reaches EWA 13.94 against 15.29 for real pairs.

W2. MediConfusion and pair accuracy

We now cite MediConfusion in Sec. 2 and report its metrics on our counterfactual pairs. We randomly sampled 306 pairs from the CleanSlide paired test. The two slides of a pair come from different patients with the same cancer type and tissue source site, and have opposite answers for one report-verified finding. Following MediConfusion, set accuracy is the share of pairs with both slides answered correctly, and individual accuracy is the share of single slides answered correctly. Each answer is the option letter with the highest next-token probability.

ModelSet acc. (%)Indiv. acc. (%)
SlideChat, published2.9436.93
+ CE control20.5944.44
+ Pair-DPO22.5547.55

The published SlideChat answers both slides of a pair correctly in only 2.94% of pairs. This pattern is consistent with MediConfusion’s observation that models tend to give the same answer to both images of a pair. Both models trained on CleanSlide perform better, reaching a set accuracy of 20.59% with the CE control and 22.55% with Pair-DPO. Among the three models, Pair-DPO achieves the highest set and individual accuracy.

W3 & W4. PathBench and Howard et al.

Thank you for pointing out these two missing references. We now cite and discuss both in Sec. 2, and both findings motivate the design of CleanSlide. PathBench showed that pathology LMMs can answer multiple-choice questions without the image. CleanSlide builds on this observation: it reports a blind score next to every accuracy, so that the prior and the image contribution can be separated for each model, and it adds a paired counterfactual test. Howard et al. (2021) showed that site-specific signatures in TCGA slides bias model accuracy. This is the reason CleanSlide splits by tissue source site as well as by patient.

W5. Layer-C

We rebuilt Layer-C from the three members that Table 1 does not evaluate: the text-only LLMs Qwen3-32B, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-7B. The revised subset keeps 10,465 questions. Qwen2.5-7B-Instruct is the language model of PairSlide, so this screen, if anything, works against our model.

Layer-C accuracy (%)QuestionsConstant answerSlideChatPatho-R1WSI-LLaVASlideChat*PairSlide
Submitted screen, 7 blind models2,50138.668.7616.0330.7642.8646.78
Revised screen, 3 text-only LLMs10,46529.1549.4732.5242.9664.9868.39

On the revised Layer-C, PairSlide still has the highest accuracy. The revised screen contains no pathology model, so the revised Layer-C removes only the questions that a text-only LLM can answer. In the revision, Table 1 reports the revised Layer-C for every model, and we give the new definition of Layer-C accordingly.

W6. 15.29% gain in the abstract

Thank you for pointing this out.15.29 is the evidence-weighted accuracy (EWA) of PairSlide. The raw gain, Layer-B accuracy with the slide minus the blind score, is 9.59 points for PairSlide (67.35 vs. 57.76) and 3.71 points for SlideChat (44.72 vs. 41.01). We revise the abstract and Sec. 6.

W7. Seed spread for Table 2

The intervals in Table 2 are patient-level bootstrap intervals for one seed of each arm. The EWA of the three seeds is:

EWACE controlPair-DPO
Seed 012.5515.29
Seed 112.2015.45
Seed 213.3115.22

Every Pair-DPO seed is above every CE seed, and the difference in means is 2.63. We have added this table to the revised paper.

W8. Residual differences between paired slides

Pair-DPO cancels what the two slides of a pair share, but the two slides can still differ in finer-grained properties that correlate with the finding, such as tumour fraction, size, and stage. So we ran a masking experiment to test whether the answer depends on the region that carries the finding. If it does, removing that region should change the answer, while removing the same number of other tiles should not.

Here, we use CAMELYON16, a public set of breast-cancer sentinel lymph-node slides in which pathologists outlined every metastasis. Every slide is asked one four-option question, “Does this sentinel lymph node section contain metastatic breast carcinoma?”, with “Yes” correct for metastatic slides and “No” for normal ones. We trained two models from our SFT checkpoint, one with CE and one with Pair-DPO, where a pair is one metastatic and one normal slide asked the same question. We then compare four interventions on the test slides: removing the expert-annotated lesion, removing the same number of random non-lesion tiles, removing the same number of non-lesion tiles that CONCH ranks as most tumour-like, and keeping only the annotated lesion.

CAMELYON16, accuracy (%)CE controlPair-DPO
Full slide45.1248.35
Annotated lesion removed (expert-annotated)40.57 (−4.55)41.64 (−6.71)
Only the annotated lesion kept64.61 (+19.50)77.50 (+29.15)
Same number of random non-lesion tiles removed45.66 (+0.54)48.92 (+0.57)
Same number of most tumour-like non-lesion tiles removed (CONCH)45.54 (+0.42)48.60 (+0.25)

Removing the annotated lesion lowers the accuracy of Pair-DPO by 6.71 points, whereas removing the same number of non-lesion tiles changes it by at most 0.57 points, even when these are the most tumour-like tiles. The effect is also larger for Pair-DPO than for the CE control, both when the lesion is removed (−6.71 vs. −4.55) and when only the lesion is kept (+29.15 vs. +19.50). On this task, the answer therefore responds to the target region and not to other tissue of equal area.

W9. Pathologist review

Two pathologists audited a stratified sample of 80 questions against their slides. Both rated every question independently, and they agree for 86.25% of questions. Table 5 shows where it comes from. The text-level criteria are at or near 100%: no stem answers away, the wording is sound, and the distractors are plausible for 96% of questions. The losses are on clinical validity, from two sources. About 10% of questions can be answered from the text alone, which is the prior that CleanSlide measures with the blind score. About 8% cannot be answered from the slide, and these concentrate in staging, molecular subtype, and treatment, whose answers come from the clinical record and not from the H&E slide. We also correct and specify the scope of the review in the revised paper.

W10. Option-shortcut rate

The reviewer is right that 25% is the chance for four options, and this is what the value means. Under our definition, 25% is the lowest possible option-shortcut rate, reached when the options carry no cue to the answer. Under the same definition, WSI-Bench scores 49% and SlideBench 35%. We give a clear definition in Sec. 3.1 in the revised paper.

We hope these responses address your concerns and would be grateful if you could reconsider your score. We are happy to provide further clarification.

Reviewer_prNyOpenReview ↗

Thanks for the rebuttal

Thanks for the rebuttal and the new experiments. Many points were resolved. I keep my score for the following reasons:

  • Layer-B is screened with the same blind models that are then evaluated on it. Re Table 1, three of them have larger raw gains than PairSlide
  • The pathologist review covers 80 imported questions. The generated tier, which supplies the Pair-DPO labels, was not reviewed.
  • The effect of the loss is small.
  • The CAMELYON masking test is on a yes/no finding, an the accuracy is below of the majority-class baseline. It does not address the tumour-fraction parts in the TCGA pairs for diffuse findings.
Reviewer_UwvWOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

When pathology vision-language models answer questions correctly, are they actually examining the slides, or relying on medical knowledge, textual cues, or memorized training data? To investigate this question, the authors introduce the CleanSlide benchmark, the Pair-DPO training method, and PairSlide, a model trained using this method.

Critical strengths
  1. The paper is well motivated and addresses an important issue in pathology vision-language models: high answer accuracy does not necessarily indicate effective use of image evidence. The paper substantiates its motivation through experiments comparing predictions under three input conditions: blank images, whole-slide images, and manually annotated lesion regions. Distinguishing visual evidence from clinical priors and potential data contamination provides a valuable perspective for evaluating these models.

  2. The paper presents a comprehensive benchmark and a clearly defined training strategy. CleanSlide is a valuable benchmarking contribution, comprising approximately 149K questions across 9,985 slides and 32 cancer types. Its patient- and tissue-source-site-disjoint splits, question-level audits, and comparisons against blind baselines enable a more comprehensive evaluation than reporting accuracy alone. Pair-DPO also offers a practical approach to improving image use by training on real slide pairs with opposing findings verified through pathology reports, avoiding the difficulty of synthesizing reliable counterfactual pathology images.

Critical weaknesses
  1. The comparisons in Table 1 involve different training conditions, making it difficult to attribute the performance gains. Most baseline models are evaluated using their published checkpoints without additional training on CleanSlide, whereas PairSlide is trained on this dataset. Although the retrained SlideChat* baseline and the matched-step ablations partially address this concern, the improvements over published checkpoints may still reflect adaptation to the dataset and question distribution rather than the benefits of Pair-DPO itself.

  2. Pair-DPO can cancel shared textual priors, but this does not guarantee that the model learns the target pathological features. The paired slides come from different patients and may differ in tissue composition, tumor proportion, and slide quality, even when cancer type and tissue source site are matched. Equation (4) ensures that shared reward components cancel in the difference, but it does not rule out the use of other visual cues correlated with the labels. Therefore, the argument in Section 4.2 that the loss gradient acts entirely on tissue differences and thereby encourages the model to read the target pathological evidence is overstated. The sensitivity and directional accuracy results support improved responsiveness to image changes, but do not establish whether these responses are driven by target features such as necrosis or invasion. Additional validation using target-region masking or expert-annotated regions would strengthen the central claim. For example, masking the target region should affect the corresponding answer more strongly than masking a non-target region of equal area.

Questions for authors

Please address the questions raised in the weaknesses above.

1 comment

We thank the reviewer for the careful assessment and suggestions. We address your concerns with two new experiments and report them below.

W1. Different training conditions in Table 1

To separate the contribution of the CleanSlide benchmark and the pair loss, we fine-tuned three more published models on the CleanSlide training split, in addition to SlideChat.

Model fine-tuned on CleanSlideLayer-B acc. (%)Blind (%)EWA
SlideChat*62.7056.129.40
Patho-R1-7B*46.0340.664.16
Quilt-LLaVA*44.3641.122.46
WSI-LLaVA*52.5248.424.17
PairSlide67.3557.6715.29

Compared with their published checkpoints in Table 1, fine-tuning raises the Layer-B accuracy of these models, but their blind scores rise with it. Their EWA therefore stays between 2.46 and 9.40, against 15.29 for PairSlide. Adaptation to the dataset alone does not produce the gain in evidence use. The effect of the loss itself is isolated in Table 2, where the CE control and Pair-DPO start from the same SFT base and use the same data and number of steps.

Furthermore, Table 3 evaluates on two external cohorts, CPTAC and BCNB, which are not part of CleanSlide. PairSlide has the highest accuracy among the ten medical and pathology models there (57.64% and 58.98%), so its advantage is not limited to the CleanSlide distribution.

W2. Whether the model learns the target finding

We softened the statement in Sec. 4.2 accordingly. To test the point empirically, we ran the masking experiment you proposed.

We use CAMELYON16, a public set of breast-cancer sentinel lymph-node slides where pathologists outlined every metastasis. Every slide is asked one four-option question, "Does this sentinel lymph node section contain metastatic breast carcinoma?", with "Yes" correct for metastatic slides and "No" for normal ones. We trained the models with the CE and Pair-DPO from our SFT checkpoint. A Pair-DPO pair is one metastatic and one normal slide asked the same question. We compare four interventions: removing the expert-annotated lesion, removing the same number of random non-lesion tiles, removing the same number of non-lesion tiles that CONCH ranks as most tumour-like, and keeping only the annotated lesion.

CAMELYON16, accuracy (%)CE controlPair-DPO
Full slide45.1248.35
Annotated lesion removed (expert-annotated)40.57 (−4.55)41.64 (−6.71)
Only the annotated lesion kept64.61 (+19.50)77.50 (+29.15)
Same number of random non-lesion tiles removed45.66 (+0.54)48.92 (+0.57)
Same number of most tumour-like non-lesion tiles removed (CONCH)45.54 (+0.42)48.60 (+0.25)

As shown here, removing the annotated lesion lowers the accuracy of Pair-DPO by 6.71 points. Removing an equal number of non-lesion tiles changes it by at most 0.57 points, even when those tiles are the most tumour-like ones. Therefore, masking the target region affects the answer more than masking a non-target region of equal area. The dependence on the lesion is also stronger for Pair-DPO than for the CE control, both when the lesion is removed (−6.71 vs. −4.55) and when only the lesion is kept (+29.15 vs. +19.50).

We hope these results address your concerns, and we are happy to provide further clarification if needed.

Reviewer_som8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

this work tackles a common issue in mllm, hallucination. in particular, authors focus on the case where models discard the impact of the image and use only text input with application to wsi image analysis. first, authors propose to construct a new benchmark, cleanslide, with multiple questions/wsi based on tcga, existing questions from other benchmarks, and generated questions. questions are then reviewed by pathologists. this dataset is used in this work for training. authors also proposed new evaluations cases and 3 metrics. in addition, a variant of DPO loss is considered based on pairs of wsi. results are reported over cleanslide test set, in comparison to other models. in addition, test over 2 other sets. authors mentioned that the dataset and code will be released.

Critical strengths
  • the writing is good.
  • the paper tackles an important issue, hallucination in mllms, for histology applications.
  • results and comparisons are reported.
Critical weaknesses

method:

  • no major contribution. the paper is composed of 3 elements that are packaged under the form of a paper with very limited novelty overall. each element contains a minor contribution:

a- a benchmark: it is all based on existing material (wsi from tcga, questions fro existing benchmarks, in addition to generated questions), which on its own drastically limits its novelty. the introduction of this benchmark was not justified in the paper. in the paper, suddenly, it was introduced (abstract - line019, introduction - line093). why it is introduced remains unclear. also, there was no mention to how many pathologists check the questions. in all instances, authors used 'pathologistS'. dataset publication requires a separate dedicated paper with much more details than 1 paragraph as done here (sec.3.1).

b- authors decided to change the evaluation and metrics without justification. they introduced 3 new evaluation cases and called them layers (sec.3.2). they were given as facts. then, provide 3 new metrics, also as facts.

c- minor change of dpo with fixed question on 2 different wsi with same cancer type while their reports give different finding (not clear why reports give different finding). how many pairs fits these conditions in the new datasets and does this extend to standard wsi datasets remains unclear.

  • why the dataset and code are not released yet? or included in the submission? why they are released in the future? what is preventing the release?

experiments:

  • the comparisons in this work (tab.1,3) do not seem fair: a- models operate on different levels (tiles+ majority voting vs wsi). b- specialized models in histology vs generic models on natural scene images. c- zero-shot vs finetuned evaluation (slidechat* + pairslide are finetuned). c- it is not clear how the proposed model (pairslide) was trained in terms of model selection criteria: if the 3 new metrics are used for model selection, this already gives it an edge to max out the test performance. other models were not necessarily trained to max these new metrics. d- not clear how slidechat* was finetuned.
  • no explanation on how the hyperparameters of the proposed model are found (sec.5.1).

overall, the paper is composed of 3 elements with very limited to questionable novelty. in addition, results and comparison fairness are also in question.

please stop using ai.

Questions for authors

please see weaknesses.

2 comments

Thank you for your comments. Before the point-by-point responses, we would like to briefly introduce the background of this topic. Many slide-level pathology VQA benchmarks are built on TCGA cases. A high accuracy therefore does not necessarily show that a model has read the slide. It may come from dataset contamination or prior knowledge. Our paper measures this with a contamination-controlled benchmark and a blind score next to every accuracy, and improves it by training on real slide pairs with opposite findings.

W1. Benchmark

Building on TCGA is the common practice in this field. SlideBench [1], WSI-Bench [2], PathGen-1.6M [3], and WSI-VQA [4] are all constructed from TCGA, which offers public slides together with pathology reports. This shared source is also the motivation for our benchmark. In Sec. 3, we report the case overlap and site overlap in this commonly used benchmark. Removing this overlap requires re-splitting and re-auditing the existing questions. So CleanSlide splits the patient and tissue source site, performs four audits of every question. Two pathologists independently audited a stratified sample of 80 questions against their slides and agreed on 86.25% of them. The revised Sec. 3.1 states this and describes the review.

W2. Evaluation and metrics

Existing slide-level benchmarks primarily report accuracy [1, 2, 4]. We are motivated by the experiment in Figure 1: most models give the same prediction whether they receive a blank image, the whole slide, or the annotated lesion, so accuracy alone cannot tell whether a model has read the slide. But the standard evaluation is kept. Layer-A is the conventional benchmark accuracy and is reported for every model, and Layers B and C are subsets of it that progressively remove questions answerable without the slide. Among the metrics, EWA is the one we introduce as new, because the raw gain over the blind score does not account for the headroom left by the prior. With these metrics, Table 1 shows what accuracy alone hides, and our model achieves: no published model exceeds an EWA of 4.32, while PairSlide reaches 15.29.

W3. Pair-DPO

Image-side preference optimization, like mDPO [7], keeps the answer fixed and contrasts two images. These methods create the contrasting image synthetically, by cropping or otherwise corrupting the original. But in pathology, a finding such as necrosis cannot be faithfully inserted into or removed from a slide. So our contribution is how to build the contrast: a second real slide from a different patient with the same cancer type and tissue source site, whose report records the opposite state of the same finding (for example, necrosis present in one specimen and absent in the other). As shown in Table 2, the synthetic variant reaches EWA 13.94 against 15.29 for real pairs.

To further address your concern, we ran an additional experiment. We tested the Pair-DPO on CAMELYON16, a standard public dataset in which pathologists outlined every metastasis. Every slide is asked one four-option question on whether the lymph node section contains metastatic breast carcinoma, and a pair consists of one metastatic and one normal slide. We trained a CE control and a Pair-DPO model from our SFT checkpoint, and then removed or kept the annotated lesion at test time.

CAMELYON16, accuracy (%)CE controlPair-DPO
Full slide45.1248.35
Annotated lesion removed (expert-annotated)40.57 (−4.55)41.64 (−6.71)
Only the annotated lesion kept64.61 (+19.50)77.50 (+29.15)
Same number of random non-lesion tiles removed45.66 (+0.54)48.92 (+0.57)
Same number of most tumour-like non-lesion tiles removed (CONCH)45.54 (+0.42)48.60 (+0.25)

Removing the annotated lesion lowers the accuracy of Pair-DPO by 6.71 points, whereas removing the same number of non-lesion tiles changes it by at most 0.57 points. The effect of the lesion is also larger for Pair-DPO than for the CE control. This suggests that the same loss can be trained on another WSI dataset from slide-level labels.

W4. Release of data and code

We have uploaded the benchmark as supplementary material in two files. cleanslide_questions.json contains all the questions, each with its slide, split, source, task, options, and answer. cleanslide_slides.csv lists the 9,985 TCGA diagnostic slides with their patient, tissue source site, cancer type, and split, together with the GDC file ID, file name, size, and MD5 checksum of each slide. The slides themselves are TCGA data distributed by the NCI Genomic Data Commons (GDC) and amount to about 11 TB, so we release their identifiers rather than copies. With this index, anyone can download exactly the slides we used with the GDC Data Transfer Tool, check each file against its checksum, and verify that the splits share no patient or tissue source site.

W5. Fairness of the comparisons

Our comparison follows the common protocol of slide-level VQA work. SlideChat [1] and WSI-LLaVA [2] also evaluate tile-level and general-purpose models on slide-level questions with sampled tiles or a downsized slide, because these models cannot take a whole slide, and we use the 30-tile majority vote of [1]. General-purpose models are routinely included as a reference [1–3], and in [1, 3] the proposed model is likewise trained on its own data while the other models are used as published checkpoints. Moreover, to further address your concern, we fine-tuned three more published models on the CleanSlide training split.

Fine-tuned on CleanSlideLayer-B acc. (%)Blind (%)EWA
SlideChat∗62.7056.129.40
WSI-LLaVA∗52.5248.424.17
Patho-R1-7B∗46.0340.664.16
Quilt-LLaVA∗44.3641.122.46
PairSlide67.3557.7615.29

Fine-tuning raises accuracy, but the blind score rises with it, so EWA stays between 2.46 and 9.40, against 15.29 for PairSlide. This suggests that adaptation to the dataset alone does not produce the gain in evidence use. The result also holds on the two external cohorts in Table 3, where PairSlide is compared only with 10 medical speicifc models and has the highest accuracy on both (57.64% on CPTAC and 58.98% on BCNB).

W6. Hyperparameters

As is common when building on an existing model, stages one and two follow the public SlideChat. Sec. 5.1 lists the values, and Appendix D gives the full settings. λ and the number of updates come from the sweeps over training steps, number of pairs, and λ in Sec. 5.3 and Figure 3.

W7. AI usage

We are not sure which part of the paper it refers to. We would be grateful if you could point us to the specific places, and we will be glad to clarify them. We would also like to mention that our use of LLMs is already disclosed in the paper, in the LLM usage statement before the references, in line with ICLR’s AI Policy for Authors.

We hope that these clarifications and new results address your concerns, and we would be very grateful if you would consider updating your assessment.

References

[1] Ying Chen, Guoan Wang, Yuanfeng Ji, et al. SlideChat: A large vision-language assistant for whole-slide pathology image understanding. In CVPR, 2025.

[2] Yuci Liang, Xinheng Lyu, Wenting Chen, et al. WSI-LLaVA: A multimodal large language model for whole slide image. In ICCV, 2025.

[3] Yuxuan Sun, Yunlong Zhang, Yixuan Si, et al. PathGen-1.6M: 1.6 million pathology image-text pairs generation through multi-agent collaboration. In ICLR, 2025.

[4] Pingyi Chen, Chenglu Zhu, Sunyi Zheng, et al. WSI-VQA: Interpreting whole slide images by generative visual question answering. In ECCV, 2024.

[5] Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don’t just assume; look and answer: Overcoming priors for visual question answering. In CVPR, 2018.

[6] Lin Chen, Jinsong Li, Xiaoyi Dong, et al. Are we on the right way for evaluating large vision-language models? In NeurIPS, 2024.

[7] Fei Wang, Wenxuan Zhou, James Y. Huang, et al. mDPO: Conditional preference optimization for multimodal large language models. In EMNLP, 2024.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.