acceptodds
Withdrawn from ICLR 2027

PATHOARGUS: ADVANCING COMPREHENSIVE, EVIDENCE-GROUNDED VISUAL REASONING FROM GIGAPIXEL SLIDES TO MULTI-SLIDE CASES

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Whole-slide pathology question answering requires finding relevant tissue and an- swering from visual evidence, yet textual cues can also yield correct answers. We introduce PATHOARGUS-BENCH to evaluate six pathology capabilities and test whether answers follow controlled changes in target evidence. It contains 22,078 four-choice questions covering 4,809 target cases across 15 TCGA projects, in- cluding 4,281 bench questions. Evidence-Set Grounding (ESG) forms counterfac- tual quartets by placing a target in each of three candidate sets or removing it; quar- tet exact-match accuracy (QExact) requires all four answers to be correct. Across 24 baselines, GPT-5.6 attains 57.09% overall accuracy but only 3.93% QExact on 483 quartets. We further propose PATHOARGUS, a framework that selects question-relevant evidence patches from whole-slide images to answer pathology questions. It reaches 65.15% overall accuracy, 68.27% ESG accuracy, and 43.27% QExact using only 512 selected patches per question, retaining 1.51% of avail- able bench features. These results show more consistent answers across evidence states, while fine-grained local findings remain challenging.

Primary area:
datasets and benchmarks
Keywords:
Computational pathology, Whole-slide image question answering, Multimodal large language model, Evidence-Grounded reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_73KnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper points out that existing pathology VQA models may obtain correct answers by relying on textual priors rather than genuinely using visual evidence. To address this issue, the authors construct the PathoArgus dataset and propose the ESG counterfactual evaluation mechanism, which tests whether model predictions change with visual evidence by moving or removing the target case from candidate sets. Methodologically, PathoArgus selects question-relevant evidence from large numbers of WSI patches through question-conditioned hierarchical evidence allocation, tissue-aware patch selection, and source-aware answer generation. Overall, this paper introduces a valuable approach for evaluating visual evidence grounding in pathology and designs a corresponding evidence selection framework. However, the validity of the benchmark evidence, the rigor of the counterfactual experiments, and the method's advantages in real-world multi-slide pathology reasoning require further validation.

Critical strengths
  1. The paper points out that conventional VQA accuracy cannot sufficiently demonstrate whether models truly utilize visual evidence. It proposes ESG and QExact to evaluate models through counterfactual changes in evidence. This design helps identify cases where models rely on textual priors or fixed answers.

  2. The paper constructs a dataset covering six pathology capabilities and 15 TCGA projects. It also provides text-only controls, counterfactual evaluations, and multiple component ablations, offering a relatively systematic experimental foundation for analyzing the visual evidence utilization capabilities of pathology models.

Critical weaknesses
  1. ESG constructs counterfactual samples by moving or removing target cases, but approximately half of the evaluations use cross-organ donor cases. In these settings, models may identify the target set simply by recognizing organ types rather than specific pathological features. Furthermore, for same-organ donors, it remains unclear whether cases containing the same target pathological findings are strictly excluded. Therefore, the current ESG results may reflect both organ recognition and pathological evidence localization capabilities, making it difficult to distinguish between them.

  2. The paper primarily generates questions and answers from diagnostic fields in pathology reports. However, not all information in these reports can be directly determined from the provided WSIs. Although the authors conduct data integrity checks and report expert performance on a subset of questions, systematic WSI-level evidence validation is still lacking. Furthermore, the paper does not directly evaluate the pathological relevance or completeness of the selected patch evidence. This limits the reliability of the benchmark as a resource for evaluating evidence-grounded pathology VQA.

  3. The paper emphasizes extending from single WSIs to multi-slide cases, but the dataset contains 4,809 cases and 5,400 WSIs, averaging only approximately 1.12 slides per case. Therefore, the proportion of cases containing multiple WSIs needs further clarification. Moreover, the Cross-Slide Integration (CSI) results do not clearly establish whether answering these questions genuinely requires integrating information from different slides. Although the paper states that CSI uses multiple slides from the same case, providing multiple slides does not necessarily imply that cross-slide information integration is required. If a single slide already contains sufficient evidence, the task may essentially remain one of single-slide recognition or evidence retrieval.

  4. The paper compares 24 baseline systems, but PathoArgus is jointly trained on a specifically constructed training set, whereas most existing models are evaluated primarily using their original pretrained capabilities. Although the paper provides an SFT baseline and component ablations, it still lacks comparisons with strong evidence selection methods retrained under the same data, backbone, and training budget, as well as simpler question-conditioned retrieval methods. Therefore, it is difficult to determine whether the performance improvements primarily come from the proposed hierarchical selection mechanism or from task-specific supervision and training adaptation.

  5. The paper emphasizes that only 512 patches are selected, retaining approximately 1.51% of the available features. However, the method still requires LongNet to encode all patch features from candidate slides, while the 512-patch limit mainly applies to the visual inputs passed to the Reader. Therefore, the current analysis primarily reflects Reader-side token consumption rather than the computational efficiency of the entire system. The lack of comparisons in end-to-end inference time, GPU memory consumption, and feature processing costs leaves the practical deployment efficiency unclear.

  6. There is an inconsistency in the overall accuracy reported for the PathoArgus–Lingshu configuration between Table 2 and Figure 5a. Table 2 reports an overall accuracy of 61.39%, whereas Figure 5a reports 62.05% for the corresponding WSI-input setting. Could the authors clarify the reason for this discrepancy?

Questions for authors
  1. Could the authors report ESG performance separately for same-organ and cross-organ donor settings and explain how they ensure that donors do not contain the same target pathological findings?

  2. How do the authors verify that report-derived answers are genuinely supported by WSI visual evidence? Could they provide expert annotations or patch relevance evaluations?

  3. How many cases contain multiple WSIs, and how many questions genuinely require cross-slide reasoning? Could the authors report performance specifically on these tasks?

  4. Could the authors compare PathoArgus with stronger evidence selection methods under matched training data, backbone models, and training budgets?

  5. Could the authors report end-to-end inference time, GPU memory usage, and computational costs, including complete WSI feature encoding and evidence selection?

Reviewer_dN2JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a pathology question-answering benchmark that tests whether answers track changes in whole-slide evidence. Its counterfactual protocol moves a target among candidate slide sets or removes it, requiring correct answers across all four states. A question-conditioned patch selector with source-aware answering improves both overall accuracy and counterfactual consistency over the evaluated baselines.

Critical strengths

Testing the same question with changed evidence patches is a strong idea. ESG moves the target between three slide groups and then removes it. The question and answer choices stay the same. QExact counts a success only when all four answers are correct. This tests image use more directly than accuracy alone. GPT-5.6 scores 57.04% on ESG but only 3.93% on QExact, showing that there is still room to improve.

The proposed model shows strong gains. It reaches 43.27% QExact and 65.15% overall accuracy. Also, removing images from the same trained model lowers its scores, which supports the value of the image patches. The ablations that remove parts of the method also support importance of patch selection and source tracking.

Critical weaknesses

I like the counterfactual benchmark and the size of the reported gains. My concern is whether the labels and comparisons support the main claim. The following details would help in the rebuttal:

It is unclear whether the reference answers alway match the evidence patch information. Section 3 takes answers from patient reports and pairs them with the available slides. But a report may describe a finding that is not visible in those slides. The other cases used to fill the ESG slide groups must also lack the target finding. So, matching the organ and number of slides does not necessarily prove this. Without ground truth annotations, a wrong answer could reflect a label error rather than poor evidence patch identification/use. The paper notes this limitation though.

The comparisons do not clearly separate the method's value from the value of extra training. Table 2 mainly compares trained PATHOARGUS models with existing models used without that training. The Qwen2.5 SFT comparison helps mitigate this to a degree, but its input and training details are missing. Since many models score near chance on the new slide-group task, a fair comparison against equal amount of training is important to do.

The efficiency claim also needs more validation. Only 512 patches reach the answering model, but Section 4.2 processes all patches before selection. Only counting the answer-stage tokens does not show the full cost incurred during inference.

Questions for authors

What fraction of questions has been independently checked against the actual supplied slides? It is important to report errors separately for local findings, case-level findings, and ESG. How do you verify that each donor lacks the target finding and that each ESG row has exactly one valid answer?

What are ESG accuracy and QExact on same-organ donors alone? Do the gains persist when donors share the broad diagnosis and differ only in the queried finding? This would distinguish fine-grained evidence grounding from easier case discrimination.

Can you provide the missing implementation details and compare against an equally trained, source-aware selector baseline on the same backbone? Please also report end-to-end inference cost, including candidate encoding.

Reviewer_dJZhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces PathoArgus, a benchmark and framework for evidence-grounded visual question answering over whole-slide pathology images (WSIs). The benchmark contains 22,078 four-choice questions covering six pathology capabilities, including 4,281 test questions. Its key contribution is Evidence-Set Grounding (ESG), which evaluates whether model predictions correctly follow controlled changes in the location or presence of target evidence. Quartet exact-match accuracy (QExact) measures correctness across all four counterfactual configurations.

The proposed framework combines question-conditioned hierarchical patch allocation, tissue-aware ranking, and source-aware encoding to select relevant visual evidence across slides and candidate sets. Experiments against 24 baselines show that PathoArgus achieves 65.15% overall accuracy and 43.27% QExact, compared with 57.09% and 3.93%, respectively, for GPT-5.6. Ablations further support the effectiveness of structured evidence selection under a limited reader-side visual-token budget.

Critical strengths
  1. Meaningful benchmark innovation for evaluating visual evidence dependence. The ESG protocol and QExact metric address an important limitation of conventional WSI QA benchmarks: high answer accuracy does not necessarily indicate reliance on visual evidence. By controlling target placement while preserving questions and answer choices, the benchmark provides a more stringent evaluation of evidence responsiveness. The substantial discrepancy between conventional accuracy and QExact across existing models demonstrates the practical significance of this evaluation design.

  2. Effective structured evidence selection with substantial empirical improvements. The proposed framework explicitly balances question relevance and evidence coverage across candidate sets, slides, and spatial regions. Strong improvements over uniform sampling, global Top-K selection, and source-encoding ablations support the utility of this architecture. Consistent gains across three reader backbones and substantial improvements in counterfactual accuracy further strengthen the empirical contribution.

Critical weaknesses
  1. Insufficient validation of genuine pathological evidence grounding. ESG evaluates whether predictions follow changes in target-case placement, but does not establish whether the selected patches contain the diagnostic findings supporting the answers. Report-derived labels do not guarantee that every supplied WSI visually supports its corresponding question, and donor cases are not systematically verified as negative for the queried findings. Although the expert evaluation provides useful evidence, its limited sample does not resolve this concern. Moreover, approximately half of ESG quartets use cross-organ donors, potentially allowing coarse organ recognition rather than fine-grained pathological reasoning. This limits the central evidence-grounding claim.

  2. Incomplete causal attribution of performance gains. The framework combines architectural modifications, pretrained evidence-selection components, and substantial counterfactual ESG training supervision. Existing ablations demonstrate improvements from structured selection but do not isolate the contribution of counterfactual training or fully disentangle individual architectural components. Most external baselines are also evaluated without equivalent task-specific adaptation. Since ESG accounts for 45.13% of benchmark questions, the overall improvement may overstate gains in broader pathology reasoning. Indeed, the best-overall Qwen3-VL configuration remains relatively weak on morphological analysis, fine-grained recognition, and cross-slide integration. These limitations weaken the attribution and generality of the claimed improvements.

Questions for authors

Validity of evidence grounding: How are ESG donor cases verified to exclude the queried pathological finding, particularly for same-organ candidates? Can the authors report ESG accuracy and QExact separately for same-organ and cross-organ quartets? Is expert-annotated evidence localization available to verify that selected patches contain the actual diagnostic findings?

Attribution of counterfactual improvements: How much of the QExact improvement comes from ESG counterfactual training rather than the proposed selection architecture? Can the authors provide a controlled comparison with and without ESG training under identical backbone, training budget, and evidence-selection settings?

Component contributions and generalization: Can the authors report task-specific and QExact ablations for hierarchical allocation, candidate-set balancing, and source encoding? Additionally, can they clarify whether CSI questions genuinely require information from multiple slides, rather than being answerable from a single slide?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.