acceptodds
Withdrawn from ICLR 2027

Pathology Harness: Evidence-Guided Diagnostic Finalization for Pathology Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in computational pathology have enabled systems that combine whole-slide image analysis with diagnostic reasoning. These systems aggregate regional features, generate morphological descriptions, or navigate tissue at multiple magnifications to formulate diagnostic proposals. However, producing a plausible proposal does not ensure that the collected observations document the morphological findings and differential checks needed to support it. This gap raises a practical question: when is the available evidence sufficient to finalize a diagnosis, and what should be examined when it is not? To address this problem, we introduce Pathology Harness, a framework that governs diagnostic finalization through explicit evidence requirements. The framework binds each fixed diagnostic proposal to an evidence contract, audits the recorded observations against its requirements, and determines whether to release the proposal, request further observation, or block finalization. To resolve recoverable evidence gaps, we develop Differential-Contract Evidence Acquisition (DCE-Scout), which selects complementary regions and magnifications under a limited observation budget. Newly acquired findings can support or contradict the proposal and are recorded for renewed audit, preserving the basis for each release decision. Experiments reveal that even correct diagnostic proposals can lack the recorded evidence required for finalization, highlighting a gap between diagnostic prediction and evidential support. Pathology Harness addresses this gap through targeted evidence acquisition and selective release, allowing more cases to meet explicit finalization criteria than initial-record auditing alone and lowering the error rate among released diagnoses compared with unconditional release. These findings demonstrate the value of coupling evidence acquisition with explicit finalization criteria and suggest a promising direction for pathology agents that produce traceable, evidence-supported diagnostic outputs.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computational pathology, Whole-slide image analysis, AI agents, Evidence-grounded decision making, Case closure, Audit-guided evidence acquisition

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_ZJsKOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Pathology Harness, a framework for evidence-guided diagnostic finalization in computational pathology. The authors address the problem that a diagnostic prediction may be correct or plausible without being supported by sufficiently documented morphological evidence. The proposed framework introduces evidence contracts that specify the morphological observations and differential diagnostic requirements associated with a fixed diagnostic proposal. A Contract Auditor and a P3 Gate assess whether the collected evidence satisfies these requirements. When evidence is insufficient, the proposed DCE-Scout module selects additional regions and magnifications for further examination before making a finalization decision. Experiments are conducted primarily on BRACS and EBRAINS, comparing unconditional diagnostic release, evidence auditing without further observation, and evidence-guided follow-up. The results show that the proposed framework reduces the error rate among finalized diagnoses, from 55.6% to 18.9% on BRACS and from 28.1% to 11.0% on EBRAINS. However, these improvements come with substantially reduced diagnostic coverage.

Critical strengths
  1. The paper addresses a meaningful problem in pathology agents by distinguishing diagnostic prediction from evidence-supported diagnostic finalization. A plausible prediction does not necessarily imply that the underlying morphological evidence has been adequately collected or documented. Introducing explicit evidence requirements and traceable decision records is a potentially valuable direction for improving the reliability and accountability of pathology agents.

  2. The paper attempts to separate diagnostic prediction from finalization decisions by evaluating different policies using the same frozen diagnostic proposals and initial evidence records. This experimental design helps isolate the effect of evidence auditing and follow-up observation from changes in the underlying classifier. The reported risk–coverage analysis also provides useful information about the trade-off between reducing incorrect finalizations and withholding potentially correct diagnoses.

Critical weaknesses

1.The proposed framework is described at a rather abstract level, making it difficult to assess its technical novelty and reproducibility. Although the paper introduces several components, including evidence contracts, the Contract Auditor, P3 Gate, and DCE-Scout, their actual implementations are not sufficiently specified. For example, Appendix A.7 describes several possible implementations of the Contract Auditor without clearly identifying the one used in the experiments. Similarly, the construction of the actual evidence contracts and the scoring functions, optimization procedure, and hyperparameters used by DCE-Scout remain unclear. More importantly, the framework relies on these components to determine whether morphological evidence is sufficient, yet the validity of the evidence requirements and auditing decisions is not independently established. Consequently, it is difficult to determine whether the proposed system genuinely assesses diagnostic evidence or primarily enforces a set of predefined procedural rules. The presentation further complicates understanding. The main framework diagram (Figure 2) appears in the Related Work section, before the methodology, and the paper relies heavily on specialized terminology without providing a sufficiently concrete explanation of how the overall system operates.

2.The experimental evaluation is insufficient to establish the advantages of the proposed framework over existing pathology agents. Although the paper discusses several recent agent-based approaches, including CPathAgent, WSI-Agents, SlideSeek, and BEACON, these methods are not directly compared with Pathology Harness in terms of evidence acquisition, verification, and diagnostic finalization under comparable conditions. The reported comparisons mainly involve internal policy variants, unconditional release, and confidence-based selection. These baselines do not adequately establish the incremental contribution of the proposed framework relative to existing agentic approaches. Furthermore, the improvements in selective accuracy should be interpreted cautiously, as they are accompanied by a substantial reduction in diagnostic coverage. While the paper includes a confidence-based comparison at matched coverage on BRACS, the proposed method additionally acquires new observations, making the comparison unequal in terms of available information. Stronger controls, such as budget-matched alternative evidence acquisition strategies, are needed to determine whether DCE-Scout itself provides meaningful benefits. Overall, the current experiments demonstrate that selective finalization can reduce the error rate among released diagnoses, but they do not convincingly establish that the proposed framework offers a substantial advantage over existing evidence-seeking or verification-based approaches.

Questions for authors

1.Could the authors clarify the exact implementations used for the key components in the reported experiments? In particular, how were the evidence contracts constructed for BRACS and EBRAINS? What specific implementation was used for the Contract Auditor? How were the observability, differential relevance, diversity, and cost terms in DCE-Scout computed and combined? Providing the actual configurations and a complete execution example would be important for assessing the technical contribution and reproducibility of the method.

2.In Table 2, adding the P3 Gate to the Contract Auditor reduces the number of finalized BRACS cases from 152 to 64, eliminating 20 incorrect finalizations but also withholding 68 additional correct predictions. Could the authors explain which specific P3 checks are responsible for these decisions and why these checks are so strongly associated with diagnostic correctness? This clarification is important for understanding whether the reported improvement comes from meaningful evidence assessment or from restrictive release rules.

3.How is the clinical validity of the proposed evidence contracts and auditing decisions established? Were the morphological claims and evidence-sufficiency judgments independently evaluated by pathologists or against external annotations? In particular, can the authors provide an analysis of incorrect finalized cases to distinguish errors caused by inaccurate morphological observations, incomplete evidence requirements, and failures of the auditing procedure? Such evidence would be important for determining whether the framework genuinely improves evidence-grounded diagnostic reliability.

4.Could the authors provide comparisons with recent evidence-seeking or verification-based pathology agents, such as SlideSeek, WSI-Agents, or BEACON, under comparable evaluation settings? In particular, comparisons using the same initial diagnostic proposals and matched observation budgets would help establish whether Pathology Harness provides benefits beyond existing approaches. If direct comparisons are not feasible because of differences in task formulation, could the authors provide stronger budget-matched acquisition baselines and clarify the limitations of the current comparisons?

Reviewer_dj4qOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents Pathology Harness, a framework that checks whether a pathology prediction has enough supporting evidence to be released. It reviews image observations against class-specific requirements and requests additional regions or magnifications when evidence is missing. The system then releases or blocks the prediction. Experiments on BRACS and EBRAINS (two WSI classification datasets) show fewer errors among released predictions at the cost of withholding some cases. Additional image observations allow more cases to be released than reviewing the initial evidence alone.

Critical strengths

The paper explores a relatively novel idea for medical agents, which is checking the evidence that supports a diagnosis before allowing it to be released. Requesting additional observations or blocking an insufficiently supported prediction is a promising design for future medical-agent workflows. The explicit evidence requirements and structured records also make the process easier to inspect and could support clinician oversight.

Critical weaknesses
  1. The evaluated system uses a custom pipeline with a frozen classification model, followed by evidence collection and a release-or-block decision. The paper does not demonstrate how the harness works with existing agents such as PathAgent or PathNavigate, or compare their performance before and after integration. The current evidence can only support a narrow application to selective classification, while the conception of the pathology harness is a bit overclaimed.

  2. The system can collect additional evidence before its final decision, but once a case is blocked, it does not show how to reconsider the original diagnosis, explore alternative diagnoses, or continue gathering evidence under a revised hypothesis. Because the proposed label remains fixed, additional observations can only support or challenge that label. This limits the framework’s demonstrated value as a complete medical-agent workflow: it identifies unsupported predictions but doesn't show the backup solutions.

Questions for authors
  1. Can you provide paired results for an existing agent, such as PathAgent or PathNavigate, with and without the proposed Harness? This would help show the usefulness of this work beyond the current simple classification pipeline.

  2. Can you demonstrate a workflow in which a block decision triggers further evidence gathering, reconsideration of the initial diagnosis, or evaluation of alternative diagnoses? This would clarify whether the framework can really support a complete pathology diagnostic workflow.

Reviewer_ctsrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript proposes Pathology Harness, a framework that determines whether a pathology agent's diagnostic proposal has sufficient evidence to be finalized. It links each proposal to an evidence contract that specifies the required morphological findings, reviews the observations recorded in the contract, and decides whether to make a definitive determination, request further observation, or prevent a diagnosis. Experiments on BRACS and EBRAINS show that the framework reduces the error rate among finalized diagnoses compared with unconditional release.

Critical strengths
  1. This manuscript explores a meaningful but under-explored question in computational pathology: whether the evidence gathered truly supports the issuance of a diagnosis.

  2. The framework is traceable because it separates proposal generation, evidence auditing, and release authorization, and retains complete evidence and review history for each diagnostic decision.

Critical weaknesses
  1. The core argument of this manuscript is that "even if a proposal is correct, it may lack sufficient evidence." However, whether the evidence is sufficient is entirely determined by the system's own Contract Auditor and P3 Gate. So how can we prove that records deemed sufficient evidence are indeed enough to support a diagnosis? Or how can we prove that records deemed insufficient evidence truly lack the necessary evidence?

  2. The "Evidence Contract" serves as the foundation of the entire framework. However, the manuscript does not appear to present any concrete examples of these contracts, such as those used for specific categories in datasets like BRACS or EBRAINS. Furthermore, the manuscript does not seem to explain how these contracts are constructed, nor does it clarify whether they have been validated by pathologists for both correctness and the ability to effectively discriminate between categories.

  3. The proposed Harness is evaluated based on a simple logistic regression proposal generator, and since the base classifier is weak, it is much easier to reduce the selection risk by abstaining. Therefore, it is unclear whether this advantage can be maintained for stronger proposals, such as some MIL baseline models, which have never been integrated into Harness. Therefore, the proposed framework does not offer significant practical advantages over the standard foundation model + MIL pipeline.

Questions for authors
  1. Could the authors provide independent validation, such as a human review of a subset of released and blocked cases, to verify that the Contract Auditor's sufficiency judgments are correct?

  2. Could the authors present concrete contract examples for BRACS and EBRAINS and explain how these contracts were constructed and whether they were validated by pathologists?

  3. When Harness is applied on top of a more robust MIL model, does the risk reduction effect still exist?

Reviewer_rGsuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a pathology system has enough recorded evidence to release a proposed diagnosis. It checks a list of required findings and uses DCE-Scout to inspect more tissue when evidence is missing, without changing the diagnosis. On BRACS and EBRAINS, it releases fewer diagnoses but makes fewer errors among those it releases.

Critical strengths

The paper separates making a prediction from having enough evidence to release it. I like this separation. All the methods explored start with the same diagnosis and evidence record, making the effects of checking evidence and inspecting more tissue easier to understand. Since earlier observations are kept around, it also makes the final decision easier to review.

The EBRAINS results test the approach on a larger set of cases. The paper also reports how many correct diagnoses are held back. This makes the cost of being more cautious clear, which others can build on.

Critical weaknesses

The problem is important, and keeping the proposed diagnosis fixed makes the tests easier to interpret. I am not yet convinced that the system checks evidence reliably or chooses better follow-up views. Expert review of the records and fairer follow-up comparisons could change my rating.

The paper does not independently show that accepted evidence records are correct and complete. The Interpreter describes the tissue, the Auditor checks those descriptions, and P3 checks the stored records. Passing these checks does not prove that the descriptions are right or that the required findings are enough to support the diagnosis. The paper notes this risk but does not provide a broad expert review of the records. Two examples are not enough. Important setup details are also missing. This makes the system hard to assess or repeat. Perhaps the authors can do a more thorough job here?

The tests do not show whether DCE-Scout chooses better follow-up views than simpler methods. Evidence Repair sees new images, while the confidence-based comparison uses only its initial scores. Comparing NoFU with Full tests the whole follow-up process but it does not test the choice of tissue regions alone.

There is also no equal-budget comparison with random sampling or a selection based on diversity. The gains could come from seeing more images and checking again. The paper also does not report the cost of those extra steps.

Questions for authors

Which model or rules were used for the Contract Auditor? How were the required findings for BRACS and EBRAINS written and checked? Please provide these lists, the prompts, the limits on follow-up requests, and the rules and weights used to select tissue regions.

In a sample of released cases, how often do independent pathologists agree that the tissue descriptions are correct and the evidence is enough? Please separate correct diagnoses with weak evidence from wrong diagnoses that pass the checks.

With the same number of image requests and model calls, does DCE-Scout beat simpler follow-up sampling followed by the same checks? Does it still beat confidence-based selection when that method can also use the new images?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.