acceptodds
Withdrawn from ICLR 2027

XPathAgent: Concept-to-Evidence Reasoning via Reliability-Guided Routing on Heterogeneous Pathology tools

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Computational pathology provides diverse specialist models spanning organs, spatial resolutions, and tissue properties. Integrating these heterogeneous pathology tools is essential for whole-slide image analysis, yet existing pathology agent methods rely on fixed foundation models and validation strategies, limiting generalizability while providing limited transparency into how tool outputs are selected, validated, and integrated. We present XPathAgent, a plug-and-play framework for concept-to-evidence reasoning over heterogeneous pathology tools via reliability-guided routing. The framework operates through three key modules: (1) a Diagnostic Atlas that maps clinical queries to pathology concepts and corresponding feasible tool candidates; (2) a Concept–Evidence Tree that dynamically refines tool-to-region assignments as new evidence accrue; and (3) Invariance-Calibrated Reliability (ICR), which guides tool routing by measuring tool invariance under controlled input degradation while penalizing correlated errors. Validated measurements propagate into concept- and task-level estimates, yielding a fully traceable reasoning path. Compared with existing foundation models and agentic approaches, XPathAgent attains top performance on all 13 benchmarks across subtyping, grading, metastasis detection, stromal TIL scoring, and survival prediction, establishing concept-driven orchestration as a robust paradigm for trustworthy pathology agents.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
computational pathology, whole-slide image analysis, agentic systems, tool-augmented LLMs, pathology foundation models, input-conditional reliability, explainable AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dFWwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes XPathAgent, a concept-driven, multi-tool collaborative reasoning framework for computational pathology, aiming to address the lack of explicit diagnostic evidence organization in existing pathology agents and the difficulty of assessing the reliability of outputs from different specialized tools. The authors evaluate the framework on 13 benchmark-task combinations covering cancer grading, subtyping, metastasis detection, stromal tumor-infiltrating lymphocyte (sTIL) scoring, and survival prediction, and validate its effectiveness through component ablations, leave-one-tool-out analyses, and in-context learning experiments. Overall, this paper presents a clinically reasonable framework for evidence organization and tool collaboration, with relatively broad experimental coverage. However, whether the output stability measured by ICR can represent diagnostic correctness, and the extent to which the framework depends on predefined medical knowledge and tool configurations, require further justification.

Critical strengths
  1. XPathAgent decomposes pathology tasks into explicit morphological concepts, measurement tools, and evidence integration rules through the Diagnostic Atlas and Concept–Evidence Tree, allowing the reasoning process to be traced back to specific tissue regions and tool outputs. This explicit organization helps improve the transparency and auditability of the reasoning process.

  2. ICR does not assume that all tool outputs are equally reliable. Instead, it adjusts evidence weights by considering image-quality perturbations, tool output stability, and cross-tool correlations, providing a structured mechanism for integrating heterogeneous pathology models.

Critical weaknesses
  1. ICR measures stability using Spearman correlations between tool outputs before and after controlled perturbations and determines evidence weights accordingly. However, stable outputs are not necessarily correct, and models with systematic diagnostic biases may also remain consistent under perturbations. Furthermore, correlations between cross-tool perturbation responses do not directly represent correlations between diagnostic errors. The paper lacks direct validation of the relationship between ICR scores and actual tool prediction accuracy, leaving the effectiveness of ICR as a reliability assessment mechanism insufficiently supported.

  2. The paper constructs offline stability curves using a small number of unlabeled images from each benchmark and then estimates tool reliability based on the severity of image-quality degradation in test regions. However, how the severity of resolution degradation, defocus, and staining deviations is automatically determined in test images is not sufficiently explained. Furthermore, each benchmark requires a separate calibration set, making it unclear whether the method can be directly applied to new data distributions.

  3. Although XPathAgent is positioned as a plug-and-play framework, its task decomposition, concept selection, tool matching, and final evidence aggregation depend on the manually curated Diagnostic Atlas. New organs or diagnostic tasks still require additional construction and review of relevant concepts and rules. Furthermore, the organ set defined in the methodology does not include the lung, bladder, and endometrium, which are involved in the experiments. These configurations require further clarification.

  4. The paper emphasizes its ability to construct reliable and traceable concept-to-evidence reasoning chains, but the main experiments still focus on final classification, grading, and survival prediction performance. Independent evaluations of ROI localization accuracy, tool selection appropriateness, concept measurement accuracy, and the clinical validity of evidence chains are lacking. Therefore, improvements in final prediction performance are insufficient to demonstrate that the intermediate evidence itself is more reliable.

  5. The paper claims that XPathAgent achieves the best performance across all 13 benchmark-task combinations. However, Table 2 reports a QWK of 0.638 on TCGA-UCEC, which is lower than HistoSelect's 0.641. Furthermore, the gains from co-failure correction in Table 13 are only 0.002–0.008, and the paper does not provide sufficient confidence intervals or significance tests to establish the stability of these small improvements. This weakens the credibility of some experimental conclusions.

  6. XPathAgent requires invoking multiple specialized tools across different tissue regions while performing reliability assessment, tool routing adjustments, and LLM reasoning. The paper does not sufficiently report end-to-end inference time, the number of tool calls, GPU memory usage, or computational cost comparisons with baselines. Therefore, its practical efficiency for large-scale WSI analysis remains unclear.

Questions for authors
  1. Could the authors provide a correlation analysis between ICR scores and actual tool prediction accuracy, and explain how the framework handles tools that produce stable but diagnostically incorrect outputs?

  2. How is the severity of image-quality degradation automatically estimated during inference? When the model is applied to unseen datasets, is it necessary to reconstruct the ICR calibration set?

  3. How does XPathAgent support new organs and tasks not covered by the Diagnostic Atlas? Furthermore, why does the organ set defined in the methodology not fully match the organs included in the experiments?

  4. Could the authors directly validate the correctness of ROI localization, concept measurements, and final evidence chains using pathologist annotations or independent reference standards?

  5. Could the authors explain the inconsistency between the TCGA-UCEC results in Table 2 and the claim of achieving the best performance across all benchmarks, and provide statistical significance analyses for the improvements introduced by ICR?

  6. Could the authors report the inference time, GPU memory consumption, and tool invocation costs of the complete system and compare them with the main baselines?

1 comment
AuthorsOpenReview ↗

Response to Reviewer dFWw

We thank the reviewer for recognizing the transparency of our concept-to-evidence design and the structured reliability mechanism in ICR. We address each concern below.

[W1/Q1] ICR stability versus diagnostic correctness

We agree that stability does not guarantee correctness. ICR estimates measurement stability/informativeness, not intrinsic diagnostic accuracy. We therefore added ground-truth validation on 91 SICAPv2 biopsies from 63 patients using CONCH (Table 5(b); Appendix E.2). For cancer detection, where CONCH has strong clean performance (AUROC=0.92), lower ICR admissibility is significantly associated with more perturbation-induced errors (Kendall τ=0.67, Holm-adjusted p=0.002). Across 48 acquisition-condition comparisons, ICR ordering agrees with diagnostic-error ordering in 83% of comparisons versus 65% for nominal severity. At region level (1,584 pairs), lower admissibility is associated with higher error rates (Spearman ρ=-0.43, 95% CI: -0.52 to -0.33), and rejected measurements exhibit 25.9 percentage points more new errors than admitted measurements. For Gleason grading, CONCH is near chance on clean images (AUROC=0.56), so that analysis is inconclusive. Thus, ICR cannot rescue a diagnostically weak but stable tool; it identifies degradation susceptibility when the underlying specialist is competent.

[W2/Q2] Severity estimation and recalibration

We clarified inference-time severity estimation in Appendix C.2. Resolution uses slide micrometers-per-pixel metadata relative to the reference-set median; focus uses median log10 Laplacian variance mapped to an equivalent calibrated blur level; and stain deviation uses the RMS z-score of mean H/E/D concentrations mapped to an equivalent calibrated stain shift. Values are interpolated within the calibrated range and clipped/flagged outside it. Calibration is intentionally dataset-specific because acquisition distributions differ. For an unseen dataset, ICR requires a representative unlabeled reference set, but no diagnostic labels, specialist fine-tuning, or parameter updates.

[W3/Q3] Diagnostic Atlas and organ/task extension

The Diagnostic Atlas is constructed offline from established pathology literature/reporting conventions and subsequently clinician-reviewed (Appendix D.3); it is fixed during inference. New organs/tasks therefore require offline extension and clinical review of relevant concepts/mappings. “Plug-and-play” refers to integrating compatible specialist tools without fine-tuning them, not automatic ontology creation. We also corrected the organ inventory to include lymph node, lung, bladder, and endometrium (Lines 160–161), matching the experiments.

[W4/Q4] Intermediate evidence validity

We agree that final performance alone does not establish clinical correctness of every intermediate evidence chain. Direct ROI/concept-chain validation would require task-specific spatial annotations unavailable across our benchmarks. We therefore add complementary evidence: component ablations (Fig. 3), leave-one-tool-out analysis (Table 12), measurement-level ICR validation (Table 5(b), Appendix E.2), and complete qualitative traces including a failure case (Figs. 5–7). Appendix E.2 explicitly states that these analyses do not establish native ROI-localization accuracy or clinical correctness of full concept-to-region chains.

[W5/Q5] TCGA-UCEC inconsistency and statistical significance

We thank the reviewer for identifying the inconsistency. HistoSelect is slightly higher on TCGA-UCEC (0.641 vs. 0.638); we will correct the claim to state that XPathAgent achieves the strongest point performance on 12/13 benchmark-task combinations. We also added statistical analysis of ICR (Table 16): mean±SD over five runs, 95% bootstrap CIs, and paired permutation tests (10,000 permutations) with Holm correction. Full ICR improves over no reliability by +0.013 on SICAP (pHolm=0.028) and +0.013 on CAMELYON16 (pHolm=0.036), with gains of +0.005 on TCGA-RCC and +0.006 on TCGA-BRCA. Co-failure correction adds +0.006/+0.008/+0.002/+0.002 across the four benchmarks.

[W6/Q6] Computational efficiency

We added end-to-end profiling on TCGA-RCC using one NVIDIA A100 80GB GPU (Table 19). XPathAgent requires 17.0 min/WSI, 45 GB peak memory, and 34 LLM calls, achieving 0.939 BACC. PRISM, TITAN, and HistoSelect require 2.9/3.3/4.7 min, 12/15/19 GB, and achieve 0.615/0.663/0.875 BACC, respectively. XPathAgent provides relative BACC gains of 52.7%, 41.6%, and 7.3%, at additional compute cost. All specialist execution and route refinement are included in the end-to-end timing.

Reviewer_LtkUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents XPathAgent, a concept-to-evidence reasoning framework for heterogeneous pathology tools via reliability-guided routing. The framework consists of three key modules: (1) a Diagnostic Atlas that maps clinical queries to pathology concepts and corresponding feasible tool candidates; (2) a Concept–Evidence Tree that dynamically refines tool-to-region assignments as new evidence accrues; and (3) Invariance-Calibrated Reliability (ICR), which guides tool routing by measuring tool invariance under controlled input degradation while penalizing correlated errors. Validated measurements propagate into concept- and task-level estimates, yielding a traceable reasoning path. The authors evaluate XPathAgent across 13 benchmark–task combinations covering tumor subtyping, grading, metastasis detection, stromal tumor-infiltrating lymphocyte (sTIL) scoring, and survival prediction, and report the best performance on all 13 benchmarks.

Critical strengths
  1. The paper proposes a unified framework for orchestrating heterogeneous specialist pathology tools. By connecting pathology tasks, morphological concepts, and specialist tools through the Diagnostic Atlas, Concept–Evidence Tree, and reliability-guided routing, the framework provides a structured approach to evidence collection and integration across tools.

  2. The framework is evaluated across a broad range of pathology tasks.The authors evaluate it on 13 benchmark–task combinations and provide component ablations and leave-one-tool-out experiments, offering empirical evidence for the effectiveness of the proposed tool orchestration pipeline.

Critical weaknesses

1.The paper does not sufficiently demonstrate genuine autonomous decision-making. Although XPathAgent supports tool selection and local refinement, its overall process still relies heavily on predefined pathology concepts and routing rules, with limited evidence of a closed loop that dynamically determines the next action based on current evidence. It therefore appears closer to a tool orchestration workflow than a fully autonomous agent. 2.The zero-shot results are valuable, but the added value of the framework in real-world clinical settings remains unclear. Dedicated slide-level models already exist for tasks such as grading, subtyping, and metastasis detection, and may achieve better performance. The authors should clarify how their framework can work with these specialist models to improve their performance, or outperform existing methods in accuracy, robustness, or clinical diagnostic utility, rather than simply orchestrating multiple tools.

Questions for authors
  1. How does XPathAgent autonomously decide what to do next based on the current diagnostic state and evidence gaps? Can the authors provide execution traces showing how tool calls are dynamically adjusted rather than simply following a predefined workflow?

  2. In real-world clinical settings, dedicated slide-level models may already perform well on grading, subtyping, and metastasis detection. How can this framework work with these models to improve their performance, or outperform them in accuracy, robustness, and clinical diagnostic utility?

  3. Does the stability measured by ICR under image perturbations actually correlate with the diagnostic correctness of tool outputs? Can the authors validate this relationship using ground-truth pathology annotations?

3 comments
AuthorsOpenReview ↗

Response to Reviewer LtkU

We thank the reviewer for recognizing the value of our unified multi-tool pathology framework, structured concept-to-evidence reasoning, and broad evaluation. We also appreciate the constructive questions regarding autonomy, dedicated slide-level models, and ICR validity.

[W1/Q1] Autonomous decision-making versus predefined workflows

XPathAgent is designed around morphological grounding and reliability-guided evidence reasoning (Lines 052–053, 091–098). Diagnostic planning is constrained by the clinician-validated Diagnostic Atlas so that evidence collection follows established pathological criteria. The Atlas is constructed offline from pathology literature and reporting conventions, and its concepts and task–concept mappings are subsequently verified through clinical review (Appendix D.3, Lines 1102–1129).

The Atlas is fixed during inference, while evidence collection and refinement are updated dynamically case-by-case. To demonstrate this, we conducted two experiments on CAMELYON16 (Table 5(a), Lines 433–447; Appendix E.1).

(i) Evidence-conditioned routing across regions. Across 20 slides and 10,808 tool–concept–region measurements, 25.5% required refinement, rejection, or LLM adjudication. Moreover, 135/140 (96.4%) slide–tool–concept combinations exhibited different action sequences across regions of the same slide.

(ii) Causal effect of evidence on subsequent actions. We replayed the same regions after controlled changes in blur, staining, or resolution while keeping all other settings fixed. These changes altered subsequent tool calls or delivered evidence in 115/369 (31%) comparisons, whereas identical-input replays produced no changes (0/123).

Thus, while XPathAgent follows a clinically grounded diagnostic structure, its subsequent evidence collection and refinement are dynamically determined by observed measurements, yielding closed-loop evidence-conditioned execution rather than a fixed workflow.

[W2/Q2] Added value relative to dedicated pathology models

We additionally evaluate whether XPathAgent can augment a dedicated supervised slide-level model. We train UNI+ABMIL as a metastasis specialist and incorporate it into XPathAgent. On CAMELYON16/17, UNI+ABMIL alone achieves 0.803/0.759 BACC, XPathAgent achieves 0.822/0.815, and XPathAgent+UNI+ABMIL reaches 0.899/0.908 (Table 17). This shows XPathAgent can build on a task-specific specialist rather than simply replace it.

We also outperform strong whole-slide models PRISM and TITAN across subtyping and metastasis benchmarks (Table 3). For survival, XPathAgent achieves a mean C-index of 0.726, compared with 0.704 for the strongest survival-specific baseline, MDCS-MoAME (Table 1).

Beyond accuracy, ICR provides input-conditional reliability for specialist evidence; with all other components fixed, full ICR improves CAMELYON16 BACC from 0.809 to 0.822 over unweighted integration (Table 15; Lines 458–460). XPathAgent also retains the concepts, supporting regions, and reliability values underlying each prediction, providing an inspectable evidence chain.

[Q3] Relationship between ICR stability and diagnostic correctness

We agree that stability and correctness are distinct. ICR does not directly estimate intrinsic diagnostic accuracy; it measures whether a tool's outputs remain stable and informative under observed image conditions.

We therefore conducted an additional ground-truth validation on 91 SICAPv2 biopsies from 63 patients using CONCH (Table 5(b); Appendix E.2). ICR was calibrated separately using 20 unlabeled biopsies.

For cancer detection, where CONCH has strong clean performance (AUROC = 0.92), lower ICR admissibility was significantly associated with more perturbation-induced diagnostic errors (Kendall τ\tau = 0.67, Holm-adjusted pp = 0.002). Across 48 acquisition-condition comparisons, ICR ordering agreed with diagnostic-error ordering in 83% of comparisons (95% CI: 77–90%), versus 65% for nominal perturbation severity.

At the region level, across 1,584 region–perturbation pairs, lower admissibility was associated with higher degradation-induced error rates (Spearman ρ\rho = -0.43, 95% CI: -0.52 to -0.33), and ICR-rejected measurements exhibited 25.9 percentage points more new errors than admitted measurements (95% CI: 16.2–32.8).

For Gleason grading, however, CONCH performed near chance on clean images (AUROC = 0.56), so the predefined primary analysis was inconclusive. Our end-to-end grading replay also did not establish a significant improvement in final accuracy (Appendix E.2, Lines 1329–1339). This reinforces that ICR cannot compensate for a diagnostically weak underlying tool, nor does stability guarantee correctness. Overall, the results support ICR as a label-free measure of evidence stability associated with degradation-induced diagnostic failures when the underlying specialist has adequate diagnostic capability.

Reviewer_LtkUOpenReview ↗

Follow-up on the Authors' Response

Q1

Question 1 is only partially addressed. The reported action changes show that XPathAgent can react to different inputs, but they may still result from predefined thresholds and routing rules within the fixed Diagnostic Atlas. Therefore, the evidence supports input-conditioned workflow adaptation, but not autonomous planning based on evolving diagnostic states or evidence gaps.

Q2

The additional zero-shot comparisons with PRISM and TITAN and the survival-specific evaluation against MDCS-MoAME are useful. However, they do not fully address the central concern regarding strong supervised slide-level models for grading, subtyping, and metastasis detection: PRISM and TITAN are evaluated as frozen zero-shot foundation models, survival is outside the main tasks raised in this question, and no supervised grading or subtyping specialist is included.

The only supervised specialist experiment is UNI+ABMIL in Table 17. Yet the manuscript does not disclose whether it was trained separately on CAMELYON16 and CAMELYON17, trained on CAMELYON16 and externally evaluated on CAMELYON17, or trained on pooled data, nor does it report the exact data split and training protocol. This is critical because the original UNI study achieved 0.957 BACC (95% CI: 0.911–0.991) on CAMELYON16 using UNI features with ABMIL and the official protocol (Lu et al., 2024, Supplementary Data Table 22, p. 46), whereas Table 17 reports only 0.803. If the same protocol was used, the baseline was not faithfully reproduced; if a different protocol was used, the comparison is uninterpretable without full disclosure. Therefore, the current evidence does not support the conclusion that XPathAgent improves an established supervised slide-level specialist.

Q3

Question 3 has been adequately addressed, as the new results support ICR as a measure of evidence stability associated with perturbation-induced errors, while appropriately distinguishing it from intrinsic diagnostic correctness.

Follow-up on [W1]:

We thank the reviewer for the follow-up. We respectfully clarify that the offline thresholds and Diagnostic Atlas do not replace autonomous planning; they provide sensor calibration and clinical guardrails within which the agent dynamically navigates, refines, and synthesizes evidence across a gigapixel WSI.

1. Calibration addresses domain shift rather than prescribing execution trajectories.
WSIs exhibit substantial variation from staining, scanner optics, and preparation artifacts. ICR bounds (akmin⁡,dkmin⁡a_k^{\min}, d_k^{\min}) are calibrated offline per tool and benchmark using 20 unlabeled images to characterize degradation under stain deviation, defocus, and resolution loss (Lines 228–250, 370–375; App. C.1–C.2, Lines 911–1005). At inference, however, reliability wk(r)w_k(r) and corroboration g~AB(r)\tilde{g}^{AB}(r) are computed dynamically from observed region statistics and within-slide variance (Eqs. 4–9, Lines 251–269, 291–298). Empirically, admissibility tracks degradation-induced errors (τ=0.67\tau=0.67, p=0.002p=0.002), while rejected measurements exhibit a 25.9 pp higher new-error rate (Table 5(b), Lines 468–472; App. E.2, Lines 1294–1328).

2. Execution remains region- and evidence-conditioned.
Across dozens of regions and hundreds of tiles (Lines 157–163; App. A, Lines 758–761), XPathAgent performs a closed-loop process (Figs. 5–7, Lines 1387–1428). The triage module prioritizes candidate ROIs from observed regional distributions; within each ROI, the Concept–Evidence Tree dynamically prunes or re-parameterizes tool routes according to morphological and reliability states (Lines 218–222, 270–276; Alg. 1, Lines 1026–1045). Low-informativeness routes can trigger up to Rmax⁡=3R_{\max}=3 refinement rounds, recovering 1,057 routes, while borderline measurements trigger LLM keep/discard adjudication (459 instances; Table 18, Lines 1266–1271, 1359). The synthesis stage then integrates surviving evidence using reliability and co-failure-discounted corroboration, resolves tool conflicts, and aggregates regional evidence into the final diagnosis (Lines 295–323; Fig. 6).

3. Clinical grounding is intentional rather than a limitation on autonomy.
Pathology diagnosis follows established medical criteria such as WHO, CAP, AJCC, and Nottingham guidelines (Table 9, Lines 864–906). Allowing an agent to invent diagnostic concepts at inference time would undermine clinical validity and auditability (Lines 167–172, 465–467; App. G, Lines 1473–1476). Similar knowledge grounding is used by recent pathology agents including PathAgent [1] and WSI-Agents [2]. Importantly, removing XPathAgent's structured grounding and calibration while retaining the same tools and reasoning model (XPathAgent-Base) consistently reduces performance, including 3.5 points in survival C-index, 3.8 BACC points on CAMELYON16, and 3.6 QWK points on SICAP (Tables 1–3, 7; Lines 450–453).

Thus, the Atlas and calibration define clinically valid operating constraints, while the regions examined, tools invoked, refinement actions, adjudications, and final evidence synthesis remain dynamically conditioned on each WSI.


Follow-up on [W2]:

We thank the reviewer for this suggestion. We have expanded the supervised-specialist evaluation to all three requested task families. For CAMELYON16, we now follow the official train/test protocol and evaluate all methods on the same held-out test set: UNI+ABMIL achieves 0.927 BACC, XPathAgent 0.888, and XPathAgent+UNI+ABMIL 0.946. The previously reported 0.822 XPathAgent result used the full 399-slide cohort, explaining the difference. We further integrate SMMILe, a supervised slide-level MIL specialist, for grading and subtyping. On SICAP, SMMILe/XPathAgent/XPathAgent+SMMILe achieve 0.844/0.809/0.863 QWK, while on TCGA-RCC they achieve 0.946/0.935/0.959 BACC, respectively. These matched comparisons show that XPathAgent can improve upon strong task-specific supervised specialists rather than relying only on zero-shot foundation models [3,4].


References

[1] Chen, Jingyun, et al. “PathAgent: Toward interpretable analysis of whole-slide pathology images via large language model-based agentic reasoning.” arXiv:2511.17052, 2025.

[2] Lyu, Xinheng, et al. “WSI-Agents: A collaborative multi-agent system for multi-modal whole slide image analysis.” arXiv:2507.14680, 2025.

[3] Chen, Richard J., et al. “Towards a general-purpose foundation model for computational pathology.” Nature Medicine, 2024.

[4] Gao, et al. “Accurate spatial quantification in computational pathology with multiple instance learning.” Nature Cancer, 2025.

Reviewer_gn56OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents an agent framework which orchestrates a range of pathology foundation models as tools and combines their predictions with an orchestrator LLM (a Qwen3.5-35B model). The orchestration happens with three modules (1) a Diagnostic Atlas, (2) a Concept–Evidence Tree and an (3) Invariance Calibrated Reliability (ICR) score. The authors evaluated the model on a wide range of tasks including survival prediction, sTIL scoring, subtyping, grading and metastasis detection. Comparing their model to MIL baselines with pathology foundation model encoders and slide level models, their agent-based approach significantly outperforms all other baselines across the tasks. Through a range of ablations (removing individual components), the authors show the importance of the individual foundation models and modules.

Critical strengths
  • The idea of orchestrating multiple pathology specific tools with a general LLM and reason about its outputs is compelling and is a promising direction to allow for large-scale reasoning across pathology datasets.

  • The comparison includes several typical tasks in histopathology and includes multiple strong baselines (various foundation model encoders + MIL) as well as slide foundation models. The gains over these baselines are quite significant.

  • There are ablations on the performance when removing one of the three main modules and removing one of the four models. In all cases, we can see a significant performance drop.

Critical weaknesses
  • Using the TITAN model as a slide-level encoder is missing from the results, although it is used in the agent framework as a tool. TITAN significantly outperforms Prism on survival tasks (results from the TITAN paper). This model should also be added as a baseline.

  • The efficiency of the model is not really discussed or compared. The agent needs to run four total patch encoding models and two slide encoder models. A discussion and measurement of the added compute would strengthen the paper.

  • Some qualitative examples about how the model calls the tools and reasons about a specific case would be helpful to understand the complex architecture and what the model uses to improve upon the baselines better.

  • Minor: The missing parentheses around the citations sometimes make the paper quite hard to read (\citep would help to make the paper more readable.)

Questions for authors
  • Did you try to use an ensemble of the various models (Conch, TITAN, Prism) with maybe something similar as the ICR score on top? This would help to disentangle how much performance contribution comes from combining multiple pathology foundation models vs. the reasoning on top of them.

  • How did you end up with this composition of models? Did you also experiment with other models and observed overall weaker performance?

1 comment
AuthorsOpenReview ↗

Response to Reviewer gn56

We thank the reviewer for the positive assessment of our multi-tool reasoning framework, broad evaluation across pathology tasks, and component/tool ablations. We also appreciate the constructive suggestions, which helped us strengthen the revised manuscript. We address each point below.

[W1] TITAN as a slide-level encoder results

We thank the reviewer for this suggestion. We have added TITAN as a standalone slide-level baseline across the applicable tasks in the revised results tables. Table 1 now includes TITAN for survival prediction, and the corresponding zero-shot tables (2 & 3) include TITAN for subtyping, metastasis detection, and sTIL scoring. All results are evaluated under the same protocol as the other foundation-model baselines, enabling a direct comparison with PRISM and XPathAgent. Under our evaluation, XPathAgent consistently outperforms TITAN across these tasks, including a mean C-index of 0.726 versus 0.638 for survival prediction.

[Q1] Did you try to use an ensemble of the various models

We added a direct ensemble of the foundation models in our pipeline using equal-weight soft voting, without the Diagnostic Atlas, CET, ICR, or reasoning model. As reported in Table 14, the ensemble achieves BACC scores of 0.759 on CAMELYON16 and 0.834 on TCGA-RCC, compared with 0.784/0.921 for XPathAgent-Base and 0.822/0.939 for the full XPathAgent. This shows that the gains cannot be explained by model ensembling alone, but instead arise from structured concept-based evidence selection and reliability-aware integration.

[Q2] Did you also experiment with other models

We initially considered a broader pool of pathology foundation models, including UNI, MUSK, PLIP, and VIRCHOW. In preliminary experiments, these models showed weaker downstream performance when integrated with the other tools, and were therefore not retained in the final agent.

[W2]

We have added an end-to-end computational-efficiency analysis on TCGA-RCC using a single NVIDIA A100 80GB GPU, reporting inference time per WSI, peak GPU memory, LLM calls, and predictive performance (Table-19). XPathAgent requires 17.0 min/WSI, 45 GB peak GPU memory, and 34 LLM calls per WSI.

For comparison, PRISM, TITAN, and HistoSelect require 2.9, 3.3, and 4.7 min/WSI, respectively, with peak memory of 12, 15, and 19 GB and BACC scores of 0.615, 0.663, and 0.875. XPathAgent achieves a BACC of 0.939, corresponding to relative improvements of 52.7%, 41.6%, and 7.3% over PRISM, TITAN, and HistoSelect, respectively. Thus, XPathAgent incurs additional computational cost from multi-tool evidence collection and iterative reasoning, while providing substantial performance gains over the compared slide-level and reasoning baselines. All specialist-model execution and route refinement are included in the end-to-end timing.

[W3] Qualitative Examples

Figures 5–7 have been added to the supplementary material, providing qualitative examples across different cases.

[W4] Citation formatting

We thank the reviewer for noting this. We will revise the citation formatting throughout the manuscript to use parenthetical citations where appropriate for improved readability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.