- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PlanPath addresses patient-level pathology visual question answering (VQA), answering questions from a patient’s full collection of whole-slide images (WSIs) rather than isolated slides. Its core workflow integrates several specialized components: an Agent Brain (Qwen3.5-4B) translates queries into exploration plans, a TITAN-based Slide Router ranks candidate slides via text–slide similarity, and PLIP performs ROI relevance ranking. Multi-magnification region examinations are further supported by Patho-R1-7B for morphological descriptions, alongside tissue-prototype analysis and HistoPLUS-based cell quantification for spatial or numerical evidence. By tracking slide, ROI, and magnification provenance, the system iteratively integrates evidence before generating a final answer. To evaluate PlanPath, the authors introduced gastric and renal datasets spanning 213 patients, 3,245 WSIs, and 1,623 expert-reviewed question–answer pairs. PlanPath achieved 60.24% accuracy on PatientVQA-Gastric and 72.71% on PatientVQA-Renal, outperforming the strongest baselines by 9.43 and 12.14 percentage points, respectively. Additional experiments cover agent compatibility, component ablations, quantitative evaluation, and qualitative cases. However, whether the evaluation rigorously isolates visually grounded, cross-slide reasoning and validates specific efficiency claims remains an open question.
Critical strengths
- Meaningful Problem and Principled System Architecture:
The paper addresses a challenging, understudied problem by transitioning from isolated-slide VQA to patient-level synthesis across multiple WSIs. PlanPath offers a coherent, provenance-aware architecture combining slide routing, multi-scale ROI exploration, and specialized tools. Crucially, its explicit exploration plans and evidence gates enhance inspectability over opaque end-to-end models.
- Extensive Empirical Validation and Strong Results:
The study provides a broad evaluation across two novel datasets (213 patients, 3,245 WSIs, 1,623 QA pairs), demonstrating notable accuracy gains over strong baselines (+9.43% on Gastric, +12.14% on Renal). Beyond headline metrics, comprehensive ablations, compatibility tests, and qualitative cases effectively illuminate the system's routing and evidence integration mechanisms.
Critical weaknesses
- Unclear Evaluation of True Cross-Slide Reasoning and Visual Grounding:
While the paper emphasizes transitioning to patient-level VQA, the benchmark evaluation fails to isolate genuine cross-slide reasoning. Because pathology report-derived QA pairs may be answerable from a single informative WSI, local visual recognition, or clinical priors, it remains unclear how many questions strictly require multi-slide synthesis. The pathologist review protocol lacks crucial details (e.g., annotation guidelines, inter-annotator agreement, and explicit counts of multi-slide dependency), and performance is not disaggregated by single- vs. multi-slide sufficiency. Without separating these cases, the reported gains cannot convincingly prove that the improvement stems from cross-slide integration rather than single-slide recognition.
- Insufficient Evidence for Quantitative Claims, Statistical Rigor, and True Efficiency:
The validation of quantitative reasoning is constrained by an unconfirmed, small sample size (appearing to be only 18 questions based on step increments), where tolerance metrics ( to ) lack clinical justification and tools like cell quantification yield a marginal ablation impact (+0.43%). Furthermore, accuracy metrics are presented solely as point estimates without confidence intervals or significance testing to account for patient-level clustering. Finally, the efficiency analysis relies strictly on prompt-token counts, omitting critical end-to-end operational costs such as tool latency, GPU time, generated output tokens, and monetary expense.
Questions for authors
Q1. Cross-Slide Dependency and Evaluation Protocol: Could the authors clarify what proportion of evaluation questions genuinely require cross-slide or multi-site synthesis, as opposed to single-slide sufficiency? Please provide the specific pathologist screening protocol, inter-annotator agreement, and per-question evidence mappings (e.g., required WSIs/ROIs). Reporting separate accuracy for single- vs. multi-slide questions is essential to validate the core task claims.
Q2. Quantitative Sub-benchmark and Tool Grounding: Please confirm the exact sample size for the quantitative subset (the 5.56% increments suggest ) and specify how ground-truth values were derived. Were tissue-prototype and HistoPLUS outputs validated against expert annotations? Additionally, what are the clinical rationales and units for the to tolerance thresholds?
Q3. Statistical Rigor and Patient-Level Uncertainty: Could the authors provide patient-level resampled confidence intervals (e.g., paired bootstrap) or significance tests for the primary accuracy results? Since multiple questions originate from the same patient, treating QA pairs as independent observations may underestimate uncertainty and obscure whether the headline gains (+9.43% and +12.14%) are statistically significant.
Q4. End-to-End Efficiency and Implementation Details: To provide a comprehensive efficiency profile, could the authors report wall-clock latency, tool-call frequency, output token counts, GPU runtime, and monetary costs alongside prompt tokens? Please also clarify the "Slide-T" setting, exact model/API decoding parameters, and whether any WSIs in the tissue prototype banks overlap with the evaluation cohorts.