- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PathHunter, a hierarchical reinforcement learning framework for weakly supervised whole-slide image (WSI) classification. Rather than treating WSI classification solely as a passive multiple-instance learning (MIL) problem, the authors formulate it as sequential evidence acquisition. A high-level Manager selects candidate regions of interest, while a low-level Worker identifies informative patches within each region. The selected evidence is subsequently aggregated by a MIL classifier. The framework adopts a three-stage training strategy consisting of MIL pretraining, pseudo-expert initialization of the Worker, and joint PPO optimization with MIL classifier adaptation. The reward combines changes in label-consistent prediction confidence with instance-score-based shaping. Experiments on Camelyon16, TCGA-NSCLC, and PANDA demonstrate improvements across several pathology foundation models and MIL architectures. The authors also provide comparisons with sequential-observation methods, component ablations, evidence-quality analysis, and computational efficiency measurements.
Critical strengths
-
A meaningful reformulation of weakly supervised WSI classification. The paper addresses an important limitation of conventional MIL approaches, which generally aggregate predefined patch bags without explicitly optimizing evidence acquisition. Formulating evidence selection as a sequential coarse-to-fine decision process is a potentially useful direction, particularly for sparse-lesion detection. The interaction between the learned selection policy and the downstream MIL classifier provides a coherent optimization framework.
-
Extensive evaluation across different models and pathology tasks. The evaluation covers multiple pathology foundation models and MIL architectures, with particularly strong improvements reported on Camelyon16. The paper also goes beyond standard classification comparisons by examining static selection, independently trained policies, sequential-observation baselines, and pathology annotation-based evidence quality. The reported improvements in tumor enrichment and lesion coverage provide useful evidence that the selected patches can be diagnostically relevant.
Critical weaknesses
-
The necessity of the hierarchical architecture and the proposed reward shaping is not sufficiently established. Although the hierarchical Manager–Worker design is central to the proposed method, the results do not consistently demonstrate that both levels are necessary. In Table 3, disabling the Worker during evaluation produces exactly the same accuracy (99.23%), macro-F1 (99.18%), and AUC (100.00%) as the full model. In contrast, removing the Manager leads to a noticeable performance decrease. This suggests that most of the benefit in the canonical setting may originate from high-level ROI selection rather than the proposed hierarchical coordination. Moreover, the PANDA experiments in Tables 17 and 18 show that removing reward shaping improves weighted kappa and accuracy compared with the full-reward configuration. This raises questions about the necessity and general applicability of the instance-score-based shaping mechanism. While the existing experiments support the usefulness of learned evidence selection, they provide less convincing evidence that the complete hierarchical architecture and reward design consistently outperform simpler alternatives.
-
Several evaluation and reproducibility issues weaken confidence in the reported performance. A number of configurations achieve nearly perfect performance on Camelyon16, including 100% precision, recall, accuracy, and AUC. Although the paper includes multiple-seed evaluations, such strong results on a relatively small test cohort warrant particularly careful examination of data partitioning, model selection, and evaluation robustness. There are also ambiguities in the experimental description. Appendix D specifies a 90%/10% training-validation partition of the 270 official Camelyon16 training slides, whereas the evidence-quality analysis refers to 27 positive validation slides and also reports negative-slide statistics. The relationship between these evaluation subsets is unclear. Furthermore, Appendix M states that test metrics were recorded during training, while the main text indicates that test data were not used for checkpoint selection. These statements are not necessarily contradictory, but the procedure should be clarified to establish that test-set observations did not influence model development. Finally, the reported PANDA patch-budget settings require clarification regarding the distinction between the maximum budget and the number of patches actually acquired. These concerns do not establish experimental invalidity, but they limit the ability to independently assess the reliability and reproducibility of the reported gains.
Questions for authors
Q1. Contribution of the Worker and hierarchical coordination. Table 3 shows identical performance for the full model and the Manager-only evaluation variant. Could the authors explain what additional benefit the Worker provides in this setting? Would a jointly trained Manager-only model achieve comparable performance across the main datasets under matched computational and evidence budgets? Additional results clarifying when hierarchical selection is preferable to a simpler single-level policy would materially strengthen the central contribution.
Q2. Necessity of reward shaping.In the PANDA experiments, removing instance-score-based reward shaping improves both accuracy and weighted kappa over the full-reward configuration. Does this behavior also occur on Camelyon16 and TCGA-NSCLC? Could the authors provide matched comparisons with and without shaping across representative backbone–MIL combinations to clarify whether the proposed reward design consistently benefits policy learning?
Q3. Data partitioning and evaluation protocol. Could the authors clarify the construction of the Camelyon16 validation subsets, particularly the relationship between the stated 10% validation split and the 27 positive validation slides used for evidence-quality analysis? Additionally, Appendix M mentions recording test metrics during training. At what stage were these metrics computed, and what procedures ensured that test-set observations did not affect checkpoint selection, hyperparameter tuning, or subsequent methodological decisions?
Q4. Evidence budget and computational interpretation. Could the authors clarify the distinction between the maximum patch budget B and the actual number of patches selected per episode, especially for the PANDA configuration with B = 2048 and (K, m) = (8, 8)? Reporting the realized evidence counts alongside computational costs would help determine the trade-offs between evidence coverage, classification performance, and inference efficiency.
We thank the reviewer for the constructive assessment. We have re-run all evaluations label-blind with the Table 1 checkpoints, and uploaded every checkpoint and training log to the anonymous repository https://anonymous.4open.science/r/pathhunter-08F1/README.md to ensure reproducibility.
Q1/W1: What does the Worker add?
In the canonical setting, a region is one 5× tile holding a 4×4 grid of 16 patches, and m = 16. Any in-region selector therefore keeps the whole region. Full and Worker-disabled select identical patch sets on 100% of test slides, which explains the identical metrics in Table 3. The Worker matters when regions are larger than m. With K = 32 regions and m = 8 patches (same budget), the learned Worker beats random in-region selection in 11 of 12 decided runs on Camelyon16 (Acc 92.59 vs 91.95). It keeps 17% more tumour evidence, measured against pathologist annotations (tumour ratio 0.202 vs 0.172). Joint training also pays off consistently: on TCGA-NSCLC, the full model has a higher AUC than the jointly trained Manager-only model in every seed (96.45 / 96.36 / 95.87 vs 96.22 / 95.92 / 95.78).
The hierarchy is preferable to single-level patch selection. Under an independent frozen classifier (Camelyon16, ABMIL, 3 backbones × 3 seeds, same budget), PathHunter reaches 89.49 / 93.65 Acc / AUC, against 80.45 / 91.91 for flat attention top-B patch ranking (+9.0 / +1.7). The RL-trained Manager is also far better than random regions: 93.02 vs 75.80 Acc, with 7.2× more tumour evidence.
Q2/W1: Is reward shaping necessary?
We ran the matched comparison the reviewer asks for: same configuration (K = 32, m = 8), with vs without instance-score shaping, 3 seeds. Acc / AUC with the Stage 3 classifier:
Shaping improves accuracy and AUC on Camelyon16. With CTransPath it gains +1.3 Acc. It also helps on TCGA-NSCLC with CONCH1.5.
Q3/W2: Data partitioning, the 27 slides, and test metrics.
Q4/W2: Budget B vs. realized evidence. B is an upper cap. The patches actually acquired per episode are at most K·m, and fewer when a region at the tissue border holds fewer than m tissue patches.
Evidence coverage trades off smoothly against accuracy. Camelyon16, 12 configurations × 3 seeds, with the full-bag Stage 1 baseline (~11k patches) at 90.12 Acc:
With only 16 patches per slide, PathHunter already matches the full-bag baseline. With 256 patches (about 2% of the slide), it exceeds the baseline by 3.4 Acc.