acceptodds
Under review as a conference paper at ICLR 2027

PathHunter: Hierarchical Evidence Acquisition for Weakly Supervised WSI Classification

18 views

Abstract

Histopathology remains the gold standard for definitive disease diagnosis, yet locating decisive evidence within gigapixel whole-slide images (WSIs) under slide-level supervision is difficult. Most weakly supervised pipelines still treat diagnosis as passive bag aggregation, even though clinical reading is inherently sequential and multi-scale. We propose PathHunter, a hierarchical reinforcement learning (HRL) framework that reframes weakly supervised WSI diagnosis as sequential evidence acquisition using visual features alone. A high-level Manager selects candidate regions of interest (ROIs), while a low-level Worker retrieves discriminative patches within each ROI, transforming passive aggregation into active evidence construction. PathHunter is trained in a weakly supervised loop that also acts as a curriculum over action-space complexity: Stage1 learns static evidence aggregation with MIL, Stage2 learns local evidence refinement from pseudo-expert rankings, and Stage3 learns full-slide evidence acquisition by jointly optimizing the Manager, the Worker, and the downstream MIL classifier on policy-selected trajectories. To improve credit assignment, we combine step-wise rewards from gains in label-consistent confidence with instance-score-based reward shaping. Across Camelyon16 and TCGA-NSCLC, PathHunter consistently improves diverse backbone-head combinations, with the largest gains on sparse-evidence binary detection and smaller but still mostly stable gains on harder morphology-driven classification. On Camelyon16, the improvement reaches +27.21% AUC for a weak Stage 1 pipeline; on PANDA, the same training recipe improves weighted kappa on all evaluated backbone-domain pairs. PathHunter additionally yields interpretable coarse-to-fine trajectories that expose how a compact evidence set is constructed during inference. Overall, the results indicate that hierarchical evidence acquisition is most beneficial when slide labels depend on sparse decisive regions, while broader morphology tasks benefit mainly through cleaner evidence aggregation. Anonymous code is https://anonymous.4open.science/r/pathhunter-08F1/README.md.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Reinforcement Learning, Medical AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sj8sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PathHunter, a hierarchical reinforcement learning framework for weakly supervised whole-slide image (WSI) classification. Rather than treating WSI classification solely as a passive multiple-instance learning (MIL) problem, the authors formulate it as sequential evidence acquisition. A high-level Manager selects candidate regions of interest, while a low-level Worker identifies informative patches within each region. The selected evidence is subsequently aggregated by a MIL classifier. The framework adopts a three-stage training strategy consisting of MIL pretraining, pseudo-expert initialization of the Worker, and joint PPO optimization with MIL classifier adaptation. The reward combines changes in label-consistent prediction confidence with instance-score-based shaping. Experiments on Camelyon16, TCGA-NSCLC, and PANDA demonstrate improvements across several pathology foundation models and MIL architectures. The authors also provide comparisons with sequential-observation methods, component ablations, evidence-quality analysis, and computational efficiency measurements.

Critical strengths
  1. A meaningful reformulation of weakly supervised WSI classification. The paper addresses an important limitation of conventional MIL approaches, which generally aggregate predefined patch bags without explicitly optimizing evidence acquisition. Formulating evidence selection as a sequential coarse-to-fine decision process is a potentially useful direction, particularly for sparse-lesion detection. The interaction between the learned selection policy and the downstream MIL classifier provides a coherent optimization framework.

  2. Extensive evaluation across different models and pathology tasks. The evaluation covers multiple pathology foundation models and MIL architectures, with particularly strong improvements reported on Camelyon16. The paper also goes beyond standard classification comparisons by examining static selection, independently trained policies, sequential-observation baselines, and pathology annotation-based evidence quality. The reported improvements in tumor enrichment and lesion coverage provide useful evidence that the selected patches can be diagnostically relevant.

Critical weaknesses
  1. The necessity of the hierarchical architecture and the proposed reward shaping is not sufficiently established. Although the hierarchical Manager–Worker design is central to the proposed method, the results do not consistently demonstrate that both levels are necessary. In Table 3, disabling the Worker during evaluation produces exactly the same accuracy (99.23%), macro-F1 (99.18%), and AUC (100.00%) as the full model. In contrast, removing the Manager leads to a noticeable performance decrease. This suggests that most of the benefit in the canonical setting may originate from high-level ROI selection rather than the proposed hierarchical coordination. Moreover, the PANDA experiments in Tables 17 and 18 show that removing reward shaping improves weighted kappa and accuracy compared with the full-reward configuration. This raises questions about the necessity and general applicability of the instance-score-based shaping mechanism. While the existing experiments support the usefulness of learned evidence selection, they provide less convincing evidence that the complete hierarchical architecture and reward design consistently outperform simpler alternatives.

  2. Several evaluation and reproducibility issues weaken confidence in the reported performance. A number of configurations achieve nearly perfect performance on Camelyon16, including 100% precision, recall, accuracy, and AUC. Although the paper includes multiple-seed evaluations, such strong results on a relatively small test cohort warrant particularly careful examination of data partitioning, model selection, and evaluation robustness. There are also ambiguities in the experimental description. Appendix D specifies a 90%/10% training-validation partition of the 270 official Camelyon16 training slides, whereas the evidence-quality analysis refers to 27 positive validation slides and also reports negative-slide statistics. The relationship between these evaluation subsets is unclear. Furthermore, Appendix M states that test metrics were recorded during training, while the main text indicates that test data were not used for checkpoint selection. These statements are not necessarily contradictory, but the procedure should be clarified to establish that test-set observations did not influence model development. Finally, the reported PANDA patch-budget settings require clarification regarding the distinction between the maximum budget and the number of patches actually acquired. These concerns do not establish experimental invalidity, but they limit the ability to independently assess the reliability and reproducibility of the reported gains.

Questions for authors

Q1. Contribution of the Worker and hierarchical coordination. Table 3 shows identical performance for the full model and the Manager-only evaluation variant. Could the authors explain what additional benefit the Worker provides in this setting? Would a jointly trained Manager-only model achieve comparable performance across the main datasets under matched computational and evidence budgets? Additional results clarifying when hierarchical selection is preferable to a simpler single-level policy would materially strengthen the central contribution.

Q2. Necessity of reward shaping.In the PANDA experiments, removing instance-score-based reward shaping improves both accuracy and weighted kappa over the full-reward configuration. Does this behavior also occur on Camelyon16 and TCGA-NSCLC? Could the authors provide matched comparisons with and without shaping across representative backbone–MIL combinations to clarify whether the proposed reward design consistently benefits policy learning?

Q3. Data partitioning and evaluation protocol. Could the authors clarify the construction of the Camelyon16 validation subsets, particularly the relationship between the stated 10% validation split and the 27 positive validation slides used for evidence-quality analysis? Additionally, Appendix M mentions recording test metrics during training. At what stage were these metrics computed, and what procedures ensured that test-set observations did not affect checkpoint selection, hyperparameter tuning, or subsequent methodological decisions?

Q4. Evidence budget and computational interpretation. Could the authors clarify the distinction between the maximum patch budget B and the actual number of patches selected per episode, especially for the PANDA configuration with B = 2048 and (K, m) = (8, 8)? Reporting the realized evidence counts alongside computational costs would help determine the trade-offs between evidence coverage, classification performance, and inference efficiency.

1 comment

We thank the reviewer for the constructive assessment. We have re-run all evaluations label-blind with the Table 1 checkpoints, and uploaded every checkpoint and training log to the anonymous repository https://anonymous.4open.science/r/pathhunter-08F1/README.md to ensure reproducibility.

Q1/W1: What does the Worker add?

In the canonical setting, a region is one 5× tile holding a 4×4 grid of 16 patches, and m = 16. Any in-region selector therefore keeps the whole region. Full and Worker-disabled select identical patch sets on 100% of test slides, which explains the identical metrics in Table 3. The Worker matters when regions are larger than m. With K = 32 regions and m = 8 patches (same budget), the learned Worker beats random in-region selection in 11 of 12 decided runs on Camelyon16 (Acc 92.59 vs 91.95). It keeps 17% more tumour evidence, measured against pathologist annotations (tumour ratio 0.202 vs 0.172). Joint training also pays off consistently: on TCGA-NSCLC, the full model has a higher AUC than the jointly trained Manager-only model in every seed (96.45 / 96.36 / 95.87 vs 96.22 / 95.92 / 95.78).

The hierarchy is preferable to single-level patch selection. Under an independent frozen classifier (Camelyon16, ABMIL, 3 backbones × 3 seeds, same budget), PathHunter reaches 89.49 / 93.65 Acc / AUC, against 80.45 / 91.91 for flat attention top-B patch ranking (+9.0 / +1.7). The RL-trained Manager is also far better than random regions: 93.02 vs 75.80 Acc, with 7.2× more tumour evidence.

Q2/W1: Is reward shaping necessary?

We ran the matched comparison the reviewer asks for: same configuration (K = 32, m = 8), with vs without instance-score shaping, 3 seeds. Acc / AUC with the Stage 3 classifier:

With shapingWithout shaping
Camelyon16, CTransPath (ABMIL + DSMIL)89.15 / 91.1687.86 / 90.53
TCGA-NSCLC, CONCH1.5 (ABMIL + DSMIL)91.19 / 96.0191.00 / 95.82

Shaping improves accuracy and AUC on Camelyon16. With CTransPath it gains +1.3 Acc. It also helps on TCGA-NSCLC with CONCH1.5.

Q3/W2: Data partitioning, the 27 slides, and test metrics.

  • Table 1 protocol. MIL is trained on the 270 official training slides only. Its 90/10 split (243/27) is internal to these training slides and is used only for Stage 1 model selection. All 129 official test slides are held out from every training and selection step.
  • The 27 positive slides. We re-ran the analysis on the official test set with the Table 1 checkpoints, using 44 annotated positive and 80 negative slides. The baseline is the Stage 1 static ranking with the same budget. CTransPath + TransMIL, mean ± std over 3 seeds, %:
MetricStage 1 staticPathHunter
Hit@budget ↑66.67 ± 11.2178.03 ± 1.31
Tumour-patch recall ↑36.37 ± 8.9448.65 ± 2.98
Lesion coverage ↑34.41 ± 9.3544.98 ± 2.18
False-positive rate ↓ (neg.)12.08 ± 8.785.42 ± 1.91
  • Test metrics during training were logged for monitoring only. They never fed into checkpoint selection, early stopping or hyperparameter choice, as the uploaded logs show.
  • Near-perfect scores come from UNI2-H, whose Stage 1 baseline already reaches 100% precision. They are 3-seed means, obtained label-blind: the policy always searches for tumour.

Q4/W2: Budget B vs. realized evidence. B is an upper cap. The patches actually acquired per episode are at most K·m, and fewer when a region at the tissue border holds fewer than m tissue patches.

  • Camelyon16 (K = m = 16, cap 256): 200.6 patches on average.
  • TCGA-NSCLC (K = m = 16, cap 256): 222.7 patches on average.

Evidence coverage trades off smoothly against accuracy. Camelyon16, 12 configurations × 3 seeds, with the full-bag Stage 1 baseline (~11k patches) at 90.12 Acc:

K = m481216
Patches1664144256
Stage 3 Acc90.2991.2692.1493.48
Stage 3 AUC92.5992.6294.3095.51

With only 16 patches per slide, PathHunter already matches the full-bag baseline. With 256 patches (about 2% of the slide), it exceeds the baseline by 3.4 Acc.

Reviewer_79ssOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PathHunter, a hierarchical reinforcement learning framework for evidence selection in weakly supervised whole-slide image (WSI) classification. The method employs a high-level Manager to select regions of interest and a low-level Worker to identify informative patches within them. It adopts a three-stage training strategy, consisting of MIL baseline training, pseudo-expert initialization of the Worker, and joint optimization of the hierarchical policies using PPO while adapting the MIL classifier to the selected patches.

The authors evaluate PathHunter on Camelyon16, TCGA-NSCLC, and PANDA, demonstrating performance improvements across multiple feature extractors and MIL architectures, particularly for sparse tumor detection. The paper also includes comparisons with existing sequential observation methods, ablation studies, and evidence localization analyses.

The main strengths of this work are its meaningful problem formulation and relatively comprehensive experimental evaluation. However, the improvements are less consistent across more complex pathology tasks. More importantly, the ablation results raise questions about the necessity of the full hierarchical architecture, as disabling the Worker at inference does not affect performance in the main ablation setting. Thus, while the overall effectiveness of learned evidence selection is supported, the individual contribution of the hierarchical design remains insufficiently established.

Critical strengths

1. Meaningful problem formulation.

The paper addresses an important problem in weakly supervised WSI classification: how to identify informative regions and construct compact evidence sets using only slide-level supervision. The proposed hierarchical ROI-to-patch selection framework provides an intuitive approach to this problem, particularly for tasks involving sparse diagnostic evidence.

2. Relatively comprehensive experimental evaluation.

The authors evaluate PathHunter across multiple pathology datasets, feature extractors, and MIL architectures. The experiments demonstrate promising improvements, particularly on Camelyon16. Comparisons with existing sequential observation methods, including ZoomMIL and SASHA, further strengthen the empirical evaluation and support the effectiveness of the overall framework.

3. Evidence quality evaluation beyond classification performance.

Beyond slide-level classification metrics, the paper evaluates the quality of selected evidence using tumor annotations, demonstrating improvements in tumor enrichment and lesion coverage. The authors also provide component ablations and transferability analyses. These evaluations are valuable because they attempt to verify whether the selected patches contain meaningful diagnostic information rather than merely improving classification scores.

Critical weaknesses

1. Inconsistencies in experimental reporting.

There are inconsistencies between the reported performance values and their corresponding improvements. For example, in Table 1, the UNI2-H + ABMIL results on Camelyon16 show an accuracy improvement from 87.60 to 95.66, but the reported gain is +8.52 rather than +8.06. Similarly, the recall improvement from 83.67 to 91.84 is reported as +6.13 rather than +8.17. Interestingly, these reported gains correspond to different absolute values in Appendix Table 15. While this may reflect differences between experimental runs, the relationship between these results is not clearly explained. These inconsistencies raise concerns about the reliability and reproducibility of the reported experimental results.

2. Insufficient evidence for the necessity of the hierarchical components.

Although the proposed framework achieves promising classification performance, the ablation studies do not convincingly establish the necessity of the full Manager–Worker architecture. In Table 3, disabling the Worker during inference produces exactly the same accuracy, Macro-F1, and AUC as the full model. The independently trained Manager-only model also achieves very similar performance. While Appendix Table 11 reports a modest AUC improvement from including the Worker under a different setting, its contribution appears limited and potentially configuration-dependent. Since hierarchical evidence selection is a central contribution of this paper, stronger and more consistent evidence is needed to justify the additional complexity of the Worker and the overall hierarchical design.

3. Discrepancies between the released implementation and the described methodology, with concerns about evaluation validity.

The released implementation raises several important concerns. In the evaluation pipeline, the ground-truth slide label appears to be passed into the RL environment and used as class_idx when computing class-dependent patch instance scores. Since these scores influence candidate construction and ROI selection, this could introduce label leakage during inference for certain MIL architectures. Additionally, the default code configuration appears to construct only 16 candidate bins per ROI while allowing the Worker to select 16 bins, potentially leaving little meaningful selection flexibility. This also differs from the 16×16 bin structure described in the paper. Furthermore, the released advantage computation does not appear to implement the discounted returns and GAE procedure described in the manuscript. These issues require clarification, as they directly affect the validity, reproducibility, and interpretation of the proposed method. It is currently unclear whether the released implementation exactly matches the configuration used to obtain the reported results.

Questions for authors

Q1. Experimental result consistency

Could the authors clarify the discrepancies between the absolute performance values and the reported improvements in Table 1, particularly for UNI2-H + ABMIL on Camelyon16? Some improvements appear to correspond to the results in Appendix Table 15 rather than the main table. Were these results obtained from different experimental runs or evaluation protocols? Providing consistent results and the corresponding per-seed statistics would help resolve this concern.

Q2. Necessity and effectiveness of the Worker

In Table 3, disabling the Worker at inference yields identical performance to the full PathHunter model. However, Appendix Table 11 reports a small AUC difference under another setting. Could the authors explain these observations and clarify the specific contribution of the Worker? In particular, could they provide matched comparisons between the full model and Manager-only variants across multiple backbones, using the same evaluation protocol and patch budget?

Q3. Evaluation protocol and potential label leakage

In the released code, the evaluation pipeline appears to pass the ground-truth slide label into the environment, where it is used to compute class-dependent instance scores for certain MIL architectures. Since these scores influence candidate construction, could the authors clarify whether the reported results were obtained using this evaluation path? If so, could they provide label-blind evaluation results in which ground-truth test labels are not available to the evidence-selection policy?

Q4. Implementation consistency and sequential decision making

The released configuration appears to use 16 candidate bins per ROI while selecting up to 16 bins, whereas the paper describes a 16×16 candidate structure. Could the authors clarify the actual configuration used in the experiments and how meaningful Worker selection is maintained? Additionally, could the authors explain how previously selected evidence affects subsequent policy decisions, and reconcile the released advantage computation with the PPO/GAE formulation described in the manuscript?

Overall assessment and potential rating update

My current assessment is 2/4. I appreciate the meaningful problem setting and the promising experimental results. However, the concerns above prevent me from fully assessing the validity and necessity of the proposed hierarchical RL framework. I would consider increasing my rating to 3/4 if the authors can convincingly resolve the reporting and implementation discrepancies, demonstrate the independent contribution of the Worker, and confirm that the main results remain valid under a strictly label-blind evaluation protocol.

3 comments

We thank the reviewer for the careful reading, including of the released code. All new results are label-blind and come from the released code, and we have uploaded every checkpoint and training logs to the anonymous repository https://anonymous.4open.science/r/pathhunter-08F1/README.md to ensure reproducibility.

Q1/W1: Inconsistent values in Table 1.

The previous Table 1 inadvertently mixed results from different evaluation runs, leaving some reported improvements (Δ) inconsistent with their absolute values. The original experiments were trained on H20 GPUs using one training seed and five evaluation seeds. Following checkpoint issues, some results were re-evaluated on AMD MI250 GCD, contributing to the reporting inconsistencies. To resolve this, we reran all experiments on LUMI using three independent training seeds (0, 1, 2), with consistent evaluation protocols. All Δ values have been recomputed, and per-seed results has provided in the uploaded logs.

For UNI2-H + ABMIL on Camelyon16, the corrected results are:

Metric (%)Stage 1PathHunterΔ
Accuracy91.73 ± 1.3297.93 ± 1.32+6.20
Recall78.23 ± 3.4794.56 ± 3.47+16.33
AUC97.52 ± 1.6498.72 ± 1.35+1.20

The main conclusions remain unchanged: PathHunter improves accuracy in 11 of 12 Camelyon16 configurations.

Q2/W2: Contribution of the Worker. In the default setting, a region is one 5× tile holding a 4×4 grid of 16 patches, and m = 16. Any in-region selector therefore keeps the whole region, so Full and Worker-disabled select identical patch sets on 100% of test slides. The identical metrics in Table 3 follow from this, and we will state it in the caption.

When the Worker has a real choice (K = 32 regions, m = 8 patches, same budget), it learns useful selection. On Camelyon16, over 3 backbones × {ABMIL, DSMIL} × 3 seeds, the learned Worker beats random in-region selection in 11 of 12 decided runs (Acc 92.59 vs 91.95). It keeps 17% more tumour evidence (tumour ratio 0.202 vs 0.172), measured against lesion annotations, and the gap holds in every K = 32 variant we trained.

The joint training also pays off consistently. On NSCLC, the full model has a higher AUC than the independently trained Manager-only model in all three seeds:

  • Seed 0: 96.45 vs 96.22
  • Seed 1: 96.36 vs 95.92
  • Seed 2: 95.87 vs 95.78

Q3/W2: Label leakage at evaluation. The core idea of PathHunter is to hunt for tumour evidence: the policy is trained to find tumour regions, and at test time it always searches for tumour. In the RL training process on Camelyon16, the search class is fixed to tumour (class 1) for every slide, in training and in testing alike, positive or negative, so it is not test-leakage. A negative slide is classified as negative because the tumour hunt finds no convincing evidence. But for MIL we still use true label to training, but for testing it uses RL force to tumour to search cancer patch and predict with the label use MIL, so the results are obtained with this label-blind path.

On TCGA-NSCLC, the goal was to identify two types of tumor patches for prediction by MIL, never the ground truth. In the strictest test, evidence quality is measured with pathologist lesion annotations that are read only after selection. On Camelyon16, PathHunter covers far more of the lesions than the static ranking with the same budget:

  • Hit@budget 66.7 → 78.0
  • Tumour-patch recall 36.4 → 48.7
  • Lesion coverage 34.4 → 45.0

It also lowers the false-positive rate on negative slides from 12.1% to 5.4% (CTransPath + TransMIL, 3 seeds).

Ground-truth slide labels are used only where weak supervision allows. They train the Stage 1 MIL classifier, as in any MIL method, and they form the slide-level RL reward during training. Test labels never enter candidate construction, region selection, patch selection or checkpoint selection. The uploaded training logs show this end to end.

Q4/W4: Configuration, sequential decisions, GAE.

  • Configuration: "16 × 16" in the paper means K × m = 16 rounds × 16 patches (B = 256), not 16 candidate bins of 16 patches each. A region holds up to 16 patches. The released code matches the experiments.
  • Sequential decisions: the Manager scores regions with a shared network, and a visited mask removes selected regions, so it acts as a learned region ranker trained with RL. RL is needed because top-k selection is non-differentiable and no region labels exist. With the same budget, the trained ranker puts 7.2× more tumour evidence into the bag than random regions (Camelyon16 ABMIL: Acc 93.02 vs 75.80).
  • Advantage: the code uses reward-to-go returns with a learned value baseline (A = G − V), a special case of GAE (γ = λ = 1) that is suited to our short, fixed-length episodes.

Thank you for the clarifications and re-runs. That said, the response confirms that in the main Table 1 setting (K=m=16) the Worker keeps all 16 patches per region and the Manager acts as a history-free region ranker, while gains under K=32, m=8 are minor. In addition, the original Table 1 mixed inconsistent runs without a revised PDF updating all tables, and the label-blind handling of class_idx on multi-class tasks (NSCLC and PANDA) is still not clearly explained. I will keep my rating at 2.

We thank the reviewer for the follow-up and respect the decision. We need to clarify for the revision that we have already uploaded the new version of the PDF previously, especially for the table1 and table13 we also hight it with yellow color, now we update it again and it is showed the result of 3 training seed mean±var. And now it has the same consistent value.

BackboneMethodPrec.Rec.Acc.AUCWK
CTransPathABMIL92.76±1.4370.07±3.4786.56±1.5986.28±3.6170.04±3.72
CTransPathPathHunter + ABMIL95.06±3.49 (+2.30)74.83±2.55 (+4.76)88.89±0.37 (+2.33)90.54±4.36 (+4.26)75.41±0.75 (+5.37)
UNI2-HABMIL100.00±0.0078.23±3.4791.73±1.3297.52±1.6481.65±3.06
UNI2-HPathHunter + ABMIL100.00±0.00 (+0.00)94.56±3.47 (+16.33)97.93±1.32 (+6.20)98.72±1.35 (+1.20)95.55±2.87 (+13.89)
CONCH1.5ABMIL96.93±2.1781.63±5.0091.99±1.3293.05±0.8082.43±3.09
CONCH1.5PathHunter + ABMIL94.68±2.12 (-2.25)84.35±1.92 (+2.72)92.25±1.27 (+0.26)95.04±2.20 (+1.99)83.19±2.75 (+0.76)

Response on K and m. Actually according to our findings in different datasets, K and m should be adjusted to the tissue characteristics of different dataset. In CAMELYON16, tumor regions are often spatially concentrated, so K=m=16 works well. In contrast, diagnostic regions in NSCLC and PANDA are more widely distributed, making a larger K useful for broader coverage, while a smaller m reduces the number of patches selected per region under a fixed budget. For example, with B=256, K=32, m=8 improves CONCH1.5+ABMIL on NSCLC from 90.0 to 91.7 in accuracy and 89.7 to 91.3 in F1, compared with K=m=16. Similarly, on CAMELYON16, CTransPath+DSMIL improves from 87.3 to 88.9 in accuracy and 81.0 to 83.4 in F1. These results suggest that K=m is not always optimal, and modest task-specific adjustments can improve performance. We provide the full sensitivity analysis in Figure 3. One more thing we need to clarify is that our pathhunter's novelty is not just improve the performance but also give a validate trajectory for the doctor to check the result. We also add more PathBench results,

TaskEvidence characteristicBackboneStage 1 → Stage 3Δ Macro-F1
CPTAC-COAD ACVR2A mutationHeterogeneous tumor morphologyUNIAUC: 0.693 → 0.880 (+0.187)+0.139
CPTAC-BRCA TP53 mutationBreast-tumor morphologyH-optimus-1AUC: 0.781 → 0.854 (+0.073)+0.295
CPTAC-COAD MSI-HTumor and immune contextCONCH1.5AUC: 0.893 → 0.964 (+0.071)+0.296
CRC lymphovascular invasionTumor–vascular and tissue contextUNIAUC: 0.806 → 0.847 (+0.042)+0.048

The cases suggest that the gains are not confined to locating a single malignant focus. Mutation prediction may depend on heterogeneous tumor morphology, MSI-H may involve tumor- and immune-associated patterns, and lymphovascular invasion requires evidence around tumor boundaries, vascular structures, and surrounding tissue. The results show that add the search area for k/m/t can also improve the performance.

class_idx at test time. I want to claim our contribution again, our core idea is to use RL to find cancer area, so on CAMELYON16 it is fixed to the positive class for every slide. On NSCLC/PANDA it is using RL to search both the cancer area, for example it lets RL to search 0 and 1 patches and give MIL to predict, when test time prediction, it lets RL to search the both possible cancer patches and let MIL according to the selected batches from RL to predict. We have detailed the process in the Appendix D.

Reviewer_oSxSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents PathHunter, which uses reinforcement learning to select patches for whole-slide image classification. It first chooses regions, then selects patches within those regions for a MIL classifier. The method uses precomputed image features and jointly trains the selector and classifier after an initial warm-up. Experiments on three pathology datasets report improved classification performance, with the largest gains on Camelyon16.

Critical strengths
  1. The experiments cover several feature extractors, MIL heads, and pathology datasets, with particularly strong gains on Camelyon16. This provides useful evidence that the overall approach can improve classification when diagnostic evidence is sparse.

  2. The authors evaluate the selected patches against lesion annotations and independent classifiers. The improved lesion coverage and positive results with some independent classifiers suggest that the selected patches contain useful information beyond what benefits the original classifier.

Critical weaknesses
  1. The benefit of sequential RL over a learned ranking model is unclear. From the description, the Manager scores regions independently and mainly updates the visited mask. It is not clear how previously selected evidence changes its preference among the remaining regions. The static-ranking and PPO ablations show that training helps, but do not establish that sequential decision-making is needed. A matched comparison with a trainable ranking model is missing.

  2. The selector and classifier are trained together, so some gains may depend on their adaptation to each other. The frozen-classifier and lesion-annotation results partly address this concern. However, the independent-evaluator results are mixed and compare selected bags with full bags. It remains unclear whether PathHunter outperforms strong ranking-based selectors at the same patch budget when evaluated by an independent classifier.

Questions for authors
  1. Does the Manager update its scores for unvisited regions based on the patches already selected? If not, is its selection equivalent to ranking regions once? What evidence supports the sequential setup over a comparably trained ranking model?

  2. The frozen-classifier comparison is helpful. Under an independent frozen classifier that was not involved in policy training, how does PathHunter compare with strong static or learned-ranking selectors using the same candidate pool and patch budget?

  3. In the evaluation-only ablations in Table 3, Worker-only matches Stage-3 static selection, while Manager-only matches the full model. What replaces each disabled policy? Are the selected patches and predictions also identical, or only the reported metrics? Does the Worker mainly help during training, and is the small gain over independently trained Manager-only consistent across seeds?

1 comment

We thank the reviewer for the precise questions. Comparisons within each experiment use matched candidate pools and patch budgets; all test-time selection is label-blind. Results are averaged over three training seeds.

W1/Q1: Is the Manager truly sequential, and why RL rather than a learned ranker?

The Manager applies a shared MLP to candidate regions and masks previously visited regions. It does not update unvisited-region scores using acquired patch content; its behavior is thus best described as an RL-trained region ranker, not history-conditioned reasoning. We will revise this terminology. RL allows optimization from slide-level rewards without region annotations.

(a) Learned vs. random region selection (Camelyon16, ABMIL, 3 backbones × 3 seeds; trained Worker; B=256):

SelectorAcc / AUC (%)Tumor ratio
PathHunter93.02 / 94.770.187
Random regions75.80 / 69.070.026

The learned ranking enriches tumor evidence 7.2× relative to random selection.

(b) Learned vs. Stage-1 static ranking (Camelyon16, CTransPath+TransMIL, 3 seeds; annotations used only post hoc):

Positive-slide metric (%)StaticPathHunter
Hit@budget66.67±11.2178.03±1.31
Tumor-patch recall36.37±8.9448.65±2.98
Lesion coverage34.41±9.3544.98±2.18
Patch precision20.64±2.2524.00±0.54

These annotation-based gains do not depend on an evaluating classifier. On negative slides, false-positive rate also falls from 12.08% to 5.42%.

W2/Q2: Does the advantage persist under an independent frozen classifier?

We use a frozen Stage-1 classifier from a different seed, never involved in policy training. On Camelyon16 (ABMIL, 3 backbones × 3 seeds), all selected-bag methods use the same candidate pool and B=256:

Evidence selectorPatchesIndependent Acc / AUC (%)
Full bag~11k90.09 / 92.28
PathHunter25689.49 / 93.65
Stage-1 attention top-B25680.45 / 91.91
Random B25684.32 / 81.77

PathHunter exceeds equal-budget static ranking by +9.04 accuracy and +1.74 AUC points. With ~2% of the patches, it is only 0.60 accuracy points below full-bag evaluation while improving AUC by 1.37 points. Together with post-hoc lesion localization, this supports a selection benefit beyond direct selector–classifier co-adaptation.

Q3: What replaces disabled policies? Are outputs identical? Does the Worker help?

In Table 3, Manager-only replaces the Worker with MIL-score top-m patches within learned-Manager ROIs; Worker-only replaces the Manager with static-ranked ROIs while retaining the learned Worker.

Under the default K=m=16 setting, each ROI contains 16 spatial positions, all retained. Consequently, Manager-only and Full select identical patches on 100% of test slides across 72 runs (2 datasets × 12 configurations × 3 seeds); predictions agree on 99.7% (Camelyon16) and 99.9% (NSCLC), with residual differences attributed to GPU nondeterminism. Worker-only similarly reduces to static region selection.

When the Worker has genuine choices (K=32, m=8; unchanged B=256), it improves in-region evidence selection (Camelyon16, 3 backbones × 2 MIL heads × 3 seeds):

In-ROI selectorAcc / AUC (%)Tumor ratio
Learned Worker92.59 / 94.940.202
Random m patches91.95 / 94.610.172

The Worker yields 17% greater tumor enrichment . This persists with a history-conditioned Manager (0.201 vs. 0.172) and without reward shaping (0.202 vs. 0.173).

Joint-training contribution (NSCLC, K=32, m=8, six configurations per seed): Full vs. independently trained Manager-only AUC is 96.45 vs. 96.22 (seed 0), 96.36 vs. 95.92 (seed 1), and 95.87 vs. 95.78 (seed 2), with means 96.22 vs. 95.97. Full wins in every seed.

Comments on the paper

1 comment

We thank the reviewers for their careful assessment and constructive feedback. The discussion helped us clarify both the contribution and the limitations of PathHunter, correct inconsistencies in the original reporting, and strengthen the evidence for the method through additional experiments and analyses.

Core contribution. PathHunter addresses a fundamental question in weakly supervised whole-slide image classification: how can a model learn to construct an informative evidence bag when only slide-level labels are available? We use RL to guide the selection of regions and patches relevant to the target class. The selected patches are passed to a downstream MIL classifier, which adapts to the policy-selected evidence during joint training. Slide-level labels supervise policy learning through the training reward, without requiring patch-level annotations. At inference, selection is label-blind: for example, on Camelyon16 the policy searches for tumor evidence on every slide, regardless of its ground-truth label.

We believe this formulation makes a contribution beyond improving a particular MIL architecture. It connects weakly supervised evidence acquisition and downstream prediction: the selector learns which evidence to provide, while the MIL model learns to classify the resulting bag. The selected regions, patches, and acquisition trajectory also provide a trace that can be inspected alongside the prediction.

What changed during rebuttal. In response to the reviewers’ concerns, we made the following changes and clarifications:

  1. Corrected reporting and improved reproducibility. We acknowledged that the original Table 1 mixed values from different evaluation runs. We reran the experiments under a consistent protocol with three independent training seeds, corrected the reported improvements, updated the manuscript tables, and released checkpoints and training logs. The corrected results retain the main finding: PathHunter improves accuracy in 11 of 12 Camelyon16 backbone–MIL configurations. It also surpass the performance than RL method SASHA (NIPS 2025).

  2. Clarified label-blind evaluation. We explained how target classes are used during selection and distinguished training supervision from test-time inference. On Camelyon16, the target is fixed to tumor for every test slide; on the multiclass tasks, the policy searches for evidence associated with the possible classes, and MIL makes the final prediction. Ground-truth test labels are not used to construct candidates or select regions and patches.

  3. lmitations of the Manager and Worker. We now describe the Manager as an RL-trained region ranker, rather than claiming that its scores are conditioned on previously acquired patch content. In the main K=m=16K=m=16 configuration, each selected region contains at most 16 patches, so the Worker retains the entire region; this explains why disabling it at inference produces the same patch sets. When the Worker has a genuine choice under the same patch budget (K=32,m=8K=32,m=8), it increases the tumor ratio from 0.172 to 0.202 and improves average accuracy from 91.95% to 92.59% over random in-region selection. Its contribution is therefore meaningful but configuration-dependent.

  4. Added evidence beyond the jointly adapted classifier. PathHunter improves accuracy from 80.45% to 89.49% and AUC from 91.91% to 93.65% under an independent frozen classifier. Post-hoc lesion-annotation analysis also shows increased tumor-patch recall, from 36.37% to 48.65%, and lesion coverage, from 34.41% to 44.98%, on the reported Camelyon16 configuration. These analyses support the conclusion that the selected evidence has value beyond adaptation to the jointly trained classifier.

Why we believe the paper is above the acceptance threshold. PathHunter offers a clear approach to learning compact evidence sets from slide-level supervision, and its evaluation goes beyond reporting classification accuracy alone. Across Camelyon16, TCGA-NSCLC, and PANDA, the study covers multiple feature backbones and MIL architectures; the strongest gains occur on sparse-evidence metastasis detection, while gains on more complex morphology-driven tasks are smaller and task-dependent. The rebuttal further strengthens the case through corrected multi-seed results, label-blind evaluation clarifications, controlled Worker experiments, an independent-classifier comparison, and annotation-based evidence analysis.

We recognize that the current Manager is not a history-conditioned reasoning agent, that the Worker’s benefit depends on the selection configuration, and that a matched comparison against a separately trained ranking model remains valuable future work. Nevertheless, we believe the combination of a useful weakly supervised evidence-acquisition formulation, broad empirical evaluation, improved lesion localization, and an inspectable selection trajectory constitutes a meaningful contribution that merits acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.