acceptodds
Withdrawn from ICLR 2027

BU-HIM: Bayesian Uncertainty-Guided Hard Instance Mining for Whole Slide Image Classification

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Whole slide image (WSI) classification is a fundamental task in computational pathology, yet its large-scale and weakly annotated nature poses significant challenges for effective representation learning. Existing multiple instance learning (MIL) methods typically rely on deterministic attention mechanisms to identify informative instances, but instances with comparable attention may exhibit substantially different levels of epistemic uncertainty, leading to suboptimal instance selection. To address this issue, we propose BU-HIM, a plug-and- play Bayesian uncertainty-guided hard instance mining framework for weakly supervised WSI classification. Specifically, aBayesian variational attention mechanism is introduced to jointly characterize instance attention and epistemic uncertainty, where attention variance serves as an uncertainty-aware criterion for identifying informative hard instances. Furthermore, a teacher–student collaborative learning strategy is developed to adaptively optimize the model using the selected hard instances while preserving global contextual information from the entire slide. Extensive experiments on the CAMELYON16 and PANDA benchmark datasets demonstrate that BU-HIM consistently improves the performance of various MIL backbones and achieves superior uncertainty estimation compared with MC-Dropout-based approaches. These results validate BU-HIM as an effective, uncertainty-aware, and backbone-compatible training framework for weakly supervised WSI classification.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Uncertainty Modeling;Multiple Instance Learning;Hard Instance Mining;Bayesian Variational Attention;Computational Pathology

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_E425OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The proposed BU-HIM identifies a limitation of existing MIL methods for WSI classification, which select patches based on attention scores alone. Even patches with similar attention scores may differ in model confidence, but existing methods do not adequately account for this epistemic uncertainty. To address this limitation, BU-HIM introduces a plug-and-play framework based on Bayesian variational attention that jointly estimates attention importance and epistemic uncertainty. It constructs a hard instance set by selecting both highly important and highly uncertain patches, and improves diagnostic performance through teacher-student collaborative learning that combines local hard-instance learning with the preservation of slide-level context. Experiments across three MIL backbones and two benchmark datasets demonstrate the effectiveness of the proposed framework. The results further show that among patches with similar attention scores, those with higher uncertainty can have an important influence on slide-level classification.

Critical strengths
  • The motivation of the paper is well articulated, maintaining a consistent argument across the Introduction, Related Work, and the beginning of the Method section.

  • The proposed plug-in epistemic uncertainty methodology offers strong scalability. Furthermore, unlike prior work that remains limited to uncertainty quantification or interpretability, this approach directly addresses the mining process itself.

Critical weaknesses
  1. I would like to comment on the claim made in Lines 46-52, which serves as a central motivation repeated throughout the paper. The statement following “However” appears to be an assertion, without supporting experimental evidence, mathematical justification, or citations to prior works. Is the intended argument that MIL, which involves a large number of instances, inherently suffers from uncertainty due to the use of deterministic gated attention? However, state-of-the-art MIL models have already achieved substantial performance improvements without explicitly accounting for this uncertainty [1-3]. As currently presented, the claim seems overly broad and ambiguous, potentially suggesting that deterministic attention mechanisms themselves are fundamentally problematic.

  2. Dual-teacher methods that rely on Bayesian components, such as BU-HIM, are relatively common approaches [17-19]. It is difficult to identify contributions beyond applying an existing paradigm to MIL, making the proposed methodology appear incremental. Furthermore, top-k selection has already been used in score-based MIL methods [4,5]. While I understand that the method iteratively updates Bayesian mining through training, BU-HIM ultimately still performs deterministic inference.

  3. I disagree with generalizing the attention mechanism presented in Equation (1) to all MIL methods. Equation (1) describes gated attention, which was first introduced in AB-MIL [6]. This was a significant contribution that incorporated gated attention into MIL and demonstrated strong performance and interpretability. However, following the introduction of self-attention, most modern MIL architectures have adopted Transformer-based structures [7-10]. These architectures have moved beyond the gated-attention paradigm, in which each instance is assigned a single attention score aia_i.

    • Graph-based models employ more sophisticated message-passing mechanisms, such as GAT and Graph Transformers, while sequential approaches have also been explored [11-15]. Therefore, I find it difficult to agree with the generalization of the mechanism in Equation (1) to MIL as a whole, particularly when it is used to support the arguments presented in Lines 46–52.
  4. The experimental results presented in Table 1 require a more thorough discussion. While I acknowledge that AUC is an important metric for diagnostic tasks, the ACC of AB-MIL+BU-HIM on CAMELYON16 is lower than that of the vanilla AB-MIL baseline. A decrease in ACC indicates an increase in the overall classification error rate. Notably, AB-MIL also exhibits a decline in diagnostic performance when combined with BU-HIM on PANDA. Furthermore, although RRT-MIL+BU-HIM achieves the highest recall on CAMELYON16, its F1 score is not comparably strong. This raises concerns about potential over-calling by the diagnostic model.

    • These inconsistencies across the reported quantitative results make it difficult to establish the practical justification for BU-HIM and assess whether its benefits outweigh the potential drawbacks.
  5. Experimental Design: Restricting the plug-in targets of BU-HIM to gated-attention-based MIL methods, which were proposed relatively early, rather than evaluating it on more recent MIL architectures weakens the validation of the proposed methodology. This experimental setting appears to demonstrate BU-HIM's effectiveness within a self-imposed scope rather than establish its broader applicability.

    5-1. More importantly, this work primarily concerns instance mining and uncertainty-aware selection in MIL, rather than proposing a new MIL architecture. Accordingly, the methods discussed in Section 2.3 that address the relationship between uncertainty and MIL should have been included as baselines, where applicable. The experimental section should instead establish fixed MIL backbones and conduct a multifaceted analysis of their performance, interpretability, and hard-instance mining effectiveness with and without the proposed method.

    5-2. In this regard, Table 2 provides only a limited evaluation. To my knowledge, several other studies have explored instance mining in MIL [20-22]. However, Random Selection and MCDO are overly naive baselines, and outperforming these methods alone is insufficient to establish the superiority of BU-HIM. Moreover, the reported improvements appear statistically inconclusive. BU-HIM yields only a 0.54 percentage-point performance gain over vanilla AB-MIL, and this improvement remains within the standard deviation of AB-MIL's performance.

  6. RRT-MIL [16] appears to be mentioned for the first time in Line 294, but I could not find a corresponding citation. More generally, citations should be provided for the models referenced when describing the experimental setup.


References

[1] Chen, Richard J., et al. "Scaling vision transformers to gigapixel images via hierarchical self-supervised learning." 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2022.

[2] Shao, Zhuchen, et al. "Transmil: Transformer based correlated multiple instance learning for whole slide image classification." Advances in neural information processing systems 34 (2021): 2136-2147.

[3] Yang, Shu, Yihui Wang, and Hao Chen. "Mambamil: Enhancing long sequence modeling with sequence reordering in computational pathology." International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland, 2024.

[4] Javed, Syed Ashar, et al. "Additive mil: Intrinsically interpretable multiple instance learning for pathology." Advances in Neural Information Processing Systems 35 (2022): 20689-20702.

[5] Zhang, Yunlong, et al. "Attention-challenging multiple instance learning for whole slide image classification." European conference on computer vision. Cham: Springer Nature Switzerland, 2024.

[6] Ilse, Maximilian, Jakub Tomczak, and Max Welling. "Attention-based deep multiple instance learning." International conference on machine learning. Pmlr, 2018.

[7] Chen, Richard J., et al. "Whole slide images are 2d point clouds: Context-aware survival prediction using patch-based graph convolutional networks." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2021.

[8] Thandiackal, Kevin, et al. "Differentiable zooming for multiple instance learning on whole-slide images." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2022.

[9] Henry, Emerald U., Onyeka Emebob, and Conrad Asotie Omonhinmin. "Vision transformers in medical imaging: A review." arXiv preprint arXiv:2211.10043 (2022).

[10] Hong, Sungrae, et al. "Diagnose Like A REAL Pathologist: An Uncertainty-Focused Approach for Trustworthy Multi-Resolution Multiple Instance Learning." 2026 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, 2026.

[11] Hou, Wentai, et al. "H^ 2-MIL: exploring hierarchical representation with heterogeneous multiple instance learning for whole slide image analysis." Proceedings of the AAAI conference on artificial intelligence. Vol. 36. No. 1. 2022.

[12] Chen, Zhen, et al. "Diagnose like a pathologist: Weakly-supervised pathologist-tree network for slide-level immunohistochemical scoring." Proceedings of the AAAI conference on artificial intelligence. Vol. 35. No. 1. 2021.

[13] Bontempo, Gianpaolo, et al. "Das-mil: Distilling across scales for mil classification of histological wsis." International conference on medical image computing and computer-assisted intervention. Cham: Springer Nature Switzerland, 2023.

[14] Shi, Jiangbo, et al. "Vila-mil: Dual-scale vision-language multiple instance learning for whole slide image classification." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[15] Ye, Guo, et al. "Graph-based multi-instance learning: a survey." International Journal of Machine Learning and Cybernetics 17.6 (2026): 274.

[16] Tang, Wenhao, et al. "Feature re-embedding: Towards foundation model-level performance in computational pathology." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

[17] Yu, Lequan, et al. "Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation." International conference on medical image computing and computer-assisted intervention. Cham: Springer International Publishing, 2019.

[18] Korattikara Balan, Anoop, et al. "Bayesian dark knowledge." Advances in neural information processing systems 28 (2015).

[19] Tang, Wenhao, et al. "Multiple instance learning framework with masked hard instance mining for whole slide image classification." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.

[20] Huang, Wentao, et al. "Hard negative sample mining for whole slide image classification." International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2024.

[21] Shao, Zhuchen, et al. "Lnpl-mil: Learning from noisy pseudo labels for promoting multiple instance learning in whole slide image." 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023.

[22] Zheng, Tingting, Kui Jiang, and Hongxun Yao. "Dynamic policy-driven adaptive multi-instance learning for whole slide image classification." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

Questions for authors

Please refer to the "Critical Weaknesses" section.

Reviewer_S4EZOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes BU-HIM, a Bayesian uncertainty-guided hard instance mining framework for weakly supervised whole-slide image classification. The authors introduce a Bayesian variational attention mechanism to estimate instance importance and its associated uncertainty. Instances are selected according to the union of high-importance and high-uncertainty subsets, and a teacher-student training framework jointly optimizes predictions from the selected subset and the complete slide.

The framework is evaluated on CAMELYON16 and PANDA using ABMIL, DSMIL, and RRT-MIL as student backbones. The authors report improvements in AUROC across all six dataset-backbone combinations and provide comparisons with random sampling and MC-Dropout selection. Additional analyses examine prediction sensitivity under instance removal and qualitative attention localization.

Critical strengths
  1. Relevant motivation and evaluation across multiple MIL architectures.

The paper addresses a meaningful limitation of attention-based MIL: the estimated importance of an instance may be uncertain, and considering this uncertainty during training could potentially improve hard-instance selection.

The authors evaluate the proposed framework with three different MIL backbones and report results on both binary metastasis detection and multiclass prostate grading. The improvement obtained with DSMIL on PANDA is particularly encouraging.

The teacher-student design also retains the original student architecture at inference time, which could be useful in practical applications if the training benefits are confirmed.

Critical weaknesses
  1. The theoretical formulation and optimization of the proposed epistemic uncertainty are insufficiently justified.

My primary concern is whether the proposed uncertainty measure actually captures the epistemic uncertainty claimed by the authors.

In Section 3.2, the Bayesian attention network uses a variational posterior over its parameters and predicts a latent Gaussian attention response with mean μ_i and variance v_i. The proposed uncertainty score is then defined using a first-order approximation as:

u_i ≈ [m_i(1−m_i)]²v_i.

However, this expression directly uses the predicted conditional variance of the latent attention response. Under a conventional hierarchical probabilistic interpretation, predictive variance contains both expected conditional variance and variance of the conditional mean across posterior parameter samples. The latter is the component more directly associated with epistemic uncertainty.

The proposed expression does not explicitly quantify this posterior variability. Consequently, the interpretation of u_i as epistemic uncertainty requires substantially stronger mathematical justification.

There is also an important ambiguity in the training procedure. According to Eqs. 5–7, the classification objective supervises only student predictions, while the consistency objective applies stop-gradient to the teacher prediction. The instance selection operation is based on TopK ranking, which is non-differentiable. Furthermore, only structurally compatible teacher parameters are updated through EMA.

It is therefore unclear how teacher-specific Bayesian parameters, especially the predicted variance and variational posterior parameters, receive data-dependent training signals. The KL regularization term alone does not establish task-dependent uncertainty learning.

The paper should explicitly specify the computational graph, the parameter groups updated by gradient descent versus EMA, and the likelihood or task objective used to optimize the Bayesian attention distribution. Without these details, it is difficult to determine whether the uncertainty scores are meaningfully learned.

This is a fundamental concern because uncertainty-guided instance selection constitutes the paper's central methodological contribution.

  1. The experimental evidence does not adequately isolate the benefit of uncertainty-guided mining or demonstrate its practical significance.

The reported improvements are generally modest. On CAMELYON16, the AUROC gains are between 0.32 and 0.70 percentage points across three backbones. On PANDA, the improvements range from 0.03 to 1.04 percentage points. Some combinations show decreases in accuracy or F1-score despite improved AUROC.

The comparison with MC-Dropout is particularly inconclusive. In Table 2, BU-HIM achieves an AUROC of 91.57%, compared with 91.34% for MC-Dropout selection. This difference of 0.23 percentage points is small relative to the reported variability, and no paired statistical evidence is provided.

More importantly, the comparison does not clearly isolate the contribution of Bayesian uncertainty. BU-HIM combines uncertainty estimation, importance-based selection, a teacher-student framework, dual-view supervision, and consistency regularization. The paper lacks a factorial ablation separating these components, particularly importance-only versus uncertainty-only selection under otherwise identical training conditions.

The actual instance-selection mechanism also raises concerns. The authors set the selection ratio to β = 0.7 for both importance-based and uncertainty-based TopK sets and retain their union. Therefore, the resulting subset necessarily contains between 70% and 100% of all instances. If the selections were approximately independent, the expected retention would be around 91%.

This raises the question of whether the method performs sufficiently selective hard-instance mining or mainly benefits from training on highly overlapping full-bag and selected-bag views. The paper does not report actual retention ratios, overlap statistics, or sensitivity to the selection budget.

The empirical validation is further limited to CAMELYON16 and PANDA, both extensively studied benchmarks. All experiments use ResNet-50 patch features, without evaluating whether the proposed mechanism remains beneficial with modern pathology-specific foundation model embeddings or under independent external evaluation.

For PANDA, the absence of quadratic weighted kappa, an established metric for ordinal prostate grading, also limits the clinical interpretation of the results.

Finally, the claimed interpretability benefits are supported mainly by four qualitative attention heatmaps and instance-removal sensitivity experiments. Although the relative differences in removal sensitivity are substantial, the absolute probability and loss changes are very small. These experiments do not directly establish calibrated uncertainty or more accurate tumor localization.

Given that CAMELYON16 provides tumor-region annotations, quantitative evaluation of uncertainty-guided region localization, false-positive regions, and small metastasis detection would provide a much stronger validation of the proposed mechanism.

Overall, the current results demonstrate limited classification improvements but do not convincingly establish that these gains arise from correctly modeled epistemic uncertainty.

Questions for authors
  1. What exactly does the proposed uncertainty score measure?

Could the authors clarify why the predicted latent attention variance v_i is interpreted as epistemic uncertainty? How does the proposed formulation account for uncertainty arising from the posterior distribution over network parameters?

Would the results remain consistent if epistemic uncertainty were explicitly estimated from the variance of attention predictions across posterior parameter samples?

  1. How are the Bayesian teacher parameters optimized?

Please specify which parameters are updated through backpropagation, which are updated through EMA, and how the Bayesian-specific parameters receive task-dependent gradients.

In particular, how is the predicted variance v_i trained, given that the classification objective supervises student predictions, the teacher prediction is detached in the consistency objective, and TopK instance selection is non-differentiable?

An explicit computational graph or implementation-level description would be helpful.

  1. How many instances are actually retained after taking the union of the two TopK sets?

With β = 0.7, each selection criterion already retains 70% of all patches. Please report the average final retention ratio, overlap between importance-selected and uncertainty-selected instances, and sensitivity to β.

How much performance remains when using an equally sized random subset or attention-only subset within the identical dual-view teacher-student training framework?

  1. Can the authors provide controlled ablations isolating the contribution of Bayesian uncertainty?

Please compare importance-only, uncertainty-only, and combined selection while keeping the final instance count, teacher architecture, student backbone, training budget, and consistency objective identical.

It would also be useful to compare the proposed uncertainty measure against posterior-sampling variance, MC-Dropout uncertainty, and simpler measures of attention instability.

  1. Can the authors provide quantitative evidence that uncertainty improves interpretability and generalization?

For CAMELYON16, please consider using the available tumor-region annotations to assess whether uncertainty-guided selection improves localization of small metastases, detection of difficult tumor regions, or identification of model errors.

For the classification experiments, please report paired statistical comparisons, results with modern pathology foundation model embeddings, and performance on independent external datasets.

For PANDA, please additionally report quadratic weighted kappa to characterize ordinal grading performance.

Reviewer_3PXuOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript proposes BU-HIM, a Bayesian uncertainty-guided hard instance mining framework for weakly supervised whole slide image classification. The method combines Bayesian variational attention, importance- and uncertainty-based instance selection, and dual-view teacher-student learning to improve MIL classification. The research motivation is interesting, and the experimental results show modest improvements across several MIL backbones. However, the manuscript has substantial weaknesses in terms of methodological justification, consistency between the central claims and experimental evidence, experimental rigor, and practical applicability. These concerns significantly undermine the validity and significance of the claimed contributions.

Critical strengths
  1. The paper highlights a potentially meaningful limitation of deterministic attention-based MIL: instances with comparable attention scores may exhibit different levels of uncertainty in their importance estimation. Exploring uncertainty as a complementary criterion for instance selection is a reasonable research direction in weakly supervised WSI classification.
  2. BU-HIM can be integrated with different MIL backbones without additional inference overhead. The experiments cover two WSI benchmarks and include comparisons of instance selection strategies and qualitative analyses.
Critical weaknesses
  1. The paper's central motivation is that “instances with comparable attention scores may exhibit substantially different levels of epistemic uncertainty, leading to suboptimal instance selection”. However, the proposed methodology and experiments do not adequately demonstrate how uncertainty-guided mining resolves this challenge. In particular, Section 4.3 compares the decision sensitivity of high-uncertainty and low-uncertainty instances with similar attention scores by measuring prediction changes after instance removal. These results merely indicate that such instances can have different effects on model predictions. They do not establish that high-uncertainty instances are more informative for training or that prioritizing them improves generalization. Moreover, the experiment focuses on high-attention instances rather than those additionally introduced by uncertainty-based selection, which are more directly relevant to the claimed contribution.
  2. The first-order Delta approximation estimates the variance of transformed attention responses but does not establish that this variance represents epistemic uncertainty arising from the Bayesian parameter posterior. Moreover, the optimization of Bayesian-specific parameters within the teacher–student framework remains unclear.
  3. The reported performance gains are modest and inconsistent across metrics, with no statistical significance analysis. Furthermore, the experiments do not sufficiently attribute performance gains to uncertainty-guided selection rather than to teacher–student learning and other training components.
  4. All experiments rely on ResNet-50 patch features, without evaluating modern pathology foundation models such as UNI. Consequently, the effectiveness and generalizability of the proposed method under more representative contemporary feature representations remain uncertain.
  5. The paper provides no training-time or memory-cost analysis. Since importance-based and uncertainty-based selection each retain 70% of instances before taking their union, the final subset may include nearly all patches, raising concerns about the effectiveness of instance mining and the additional computational overhead.
Questions for authors

Please refer to weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.