- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes H2PE, a positional encoding for multi-resolution multiple instance learning on whole slide images. H2PE uses MPP to map patches at multiple magnifications onto a shared hyperbolic upper half-space to capture spatial and cross-scale relationships. Experiments with three MRMIL frameworks on CAMELYON16, BRACS, and BCNB show consistent improvements over existing positional encodings.
Critical strengths
- The paper points out a gap that has not been fully explored. Existing location encoding either models each resolution independently or ignores the scale-dependent geometry of multi-resolution WSI.
- H2PE is model-independent (just for the multi-resolution MIL methods) and provides injection schemes for different types of aggregators. Also, its runtime overhead is small, making it practical to adopt.
- H2PE uses MPP to convert patches from multiple resolution levels into real physical coordinates, so that every level shares a unified space. This lets the resulting hyperbolic distance capture both spatial neighborhood and scale hierarchy in a simple closed form.
Critical weaknesses
-
All experiments use only three levels (~5×/10×/20×). Under this setting, the distortion produced by Euclidean embedding was small, so the theoretical advantage of hyperbolic space growing exponentially was hardly verified.
-
H2PE is evaluated only on ZoomMIL, H2MIL, and DSMIL. No hierarchical sequence-modeling encoders like [1,2,3] have been validated. Since recent MIL methods and slide encoders are increasingly built on such architectures, validating the effectiveness of H2PE on these architectures will be of great value and interesting in demonstrating its capability.
[1] "Scaling vision transformers to gigapixel images via hierarchical self-supervised learning." In CVPR.
[2] "Retmil: Retentive multiple instance learning for histopathological whole slide image classification." In MICCAI.
[3] "Mambamil: Enhancing long sequence modeling with sequence reordering in computational pathology."In MICCAI.
Questions for authors
- Could the authors report results with an additional 0.25 mpp level, and show whether the gap between H2PE and its Euclidean variant grows as the hierarchy gets deeper? This would directly test whether the exponential-growth advantage of hyperbolic space holds in practice.
- Does H2PE benefit fine-grained tasks that depend on nuclear-level morphology at high magnification, such as biomarker prediction (e.g., ER/PR/HER2 prediction in BCNB)?
- How would H2PE be integrated into hierarchical sequence-modeling encoders, which already have built-in positional mechanisms? Should H2PE replace or complement them?
4 comments
cbXU#W2: H2PE on Hierarchical Sequence Models
cbXU#W2: H2PE on Hierarchical Sequence Models
We appreciate your valuable suggestion, which helped us further improve our work. Following your suggestion, we conducted additional experiments using HIPT and have included the results in Table 1 of the revised manuscript. Since this question is also related to your Question #3, we kindly invite you to refer to our response to that question for further details.
We sincerely appreciate your valuable comments, which provided useful insights for improving our manuscript and designing additional experiments.
cbXU#Q2: Nuclear-level Morphology
cbXU#Q2: Nuclear-level Morphology
We sincerely appreciate your suggestion and are pleased to share an interesting finding from our additional experiments.
- We conducted HER2 prediction on the BCNB dataset using the (1, 0.5) MPP setting and HIPT MIL. As shown in the table below, all PE methods perform within the variation range of the performance without PE. This suggests that positional information may offer limited benefits for this particular task. One possible interpretation is that, for labels such as HER2 status, which are determined by the overall morphology of tumor cells, the spatial arrangement of patches may provide little additional information.
| PE | ACC | AUC |
|---|---|---|
| none | 72.6$\pm$2.0 | 72.4$\pm$3.1 |
| PPEG | 72.2$\pm$1.8 | 72.2$\pm$2.6 |
| RoPE | 73.6$\pm$2.6 | 71.2$\pm$3.2 |
| TNAM | 72.3$\pm$2.0 | 71.1$\pm$2.2 |
| Sin.PE | 72.6$\pm$3.1 | 71.4$\pm$4.0 |
| HyPE | 72.5$\pm$3.6 | 70.5$\pm$4.3 |
| TreePE | 72.4$\pm$1.9 | 70.8$\pm$3.4 |
| GPSE | 72.7$\pm$2.3 | 71.3$\pm$3.6 |
| H2PE | 73.1$\pm$1.5 | 72.1$\pm$3.1 |
We believe this finding may provide a useful starting point for future MIL research on more advanced biomarker prediction tasks beyond conventional diagnosis. We also discuss this direction in the Future Work section. We sincerely thank you for this insightful suggestion.
cbXU#Q3: Intergrating into Hierarchical Sequence-modeling Encoders
cbXU#Q3: Intergrating into Hierarchical Sequence-modeling Encoders
We appreciate your insightful question, which has helped us further develop our work. Our primary motivation is to provide positional information for MRMIL. As described in Section 3.2.4, H2PE is designed to be flexibly integrated into the general MRMIL architecture (encoder).
-
Accordingly, when positional information is provided by a replaceable module within an architecture, H2PE can replace that module. HIPT, which we evaluated following your suggestion, is an example of this case. HIPT incorporates absolute positional encoding (APE), which we replaced with H2PE and other PE methods for comparison. In contrast, RetMIL and MambaMIL rely on sequence order itself for positional information rather than a separate PE module. In these architectures, H2PE can serve as a complementary component rather than a direct replacement. Specifically, the node term can be added to the input features, while the pairwise term can be incorporated through Equation 6.
-
We sincerely thank you for bringing up this insightful point, which we had not fully considered during our initial investigation. We believe this question provides an opportunity to enrich the manuscript with a broader discussion of H2PE's applicability. Accordingly, we have added a relevant discussion to the Appendix.
cbXU#W1Q1: Evaluation with an Additional Resolution Level
cbXU#W1Q1: Evaluation with an Additional Resolution Level
Following your valuable feedback, we conducted additional experiments incorporating 40x magnification (0.25 MPP).
We would like to clarify the following regarding these results. At 40x magnification, the number of patches is 64 times greater than at 5x, resulting in approximately 64-fold increases in preprocessing time and storage requirements. We initially attempted to train and evaluate MR-MIL models using all 40x patches but encountered GPU out-of-memory (OOM) errors due to the excessive memory demands. We therefore imposed an upper limit on the number of instances in the image pyramid for both training and inference. Consequently, the results may differ slightly from those in the main experiments. However all methods were evaluated under identical conditions to ensure fair comparisons.
At shallow hierarchies (i.e., (2, 1) MPP), performance differences among PE strategies are relatively small. As the hierarchy deepens, all distance exhibit performance improvements. Notably, normalized Euclidean distance, as suggested in Reviewer PaCM's Weakness 2, provides a slight performance advantage over naive Euclidean distance. While deeper hierarchies generally benefit all PE strategies, H2PE achieves the highest performance, demonstrating its effectiveness in deeper multi-resolution hierarchies.
At seven levels, H2PE achieves a distortion ratio of 1.3, indicating that its distance-to-hop ratios vary by only a factor of 1.3 between the 1st and 99th percentiles, closely preserving proportionality between distance and hop count. In contrast, normalized Euclidean distance yields 11.3, indicating substantially greater distortion. As your feedback, these results highlight H2PE's increasing advantage as hierarchy depth grows. We present the corresponding visualization in Fig. 6(b) of the revised manuscript.