acceptodds
Withdrawn from ICLR 2027

Why Patch Selection Rarely Saves Compute, and How to Fix It

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

GigaPixel Whole slide image (WSI) analysis is computationally expensive because pathology foundation models have to process large numbers of image patches extracted from WSI. Recent advances tend to select only a candidate patches for slide-level aggregation and representation. However, the selection process is usually carried on high-magnification resolution which requires heavy computation for the task processing. To avoid such computation burden, the process of patch selection/embedding can be utilized at low-magnification level of WSI; but coming with the price of losing pixel information. We address this problem with a three-stage coarse-to-fine cascaded process to progressively select patch regions from low- to high- magnification levels in WSI; imitating a very similar process of pathologists during microscopy diagnosis. A frozen thumbnail encoder first screens all candidate regions (i.e. global tissue view), followed by a frozen pathology foundation model that re-ranks the survivors using one coarse view per region. Only the final selected regions are then encoded at higher resolution. All encoders are frozen, and two lightweight linear attention scorers are trained with attention distillation to transfer the coarse ranking to the thumbnail stage. Across three classification and five survival datasets, we demonstrate the our cascade processing significantly reduces the inference GPU time and resource by –, while preserving the task performance evaluation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Computation Pathology, Multi instance learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_9ua4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper shows that patch selection in WSI analysis often fails to reduce compute when selection occurs after high-resolution feature extraction. To address this, it proposes a coarse-to-fine cascade: a frozen thumbnail encoder first screens candidate regions, a frozen pathology foundation model reranks survivors at coarse resolution, and only the final selected regions are encoded at fine resolution for MIL prediction. The method aims to reduce fine-resolution encoder evaluations while preserving slide-level classification and survival performance across CAMELYON16 and TCGA datasets.

Critical strengths
  1. The proposed cascade is simple. It uses cheap thumbnail screening before coarse pathology reranking and reserves expensive fine-resolution encoding only for selected regions.
Critical weaknesses
  1. The novelty is somewhat limited. The main observation that patch-selection cost depends on where selection is performed is intuitive, and coarse-to-fine WSI inference has been explored in prior work such as ZoomMIL, HDMIL, WISE-FUSE, and multiscale DSMIL. The main contribution is clearer compute accounting plus an engineering combination of thumbnail screening, coarse reranking, and distillation. The title’s claim that patch selection “rarely saves compute” also seems overly broad, since not all patch-selection methods perform selection after full 20× encoding.

  2. The claim of preserved performance is not uniformly supported. The CAMELYON16 task shows a clearer gap: the cascade is below full-bag encoding and also below some published selection baselines. For survival prediction, the standard errors are large, and cascade variants are often slightly below their full-bag counterparts.

  3. The paper should better justify the practical importance of compute reduction relative to accuracy. In pathology, users may prefer a slower but more accurate full-bag model, especially for high-stakes diagnosis.

  4. Since the method already encodes all candidate regions at 5× during training, the paper should report how well an MIL model performs using all 5× features directly, without any 20× encoding. If this baseline is competitive, the necessity of the full three-stage cascade would be less clear. The comparison with published methods also needs more detail about whether the same encoder, tiling, preprocessing, and compute accounting are used.

  5. Several Table 3 results are counterintuitive and need deeper explanation. The tumor oracle performs only slightly better than random selection on CAMELYON16. Similarly, 20× fine-resolution selection performs worse than the cascade despite using more detailed features. These results may indicate issues with the oracle setup, negative-slide handling, optimization, or variance, but the current explanation is not fully convincing.

  6. The method depends on manually chosen budgets such as K=500 and b=10. The sensitivity analysis shows that too small a K can substantially hurt CAMELYON16 performance, so the compute–accuracy tradeoff appears task-dependent and may require tuning.

  7. The evaluation is limited to public CAMELYON16 and TCGA datasets. Since the method discards most tissue before prediction, stronger cross-institution validation would be important.

  8. The writing and presentation need improvement. The paper has several grammar issues and dense explanations that make the method and experimental protocol harder to follow. Clearer writing, more precise terminology, and better separation between motivation, method, and ablation analysis would improve readability.

Questions for authors
  1. Can the authors clarify the novelty relative to prior coarse-to-fine WSI methods? The title’s claim that patch selection “rarely saves compute” also seems broad, since some prior methods already perform coarse-to-fine selection.

  2. Can the authors justify the practical importance of compute reduction in pathology settings where users may prefer a slower but more accurate model, especially for high-stakes diagnosis?

  3. Can the authors add a low-cost baseline using all 5× features directly, without any 20× encoding? Since the method already encodes all candidate regions at 5× during training, this baseline would clarify whether the full three-stage cascade is necessary.

  4. Can the authors explain the counterintuitive Table 3 results, especially why the tumor oracle is only slightly better than random selection and why 20× fine-resolution selection performs worse than the cascade?

  5. Can the authors provide more sensitivity analysis for K and b?

  6. Can the authors discuss robustness beyond CAMELYON16 and TCGA, preferably with cross-institution validation?

Reviewer_wnPnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a three-stage cascade for efficient whole-slide image analysis: thumbnail screening, coarse-resolution reranking, and high-resolution encoding of only selected regions. Its main contribution is distinguishing the final selection budget from the actual encoding cost, using frozen encoders and attention distillation to discard regions before expensive processing. Evaluation spans three classification and five survival datasets, with multiple MIL aggregators and selection baselines, reporting approximately 14–28× lower encoder GPU time with task-dependent performance losses.

Critical strengths
  1. This paper targets the computational bottleneck directly. The paper distinguishes reducing the final MIL bag from avoiding expensive feature extraction, and its cascade rejects regions before high-resolution encoding. The reported 14–28× reduction in encoder GPU time makes this a potentially consequential efficiency contribution, even though end-to-end speedups remain unverified in the supplied version.

2.The controlled comparisons of this paper strengthen the empirical argument. Comparing thumbnail, coarse, fine-resolution, random, and oracle selection at matched region budgets helps separate selection quality from subset size. Combined with evaluation across classification and survival tasks and multiple MIL aggregators, this provides meaningful evidence that the approach has utility beyond a single model or benchmark.

Critical weaknesses
  1. Matched region budgets do not guarantee matched computational budgets. Selected regions contain different numbers of high-resolution patches. Consequently, fixing b=10b=10 does not fully isolate ranking quality from the quantity of tissue processed; accuracy–compute curves would provide stronger evidence.

2.The headline claim is broader than the evidence. “Patch selection rarely saves compute” suggests a general finding, whereas the experiments establish a narrower conclusion about selectors requiring exhaustive high-resolution encoding. The paper would need broader evidence or more qualified framing.

Questions for authors

1.How is the tumor oracle implemented, particularly for negative slides and tied scores? Is the downstream predictor retrained for each selection strategy, and what explains the oracle’s near-random performance? 2.Does downstream MIL receive region-aggregated embeddings or individual high-resolution patch embeddings? At fixed bb, how much do actual encoded patch counts differ across selection methods?

Reviewer_PCh6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates why patch selection does not necessarily reduce computational cost in whole-slide image analysis. It addresses this problem with a three-stage coarse-to-fine cascaded process to progressively select patch regions from low- to high- magnification levels in WSI. Experiments on classification and survival prediction demonstrate substantial reduction of computational cost.

Critical strengths

The paper addresses a practically relevant problem. The proposed cascade provides a reasonable approach to reducing expensive feature extraction. Evaluation across classification and survival tasks, together with different downstream aggregation methods, broadens the experimental assessment. The reported computing time savings suggest effectiveness and look promising.

Critical weaknesses
  1. One concern is the decline in predictive performance. On CAMELYON16, ABMIL achieves an AUROC of 0.9844, compared with 0.9491 for the cascade. On TCGA-STAD, the C-index decreases from 0.654 to 0.604, and the TransMIL C-index decreases from 0.625 to 0.578. These are notable decreases that challenge the general claim of performance preservation. The authors should provide paired comparisons and performance vs. cost curves and soften this claim.

  2. Table 3 fixes the number of selected regions at b=10, but Section 3 states that each region contains a tissue-dependent number of fine-resolution patches. Different selectors may therefore retain different number of tissue patches. The authors should report the actual patch counts for each selector.

  3. Some results require further explanation. In Table 3, the tumor oracle achieves an AUROC of 0.6423, only slightly above random selection at 0.629. The authors should clarify the selection procedure and investigate why this scenario provides such a limited improvement.

  4. Although the computational savings are promising, the technical novelty appears limited. The selection components largely build on existing approaches, with the main contribution appearing to be thumbnail screening within the proposed cascade. The authors should more clearly distinguish their contributions from prior work and isolate their benefits through comparisons with simpler cascades.

  5. The practical implications of the performance cost trade-off need further discussion. Computational savings come with reduced predictive performance in some settings. The authors should discuss when this trade-off is acceptable and which tasks or tissue characteristics make the selection strategy more suitable, helping practitioners decide when to adopt it.

Questions for authors

Please refer to the questions raised in the Weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.