acceptodds
Under review as a conference paper at ICLR 2027

WEAVE: Adaptive Patchification for Gigapixel Whole-Slide Images

Abstract

Transformer-based architectures for computer vision, such as Vision Transformers, require patchification, which is a process in which the input image is split into patches and mapped to tokens. In contrast to uniform patchification, adaptive patchification allocates more tokens to information-rich image regions. While adaptive patchification is effective for natural images, extending this approach to high-resolution images, such as gigapixel whole-slide images (WSIs) in histopathology, introduces new challenges. Uniform patchification can produce prohibitively long token sequences, making slide-level aggregation expensive, particularly for quadratic-cost architectures such as Transformers. Adaptive patchification can reduce this cost by allocating tokens according to image content, but highly redundant and long-tailed WSI feature statistics destabilize existing scoring mechanisms. We propose SI -based daptive ision ncoder (), an adaptive patchification method for gigapixel WSIs. We introduce a Hopfield energy-based scoring function that provides stable importance estimates under WSI feature statistics and a lightweight mLSTM encoder for subquadratic token aggregation. Together, these components enable content-adaptive tokenization at gigapixel scale and achieve competitive or improved performance across five WSI classification benchmarks and much higher efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Adaptive Tokenization, Whole-slide images, Subquadratic architectures, Multiple instance learning, Gigapixel image processing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Qjj8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a new method called WEAVE for extracting patches with arbitrary sizes from gigapixel whole slide images (WSIs). The core idea is to model the patch extraction as an adaptive process where any given WSI is decomposed into variable-sized patches based on their informativeness with less informative WSI regions are represented with larger patches and more informative ones with more. This idea has been adapted for in the literature for natural images for adaptive tokenization. In the paper, the authors propose a Hopfield-energy based measure for assessing informativeness of pathches. The tokens from these variable size patches are then processed by an mLSTM module, introducing further computational efficiency as compared to pairwise self-attention. The experiments demonstrate that the suggested WEAVE method outperforms the evaluated MIL-based baselines on five different classification oriented datasets.

Critical strengths

S1. The paper tackles with an important issue in processing WSIs: As WSIs are gigapixel images, efficient techniques are required. For this purpose, the authors propose an adaptive token merging technique, which is built upon extracting variable-size patches by using a Hopfield-energy based informativeness measure and an mLTSM token aggregation module.

S2. Employing the Hopfield energy as a means for measuring spatial importance of image patches for obtaining arbitrary size patches is an interesting idea, which seems to work for WSIs containing highly sparse informative regions.

S3. Experiments on five different classification-oriented WSI datasets demonstrates the effectiveness of the suggested framework against the evaluated MIL-based baselines in terms of classification performance.

Critical weaknesses

W1. The technical novelty is relatively weak. The proposed adaptive patch extraction method is just DART with its importance mechanism being replaced with a Hopfield-energy based importance scoring.

W2. Experiments were carried out only on classification tasks which demonstrates the effectiveness of the proposed method. However, no experiments are carried out on regression or segmentation-oriented tasks, which are also important for WSI analysis.

W3. The suggested method has close connections with token merging or token compression methods such as ToMe, DivPrune, PruMerge. However, the proposed method is only compared against MIL-based baselines.

W4. Some state-of-the-art models are missing in the evaluations:

  • Chen et al., Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning, arXiv 2026
  • Rahman et al., DTC-WSI: Dynamic Token Compression for Whole-Slide Images, MIDL 2026

W5. Number of learnable prototypes, L, is an hyperparameter of the proposed method, but no analysis is given regarding the effect of L on the model performance.

Questions for authors

Q1. Could the suggested method can be easily extended to multiscale analysis of WSIs as it has been demonstrated in the literature that multiscale analysis could boost the model performances?

Q2. How sensitive the model performance is with respect to the number of learnable prototypes?

Q3. How does the proposed method’s performance compare against those of the missing token merging/pruning methods?

Q4. What is the reason behind poorer performance on the BRACS dataset as compared to other datasets?

Q5. How computationally efficient the suggested framework is as compared to the evaluated MIL baselines?

4 comments

We thank the reviewer for the feedback and the helpful pointers to related work, which we now discuss in the revised manuscript. Below, we answer each point and summarize the relevant new results. All changes are marked in the revised PDF.

W1: Novelty

"The technical novelty is relatively weak. The proposed adaptive patch extraction method is just DART with its importance mechanism being replaced with a Hopfield-energy based importance scoring."

WEAVE keeps only one of DART's three components, the differentiable quantile partitioning, and replaces the other two and adds a linear-cost aggregator:

  • Scoring. The original DART scorer, designed for natural images, does not train stably on WSI features. Replacing it is what makes adaptive patchification trainable on WSIs in the first place. On BRACS, our annotation analysis further shows that the learned scores capture diagnostically relevant differences within tissue (General Response §2).
  • Tokenization. DART resizes each region from raw pixels and re-encodes it, which is infeasible at gigapixel scale. WEAVE instead constructs each token by overlap-weighted aggregation over a precomputed feature map.
  • Aggregation. WEAVE pairs the partition with a linear-cost mLSTM aggregator, which keeps the resulting long token sequences tractable.

To our knowledge, WEAVE is the first method to apply differentiable adaptive patchification to gigapixel WSIs.

W2: Regression and segmentation

"Experiments were carried out only on classification tasks […]. However, no experiments are carried out on regression or segmentation-oriented tasks, which are also important for WSI analysis."

We agree and state this in the Limitations. Slide-level regression, such as survival prediction, would in principle only require a different task head and loss. Dense segmentation fits less naturally, because WEAVE deliberately produces coarse tokens in low-importance regions. We consider both directions for future work.

W3 / Q3: Relation to token merging and pruning

"The suggested method has close connections with token merging or token compression methods such as ToMe, DivPrune, PruMerge. However, the proposed method is only compared against MIL-based baselines." "How does the proposed method's performance compare against those of the missing token merging/pruning methods?"

WEAVE shares the goal of these methods, retaining relevant information with fewer tokens, but uses another mechanism. It partitions a feature map into spatially contiguous regions that cover the full WSI. Unlike in pruning, no location is discarded and unlike in merging the resulting regions are spatially contiguous. A direct quantitative comparison was not feasible within the rebuttal period, but we clarified this distinction (Sec. 1 Introduction and Sec. 2 Related work), list DivPrune and PruMerge and state the missing comparison in the Limitations.

W4: Missing methods

"Some state-of-the-art models are missing in the evaluations"

Thank you for these pointers, which are both now discussed in the related work. Chen et al. (arXiv, June 2026) addresses a different task: they select a small set of tokens for vision-language slide reasoning and question answering, rather than WSI classification. DTC-WSI (Rahman et al., 2026) is closer to our setting, but its evaluation protocol differs from ours, as they use CONCH features and treat PANDA as a binary rather than 6-class ISUP grading task. Their reported numbers are therefore not comparable to ours. Since no public implementation is available, reproducing it under our protocol was not feasible within the rebuttal period.

W5 / Q2: Number of prototypes L

"Number of learnable prototypes, L, is an hyperparameter of the proposed method, but no analysis is given regarding the effect of L on the model performance." "How sensitive the model performance is with respect to the number of learnable prototypes?"

We evaluated L ∈ {1, 5, 10, 20, 50, 100, 1000} on DHMC (Appendix C.4 Prototype Sensitivity). To keep the sweep affordable, it uses fewer training epochs than the main experiments, so absolute values are lower than in Table 2 and should be compared only within the sweep. Values are mean ± standard deviation over three runs.

LAccF1AUC
10.781 ± 0.0250.744 ± 0.0120.964 ± 0.011
50.790 ± 0.0740.778 ± 0.0780.960 ± 0.024
10 (default)0.803 ± 0.0430.797 ± 0.0680.970 ± 0.010
200.826 ± 0.0370.811 ± 0.0190.973 ± 0.010
500.841 ± 0.0370.797 ± 0.0480.968 ± 0.009
1000.864 ± 0.0040.833 ± 0.0100.972 ± 0.004
10000.850 ± 0.0390.801 ± 0.0300.960 ± 0.010
WEAVE is robust to L: AUC varies little across all values (0.960–0.973). Under this training budget, accuracy tends to increase with more prototypes up to L = 100, although most differences lie within one standard deviation. Even a single prototype trains stably and reaches comparable AUC. The default L = 10 was not extensively tuned and is likely not optimal. We kept it fixed for consistency across all experiments.

Q1: Multiscale extension

"Could the suggested method can be easily extended to multiscale analysis of WSIs [...]?"

Yes, we see two natural directions for WEAVE:

  • Multi-scale: apply adaptive patchification independently at each magnification, with one scorer and one token budget per level, and aggregate the resulting tokens jointly, similar to multi-scale MIL approaches such as DSMIL (Li et al., 2021).
  • Hierarchical zooming: apply adaptive patchification to the full slide at the lowest magnification, and at each higher magnification only within the regions retained at the previous level, similar to ZoomMIL (Thandiackal et al., 2022) or PathCTM (Ge et al., 2026). We have not evaluated either direction, but added them to the future-work discussion.

Q4: BRACS

"What is the reason behind poorer performance on the BRACS dataset as compared to other datasets?"

We do not have a conclusive explanation, but our new results narrow it down:

  • Encoder. With UNI features, the encoder used by the baselines, WEAVE does not reduce the gap on BRACS (Appendix C, Encoder Ablation), so the encoder is unlikely to be the main cause.
  • Magnification. Separating atypical from benign and malignant lesions may depend more strongly on magnification than the tasks of the other datasets. Both our RetCCL and UNI variants extract features at 10×, while the baselines use 20×.
  • Data regime. BRACS combines one of the smallest training sets (395 slides) with the longest token sequences (13.6k tokens per slide on average).

Q5: Efficiency

"How computationally efficient the suggested framework is as compared to the evaluated MIL baselines?"

We compared WEAVE with 2DMambaMIL, the strongest baseline, on the same features (General Response §4). WEAVE needs 2.5–4.1× fewer inference FLOPs and up to 2.8× less training memory, with 12–31% faster training steps; inference is slower on DHMC due to our unoptimized PyTorch mLSTM. Dense feature extraction dominated the cost of our original pipeline, but it is not necessarily required: on PANDA, reducing the extraction density by about 25× yields comparable performance (Appendix C, Feature Extraction Density).


We hope our response clarifies the novelty of WEAVE and that the new results answer the reviewer's questions. If so, we would be grateful if they would consider updating their assessment. We are happy to answer any further questions.

Thank you for the response.

I still think the proposed method has limited novelty as the replacements mentioned by the authors appear to lack complexity of a certain extent. If a more comprehensive evaluation had been carried out with the experiments including the state-of-the-art models and the experiments being carried out on regression or segmentation-oriented tasks, assessing the contribution of the proposed method could be more convincing.

Based on these, I am keeping my initial rating.

We thank the reviewer for the reply but respectfully disagree with the remaining concerns. In contrast to the reviewer's claims, we would like to point out the following.

WEAVE provides substantial novelty over DART. DART was developed and evaluated on natural images and videos at standard resolution (e.g., 224×224 on ImageNet-1K). Extending it to gigapixel whole-slide images is not a straightforward transfer: in our experiments, DART's scorer did not train stably on WSI features, and its pixel-level re-encoding of each region is infeasible at gigapixel scale. Our energy-based scoring and feature-space tokenization address both. As the other reviewers pointed out, this direction is "under-explored" (pEsr), and our approach offers "a plausible solution" and is "practically relevant" (e8nV).

Our experiments include state-of-the-art methods for WSI classification. We compare against the reported results of nine MIL methods spanning attention-based, Transformer-based and state-space aggregators, the full baseline suite of Zhang et al. (2025), on five benchmarks using their released splits. Chen et al. (2026) addresses vision-language question answering rather than slide classification, and DTC-WSI uses a different evaluation protocol without a public implementation, so neither is directly comparable.

We present a comprehensive and controlled evaluation. Beyond the main benchmark, we evaluate two feature encoders, controlled ablations of both components, the importance scores against pathology annotations, the number of prototypes, the computational cost, and the feature-extraction density. Slide-level classification is the most widely used benchmark task for WSI aggregation methods, and all our baselines are evaluated on it. We state the absence of regression and segmentation tasks in the Limitations.

We appreciate the reviewer's engagement and remain happy to answer further questions.

Reviewer_e8nVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes WEAVE, a framework for adaptive patchification and efficient aggregation of gigapixel whole-slide images. The method first extracts dense spatial features using a frozen RetCCL encoder and employs a prototype-based Hopfield energy function to estimate spatial importance. It then applies differentiable quantile partitioning, adapted from DART, to construct nonuniform spatial regions. Each region is converted into a token through overlap-weighted feature aggregation, followed by an mLSTM-based slide encoder for classification.

The approach is evaluated on five WSI classification benchmarks, demonstrating competitive performance against existing MIL methods. Controlled experiments on two datasets suggest that energy-based adaptive partitioning improves over uniform partitioning with mean-max pooling, while mLSTM aggregation provides additional benefits. Computational scaling experiments demonstrate lower memory requirements and better scaling than conventional self-attention.

Critical strengths
  1. The paper addresses a meaningful efficiency problem in gigapixel pathology by investigating how a limited token budget should be distributed spatially. The Hopfield energy-based scoring mechanism offers a plausible solution to the numerical instability encountered when applying DART to WSI feature distributions. The integration of adaptive partitioning and feature-space token construction is technically coherent and practically relevant.

  2. The evaluation covers five WSI classification benchmarks and includes useful component-level experiments. The results demonstrate improvements from energy-based adaptive partitioning and mLSTM aggregation under controlled settings. The computational scaling experiments also provide evidence that the proposed aggregation architecture is more memory-efficient than full self-attention for long token sequences.

Critical weaknesses
  1. The independent contribution of adaptive patchification is not sufficiently established. Although Table 2 evaluates uniform and adaptive partitioning, the critical Uniform + mLSTM baseline is missing. Therefore, it remains unclear whether the improvements originate from adaptive token allocation or the stronger aggregation architecture. A controlled comparison using identical features, token budgets, and mLSTM settings is necessary.

  2. The learned importance scores lack quantitative validation of their diagnostic relevance. Figure 5 only provides qualitative visualizations showing that high-scoring regions receive finer patches, without demonstrating that these regions contain diagnostically meaningful information. Since the Hopfield prototypes are initialized using tissue-background contrast, the importance scores may primarily reflect tissue presence rather than diagnostic relevance. Comparisons with random and tissue-guided adaptive partitioning, or evaluation against available pathology annotations, would strengthen this claim.

  3. The generalizability of the proposed method across pathology foundation models remains unclear. WEAVE exclusively uses the older RetCCL encoder, while several reported baselines rely on UNI features and different image resolutions. Although the lightweight encoder is motivated by computational efficiency, these differences prevent strictly controlled comparisons. Additional experiments using a stronger encoder, such as UNI or CONCH, would help establish whether the proposed adaptive patchification strategy generalizes beyond RetCCL.

  4. The efficiency evaluation does not adequately reflect the complete WSI processing pipeline. WEAVE requires dense, fixed-resolution RetCCL feature extraction before adaptive partitioning, meaning that the initial patch encoding cost is not reduced. The reported computational analysis primarily demonstrates the efficiency of mLSTM aggregation compared with self-attention. End-to-end runtime, FLOPs, and memory measurements are needed to substantiate the overall efficiency claims.

Questions for authors
  1. Have the authors evaluated Uniform Patchification + mLSTM under the same RetCCL features, token budget, and training settings as WEAVE? This comparison is essential to isolate the performance gain attributable specifically to adaptive patchification.

  2. What evidence demonstrates that the Hopfield importance scores identify diagnostically informative regions rather than simply separating tissue from background? Are quantitative comparisons with pathology annotations, random adaptive partitioning, or tissue-guided partitioning available?

  3. Have the authors evaluated WEAVE using more recent pathology foundation models, such as UNI or CONCH? If not, how can the authors establish that the observed gains are not specific to the feature distributions produced by RetCCL?

  4. Can the authors report end-to-end WSI processing time, FLOPs, and peak memory usage, including dense feature extraction, Hopfield scoring, adaptive partitioning, token construction, and mLSTM aggregation? How does this compare with conventional WSI pipelines using the same feature encoder and a matched token budget?

1 comment

We thank the reviewer for the constructive review. Indeed, we think that this is a critical problem in pathology and for AI on high-res biomedical images in general. All four concerns are shared with other reviewers and are addressed with new results in the General Response. We summarize the key findings here. All changes are marked in the revised PDF.

W1 / Q1: Uniform + mLSTM

“The independent contribution of adaptive patchification is not sufficiently established. [...] the critical Uniform + mLSTM baseline is missing.”
“Have the authors evaluated Uniform Patchification + mLSTM under the same RetCCL features, token budget, and training settings as WEAVE?”

We added Uniform + mLSTM to Table 2 under the same protocol (General Response §1). Under mLSTM aggregation, adaptive patchification still clearly improves over uniform patchification on DHMC (Acc 0.716 → 0.875), while the gain on TCGA-NSCLC is small (0.885 → 0.897, equal AUC). Both components therefore contribute, but part of the benefit of adaptive patchification can be absorbed by the stronger aggregator, depending on the dataset. We state this explicitly in the revised ablation paragraph.

W2 / Q2: Diagnostic relevance of the importance scores

“The learned importance scores lack quantitative validation of their diagnostic relevance.” “What evidence demonstrates that the Hopfield importance scores identify diagnostically informative regions rather than simply separating tissue from background? Are quantitative comparisons with pathology annotations, random adaptive partitioning, or tissue-guided partitioning available?”

Following the reviewer’s suggestion, we evaluated initial and learned scores against the pixel-level annotations of PANDA and BRACS, both over the full slide and within tissue only (General Response §2, new Table 3). The answer depends on the dataset. On BRACS, training improves the within-tissue separation of annotated regions (AUC 0.631 → 0.721), so the scores capture diagnostically relevant differences beyond tissue presence. On PANDA, the scores separate tissue from background almost perfectly but differentiate only weakly within tissue (0.576 → 0.596), so the allocation mainly reflects tissue presence. Random and tissue-guided partitioning were not feasible within the rebuttal period. We state this and the dataset dependence in the Limitations.

W3 / Q3: Generalization beyond RetCCL

“Additional experiments using a stronger encoder, such as UNI or CONCH, would help establish whether the proposed adaptive patchification strategy generalizes beyond RetCCL.” “Have the authors evaluated WEAVE using more recent pathology foundation models, such as UNI or CONCH?”

We re-ran WEAVE with UNI features on all five benchmarks, applied to exactly the same patches as RetCCL and with the RetCCL hyperparameters unchanged (General Response §3). The Hopfield scorer trains stably on UNI features without modification, and performance stays close to the RetCCL variant, with slightly higher AUC on all datasets except PANDA. WEAVE is therefore not specific to RetCCL feature statistics.

W4 / Q4: End-to-end efficiency

“WEAVE requires dense, fixed-resolution RetCCL feature extraction before adaptive partitioning, meaning that the initial patch encoding cost is not reduced. […] End-to-end runtime, FLOPs, and memory measurements are needed to substantiate the overall efficiency claims.” “Can the authors report end-to-end WSI processing time, FLOPs, and peak memory usage, including dense feature extraction, Hopfield scoring, adaptive partitioning, token construction, and mLSTM aggregation? How does this compare with conventional WSI pipelines using the same feature encoder and a matched token budget?”

We agree that our original analysis covered only the aggregation stage. We now compare WEAVE with 2DMambaMIL on the same RetCCL features (General Response §4). WEAVE needs 2.5–4.1× fewer inference FLOPs and up to 2.8× less training memory, with 12–31% faster training steps. Inference is slower on DHMC due to our unoptimized PyTorch mLSTM.

We also agree that dense extraction was the dominant cost of our original pipeline. On PANDA, reducing the extraction density by about 25× yields comparable performance (Appendix C, Feature Extraction Density), so this cost can be reduced substantially. We did not break the slide-level cost down further per stage within the rebuttal period, and we revised the abstract’s efficiency claim to “at lower computational and memory cost.”


We believe the new experiments address all four of the reviewer's concerns. If the reviewer agrees, we would kindly ask them to consider raising their score. We are happy to answer any further questions.

Reviewer_pEsrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WEAVE, an adaptive patchification pipeline for gigapixel WSI classification. A frozen RetCCL encoder first precomputes a dense feature map. A prototype-based modern Hopfield energy score then replaces DART’s scorer, which the authors report produced NaN gradients on WSI features. DART’s differentiable quantile partitioning uses these scores to form variable-size regions, and each region becomes one token by overlap-weighted feature averaging. A lightweight mLSTM encoder aggregates the tokens at subquadratic cost.

On five benchmarks, WEAVE is competitive with nine MIL baselines whose results are taken from 2DMamba, which used UNI features. Its clearest gain is on PANDA, while AUC is mixed. Ablations on two datasets support both adaptive partitioning and mLSTM aggregation, and a scaling study shows mLSTM stays feasible where self-attention runs out of memory.

Critical strengths
  1. A practical problem, a clean pipeline, and promising results. Applying differentiable adaptive partitioning to gigapixel WSIs is a reasonable and under-explored direction, and the modular design (frozen features → prototype scoring → overlap-weighted pooling → linear-cost aggregator) is easy to follow. Results are competitive using a smaller encoder. There are reported gains on PANDA (Acc 0.5075 → 0.5477, F1 0.4562 → 0.4821) and in macro-F1 on DHMC (0.8027 → 0.8653). The partially controlled ablation (Table 2) also shows that adaptive partitioning outperforms uniform partitioning under the same features, token budget and Mean-Max aggregator.

  2. Candid scoping and a useful feasibility result. The paper openly acknowledges its uncontrolled baselines, potential positional shortcuts, and the BRACS gap. The mLSTM study (Fig. 4, Table 7) shows that accuracy is comparable to self-attention up to ~6k tokens and that mLSTM remains feasible at ~22k tokens.

Critical weaknesses
  1. The evidence that adaptive partitioning adds value in the final model is incomplete. Table 1 is not a controlled comparison: encoder, magnification and sampling density all differ from the baseline results taken from prior work, as the authors acknowledge. A smaller encoder does not by itself make the overall input configuration less favorable. Table 2 is controlled, but it covers only two datasets at ~700/850 tokens (the main experiments use 2.8k–13.6k), with three runs and high variance for the uniform baseline on DHMC. Most importantly, the design lacks Uniform + mLSTM. The current results show that adaptivity helps under Mean-Max and that mLSTM helps on adaptive inputs. They do not show that adaptivity still helps when the final aggregator is used, since a stronger aggregator may already suppress irrelevant tokens. In addition, the prototypes are initialized along a tissue–background direction (App. A.3), so it remains untested whether the gain comes from learned reallocation within tissue or from tissue-aware allocation alone. Fig. 5 does show within-tissue variation, but its added value is not evaluated.

  2. The stability and efficiency claims need stronger validation, and the method’s positioning needs clarification.

    Stability: the DART failure is described qualitatively (Table 2 footnote), without failure rates or gradient statistics. The paper also does not isolate whether the improvement comes from prototype scoring or simply from removing global standardization. Under ℓ2 normalization, the scorer reduces to s~d=1βlog⁡1L∑lexp⁡(β⟨θl,hd⟩)\tilde s_d=\tfrac1\beta\log\tfrac1L\sum_l\exp(\beta\langle\theta_l,h_d\rangle), a log-mean-exp of cosine similarities. Since this is the key new component, it should be compared against simpler scorers, such as an MLP without global standardization or a single prototype.

    Efficiency: Fig. 4 compares mLSTM only against full self-attention, whereas several baselines (ABMIL, S4/Mamba-MIL) are already linear in token count. The abstract’s claim of “much higher efficiency” is therefore not measured end-to-end, and the cost of dense overlapping feature extraction is unreported.

    Positioning: partitioning operates on a fixed-resolution feature map, so it adapts aggregation granularity rather than encoding resolution. The relationship to post-extraction pooling/merging methods should be clarified.

Questions for authors
  1. Does Uniform + mLSTM match WEAVE at the main-experiment token budgets, particularly on PANDA? Please also include a tissue-aware uniform baseline at a matched budget.

  2. Can you compare against at least two MIL baselines using the same RetCCL features and splits, with multiple seeds?

  3. How often does DART fail, and does removing global standardization from its MLP scorer resolve the instability?

  4. What are the units of sscore=96s_{\text{score}}=96 and spatch=48s_{\text{patch}}=48? Please report feature-extraction and downstream costs separately, with comparisons against ABMIL and an efficient sequence-based MIL baseline.

1 comment

We thank the reviewer for the thorough and precise review, and for recognizing the clean pipeline and the candid scoping of our work. Several concerns are shared with the other reviewers and are addressed with new results in the General Response. Below, we summarize the results in the context of each point and answer the remaining points directly. All changes are marked in the revised PDF.

W1a / Q1: Uniform + mLSTM

“Most importantly, the design lacks Uniform + mLSTM. […] a stronger aggregator may already suppress irrelevant tokens.”

Added to Table 2 (General Response §1). Under mLSTM, adaptive patchification still clearly helps on DHMC (Acc 0.716 → 0.875), but only slightly on TCGA-NSCLC (0.885 → 0.897, equal AUC), so the reviewer’s concern is partly confirmed. We state this in the revised ablation paragraph. Ablations at the main-experiment budgets and on PANDA were not feasible within the rebuttal period. We now note this in the Limitations.

W1b: Scope of the controlled ablation

“Table 2 is controlled, but it covers only two datasets at ~700/850 tokens […] with three runs and high variance for the uniform baseline on DHMC.”

Agreed. The new Uniform + mLSTM row shares this high variance on DHMC, and we added the limited scope of the ablations to the Limitations.

W1c / Q1: Tissue-aware vs. learned allocation

“[…] it remains untested whether the gain comes from learned reallocation within tissue or from tissue-aware allocation alone.”

We evaluated initial and learned scores against pixel-level annotations (General Response §2, new Table 3). The answer is dataset-dependent: on BRACS, training improves within-tissue separation (AUC 0.631 → 0.721); on PANDA, the allocation remains largely tissue-aware (0.576 → 0.596).

W1d / Q2: Controlled baselines

“A smaller encoder does not by itself make the overall input configuration less favorable.”
“Can you compare against at least two MIL baselines using the same RetCCL features and splits, with multiple seeds?”

We agree with the reviewer and revised the corresponding statements (Sec. 5 Implementation and Sec. 6 Discussion). 

Training further MIL baselines on RetCCL features was not feasible within the rebuttal period. Instead, we ran WEAVE with UNI features on the same patches (General Response §3) and observed performance close to RetCCL, so the encoder is not the main factor behind the differences to the baselines.

W2a / Q3: DART failure and simpler scorers

“How often does DART fail, and does removing global standardization from its MLP scorer resolve the instability?”

During development, we tried several variants of DART’s scorer, including removing its global standardization. Across all variants, we did not obtain a single run that avoided collapse, and none reached competitive performance. The original scorer produced NaN gradients or, with gradient clipping, collapsed to concentrated or near-uniform scores. Without global standardization, the MLP scorer still collapsed. We found that removing standardization alone is therefore not sufficient.

A single prototype corresponds to L = 1 in our new sweep (Appendix C.4, Prototype Sensitivity). It trains stably with comparable AUC to L = 10 (0.964 vs. 0.970) but lower accuracy and F1, so the prototype formulation itself is what enables stable training.

W2b / Q4: Efficiency and units

“What are the units of s_score and s_patch? Please report feature-extraction and downstream costs separately, with comparisons against ABMIL and an efficient sequence-based MIL baseline.”

sscores_{\mathrm{score}} and spatchs_{\mathrm{patch}} are strides in pixels, which is now defined in the notation table. Thank you for catching this error.

For costs (General Response §4): compared with 2DMambaMIL on the same features, WEAVE needs 2.5–4.1× fewer inference FLOPs and up to 2.8× less training memory, with 12–31% faster training steps; inference is slower on DHMC due to our unoptimized PyTorch mLSTM. Reducing extraction density about 25× on PANDA gives comparable performance. We did not benchmark ABMIL within the rebuttal period. We revised the abstract’s efficiency claim to “at lower computational and memory cost.”

W2c: Positioning

“[…] it adapts aggregation granularity rather than encoding resolution. The relationship to post-extraction pooling/merging methods should be clarified.”

We agree. WEAVE partitions a fixed-resolution feature map into spatially contiguous regions that jointly cover the slide, with geometry from a learned, differentiable importance distribution. Unlike pruning, no location is discarded and unlike similarity-based merging, merged regions are always spatially contiguous. We revised the introduction and related work accordingly.


We hope the new ablation, annotation analysis, encoder ablation and cost comparison address the reviewer's main concerns. If so, we would be grateful if they would consider updating their score. We are happy to answer any further questions.

Comments on the paper

4 comments
AuthorsOpenReview ↗

General Response to All Reviewers #1

We thank all reviewers for their careful and constructive reviews. Their suggestions led us to run several new experiments, which substantially improved the soundness of our results. In particular, running WEAVE with a second encoder, as suggested by Reviewers pEsr and e8nV, led us to identify and fix a bug in our PANDA pipeline (§3). Below, we summarize the new results that address concerns shared by several reviewers. Individual points are answered in the replies to each reviewer. All changes are marked in in the revised PDF.

1. Uniform + mLSTM: does adaptive patchification still help under the final aggregator? (pEsr, e8nV)

We added the missing Uniform + mLSTM configuration to Table 2, under exactly the same protocol: identical RetCCL features, token budget, training settings and mLSTM configuration. Values are mean ± std over three runs.

MethodDHMC AccDHMC F1DHMC AUCNSCLC AccNSCLC F1NSCLC AUC
Uniform + Mean-Max0.718 ± 0.1010.655 ± 0.1020.921 ± 0.0500.851 ± 0.0350.850 ± 0.0350.959 ± 0.006
Uniform + mLSTM (new)0.716 ± 0.1070.725 ± 0.0740.966 ± 0.0130.885 ± 0.0110.885 ± 0.0110.962 ± 0.004
Adaptive (Energy) + Mean-Max0.821 ± 0.0140.792 ± 0.0380.953 ± 0.0140.866 ± 0.0290.865 ± 0.0290.967 ± 0.003
WEAVE (Adaptive + mLSTM)0.875 ± 0.0160.841 ± 0.0230.977 ± 0.0110.897 ± 0.0110.896 ± 0.0110.962 ± 0.003
Under mLSTM aggregation, adaptive patchification still clearly improves over uniform patchification on DHMC (accuracy +15.9 points). On TCGA-NSCLC the gain is small (+1.2 points, equal AUC). Both components therefore contribute, but part of the benefit of adaptive patchification can be absorbed by a stronger aggregator, depending on the dataset. We state this explicitly in the revised ablation paragraph and in the Limitations. We were not able to repeat these ablations at the main-experiment token budgets within the rebuttal period.

2. Do the importance scores capture more than tissue presence? (pEsr, e8nV)

Because the prototypes are initialized along a tissue–background direction, the scores might mainly encode tissue presence. We tested this against the pixel-level annotations of PANDA (744 annotated slides) and BRACS (37), which are used neither for initialization nor for training, comparing initial and learned scores. We report per-slide ROC-AUC, averaged over annotated slides, as mean ± std over three runs. Full includes background locations; Tissue is restricted to tissue locations.

ScorerPANDA fullPANDA tissueBRACS fullBRACS tissue
Initialized prototypes0.965 ± 0.0010.576 ± 0.0190.752 ± 0.0120.631 ± 0.012
Learned prototypes0.966 ± 0.0090.596 ± 0.0820.833 ± 0.0140.721 ± 0.046
The results differ between datasets. On BRACS, training clearly improves the separation of annotated regions, both over the full slide and within tissue, so the scorer learns diagnostically relevant differences beyond tissue presence. On PANDA, both initial and learned scores separate tissue from background almost perfectly, but differentiate only weakly within tissue, so there the allocation mainly reflects tissue presence. This analysis is now part of Sec. 5 (new Table 3), and the dataset dependence is stated in the Limitations. We did not run random or tissue-guided allocation baselines within the rebuttal period, and we note this in the Limitations as well.
AuthorsOpenReview ↗

General Response to All Reviewers #2

3. Encoder ablation with UNI, and a corrected PANDA result (pEsr, e8nV, Qjj8)

Encoder ablation. To test whether WEAVE depends on RetCCL feature statistics, we re-ran WEAVE with UNI features, the encoder used for all baseline results in Table 1. To isolate the effect of the encoder, we applied UNI to exactly the same patches as RetCCL (overlapping 256×256 patches at 10×), and reused the RetCCL hyperparameters without retuning. The energy-based scorer trains stably on UNI features without modification. Values are mean ± std over five runs.

DatasetRetCCL AccRetCCL F1RetCCL AUCUNI AccUNI F1UNI AUC
BRACS0.7126 ± 0.01410.6837 ± 0.02090.8617 ± 0.01570.6828 ± 0.02890.6362 ± 0.03670.8666 ± 0.0122
DHMC0.8926 ± 0.03070.8653 ± 0.02920.9609 ± 0.01070.8792 ± 0.03830.8469 ± 0.04270.9755 ± 0.0130
PANDA0.8134 ± 0.00840.7892 ± 0.00880.9581 ± 0.00290.7947 ± 0.01090.7573 ± 0.01600.9523 ± 0.0046
TCGA-NSCLC0.8889 ± 0.01760.8886 ± 0.01760.9613 ± 0.00980.8943 ± 0.01890.8939 ± 0.01890.9651 ± 0.0079
TCGA-BRCA0.9544 ± 0.00610.9293 ± 0.00860.9693 ± 0.00350.9474 ± 0.01050.9188 ± 0.01560.9861 ± 0.0061
The choice of encoder makes only a modest difference. On DHMC, PANDA, TCGA-NSCLC and TCGA-BRCA, accuracy and F1 lie close to each other, and UNI achieves slightly higher AUC on all datasets except PANDA. On BRACS, UNI does not close the gap to the strongest baseline, so the encoder is unlikely to be its main cause, whereas magnification, which is 10× for both of our variants, remains a possible factor. These results show that WEAVE transfers to a stronger encoder without changes (Appendix C, Encoder Ablation). We did not additionally evaluate CONCH within the rebuttal period.

Correction of our PANDA results. When comparing WEAVE with RetCCL and UNI features, results agreed closely on four datasets, but differed substantially on PANDA. This led us to re-inspect our RetCCL pipeline for PANDA, where we found a bug affecting only PANDA. The results on the other four datasets are unchanged. After fixing it and verifying that our splits match those of Zhang et al. (2025), we re-ran WEAVE with RetCCL features on PANDA:

AccF1AUC
WEAVE (RetCCL), original submission0.5477 ± 0.00660.4821 ± 0.00820.8239 ± 0.0039
WEAVE (RetCCL), corrected0.8134 ± 0.00840.7892 ± 0.00880.9581 ± 0.0029
We updated Table 1 accordingly. The corrected RetCCL result is now consistent with the UNI result on PANDA (Acc 0.795). The gap to the reported baselines on PANDA is now large. Since those baselines were evaluated in a different pipeline, we do not attribute this gap to WEAVE alone, and we state this in Sec. 5.
AuthorsOpenReview ↗

General Response to All Reviewers #3

4. Computational cost and extraction density (pEsr, e8nV, Qjj8)

Slide-level model. We compared WEAVE and 2DMambaMIL on DHMC and TCGA-BRCA, using the same precomputed RetCCL features for both models, so that feature extraction is identical and the comparison isolates the downstream model (Appendix C, Computational Cost). Costs are per slide, measured on five matched slides per dataset on an NVIDIA A40 in FP32 with batch size 1, using five warm-ups and 20 timed repetitions. FLOPs are computed analytically and exclude normalization, nonlinearities and non-matrix tokenizer arithmetic.

DatasetModelInference GFLOPsInference (ms)Training FW+BW (ms)Training peak (GiB)
DHMCWEAVE46.4153.1251.75.38
DHMC2DMambaMIL191.9107.8285.08.16
TCGA-BRCAWEAVE127.2201.7421.45.56
TCGA-BRCA2DMambaMIL316.9215.2607.815.66
WEAVE requires 2.5–4.1× fewer inference FLOPs and up to 2.8× less peak training memory, and a training step is 12–31% faster. Its memory stays nearly constant across datasets because the token budget is fixed, whereas the memory of 2DMambaMIL grows with slide size. Inference is slightly faster on TCGA-BRCA but slower on DHMC. We attribute this mainly to implementation: our mLSTM uses a plain PyTorch implementation, whereas 2DMambaMIL uses an optimized CUDA scan kernel. Optimized mLSTM kernels offer a path to further acceleration. We did not additionally benchmark ABMIL within the rebuttal period.

Extraction density. We agree with the reviewers that dense overlapping feature extraction dominated the cost of our original pipeline, and we found that it is not necessarily required. On PANDA with UNI features, we kept only every fifth position of the dense feature map along each axis. This increases the effective stride from 48 to 240 pixels, leaving only 16 pixels of overlap between neighboring patches, and corresponds to encoding about 25× fewer patches. Over five seeds at the same median budget of 360 tokens:

ExtractionAccF1AUC
Dense (stride 48)0.784 ± 0.0030.750 ± 0.0070.950 ± 0.006
Sparse (stride 240)0.780 ± 0.0120.748 ± 0.0100.946 ± 0.005
All differences lie within one standard deviation (Appendix C.8, Feature Extraction Density). This experiment covers a single dataset at a reduced token budget, but it indicates that extraction density, and with it the dominant cost of the pipeline, can be reduced substantially without a large loss in performance.

Following the reviewers’ comments, we revised the efficiency claim in the abstract from “much higher efficiency” to “at lower computational and memory cost,” which reflects what we measured.

Area_Chair_ETfFOpenReview ↗

Reminder – Please engage with author rebuttal

Hi, Reviewers,

We are now approaching the end of the discussion phase. The authors have posted their rebuttal, and I encourage all reviewers to engage actively in the discussion.

Please take the time to: i) read the authors' responses carefully; ii) check whether your concerns and questions have been adequately addressed; and iii) provide the authors with further feedback where needed.

Your active participation at this stage is important for reaching a well-informed final decision.

Thank you for your time and effort.

Best,

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.