acceptodds
Under review as a conference paper at ICLR 2027

Stabilizing Slide Representations for Continual Whole-Slide Image Learning: Frozen Attention and Prototype Rehearsal

Abstract

Class-incremental whole-slide image (WSI) classification must learn newly arriving cancer types without catastrophic forgetting of previously learned classes and without access to data from earlier tasks. Compact class prototypes offer an exemplar-free way to rehearse earlier classes, but a prototype is valid only in the representation space that produced it. Freezing only the patch encoder does not stabilize slide-level representations: a trainable multiple-instance learning attention aggregator maps the same patch bag to different embeddings as its attention changes across tasks, making stored prototypes stale. We introduce PACE (Prototype-Anchored Continual Expansion), which fixes a pretrained gated-attention aggregation map so that identical patch features yield identical embeddings across tasks, and rehearses K class-conditional K-means centroids in this fixed space; only an expanding linear classifier adapts to the new cancer types. PACE retains no slides, patch bags, or pseudo-samples and needs no task identity at inference. Across seven public cohorts, 11 class-incremental curricula, and 21 continual baselines, PACE attains the highest mean end-of-stream average balanced accuracy (AACC) on eight curricula and ties on one; its margins over the strongest competitor reach 10.5 points, and slide-retaining methods lead on the other two curricula, where classes are smallest or most heterogeneous. A matched ablation shows that fixing the aggregation map, not its initialization, is what makes rehearsal effective, and that one prototype per class (K = 1) recovers 82% of the AACC gain achieved with K = 16. For continual prognosis, a prototype-free extension with frozen cohort-specific Cox heads also attains the highest macro-averaged concordance index among the evaluated continual methods under all five orderings of a six-cohort stream, though its margins over the strongest competitor (0.8–2.5 points) are smaller than in diagnosis and within the standard deviation across folds. Stabilizing the slide representation at the aggregation map thus makes compact, slide-free prototype rehearsal effective for continual WSI diagnosis and, without prototypes, extends to cohort-incremental prognosis.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Continual learning, Whole-slide image classification, Multiple-instance learning, Computational pathology, Prototype rehearsal, Representation stability

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_jY23OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper looks at the instability of whole-slide image (WSI) representations in continual learning. Even when patch features are frozen, updates to the attention-based aggregation module can still change slide-level representations. The proposed PACE framework fixes both the projection and attention aggregation modules to maintain a stable slide-level feature space. It then stores a small set of clustering-based prototypes for each class and trains the classifier through prototype replay. The experiments evaluate classification across multiple datasets, task partitions, and arrival orders, and also include an extension to survival prediction.

Critical strengths

The experimental evaluation is extensive, covering multiple classification datasets, different task partitions and arrival orders, and a broad range of baselines. The paper also systematically examines freezing strategies, prototype counts, and replay weights. The problem is practically relevant, and the connection between the identified source of representation instability and the proposed design is clearly articulated.

Critical weaknesses

There is not much methodological novelty. The core ideas maintaining a stable representation space, preserving past knowledge through prototypes, and using multiple prototypes to capture intra-class variation have clear precedents in prior work. The methodological contribution therefore lies primarily in integrating these ideas into a continual learning framework for WSIs and validating their effectiveness through extensive experiments. This integration has practical value, but the specific methodological advances beyond existing work remain insufficiently clear.

Questions for authors
  1. The results across task-granularity settings in Table 5 use different class weights. AACC averages class recalls within each task and then weights tasks equally, so moving from six tasks to three increases UBC-OCEAN’s total weight from 1/6 to 1/3. The reported one-percentage-point variation therefore reflects both changes in model predictions and changes in evaluation weights, and does not by itself establish robustness to task granularity. I suggest using the final predictions on the existing shared test splits to additionally report global macro recall with equal weights across all 16 classes. This would provide a consistent basis for comparison and better substantiate the robustness claim.
  2. Several standard deviations are inconsistent between Table 1 and the detailed results in Tables 10 and 12. For CF, Fine-tuning is reported as 0.167 ± 0.036 in Table 1 but 0.167 ± 0.009 in Table 10. For RF, MICIL and LoRA are both reported as 0.077 ± 0.000 in Table 1, whereas Table 12 reports 0.077 ± 0.009 and 0.077 ± 0.005, respectively. All three tables specify means and standard deviations over five runs, but these discrepancies are unexplained. Please verify the statistics against the corresponding per-run records and ensure consistency between the main text and the appendix. If different run sets or statistical procedures were used, please state this explicitly. I would consider raising my score if the authors provide clear explanations and targeted supporting evidence during the rebuttal.
4 comments
AuthorsOpenReview ↗

We sincerely thank the reviewer for the detailed and constructive feedback. All table, figure, and section numbers refer to the revised manuscript, in which changes are highlighted in green for ease of review.

We sincerely appreciate your thoughtful review and the care you have taken to examine the standard deviations reported in the tables. We provide our responses to each of your questions below.

Q1: Table 5 uses different class weights across task granularities; please report global macro recall with equal weights over the 16 classes on the existing test splits.

We report global macro recall in Table 7, for the same runs as the task-granularity table (Table 5 is now Table 6), and it changes the conclusion as you anticipated:

Cross-cohort benchmark: means over five seeds, with the best baseline of each column in parentheses.

Metric6 tasks4 tasks3 tasks
AACC (tasks weighted equally), PACE0.8530.8490.843
AACC, best baseline0.822 (iCaRL)0.833 (AdvMIL)0.843 (AdvMIL)
Macro recall (16 classes weighted equally), PACE0.8310.8430.858
Macro recall, best baseline0.813 (iCaRL)0.824 (AdvMIL)0.841 (AdvMIL)

You are right that the one-point AACC variation mixes model behavior with evaluation weights (UBC-OCEAN's weight rises from 1/61/6 to 1/31/3). Under equal class weights, PACE keeps the highest mean at all three groupings, by 1.7–1.9 points, within the runner-up's sd (0.047, 0.039, and 0.018), which we read as parity. Its spread widens from 1.0 to 2.7 points and the trend reverses: macro recall rises as tasks are merged, whereas AACC falls. The result therefore does not establish robustness to task granularity, and we no longer claim it: we state only that PACE has the highest or tied-highest mean at all three granularities under both weightings (Secs. 6.1 and C).

Q2: Several standard deviations differ between Table 1 and the detailed tables (fine-tuning on CF; MICIL and LoRA on RF). Please verify against the per-run records and state whether different run sets or procedures were used.

You are right: the entries should not have differed, and they now agree. Both tables report the same statistic, the mean ±\pm sd over five seeds of each method's validation-selected configuration (Secs. 5 and E). Fine-tuning on Cancer family now reads 0.167±0.0090.167\pm0.009 in both (Tables 1 and 17 in the revised manuscript; originally Tables 1 and 10), and MICIL and LoRA on Rare-first read 0.077±0.0090.077\pm0.009 and 0.077±0.0050.077\pm0.005 (Tables 1 and 19 in the revised manuscript; originally Table 12).

Cause. Different run sets, not different statistical procedures. The sds that disagreed came from separate executions of the same configurations, and executions can differ through seed-to-seed variation (including weight initialization) and library nondeterminism. The means agreed at the reported precision in the three entries you list because these baselines collapse to approximately 1/T1/T (0.167 for six tasks, 0.077 for thirteen).

W1: Limited methodological novelty: stable representations, prototype memory, and multiple prototypes have precedents.

We agree that the core ideas have precedents (the revised manuscript now says that the components are established; Sec. 2 and Table. 5) and that the contribution is their integration for WSIs and its empirical validation. Relative to the compared frozen-representation methods, what is specific to PACE is the combination of a complete bag-to-slide map fixed before the first task, a capped multi-centroid memory, and rehearsal through the expanding head only (Table 5 of the revised manuscript). The evidence that fixing the map matters is the matched single-run ablation: trained throughout, the same map is 14.1–28.3 points lower in every matched pair on three curricula (λproto=10\lambda_{\mathrm{proto}}=10; Tables 12–14). Controls also show that the fixed map need not be the pretrained attention map (Table 10), so what we claim is an empirical finding about where to fix the representation, not a new aggregator.

The authors have made the necessary changes.

We sincerely thank the reviewer for the follow-up and constructive feedback. The comments have helped us to further strengthen the evaluation, improve the consistency of the reported results, and clarify the scope of our contribution. We appreciate the reviewer’s time and careful consideration of the revisions.

AuthorsOpenReview ↗

A Gentle Request for Score Reconsideration

As the discussion period comes to a close, we thank you for reviewing our revisions and acknowledging the changes. We summarize the main updates below:

  • Task granularity (Q1). Table 7 adds global macro recall with equal weights over the 16 classes, as suggested, and the conclusion on task granularity is refined accordingly.
  • Standard deviations (Q2). Table 1 now agrees with the detailed tables (Tables 17 and 19), and our response explains the source of the earlier difference.
  • Scope of contribution (W1). The manuscript clarifies that PACE builds on established components and what is specific to it (Secs. 1 and 2, Table 5).

If these revisions address your main concerns, we would appreciate you reconsidering your overall assessment and score. Please let us know if any concerns remain.

Reviewer_awtNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

I view this paper as primarily an empirical contribution supported by extensive experimental validation. While I appreciate the practical motivation and the broad evaluation, I still have some concerns regarding the methodological novelty.

In particular, (i) the use of K-means to summarize previous classes for memory rehearsal and (ii) the use of an expanding linear classifier for adaptation are both established techniques in representation learning and class-incremental learning.

I believe the paper would benefit from a clearer discussion of how the proposed method differs from existing approaches beyond its application to WSI classification.

Critical strengths
  1. I find the observation regarding representation drift in the MIL aggregation module clear and relevant to class-incremental learning. In my view, identifying this source of instability provides a useful motivation for the proposed design, particularly when adapting to new tasks while preserving knowledge from previous classes.

  2. I appreciate the extensive empirical evaluation, which covers a broad range of datasets, class-incremental settings, and baseline methods. I believe this is one of the main strengths of the paper, as the experiments provide useful evidence of the method's effectiveness across different settings.

  3. I also find the ablation studies informative. The comparisons between frozen and continually trainable attention, different initialization strategies, prototype counts, etc

Critical weaknesses
  1. My main concern is the limited novelty of the proposed method. The framework mainly combines established components, including K-means clustering, cross-entropy-based rehearsal in feature space, and an expanding linear classifier. While I acknowledge the relevance of applying these components at the patch-to-slide MIL boundary, I would like the authors to clarify the methodological contribution beyond combining existing techniques. In my view, the contribution is more empirical than algorithmic.

  2. I find the zero-drift result for identical deterministic inputs somewhat expected, as freezing the entire aggregation mapping naturally prevents representation changes (Eq. 6). Furthermore, I noticed that frozen Xavier-initialized attention achieves performance close to pretrained frozen attention, while retaining the pretrained projection. This raises a question about whether the attention mechanism itself is essential to the proposed framework. I would encourage the authors to compare their method with simpler aggregation approaches, such as mean pooling, under the same experimental settings.

  3. In Section 4.3, the authors use K-means to construct class prototypes. While this is a reasonable and straightforward choice, I am concerned about its effectiveness for heterogeneous cancer classes. K-means relies on Euclidean distance and favors relatively compact clusters, which might not fully capture complex tissue representations. I believe the paper would benefit from discussing this limitation and comparing K-means with alternative prototype selection strategies

Questions for authors
  1. Regarding the proposed design, could the authors clarify the main methodological novelty of PACE compared with existing fixed-representation continual learning methods?
  2. In Section 6.1, I noticed that frozen Xavier-initialized attention performs close to pretrained frozen attention. Does this suggest that learned attention is not essential to PACE?
  3. Although PACE uses K-means across all curricula, I noticed that the detailed prototype ablation focuses on CANCER FAMILY. Given the lower performance on FREQUENT-FIRST and METASTATIC SITE, could the authors analyze whether prototype quality or class heterogeneity contributes to these results?
4 comments
AuthorsOpenReview ↗

We sincerely thank the reviewer for the detailed and constructive feedback. All table, figure, and section numbers refer to the revised manuscript, in which changes are highlighted in green for ease of review.

We sincerely appreciate the time and care you have taken to review our work. We respond to each of your three questions below.

Q1: What is the main methodological novelty of PACE compared with existing fixed-representation continual-learning methods?

We agree: KK-means prototypes, feature-space rehearsal, and an expanding linear head are established, as the revised manuscript now says (Sec. 2), and we regard the contribution as their adaptation and empirical evaluation in WSI-MIL. Table 5 separates what PACE shares with SimpleCIL, FeTrIL, PASS, and ACIL from what is specific to it: (i) the complete bag-to-slide map, including the attention, is fixed before the first task, a component the original methods do not have because they assume one feature vector per input; (ii) the memory is a capped set of Kc≤KK_c\le K centroids per class instead of one mean; (iii) the centroids are rehearsed through the expanding linear head only.

Two results bear on these elements separately. For (i), with the same map trained throughout, the same rehearsal is 14.1–28.3 points lower in every matched single-run pair on three curricula (λproto=10\lambda_{\mathrm{proto}}=10; Tables 12–14). For (ii) and (iii) together, PACE's mean AACC is above SimpleCIL's and FeTrIL's on all eight CHTN curricula (mean differences +16.2 and +5.3 points; signed-rank p=0.008p=0.008 each, uncorrected; Table 9); as ported here, both freeze their aggregator after the first task, so this comparison is between fixed-map methods. A single centroid already recovers 82% of the K=16K=16 gain on Cancer family at λproto=10\lambda_{\mathrm{proto}}=10, so most of the gain in that setting is achieved without multiple centroids.

Q2: Frozen Xavier-initialized attention performs close to pretrained attention: is learned attention essential? Please compare with mean pooling under the same settings.

Learned attention is not essential, and the revision reports the mean-pooling comparison. In seed-matched fixed-map controls across Grouped increments, Diagnostic triage, and Organ subtype with PACE's rehearsal (K=8K=8; Table 10, lower block), mean pooling over the PC-108 projection is within 1.8 points of PACE, Xavier attention is within 4.0 points, and mean pooling over unprojected UNI features, a map with no learned parameter, is 3.5–5.7 points better on all three. The revised manuscript now states that neither learned attention, a particular initialization, nor its pretraining is needed there, and that the PC-108 map is not claimed to be the best fixed map (Secs. 6.1 and 7). The regime ablation also favors a fixed map: trained throughout, the same map is 14.1–28.3 points lower in matched run pairs (λproto=10\lambda_{\mathrm{proto}}=10; Tables 12–14). We also agree that zero drift under a fixed map is expected; the paper now presents it as holding by construction (Fig. 1).

Q3: Do prototype quality or class heterogeneity contribute to the lower results on Frequent-first and Metastatic site? KK-means favors compact clusters; please compare alternative prototype strategies.

Prototype construction. We agree that Euclidean KK-means favors compact clusters, so Table 11 compares it at matched memory (K=8K=8) with spherical KK-means, kk-medoids, herding, random embeddings, and class-mean-plus-Gaussian sampling on five curricula. KK-means has the highest mean on Frequent-first (0.652; next 0.625) and Metastatic site (0.643; next 0.601), so no tested alternative closes the gap to AdvMIL or iCaRL. It is not uniformly best: spherical KK-means, a test of sensitivity to distance geometry, is higher on Cancer family and lower on the other four, with every interval including zero, and herding is higher on Organ subtype.

Prototype count. AACC still rises with KK up to the grid maximum on both curricula (Tables 12, 13, 14, and Fig. 4).

Class heterogeneity. Frequent-first contains the same 14 classes as Rare-first, where PACE has the highest mean, so the heterogeneity of these classes alone does not explain the difference in ranking, and AdvMIL's 5.6-point lead is not distinguishable from seed variation (interval −18.7 to +7.5 points; Table 9).

Class size. PACE's per-class recall rises only weakly with class size (Spearman ρ\rho: median +0.23 over the eight curricula, in the range of iCaRL, SimpleCIL, and FeTrIL; Table 16). On Metastatic site, its two largest recall deficits against iCaRL are in classes with 24 and 36 training slides (0.48 and 0.17 against 0.68 and 0.46), but the last four tasks have 5–7 test slides each, so the 5.8-point gap is on the scale of two or three predictions. This is consistent with, but does not establish, a centroid summary being limited by class size (Sec. 7).

Reviewer_awtNOpenReview ↗

Responses to Authors

I thank the authors for the detailed rebuttal. Sorry for delayed responses

Regarding Q1. The authors agree that the components are established and that the contribution is empirical. I accept this, but it confirms my original concern. I thank the authors for Table 5, which helps clarify the differences from existing methods. I suggest presenting this comparison in the introduction rather than in the appendix.

Regarding Q2. I appreciate the new controls in Table 10. However, mean pooling over raw UNI features, with no learned parameters, is 3.5–5.7 points better than PACE on all three curricula. If I am not mistaken, the frozen gated attention and the PC-108 prior, which define the method and the title, are therefore not needed.

Regarding Q3. Table 11 addresses my question, and I consider it resolved. However, the cause of the lower performance on Frequent-first and Metastatic site remains open.

Suggestions

Reframe the title and contributions. The results show that fixing the aggregation map matters, not the frozen attention itself. A title and abstract centred on “a fixed aggregation map” would match the evidence better than “Frozen Attention”. Add a limitations paragraph. It could cover the missing replay baselines under the same fixed map, the lower performance on Frequent-first and Metastatic site, and the fact that the fixed-map ablations are single runs.

Overall. The rebuttal is honest and improves the soundness of the paper. However, the new results suggest that the gain comes mainly from frozen features with a linear classifier and centroid replay, rather than from the proposed frozen attention

Thank you for the follow-up comments and the concrete suggestions. We respond to each point below. The changes made for this response are highlighted in blue in the revised manuscript; changes from the previous round remain in green.

Questions: Q1–Q3. Sec. 1 (p. 2) now clarifies that PACE builds on established components and differs from the frozen-representation methods SimpleCIL, FeTrIL, PASS, and ACIL in its point of fixation (the bag-to-slide map), its capped multi-centroid memory, and rehearsal through the expanding head only. It also directs readers to Table 5, retained in Appendix B.4 to accommodate the ten-page main-text limit. The abstract now specifies that neither learned attention nor map pretraining is required in the controls of Table 10, and the title and contributions have been reframed (see Suggestion 1). The benchmark results use the gated-attention map initialized from the PC-108 prior as one effective fixed-map choice. The new final paragraph of Sec. 7 (see Suggestion 2) discusses curriculum-specific performance on Frequent-first and Metastatic site and identifies broader comparisons, alternative representations, and evaluation across training seeds as directions for future study.

Suggestion 1. Reframe the title and contributions; a title and abstract centred on “a fixed aggregation map” would match the evidence better than “Frozen Attention”.

We have done so.

  • Title: “Stabilizing Slide Representations for Continual Whole-Slide Image Learning: A Fixed Aggregation Map and Prototype Rehearsal”.
  • Abstract: PACE is now defined as fixing the bag-to-slide aggregation map, with pretrained gated attention as the instance evaluated.
  • Introduction: the paragraph introducing PACE and contribution 2 now refer to the aggregation map instead of the attention aggregator, and contribution 3 concludes that “the fixed map need not be pretrained or attention-based”.

Suggestion 2. Add a limitations paragraph covering the missing replay baselines under the same fixed map, the lower performance on Frequent-first and Metastatic site, and the fact that the fixed-map ablations are single runs.

The last paragraph of Sec. 7 (p. 10) is now discussing the “Limitations” and covers the three items as you suggested.

  • Replay baselines under the same fixed map: stated as not run, together with what they would settle: separating rehearsal type from map fixation.
  • Frequent-first and Metastatic site: the paragraph states that PACE trails on both and that the cause remains open.
  • Single runs: the ablations that compare a fixed with a trained map are single runs, and the paragraph now says that they give no estimate of seed variance.

Overall. The new results suggest that the gain comes mainly from frozen features with a linear classifier and centroid replay, rather than from the proposed frozen attention.

We agree. Sec. 7 already attributes the class evidence mainly to the pretrained encoder, with the classifier handling continual learning provided the map stays fixed. The title and abstract no longer present frozen attention as the source of the gain.

AuthorsOpenReview ↗

Suggestions Incorporated: A Gentle Request for Score Reconsideration

As the discussion period draws to a close, we thank you for the follow-up, for appreciating Table 5 and the new controls in Table 10, for considering the question on prototype strategies resolved by Table 11, and for noting that the rebuttal “is honest and improves the soundness of the paper.” We have incorporated your suggestions and made the following updates to the manuscript (changes marked in blue).

  • Reframing (Suggestion 1, Q2). The title now reads “…: A Fixed Aggregation Map and Prototype Rehearsal.” The abstract and contributions are centred on the fixed aggregation map, and the abstract notes that neither learned attention nor map pretraining is needed in the controls of Table 10.
  • Limitations (Suggestion 2, Q3). The final paragraph of Sec. 7 discusses Frequent-first and Metastatic site and outlines replay baselines under the same fixed map, broader comparisons, and evaluation across training seeds as future directions.
  • Comparison with existing methods (Q1). Sec. 1 now clarifies that PACE builds on established components and how it differs from SimpleCIL, FeTrIL, PASS, and ACIL, with a pointer to Table 5.

If these changes address your main concerns, we would appreciate your reconsidering your overall assessment and score. Please let us know if any concerns remain.

Reviewer_kfHYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies continual learning for whole-slide image (WSI) classification. The main issue considered is that, even when the patch encoder is frozen, a trainable MIL aggregator can still change the slide-level representation across tasks. This makes previously stored prototypes less useful because they were constructed in an old representation space.

The authors propose PACE, which freezes both the patch encoder and the Gated ABMIL aggregation map, and only updates an expanding linear classifier. After each task, the method stores several class-conditional K-means centroids in the fixed slide-level embedding space. These prototypes are replayed when learning later tasks.

The method is evaluated on 11 diagnostic curricula constructed from several public WSI datasets and compared with 21 continual-learning baselines. PACE performs strongly on most curricula while storing no previous WSIs or patch bags. The paper also includes ablations on prototype number, replay weight, aggregator freezing, and attention initialization. In addition, the authors study a continual survival setting using a fixed shared aggregator and frozen cohort-specific Cox heads.

Critical strengths
  1. Strong empirical evaluation. The paper evaluates the method on many diagnostic curricula and compares against a large number of continual-learning baselines. The results are fairly consistent across settings, which supports the main empirical finding that fixing the slide representation can make compact prototype rehearsal effective.

  2. The method is simple and practical. PACE directly addresses a useful issue in continual WSI learning. A trainable MIL aggregator can change slide-level features even when the patch encoder is frozen, which can make stored prototypes outdated. Freezing the aggregation map avoids this issue and allows the method to use a very small memory instead of keeping old slides or patch bags.

Critical weaknesses
  1. Some of the cross-cohort conclusions are difficult to interpret under the current metric.

The comparison across CROSS-COHORT-6/4/3 is presented as measuring robustness to task granularity. However, AACC gives equal weight to tasks rather than to classes, while these three curricula group the same 16 classes very differently. As a result, changing the task partition also changes how much each class contributes to the final score. This makes it hard to separate a true task-granularity effect from a change induced by the metric itself.

I do not think this invalidates the main results, but it weakens the specific claim that PACE is insensitive to task granularity.

  1. The ablation does not fully isolate what part of the frozen representation is actually important.

The paper argues that freezing is more important than initialization, based partly on the comparison between PC-108 and Xavier-initialized attention. However, the pretrained PC-108 projection is retained in both cases. Thus, the random condition is still using part of the pretrained aggregation map.

This matters because the paper further interprets the result as evidence that the pretrained attention contributes little and that most useful information already comes from the patch encoder. I think the current experiment supports the first statement only partially and does not cleanly support the second one.

  1. I found the survival results difficult to reconcile with the method description.

For PACE, the survival setting uses a fixed shared aggregator and an independent Cox head for each cohort, with previous heads frozen. As written, there seems to be no learned state passed from one cohort to the next. Therefore, I would expect the final PACE result to be essentially independent of the cohort ordering.

However, the reported PACE C-index changes noticeably across the five orderings. This could be due to implementation details or randomization, but the current paper does not explain it. Since the survival section uses these orderings as part of its evaluation, I think this needs clarification.

Overall, these concerns appear potentially fixable rather than fundamental to the main diagnostic contribution. In particular, if the authors can clarify the survival-ordering discrepancy, provide a fairer cross-granularity comparison, and narrow or better support the initialization claim, I would be open to revising my overall assessment.

Questions for authors
  1. For the CROSS-COHORT experiments, could you report a metric that assigns the same weight to each of the 16 classes across the 6-, 4-, and 3-task versions? I would like to know whether the conclusion about task-granularity robustness remains the same under fixed class weighting.

  2. In the survival experiments, what specifically causes PACE to obtain different results for different cohort orderings? Given that the aggregator is fixed and each cohort head appears to be trained independently, I would expect the same cohort to produce the same model regardless of when it appears in the stream. A description of the source of this variation would materially affect my assessment.

  3. Could you clarify the intended scope of the initialization claim? In the Xavier setting, the attention parameters are randomized but the PC-108 projection remains pretrained. I would be more convinced by either a full random/identity projection control or a narrower claim explicitly referring only to attention initialization.

4 comments
AuthorsOpenReview ↗

We sincerely thank the reviewer for the detailed and constructive feedback. All table, figure, and section numbers refer to the revised manuscript, in which changes are highlighted in green for ease of review.

Thank you for your careful review. We address each of your three questions below.

Q1: A metric with equal weight on each of the 16 classes across the 6-, 4-, and 3-task versions: does the conclusion about task-granularity robustness remain?

PACE still has the highest mean at all three groupings, with leads within the runner-up's seed variation; we therefore no longer interpret the small AACC spread as evidence of task-granularity robustness. Table 7 reports macro recall with equal weight on each of the 16 classes for the runs of Table 6:

Cross-cohort benchmark: means over five seeds, with the best baseline of each column in parentheses.

Metric6 tasks4 tasks3 tasks
AACC (tasks weighted equally), PACE0.8530.8490.843
AACC, best baseline0.822 (iCaRL)0.833 (AdvMIL)0.843 (AdvMIL)
Macro recall (16 classes weighted equally), PACE0.8310.8430.858
Macro recall, best baseline0.813 (iCaRL)0.824 (AdvMIL)0.841 (AdvMIL)

Under macro recall the leads are 1.7–1.9 points, within the runner-up's sd (0.047, 0.039, and 0.018). PACE's spread across groupings widens from 1.0 point under AACC to 2.7 under macro recall, and the trend reverses: AACC falls and macro recall rises as tasks are merged. As you argued, the AACC spread mixed task granularity with class weighting (the UBC-OCEAN task rises from 1/61/6 to 1/31/3 of the score). We now claim only that PACE has the highest or tied-highest mean at all three granularities under both weightings (Secs. 6.1 and C).

Q2: In the survival experiments, what specifically causes PACE to obtain different results for different cohort orderings?

Run-to-run randomness, not order; your expectation is correct. The aggregator is fixed, each cohort's head is trained alone, and no learned state crosses cohorts. The five orderings in Table 8 were five separate runs that used PyTorch's default, unseeded generator and independent hyperparameter selection, so head initialization, minibatch order, and dropout masks differed between runs. We verified this for PACE by re-running all five orderings with seeds derived from a fixed base seed and each cohort's name: its per-cohort C-indices then agree within 10−410^{-4} across orderings for every fold and configuration. Under the same selection rule, the six available runs give 0.578–0.603 (mean 0.589, sd 0.009, below the fold-to-fold sd of 0.016–0.035), and the seeded re-run gives 0.582±0.0290.582\pm0.029 under every ordering (Table 28); the value reported for the stream, 0.603, is the highest of the six. PACE's margins over the runner-up (0.8–2.5 points) are no larger than this range, so we now read the survival result as parity with the best adapting methods at zero forgetting, not as a ranking (Secs. 6.2 and E.3).

Q3: Scope of the initialization claim: in the Xavier setting the PC-108 projection remains pretrained. Either a full random or identity projection control, or a narrower claim.

We did both: we ran the projection controls and narrowed the claim. On three curricula Grouped increments, Diagnostic triage, and Organ subtype, five alternative maps were fixed before task 1 and used with PACE's rehearsal (K=8K=8; Table 10, lower block):

Mean seed-matched difference PACE − control in AACC points (negative: the control is higher).

ProjectionPoolingGrouped incrementsDiagnostic triageOrgan subtype
PC-108Xavier gated attention+4.0−2.9+1.0
Random (Xavier)Xavier gated attention−2.6−3.0−1.3
IdentityXavier gated attention−1.4−4.2−4.1
PC-108Mean pooling−0.1−1.8+1.0
IdentityMean pooling−5.7−3.5−5.3

PACE is ahead in three of the fifteen mean differences, never with an interval that excludes zero, and the last row, which uses no PC-108 parameter, is higher on all three curricula. Neither the pretrained attention nor the pretrained projection is therefore required for effective rehearsal on these curricula. This control, not the original Xavier comparison, supports the interpretation that the class evidence is already present in features from the pre-trained encoder, as now stated in the revised manuscript (Sec. 7). The Cancer family ablation is now described as varying the attention initialization only, with the PC-108 projection kept (Secs. 1 and 6.1), and we do not claim that the PC-108 map is the best fixed map.

Thank you for the detailed response and additional experiments.

The rebuttal addresses my main concerns, particularly regarding the evaluation metric and survival ordering. However, the survival results appear less convincing given the run-to-run variation and best-run reporting. The new ablations also suggest that the specific pretrained aggregation map may not be necessary.

We sincerely thank the reviewer for the thoughtful follow-up and constructive feedback, which helped further strengthen our evaluation, refine our conclusions, and gain further insight into the role of a fixed representation in prototype rehearsal.

The survival experiments complement our main study of continual WSI classification. We have revised the discussion to emphasize comparable performance with zero forgetting rather than a definitive ranking, while keeping our main conclusions grounded in the classification results.

We again thank the reviewer for helping us to further strengthen the paper.

AuthorsOpenReview ↗

A Gentle Request for Score Reconsideration

As we approach the end of the discussion period, we would like to thank you for your time and constructive feedback. We appreciate your confirmation that the rebuttal addresses your main concerns, particularly regarding the evaluation metric and survival ordering. We summarize the updates below.

  • Evaluation metric (Q1, acknowledged). Table 7 adds macro recall with equal weight on each of the 16 classes, and the conclusion on task granularity is refined accordingly.
  • Survival ordering (Q2, acknowledged). The variation is traced to run-to-run randomness, not cohort order, and verified by seeded re-runs (Table 28).
  • Initialization claim (Q3). Projection and pooling controls were added (Table 10), and the claim was narrowed.
  • Survival results (follow-up). The extension complements the main classification study. Sec. 6.2 presents it as comparable performance at zero forgetting rather than a ranking, and Table 28 reports all runs.
  • Pretrained map (follow-up). As you observed, the new controls indicate that rehearsal does not depend on a specific pretrained map. The title, abstract, and contributions now reflect this, with the PC-108 map as one effective choice.

In light of these revisions, we would appreciate your reconsidering your overall assessment and score. Please let us know if any concerns remain.

Reviewer_R2vxOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies exemplar-free class-incremental WSI classification and argues that freezing only the patch encoder is insufficient, because a trainable MIL aggregator changes slide embeddings across tasks and invalidates stored class prototypes. It proposes PACE, which fixes the entire Gated-ABMIL aggregation map (initialized from the PC-108 prior), trains only an expanding linear classifier, and rehearses K K-means centroids per earlier class in the fixed slide-embedding space, with a prototype-free extension for cohort-incremental survival prediction. Across 11 curricula and 21 baselines, PACE reports the highest mean final balanced accuracy on 8 curricula and a tie on 1, supported by an ablation on one curriculum (CANCER FAMILY) showing that freezing the aggregator is what makes prototype rehearsal effective; results are mean ± sd over five runs without significance tests.

Critical strengths

The paper evaluates 11 curricula (including order-reversal and task-granularity controls) against 21 continual-learning baselines, with patient-stratified splits and validation-based tuning. This makes the benchmark a useful reference for exemplar-free WSI continual learning, and it shows that a simple frozen-feature approach is competitive with, and on several curricula better than, specialised WSI continual-learning methods.

The authors state where PACE loses (Frequent-First, Metastatic Site), treat margins within one standard deviation as parity, and include no-rehearsal controls for each aggregator regime. These controls show that neither freezing the aggregator nor prototype rehearsal suffices alone, which gives a clear and transparent basis for assessing the claims.

Critical weaknesses

The paper argues that fixing the aggregation map is what makes prototype rehearsal effective. Zero drift under a frozen map is true by construction (Eq. 6, Fig. 1), and the ablation only shows that stale prototypes do not help a drifting map. The decisive controls are missing: replay/bias-correction baselines (iCaRL, ER-ACE, DER++, BiC) run with the same frozen aggregator; frozen-feature class-incremental learners (e.g., RanPAC, FeCAM, covariance replay); a fixed mean-pooling or pretrained slide-level encoder (e.g., TITAN); and trainable aggregators with drift compensation. The paper itself finds that frozen random (Xavier) attention is nearly as good as the PC-108 prior (0.717 vs. 0.738), so the contribution reduces to frozen features plus a linear classifier with class-centroid replay, a recipe already used by SimpleCIL, FeTrIL and PASS. Several baselines also perform at fine-tuning level (e.g., MICIL, listed under "No Memory" despite being described as rehearsal-based), which raises the question of whether the gap reflects the method or insufficiently tuned baselines.

Under the paper's own rule (a margin within one sd is parity), only 2 of 8 CHTN curricula (Diagnostic Triage, Organ Subtype) are clear wins; the other four "leads" are within sd, with five runs and no significance tests. The ablation, which carries RQ2–RQ3 and the "82%" claim, covers a single curriculum and a single ρ_proto. Table 1 reports 0.779 for PACE on Cancer Family while the ablation reports 0.859 at K=16 on the same curriculum, and how K and ρ_proto were selected is unclear. The survival extension is a multi-head setting with known cohort identity, where zero forgetting is guaranteed; its margins (0.8–2.5 points) are comparable to the spread across orderings (~1.9 points) that should be zero for an order-invariant pipeline, and the text is inconsistent about which aggregator it uses (PC-108 vs. an independent UNI-v2 map).

Questions for authors
  1. Which K and ρ_proto produce the PACE rows in Tables 1 and 5, and how were they selected (was validation data from the whole stream used)? Table 1 reports 0.779 on Cancer Family, whereas the ablation reports 0.859 at K=16 on the same curriculum. Are the ablation numbers validation or test results?
  2. What do the "five runs" vary (seeds only, or splits)? Please report paired differences or confidence intervals against the strongest baseline, and a significance test across curricula.
  3. For the survival extension, which aggregator is used (PC-108 weights, or an independent map on UNI-v2 features)? Since the pipeline is order-invariant by construction, why does PACE's macro C-index vary by about 1.9 points across orderings?
  4. MICIL is described as rehearsal-based but is listed under "No Memory". How were the baselines adapted and tuned, and why do several (MICIL, CoMEL, EWC, LwF) match fine-tuning?
  5. How many classes have fewer than K training slides, so that a stored "centroid" is a single slide embedding?
4 comments
AuthorsOpenReview ↗

We sincerely thank the reviewer for the detailed and constructive feedback. All table, figure, and section numbers refer to the revised manuscript, in which changes are highlighted in green for ease of review.

Thank you for your thoughtful and detailed review. We address each of your five questions below.

Q1: Which KK and replay weight give the PACE rows of Tables 1 and 5, and how were they selected? Why 0.779 in Table 1 but 0.859 in the ablation? Are the ablation numbers validation or test results?

For each curriculum, the PACE row uses the configuration ranked highest on validation AACC within the grid K∈{1,4,8}K\in\lbrace1,4,8\rbrace, λproto∈{0.1,1,10}\lambda_{\mathrm{proto}}\in\lbrace0.1,1,10\rbrace (λproto\lambda_{\mathrm{proto}} is ρproto\rho_{\mathrm{proto}}; the grid also covers the classifier learning rate and other hyper-parameters; Table 2), then frozen and run with five seeds. Yes, validation data from the whole stream are used: validation AACC is an end-of-stream quantity, so the ranking uses the validation splits of all tasks. This is a stream-level selection, not an online one; it is applied in the same way to every method, and no test result enters it (Sec. B.3). The 0.859 at K=16K=16 comes from a single run outside this grid and is shown only for detailed ablation analysis (Fig. 3). The 0.779 at K=8K=8 is the five-seed mean of the selected in-grid configuration, with a standard deviation of 0.070. The ablation numbers are test results (AACC is computed on the task test sets; Sec. 5); we report them for analysis, and configuration selection used only validation AACC. We believe the reviewer intended to refer to Table 7 rather than Table 5, as Table 7 presents the detailed ablation of AACC across prototype counts on the CF curriculum. To improve readability, we have plotted these results in Fig. 3 in the revised manuscript.

Q2: What do the five runs vary? Please report paired differences or confidence intervals against the strongest baseline, and a significance test across curricula.

Seeds only. Each method's configuration is selected on validation AACC and frozen, and the five runs are five seeds of that configuration on fixed patient-level splits, so the sd is seed-to-seed variation on a fixed test split, not variation across splits (Secs. 5 and B.3). Table 9 reports the requested statistics:

PACE against the strongest baseline on each CHTN curriculum: AACC and difference in points, with an approximate 95% tt interval over seeds (†: the interval excludes zero).

CurriculumStrongest baselinePACEDifference [95% CI]
Cancer familyFeTrIL (0.775)0.779+0.4 [−9.5, +10.3]
Frequent-firstAdvMIL (0.708)0.652−5.6 [−18.7, +7.5]
Rare-firstFeTrIL (0.676)0.679+0.3 [−9.8, +10.4]
Grouped incrementsAdvMIL (0.692)0.711+1.9 [−5.3, +9.1]
Diagnostic triageACIL (0.616)0.662+4.6 [+0.3, +8.9] †
Organ subtypeiCaRL (0.556)0.661+10.5 [+6.2, +14.8] †
Metastatic siteiCaRL (0.701)0.643−5.8 [−11.0, −0.6] †
Primary siteFeTrIL (0.689)0.723+3.4 [−6.5, +13.3]

Across the eight curricula, the exact two-sided signed-rank test on the paired differences gives uncorrected p=0.008p=0.008 against FeTrIL, ACIL, and SimpleCIL (PACE higher on all eight, against ACIL by only 0.001 on Metastatic site; 0.008 is the smallest attainable value) and p=0.078p=0.078 against iCaRL and AdvMIL (higher on six), which is not significant at the 0.05 level. The paper now states these distinctions explicitly: only Organ subtype and, narrowly, Diagnostic triage have 95% difference intervals above zero; the other four leads have intervals that include zero, and iCaRL's lead on Metastatic site has an interval excluding zero (Sec. 6.1); the abstract now restricts its claim to these two leads.

Q3: Which aggregator does the survival extension use, and why does PACE's macro C-index vary by about 1.9 points across orderings of an order-invariant pipeline?

The aggregator is the PC-108 Gated ABMIL prior released for UNI-v2 features, including its 1536→5121536\to512 projection, frozen from the first cohort; no aggregation parameter is trained on the survival cohorts, and the revision now makes this explicit (Secs. 4.2, 5, and E.3). The variation is not an order effect, since no learned state crosses cohorts. The five orderings were separate runs that used PyTorch's default, unseeded generator and their own hyperparameter selection, so head initialization, minibatch order, and dropout masks differed between runs. When all five orderings are re-run with seeds derived from a fixed base seed and each cohort's name, PACE's per-cohort C-indices agree within 10−410^{-4} for every fold and configuration (Table 28); the value reported for the stream, 0.603, is the highest of the six.

We agree with your assessment of this extension. It is a multi-head setting with known cohort identity in which zero forgetting holds by construction, and its margins (0.8–2.5 points) are no larger than this run-to-run range (2.5 points). We now read it as parity with the best adapting methods at zero forgetting, not as a ranking (Sec. 6.2), and the abstract now states that the margins are within fold-to-fold and run-to-run variation.

Q4: MICIL is described as rehearsal-based but listed under “No memory”. How were the baselines adapted and tuned, and why do several (MICIL, CoMEL, EWC, LwF) match fine-tuning?

MICIL is run in its distillation-only form, without retained slides, and is therefore listed under “No memory”; the manuscript now says so (Secs. 2 and B.3). All baselines are ported to one pipeline: the frozen UNI features, the PC-108-initialized Gated ABMIL aggregator, the expanding classifier, the splits, the optimizer, and the schedule; CoMEL is run as its orthogonal low-rank adaptation (Table 3). Each baseline searches learning rate and weight decay ({10−3\lbrace10^{-3}, 10−4}10^{-4}\rbrace for each), plus the rank for LoRA, InfLoRA, and CoMEL, and is selected on validation AACC (Table 2).

Two observations nevertheless suggest that the collapse reflects the lack of old-class inputs during shared-head training (Secs. 6.1 and B.3). (i) Fine-tuning scores about 1/T1/T on Cancer family, Rare-first, and Metastatic site (0.167, 0.078, and 0.143 for T=6T=6, 13, and 7 tasks): only the last task is recognized. Every method that regularizes or adapts a trainable aggregator without stored slides (EWC, LwF, LoRA, InfLoRA, MICIL, CoMEL) and A-GEM stays within 0.19 AACC of fine-tuning on every CHTN curriculum; MICIL and LwF distill the previous outputs, but only on new-task slides. (ii) PACE's own head, trained without rehearsal, collapses to the same level even with the map frozen (0.173 on Cancer family; 0.10–0.18 in Tables 12–14). Our most direct comparisons therefore rest on the methods that present the head with old-class inputs or statistics: FeTrIL, ACIL, SimpleCIL, iCaRL, and AdvMIL.

Q5: How many classes have fewer than KK training slides, so that a stored centroid is a single slide embedding?

The smallest training class in the CHTN curricula has 24 slides (Metastatic site); the minima for the other curricula are 30 (four curricula) and 254 (Table 15). The cap Kc=min⁡(K,nc)K_c=\min(K,n_c), with ncn_c the number of training slides of class cc, therefore does not bind at K=8K=8. At this setting, the smallest classes (24–30 slides) average only three to four slides per centroid. The cap binds only in the analysis sweep on Cancer family (Fig. 3), from K=256K=256 upward: the smallest class has 254 training slides, so each of its slides becomes its own centroid. For the cross-cohort benchmark, the manuscript reports per-cohort counts in Fig. 32.

W1: Zero drift under a frozen map holds by construction; the decisive controls are missing; the contribution reduces to frozen features with class-centroid replay (SimpleCIL, FeTrIL, PASS).

The revision clarifies the scope of our contribution. Zero drift under a fixed map holds by construction (Fig. 1, Sec. 6). Our approach builds on established components (Sec. 2). The ablation shows that a fixed map suffices for effective rehearsal (Secs. 1 and 7).

The revision reports fixed-map controls on three curricula (Table 10, seed-matched differences): mean pooling over the PC-108 projection is within 1.8 points of PACE, and a parameter-free map (identity projection, mean pooling) is 3.5–5.7 points better on all three. These controls show that simpler fixed maps can achieve comparable or better performance than the PC-108 attention map. Compact rehearsal also achieves higher AACC with a fixed map than with the same map trained throughout (14.1–28.3 points at λproto=10\lambda_{\mathrm{proto}}=10 in every matched pair; Tables 12–14). These findings support applying frozen-feature class-incremental learning to WSI-MIL with a fixed bag-to-slide map.

PACE's mean AACC exceeds that of SimpleCIL and FeTrIL on all eight CHTN curricula (mean differences +16.2 and +5.3 points; signed-rank p=0.008p=0.008 each; Table 9) and that of PASS on all eight (Table 1). SimpleCIL, FeTrIL, and ACIL freeze their aggregator after the first task. The comparisons with SimpleCIL and FeTrIL therefore assess PACE's multi-centroid memory and rehearsal through the head relative to single-mean approaches within a fixed-map setting. PACE's advantage over SimpleCIL and FeTrIL is associated with improved learning of new classes rather than reduced forgetting (Sec. 6.1).

Baseline tuning is addressed under Q4. The revised discussion also outlines directions for future evaluation: iCaRL, ER-ACE, DER++, and BiC under the same fixed map; RanPAC, FeCAM, and covariance replay; slide-level encoders such as TITAN; drift-compensated trainable aggregators to further assess the role of map fixation; and the baselines under the parameter-free map (Sec. 7).

W2: The ablation behind RQ2–RQ3 and the “82%” claim covers a single curriculum and a single replay weight.

The revision extends the regime ablation to three additional curricula—Frequent-first, Metastatic site, and Diagnostic triage (K∈{1,4,8}K\in\lbrace1,4,8\rbrace, λproto∈{0,1,10}\lambda_{\mathrm{proto}}\in\lbrace0,1,10\rbrace; comparing the fixed PC-108 map with the same map trained throughout (Tables 12–14) and scopes the 82% claim. Without rehearsal, both regimes remain at the fine-tuning level (AACC 0.09–0.18).With rehearsal, the fixed map achieves higher AACC in every matched pair, by 14.1–28.3 points at λproto=10\lambda_{\mathrm{proto}}=10 and by 18.4–39.8 points at λproto=1\lambda_{\mathrm{proto}}=1. Rehearsal also benefits the trained map, whose best AACC values are 0.493, 0.495, and 0.438, respectively, while the fixed map yields larger gains.

Relative to the mean no-rehearsal control, one prototype recovers 62–93% of the K=8K=8 gain at λproto=1\lambda_{\mathrm{proto}}=1 and 89–99% of the K=4K=4 gain at λproto=10\lambda_{\mathrm{proto}}=10. On Cancer family, the sweep covers eleven replay weights (Figs. 3 and 4). The 82% result holds at λproto=10\lambda_{\mathrm{proto}}=10 on that curriculum, and the abstract now explicitly limits this claim to one curriculum.

AuthorsOpenReview ↗

Gentle Reminder: Summary of Revisions and a Request for Score Reconsideration

Thank you for the time and care you invested in your detailed review. As we approach the end of the discussion period, we summarize the revisions made in response to your main concerns:

  • Contribution and controls (W1). PACE is now framed as building on established components, with zero drift holding by construction. Fixed-map controls (Table 10) show that simpler maps, including mean pooling, perform comparably or better, so the title, abstract, and contributions were reframed around a fixed aggregation map. PACE's mean AACC exceeds that of SimpleCIL, FeTrIL, and PASS on all eight CHTN curricula (Tables 1 and 9). Further suggested comparisons, such as replay baselines under the same fixed map, are outlined as future work (Sec. 7).
  • Ablation scope (W2). The fixed-versus-trained ablation now spans three further curricula (Tables 12–14), with the fixed map at least 14 points higher in every matched pair. The 82% result is reported for one curriculum.
  • Selection and statistics (Q1, Q2). Selection uses validation AACC only. Table 9 adds 95% intervals and a signed-rank test across curricula, and the abstract reports the leads accordingly.
  • Survival extension (Q3). The extension uses the frozen PC-108 prior for UNI-v2 features. We trace its ordering variation to run-to-run randomness, verified by seeded re-runs (Table 28), and present the result as comparable performance at zero forgetting.
  • Baselines and class sizes (Q4, Q5). Baseline settings and tuning are documented (Sec. B.3), Sec. 6.1 explains why shared-head methods without old-class inputs behave like fine-tuning, and the prototype cap does not bind at K = 8 (Table 15).

If these revisions address your main concerns, we would appreciate your reconsidering your overall assessment and score. Please let us know if any concerns remain.

Comments on the paper

1 comment
Area_Chair_Pwa9OpenReview ↗

A Gentle Nudge: Let's Keep the Discussion Going!

Dear Reviewers,

The authors have now posted their responses, addressing your concerns and incorporating your suggestions. We are already halfway through the discussion period!

One of the distinguishing features of ICLR is its interactive review process, where constructive discussions not only help strengthen individual papers but also contribute valuable insights to the broader scientific community.

So, let's not let the discussion be a one-way street! I encourage you to revisit the authors' responses, acknowledge their efforts, and engage in further discussion where needed. After all, a good scientific discussion is often as valuable as the final verdict!

-AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.