acceptodds
Under review as a conference paper at ICLR 2027

ProtoCIL: Class-Incremental Learning for Protein Representations with Adaptive Prototypes

Abstract

Protein function annotations continually expand as new enzyme classes are discovered, requiring models to incorporate new classes without retraining from scratch. Yet class-incremental learning for protein function prediction remains underexplored, especially under frozen protein encoders with highly heterogeneous embedding geometries. We present the first systematic benchmark for protein class-incremental learning, covering two enzyme datasets, five frozen protein encoders, and eleven representative CIL baselines. Our benchmark reveals that no fixed-granularity approach, whether shared covariance, random projection, or learned subspace, scales reliably across encoders and class counts. We propose ProtoCIL, a training-free prototype method that adaptively learns group-wise covariance structure over frozen protein embeddings for Mahalanobis classification. ProtoCIL derives its target group count from intrinsic rank and sample size, then refines groups incrementally through validation-based splits as new classes arrive. Across all encoder-dataset configurations, ProtoCIL achieves the highest average incremental accuracy, outperforming the strongest baseline by 8.8% on Enzyme Reaction and 18.5% on EC Number, while maintaining average forgetting below 5%. These results demonstrate that adaptive group-wise covariance sharing provides an effective principle for protein class-incremental learning. Code is provided for review and will be released upon acceptance.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein Representation Learning, Prototype-Based Learning, Class-Incremental Learning, Catastrophic Forgetting, Covariance Modeling

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LFspOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a benchmark for class-incremental protein function prediction under frozen pretrained encoders and proposes ProtoCIL, a training-free prototype classifier. The benchmark covers two enzyme datasets, five sequence/structure encoders, and eleven CIL baselines. ProtoCIL maintains class prototypes while sharing covariance estimates among adaptively formed class groups. Its target number of groups is derived from effective embedding rank and sample size; groups are refined incrementally using held-out validation samples, Ledoit–Wolf shrinkage, and a Mahalanobis classifier with log-determinant normalization. Across the ten encoder–dataset combinations, ProtoCIL reports the best average incremental accuracy (74.3% on Enzyme Reaction and 91.8% on EC Number) and low forgetting (3.8% and 1.8%). Ablations, alternative phase constructions, and efficiency measurements are also provided.

Critical strengths
  1. The benchmark is broad and useful: it evaluates multiple CIL paradigms across five heterogeneous protein encoders, two class scales, and several phase constructions. The consistent advantage across all encoder–dataset combinations makes the empirical result more convincing than a single-backbone demonstration.

  2. The method is technically coherent and well motivated for the sample-starved frozen-feature regime. Adaptive covariance sharing, shrinkage, and log-determinant normalization directly address the instability of class-wise covariance estimates. The component ablations and grouping-strategy comparisons substantiate the design choices.

Critical weaknesses
  1. The biological validity of the evaluation is not yet sufficiently established. Both datasets are originally multi-label, but the benchmark converts them to single-label problems by argmax/first annotation. More importantly, the paper analyzes sequence identity across phases, but does not report a homology-controlled train/test split. Cross-phase dissimilarity does not rule out highly homologous train/test proteins within the same class. The near-99.5% joint-MLP accuracy on the 4,788-class EC dataset makes this concern especially important. If train/test near-neighbors are present, the very large gains may not reflect generalization to genuinely novel proteins.

  2. The resource and comparison accounting needs clarification. ProtoCIL reserves 10% of every class for repeated split validation and stores dense group covariances, while most baselines use globally fixed hyperparameters and only a subset receive per-encoder tuning. It is unclear whether the reported storage includes retained validation embeddings and whether all competitors receive comparable validation and memory budgets. This matters to the claims of a practical, tuning-free continual learner.

Questions for authors
  1. What is the maximum sequence identity between training and test proteins, and how does ProtoCIL perform under a cluster-based homology split (e.g., 30% or 50% identity)? This result would materially affect my confidence in the benchmark.

  2. Does the reported storage include all held-out validation embeddings? Must old validation samples remain accessible after each phase, or can they be discarded? Please compare against baselines under matched total memory and validation budgets.

  3. How robust are the conclusions when preserving the original multi-label annotations and evaluating with appropriate multi-label metrics?

1 comment

We thank the reviewer for the careful review. All revisions are included in the revised PDF, and new text and new results are highlighted in blue.

Q1 (homology and the high joint accuracy).

The reviewer's concern was justified. The training and test sets we built from CLEAN's data overlapped, so some test proteins had already been seen in training. This is why joint accuracy was so high. We rebuilt the split by removing from training every protein that also appears in the test set (64,161 train, 8,106 test, 3,932 classes). All EC Number results in the revised paper use this new split, and the Joint MLP now reaches 68.9%.

On the new split, the maximum sequence identity of each test protein to the training set (MMseqs2) is:

Max identity to training set≥30%≥50%≥70%≥90%
Same phase86.8%44.4%2.8%0.3%
Any phase87.4%44.7%2.8%0.3%

The reviewer's concern is about close homologs within the same class. A class always belongs to one phase, so the Same phase row covers this case, and there only 2.8% of test proteins have a training protein with 70% or more identity. Following the reviewer, we also evaluated two homology-controlled test sets that keep only the test proteins whose maximum identity to any training protein is below 50% or 30% (4,482 and 1,018 of the 8,106 test proteins). Average accuracy (%), mean over the five encoders and 3 seeds, including Deep SLDA and KLDA, two baselines we added following Reviewer yQH4:

SplitProtoCILDeep SLDAStrongest other baselineJoint MLP
New split72.965.368.5 (KLDA)68.9
Below 50% identity62.355.356.5 (KLDA)55.2
Below 30% identity31.029.926.7 (KLDA)23.7

The 30% test set contains only proteins without close homologs in training, so all methods drop sharply. ProtoCIL remains the most accurate on average and stays well above joint training.

Q2 (storage, held-out samples and matched budgets).

The storage of ProtoCIL in the revised Table 2 includes the frozen embeddings of past training samples, about 313 MB on EC Number with ESM2 650M (64,161 embeddings of dimension 1,280 in float32), so ProtoCIL needs 557.5 MB in total, still below the 879.5 MB of RanPAC.

Old held-out samples must remain accessible under the current design. When a group that contains classes from earlier phases is considered for a split, candidates are scored on the held-out samples of all classes in the group, and the final prototypes and covariances are refit on all training samples. ProtoCIL stores no raw proteins and performs no replay training or gradient updates.

On matched memory, the Joint MLP keeps the same embeddings as ProtoCIL and retrains on them at every phase. Under the efficiency setting of Table 2, ProtoCIL reaches 80.0% against 71.5% for the Joint MLP and is also faster, 212.8 s against 220.5 s. In the main experiments, ProtoCIL is more accurate than it on both datasets, 74.9% against 66.4% on Enzyme Reaction and 72.9% against 68.9% on EC Number.

On validation budgets, ProtoCIL uses the same fixed settings for every encoder, and its held-out samples are used only for split decisions, not for tuning any hyperparameter. Baselines given per-encoder hyperparameter tuning reach at most 71.3% on Enzyme Reaction and 60.0% on EC Number, both below ProtoCIL without tuning.

Q3 (multi-label annotations).

We have not evaluated multi-label metrics, so we cannot claim robustness under them. Two facts limit the effect of the single-label conversion. First, more than 88% of proteins in every split carry a single EC annotation, so the conversion changes the label of a small fraction of proteins only. Second, the retained label is always one of the protein's true labels, and the same conversion is applied to every method. A prediction of another true label is counted as wrong, so the single-label accuracy is a lower bound on any-label accuracy for all methods alike. Single-label prediction is also the standard setting for class-incremental learning, while multi-label class-incremental learning is a different problem that we leave to future work.

Reviewer_SXFaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies class-incremental learning for protein function prediction when the protein encoder is held frozen. It makes two contributions. The first is a benchmark: two enzyme datasets, five frozen encoders spanning sequence, structure and fusion architectures, and eleven methods, reported as 110 evaluation configurations. The second is ProtoCIL, a gradient free prototype classifier that partitions classes into covariance groups, derives the number of groups from shrinkage theory and sample size, and refines the groups as new classes arrive. ProtoCIL reports the best average accuracy on every encoder and dataset combination, with 8.8 and 18.5 points over the strongest baseline on Enzyme Reaction and EC Number respectively, and average forgetting below 5. The diagnostic that motivates the method is the strongest part of the paper. Once the encoder is frozen, gradient based stability mechanisms stop helping, and no fixed covariance granularity survives the range of embedding geometries in use. My concerns are about how the problem is framed, and about controls on the label space that the paper does not report.

Critical strengths
  1. The experimental grid is substantial and it is controlled. Five encoders, two datasets, three seeds, and hyperparameters fixed across every configuration instead of tuned per setting. The authors also verify that this does not disadvantage the baselines by tuning RanPAC, FeCAM and DER++ per encoder in Appendix D. Most class incremental papers do not do this, and it is what makes the negative result credible.

  2. The negative result is clean. EWC gives inconsistent gains over the Naive lower bound, and RER matches vanilla replay almost exactly. The explanation, that gradients flowing only through a small head carry little stability signal, is simple and fits the numbers.

  3. ProtoCIL follows directly from the diagnosis. Deriving the group count from intrinsic rank and sample size, rather than fixing it in advance, is the right response to the failure mode the benchmark exposes, and the method needs no per encoder tuning. The ablation separating global, per class and grouped covariance supports the claim that granularity is the relevant axis.

  4. The four alternative phase constructions in Appendix D are more informative than the main table. Random, finer EC, mixed hierarchy and temporal protocols, with the conclusion holding across all of them, is the robustness check that this literature usually skips.

  5. Table 2 is a sensible addition. If the selling point is that no backpropagation is needed, then wall clock time and memory next to accuracy is the right way to argue it.

Critical weaknesses

W1. The class definition is never stated. The paper says "enzyme class" throughout, but the objects are individual EC numbers, and 4,788 implies the fourth and finest level of the tree. In enzymology the word class refers to the seven top categories. The reader has to infer the granularity from the class count alone, and the growth statistics in the introduction only make sense once that inference is made. For a paper whose central object is the label space, this should be stated once and used consistently.

W2. The problem is not what the paper says it is. Every class is defined in advance. Each phase hands the model a batch of new labels together with their training samples. Nothing is discovered, nothing is unlabeled, and the encoder never moves. What is actually learned is how to distribute covariance templates over a fixed label set, which is a static supervised classification problem evaluated in stages. The word incremental is carrying a lot of weight here. The motivating numbers do not help. Roughly 130 EC numbers are assigned per year against 4,788 classes, so the label space grows at 2.7 percent, and the additions cluster in a few branches rather than arriving in comparable blocks. The main protocol cuts phases by EC first digit, which resembles no arrival process. The temporal protocol in Table 12 is the only construction that matches the story the introduction tells, and it sits in the appendix.

W3. The label space also shrinks and renumbers. Class incremental learning assumes classes are only added. The EC list transfers and deletes entries routinely. In the 2024 supplement at least 5 of roughly 126 entries are transfers or deletions, for example strictosidine synthase moving from EC 4.3.3.2 to EC 3.5.99.13 after its mechanism was reclassified. For such a class the model needs relabeling rather than learning, and the benchmark instead splits one distribution into two and penalizes every method for it. Deletion and merging are never tested.

W4. Controls on the data split are missing. The identity control covers cross phase pairs only. At 4,788 classes with about 15 training samples each, a frozen encoder plus an MLP reaches 93.3 percent joint accuracy, which is hard to read without a within phase control. The dataset sizes are also unexplained: Table 7 reports 348 classes and 16,983 proteins for Enzyme Reaction against 384 and 37,428 in the cited source. Classes are kept at 91 percent and proteins at 45 percent, so a single filter cannot account for both, and the table note describes these as sequence encoder splits while the text attributes the loss to missing structures.

W5. Table 6 undercuts the title claim. Random grouping reaches 90.2 on EC Number against 91.8 for the proposed method, with encoder dispersion of 12.3, and taxonomy grouping lands at 83.6, which is exactly the Global pooled number in Table 4. Read together, these say that having several groups matters and choosing them does not, at least at this scale. The number of groups each strategy actually reaches is never reported, so it is also unclear whether taxonomy grouping was executed or collapsed to a single group.

W6. The partition is irreversible and the selection signal is thin. Groups never release members, so an early split is permanent. Under the main protocol the first phase holds a single top level category, and phase order is shuffled per seed, which means the initial task size likely varies across seeds by an order of magnitude. Neither the per phase class counts nor the variance of the final structure across phase orders is reported. Separately, the split search selects among up to 3k candidate partitions using 10 percent held out samples, which is about 1.5 samples per class at 4,788 classes.

Questions for authors

Q1. Which EC level are the classes? Please name it, give the count at each of the four levels in both datasets after filtering, and replace "class" with "EC number" where that is what is meant.

Q2. If every new class arrives with its labels and training samples, what does class incremental learning buy over periodic refitting of the head? And if the motivation is growth over time, why is the by digit construction the primary protocol rather than the temporal one in Table 12?

Q3. How many classes in the temporal protocol are renumberings of activities that already existed under a previous EC number, and do the reported gains and forgetting rates survive when those classes are excluded?

Q4. What is the within phase train test identity distribution on EC Number, and what happens to the joint and ProtoCIL accuracies under a hard identity cutoff? Also, please give the per class table behind the move from 384 classes and 37,428 proteins to 348 and 16,983.

Q5. What is the number of groups reached by each strategy in Table 6? Random grouping is within 1.6 points of the proposed method on EC Number, and taxonomy grouping equals the single covariance baseline. Does the claim survive if the group count is fixed and the assignment is random?

Q6. How much does the final group structure change when the phase order is permuted, and what are the per phase class counts? Also, how often does the selected split change across resampled held out sets, given roughly 1.5 samples per class to choose among up to 3k candidates?

6 comments

We thank the reviewer for the detailed review. All revisions are included in the revised PDF, and new text and new results are highlighted in blue.

Ethics concern. The NeurIPS 2026 workshop is non-archival, and ICLR 2027 allows prior presentation at workshops without proceedings. The workshop paper was accepted and made public in October 2026, after the ICLR deadline, so it could not be cited at submission. The two papers differ in their contribution. The workshop paper is a benchmark study that proposes no method and calls for encoder-aware methods. This submission shares its benchmark setup and contributes such a method, ProtoCIL, whose adaptive grouping, target group count and validated splitting are all new. We now cite it in the revised PDF.

W1/Q1.

We agree this should be explicit. In EC Number every class is a full 4th-level EC number. Enzyme Reaction follows the DeepFRI label set, which mixes levels: each protein keeps one label, which is a 4th-level number for 193 classes and a 3rd-level number (e.g. 3.4.21.-) for 155 classes. Counts after filtering:

DatasetClassesLevel 1Level 2Level 3Level 4 (full EC number)
EC Number (new split)3,9327702453,932
Enzyme Reaction348755161193 (+155 at level 3)

We keep "class" in the class-incremental sense of one label, because Enzyme Reaction also contains third-level labels, so "EC number" would not be accurate for every class. We now state the label level in Appendix B.1.

W2/Q2.

We respectfully disagree that the setting is static classification in stages. Class-incremental learning is defined by updating the model with only the current phase's data and then classifying among all seen classes, as in split CIFAR-100. This constraint is what makes the problem hard, because a model that sees only the current phase forgets earlier classes. An MLP trained this way reaches 11.6% and 16.1% average accuracy on Enzyme Reaction and EC Number, while the same MLP trained on all data at once reaches 66.4% and 68.9%.

Periodic refitting retrains the head on all data seen so far after every release. ProtoCIL instead adds new classes by computing their statistics, without any gradient training, and it is still more accurate than the refitted head, with 74.9% and 72.9% against the 66.4% and 68.9% of the joint MLP above.

Following the reviewer, we report the temporal protocol with all baselines for every encoder in Table 14 of Appendix D, where all methods use exactly the same phases. ProtoCIL achieves the highest accuracy for every encoder on both datasets, 1.8% to 7.0% above the strongest baseline. Average accuracy (%), mean over 3 seeds:

MethodER 650MER 150MER GearNetER SaProtER ProtBertEC 650MEC 150MEC GearNetEC SaProtEC ProtBert
ProtoCIL79.575.361.574.771.381.776.955.476.973.6
KLDA75.372.454.570.767.277.374.052.073.667.5
Deep SLDA67.563.042.070.267.472.973.446.275.168.1
RanPAC74.069.223.171.065.770.765.223.165.158.0
FeCAM (orig.)59.645.734.156.135.374.467.941.166.151.7
FeCAM (ridge)71.064.640.867.567.643.940.325.034.517.3
DER++57.449.834.553.747.853.345.325.345.346.3

ER: Enzyme Reaction, EC: EC Number, 650M and 150M: ESM2.

We keep the by-digit protocol as a harder stress test, since entire top-level categories are absent from earlier phases. ProtoCIL ranks first under both protocols, so the conclusions do not depend on which one is primary.

W3/Q3.

Using the ExplorEnz history, 533 of the 3,932 EC numbers (13.6%) are renumberings of earlier activities. In our temporal protocol, a renumbered EC number is placed at the year its enzyme was first listed, not the year it received its current number. For example, EC 1.1.1.203 was created in 1972 as EC 1.2.1.35 and renumbered in 1984, so it is placed in 1972. A renumbered enzyme is therefore not treated as a new class, and only two of them have their old number also present as a label.

Excluding all 533, ProtoCIL remains the best on both metrics, averaged over the five encoders:

MethodAvg Acc (%)Forgetting (%)
ProtoCIL74.82.7
KLDA70.63.0
Deep SLDA68.75.5
FeCAM (orig.)61.23.8
RanPAC59.34.4
DER++50.523.8
FeCAM (ridge)33.06.9

Deletion and merging change the label space rather than extend it. They define a different problem from class-incremental learning and are outside the scope of this work. However, it is an inspiring direction and we will list it as future work.

W4/Q4.

We thank the reviewer for these checks. The training and test sets we built from CLEAN's data overlapped, so some test proteins had already been seen in training, which is why joint accuracy was so high. We rebuilt the split by removing from training every protein that also appears in the test set (64,161 train, 8,106 test, 3,932 classes), and all EC Number results in the revised paper use this new split.

On the new split, the maximum identity of each test protein to the training set is:

Max identity to training set≥30%≥50%≥70%≥90%
Same phase86.8%44.4%2.8%0.3%
Any phase87.4%44.7%2.8%0.3%

We also built two stricter splits that remove every training protein above 50% or 30% identity to any test protein. Average accuracy (%):

SplitProtoCILDeep SLDAStrongest other baselineJoint MLP
New split72.965.368.5 (KLDA)68.9
Below 50% identity62.355.356.5 (KLDA)55.2
Below 30% identity31.029.926.7 (KLDA)23.7

All methods drop sharply at 30%, where almost no close homologs remain, but ProtoCIL stays the most accurate on every split (Appendix D, Table 11).

On Enzyme Reaction, the citation was wrong, and the 384 classes and 37,428 proteins belong to another dataset. Our data are the DeepFRI EC dataset (Gligorijević et al., 2021) from TorchDrug, used identically by all methods. Its 15,550 training and 1,919 test chains become 15,128 and 1,855 after the TorchDrug loader, and keeping one label per protein turns 538 EC terms into 348 labels. Table 7 counts the sequence-encoder splits, and only the structure-based encoders also drop proteins without structures. Details are in Appendix B.1, and the per-class table is long, so we can post it as a comment on request.

W5/Q5.

Random and k-means in Table 6 already use ProtoCIL's target group count, so they are the control you describe. Accuracy (%) on the new split, groups at seed 0 in brackets:

StrategyESM2 650MESM2 150MGearNetSaProtProtBertAvg
ProtoCIL80.0 [36]76.9 [73]57.0 [1]77.8 [10]72.8 [3]72.9
Random, same G80.5 [36]76.2 [75]30.5 [16]76.0 [40]66.9 [31]66.0
K-means, same G79.4 [36]75.3 [75]31.8 [16]73.2 [40]65.7 [31]65.1
EC hierarchy77.2 [1]73.1 [1]57.0 [1]77.1 [1]72.9 [1]71.4

Taxonomy grouping was executed, but every split along the EC hierarchy was rejected on the held-out set, so it stays at one group and equals ProtoCIL with one shared covariance.

With the group count fixed and the assignment random, ProtoCIL leads by 6.9% on average, and the encoder dispersion shows why. On the ESM2 encoders many groups help, and random assignment with the same count does about as well. On GearNet and ProtBert splitting hurts, so ProtoCIL keeps 1 and 3 groups, while forcing the target count costs 27% and 6%. The main benefit of ProtoCIL is therefore deciding how many groups each encoder needs, which ranges from 1 to 75, without tuning. We clarified this in Sections 4.3 and 5.

W6/Q6.

Can an early wrong split hurt later? In most settings we see no lasting harm. A split is made only when it improves held-out accuracy (for GearNet every split was rejected), a group can still be split further later, and changing the phase order, which changes which splits happen early, barely changes final accuracy (below). We do not merge groups, because re-partitioning would change the covariances old classes rely on and cause forgetting.

Per phase class counts. On the new EC Number split, the phases hold 1,111, 1,189, 764, 444, 194, 159 and 71 classes (EC classes 1 to 7), so the first task does vary by over an order of magnitude.

How much does the structure change with phase order? Across three phase orders:

ESM2 650MESM2 150MGearNetSaProtProtBert
Groups (3 orders)33 to 3673 to 7519 to 123 to 31
Accuracy std (%)0.20.20.00.51.7

The class assignment itself changes a lot (adjusted Rand index 0.06 to 0.18 on Enzyme Reaction), so many partitions perform equally well, and what matters is the number of groups, which is stable.

How often does the selected split change with a resampled held-out set? Over 10 resamples (ESM2 650M, EC Number), the number of groups was 36 in 9 runs, the assignment changed each time, and the accuracy std was 0.5%.

Is the selection signal too thin? In Section 4.3, "3k" means 3 × k with k ≤ 5, so each split decision compares at most 15 candidates, not thousands. Each candidate is scored on the held-out samples of all classes in the group together, not about 1.5 samples per class, and the final prototypes and covariances are refit on all training samples.

We added these stability results to Appendix D.

Thanks for the detailed response. Several points are settled and I will not raise them again: the label level and per-level counts (Q1); the 533 renumberings at 13.6 percent and the exclusion experiment (Q3); the overlap between train and test in the original split, its repair, and the identity-stratified results, which is the most valuable part of the revision (Q4); the per-strategy group counts (Q5); and the phase-order stability (Q6). I also accept that I misread "3k" as three thousand, though the notation should be written out explicitly in the text.

Some points remain unresolved.

R1 (Q2, decisive). ProtoCIL reaches 74.9 and 72.9 while the jointly trained MLP reaches 66.4 and 68.9, and forgetting stays under 5. Read together, these suggest the incremental constraint is not the binding one, and that the gain comes from the classifier family rather than from any stability mechanism. Please report the tuning budget behind the joint MLP, and add a control in which the same grouped-covariance head is fitted on the full dataset at once. If that control is at or above 74.9, the CIL framing needs to be re-argued rather than defended by definition.

R2 (Q4, follow-up). Absolute accuracy falls from 72.9 to 62.3 to 31.0 as the identity cutoff tightens. What I need is the gap to the strongest baseline on each of the three splits, not the absolute number. If the gap shrinks with homology, the main table is still partly carried by homolog leakage.

R3 (Q5, scope). At matched group count, random assignment is 80.5 against 80.0 on ESM2 650M, and on GearNet ProtoCIL selects a single group, that is, no partition. The evidence supports "deciding how many groups" rather than "adaptive grouping". Please reflect this in the title and abstract, or show one configuration where the partition itself, at matched group count, is worth more than 1 point.

R5 (Q6, provenance). Where does the held-out set used for split selection come from: the current phase only, or across phases? If it spans future phases, selection is using information that the sequential setting is supposed to withhold. Separately, please post the per-class table you offered rather than offering to post it.

Reviewer_yQH4OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper builds a class-incremental learning benchmark for protein function prediction with frozen encoders and proposes ProtoCIL, a training-free prototype method. The main claim of ProtoCIL is that adaptive group-wise covariance sharing beats all baselines in average accuracy, by 8.8% on Enzyme Reaction and 18.5% on EC Number, with average forgetting below 5%.

Critical strengths

S1. The benchmark fills a real gap. It covers sequence, structure and fusion encoders with very different dimensionality, and the temporal protocol ordered by EC creation year (Table 12) makes it a useful testbed beyond this one method.

S2. ProtoCIL is simple, needs no backpropagation, and gives strong accuracy with very low forgetting across encoders and phase protocols.

S3. Extensive efforts in ablation studies: per-encoder tuning for RanPAC, FeCAM (orig.) and DER++ (Table 16), threshold sensitivity (Tables 3 and 15), and the cross-phase sequence identity check in Appendix B.5.

Critical weaknesses

W1. The Global pooled row of Table 4 (one shrunk shared covariance) reaches 72.1 and 83.6, already above the best baseline averages in Table 1 (65.5 and 73.3), and adaptive grouping adds 2.2% and 8.2% on top. FeCAM (ridge) uses a fixed ridge of 1e-4 and is not tuned in Table 16, so the baseline set has no tuned shrunk shared-covariance head of the kind used in Deep SLDA (Hayes and Kanan [1]), the LDA head of Panos et al. [2], or KLDA (Momeni et al. [3]). None of these is cited. In my view the experiments do not yet separate the contribution of adaptive grouping from that of shrinkage on a shared covariance, which is the comparison that matters most for the method's novelty.

W2 In section 4.3, validated splitting at later phases scores splits of groups holding old classes on a held-out subset. A split also changes old-class covariances. So the "no replay buffer" and "zero modification" claims (Section 4.2) would benefit from qualification.

[1] Hayes and Kanan, 2020, "Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis", CVPR Workshops [2] Panos, Kobe, Olmeda Reino, Aljundi, and Turner, 2023, "First Session Adaptation: A Strong Replay-Free Baseline for Class-Incremental Learning", ICCV [3] Momeni, Mazumder, and Liu, 2025, "Continual Learning Using a Kernel-Based Method Over Foundation Models", AAAI

Questions for authors

Q1. Table 4 reports that Global pooled, a single shrunk shared covariance with a Mahalanobis rule, already reaches 72.1 and 83.6. Could you clarify the novelty of ProtoCIL relative to Deep SLDA (Hayes and Kanan [1]), which also keeps class means and a shrinkage-regularized shared covariance over a frozen backbone, ideally by reporting Deep SLDA on your benchmark next to Global pooled and ProtoCIL?

Q2. Section 4.3 scores candidate splits on a held-out subset. Could you clarify whether splitting a group at phase t uses held-out samples of classes from earlier phases, and if so, how many embeddings ProtoCIL stores for this?

3 comments

We thank the reviewer for the careful review and the pointers to related work. All revisions are included in the revised PDF, and new text and new results are highlighted in blue. EC Number results now use a rebuilt train and test split (see our response to Reviewer SXFa, W4), so some numbers differ from the submission.

Ethics flag. Please see our response to Reviewer SXFa.

W1/Q1.

We added Deep SLDA (Hayes and Kanan, 2020) and KLDA (Momeni et al., 2025) as baselines and now cite all three works. Both are implemented as in the original papers. Deep SLDA keeps class means and one shared covariance updated sample by sample, with the fixed regularizer of the original paper. The LDA head of Panos et al. (2023) also uses class means and one shared covariance, so Deep SLDA represents it. KLDA applies an LDA head over 5,000 random Fourier features. Average accuracy (%), mean over the five encoders and 3 seeds:

MethodEnzyme ReactionEC Number
ProtoCIL74.972.9
Global pooled (ProtoCIL with one group)72.671.4
KLDA69.668.5
Deep SLDA64.265.3
Strongest other baseline65.5 (FeCAM ridge)58.3 (FeCAM orig.)

ProtoCIL is the most accurate method for every encoder on both datasets, ahead of the strongest baseline by 2.8% to 7.5% per encoder.

Global pooled answers this question directly, since it is ProtoCIL with a single group that keeps the same shrinkage and the same classifier but drops the grouping. Shrinkage clearly matters, because Global pooled alone reaches 72.6% and 71.4%, which is above Deep SLDA and KLDA with their fixed small regularizer or no shrinkage at all, and removing shrinkage from ProtoCIL costs 15.4% and 7.9%. Grouping then adds on top of shrinkage, so ProtoCIL is 2.3% and 1.5% above Global pooled on average and up to 7.4% and 3.8% on ESM2 150M, while on GearNet ProtoCIL keeps a single group and therefore matches Global pooled exactly.

So the novelty of ProtoCIL relative to Deep SLDA is not shrinkage of a shared covariance but choosing the covariance granularity for each encoder. ProtoCIL derives a target group count from effective rank and sample size, and accepts a split only when it improves held-out accuracy. A single shared covariance is the special case of one group, which ProtoCIL recovers on its own when grouping does not help. The best number of groups ranges from 1 to 75 across encoders, so no fixed choice works for all of them, and ProtoCIL finds it without per-encoder tuning.

W2/Q2.

Yes. When a group that contains classes from earlier phases is considered for a split at phase t, each candidate partition is scored on the held-out samples of all classes in the group, including the earlier ones. To support these decisions and the final refit, ProtoCIL keeps the frozen embeddings of past training samples. On EC Number with ESM2 650M this is 64,161 embeddings of dimension 1,280, about 313 MB in float32 (74 MB on Enzyme Reaction). It stores no raw proteins and performs no replay training or gradient updates.

We agree the claims needed qualification and revised Section 4.2. Old class prototypes never change. A group's covariance changes only when new classes join it or when a validated split refines it, and a split is accepted only if it improves held-out accuracy over all classes in the group, so it does not trade old-class accuracy for new classes. The stored embeddings are now described explicitly as a form of feature storage. Forgetting stays low, 4.3% on Enzyme Reaction and 2.6% on EC Number on average.

Thank you for your detailed response, which has addressed my concerns. As my original rating was already positive, I will maintain my current rating.

We thank the reviewer for the positive assessment and constructive feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.