acceptodds
Under review as a conference paper at ICLR 2027

Continual Learning for Virtual Cell models

Abstract

Recent virtual cell or single-cell foundation models are typically pretrained on fixed datasets. However, with the growth of biological data, there is a pressing need to continually train such models on new data distributions without sacrificing performance on previously seen domains. We investigate this continual-learning problem in three existing models, scGPT, scDFM, and X-Cell, using a four-stage Perturb-seq benchmark. Across these backbones, naive fine-tuning loses 69–84% of old-domain performance. Continual training with 5% replay reduces but does not remove this forgetting. On X-Cell, larger replay budgets can reduce forgetting but improve new-domain learning only marginally, so retention trades off against acquisition. Motivated by interference in shared memory, we introduce Profile-Routed Residual Memory (PRM), residual product-key memory layers with a boundary-aware mechanism that separates read composition from write ownership, building on prior work on sparse product-key memory layers. This boundary-aware continual learning method requires observable context and awareness of training boundaries, but not task identifiers, replay of earlier perturbed examples, or distillation. Integrated into X-Cell, a recent single-cell perturbation model, PRM achieves a mean acquisition Pearson- of 0.575 with a final old-domain Pearson- of 0.583, corresponding to 0.6% relative forgetting and both retains and acquires more learning throughout the training campaign compared to other continual learning methods. On other model backbones, PRM shows minimal forgetting. These results provide initial evidence that asymmetric soft reads and private writes are a useful design principle for stable, plastic adaptation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Continual Learning, Memory Model, Single-cell omics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_spjBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3(+1)
Read review
Paper summary

The paper successfully demonstrates that the ‘profile routed residual memory’ method alleviated catastrophic forgetting during sequential training on four tasks.

The data are in the form (cells, gene activity) + intervention label and the model predicts activity under an intervention, the continual learning problem is training this model sequentially on new data.

I was not familiar with the virtual cell models used before reading this paper, much of the unique contribution seems to derive from creating a continual learning method specially designed for these models, trained on cell line data.

Critical strengths

The authors demonstrate that it does alleviate catastrophic forgetting for foundational virtual cell models. They carefully select which memory entries can be updated in their constructed PRM memory matrix, and this along with the sparse writes and protection of old writes, appears to be why it works.

The parts of the sparse memory mechanism are taken from elsewhere but much of it appears to be a new contribution.

Critical weaknesses

The method seems tremendously complex and my main concern is whether this complexity is justified.

PRM works by training sparse values in their memory matrix and then protecting them for future tasks. It took me a while to understand how this was injected into the transformer foundational models (equation 7).

This seems to be a classic continual learning problem using a pre-trained transformer, which is why a very complicated bespoke solution seems odd. Though they do compare against some classical baselines.

In the appendix table 4, it appears that performance on the pre-training data drops (if I've understood it correctly). This weakens the continual learning claim somewhat.

Questions for authors

One check I would like to see is if the careful selection of the memory entries are necessary or if you could just use a random sparse selection with the same number of writable entries. Is most of the gain coming from simply updating a small set of parameters and protecting earlier ones? I see some controls for a ‘frozen router’ but not this.

The drop in performance for X-Cell on the pre-training data seems to be a crucial result. Would you be able to see if this is the case for the other foundational models?

3 comments

Thank you for asking whether simpler protected sparse updates explain the gains and whether pretraining knowledge is retained. We added random-allocation and retention tests, and extended protection to pretraining controls in X-Cell. The corresponding manuscript changes are marked in blue.

Regarding Q1, Would random sparse allocation with the same number of protected writable entries work:

Matching the number of randomly reserved rows does not recover PRM's acquisition in this control. Across three campaigns, uniform sampling from all unprotected rows matches PRM's nominal allocation counts at every stage and table, but gives acquisition 0.1264 ± 0.0021 compared with PRM's 0.5752 ± 0.0032.

The router reached only 613 of the random arm's 260,869 reserved row instances during allocation probes, compared with all 260,869 for PRM. Equal nominal counts therefore left the random arm with far less capacity that the router actually used.

Whether ranking equally active rows adds a benefit remains open. The completed control supports PRM's use of routing information, because equal nominal sparsity did not provide equal access to trainable memory.

Regarding Q2 and W4, Does pretraining performance also decline for scDFM and scGPT:

Yes, the new scDFM evaluation also shows a substantial decline. Across 352 matched HCT116 training groups, perturbation-Δ Pearson falls from 0.4199 before sequential adaptation to 0.1219 afterward, an absolute drop of 0.2980. This single checkpoint pair uses identical group keys, the fixed 1,898-gene panel and the same sampling protocol on a training cohort.

A separate scGPT PRM rerun falls from 0.7121 to 0.0056 across 366 matched HCT116 training groups. This rerun uses a different checkpoint family from the submitted scGPT row, and the appendix labels it separately.

For X-Cell, extending protection to pretraining controls gives final cohort performance of 0.6170 ± 0.0052 across three seeds at the 90% route-mass target. Performance improves from 0.1463 for standard PRM to 0.6230, while sequential forgetting rises from 0.59% to 18.87%.

These results show that pretraining-cohort forgetting also occurs in the tested scDFM and scGPT campaigns. The X-Cell extension shows that adding protection for pretraining contexts can reduce this forgetting, albeit with a cost to sequential retention.

Regarding W1, Is the complexity justified:

The component experiments show why protection and calibration serve different roles within PRM. Ownership protects earlier memory values, while calibration restores access to rows that remain trainable.

Under full protection, K562 acquisition is 0.5310 with calibration versus 0.1143 with frozen routing in one run each, while permuted protected-risk information gives 0.1138 ± 0.0003 across three runs. The shared-checkpoint X-Cell controls also favor PRM over host fine-tuning and LoRA with 5% replay across three adaptation seeds, although progressive slots remain stronger on this benchmark.

These interventions justify calibration in the tested PRM design by linking it to recovered acquisition under protection.

Regarding W2, How does Equation (7) inject memory into the backbone:

Equation (7) adds a transformed memory read to the backbone hidden state through a residual connection. The current hidden state modulates how the retrieved information affects the prediction.

During response training, the host, router and base memory stay frozen while selected residual rows learn, and separate boundary calibration adjusts routing before it is frozen again. We now place this explanation before the equations. Equation (7) therefore lets learned memory change predictions without updating the backbone weights.

Regarding W3, Why use a specialized method for a classical continual-learning problem:

PRM addresses a general failure in sparse-memory adaptation, where new inputs retrieve protected entries they can no longer update. Its calibration rule uses current retrieval patterns and earlier ownership to restore writable access while anchoring earlier routes. We test this rule across three backbones, with gene identities and control profiles supplying the observable content and context.

We hypothesize that prompt embeddings or support-set summaries could provide context in other applications, provided earlier context representations remain available for anchoring. The methodological contribution is the coordination of routing and protected updates, with control cells providing a concrete biological implementation for this method.

Thank you again for highlighting pretraining retention. We are following up with additional completed experiments that test a remedy using PRM's existing protection and calibration mechanism.

Regarding Q2 and W4, Can PRM also protect pretraining performance:

Extending X-Cell PRM's ownership and anchoring to pretraining controls raises final Pearson-Δ from 0.1202 to 0.6220 on the same 860 held-out HCT116 groups, from an initial score of 0.6788. This provides direct evidence that the mechanism can protect more pretraining knowledge.

We then test the existing anchoring weight at a fixed starting checkpoint, memory size, gene panel and response-update budget. Each row is one adaptation campaign.

Anchoring weightAcquisitionSequential forgetting (%)Pretraining retention (Pearson-Δ)K562 acquisition
10.574821.220.62110.5494
20.577014.240.62950.5512
40.56776.520.62240.5126
80.54072.620.63270.4091

Pretraining retention is final Pearson-Δ on the same 860 held-out HCT116 groups, from an initial score of 0.6788.

Doubling the anchoring weight from 1 to 2 raises held-out Pearson-Δ from 0.6211 to 0.6295 and lowers sequential forgetting from 21.22% to 14.24%, while acquisition changes from 0.5748 to 0.5770. It improves both measured retention outcomes by tuning the existing objective, without adding a model component. The completed weight-8 evaluation gives held-out Pearson-Δ of 0.6327 and sequential forgetting of 2.62% in the same campaign, with lower acquisition of 0.5407.

The appendix subsection “Protection of pretraining knowledge” reports the pretraining gains, their cost to sequential retention, and the full sweep showing the acquisition trade-off at larger weights. First-round changes remain blue, and these second-round additions are red.

The new evidence addresses Q2 and W4 with an intervention that extends protection to pretraining knowledge and a calibration adjustment that improves retention. We would be grateful if you would reconsider your assessment and score in light of these additional results.

Thank you to the authors for the thorough response.

I now understand the protection of sparse updates a little better. However, it remains unclear to me whether the additional complexity of the procedure provides an advantage over simpler methods.

The substantial drop in performance on the pretraining data has been investigated more carefully, including experiments aimed at preserving pretraining performance. I consider this to have strengthened the paper’s soundness, and I have raised the soundness score to 3.

I still find the paper quite hard to understand, partly due to its complexity. The authors have attempted to address this and justify the need for the complexity, but it remains quite hard for me to assess. This is reflected in my low confidence score of 2.

Reviewer_iwfEOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies continual adaptation of virtual-cell models to sequentially arriving cellular contexts while preserving previously acquired perturbation knowledge. The authors propose Profile-Routed Residual Memory (PRM), which combines product-key memory with context-dependent routing, protected residual writes, and boundary-time routing calibration. Gene identity and unperturbed control-cell profiles determine memory retrieval, while ownership masks prevent subsequent stages from directly overwriting previously allocated memory rows. Calibration redirects incoming contexts toward writable memory without replaying earlier perturbed examples. Experiments evaluate PRM on scGPT, scDFM, and X-Cell using a four-stage Perturb-seq benchmark. The results show substantially reduced forgetting across the evaluated backbones, with particularly strong acquisition-retention performance on X-Cell, alongside ablations of memory protection, routing calibration, replay budgets, and boundary spillover.

Critical strengths
  1. The main methodological contribution is the distinction between allowing every context to access previously learned memory and restricting which memory values can be updated during subsequent training, which addresses two different requirements of continual learning: preserving earlier information while ensuring that new contexts retain access to trainable capacity. Equations (4)–(7) (pp. 4–5) define the product-key routing mechanism, where gene identity and observable control-cell profiles jointly determine memory retrieval, while perturbation-conditioned hidden states modulate the retrieved information. Equations (8)–(12) (pp. 5–6) introduce protected-risk calibration, writable-row allocation, and row-exact gradient masking, and these components provide an explicit mechanism for preventing direct overwriting while maintaining access to writable memory. Figure 4 and Table 7 (pp. 8, 18) provide direct evidence for the separate roles of protection and calibration: under full protection, a frozen router achieves only 0.1143 K562 acquisition, whereas progressive calibration achieves 0.5310, and a permuted-risk control achieves only 0.1138, supporting the importance of the actual protected-risk information. These experiments provide more than an aggregate performance comparison: they investigate why the individual components are necessary and demonstrate the importance of coordinating write protection with adaptive routing.
  2. The paper evaluates the proposed framework under sequential cellular-context changes rather than relying exclusively on final-model performance, and Section 3 (pp. 3–4) formally defines acquisition, retention, and relative forgetting, providing a consistent evaluation protocol for comparing performance immediately after learning each context and after subsequent updates. Table 1 and Figure 3 (p. 7) report results across scGPT, scDFM, and X-Cell; for example, scGPT PRM achieves 0.5735 acquisition with 0.01% relative forgetting, while X-Cell PRM achieves 0.5752 acquisition with 0.59% relative forgetting. Figure 5 and Table 8 (pp. 9, 18) compare different replay budgets and boundary-spillover configurations, providing additional evidence that PRM can maintain earlier sequential performance without retaining earlier perturbed examples. The evaluation provides useful evidence about the stability-plasticity trade-off in virtual-cell adaptation and offers a foundation for further continual-learning studies in this application domain.
Critical weaknesses
  1. Differences in pretraining initialization make the main cross-method comparisons difficult to interpret, because the experimental results do not consistently isolate the contribution of the proposed continual-learning mechanism from differences introduced during model and memory preparation. Appendix B (p. 13) states that X-Cell full fine-tuning and replay baselines start from a dense checkpoint containing 108.5M parameters, whereas PRM starts from a separately pretrained memory-augmented checkpoint containing 2.28B parameters, and its host, router, and base memory values are jointly pretrained for 10 epochs. Appendix B (p. 14) reports that scDFM PRM undergoes an additional six-epoch memory warm-start procedure, which changes held-out pretraining Pearson from 0.510 to 0.458, and moreover the clean scDFM baselines use 1,000 genes per training set, whereas PRM uses the fixed 1,898-gene panel. Table 1 (p. 7) reports substantial acquisition differences between PRM and conventional baselines, particularly on X-Cell; however, because the methods do not consistently start from equivalent representations and memory configurations, these differences cannot be attributed exclusively to the continual-learning mechanism. This is an important experimental design limitation: the results demonstrate that the complete PRM system works under its particular initialization and training procedure, but they do not fully establish that its continual-learning strategy itself outperforms the competing methods under matched initialization, parameter budget, and pretraining conditions.
  2. The proposed retention mechanism does not preserve knowledge from the original pretraining distribution, because the primary forgetting metric measures retention of contexts encountered during the four-stage continual-learning sequence but does not capture forgetting of the original pretraining distribution. Section 3 (pp. 3–4) defines relative forgetting using the acquisition and final performance of previously encountered sequential contexts, and the original pretraining distribution is not included in this primary retention metric. Table 4 (Appendix B, p. 15) shows that X-Cell PRM’s held-out pretraining Pearson decreases from 0.7335 before sequential adaptation to 0.1358 after the final K562 stage, corresponding to a 0.5978 absolute drop, and Appendix B (p. 15) explains that pretraining contexts are not assigned continual ownership or included in boundary-time route anchoring; consequently, their memory routes can access newly written residuals or change during calibration. This is a substantial limitation for a method intended to support continual adaptation of pretrained virtual-cell models: although PRM protects knowledge acquired within its sequential training campaign, it does not demonstrate preservation of the broader pretrained capabilities that existed before the campaign began, and this distinction materially limits the scope of its continual-learning claims.
Questions for authors
  1. Can you provide controlled comparisons in which PRM and the competing continual-learning methods start from the same pretrained representation and comparable memory-augmented architecture? This matters because the X-Cell PRM checkpoint undergoes separate memory pretraining and has a substantially different parameter configuration from the dense baselines, and the scDFM PRM checkpoint also undergoes additional preparation that changes its pretraining performance, so these differences make it difficult to isolate the benefits of PRM itself. Please clarify the exact initialization used for each backbone and, ideally, provide matched-initialization experiments comparing full fine-tuning, replay, and other continual-learning baselines against PRM under the same model and memory pretraining conditions.
  2. Can you explain why the original pretraining distribution is not included in the memory ownership and boundary-calibration mechanism, and could PRM protect the pretraining memory routes in addition to the sequentially acquired contexts? This matters because the reported pretraining Pearson decreases from 0.7335 to 0.1358 after the continual-learning sequence, which suggests that near-zero forgetting on sequential contexts does not necessarily imply preservation of the original pretrained capabilities. Please provide an experiment that includes the pretraining distribution in the retention evaluation and investigate whether assigning ownership or route anchoring to pretraining-related memory can reduce this degradation without substantially compromising acquisition on new contexts.
  3. Does PRM maintain similar acquisition and retention performance under different orders of HepG2, RPE1, Jurkat, and K562, and can you report PRM-specific stage-order experiments rather than only the progressive-slot reference? This matters because the order of contexts may influence memory-row ownership, routing calibration, and the availability of writable capacity, while the current main experiments use a fixed four-stage order. Please evaluate several alternative stage orders and report the corresponding acquisition and forgetting statistics, including whether any particular biological context is consistently disadvantaged when introduced later.
2 comments

Thank you for identifying the initialization and pretraining-retention concerns. We added shared-checkpoint controls and tested protection of the pretraining cohort. The corresponding manuscript changes are marked in blue.

Regarding Q1 and W1, Can the methods start from the same pretrained representation and memory architecture:

Three-seed X-Cell controls start from the same pretrained PRM checkpoint and retain the memory forward path. This addresses the original difference in starting representations.

PRM achieves mean acquisition 0.5752 and relative forgetting 0.591%, compared with 0.4669 and 15.27% for host fine-tuning with 5% replay and 0.4141 and 7.83% for rank-64 LoRA with 5% replay.

PRM has 8.0–14.6M update-eligible parameters versus 16.6M for LoRA, with 810 response updates versus 870. Storage, calibration, optimization and revisions do differ, so this is not an exact parameter or compute match. Conclusively, however, PRM's advantage over host fine-tuning and LoRA nevertheless persists when the starting representation is shared.

Regarding Q1 and W1, Do matched initialization and gene-panel controls support the scDFM result:

The new scDFM control starts from the same warmed PRM checkpoint and retains its memory forward path while fine-tuning the host with frozen memory and 5% replay. Both arms use the fixed 1,898-gene panel and 810 response-training updates across the same three adaptation seeds.

scDFMSeedsAcquisitionRelative forgetting
PRM30.3739 ± 0.0041−0.03 ± 0.03%
Host fine-tuning with 5% replay, frozen memory30.3703 ± 0.00659.70 ± 7.52%

Values are mean ± sample SD. PRM has lower forgetting and slightly higher acquisition in every paired seed.

The three-seed panel control gives acquisition 0.3783 ± 0.0094 with 1,898 genes versus 0.4057 ± 0.0037 with 1,000 genes. Relative forgetting is 14.52 ± 6.68% versus 12.80 ± 4.05%, with changes in both directions across seeds. The appendix retains these supporting controls and the three-seed host-plus-PRM ablation.

PRM's retention advantage therefore persists across three paired seeds after matching the starting checkpoint, panel and response-update budget.

Regarding Q2 and W2, Why is pretraining knowledge unprotected, and could ownership or anchoring preserve it:

Thank you for motivating this experiment. The original protocol protected sequential contexts but did not include pretraining contexts in ownership or anchoring (by arbitrary design). However, extending ownership and anchoring to pretraining controls, together with calibration before the first stage, substantially reduces forgetting on the tested X-Cell cohort. Final performance improves from 0.1463 without this protection to 0.6230 with the stronger setting. The new experiment tests protection on a separate cohort of 352 HCT116 training groups, with initial Pearson-Δ of 0.7202 in every arm.

Pretraining protectionFinal training-cohort Pearson-ΔSequential acquisitionSequential forgetting
None, standard PRM0.14630.57520.59%
50% route-mass target0.21420.58074.47%
90% route-mass target0.62300.579318.87%

Including pretraining contexts in protection substantially reduces forgetting on the measured pretraining cohort while maintaining acquisition. The stronger setting increases forgetting of sequential contexts, showing that protecting both sources of knowledge remains a trade-off. This answers whether PRM can extend protection to pretraining contexts, while identifying the remaining challenge of preserving both simultaneously.

Regarding Q3, Does PRM retain its performance under different context orders:

The appendix already reports 22 of 24 possible stage orders for progressive slots. We now add a PRM-specific comparison across four cyclic orders, placing each context once at every position.

Our canonical HepG2 → RPE1 → Jurkat → K562 order is not the best observed PRM case. RPE1 → Jurkat → K562 → HepG2 achieves higher acquisition and lower forgetting, at 0.5823 and 0.31%, compared with 0.5752 and 0.59% for the canonical sequence.

Across all four PRM orders, acquisition remains between 0.5752 and 0.5823, while relative forgetting ranges from 0.31% to 9.51%. PRM therefore learns new contexts consistently across the tested arrival orders, and the canonical sequence is not its best observed result. These experiments broaden the evidence beyond the original sequence while identifying some order sensitivity in retention.

Thank you again for motivating the pretraining-protection experiment. We are following up with additional held-out evaluations and an anchoring sweep that directly test whether PRM can protect pretraining knowledge while retaining sequentially learned contexts.

Regarding Q2 and W2, Protecting pretraining knowledge while retaining new contexts:

Extending X-Cell PRM's ownership and anchoring to pretraining controls raises final Pearson-Δ from 0.1202 to 0.6220 on the same 860 held-out HCT116 groups, from an initial score of 0.6788. These held-out results demonstrate that protection can extend beyond the sequential training contexts.

In a separate matched sweep, we vary the anchoring weight while fixing the starting checkpoint, memory size, gene panel and response-update budget. Each row is one adaptation campaign.

Anchoring weightAcquisitionSequential forgetting (%)Pretraining retention (Pearson-Δ)K562 acquisition
10.574821.220.62110.5494
20.577014.240.62950.5512
40.56776.520.62240.5126
80.54072.620.63270.4091

Pretraining retention is final Pearson-Δ on the same 860 held-out HCT116 groups, from an initial score of 0.6788.

Doubling the anchoring weight from 1 to 2 raises held-out Pearson-Δ from 0.6211 to 0.6295 while lowering sequential forgetting from 21.22% to 14.24%. This improves both measured retention outcomes without enlarging memory, with acquisition changing from 0.5748 to 0.5770. The completed weight-8 evaluation gives held-out Pearson-Δ of 0.6327 and sequential forgetting of 2.62% in the same campaign, with lower acquisition of 0.5407.

The appendix subsection “Protection of pretraining knowledge” reports the protection gains, their cost to sequential retention, and the full anchoring sweep, including the acquisition trade-off at larger weights. First-round changes remain blue, and these second-round additions are red.

These results directly address Q2 and W2 by extending PRM to pretraining retention and showing how an existing calibration weight improves the measured retention balance. We would be grateful if you would reconsider your assessment and score in light of this new evidence.

Reviewer_WTwSOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies continual learning for virtual-cell models as new cellular contexts become available. Using a four-stage Perturb-seq benchmark, the authors examine catastrophic forgetting in scGPT, scDFM, and X-Cell. They propose Profile-Routed Residual Memory (PRM), which uses gene identities and control-cell profiles for memory routing, protects previously updated memory rows, and recalibrates routing to provide writable capacity for new contexts. Experiments compare PRM with existing continual learning methods, including fine-tuning, replay, and parameter-efficient adaptation. The results show substantially reduced forgetting while maintaining competitive acquisition performance, although the gains vary across models. Additional experiments examine memory protection, routing calibration, and replay budgets.

Critical strengths
  1. PRM uses gene identities and unperturbed control-cell profiles for memory routing. This allows adaptation to new cellular contexts without task identifiers at inference or replaying earlier perturbed examples. The design makes use of information naturally available in the perturbation prediction setting.
  2. The paper evaluates PRM across three virtual-cell backbones and compares it with several continual learning baselines. The results show substantially reduced forgetting. More importantly, the ablation studies demonstrate that protecting earlier memory reduces forgetting but can limit new-context learning, while routing calibration restores access to writable capacity. These experiments support the main design choices.
Critical weaknesses
  1. The paper demonstrates catastrophic forgetting in virtual-cell models, but does not clearly explain how this problem differs from conventional domain-incremental learning. Although PRM uses gene identities and control-cell profiles for memory routing, the main challenges motivating the method—memory overwriting and limited writable capacity—are general to sparse-memory continual learning. The distinct research gap in virtual-cell adaptation therefore remains unclear.
  2. The benefit of shared memory over stage-private memory is not clearly demonstrated. In Appendix Table 5, the progressive-slots reference reports higher acquisition (0.5921 vs. 0.5752) and lower forgetting (0.01% vs. 0.59%) than PRM. Although the comparison is limited by different pretrained checkpoints and only one run for progressive slots, it raises questions about what shared read access contributes beyond stage-private memory.
Questions for authors
  1. What specific challenges distinguish continual perturbation prediction in virtual-cell models from conventional domain-incremental learning, and how do they motivate PRM's design?
  2. What advantages does shared read access offer over stage-private memory? Given the progressive-slots results in Appendix Table 5, could the authors provide a more controlled comparison with comparable initialization and multiple runs?
1 comment

Thank you for recognizing the value of control-based routing and the protection and calibration experiments to justify the design. Your questions motivated a more controlled memory comparison and a clearer explanation of the continual-learning setting. The corresponding manuscript changes are marked in blue.

Regarding Q1 and W1, How does this differ from conventional domain incremental learning:

Our contribution is a calibration rule that restores access to writable memory while protecting earlier entries and limiting changes to earlier routes. Domain-incremental learning describes our setting. The methodological question we focus on is how a frozen backbone can acquire new contexts through memory when earlier writes must remain protected.

Protection alone can leave new inputs routed toward rows they cannot update. As an example, within PRM, calibration raises K562 acquisition under full protection from 0.1143 with frozen routing to 0.5310 in one run each, while permuting the protected-risk information gives 0.1138 ± 0.0003 across three runs. These interventions show why protecting values and maintaining access to writable capacity must work together.

We build on established sparse retrieval and protected updates. PRM contributes a calibration rule that restores access to writable rows, then learns new responses through residual-memory updates while the backbone remains frozen. This addresses the conventional failure mode in which protecting earlier memory blocks new learning, all without requiring task labels at inference.

Regarding Q2 and W2, Does shared memory help compared with progressive slots under matched initialization and multiple runs:

We completed the requested comparison from the same pretrained PRM checkpoint across three adaptation seeds, using the same gene panel and response-training update budget. This addresses the initialization and replication limits of the submitted reference.

Slots achieve acquisition 0.5919 ± 0.0040 and forgetting 0.0356 ± 0.0017%, compared with PRM's 0.5752 ± 0.0032 and 0.591 ± 0.131%. Slots reserve write regions by stage, while PRM selects rows using routing traffic and calibrates access to writable capacity.

The comparison favors slots on this four-stage benchmark, and both designs mix reads, so it does not establish an advantage from shared reads. PRM contributes a calibration rule that restores writable access without assigning a fixed memory region to each stage. The interventions in Q1 support this contribution by showing how calibration recovers learning when protecting earlier writes leaves new inputs unable to update the memory they retrieve, which is expected to be a more realistic setting as the number of stages scales.

Comments on the paper

1 comment

Summary of the revision for the AC

Thank you for handling our paper and taking the time to support the frontier of machine learning research for science! Thanks to the reviewers' suggestions, we conducted revisions and made three changes that make our contribution stronger, as reflected by one reviewer’s changed score. We have updated the manuscript so that round one edits are blue and round two edits are red

  1. PRM can now retain pretraining knowledge (iwfE, spjB). On X-Cell, extending ownership and anchoring to pretraining contexts raises final held-out pretraining Pearson-Δ from 0.12 to 0.62, against an initial 0.68, while sequential forgetting rises from 0.6% to about 20%. Raising the existing anchoring weight from 1 to 8 brings sequential forgetting back to 2.6% with pretraining retention maintaining at 0.63 (Appendix B.1, Table 9). Thus, merely modifying our setup (to include pretraining protection) shows additional extension of retention using our memory-layer based continual learning method.

  2. The advantage holds from a shared starting checkpoint (iwfE). On X-Cell over three seeds, PRM reaches 0.575 acquisition with 0.6% forgetting, versus 0.467 with 15.3% for host fine-tuning and 0.414 with 7.8% for LoRA, both with 5% replay. The matched scDFM control agrees, so the starting checkpoint does not explain the gain (Tables 3 and 14).

  3. Sparse protected updates alone do not explain the gains (spjB). Random rows with matched counts and the same protection reach 0.126 acquisition versus 0.575 for PRM. The router reached only 613 of the 260,869 randomly reserved rows, so allocation must follow routing to give usable capacity (Tables 3 and 5). Our method thus needs the components specified.

Clarity. We also revised the passages the reviewers asked us to clarify. A plain summary of how memory is read, written and injected through Equation 7 now opens Section 4 (spjB), and Section 5.2 now states the starting checkpoints in the main text (iwfE). Reviewer WTwS rated the presentation clear and easy to follow.

Status. Reviewer spjB replied and raised soundness to 3. Reviewers WTwS and iwfE have not yet replied to our responses. We welcome their view on the new, strengthening results.

As applications of deep learning to life sciences data keep growing, our paper now offers a benchmark for catastrophic forgetting in virtual cell models and a replay-free method whose gains over replay and other continual learning baselines hold from a shared starting checkpoint. The need to fine-tune continually has been a major challenge in adoption of single-cell models and our contribution allows future virtual cell models (and potentially other domains) to make continual updates in production without appreciable acquisition or lost retention costs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.