Continual Learning for Virtual Cell models

Recent virtual cell or single-cell foundation models are typically pretrained on fixed datasets. However, with the growth of biological data, there is a pressing need to continually train such models on new data distributions without sacrificing performance on previously seen domains. We investigate this continual-learning problem in three existing models, scGPT, scDFM, and X-Cell, using a four-stage Perturb-seq benchmark. Across these backbones, naive fine-tuning loses 69–84% of old-domain performance. Continual training with 5% replay reduces but does not remove this forgetting. On X-Cell, larger replay budgets can reduce forgetting but improve new-domain learning only marginally, so retention trades off against acquisition. Motivated by interference in shared memory, we introduce Profile-Routed Residual Memory (PRM), residual product-key memory layers with a boundary-aware mechanism that separates read composition from write ownership, building on prior work on sparse product-key memory layers. This boundary-aware continual learning method requires observable context and awareness of training boundaries, but not task identifiers, replay of earlier perturbed examples, or distillation. Integrated into X-Cell, a recent single-cell perturbation model, PRM achieves a mean acquisition Pearson-$\Delta$ of 0.575 with a final old-domain Pearson-$\Delta$ of 0.583, corresponding to 0.6% relative forgetting and both retains and acquires more learning throughout the training campaign compared to other continual learning methods. On other model backbones, PRM shows minimal forgetting. These results provide initial evidence that asymmetric soft reads and private writes are a useful design principle for stable, plastic adaptation.