CellShift: Benchmarking Single-Cell Temporal Models on Unseen Biology

Single-cell temporal modeling is the task of predicting the gene expression of individual cells at unobserved time points, either after the last measurement (forecasting) or between measurements (interpolation). Machine-learning models such as PRESCIENT, scNODE and CellTempo learn these tasks from single-cell RNA sequencing time courses, which measure thousands of genes in each of thousands of cells. Existing evaluations, however, hold out time points within the cell types and conditions seen during training, so it remains unclear whether these models learn dynamics that transfer to unseen biology. This paper introduces CellShift, a benchmark spanning seven time-course datasets, with three extended to test generalization across held-out cell fates, lineages, and starting populations. It evaluates statistical baselines, neural dynamics models, pretrained temporal models, and language-model agents under a shared evaluator with explicit tracking of information access. On lineage-traced blood development (the LARRY in vitro dataset), PRESCIENT and a population-time MLP outperform simple copy baselines when forecasting the whole day-6 population. However, when evaluated on held-out fates, no fitted model without access to target-day cells improves upon copying the latest observed snapshot by more than 5%, even when initialized from the held-out cells. Even on identical target cells, holding the fate out raises the error of nearly every fitted model. At the clone level, models trained on day-6 states assign at most 28% of predicted cells to a held-out fate that comprises 93 to 95% of the observed population, highlighting the difficulty of generalizing to unseen cell fates. Agents that choose among a fixed set of numerical tools perform on par with a selector that uses no language model.