Scryer: A scalable framework for forecasting neural population activity

Forecasting objectives have become a central component of modern large-scale models in language and video, as they enable the learning of structure in complex temporally evolving data. However, applying this paradigm to neural recordings from the brain remains challenging because neural activity is measured over varying populations of neurons and exhibits high-dimensional, sparse, and nonstationary dynamics. In this work, we introduce Scryer, a scalable framework for neural forecasting designed to learn transferable representations from heterogeneous neural recordings, built around a hierarchical multiscale architecture with learnable channel embeddings and population-conditioned retrieval. We evaluate Scryer across both electrophysiology and calcium imaging datasets spanning multiple brain regions and species. Finally, we show that Scryer's unit-identity embeddings, learned purely from the forecasting objective, support brain-region decoding, substantially exceeding an identity representation derived from other methods. Together, these results support forecasting-driven architectures, independent of any single auxiliary training objective, as a viable approach for learning representations from neural population activity.