OcMAE: Ocean Masked Autoencoders for Biogeochemistry Prediction

Learning from sparse labeled data remains a central challenge in machine learning. While large-scale pretraining has driven major advances in domains such as computer vision, its potential for scientific data remains elusive. We introduce Ocean Masked Autoencoders (OcMAE), a self-supervised framework for learning representations from large-scale satellite data to improve downstream prediction tasks in ocean biogeochemistry. OcMAE introduces two key components. First, it discretizes continuous variables into bins, improving representation learning while naturally handling missing values. Second, it leverages large-scale satellite data for self-supervised pretraining, enabling the model to learn prior over biogeochemical processes. We evaluate OcMAE on nine downstream tasks and show consistent improvements over strong baselines, including random forests (RF), extreme gradient boosting (XGB), and tabular prior-data fitted networks (TabPFN). Our results demonstrate that large-scale self-supervised pretraining on satellite data can improve generalization in ocean biogeochemistry.