acceptodds
Under review as a conference paper at ICLR 2027

OcMAE: Ocean Masked Autoencoders for Biogeochemistry Prediction

Abstract

Learning from sparse labeled data remains a central challenge in machine learning. While large-scale pretraining has driven major advances in domains such as computer vision, its potential for scientific data remains elusive. We introduce Ocean Masked Autoencoders (OcMAE), a self-supervised framework for learning representations from large-scale satellite data to improve downstream prediction tasks in ocean biogeochemistry. OcMAE introduces two key components. First, it discretizes continuous variables into bins, improving representation learning while naturally handling missing values. Second, it leverages large-scale satellite data for self-supervised pretraining, enabling the model to learn prior over biogeochemical processes. We evaluate OcMAE on nine downstream tasks and show consistent improvements over strong baselines, including random forests (RF), extreme gradient boosting (XGB), and tabular prior-data fitted networks (TabPFN). Our results demonstrate that large-scale self-supervised pretraining on satellite data can improve generalization in ocean biogeochemistry.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
masked autoencoders, self-supervised learning, remote sensing, ocean biogeochemistry

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_LDuTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Ocean Masked Autoencoders (OcMAE), a self-supervised learning framework for ocean biogeochemistry prediction using large-scale satellite observations. The method combines quantile-based feature discretization with masked autoencoder pretraining to learn spatiotemporal representations while handling missing values. The authors also provide a theoretical interpretation connecting masked reconstruction to ocean physical processes. Experiments on nine downstream prediction tasks, along with ablation studies, show that OcMAE outperforms traditional machine learning and tabular baselines on four tasks.

Critical strengths

1.Meaningful Problem and Practical Significance: The paper addresses limited labeled data and missing observations in ocean biogeochemistry. Using large-scale satellite data for self-supervised learning is well-motivated and practically relevant.

2.Large-Scale Data Preparation: The authors process satellite observations from 2003–2023 to construct a pretraining dataset with over 500 million spatiotemporal image cubes, providing a useful foundation for ocean representation learning.

Critical weaknesses
  1. Insufficient Comparison with Relevant Baselines: The comparisons mainly focus on traditional machine learning and tabular models, while a standard MAE adapted to the same setting is missing. Domain-specific models such as OXYGENERATOR could also provide relevant comparisons for oxygen-related tasks. This makes it difficult to assess the advantages of OcMAE over existing approaches.

  2. Limited Methodological Novelty: The method mainly combines existing discretization and masked autoencoder techniques, while the theoretical analysis does not rigorously establish that masked reconstruction learns the underlying physical dynamics.

Questions for authors
  1. Additional Baselines: Could the authors include comparisons with a standard MAE adapted to the same setting and relevant domain-specific models, such as OXYGENERATOR on applicable tasks, to better establish the advantages of OcMAE?

  2. Analysis of Underperforming Tasks: Could the authors provide further analysis of why OcMAE fails to improve over existing baselines on several tasks and discuss the factors that may limit its effectiveness?

  3. Ablation Studies: Could the authors provide ablations that isolate the contributions of self-supervised pretraining and quantile-based discretization to the overall performance?

3 comments
AuthorsOpenReview ↗

Response to the reviewer 3

We thank the reviewer for the thoughtful and constructive feedback. We address the concerns regarding baselines, methodological novelty, task-dependent performance, and ablation studies below.

1. Comparison with standard MAE and domain-specific models

Reviewer concern: The evaluation lacks a comparison to standard MAE baseline and domain-specific models such as OXYGENERATOR.

Response: We have additionally evaluated a standard MAE (NO_Pre and NO_Bin in Table 1) using the same pretraining dataset, model architecture, and downstream evaluation protocol. The results demonstrate that both large-scale pretraining and discretization contribute to improved predictive performance.

Regarding OXYGENERATOR, we appreciate this suggestion and will clarify its applicability to our downstream prediction tasks. OcMAE focuses on predicting surface oxygen concentrations, whereas OXYGENERATOR was developed 3D oxygen reconstruction. Surface oxygen concentrations are well constrained by solubility. This is also reflected in the high accuracy achieved by linear regression and other algorithms, suggesting that surface oxygen provides limited discrimination among model architectures. We include surface oxygen prediction to primarily establish a baseline for other downstream tasks. Our main interest lies in more challenging biogeochemical prediction tasks, e.g., biological productivity, where the underlying processes are more complex. We will clarify these distinctions in the revised manuscript.

2. Methodological novelty and theoretical interpretation

Reviewer concern: OcMAE combines existing techniques, and the theoretical analysis does not rigorously establish that masked reconstruction learns physical dynamics.

Response: Quantization and masked autoencoders are established techniques. Our contribution is to develop and evaluate a self-supervised representation learning framework for ocean biogeochemistry, by leveraging large-scale satellite observations with missing values accommodated by including a new category after quantization.

Our theoretical analysis provides a physical interpretation connecting masked reconstruction with tracer conservation equation, rather than establishing a general guarantee of physical-dynamics recovery.

To summarize, our key contributions are:

(1) large-scale pretraining on global satellite ocean-color observations, using discretization of continuous observations to accommodate missing values and nonlinear relationships;

(2) comprehensive evaluation across nine ocean biogeochemistry prediction tasks against strong baselines;

(3) an analysis connecting masked reconstruction to the tracer transport equation, providing a physical interpretation of the learned representations.

3. Analysis of underperforming downstream tasks

Reviewer concern: OcMAE does not improve over existing baselines on several tasks. Further analysis of these cases would be valuable.

Response: We agree that OcMAE improves upon existing baselines on four of the nine downstream tasks.

We would like to emphasize that several of these tasks, particularly GPP, CR, nitrification, and denitrification, are especially challenging because of the limited availability of labeled observations, with only a few hundred training samples across the global ocean for each task (Table 4 in paper). Furthermore, these biogeochemical processes involve complex underlying mechanisms and substantial spatial and temporal variability. The relatively modest performance across different models highlights the difficulty of these prediction problems rather than necessarily reflecting limitations specific to OcMAE. Nevertheless, we believe that reporting these results are valuable, as they provide useful benchmarks for future studies and may stimulate further research on these challenging biogeochemical prediction tasks.

We hypothesize that differences in model performance reflect variations in the underlying prediction problems, including the availability and quality of labeled observations, spatial and temporal scales, the strength of relationships between satellite data, including Chl, SST, and PAR, and target variables, and the extent to which pretrained representations transfer across tasks. We will expand the discussion of these results in the revised manuscript.

4. Ablation studies isolating pretraining and discretization

Reviewer concern: Ablations should isolate the contributions of self-supervised pretraining and quantile-based discretization.

Response: We have additionally evaluated a standard MAE (NO_Pre and NO_Bin in Table 1) using the same pretraining dataset, model architecture, and downstream evaluation protocol. The results demonstrate that both components contribute to improved downstream prediction performance.

We thank the reviewer again for these valuable comments, which will help us improve the clarity of the paper and better communicate its contributions.

Thank you for the detailed response. The clarification regarding OXYGENERATOR's different task setting and the more modest interpretation of the theoretical analysis are helpful. The discussion of task-dependent performance is also reasonable.

However, my main concern about methodological novelty remains, as the framework primarily combines established discretization and masked autoencoder techniques. Moreover, the additional MAE comparisons and ablations are mentioned, but their numerical results are not provided in the response, making their contributions difficult to assess. Overall, while the response addresses some of my concerns, I maintain my original rating.

AuthorsOpenReview ↗

Response to the reviewer 3

We thank the reviewer for the insightful and specific feedback. We have included the additional experimental results (NO_Pre and NO_Bin) in both Table 1 and Table 8. These additional results will also be incorporated into the revised manuscript.

Table 1. The prediction accuracies (coefficient of determination, R2R^2) of ppCO$_2$, O$_2$ concentration, air-sea O$_2$ flux (O$_2$FLUX), NCP, and CR derived by different model variants.

ModelpCO₂O₂O₂ fluxNCPCR
Base0.662 ± 0.0090.971 ± 0.0000.379 ± 0.0040.258 ± 0.0360.260 ± 0.026
BIN320.637 ± 0.0160.970 ± 0.0000.335 ± 0.0120.145 ± 0.0360.244 ± 0.048
BIN2560.655 ± 0.0130.971 ± 0.0010.371 ± 0.0070.265 ± 0.0240.297 ± 0.018
BIN5120.668 ± 0.0110.971 ± 0.0010.366 ± 0.0200.174 ± 0.0500.290 ± 0.020
BIN20480.663 ± 0.0130.971 ± 0.0010.367 ± 0.0140.189 ± 0.0230.201 ± 0.029
L40.640 ± 0.0130.969 ± 0.0020.353 ± 0.0100.282 ± 0.0230.289 ± 0.010
L80.641 ± 0.0240.970 ± 0.0010.369 ± 0.0110.305 ± 0.0180.218 ± 0.027
L320.669 ± 0.0050.972 ± 0.0000.387 ± 0.0070.297 ± 0.0220.198 ± 0.020
D320.629 ± 0.0180.970 ± 0.0000.361 ± 0.0060.293 ± 0.0120.228 ± 0.020
D1280.649 ± 0.0190.971 ± 0.0010.360 ± 0.0110.258 ± 0.0190.246 ± 0.009
S50.622 ± 0.0080.970 ± 0.0010.351 ± 0.0080.332 ± 0.0080.238 ± 0.030
S70.617 ± 0.0090.969 ± 0.0010.348 ± 0.0120.289 ± 0.0400.258 ± 0.017
T20.631 ± 0.0170.970 ± 0.0010.360 ± 0.0230.128 ± 0.0920.265 ± 0.026
T50.638 ± 0.0230.970 ± 0.0010.351 ± 0.0100.212 ± 0.0410.232 ± 0.029
PD_few0.634 ± 0.0170.962 ± 0.0000.262 ± 0.0070.310 ± 0.0200.245 ± 0.026
PD_0.30.643 ± 0.0150.972 ± 0.0000.372 ± 0.0060.273 ± 0.0100.244 ± 0.040
PD_Reg0.641 ± 0.0100.970 ± 0.0010.327 ± 0.0120.245 ± 0.0090.126 ± 0.008
NO_Pre0.579 ± 0.0260.961 ± 0.0010.238 ± 0.014-0.009 ± 0.0200.092 ± 0.013
NO_Bin0.602 ± 0.0110.970 ± 0.0010.363 ± 0.0080.202 ± 0.0140.023 ± 0.012

Table 8. The prediction accuracies (coefficient of determination, R2R^2) of GPP, NPP, nitrification, and N$_2$ fixation rate (N$_2$FIX) derived by different model variants.

ModelGPPN₂ fixNPPNitrification
Base-0.047 ± 0.056-0.050 ± 0.0200.008 ± 0.0240.046 ± 0.045
BIN320.018 ± 0.056-0.053 ± 0.0210.023 ± 0.0170.049 ± 0.045
BIN256-0.044 ± 0.053-0.053 ± 0.0210.012 ± 0.0230.049 ± 0.044
BIN5120.042 ± 0.008-0.052 ± 0.0210.010 ± 0.0240.047 ± 0.045
BIN2048-0.032 ± 0.043-0.054 ± 0.0220.007 ± 0.0250.048 ± 0.044
L4-0.055 ± 0.022-0.014 ± 0.0440.007 ± 0.0250.050 ± 0.045
L80.158 ± 0.1230.031 ± 0.0100.006 ± 0.0140.049 ± 0.044
L32-0.051 ± 0.0500.014 ± 0.0200.009 ± 0.0240.046 ± 0.045
D32-0.051 ± 0.020-0.593 ± 0.0740.008 ± 0.0230.048 ± 0.045
D128-0.057 ± 0.0230.064 ± 0.0120.006 ± 0.0150.048 ± 0.045
S5-0.052 ± 0.020-0.125 ± 0.0420.008 ± 0.0240.046 ± 0.045
S7-0.052 ± 0.0210.055 ± 0.0310.005 ± 0.0150.044 ± 0.046
T2-0.062 ± 0.042-0.053 ± 0.0210.014 ± 0.0220.048 ± 0.045
T50.179 ± 0.0370.229 ± 0.033-0.047 ± 0.0070.113 ± 0.034
PD_few0.073 ± 0.0370.031 ± 0.0470.017 ± 0.0230.050 ± 0.044
PD_0.3-0.053 ± 0.0210.020 ± 0.0060.011 ± 0.0230.047 ± 0.045
PD_Reg-0.053 ± 0.021-0.598 ± 0.0780.019 ± 0.0230.050 ± 0.045
NO_Pre-0.555 ± 0.152-0.183 ± 0.0860.004 ± 0.0180.092 ± 0.013
NO_Bin-0.578 ± 0.015-0.590 ± 0.006-0.003 ± 0.0120.054 ± 0.002
LR-0.755 ± 11.30-0.292 ± 0.470-0.023 ± 0.055-0.305 ± 0.153
RF-0.083 ± 0.151-0.063 ± 0.0740.313 ± 0.041-0.058 ± 0.119
XGB-0.183 ± 0.178-0.004 ± 0.1040.264 ± 0.064-0.088 ± 0.182
MLP-0.431 ± 0.357-0.063 ± 0.487-0.609 ± 1.123-0.179 ± 0.072
TabPFN-0.108 ± 0.134-0.267 ± 0.0930.376 ± 0.0490.021 ± 0.078
Reviewer_ezfQOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes OcMAE, a masked autoencoder pretrained on about 500 million small satellite cubes. Each cube covers 3 × 3 pixels. It holds six variables observed over three consecutive 8-day periods, plus two monthly climatologies (sea surface salinity and mixed layer depth). Each value is quantile-binned and treated as a token, with a separate token for missing data. Pretraining reconstructs masked tokens, following Lee et al. (ICML 2024).

The encoder is then fine-tuned on nine sparse in-situ biogeochemistry targets, including pCO2, oxygen, community metabolism and nitrogen fixation. It is compared with linear regression, random forest, XGBoost, an MLP and TabPFN. The paper also derives a tracer transport equation on the 3 × 3 stencil and argues that masked reconstruction learns advection, diffusion and source–sink terms.

Critical strengths
  • Evaluating one pretrained model on nine ocean biogeochemical targets, with year-based splits across the carbon, oxygen and nitrogen cycles, is useful in its own right. The benchmark could serve the community beyond this paper.
  • A dedicated token for missing values handles cloud gaps in ocean-colour data without imputation.
  • The ablations over model size, binning, window size, time length and pretraining data volume are extensive.
  • The paper reports openly that tree ensembles remain competitive and that several targets are not predicted well by any model.
Critical weaknesses
  1. The paper does not establish what produces the benefit.

    • No from-scratch control. All 17 variants are pretrained, and OcMAE is never trained without pretraining. The central claim, that large-scale satellite pretraining improves the downstream estimates, is therefore not separated from the effect of the architecture and tokenization. Dawson et al. (2025), which the paper cites, include this control.
    • The recipe itself is not new. The tokenization and reconstruction objective come from Lee et al. (2024). Pixel-level masked pretraining for sparse-label Earth-observation tasks exists on land (Presto, Galileo) and for ocean colour (Dawson et al.).
    • A new architecture is not required for an application paper. Without the control, the paper cannot show what its particular combination adds over these approaches.
  2. The gains are small and depend strongly on the variant.

    • On the five tasks in Table 2, the margins over the best baseline are +0.02, 0.00, +0.02, +0.05 and +0.08 in R². The CR gain is meaningful in relative terms.
    • On NPP, no variant exceeds 0.03, while TabPFN reaches 0.38 and random forest 0.31.
    • On GPP, N2 fixation and nitrification, Table 8 shows that the five-time-step variant reaches positive R² and beats the baselines, while most other variants stay weak. These gains depend strongly on the configuration and are not seen consistently across variants.
    • The abstract's claim of "consistent improvements over strong baselines" is not supported.
  3. Model selection and uncertainty are unclear.

    • The paper says the highest-performing variants are selected for Table 2. It does not say whether the selection used validation or test data.
    • Some reported values are not the highest in the ablation table. Table 2 reports NCP R² = 0.297, while Table 1 gives 0.332 for the 5 × 5 variant.
    • OcMAE's error bars pool three neighbouring epochs from each of 11 fine-tuning runs, while the baselines use bootstrap resampling. The two standard deviations reflect different sources of variability and are not directly comparable.
    • TabPFN is applied to pCO2 well beyond the data size it is designed for.
  4. The comparisons are limited.

    • The only conventional neural network trained on each downstream task is an MLP. The authors state that its hyperparameter search was restricted by computational cost. A stronger task-trained network, such as a modern tabular model with numerical feature embeddings or a CNN on the same cubes, would show whether OcMAE's gains reflect pretraining or limits of the supervised comparisons.
    • The resulting maps are not compared with established reference products, such as the existing pCO2 mapping products, or evaluated in a testbed (Gloege et al., 2021).
    • The paper notes "notable discrepancies" between OcMAE's maps and its baselines' maps without saying which is closer to the truth.
  5. The physical interpretation is not demonstrated.

    • The model has no explicit transport operator, and its inputs do not include ocean currents. Its attention may learn statistical dependencies that arise from transport, but the paper does not show that these correspond to the coefficients or processes in the derived tracer equation.
    • The interpretation also raises a question of scale. At about 9 km resolution, water can move many grid cells in eight days. It is unclear how the 3 × 3 stencil captures this transport, or how the derivation relates to the dependencies the model actually learns.
    • The paper's own ablation shows that larger windows mostly reduce accuracy. As written, the physical interpretation is an analogy, not a result.

References

  • Dawson, G., et al. (2025). A Sentinel-3 foundation model for ocean colour. arXiv:2509.21273.
  • Gloege, L., et al. (2021). Quantifying errors in observationally based estimates of ocean carbon sink variability. Global Biogeochemical Cycles, 35, e2020GB006788.
  • Hollmann, N., et al. (2025). Accurate predictions on small data with a tabular foundation model (TabPFN). Nature. doi:10.1038/s41586-024-08328-6.
  • Lee, K., et al. (2024). Binning as a pretext task: Improving self-supervised learning in tabular domains. ICML 2024 (PMLR 235).
  • Tseng, G., et al. (2023). Lightweight, pre-trained transformers for remote sensing timeseries (Presto). arXiv:2304.14065.
  • Tseng, G., et al. (2025). Galileo: Learning global & local features of many remote sensing modalities. ICML 2025 (PMLR 267).
Questions for authors
  1. How does the same architecture perform when trained from scratch, without pretraining, on the same downstream data and protocol?
  2. How was the variant for each task in Table 2 chosen, and how does a single fixed variant perform across all nine tasks?
  3. Do the baselines receive all cube values or only the centre pixel?
2 comments
AuthorsOpenReview ↗

Response to the reviewer 2

We sincerely thank the reviewer for the detailed and thoughtful assessment. We will address the major concerns below.

1. Contribution of self-supervised pretraining

Reviewer concern: The manuscript does not include a from-scratch control, making it difficult to establish whether the downstream performance improvements arise from self-supervised pretraining or the architecture and tokenization.

Response: We have additionally evaluated a standard MAE (NO_Pre and NO_Bin in both Table 1 and Table 8; due to limited space, the results are only attached in the response to reviewer 3) using the same model architecture and downstream evaluation protocol. The results demonstrate that both self-supervised pretraining and discretization contribute to improved downstream prediction performance.

2. Variant selection and consistency of performance

Reviewer concern: The variant selection procedure is unclear, and the reported improvements are not consistent across tasks or configurations.

Response: Now, we use a fixed OcMAE configuration (L32 in Table 1) across all downstream tasks. We will revise Table 2, the abstract, and discussion to more accurately characterize the task-dependent benefits and limitations of OcMAE.

3. Comparability of baselines and input information

Reviewer question: Do the baselines receive all cube values or only the center pixel?

Response: All models, including baselines, receive all cube values.

4. Performance variability and uncertainty

Reviewer concern: The reported gains are relatively small and vary across configurations. The uncertainty estimates are also calculated differently for OcMAE and the baselines.

Response: We will clarify the uncertainty estimation procedures in the revised manuscript and moderate our claims regarding performance improvements.

We would also like to emphasize that several of these tasks, particularly GPP, CR, nitrification, and denitrification, are especially challenging because of the limited availability of labeled observations, with only a few hundred training samples across the global ocean for each task. Furthermore, these biogeochemical processes involve complex underlying mechanisms and substantial spatial and temporal variability. The relatively modest performance across all evaluated models highlights the intrinsic challenges of these prediction problems, rather than necessarily indicating limitations specific to OcMAE. Nevertheless, we believe that reporting these results is valuable because they provide useful benchmarks for future studies and may stimulate further research on these challenging biogeochemical prediction tasks.

5. Physical interpretation and tracer transport equation

Reviewer concern: The connection between masked reconstruction and the tracer conservation equation is not demonstrated empirically. In particular, the model does not explicitly incorporate ocean currents, and the spatial and temporal scales may not support the proposed physical interpretation.

Response: We appreciate this criticism. Our intention in Appendix F is to establish a mathematical connection between masked reconstruction and the general structure of tracer conservation equation. This connection provides a physical interpretation of the spatiotemporal dependencies that may be captured by the reconstruction objective, rather than direct evidence that the model identifies the underlying physical mechanisms.

We conducted pretraining experiments using different image sizes (S5 and S7 in Table 1), which provide broader spatial contexts that may capture larger-scale spatial dependencies. However, increasing the image size did not improve downstream prediction performance. We did not explore even larger sizes because of computational constraints. We acknowledge that these experiments do not establish that the model captures physical processes across spatial and temporal scales.

6. Additional comparisons and scientific validation

Reviewer concern: Comparisons with stronger neural models and established pCO2 mapping products would improve the evaluation.

Response: We agree that these comparisons would provide valuable context. Our primary objective is to evaluate whether representations learned through self-supervised pretraining on large-scale satellite data can be transferred to biogeochemical prediction tasks with sparse labels. The current benchmark includes TabPFN, a network that is recognized as strong baseline for tabular prediction, and RF and XGBoost which are also strong baselines of tabular data and have been applied to pCO2 estimation. We will clarify the scope of the current benchmarks and discuss the limitations of the supervised baselines evaluated in this study.

We thank the reviewer again for identifying these important issues. We will revise the manuscript accordingly.

Thanks for the responses. I think my score stands. point 4 stands as the validity of the benchmark depends on the setup which may be suboptimal. External validation is still missing my point 4's (author listed as 6) novelty point stands. Points 2, 3 were not directly answered. overall, I think the paper fits a domain outlet better than this venue.

Reviewer_2XrxOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This work presents a masked auto encoder for satellite data for ocean biochemistry prediction tasks. The autoencoder discretizes continuous variables into discrete bins and handles missing values. Results improve upon baseline of random forest, XGB, and TabPFN on nine tasks. Comprehensive supplementary material consists of details about the dataset, tasks, models, experiments, theoretical analysis, and benchmarks.

Critical strengths
  1. The problem is well motivated by datasets with missing observations and non-linear relationships between inputs and outputs.

  2. The method uses discretization with a quantile binning method, and embedding into a low dimensional space. The results demonstrate improved accuracy on four of nine tasks.

  3. Ablations across model size, discretization, image resolution, time series length, pertaining data volume are valuable.

  4. Appendix F shows in detail how masked reconstruction is interpreted as learning the tracer transport equation.

Critical weaknesses
  1. Limited novelty: the method applies existing autoencoder techniques to satellite data for ocean biochemistry tasks.

  2. The tracer transport equation and analysis is a significant part of the paper. It is unclear which parts of these derivations are novel.

Questions for authors

The paper mentions that the satellite images often contain gaps due to cloud cover. Is this in the dataset used in these experiments? Can this be visualized?

1 comment
AuthorsOpenReview ↗

Response to the reviewer 1

We thank the reviewer for the thoughtful and constructive feedback. We address the concerns regarding novelty, the tracer transport equation analysis, and cloud-induced missing observations below.

1. Novelty and contributions

Reviewer concern: Limited novelty: the method applies existing autoencoder techniques to satellite data for ocean biogeochemistry tasks.

Response: We agree that masked autoencoders and quantization are established techniques. Our contribution is not to develop a novel generic autoencoder architecture, but rather to develop and evaluate a framework tailored to satellite-based ocean biogeochemistry prediction across nine tasks spanning ocean oxygen, carbon, and nitrogen cycling. These tasks share underlying data-generating processes but have traditionally been modeled independently, though sparse in situ observations unevenly sampled in space and time. We learn shared representations of these underlying processes through pretraining on abundant satellite ocean-color observations, while explicitly accommodating missing values. Quantization allows missing observations to be represented as a distinct category, enabling the model learned from incomplete satellite data. Representation learning over discrete bins may capture patterns associated with the underlying tracer transport equations through masked reconstruction.

To summarize, our key contributions are:

(1) large-scale pretraining on global satellite ocean-color observations, using discretization of continuous observations to accommodate missing values and nonlinear relationships;

(2) comprehensive evaluation across nine ocean biogeochemistry prediction tasks against strong baselines;

(3) an analysis connecting masked reconstruction to the tracer transport equation, providing a physical interpretation of the learned representations.

Importantly, our results demonstrate improvements over RF, XGBoost, and TabPFN on three of the nine tasks, highlighting the potential of the proposed framework for scientific prediction. We would also like to emphasize that several of these tasks, particularly GPP, CR, nitrification, and denitrification, are especially challenging because of the limited availability of labeled observations, with only a few hundred training samples across the global ocean for each task. Furthermore, these biogeochemical processes involve complex underlying mechanisms and substantial spatiotemporal variability. The relatively modest performance across all evaluated models highlights the intrinsic challenges of these prediction problems, rather than necessarily indicating limitations specific to OcMAE. Nevertheless, we believe that reporting these results is valuable because they provide useful benchmarks for future studies and may stimulate further research on these challenging biogeochemical prediction tasks.

We will revise the Introduction to more clearly distinguish our contributions from existing masked autoencoders and better articulate the scientific and methodological motivations underlying these design choices.

2. Novelty of the tracer transport equation analysis

Reviewer concern: The tracer transport equation and analysis is a significant part of the paper. It is unclear which parts of these derivations are novel.

Response: We appreciate this important distinction. The tracer transport equation itself and its underlying physical principles are established. We do not claim to derive a new transport equation. Rather, our analysis establishes a connection between masked reconstruction and the transport of ocean biogeochemical tracers, providing a physical interpretation of the learning objective. We will revise the Appendix to explicitly distinguish established physical results from our derivation and interpretation and clarify the novel implications of this connection for masked reconstruction.

3. Cloud-induced missing observations

Reviewer question: Are cloud-induced gaps present in the satellite datasets used in the experiments? Can these be visualized?

Response: Yes. Satellite ocean-color observations are frequently unavailable under cloud cover. We used binning technique to treat missing values as a new category. We will provide a visualization of representative satellite observations showing the observed pixels and cloud-induced gaps.

We thank the reviewer again for these valuable comments, which will help us improve the clarity of the paper and better communicate its contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.