- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Ocean Masked Autoencoders (OcMAE), a self-supervised learning framework for ocean biogeochemistry prediction using large-scale satellite observations. The method combines quantile-based feature discretization with masked autoencoder pretraining to learn spatiotemporal representations while handling missing values. The authors also provide a theoretical interpretation connecting masked reconstruction to ocean physical processes. Experiments on nine downstream prediction tasks, along with ablation studies, show that OcMAE outperforms traditional machine learning and tabular baselines on four tasks.
Critical strengths
1.Meaningful Problem and Practical Significance: The paper addresses limited labeled data and missing observations in ocean biogeochemistry. Using large-scale satellite data for self-supervised learning is well-motivated and practically relevant.
2.Large-Scale Data Preparation: The authors process satellite observations from 2003–2023 to construct a pretraining dataset with over 500 million spatiotemporal image cubes, providing a useful foundation for ocean representation learning.
Critical weaknesses
-
Insufficient Comparison with Relevant Baselines: The comparisons mainly focus on traditional machine learning and tabular models, while a standard MAE adapted to the same setting is missing. Domain-specific models such as OXYGENERATOR could also provide relevant comparisons for oxygen-related tasks. This makes it difficult to assess the advantages of OcMAE over existing approaches.
-
Limited Methodological Novelty: The method mainly combines existing discretization and masked autoencoder techniques, while the theoretical analysis does not rigorously establish that masked reconstruction learns the underlying physical dynamics.
Questions for authors
-
Additional Baselines: Could the authors include comparisons with a standard MAE adapted to the same setting and relevant domain-specific models, such as OXYGENERATOR on applicable tasks, to better establish the advantages of OcMAE?
-
Analysis of Underperforming Tasks: Could the authors provide further analysis of why OcMAE fails to improve over existing baselines on several tasks and discuss the factors that may limit its effectiveness?
-
Ablation Studies: Could the authors provide ablations that isolate the contributions of self-supervised pretraining and quantile-based discretization to the overall performance?
3 comments
Thank you for the detailed response. The clarification regarding OXYGENERATOR's different task setting and the more modest interpretation of the theoretical analysis are helpful. The discussion of task-dependent performance is also reasonable.
However, my main concern about methodological novelty remains, as the framework primarily combines established discretization and masked autoencoder techniques. Moreover, the additional MAE comparisons and ablations are mentioned, but their numerical results are not provided in the response, making their contributions difficult to assess. Overall, while the response addresses some of my concerns, I maintain my original rating.
Response to the reviewer 3
We thank the reviewer for the insightful and specific feedback. We have included the additional experimental results (NO_Pre and NO_Bin) in both Table 1 and Table 8. These additional results will also be incorporated into the revised manuscript.
Table 1. The prediction accuracies (coefficient of determination, ) of CO$_2$, O$_2$ concentration, air-sea O$_2$ flux (O$_2$FLUX), NCP, and CR derived by different model variants.
| Model | pCO₂ | O₂ | O₂ flux | NCP | CR |
|---|---|---|---|---|---|
| Base | 0.662 ± 0.009 | 0.971 ± 0.000 | 0.379 ± 0.004 | 0.258 ± 0.036 | 0.260 ± 0.026 |
| BIN32 | 0.637 ± 0.016 | 0.970 ± 0.000 | 0.335 ± 0.012 | 0.145 ± 0.036 | 0.244 ± 0.048 |
| BIN256 | 0.655 ± 0.013 | 0.971 ± 0.001 | 0.371 ± 0.007 | 0.265 ± 0.024 | 0.297 ± 0.018 |
| BIN512 | 0.668 ± 0.011 | 0.971 ± 0.001 | 0.366 ± 0.020 | 0.174 ± 0.050 | 0.290 ± 0.020 |
| BIN2048 | 0.663 ± 0.013 | 0.971 ± 0.001 | 0.367 ± 0.014 | 0.189 ± 0.023 | 0.201 ± 0.029 |
| L4 | 0.640 ± 0.013 | 0.969 ± 0.002 | 0.353 ± 0.010 | 0.282 ± 0.023 | 0.289 ± 0.010 |
| L8 | 0.641 ± 0.024 | 0.970 ± 0.001 | 0.369 ± 0.011 | 0.305 ± 0.018 | 0.218 ± 0.027 |
| L32 | 0.669 ± 0.005 | 0.972 ± 0.000 | 0.387 ± 0.007 | 0.297 ± 0.022 | 0.198 ± 0.020 |
| D32 | 0.629 ± 0.018 | 0.970 ± 0.000 | 0.361 ± 0.006 | 0.293 ± 0.012 | 0.228 ± 0.020 |
| D128 | 0.649 ± 0.019 | 0.971 ± 0.001 | 0.360 ± 0.011 | 0.258 ± 0.019 | 0.246 ± 0.009 |
| S5 | 0.622 ± 0.008 | 0.970 ± 0.001 | 0.351 ± 0.008 | 0.332 ± 0.008 | 0.238 ± 0.030 |
| S7 | 0.617 ± 0.009 | 0.969 ± 0.001 | 0.348 ± 0.012 | 0.289 ± 0.040 | 0.258 ± 0.017 |
| T2 | 0.631 ± 0.017 | 0.970 ± 0.001 | 0.360 ± 0.023 | 0.128 ± 0.092 | 0.265 ± 0.026 |
| T5 | 0.638 ± 0.023 | 0.970 ± 0.001 | 0.351 ± 0.010 | 0.212 ± 0.041 | 0.232 ± 0.029 |
| PD_few | 0.634 ± 0.017 | 0.962 ± 0.000 | 0.262 ± 0.007 | 0.310 ± 0.020 | 0.245 ± 0.026 |
| PD_0.3 | 0.643 ± 0.015 | 0.972 ± 0.000 | 0.372 ± 0.006 | 0.273 ± 0.010 | 0.244 ± 0.040 |
| PD_Reg | 0.641 ± 0.010 | 0.970 ± 0.001 | 0.327 ± 0.012 | 0.245 ± 0.009 | 0.126 ± 0.008 |
| NO_Pre | 0.579 ± 0.026 | 0.961 ± 0.001 | 0.238 ± 0.014 | -0.009 ± 0.020 | 0.092 ± 0.013 |
| NO_Bin | 0.602 ± 0.011 | 0.970 ± 0.001 | 0.363 ± 0.008 | 0.202 ± 0.014 | 0.023 ± 0.012 |
Table 8. The prediction accuracies (coefficient of determination, ) of GPP, NPP, nitrification, and N$_2$ fixation rate (N$_2$FIX) derived by different model variants.
| Model | GPP | N₂ fix | NPP | Nitrification |
|---|---|---|---|---|
| Base | -0.047 ± 0.056 | -0.050 ± 0.020 | 0.008 ± 0.024 | 0.046 ± 0.045 |
| BIN32 | 0.018 ± 0.056 | -0.053 ± 0.021 | 0.023 ± 0.017 | 0.049 ± 0.045 |
| BIN256 | -0.044 ± 0.053 | -0.053 ± 0.021 | 0.012 ± 0.023 | 0.049 ± 0.044 |
| BIN512 | 0.042 ± 0.008 | -0.052 ± 0.021 | 0.010 ± 0.024 | 0.047 ± 0.045 |
| BIN2048 | -0.032 ± 0.043 | -0.054 ± 0.022 | 0.007 ± 0.025 | 0.048 ± 0.044 |
| L4 | -0.055 ± 0.022 | -0.014 ± 0.044 | 0.007 ± 0.025 | 0.050 ± 0.045 |
| L8 | 0.158 ± 0.123 | 0.031 ± 0.010 | 0.006 ± 0.014 | 0.049 ± 0.044 |
| L32 | -0.051 ± 0.050 | 0.014 ± 0.020 | 0.009 ± 0.024 | 0.046 ± 0.045 |
| D32 | -0.051 ± 0.020 | -0.593 ± 0.074 | 0.008 ± 0.023 | 0.048 ± 0.045 |
| D128 | -0.057 ± 0.023 | 0.064 ± 0.012 | 0.006 ± 0.015 | 0.048 ± 0.045 |
| S5 | -0.052 ± 0.020 | -0.125 ± 0.042 | 0.008 ± 0.024 | 0.046 ± 0.045 |
| S7 | -0.052 ± 0.021 | 0.055 ± 0.031 | 0.005 ± 0.015 | 0.044 ± 0.046 |
| T2 | -0.062 ± 0.042 | -0.053 ± 0.021 | 0.014 ± 0.022 | 0.048 ± 0.045 |
| T5 | 0.179 ± 0.037 | 0.229 ± 0.033 | -0.047 ± 0.007 | 0.113 ± 0.034 |
| PD_few | 0.073 ± 0.037 | 0.031 ± 0.047 | 0.017 ± 0.023 | 0.050 ± 0.044 |
| PD_0.3 | -0.053 ± 0.021 | 0.020 ± 0.006 | 0.011 ± 0.023 | 0.047 ± 0.045 |
| PD_Reg | -0.053 ± 0.021 | -0.598 ± 0.078 | 0.019 ± 0.023 | 0.050 ± 0.045 |
| NO_Pre | -0.555 ± 0.152 | -0.183 ± 0.086 | 0.004 ± 0.018 | 0.092 ± 0.013 |
| NO_Bin | -0.578 ± 0.015 | -0.590 ± 0.006 | -0.003 ± 0.012 | 0.054 ± 0.002 |
| LR | -0.755 ± 11.30 | -0.292 ± 0.470 | -0.023 ± 0.055 | -0.305 ± 0.153 |
| RF | -0.083 ± 0.151 | -0.063 ± 0.074 | 0.313 ± 0.041 | -0.058 ± 0.119 |
| XGB | -0.183 ± 0.178 | -0.004 ± 0.104 | 0.264 ± 0.064 | -0.088 ± 0.182 |
| MLP | -0.431 ± 0.357 | -0.063 ± 0.487 | -0.609 ± 1.123 | -0.179 ± 0.072 |
| TabPFN | -0.108 ± 0.134 | -0.267 ± 0.093 | 0.376 ± 0.049 | 0.021 ± 0.078 |
Response to the reviewer 3
We thank the reviewer for the thoughtful and constructive feedback. We address the concerns regarding baselines, methodological novelty, task-dependent performance, and ablation studies below.
1. Comparison with standard MAE and domain-specific models
Reviewer concern: The evaluation lacks a comparison to standard MAE baseline and domain-specific models such as OXYGENERATOR.
Response: We have additionally evaluated a standard MAE (NO_Pre and NO_Bin in Table 1) using the same pretraining dataset, model architecture, and downstream evaluation protocol. The results demonstrate that both large-scale pretraining and discretization contribute to improved predictive performance.
Regarding OXYGENERATOR, we appreciate this suggestion and will clarify its applicability to our downstream prediction tasks. OcMAE focuses on predicting surface oxygen concentrations, whereas OXYGENERATOR was developed 3D oxygen reconstruction. Surface oxygen concentrations are well constrained by solubility. This is also reflected in the high accuracy achieved by linear regression and other algorithms, suggesting that surface oxygen provides limited discrimination among model architectures. We include surface oxygen prediction to primarily establish a baseline for other downstream tasks. Our main interest lies in more challenging biogeochemical prediction tasks, e.g., biological productivity, where the underlying processes are more complex. We will clarify these distinctions in the revised manuscript.
2. Methodological novelty and theoretical interpretation
Reviewer concern: OcMAE combines existing techniques, and the theoretical analysis does not rigorously establish that masked reconstruction learns physical dynamics.
Response: Quantization and masked autoencoders are established techniques. Our contribution is to develop and evaluate a self-supervised representation learning framework for ocean biogeochemistry, by leveraging large-scale satellite observations with missing values accommodated by including a new category after quantization.
Our theoretical analysis provides a physical interpretation connecting masked reconstruction with tracer conservation equation, rather than establishing a general guarantee of physical-dynamics recovery.
To summarize, our key contributions are:
(1) large-scale pretraining on global satellite ocean-color observations, using discretization of continuous observations to accommodate missing values and nonlinear relationships;
(2) comprehensive evaluation across nine ocean biogeochemistry prediction tasks against strong baselines;
(3) an analysis connecting masked reconstruction to the tracer transport equation, providing a physical interpretation of the learned representations.
3. Analysis of underperforming downstream tasks
Reviewer concern: OcMAE does not improve over existing baselines on several tasks. Further analysis of these cases would be valuable.
Response: We agree that OcMAE improves upon existing baselines on four of the nine downstream tasks.
We would like to emphasize that several of these tasks, particularly GPP, CR, nitrification, and denitrification, are especially challenging because of the limited availability of labeled observations, with only a few hundred training samples across the global ocean for each task (Table 4 in paper). Furthermore, these biogeochemical processes involve complex underlying mechanisms and substantial spatial and temporal variability. The relatively modest performance across different models highlights the difficulty of these prediction problems rather than necessarily reflecting limitations specific to OcMAE. Nevertheless, we believe that reporting these results are valuable, as they provide useful benchmarks for future studies and may stimulate further research on these challenging biogeochemical prediction tasks.
We hypothesize that differences in model performance reflect variations in the underlying prediction problems, including the availability and quality of labeled observations, spatial and temporal scales, the strength of relationships between satellite data, including Chl, SST, and PAR, and target variables, and the extent to which pretrained representations transfer across tasks. We will expand the discussion of these results in the revised manuscript.
4. Ablation studies isolating pretraining and discretization
Reviewer concern: Ablations should isolate the contributions of self-supervised pretraining and quantile-based discretization.
Response: We have additionally evaluated a standard MAE (NO_Pre and NO_Bin in Table 1) using the same pretraining dataset, model architecture, and downstream evaluation protocol. The results demonstrate that both components contribute to improved downstream prediction performance.
We thank the reviewer again for these valuable comments, which will help us improve the clarity of the paper and better communicate its contributions.