- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes GeoSelect to improve data curation processes for large-scale remote sensing self-supervised pre-training. In particular, GeoSelect processes embeddings extracted from an existing foundation model by (1) correcting acquisition-induced variability and (2) applying spatial regularization. Based on these processed embeddings, a fixed number of representative observations is then selected through semantic deduplication. Downstream performance on GeoBench for 5 classification and 5 segmentation tasks show improvements over existing sampling methods from the Major-TOM pre-training dataset when fixing the scale of reduction at 10%.
Critical strengths
- Relevant and interesting research question: improving the quality of RS pretraining datasets while reducing its size, especially in the light of needs for computational efficiency.
- Interesting methodological approach that aims to distinguish meaningful semantic from acquisition diversity when selecting RS pre-training datasets.
- Evaluation across a larger group of classification and segmentation tasks, as well as for S1 downstream tasks.
Critical weaknesses
- “No ocean“ run neither described in the paper nor used in a fair comparison - Please describe it and compare with “Spatial Regul. + SDD (no ocean)“ and “Acq. correc. + SDD (no ocean)“; the results without “No ocean“ are not fully in favor of GeoSelect
- Only one seed used - The differences of the main results in Table 1 are small; I would expect at least three independent seeds per run to demonstrate that GeoSelect is consistently outperforming other methods.
- Limited generality of GeoSelect - Please show the benefits on at least one more S2 pre-training dataset (e.g., SSL4EO-S12 [1]).
- Missing comparison with another relevant sampling strategy: the strategy introduced by Wang et al. [1] for constructing SSL4EO-S12 (which builds upon and improves the sampling strategy of Mañas et al. [2])
- Figure 5 does not consider all stronger methods - I would like to see a comparison of smaller percentages than 10% of the pre-training dataset for (1) random subset baseline, (2) Soil stratified, (3) UniverSat Hier. k-means, (4) UniverSat pHash + Hier. k-m., (5) Spatial Regul. + SDD, (6) Acq. correc. + SDD
- The introduction does not sufficiently well explain GeoSelect (particularly L045-L050) ; Which limitations of existing methods does GeoSelet address, which components are novel, which build on existing techniques?
Moderate weaknesses
- Linear probing still involves hyperparamter tuning for evaluation - I would like to see kNN results for the classification tasks. I am aware that fine-tuning on S2 datasets can lead to very similar results and does not proof representations quality well, therefore I am okay with leaving that experiment out, but kNN is needed.
- The paper does not contain an explicit limitation section
- Inference time to retrieve feature via UniverSat not stated
- Contrastive-like SSL algorithms not considered (e.g., MoCoV2, DINO)
Minor weaknesses
- UniverSat not cited in the first Figure
- When referring to the supplemental material, better explicitly ref the Figures, Tables or Section
- Figure 1 is not really informative
- Linear Probing hyperparameters are missing in experimental details in supplemental material
Formatting
- L014: Line break in abstract; L057: „Contributions“ => our contributions are; References like „Fig. 9“, but its written „Figure 9“ this is inconsistent; Figure 1 never referenced
References
[1] Y. Wang et al. "SSL4EO-S12: A large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation [Software and Data Sets]." IEEE Geoscience and Remote Sensing Magazine 11.3 (2023): 98-106.
[2] O. Manas et al. "Seasonal contrast: Unsupervised pre-training from uncurated remote sensing data.“ IEEE/CVF International Conference on Computer Vision (ICCV), 2021.
Questions for authors
- L136: The author claim that season variability does not reflect meaningful differences. I might disagree. Please justify!
- Burgert et al. [3] show that downstream performance can saturate for pre-training on S2-based dataset at around 100k and 200k samples, how can we distinguish the benefits of the proposed selection strategy from the effects of simply reducing a highly redundant dataset to a more appropriate size?
References
[3] T. Burgert et al. "Rank-based Geographical Regularization: Revisiting Contrastive Self-Supervised Learning for Multispectral Remote Sensing Imagery." IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2026.