- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The article discusses foundation models for remote sensing data. It is based on the OlmoEarth model, which combines masked latent prediction in the style of a JEPA model and local, fine-grained alignment at the pixel level. The article's main finding is that balancing these losses heavily favors solving the latent prediction problem. In the vanilla setting of OlmoEarth, the local problem gradient is very small compared to the latent one. The authors propose a reweighting strategy and an additional, finer-grained task (the original was only 80 m; they add one to 40 m). They demonstrate an improvement in segmentation tasks with no loss in classification. They also demonstrate that training from scratch is unnecessary and that the balance of the vanilla OlmoEarth can be corrected during training.
Critical strengths
Studying the differences in performance on global versus local tasks on foundation models is important because most studies have shown a preference for global models. This topic is discussed in follow-up articles on major methods, such as DINO v3 or Video JEPa v2.1. The reweighting strategy proposed by the authors seems to provide better balance than the original.
Critical weaknesses
-
The article is difficult to read. From the abstract alone, we are overwhelmed with numbers, comparisons, and technical terms that have not yet been properly introduced. In the abstract, the authors present metric differences in various tasks with different settings that have not been introduced. The article uses technical terms introduced in Section 4. To allow for a smoother read, the article should be rewritten so that the abstract and introduction present the baseline, the problem, and the proposed solution with a few experimental results. The detailed experimental report should be left for later sections.
-
The novelty is slim. The article highlights a real problem with foundation models that has already been demonstrated in DINOV3 and Video-JEPA V2.1. The reweighting strategy is an effective solution, but it is a minor contribution. Since the authors say they were inspired by DINOv3 for their investigation, they should compare their solution with the Gram anchoring proposed in DINOv3.
-
The statistical evidence of improvement is limited. Table 2 shows that it performs similarly to other models with little or no improvement over OlmoEarth on dense tasks, while sometimes performing poorly on classifications. Additionally, the fact that the weights are the result of one seed and cherry-picked across the training to achieve optimal task wise performances, weakens the articles' claims.
Questions for authors
Would it be possible to train the model on several seeds to demonstrate the range of improvements?