acceptodds
Under review as a conference paper at ICLR 2027

Anchored Latent Predictions for Earth Foundation Models

Abstract

Self-supervised Earth-observation encoders learn scene context, but mapping clouds and floods also requires local detail. In two runs of a latent-prediction recipe with weak pixel-target weights, extending training from 10B to 40B tokens cost 5.6 and 5.7 points of frozen-probe cloud mIoU; thin-cloud IoU fell by 28 to 34%, and crop-type and flood segmentation also degraded. Meanwhile, land-cover and GEO-Bench accuracy peaked at the worst cloud checkpoints, and feature rank rose. Better scene recognition can therefore hide a loss of spatial detail, motivating fixed pixel-level targets that keep evolving representations tied to the observations. We propose anchored latent prediction (ALP): complement evolving latent targets with strongly weighted, frozen random projections of the pixels, predicted at twice the token resolution with no extra encoder cost. In single-run tests, switching to ALP’s loss weights, including a higher cloud-loss weight, halted the decline; an ALP run kept improving until about 20B tokens. EarthFM, a 91M-parameter en- coder trained with ALP plus map and cloud-model targets, is on par with or better than OlmoEarth v1.2 Base on six of eight public test sets from GEO-Bench, GEO-Bench-2, and Sen1Floods11, with 1.8 to 3 times less hardware-adjusted pretraining compute: ahead on clouds, within noise on five tasks, and behind on EuroSAT and So2Sat. Its shorter 18.75B-token branch leads on clouds with a third of OlmoEarth’s compute (+1.15 mIoU, 95% CI 0.53 to 1.79); only TerraMind-L scores higher, with 7.2 times as much compute.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Earth observation, foundation models, remote sensing, representation learning, multimodal learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_6wnQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The article discusses foundation models for remote sensing data. It is based on the OlmoEarth model, which combines masked latent prediction in the style of a JEPA model and local, fine-grained alignment at the pixel level. The article's main finding is that balancing these losses heavily favors solving the latent prediction problem. In the vanilla setting of OlmoEarth, the local problem gradient is very small compared to the latent one. The authors propose a reweighting strategy and an additional, finer-grained task (the original was only 80 m; they add one to 40 m). They demonstrate an improvement in segmentation tasks with no loss in classification. They also demonstrate that training from scratch is unnecessary and that the balance of the vanilla OlmoEarth can be corrected during training.

Critical strengths

Studying the differences in performance on global versus local tasks on foundation models is important because most studies have shown a preference for global models. This topic is discussed in follow-up articles on major methods, such as DINO v3 or Video JEPa v2.1. The reweighting strategy proposed by the authors seems to provide better balance than the original.

Critical weaknesses
  1. The article is difficult to read. From the abstract alone, we are overwhelmed with numbers, comparisons, and technical terms that have not yet been properly introduced. In the abstract, the authors present metric differences in various tasks with different settings that have not been introduced. The article uses technical terms introduced in Section 4. To allow for a smoother read, the article should be rewritten so that the abstract and introduction present the baseline, the problem, and the proposed solution with a few experimental results. The detailed experimental report should be left for later sections.

  2. The novelty is slim. The article highlights a real problem with foundation models that has already been demonstrated in DINOV3 and Video-JEPA V2.1. The reweighting strategy is an effective solution, but it is a minor contribution. Since the authors say they were inspired by DINOv3 for their investigation, they should compare their solution with the Gram anchoring proposed in DINOv3.

  3. The statistical evidence of improvement is limited. Table 2 shows that it performs similarly to other models with little or no improvement over OlmoEarth on dense tasks, while sometimes performing poorly on classifications. Additionally, the fact that the weights are the result of one seed and cherry-picked across the training to achieve optimal task wise performances, weakens the articles' claims.

Questions for authors

Would it be possible to train the model on several seeds to demonstrate the range of improvements?

Reviewer_dyxrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies an empirical problem that continued pre-training can improve the scene-level representations but degrading the spatial information needed for dense prediction tasks, in the field of self-supervised earth observation models. Accordingly, the paper proposes EarthFM, which employs Anchored Latent Prediction (ALP) to preserve spatial information by combining latent prediction with fixed pixel-level targets. The experimental results show that the proposed method only outperforms the baselines on only one of eight datasets.

Critical strengths
  1. The paper provides good empirical evidence of spatial-information degradation, demonstrating a consistent decline in cloud segmentation performance across two independent pretraining runs.

  2. The authors clearly acknowledges about the limitations, negative results and the experimental constraints.

Critical weaknesses
  1. While the studied topic is interesting, the presentation requires substantial enhancement, and the novel aspects of the work appear somewhat incremental. The research problem regarding spatial information degradation is not clearly described. Although Section 3 demonstrates spatial-information degradation within the proposed method’s training recipe, it does not establish the generality of this problem across existing methods. In addition, the proposed method combines many existing techniques (loss weights, teacher architecture, data sampling, etc), but the overall innovation and individual contributions are not demonstrated.

  2. The experimental results cannot sufficiently support the main claims. While the paper the paper repeatedly says that “ EarthFM is on par or better than OlmoEarth v.12 Base on six of eight test sets”, EarthFM only outperforms OlmoEarth on 2 datasets and underperforms other the baselines on most of the datasets (see Table 2). In addition, the main cloud segmentation test set is repeatedly evaluated during training the model, this may cause potential test set leakage.

Questions for authors
  1. In the literature, there are numerous pre-training approaches for Earth foundation models, especially in the latent prediction (Galileo [2], AnySat[3], DINO[4]) what are their limitations, beyond OlmoEarth? And how does the proposed model address?
  2. In the experiment, why the encoders are frozen for segmentation task and fine-tuned for the other tasks. How are the full comparison results as shown in the Table 1 from OlmoEarth [1] paper?

[1] OlmoEarth v1.2: A more efficient family of OlmoEarth models. https://arxiv.org/pdf/2605.20804

[2] Galileo: Learning Global & Local Features of Many Remote Sensing Modalities, ICML 2025

[3] AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities, CVPR 2025

[4] DINOv3, https://arxiv.org/abs/2508.10104

Reviewer_XH5rOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Anchored Latent Predictions (ALP). Supposedly is better and cheaper to pretrain. Who knows but there's no code, data, model checkpoints made available and the paper reads almost entirely LLM written so I'm incredibly doubtful of the validity of this work.

Critical strengths

N/A

Critical weaknesses
  • The authors change many non-architectural hyperparameter choices which conflates the gains of the method they claim causes these small gains. Looking at Table 2 many of the gains aren't statistically significant. Further the authors focus on tuning their own method while not doing the same for methods they compare to. This is a classical flaw pointed out in Corley et al. [1].
  • Looking at the results in Table 2 EarthFM isn't statistically significantly better on the majority of tasks used during eval. The only one it is better on (barely) is the cloud segmentation dataset which is not worth claiming as a win because the author's proposed method pretrains on cloud labels.
  • The paper contains very LLM-like language e.g. section headers like 'Evidence that it works.', 'What it buys', "The Problem: Longer Training, Worse Maps", "we present ALP second because it answers the problem, not because it came first.", and much more in the prose including numerous LLM tells like semicolons/colons throughout. This paper needs a significant rewrite and revision because it currently reads like 90% AI-generated without any pass by a human.

[1] https://neurips.cc/virtual/2026/loc/atlanta/poster/140061

Questions for authors

Authors should rewrite the entire paper since it reads as incredibly LLM generated. It is hard to follow what the purpose of the paper is and then when you look at the results and see that it barely outperforms any other datasets which could be within noise of all the training parameters they change so I'm not sure why a 20 page ICLR submission necessary for this or all the hundreds of B200 GPU hours the paper claims it used. The authors should also rerun experiments isolating each change they make and provide fair comparisons to prior work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.