- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MIND (Matryoskha Implicit Neural Distillation), an earth embedding method with adjustable spatial scale. To do this, the model distills 4 pretrained geospatial embedding teachers (covering factors like climate, biodiversity or satellite imagery) through nested supervision over embedding prefixes. Coarser dimensions capture broader spatial patterns while later dimensions add finer variation. At inference time, downstream predictors can choose a preselection of dimensions or use the proposed chunked penalty to reduce reliance on the finer-scale chunks.
The paper is further accompanied by a benchmarked called CoordBench, composed of dozens of datasets and targets, evaluated with cross validation across random and regional heldouts. In this benchmark, MIND+CP achieves pareto-optimal performance in both regression and classification; while other experiments in the paper provide evidence that different tasks may benefit differently from different levels of detail in the representation.
Critical strengths
- This paper tackles a key problem in a growing field of Earth AI, which is the level of detail in geospatial embeddings. I find both the topic and the approach to be highly timely and relevant.
- The method is useful, well-designed, and practical. MIND provides an easy and cost effective way to adjust granularity after pretraiining, which is supported by truncation, variograms, controls and regularisation experiments.
- THe Chunked penalty improves regional prediction while preserving performance on random folds.
- The proposed benchmark is also a highlight in this paper, being very comprehensive in both datasets and tasks, and highly robust in how heldout performance is measured. This benchmark can be a highly-used resource for the future.
Critical weaknesses
- I found it hard to exactly point to what is making the model work. MIND distills 4 geospatial embeddings, but the AEF contribution is difficult to measure as it is excluded from the concatenation baselines. It would be nice to strenghen the experimental setup on this regard, to fully isolate and understand the influence of each teacher under every experiment.
- The proposed benchmark may be biased towards the MIND embedding. Particularly, many tasks are centered around cities, which could match the MIND's training coordinates. Other INR may not have this bias. This should be studied further and be clarified by the authors.
- Most INR use 256-512 dimensional embeddings, while MIND goes way beyond this. This makes it difficult to compare exactly with other lower-dimensional embeddings. The fisher information (eg Measuring the intrinsic dimension of Earth representations, ICLR 2026) could be reported for better analyses. This warrants further analysis and discussions.
Questions for authors
- How many MIND encoders were trained, and what is the variability across encoder seeds?
- How much of the improvement depends on AEF?
- The discussion could consider TerraNova: A Foundation Model for the Anthropocene, which aligns a shared representation with a set of pretrained geospatial embeddings alongside environmental and societal supervision.
2 comments
Response to rebuttal
I thank the authors for the detailed response to my review and for the rebuttal to the other reviews, which mentioned limitations that I missed, like missing analyses of related work. These rebuttals have substantially addressed my original concerns and increased my confidence about recommendign acceptance.
The analyses of the teacher ablations, the comparisons with concatenated teachers, smaller embedding dimensions, Terranova, and the rural and non-US evaluations are all signfiicant, very informative, and relevant, and suggest that MIND is likely the SOTA geospatial embedding across several dimensions.
Some limitations still remain, including variation across task categories, but the authors are candid about these points. I would also suggest to strenghen the analysis of the embedding distillation in MIND compared to what was done in Terranova, and where the differences lie, beyond the actual embeddign choices that were used for distillation.
I find the paper to be very interesting and useful and the evaluation to be unusually thorough and complete, and I'll be happy to keep or increase my rating. I'm also interested in knowing the thoughts of the other reviewers.
Rebuttal by Authors
Thank you for the careful review and for pointing us to TerraNova and the intrinsic-dimension analysis. We have added both, along with the teacher and sampling comparisons you requested.
We used one released encoder for all main results which can be found at the anonymous Hugging Face link in the Reproducibility Statement. During evals we vary the folds across 5 seeds while keeping the encoder frozen. To check pretraining variability we reran pretraining with two more random seeds. For MIND+CP the variation across the three runs is smaller than across the five fold assignments at every regional block size. Notably, we found that our released encoder is also the weakest of the three runs with the two new runs producing even stronger results for MIND.
We retrained MIND without AEF and random-fold R2 changes by 0.001 while random-fold land-cover accuracy is nearly unchanged. We evaluate across 5 seeds and accuracy changes by at most 0.6 points and R2 by at most 0.034 across the block sizes. This suggests that AEF itself isn't the main source of MIND's performance.
Thanks for pointing us to this paper. We added both TerraNova spatial and spatiotemporal encoders as baselines to Table 3 and cite it in Sec. 2. Its supervision includes variables that match or proxy 36 of CoordBench's 78 targets (see Table 20). MIND+CP and MIND-256 score above both TerraNova encoders on both metrics (e.g. 0.184 vs. 0.028 R2 at 40° for MIND+CP).
We added this comparison in App. G. On regression, AEF scores below MIND+CP at every block size (0.287 vs. 0.356 R2 at 10°), while AEF and TESSERA score higher on 13 of the land-cover datasets directly from AEF's evaluation suite.
Table 12 compares MIND+CP with each teacher alone, with the teachers concatenated (with and without AEF), and with a PCA of the four teachers. The concatenation that includes AEF has higher land-cover accuracy and higher R2 under random folds and 2° blocks. MIND+CP has higher R2 when the held-out regions are 10° or larger (0.356 vs. 0.291 at 10°).
We also retrained MIND leaving out one teacher at a time (see Q2). Without Climplicit, GeoCLIP, or SINR, random-fold R2 drops by 0.029–0.038. Without AEF, random-fold R2 changes by 0.001 and land-cover accuracy by at most 0.6 points. The Chunked Penalty improves regional R2 for every leave-one-out teacher set at all block sizes.
This is a fair concern. CoordBench is not sampled from MINDSET, and many of its datasets are global grids or station networks (e.g. WorldClim, SoilGrids, SatCLIP, air temperature). In Table 17 we now list each dataset's coverage and spatial units. To test the effect of city-centered sampling, we added an evaluation on rural points, more than 50 km from any city of at least 15,000 people (App. J, Table 14). MIND+CP scores highest on these rural points compared to SINR, GeoCLIP, and Climplicit (0.895 R2 and 94.4% accuracy), even though MINDSET is sampled around cities.
Truncated to 128 dimensions, MIND scores 0.327 R2 at 10°, above every baseline at its full width; the best is GeoCLIP at 512 dimensions with 0.303. Under random folds, MIND-256 scores 0.556 R2, above all six 256-dimensional baselines (at most 0.546 for TerraNova-S; Table 3).
We followed the approach of Rao et al. (ICLR 2026) and report 5 intrinsic-dimension estimators and the participation ratio in App. K. We found the largest difference between encoders is in the participation ratio which counts the linear directions an embedding uses. At the same width, MIND’s is the highest, with 64 at 256 dimensions vs. at most 43 for the six baselines of the same width, 123 vs. 63 for GeoCLIP at 512, and 244 vs. 89 for Climplicit at 1024. The participation ratio is also correlated with higher CoordBench scores (Spearman 0.78–0.84). In contrast, we found FisherS is about 10 or less for every encoder regardless of its embedding dimension.