- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The key contributions behind the paper are a large-scale scaling law study of pixel-wise Sentinel-1/2 foundation models trained with Barlow Twins. They find that downstream task accuracy is weakly correlated with pre-training loss. Second, the authors release embeddings from their new TESSERAv2 model trained with MRL, showing that very small prefix sizes is still viable with >90% task accuracy.
Critical strengths
The large-scale sweep for their proposed family of models is well done, detailed and thorough. The experimental rigor is very good and extends to most of this paper. The appendix ablations cover a very large set of lingering questions and is well organized.
The largest strength of the work is its impressive engineering contributions and the released data product, which can help the ML for EO community. This final product is easy to use (embedding), and practical for a lot of use cases as highlighted in the authors' evaluations.
Critical weaknesses
I see practical value in the proposed model. But at the moment, I am not convinced that the paper provides any clearly novel contribution to the representation learning for EO community. My weaknesses are organized below by the two main contributions of the paper.
Scaling Laws
While not every scaling law study has to be broad and widely applicable like [1,2], I see the immediate beneficiaries of a (i) Sentinel-1/2 (ii) pixel-wise (ii) Barlow-twins (iv) embedding-as-a-product scaling law analysis to be the authors of the paper themselves. The novel conclusions from the scaling law analysis that would carry over to other EO models is unclear to me. My comments on the scaling law takeaways:
-
SSL pre-training loss doesn't reliably predict downstream performance: This is widely known and well-established. [3,4], for example come to mind. However, I still see some value in quantifying this for EO. But why the mismatch occurs in this family of specific models or what makes this finding different from standard SSL paradigms remains unclear. The authors hypothesize about this in the appendix a little, but these experiments are not isolated and are not conclusive.
-
The scaling law analysis helps to choose a model by fixing a downstream performance target, saving 2-5x compute: This makes sense to me although model selection with supervision on downstream tasks is not particularly novel to this paper. Furthermore, the model chosen with the highest downstream score cannot score worse on that same measure than any pretraining loss selected model. My understanding is that the authors both select the model with the lowest pretraining loss, and the model with the highest downstream (composite) score. This composite is defined on 15 tasks. Using this downstream task rule, the large teacher is trained and evaluated on 14 more held-out tasks. I believe an essential missing comparison is how the model selected by minimizing pretraining loss would do on these same 14 held-out tasks. This isn't reported. Comparing both selection rules on those held-out tasks, at matched pretraining budgets, would establish whether the advantage generalizes and provide stronger support for the claimed compute savings.
MRL embeddings and the proposed compact student models.
- To the best of my knowledge, I don't believe the proposed distillation to MRL embedding approach is novel. [5], for example is the multi-teacher variant of the exact distillation contribution proposed in this paper. [5] already trains nested student prefixes through separate MLP heads to match the full frozen teacher embeddings using a cosine loss. The proposed distillation approach is [5] reduced to a single teacher. I would encourage the authors to cite related work on nested distillation, and discuss (if any) new insights or new methods they bring to embedding distillation into prefixes.
Evaluation
-
Evaluating the proposed models on more standard benchmarks such as BigEarthNet, So2Sat, EuroSAT is important, and not performed in the paper. I understand that the model is a pixel-level embedding model, in which case, direct comparisons to image/patch-based models may be murky.
-
Data volume mismatch: The authors note that the v1 of the proposed model samples a fixed 40 timesteps while v2 retains all valid observations. Compared to this, Prithvi accepts only 3 acquisitions. This massive difference in input types and scales makes me question the claimed improvements. These could be due to data volume and not due to encoder architecture, proposed distillation approach, etc. While the authors acknowledge this, a matched-observation comparison could be useful.
Writing and figures
I generally find the figures to be dense and overcrowded, and would prefer if the authors modified their figures to better align with the core findings and contributions stated at the end of the introduction. Figure 2 contains most of the evidence, but the several curves, small insets, annotations, and callouts are very distracting. There is a fair bit of redundancy in the figures, and I encourage the authors to significantly trim down their figures into necessary componenets only. For example, figure 1(f) contains two radar plots and two bar charts in one overview panel. Even inside the radar plots which cover 15 tasks and several overlapping model curves, the embedding dimension of the same model are also added. Pruning would help future readers significantly. Figure 1c and 3b are both on the same topic. Figure 1c and 4d both cover the MRL embeddings again. The small encoder vs competing products figures are in figure 1f, 4c, and 4e. I appreciate the volume of results and thorough evaluation performed in the paper, but I believe that not all of this needs to be visually communicated multiple times in the main text.
Writing is verbose and can be simplified. Phrases like: "global surface state" (intro), "adaptivity along two axes" (intro), "projector held near its optimum as a training scaffold" (S3), "we therefore treat the frozen teacher... as a fixed representation distribution" (S4) can be modified to clear alternatives.
Evaluation
Given my concerns around the (i) lack of novelty surrounding the primary contributions made in the paper, (ii) discrepancies surrounding evaluation, and (iii) presentation issues with the figures, I recommend rejection at this time. I encourage the authors to clarify/push-back on my novelty concerns and other concerns whenever applicable. I am very open to increasing my score during the rebuttal stage.
[1] Hoffmann, Jordan, et al. "Training compute-optimal large language models." arXiv preprint arXiv:2203.15556 (2022).
[2] Kaplan, Jared, et al. "Scaling laws for neural language models." arXiv preprint arXiv:2001.08361 (2020).
[3] Lourie, Nicholas, Michael Y. Hu, and Kyunghyun Cho. "Scaling Laws Are Unreliable for Downstream Tasks: A Reality Check." EMNLP (Findings). 2025.
[4] Liu, Hong, et al. "Same pre-training loss, better downstream: Implicit bias matters for language models." International Conference on Machine Learning. PMLR, 2023.
[5] Grashei, Christian, et al. "Pathryoshka: Compressing pathology foundation models via multi-teacher knowledge distillation with nested embeddings." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
Questions for authors
-
I'm curious if the pretraining losses in the scaling law analysis are actually comparable across different projector widths. I believe Barlow twins' loss is computed on the projector outputs. Therefore, wouldn't the width changes change both the capacity of the network and the number of terms in the loss? I think the loss normalization/rescaling done in the paper doesn't automatically make different projector widths an apples-to-apples comparison of encoder quality.
-
Does the model selected change depending on what downstream tasks are used? Could the authors also comment on the correlation between the tasks used for model selection and the tasks used for evaluation?
-
I am still unclear as to who the beneficiaries of the smaller student model are (the main motivation behind distillation). In section 4, the authors claim
Most users cannot run a 2 B pixel-wise encoder over global Sentinel-1/2 even once, let alone recurrently.
But the authors also assert
We expect users to consume our embeddings as data, that is, as precomputed annual pixel embeddings with lightweight task heads, without running or fine-tuning a backbone.
While I agree with the former claim most users cannot run a 2B pixel-wise encoder globally on S1/S2, Wouldn't the computational cost of a smaller student model also be intractable to generate embeddings on a global scale, especially since both models require the same large-scale Sentinel-1/2 inputs? I see that the appendix mentions a 6% preparation overhead which is promising, but further details on what the preparation entails would be helpful. For example, I assume that the storage cost associated with downloading the inputs to generate a global embedding database to be identical (and intractable) regardless of the size of the model.
- I would also appreciate the authors' insights on the true computational costs of their approach. While the smaller student model could indeed reduce GPU inference costs, how does this compare to the storage + ingress + egress costs associated with storing global embeddings for the embedding-as-data framework, especially over time? Recent discussions in EO have framed this cost as the primary bottleneck, since even with a smaller model and embeddings, storage costs are not scalable long-term.
5 comments
Response to Reviewer E5SA (2/4)
W3 (continued). You are right that the in-sample gap cannot be negative. We ran the comparison you asked for (Appendix Z.1, Table 30, Figure 2e). In each of the 9 compute buckets we took the score-selected and the loss-selected run and evaluated both on the 14 held-out tasks. The score-selected run is better in 8 of 9 buckets, though in one of them only by 0.002. In the remaining bucket the loss-selected run is ahead by 0.006. The mean gap is 0.033 (95% bucket bootstrap 0.018 to 0.047), or about 2.4x compute using the held-out score/compute slope, against 0.049 on the 15 selection tasks.
We keep the 2 to 5x of the submission as a summary of the in-sample 3.5x (bootstrap 2.6 to 5.5x) and the out-of-sample 2.4x. Section 3.2 now shows how the 3.5x is computed (Appendix G, Equations 25 to 27) and reports the held-out value. The checkpoints in this test come from the sweep, so the test covers the rule within the sweep range. We did not train a loss-selected model at the 2B budget.
W5. Thank you for anticipating the difficulty, and we agree with your reading. Our encoder takes a full year of Sentinel-1/2 observations per pixel. BigEarthNet, So2Sat and EuroSAT are single-date patch benchmarks designed for patch-based models, so a comparison on them would mostly measure the difference in input format rather than representation quality. We have therefore not added them. The closest time-series benchmarks we know of are the three Sentinel time-series tasks of GEO-Bench-2 (PASTIS-R, BioMassters, TreeSatAI-TS), and they are already in our held-out suite (Appendix M).
W6. The headline comparison is between products with different inputs, and Section 5.1 ("Input volume") now says so. To separate the encoder from input volume we matched the number of observations on one task, Austrian Crop, where the Tessera v1 paper already reports v1 with Sentinel-2 observations capped at 1 to 60 per pixel. We used that protocol for v2 and Prithvi (Appendix Z.4, Table 33, Figure 11). At every cap v2 is ahead of v1, by 0.085 at 5 observations and about 0.02 from 40 observations onwards (0.019 at 60), so with the same input v2 is at least as good as v1. With 10 or fewer observations both versions vary considerably across random draws of the dates, and v2 varies slightly less than v1 (Figure 11 shows the 5th to 95th percentile bands). Prithvi is a patch model, so we capped whole scenes and used only scenes with less than 50% cloud cover. There are at most 38 such scenes a year, so its curve stops at 30. Its score stays between 0.23 and 0.31 up to that point, and with three scenes, its usual input, it scores 0.262 against 0.402 for v2 with three observations. This test uses one task, and AlphaEarth cannot be rerun on a different input.
W7. Figure 1 no longer has the Matryoshka panel (old 1c), so the prefix schematic appears once (Figure 3b) and the prefix result once (Figure 4d). The later panels of Figure 1 move up one letter. In Figure 2 we removed the 2 to 5x box and added a panel e that shows, per bucket, how far the score-selected run is ahead of the loss-selected run on the selection tasks and on the held-out tasks. The allocation fits and the extrapolation check move to 2f and 2g. The captions of Figures 4b and 4d state the normalization. We kept the radar panel (old 1f, now Figure 1e) as a compact overview, and Figures 4c and 4e give the full comparison. We rewrote the four phrases you listed, and the Figure 1b inset no longer says that loss selection wastes compute.
Response to Reviewer E5SA (3/4)
Q1. Per entry, yes, and for encoder quality we also checked comparability directly, as described below. The divisor is the loss when every entry of the cross-correlation matrix is off by one, so the normalized loss is a per-entry mean , where and are the mean squared on- and off-diagonal errors and . This removes the extra entries of a wider projector, but falls from 4.7% to 1.2% between P = 4096 and 16384, so it is exact only when . We therefore removed width from the comparison. The whole encoder sub-sweep and most other runs use P = 4096, so we repeated the analysis on those runs alone. The loss still does not rank runs (within-bucket Spearman -0.10 on average, range -0.27 to +0.15), the score gap is 0.058 (about 4.5x, so it does not shrink), and the loss-selected encoder exponent is -0.04 (95% -0.21 to 0.10) against 0.36 by score. At a single width the two criteria still give different allocations, so pooling widths does not create the finding (Appendix Z.1). Across widths, for the same encoder in the same bucket, the loss at another width is 0.78 to 1.23 times the P = 4096 loss with no consistent direction (new, Appendix S.3).
Q2. Somewhat. Leaving out one source dataset at a time changes the best run in 0 to 3 of 9 buckets (3 when LCMAP is dropped), and the encoder exponent stays between 0.33 and 0.40 (Appendix Z.2). On the relation between the suites, we fixed the 14 held-out tasks before any v2 model was scored on them, and chose them to differ from the 15 selection tasks (Table 29). Eight of them predict urban or socioeconomic quantities (population, GDP, night lights, road density, PM2.5, surface temperature, age structure, urban land use), which no selection task does. Four of the six vegetation tasks ask for dense segmentation, multi-label species or biomass, whereas the selection tasks label single points. The other two (CropHarvest Togo and China) are binary crop/non-crop, like the selection suite's Africa crop mask. The held-out labels come from other sources and mostly other countries (Germany, France, Austria, Finland, Togo, China and the CITYREP cities) and are scored with a CNN head on up to 100% of the labels, not k-NN or linear probes on at most 300 samples. Crop mapping is therefore the main overlap, with different label sources and mostly in different countries. The suites also rank systems differently: AlphaEarth is second on the 15 tasks and last of four on the 14.
Response to Reviewer E5SA (1/4)
Thank you for the careful review, and for your positive view of the rigour of the sweep and of the released embeddings as a product for the EO community. We number the weaknesses you presented from W1 to W7 in the order raised: who benefits from the scaling laws (W1), the loss/score mismatch (W2), downstream selection (W3), nested distillation (W4), standard benchmarks (W5), input mismatch (W6), and figures and writing (W7). Your questions are Q1 to Q4. This comment covers W1 to W4. Comment 2 covers the held-out test for W3 and W5 to W7, Comment 3 answers Q1 and Q2, and Comment 4 answers Q3 and Q4. Section numbers refer to the revised PDF, and the new analyses are in Appendix Z.
W1. We agree that the direct users of our exponents are groups training pixel-wise Sentinel-1/2 encoders with Barlow Twins or a closely related objective such as VICReg. EO models differ in training data, in patch- versus pixel-level modelling and in pretraining algorithm, and we do not claim that the exponents transfer to other settings. Appendix S.1 now states this scope. We would suggest one modest practice more widely: checking a loss-based fit against a downstream metric before using it to allocate compute.
W2. The mismatch is known, and the revision notes this now (Section 3.2). Section 6 and Appendix S.2 cite your [3] (first author corrected to Lourie) and [4], and no longer contrast EO with language models. Our point is about what the mismatch does to compute allocation. We do not know why the mismatch occurs in this model family. Appendix S.2 lists four candidate causes as untested hypotheses, and the introduction now calls the cloud and orbital sampling explanation a hypothesis. We tested one simpler explanation, pooling projector widths, and it does not account for the mismatch: using only the P = 4096 runs, the mean within-bucket Spearman between loss and score is -0.10 (Appendix Z.1).
W3. We do not claim downstream model selection as new. Our claim is that, in a sweep that separates encoder, projector and data at fixed compute, the two criteria give different allocation advice. Loss selection keeps the encoder roughly fixed as the budget grows (exponent 0.02), whereas score selection grows it (0.36, Appendix S.3). This difference holds at a single projector width too (your Q1, Comment 3). The downstream-optimal split is also data-heavy (data exponent 0.63, encoder 0.36, which sum to about 1 by construction, Section 3.3). The held-out comparison you asked for is at the start of Comment 2.
W4. Thank you for pointing us to Pathryoshka. It appeared on arXiv in November 2025 and at ECCV in September 2026, which overlaps with the period in which we developed and wrote this work. We should still have found it, and not citing it was an oversight on our part. Our Equation 2 has the same form as its CLS-token objective, with one teacher. Sections 2 and 5 now cite [5] and say that we do not claim the objective as new. What we add is evidence for pixel time series. We ran a control that trains the same student with the same teacher, data and schedule, using only the full-dimensional term, and then compresses it by first-k, random-k or PCA-k (Appendix Z.3). Nested prefixes beat the best of these by 0.015 at k = 16, 0.008 at k = 32 and 0.003 at k = 64, and are level with it at k = 128 (-0.001). On the 14 held-out tasks at k = 16, nested scores 0.601 against 0.588 for PCA-16. The nested objective matters mainly for small prefixes, and PCA recovers most of the difference. Two earlier results remain: the distilled 44M student beats the same encoder trained from scratch (0.593 vs. 0.527, Table 22), and, in a preliminary experiment, prefix losses added to Barlow Twins pretraining gave weaker small prefixes than distillation (Appendix W.2).