- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SatRAP, a post-hoc correction for predictors trained on frozen satellite embeddings. The core idea is that training residuals are stored with location and embedding. At test time the M=32 geographically nearest entries are reweighted by exp(-dist/h) * exp(beta*cos sim), averaged, shrunk by a learned sigmoid and added to base prediction. The motivation is that, users would tend to discard the residuals after they used RSFMs or their embeddings for donstream tasks, but authors cleverly spotted this issue. Then, there's also SatRAP+, that adds two small MLP affinity terms on top of the analytic score. The authors did a relatively large scale evaluation: the paper has evaluated on 7140 global points, 10 inputs (7 embeddings + 3 raw sensors), 6 tasks (3 LULC, canopy, AGB, DEM), 6 predictors, 5 IID folds, against 16 location encoders. Results also look pretty good at first glance: +4.5pp OA and +0.13 R2 over base predictor, best overall vs baselines.
Critical strengths
-
The empirical observation itself is useful, i.e., residuals of embedding-based predictors keep clear geospatial semantic structure (Fig 2, 9, 10), Moran’s I of residuals predicts gain almost perfectly and the correction removes it. This is a very clean, reusable finding regardless of the method.
-
One of the biggest strengths, i think, is that this method is cheap and needs no retraining, which is what practitioners actually want.
Critical weaknesses
- The method borrows residuals from nearby labelled points and the targets are spatially autocorrelated, so every test point has labelled neighbours by construction. This measures interpolation inside the labelled region, not generalisation. I would highly recommend authors add spatial block CV tests. For an EO paper this is the central question.
- I think it's worth adding more traditional baseline methods, say, regression kriging / residual GP (not cited either, and it is the same idea). for now, I can’t tell what the retrieval adds beyond “use location”.
Questions for authors
- What is the residual head for the location-encoder baselines, i assume linear? if so, have the authors considered results with a nonlinear head, and with coordinates / e(s) concatenated into the base predictor’s input?
- Which of the 10 inputs saw the target products (or lat/lon) during pretraining? If im not mistaken, AEF has LULC map/AGB/etc. in their training pipeline (similar cases for terramind), so there might be some data leakage issue in your evaluation pipeline. so i wonder how much would this affect your results/conclusions?
- Can authors please report training details and cost for SatRAP+ and Full-Param? and was Full-Param tuned with the same effort as SatRAP+?
- Does SatRAP+ use the same shrinkage eta as SatRAP at prediction time? Eq 9 doesn’t have it.
Overall, this is a neat and good work. I will consider raising my score if authors address my concerns.