acceptodds
Under review as a conference paper at ICLR 2027

Faster, Lighter, Better: Compressed Earth Observation Foundation Models

Abstract

Earth observation foundation models (EOFMs) support diverse geospatial tasks, but adapting and deploying them could be computationally expensive. We introduce NewOrder, a framework for structured pruning of EOFMs before full downstream fine-tuning. Structural importance can change as the pretrained encoder and a newly initialized decoder learn together, while heterogeneous EO data make importance estimates sensitive to the calibration inputs. NewOrder addresses these challenges by combining output-sensitive scoring, progressive pruning, and representative calibration. Across four encoders and four tasks, NewOrder with 50% compression rate achieves the best performance among the compared pruning methods in 14 of 16 settings and exceeds dense adaptation in 8 of them. Additionally, full efficiency profiling shows up to 1.73× inference speedup, 1.72× training-step speedup, and 33.4% lower peak training memory. These results demonstrate that selecting which pretrained structures to retain during adaptation can improve both downstream accuracy and computational efficiency, making the selection of retained pretrained capacity an effective part of task adaptation

Primary area:
foundation or frontier models, including LLMs
Keywords:
compression, remote sensing, earth observation, structured pruning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_cmaTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NewOrder, a structured pruning framework for Earth Observation Foundation Models (EOFMs) that progressively removes less important attention heads and MLP neurons during early downstream adaptation. The method combines output-sensitive scoring, progressive pruning, and representative calibration to account for changes in structural importance during training. Experiments across four foundation models and four downstream tasks show that, with approximately 50% encoder parameter compression, NewOrder achieves the best performance among the compared pruning methods in 14 out of 16 settings, while providing training and inference speedups.

Critical strengths
  1. NewOrder performs structured pruning during early downstream adaptation, allowing subsequent training to operate directly on the compact model while also reducing deployment costs.

  2. The paper systematically evaluates NewOrder across multiple EOFMs and downstream tasks, reporting actual inference latency, training-step latency, and peak memory consumption. The ablation studies also demonstrate the effectiveness of the main components.

Critical weaknesses
  1. According to Appendix A.5, the baseline methods perform one-shot pruning before training, whereas NewOrder employs progressive pruning during early adaptation. Although the ablation studies demonstrate the effectiveness of individual components, comparing different pruning criteria under the same progressive pruning schedule would help better establish the independent contribution of output-sensitive scoring.

  2. The main components of NewOrder build upon established techniques, including gradient-based saliency, progressive pruning, and representative calibration. In particular, Appendix D.1 shows a high overlap between the structures selected by NewOrder and Weighted Gradient. Although their combination achieves promising results, the paper could more clearly articulate its methodological contributions beyond existing structured pruning approaches.

Questions for authors
  1. Can the authors compare existing pruning criteria under the same progressive pruning schedule to further isolate the contribution of output-sensitive scoring?

  2. Given the high overlap between NewOrder and Weighted Gradient in structural selection, as well as existing research on progressive pruning, what is the key methodological novelty of NewOrder?

3 comments

Dear Reviewer cmaT,

Thank you for the thoughtful and constructive review. We appreciate your recognition of the benefits of early pruning, the measured efficiency gains, and the component ablations. We address your concerns below.

W1 & Q1. Progressive schedule

We are happy to see that the original scoring ablation has provided useful insights on the individual effectiveness of different components of NewOrder. Following your suggestion, we have extended the comparison to Magnitude, Activation, Weighted Gradient, and Taylor under the same progressive pruning schedule on HLS over three seeds.

MethodDINOv3ClayOlmoEarthTerraMind
Dense86.27 ± 0.4389.57 ± 0.6586.53 ± 0.7890.85 ± 0.50
Magnitude79.23 ± 0.6090.31 ± 0.2485.28 ± 1.5089.72 ± 0.73
Activation85.55 ± 1.2990.50 ± 0.3086.12 ± 0.9890.44 ± 0.76
Weighted Gradient86.09 ± 0.9490.79 ± 0.2686.41 ± 0.2390.09 ± 0.89
Taylor85.20 ± 0.6290.11 ± 1.5186.47 ± 0.2990.99 ± 0.60
NewOrder86.70 ± 0.2191.42 ± 0.4086.47 ± 0.2391.24 ± 0.66

From the table, we can see that NewOrder achieves the best performance. We also observe that most baselines improve over their one-shot counterparts after adopting progressive pruning, demonstrating that the effectiveness of progressive pruning is transferable to existing pruning methods. NewOrder's continued advantage under the shared schedule suggests that its full performance gains extend beyond the progressive schedule itself. The controlled scoring and calibration ablations in the original submission further identify the contributions of these components. We have added these results to Section 3.4.

W2 & Q2. Framework contribution and structural overlap

We would like to first clarify that, as a unified pruning framework, NewOrder treats the selection of retained pretrained structures as part of downstream adaptation. Previous EOFM pruning approaches only partially address two challenges: structural rankings can change as the encoder and decoder adapt jointly, and rankings estimated from limited calibration inputs may underrepresent heterogeneous EO conditions. Moreover, loss-based saliency weights structural sensitivity by current prediction errors, which can make selection overly specific to the errors represented in the calibration set.

NewOrder addresses these challenges jointly. Output-sensitive scoring measures the current task response's dependence on each structure without loss-based error weighting. Representative calibration makes these estimates cover diverse geographic and land-cover conditions, while progressive re-estimation updates them throughout early adaptation. Together, these components allow structural selection to follow the evolving downstream task across representative EO inputs.

Within this framework, output-sensitive scoring makes a specific distinction from Weighted Gradient. For a structure gg, the two objectives give Sout(g)=1∣C∣∑x∈C∑j∈g∣θj∂J(x;θ)∂θj∣,S_{\mathrm{out}}(g)=\frac{1}{|C|}\sum_{x\in C}\sum_{j\in g}\left|\theta_j\frac{\partial J(x;\theta)}{\partial\theta_j}\right|, and Sloss(g)=1∣C∣∑(x,y)∈C∑j∈g∣θj∂L(f(x;θ),y)∂θj∣S_{\mathrm{loss}}(g)=\frac{1}{|C|}\sum_{(x,y)\in C}\sum_{j\in g}\left|\theta_j\frac{\partial\mathcal L(f(x;\theta),y)}{\partial\theta_j}\right|, respectively.

The supervised loss gradient weights output sensitivity according to the current prediction error. Our criterion directly measures how strongly the current task response depends on each structure.

Regarding Appendix D.1, the reported Jaccard overlap measures agreement in the identities of retained structures and counts each structure equally. Both criteria depend on output derivatives, making substantial agreement plausible, while the error weighting above can change the retained structures. Structures can nevertheless differ substantially in their influence on the model output. High overlap therefore leaves open whether the differing selections affect downstream performance, which the controlled comparisons in Fig. 3(a) directly evaluate (NewOrder's Avg mIoU of 89.25 compared to weighted gradient's 88.69).

We have clarified the rationale for the framework in Section 2.2 and expanded the structural-overlap discussion in Appendix D.

Happy to engage in further discussion!

Thank you again for the thoughtful review! We hope our responses address your concerns, and we would be happy to discuss further if you have any additional questions.

Thank you for your responses. I maintain my score from the positive side.

Dear Reviewer cmaT,

Thank you so much for replying. Your comments were very helpful in improving the framing and clarifying the contribution of our paper!

Best,
#5165 Authors

Reviewer_R7b3OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors present New Order, a framework for pruning EO Foundation models before full finetuning for a downstream task. The authors rely on a calibration set consisting of representative samples of a downstream dataset and progressively prunes the encoder based on a sensitivity score, while the pretrained encoder and the newly initialized decoder jointly adapt to the task at hand. The resulting encoder provides comparable results to other pruning mechanisms and unpruned models on a 50% sparsity budget.

Critical strengths

The authors offer an interesting point of view to the efficiency problem of EO foundation models. It has been well established that there exist significant levels of redundancy in both the foundation models themselves [1] as well as the EO data & tasks [2]. The authors take a pragmatic decision to optimize the process of finetuning foundation models, given that a major part of the community uses these models in such a (counterintuitive) way. Their results demonstrate a good performance/efficiency trade-off.

[1] Hackel, Leonard, Tom Burgert, and Begüm Demir. "How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models." arXiv preprint arXiv:2601.22841 (2026).

[2] Papazafeiropoulos, Tasos, et al. "Hide and Seek: Investigating Redundancy in Earth Observation Imagery." arXiv preprint arXiv:2603.13524 (2026).

Critical weaknesses

The paper takes a central decision in optimizing the finetuning process of Earth Observation foundation models. This leads me to wonder:

  1. What makes this method specifically applicable to EO foundation models and not standard computer vision models? As it stands the only thing specifc to EO is the evaluation benchmark.
  2. Why finetune? The goal of a Foundation Model is to provide semantically rich representations that can be utilized for a wide array of downstream tasks. As a result, the need for finetuning is, in theory, misplaced. Finetuning an FM is in fact supervised training with an improved (?) initialization point. What would be the difference of enforcing sparsity in a fully supervised setup e.g assuming a model pretrained to solve the BigEarthNet problem?
  3. The evaluation setup mainly examines dense tasks with relatively small class coverage. Given that the evaluation takes into consideration the nuances of the Earth Observation domain, an investigation of the robustness of the sparsification to the target granularities (i.e. how robust is the pruning mechanism as task require increasingly more finegrained information) as well as ground sampling distance (i.e how robust is the pruning mechanism when the input information is coarser and coarser (also in conjuction with progressively higher target granularity)) would be very interesting.
  4. Additionally, many EO tasks correspond to multi-label classification, if not extreme long-tailed multi-label classification. How does this framework adapt to such tasks which potentially require finegrained features?

At the same time, current EO foundation models are not truly large-scale. As can be seen also from the experiments section, most models are variants of ViT-Base and ViT-Large with very sparse examples exceeding their parameter counts (e.g SkySense). At this point most people can finetune these model on a personal computer, while the primary bottleneck would be the I/O. What is the goal of such a method in this context?

Finally, the authors seem to ignore the recent study on the redundancy and slimmability of EO FMs by Hackel et.al [1], which show that existing EO FMs can be made severely slimmer and maintain their relative performance (achieving on average 85% of their relative performance with 1% of the compute budget using post-hoc slimmability i.e using uniform width reduction of the pretrained transformer blocks). The authors should position themselves against this prior work and better articulate how their research differentiates. This also affects their claims in the listed contributions.

Minor comments:

L85-86: This assumption is debatable. I am not sure at all that the community had this assumption.

L107-108: What constitutes a structure in this case?

L130-132 & L185: How do you train for the downstream task while at the same time dont use ground truth labels? This is a bit confusing. In my understanding the sensitivity score measures the magnitude of the shift of the representation at a given output stage depending on the pruning ration. This shift depends also on the ground truth signal.

[1] Hackel, Leonard, Tom Burgert, and Begüm Demir. "How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models." arXiv preprint arXiv:2601.22841 (2026).

Questions for authors
  1. What is a pretraining structure? Should be well defined early on.
  2. What is the impact of the decoder size?
  3. How can a pruning methodology like this be adapted for linear probing and not finetuning?
  4. Why do the authors rely on a decoder for the sparsification? What if a) the task at hand is not dense, and b) the choice of a decoder is significantly more lightweight (e.g a simple convolutional decoder on top of the final layer)?
  5. Why use an UPerNet for finetuning and not rely on a linear classifier e.g restructure the grid of patches from the model's output and cast it to the original dimension via a linear projection?
  6. How does increasing the amount of classes affect the required size of the calibration set as well as its utility in the pruning process?
  7. What is the statistical significance of the results e.g mean and std across runs?
5 comments

Dear Reviewer R7b3,

Thank you for recognizing the practical relevance of efficient EOFM adaptation and NewOrder's performance–efficiency trade-off. We address your concerns below.

W1. Relevance to Earth observation

NewOrder's EO motivation extends to all three components, as discussed in Sections 2.2–2.4. Across heterogeneous geographic and land-cover conditions, loss-based rankings can become overly specific to the prediction errors represented in a small calibration set. Output-sensitive scoring avoids this error-based weighting and requires no ground-truth targets for scoring. Progressive re-estimation follows changing structural importance as the encoder and decoder adapt to the heterogeneous downstream data. Representative calibration ensures that different terrain and land-cover conditions contribute to these estimates. Together, these designs select structures supporting the evolving task response across representative EO inputs.

W2. Fine-tune

Fine-tuning is an established EOFM adaptation protocol. OlmoEarth [1] evaluates full fine-tuning across classification, regression, segmentation, and detection; Prithvi-EO-2.0 [2] uses fine-tuning across disaster-response and land-cover applications. Hackel et al. [3] also describe the field’s shift toward "fine-tuning pretrained RS FMs."

Foundation pretraining provides transferable representations, while downstream adaptation specializes them to the target task and data distribution. NewOrder reduces the cost of this widely used process. Its pruning procedure does not depend on the pretraining objective and can also operate on an encoder pretrained with supervised BigEarthNet labels.

W3. Target granularity and spatial resolution

Our evaluation already spans binary burn-scar segmentation, eight-class land-cover mapping, 24-class land-cover mapping, and continuous biomass regression, using imagery from HLS, Sentinel-2, and high-resolution Gaofen-2 (Section 3.1 and Appendix A.2). These tasks cover different semantic granularities, sensors, and spatial resolutions (GSD).

On the 24-class Five-Billion-Pixels task, NewOrder achieves a three-seed average of 52.16 mIoU, compared with 47.78 for the strongest competing pruning baseline. This supports its effectiveness for retaining information needed by more fine-grained downstream tasks.

W4. Multi-label and long-tailed classification

The framework naturally extends to a multi-label prediction head, using a scalar aggregation of its class logits as the output response JJ. Structural scoring and progressive pruning then follow the same procedure. Representative calibration can incorporate coverage of rare labels so that their examples contribute to importance estimation.

W5. Practical value and the computational bottleneck

Dense EO prediction remains computationally demanding even with ViT-Base/Large encoders. For example, our OlmoEarth configuration uses 51.24 GiB of peak training memory at batch size 4 with BF16; NewOrder reduces this to 35.72 GiB (Table 3). To examine whether data loading dominates adaptation on a larger dataset, we additionally profiled OlmoEarth on Five-Billion-Pixels, where it had the longest dense adaptation time among the evaluated encoders. With the inputs prewarmed, cumulative batch-fetch waiting time was under one minute over a complete adaptation run lasting more than one hour. Thus, waiting for input batches was not the dominant bottleneck in this configuration.

W6. Relationship to Hackel et al.

Thank you for highlighting this relevant work. Hackel et al. [3] characterize representational redundancy through uniform width reduction of pretrained EOFMs. In their post-hoc evaluation, they truncate FFN hidden dimensions and dimensions within each attention head while keeping the pretrained encoder weights frozen. Downstream performance is then evaluated using KNN classifiers or lightweight task heads.

NewOrder studies structural selection during downstream adaptation, when the encoder itself is changing. It selects attention heads and MLP neurons according to the current task response, repeatedly re-estimates their importance as the encoder and decoder adapt jointly, and uses representative calibration inputs to inform these decisions. The resulting compact encoder continues to train on the downstream task.

Their findings establish the potential for substantial compression of pretrained EO representations. Our contribution is a task-dependent procedure for selecting retained structures during adaptation and evaluating the resulting adaptation and deployment costs. We have incorporated this work into the related-work discussion and clarified these differences.

Minor comments

  • L85–86: We will replace the statement about a community-wide assumption with the direct empirical finding: retaining the full pretrained encoder is not always necessary for the best downstream performance.
  • L107–108: We will introduce the head/neuron definition earlier, as detailed under Q1.
  • L130–132 and L185: Downstream training uses supervised labels; the saliency objective JJ is computed from predictions without ground-truth targets. “Label-free” refers specifically to the scoring process during pruning.

Q1. What is a pretrained structure?

Section 2.2, L193–195, defines gg as “the set of parameter indices belonging to one head or neuron.” Each structure includes the coupled parameters needed to remove an attention head (Q/K/V/O) or MLP neuron (up/down/gate projection) consistently, as detailed in Appendix B. “Pretrained” indicates that these structures belong to the pretrained encoder. We will move this definition earlier.

Q2. Impact of decoder size

For each encoder–task pair, all methods use the same decoder architecture, width, and random-initialization protocol, controlling decoder size in the comparisons. A larger unpruned decoder increases runtime and memory consumption and can reduce the overall speedup obtained by compressing the encoder.

Q3. Application to linear probing

The scoring and pruning rules can remain unchanged. With a linear head, JJ is computed from its predictions, and structural importance is re-estimated as the head learns. Encoder parameter updates can remain frozen while gradients are enabled for sensitivity estimation; computing these gradients does not require updating the encoder weights.

Q4. Dependence on the decoder or dense prediction

NewOrder requires a differentiable task response, with no dependence on a particular decoder architecture. A classification head, lightweight convolutional decoder, or linear projection can all supply that response. The task head determines the output being scored, while pruning operates on the same encoder head/neuron groups and preserves the encoder’s output interface.

Q5. Why UPerNet?

UPerNet is an established decoder for evaluating dense EO tasks, used in Prithvi-EO-2.0 [2] and in the segmentation experiments of Hide and Seek. We adopt this established setup and keep it fixed across pruning methods. A linear patch projection is also compatible with NewOrder, as described under Q3–Q4.

Q6. Class count and calibration size

The calibration budget depends on the diversity of conditions to cover, and one dense image can contain multiple classes. Our Coverage-32 set for Five-Billion-Pixels collectively covers all 24 land-cover classes (Section 2.4). For finer-grained targets, selection should include examples containing the relevant distinctions; the budget can be increased when the existing set cannot provide that coverage.

Q7. Results across multiple runs

We have repeated the evaluation over three seeds. The table below reports mean ± standard deviation for dense adaptation and all pruning methods; segmentation uses mIoU (%), and MMEarth uses R2R^2 (%).

TaskMethodDINOv3ClayOlmoEarthTerraMindAvg.
HLSDense86.27 ± 0.4389.57 ± 0.6586.53 ± 0.7890.85 ± 0.5088.30
HLSMagnitude75.48 ± 0.2189.26 ± 1.1184.46 ± 2.9089.47 ± 0.8084.67
HLSActivation83.39 ± 0.2789.58 ± 0.6486.04 ± 1.3290.37 ± 0.4787.35
HLSWeighted Gradient83.83 ± 1.0090.46 ± 0.8386.61 ± 0.5990.17 ± 0.6887.77
HLSTaylor81.29 ± 0.6290.95 ± 0.4786.59 ± 0.5889.34 ± 0.6787.04
HLSNewOrder86.70 ± 0.2191.42 ± 0.4086.47 ± 0.2391.24 ± 0.6688.96
DFC2020Dense50.07 ± 0.3651.25 ± 0.8054.09 ± 0.5353.33 ± 0.5652.19
DFC2020Magnitude40.56 ± 1.1452.08 ± 0.4453.69 ± 0.5741.31 ± 2.2746.91
DFC2020Activation46.14 ± 4.1250.94 ± 0.7154.04 ± 0.4445.30 ± 3.7449.11
DFC2020Weighted Gradient46.97 ± 1.1251.45 ± 0.8053.37 ± 0.1346.76 ± 4.5249.64
DFC2020Taylor48.44 ± 1.0950.19 ± 1.3151.62 ± 0.4645.85 ± 6.8149.02
DFC2020NewOrder50.72 ± 0.3352.59 ± 1.0253.63 ± 0.2255.13 ± 0.3753.02
MMEarthDense45.46 ± 1.5843.47 ± 1.9952.79 ± 0.1153.70 ± 0.3848.86
MMEarthMagnitude29.20 ± 6.1340.00 ± 1.5547.67 ± 1.4047.79 ± 1.0041.17
MMEarthActivation43.41 ± 0.7343.07 ± 2.4849.74 ± 0.4250.04 ± 0.8146.56
MMEarthWeighted Gradient44.77 ± 0.9340.93 ± 4.1549.97 ± 0.3749.84 ± 0.9946.38
MMEarthTaylor43.61 ± 1.7642.85 ± 1.1650.44 ± 0.2049.44 ± 1.0846.58
MMEarthNewOrder46.99 ± 0.4644.47 ± 0.9652.05 ± 0.6652.20 ± 0.3348.93
FBPDense62.37 ± 2.7147.76 ± 0.7655.26 ± 2.9450.23 ± 1.6853.91
FBPMagnitude48.13 ± 5.7040.78 ± 1.6139.68 ± 5.1239.29 ± 2.6741.97
FBPActivation58.67 ± 1.3242.83 ± 2.4643.17 ± 0.0442.85 ± 2.0046.88
FBPWeighted Gradient58.67 ± 1.5644.24 ± 0.9344.88 ± 1.1542.05 ± 2.5547.46
FBPTaylor58.20 ± 1.7245.46 ± 0.9144.12 ± 1.9443.32 ± 1.8347.78
FBPNewOrder60.73 ± 1.6847.65 ± 0.3053.51 ± 1.1146.74 ± 2.5952.16

Across the complete three-seed comparison, NewOrder retains the highest mean performance among pruning methods in 14 of 16 settings and exceeds the dense mean in eight, while removing half of the encoder parameters.

For the direct comparison with dense, we additionally report task-level averages, win rate, and 95% confidence intervals for the average difference.

TaskDenseNewOrderAvg Δwin rate95% intervals
HLS88.3088.96+0.6558.3%[+0.32, +1.01]
DFC202052.1953.02+0.8375.0%[+0.58, +1.10]
MMEarth48.8648.93+0.0733.3%[−0.53, +0.70]
FBP53.9152.16−1.7516.7%[−2.74, −0.66]

The average gains on HLS and DFC2020 have confidence intervals entirely above zero, supporting the benefit of structural selection beyond isolated small differences in individual runs. We have updated Table 2 to the 3-seed results and add the dense comparison experiments to Sec 3.2.

Reference

[1] Herzog, Henry, et al. "Olmoearth: Stable latent image modeling for multimodal earth observation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
[2] Szwarcman, Daniela, et al. "Prithvi-eo-2.0: A versatile multi-temporal foundation model for earth observation applications." IEEE Transactions on Geoscience and Remote Sensing (2025).
[3] Hackel, Leonard, Tom Burgert, and Begüm Demir. "How Much of a Model Do We Need? Redundancy and Slimmability in Remote Sensing Foundation Models." arXiv preprint arXiv:2601.22841 (2026).

Happy to engage in further discussion!

Thank you again for the thoughtful review! We hope our responses address your concerns, and we would be happy to discuss further if you have any additional questions.

Reviewer_ZFajOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NewOrder, a structured pruning method for efficient downstream adaptation of Earth Observation Foundation Models (EOFMs). It combines gradient-free, output-sensitive importance scoring, progressive pruning with iterative re-estimation, and terrain-aware calibration sample selection. Experiments across four EOFMs and four downstream tasks show that NewOrder achieves the best pruning performance in 14 of 16 settings at 50% sparsity, outperforming dense adaptation in 8 settings, while delivering up to 1.72× training speedup and 33.4% lower peak GPU memory usage.

Critical strengths

1.The paper presents comprehensive experiments across four encoders, four tasks, and multiple pruning baselines, covering both segmentation and regression.

2.A thorough efficiency analysis reports training and inference latency, peak GPU memory, and parameter reduction.

3.Systematic ablation studies examine key design choices, including saliency scoring, progressive pruning, calibration selection, sparsity, and pruning duration.

Critical weaknesses

1.The claimed advantage of early pruning over post-hoc pruning is not validated. Table 2 changes two things at once: the scoring rule and the pruning schedule. So when NewOrder wins, we cannot tell whether it wins because of its scoring rule, its gradual pruning schedule, or both. A direct post-hoc pruning baseline or comparisons under matched pruning schedules would better support this claim.

2.The claim that NewOrder “outperforms dense adaptation in 8 of 16 settings” is accurate, but describing it as “smaller, faster, and often more accurate” overstate its overall performance. NewOrder slightly underperforms dense adaptation on MMEarth, scoring 49.12 vs. 49.14, and more noticeably on Five-Billion-Pixels, scoring 52.69 vs. 54.52, with an overall record of 8 wins and 8 losses. The authors should qualify this claim and report aggregate performance, win rates, and confidence intervals to provide a more balanced assessment.

3.The paper lacks statistical significance analysis and results from multiple runs. Table 2 reports only single-run results without standard deviations or confidence intervals, making small performance differences difficult to assess, such as 86.75 vs. 86.76 on DINOv3-HLS and 49.12 vs. 49.14 on MMEarth. Reporting means and standard deviations over three seeds or bootstrap confidence intervals would strengthen the reliability of the conclusions.

Questions for authors

1.In Algorithm 1, what is the mathematical definition of J(x; θ)? How do the authors ensure that no labels are used in its computation?

2.Do all baselines in the main table use the same Coverage-32 calibration set as NewOrder? If not, could the authors provide comparisons using the same calibration set?

5 comments

Dear Reviewer ZFaj,

Thank you for taking the time to review our paper and for highlighting the breadth of the experiments, the efficiency analysis, and the systematic component ablations. We address your concerns below.

W1. Progressive schedule

We wish to first refer the reviewer to our ablation study in Sec. 3.4, where we provided the results that isolated each component independently. In particular, the scoring ablation fixes the schedule, calibration data, structural groups, and training protocol while replacing the output-sensitive objective with the supervised loss objective.

To further address your concerns, we extend the comparison to Magnitude, Activation, Weighted Gradient, and Taylor under the same progressive pruning schedule on HLS over three seeds.

CriterionDINOv3ClayOlmoEarthTerraMind
Dense86.27 ± 0.4389.57 ± 0.6586.53 ± 0.7890.85 ± 0.50
Magnitude79.23 ± 0.6090.31 ± 0.2485.28 ± 1.5089.72 ± 0.73
Activation85.55 ± 1.2990.50 ± 0.3086.12 ± 0.9890.44 ± 0.76
Weighted Gradient86.09 ± 0.9490.79 ± 0.2686.41 ± 0.2390.09 ± 0.89
Taylor85.20 ± 0.6290.11 ± 1.5186.47 ± 0.2990.99 ± 0.60
NewOrder86.70 ± 0.2191.42 ± 0.4086.47 ± 0.2391.24 ± 0.66

From the table, we can see that NewOrder achieves the best performance. Most baselines also improve over their one-shot counterparts after adopting progressive pruning, showing that this component benefits several existing criteria. NewOrder's continued advantage under the shared schedule suggests that its full performance gains extend beyond the progressive schedule itself. The original controlled ablations identify their individual contributions, and the additional comparison under Q2 further matches both the schedule and calibration set across methods. We have included these results in Section 3.4.

We additionally evaluated post-hoc pruning on HLS. As shown below, the post-hoc baseline results in a drastic loss in performance on all settings. These results complement the comparisons under a shared progressive schedule reported below, addressing the requested post-hoc comparison alongside the analysis of different scoring criteria.

ModelDensePost-hocNewOrder
DINOv386.27±0.4351.24±2.8786.70±0.21
Clay89.57±0.6583.57±0.6991.42±0.40
OlmoEarth86.53±0.7882.30±1.0286.47±0.23
TerraMind90.85±0.5087.90±1.5591.24±0.66

W2 & W3. Dense and repeated-run results

Thanks for your suggestion. We provide the 3-seed results below and note that the two main findings persist when comparing the resulting means: NewOrder achieves the best pruned performance in 14 of 16 settings and exceeds dense adaptation in eight settings, while removing approximately half of the encoder parameters.

TaskMethodDINOv3ClayOlmoEarthTerraMindAvg.
HLSDense86.27 ± 0.4389.57 ± 0.6586.53 ± 0.7890.85 ± 0.5088.30
HLSMagnitude75.48 ± 0.2189.26 ± 1.1184.46 ± 2.9089.47 ± 0.8084.67
HLSActivation83.39 ± 0.2789.58 ± 0.6486.04 ± 1.3290.37 ± 0.4787.35
HLSWeighted Gradient83.83 ± 1.0090.46 ± 0.8386.61 ± 0.5990.17 ± 0.6887.77
HLSTaylor81.29 ± 0.6290.95 ± 0.4786.59 ± 0.5889.34 ± 0.6787.04
HLSNewOrder86.70 ± 0.2191.42 ± 0.4086.47 ± 0.2391.24 ± 0.6688.96
DFC2020Dense50.07 ± 0.3651.25 ± 0.8054.09 ± 0.5353.33 ± 0.5652.19
DFC2020Magnitude40.56 ± 1.1452.08 ± 0.4453.69 ± 0.5741.31 ± 2.2746.91
DFC2020Activation46.14 ± 4.1250.94 ± 0.7154.04 ± 0.4445.30 ± 3.7449.11
DFC2020Weighted Gradient46.97 ± 1.1251.45 ± 0.8053.37 ± 0.1346.76 ± 4.5249.64
DFC2020Taylor48.44 ± 1.0950.19 ± 1.3151.62 ± 0.4645.85 ± 6.8149.02
DFC2020NewOrder50.72 ± 0.3352.59 ± 1.0253.63 ± 0.2255.13 ± 0.3753.02
MMEarthDense45.46 ± 1.5843.47 ± 1.9952.79 ± 0.1153.70 ± 0.3848.86
MMEarthMagnitude29.20 ± 6.1340.00 ± 1.5547.67 ± 1.4047.79 ± 1.0041.17
MMEarthActivation43.41 ± 0.7343.07 ± 2.4849.74 ± 0.4250.04 ± 0.8146.56
MMEarthWeighted Gradient44.77 ± 0.9340.93 ± 4.1549.97 ± 0.3749.84 ± 0.9946.38
MMEarthTaylor43.61 ± 1.7642.85 ± 1.1650.44 ± 0.2049.44 ± 1.0846.58
MMEarthNewOrder46.99 ± 0.4644.47 ± 0.9652.05 ± 0.6652.20 ± 0.3348.93
FBPDense62.37 ± 2.7147.76 ± 0.7655.26 ± 2.9450.23 ± 1.6853.91
FBPMagnitude48.13 ± 5.7040.78 ± 1.6139.68 ± 5.1239.29 ± 2.6741.97
FBPActivation58.67 ± 1.3242.83 ± 2.4643.17 ± 0.0442.85 ± 2.0046.88
FBPWeighted Gradient58.67 ± 1.5644.24 ± 0.9344.88 ± 1.1542.05 ± 2.5547.46
FBPTaylor58.20 ± 1.7245.46 ± 0.9144.12 ± 1.9443.32 ± 1.8347.78
FBPNewOrder60.73 ± 1.6847.65 ± 0.3053.51 ± 1.1146.74 ± 2.5952.16

For the direct comparison with dense, we additionally report task-level averages, win rate, and 95% confidence intervals for the average difference.

TaskDenseNewOrderAvg Δwin rate95% intervals
HLS88.3088.96+0.6558.3%[+0.32, +1.01]
DFC202052.1953.02+0.8375.0%[+0.58, +1.10]
MMEarth48.8648.93+0.0733.3%[−0.53, +0.70]
FBP53.9152.16−1.7516.7%[−2.74, −0.66]

The average gains on HLS and DFC2020 have confidence intervals entirely above zero, supporting the benefit of structural selection beyond isolated small differences in individual runs. Our description "often more accurate" refers to the eight settings with higher performance than dense adaptation (especially on HLS and DFC2020), a finding that persists across the three-seed results. We have tamed this claim to reflect its task-dependent nature. We have also updated Table 2 to the 3-seed results and added the dense comparison experiments to Sec. 3.2.

Q1. J(x;θ)J(x;\theta) and label

Equation 1, L142–145, defines JJ as the mean model response over valid output locations. For segmentation, the response at location pp is the maximum class logit,1∣Ω(x)∣∑p∈Ω(x)max⁡czp,c(x;θ)\frac{1}{|\Omega(x)|}\sum_{p\in\Omega(x)}\max_c z_{p,c}(x;\theta). For regression, it is the predicted scalar value averaged over valid output locations. To this end, both definitions use model predictions and are label-free, thus requiring no ground-truth target values.

Q2. Calibration data

In the original Table 2, the data-dependent baselines use Random-32, while NewOrder uses the proposed Coverage-32 selection. Following your suggestion, we have additionally evaluated all criteria with the same Coverage-32 calibration set and progressive pruning schedule, holding the structural budget and training protocol fixed.

MethodDINOv3ClayOlmoEarthTerraMindAvg
Magnitude79.2190.3186.2290.4986.56
Activation85.0590.7487.0591.3188.54
Weighted Gradient85.6091.0984.7291.1188.13
Taylor86.2091.2286.8591.0688.83
NewOrder86.7591.8086.6291.8289.25

Under these matched conditions, NewOrder achieves the highest mIoU on three of four encoders and the highest average. These results establish the additional benefit of output-sensitive scoring when both the schedule and calibration inputs are shared across criteria. We have included this comparison in the appendix.

Happy to engage in further discussion!

Thank you again for the thoughtful review! We hope our responses address your concerns, and we would be happy to discuss further if you have any additional questions.

Reviewer_87ZLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NewOrder, a structured pruning framework that combines output-sensitive importance estimation, progressive pruning, and representative calibration during the early adaptation of Earth observation foundation models.

Critical strengths

Reducing both adaptation and deployment costs of Earth observation foundation models addresses an important computational challenge with broad practical relevance.

Critical weaknesses
  1. The proposed framework combines established techniques, including gradient-based importance estimation, progressive structured pruning, and representative calibration. The fundamental methodological advance beyond existing pruning approaches remains insufficiently distinguished.

  2. The main pruning baselines use one-shot importance estimation, while NewOrder progressively updates its importance scores during adaptation. Although component ablations are provided, the comparisons do not fully isolate the contributions of the proposed scoring criterion from those of the progressive pruning schedule.

Questions for authors
  1. Could the authors compare the proposed method with Taylor- or gradient-based pruning using the same progressive schedule and calibration protocol? Such comparisons would help clarify whether the improvements primarily arise from the new importance criterion or the progressive pruning strategy.

  2. Could the authors clarify the total end-to-end adaptation time, including repeated importance estimation and calibration, relative to dense training and competing pruning methods? This would help establish the overall computational benefits beyond individual training-step speedups.

2 comments

Dear Reviewer 87ZL,

Thank you for taking the time to review our paper and for recognizing the practical importance of reducing both adaptation and deployment costs of EOFMs. We address your concerns below.

W1. Methodological contribution

Thank you for this suggestion. We are happy to clarify our contribution, including how NewOrder's components work together and how the framework advances existing EOFM pruning approaches. In particular, NewOrder studies which pretrained structures should be retained during downstream adaptation.

Previous EOFM pruning approaches only partially address two challenges: structural rankings can change as the encoder and decoder adapt jointly, and rankings estimated from limited calibration inputs may underrepresent heterogeneous EO conditions. Moreover, loss-based saliency weights structural sensitivity by current prediction errors, which can make selection overly specific to the errors represented in the calibration set.

NewOrder addresses these challenges jointly. At each pruning stage, output-sensitive scoring estimates the current task response’s dependence on each encoder head and MLP neuron using representative EO calibration inputs. This objective avoids the prediction-error weighting in loss-based saliency. For example, under cross-entropy, a confidently correct prediction can have a small loss gradient even when its output remains sensitive to a particular structure. Representative calibration determines which geographic and land-cover conditions inform these estimates, while progressive re-estimation updates them as the encoder and decoder adapt jointly. Together, these components make the selection of retained pretrained capacity responsive to both the evolving downstream task and the diversity of its inputs.

W2. Progressive

We are happy to see that the original scoring ablation has provided useful insights on the individual effectiveness of different components of NewOrder. Following your suggestion, we have extended the comparison to Magnitude, Activation, Weighted Gradient, and Taylor under the same progressive pruning schedule on HLS over three seeds.

MethodDINOv3ClayOlmoEarthTerraMind
Dense86.27 ± 0.4389.57 ± 0.6586.53 ± 0.7890.85 ± 0.50
Magnitude79.23 ± 0.6090.31 ± 0.2485.28 ± 1.5089.72 ± 0.73
Activation85.55 ± 1.2990.50 ± 0.3086.12 ± 0.9890.44 ± 0.76
Weighted Gradient86.09 ± 0.9490.79 ± 0.2686.41 ± 0.2390.09 ± 0.89
Taylor85.20 ± 0.6290.11 ± 1.5186.47 ± 0.2990.99 ± 0.60
NewOrder86.70 ± 0.2191.42 ± 0.4086.47 ± 0.2391.24 ± 0.66

From the table, we can see that NewOrder achieves the best performance. We also observe that most baselines improve over their one-shot counterparts after adopting progressive pruning, demonstrating that the effectiveness of progressive pruning is transferable to existing pruning methods. NewOrder's continued advantage under the shared schedule suggests that its full performance gains extend beyond the progressive schedule itself. The controlled scoring and calibration ablations in the original submission further identify the contributions of these components. We have added these results to Section 3.4.

Q1. Progressive + Calibration data

We additionally evaluated all criteria using the same progressive schedule and Coverage-32 calibration set, with the structural budget and training protocol also held fixed.

MethodDINOv3ClayOlmoEarthTerraMind
Magnitude79.2190.3186.2290.49
Activation85.0590.7487.0591.31
Weighted Gradient85.6091.0984.7291.11
Taylor86.2091.2286.8591.06
NewOrder86.7591.8086.6291.82

Under these matched conditions, NewOrder achieves the highest mIoU on three of four encoders and the highest average. This comparison directly supports the contribution of output-sensitive scoring when the progressive schedule and calibration protocol are identical. We have included these results in the appendix.

Q2. End-to-end time

Thank you for this suggestion. We provide the end-to-end time (minutes) below for 20-epoch training of HLS.

ModelDense TimeOne-shot Gradient Time (SpeedUp)NewOrder Time (SpeedUp)Dense mIoUOne-shot gradient mIoUNewOrder mIoU
DINOv32.572.06 (1.25×)2.42 (1.06×)86.7684.6286.75
Clay3.452.95 (1.17×)3.29 (1.05×)90.1091.4291.80
OlmoEarth12.807.66 (1.67×)10.53 (1.22×)86.9586.9386.62
TerraMind3.042.43 (1.25×)2.69 (1.13×)90.4289.6191.82

NewOrder reduces total adaptation time relative to dense training for all four encoders, achieving end-to-end speedups of 1.05–1.22× after accounting for pruning overhead. Although repeated saliency estimation adds computational cost, the savings from training the progressively smaller encoder outweigh this overhead. Compared with one-shot Gradient, NewOrder takes longer but achieves higher mIoU on three of the four encoders, illustrating the trade-off between adaptation time and downstream performance. These measurements complement the training-step latency, inference latency, and memory results in Table 3. We have added the end-to-end results to Section 3.3.

Happy to engage in further discussion!

Thank you again for the thoughtful review! We hope our responses address your concerns, and we would be happy to discuss further if you have any additional questions.

Comments on the paper

1 comment

Dear Area Chairs and Reviewers,

We sincerely thank all reviewers for their thoughtful comments and constructive suggestions. We have uploaded a revised manuscript and would like to briefly summarize the additional experiments and clarifications, while inviting further feedback from reviewers who have not yet had a chance to respond.

To briefly restate our contribution: NewOrder is a structured pruning framework that integrates output-sensitive scoring, progressive pruning, and representative calibration during early downstream adaptation.

During the rebuttal, we added the following:

  • Controlled component comparisons: We evaluated Magnitude, Activation, Weighted Gradient, and Taylor under the same progressive schedule on HLS over three seeds (Sec. 3.4, Table 6). We additionally matched both the schedule and calibration inputs, and extended this comparison to MMEarth regression, broadening the evidence beyond segmentation (App. A.7, Table 13; Reviewer 87ZL, ZFaj, cmaT).
  • Early versus post-hoc pruning: We added a direct post-hoc pruning comparison on HLS across all four models, reporting three-seed means and standard deviations (Sec. 3.4, Table 6; Reviewer ZFaj).
  • Repeated-run evaluation: We updated the main results to three-seed means and standard deviations across all four models and four tasks. We also provided task-level comparisons with dense adaptation, win rates, and confidence intervals, and qualified the accuracy claims to reflect differences across tasks (Sec. 3.2, Tables 2–3; Reviewer ZFaj, R7b3).
  • End-to-end efficiency: We reported complete adaptation times on HLS, including pruning overhead, showing 1.05–1.22× speedups over dense adaptation (Sec. 3.3, Table 5). We also profiled data-loading wait time on Five-Billion-Pixels to address the concern about I/O bottlenecks (Sec. 3.3; Reviewer 87ZL, R7b3).
  • Methodological clarifications and related work: We clarified the framework’s contribution, the distinction between label-free scoring and supervised adaptation, and the definition of pruned structures (Secs. 2.1–2.2). We expanded the interpretation of structural overlap with Weighted Gradient (App. D.1) and the discussion of Hackel et al. and its relationship to our approach (Sec. 4; Reviewer 87ZL, R7b3, cmaT).

We appreciate that Reviewer ZFaj confirmed that their concerns were fully addressed and raised their score from 2 to 3, and that Reviewer cmaT maintained their positive assessment. We would welcome any remaining questions or feedback on the additional experiments and revised manuscript.

Thank you again to the AC and reviewers for your time and consideration.

Best,
#5165 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.