acceptodds
Under review as a conference paper at ICLR 2027

SafeAdapt: Selective Deployment of Local Corrections to Reduce Harmful Updates in Spatial Soil Carbon Prediction

Abstract

Local corrections can improve average prediction error while degrading some target regions. We study when a few-shot residual correction should replace a shared predictor for modeled lateral soil organic carbon. SafeAdapt combines a frozen source model, source-validation selection of correction amplitude, and a support-only empirical lower-confidence-bound (LCB) gate with source fallback. A harmful update increases held-out regional RMSE relative to the source. Across 33,620 Daily Erosion Project (DEP) episodes, selected-amplitude gating lowers harmful-update incidence from 41.56% to 8.04% at 24.35% update coverage and reduces mean RMSE by 0.89%. Method-aligned European Soil Data Centre (ESDAC) tests show similar harm-coverage trade-offs, although their mean-error gains remain uncertain. A separate ten-seed extension adds selected-candidate controls at matched update counts: LCB lowers conditional harm versus random expectation by 3.90-7.57 percentage points across DEP partitions and 5.34-9.91 points across Global2019 partitions, with exploratory paired site-bootstrap intervals below zero. Advantages over point-gain ranking are smaller and setting-dependent; European equal-coverage harm differences remain uncertain, and fixed shrinkage remains competitive. These results support selective deployment as a practical accuracy-harm-coverage trade-off, rather than a universally optimal correction or a statistical safety guarantee.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
selective adaptation, few-shot learning, transfer learning, regional calibration, negative transfer, spatial machine learning, soil organic carbon, distribution shift

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1w5FOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The authors develop a framework for selective correction for the task of erosion-induced soil carbon prediction. The issue with this task is that, generally, you have very sparse data. Additionally, you have data points that are sampled within a similar region (a watershed). As a global model does not represent all regions well, it is sensible to design a correction for each watershed. This needs to be done very carefully, as per watershed you only have a few data points and can occasionally lead to degenerate performance.

This paper develops a mechanism for this balancing act, enabling improvement where possible and preventing degradation under the condition of extremely sparse and correlated data. Specifically, they equip a frozen base model with a regularized local correction and a lower-confidence-bound (LCB) gate.

The authors find that by building a mechanism that can fall back to the original prediction (which it does in ~75% of the time) and selected shrinkage of the correction term, they can achieve average performance gains whilst keeping the harm (degraded performance) minimal. They then show that the mechanism also works on two other erosion datasets. The improvement is marginal compared to the source performance, but they substantially reduce harm compared to a forced correction. They further test whether the corrections yield targeted improvement or win by just reducing corrections compared to random coverage.

Critical strengths
  1. The authors engineer a sensible solution for a tricky problem in light of the available data and severe spatial autocorrelation. The method is highly careful in the application of the correction and also adds amplitude shrinkage with a fallback (this care is measured by the ratio of times the correction is actually applied). The authors are very transparent about the limitations of the problem, such as spatial leakage and statistical claims.
  2. Their set of experiments is convincing in showing that the mechanism works in reducing harm while the gain itself stays moderate (though the gains are expected to be marginal in this constrained and data-sparse environment).
Critical weaknesses
  1. Even if the approach is sound and pragmatic, the scientific or application contribution is thin and makes me lean towards weak accept. The combination of local correction with selective prediction to selective correction is an interesting idea, but it is deployed here rather as a highly tailored engineering patch for this problem, and the paper does not convincingly show a leap in erosion-induced soil carbon prediction.
  2. Key details about the physical tasks were hard to extract and could have been eased by the authors by clarifying the form of the data and the nature of the spatial splits upfront. The evaluation seems fragmented, and it is hard to follow its logic (e.g., the 10-seed extension with a slightly different implementation comes out of nowhere).
Questions for authors

None

2 comments

Thank you for pointing out where the prediction task and the purpose of the two DEP studies were difficult to follow. We revised the Introduction to explain what is predicted for each hillslope and how a team decides whether to use a local correction across a watershed.

On your first point, the Daily Erosion Project (DEP) prediction target is a hillslope’s modeled long-term mean annual lateral soil organic carbon flux, measured in g C m⁻² yr⁻¹. The example accompanying Figure 1 starts with a team that already has a prediction for every hillslope in a watershed, an area draining to a common outlet. Suppose labels become available for five hillslopes. We use three to fit a candidate correction and the other two to check whether it improves on the existing predictions. If the gate accepts the correction, we refit it using all five labels. If it declines, we retain the existing predictor. In either case, we evaluate the resulting predictions on the remaining hillslopes in that watershed, whose labels were held out from both fitting and selection.

On your second point, the five-seed study measures what happens when the policy is deployed. The ten-seed study asks whether the gate chooses better updates when the number of updates is held fixed. This distinction matters because a policy can reduce the total number of harmful updates simply by updating fewer watersheds.

In the five-seed study, the three DEP partitions withhold large drainage basins, one-degree grid cells, or environmental groups from source training. In each partition, the decision is whether to apply a correction to an individual watershed. Across 33,620 decisions, the gate applied a correction in 24.35% of cases. Regional root mean squared error (RMSE) increased in 8.04% of all 33,620 cases, while mean RMSE fell by 0.89% relative to the source model.

In the ten-seed study, the gate and random selection had the same candidate corrections and made the same number of updates within each split, seed, and label budget. Across the three DEP partitions, the share of applied updates that increased regional RMSE was 3.90–7.57 percentage points lower under the gate than expected under random selection. With update counts matched, this difference reflects which watersheds received corrections .

Sections 4.3 and 5.6 now distinguish these two questions and report the studies separately. Appendix F.1 documents how the 79,480-episode ten-seed study differs from the five-seed deployment study in its watershed cohort, target scaling, and rule for handling ties at the gate.

We hope these revisions clarify the prediction task and why the two DEP studies answer different questions. We welcome any further questions during the discussion.

Reviewer_1w5FOpenReview ↗

Response to authors

Dear authors,

I thank you for your response to my concerns. I do believe that your extended explanations (including those to the other reviewers) for the five-seed and ten-seed setup, and the clarification of DEP and the whole downstream task, help the readability and task framing of the manuscript.

My main concern around the thin contribution (a marginal step with a highly tailored engineering patch) still stands unaddressed. I still believe the proposed method is a careful and well-executed correction with small gains for the specific task. For this reason, my score remains weak accept, and I defer to reviewers with more direct expertise on whether it clears the ICLR bar.

Reviewer_oxvnOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper talks about soil carbon prediction and a system for incorporating corrections to some model output. The method claims to decrease harm from updates and reduce overall RMSE.

Critical strengths
  • The method tries to minimize harm from updates, and less harm is generally a good thing.
Critical weaknesses
  • The method seems far from fundamental or even applied ML/AI research and, instead, very focused on soil sciences.
  • The context/system in which the method is proposed is not explained, making it very hard to evaluate its soundness.
Questions for authors

What is the contribution to the ICLR community presented in the paper? How are the results here informative to AI/LR in general?

1 comment

Thank you for asking what this work contributes beyond an application-specific reduction in harmful updates. Your questions helped us clarify the decision a learning system faces when it already has a predictor for a new site. We revised the manuscript to explain that decision and to make the matched-update evaluation more prominent. The ML contribution is a way to test whether a few local labels can identify where to deploy a correction when retaining the existing predictor is also an option.

What decision does the gate make?

Suppose a shared predictor already serves a new watershed. A few labeled hillslopes are enough to fit a residual correction. The question is whether that correction will also improve predictions on the other hillslopes. The gate uses only the available support labels to decide whether to apply the correction. If it declines, the watershed continues to receive predictions from the existing model. We then evaluate that decision on hillslopes held out from adaptation. The five-label example accompanying Figure 1 illustrates this sequence.

The Introduction now explains negative transfer, the gate, and an episode in this setting. Section 2.2 defines λ as the amplitude of the residual correction and k as the number of local support labels. We use k to specify the size of the fitting system, split the support labels, and compare label budgets. Section 2.2 also distinguishes how often an update raises regional error from how large the increase is and what it might cost downstream.

What does the gate add at the same update count?

A rule can reduce the total number of regressions simply by making fewer updates. To test whether it chooses better updates, Section 3.4 gives competing selectors the same candidate corrections and matches the number of updates within each split, seed, and label budget. We then measure the fraction of applied updates that increase held-out root mean squared error (RMSE) relative to keeping the existing predictor. Matched random selection tests whether the local evidence helps choose sites. Ranking by estimated local improvement alone tests what the gate’s confidence penalty adds .

In the ten-seed Daily Erosion Project (DEP) study, the gate reduced this fraction relative to matched random selection by 3.90, 5.19, and 7.57 percentage points under basin, grid, and environmental holdouts, respectively. The exploratory paired site-bootstrap intervals excluded zero in all three comparisons. Against ranking by estimated improvement alone, the reductions were 2.35 points under the basin holdout and 1.21 points under the grid holdout. The environmental holdout was inconclusive. The global soil product also favored the gate over matched random selection by 5.34–9.91 points, while the European product was inconclusive (Table 4).

Does the result extend beyond modeled soil carbon?

We evaluated daily metered electricity demand on 308 held-out buildings, with earlier observations used for support and later observations for evaluation (Appendix H). At 3,670 updates per policy, 18.39% of gate-selected updates worsened RMSE, compared with 24.98% under matched random selection (difference −6.59 percentage points, 95% building-bootstrap interval [−8.25, −5.09]). Estimated-gain ranking also gave 18.39%. Always updating achieved the lowest mean RMSE, 686.64 versus 692.13 kWh/day for the gate. Because buildings from the same site could appear in different splits, we ran a further test holding out six entire sites. Across 715 buildings and 10,414 updates per rule, the gate produced 781 regressions (7.50%) and estimated-gain ranking produced 786 (7.55%).

Taken together, the matched-count comparisons show that local evidence helped identify where to update in DEP and in the held-out-building experiment. The additional value of the confidence penalty was clearest under the DEP basin and grid holdouts. The electricity results show why estimated-gain ranking is an important baseline when applying the method to a new setting.

We welcome further questions and hope these revisions help you assess the paper.

Reviewer_iRLbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper introduces an approach for applying local corrections to models for soil carbon prediction. Compared to other approaches, an modest improvement in RMSE is achieved, with a lower "Harm". To my understanding, given a local region (although it is not clear how a "region" is defined), "harm" is defined as an increase in held-out RMSE relative to the baseline.

Critical strengths

I can only identify one strength, which is that the idea of not just using e.g. global RMSE as a performance metric, but also considering local "harm", is interesting. However, the paper fails to explain why this "harm" is a suitable metric for measuring the cost of worse local performance. Isn't this the case for any model, that an update will make some predictions better than others? When and why exactly does that become problematic? How do you decide what is an acceptable level of local "harm"?

Critical weaknesses

(1) The paper is very difficult to read. The paper

  • does not provide a good introduction of the problem it is trying to solve and
  • dives straight into technical terms and abbreviations: "negative-transfer", "gate", "SOC", "HUC12", "watershed", "episode" are all terms that are not explained.

(2) The results are not presented in a coherent or convicing way. I do not understand Figure 1a. Table 3 and Figure 2 show the exact same information, what is the added value of including both? But the most important weakness is that it is not clear why "harm" is a useful measure. For example, an update that is only a little harmful in a particular region is not as bad as an update that leads to a more extreme local increase in RMSE, but this is not reflected in the current definition of "harm". Futhermore, even the best reduction of overall RMSE achieved is modest, and an explanation of the significance of this reduction is missing.

Questions for authors

Suggestions:

  • Odd formatting in places, abstract is wider and on page 9 the font changes on some lines. Missing line numbering.
  • Explain the problem you are trying to address and the relevant research and only then introduce details.
  • Since the main contribution of this paper seems to be an approach that can reduce the number of "harmful" updates, the approach should be tested on more applications than the very specific case of spatial soil carbon prediction.
  • In Section 2.2., variables like lambda are used, but never defined. On the other hand, a variable k is introduced but never used.
3 comments

Thank you for asking when regional harm matters and how large the regressions are. We clarified what harm measures and added an analysis of regression severity.

What is harm?

The operating decision is whether to replace a predictor already serving a watershed, an area draining to a common outlet. In the Daily Erosion Project (DEP), a few labeled hillslopes provide local information about modeled lateral soil organic carbon (SOC) flux. We call an update harmful when it increases root mean squared error (RMSE) on other, held-out hillslopes in that watershed relative to retaining the existing predictor. This matters because the existing predictor remains available each time a regional replacement is considered. Harm incidence tells us how often retaining it would have yielded lower regional error. To show how much worse those replacements were, we also report the magnitude of the regional RMSE increases.

How severe are the regressions?

We reanalyzed all 33,620 primary DEP episodes. In each episode, the gate uses local support labels to decide whether to update the watershed predictor. It applied 8,187 updates, of which 2,704 worsened RMSE. This is 8.04% of all decisions and 33.03% of applied updates.

Each line below gives the share of applied updates that raised RMSE by more than 1%, the share that raised it by more than 5%, and the mean increase in the worst tenth of applied updates. Each increase is relative to the existing predictor’s RMSE in that episode.

  • Always apply the same candidate (33,620 updates) — 35.40%, 24.27%, 92.85%.
  • Random selection at the gate’s count (8,187 updates, expected values) — 33.98%, 23.58%, 79.27%.
  • SafeAdapt (8,187 updates) — 29.88%, 21.82%, 81.61%.

At equal update counts, 1,786 SafeAdapt updates raised RMSE by more than 5%, versus approximately 1,931 expected under random selection. That is about 145 fewer cases across 33,620 decisions (5.31% vs. 5.74%). Resampling 20 drainage basins gave an exploratory 95% interval of [−2.66, −0.25] percentage points for the SafeAdapt-minus-random difference among applied updates at the 5% cutoff. SafeAdapt’s worst-decile mean was higher (81.61% vs. 79.27%), with an uncertain difference (95% interval [−8.2, +14.9] points).

The 1% and 5% cutoffs illustrate severity. An agronomically acceptable threshold depends on what increase matters for a particular decision and how often it can be tolerated. For example, if predictions guide field sampling, one would need to test whether an update helps identify high-loss hillslopes. Our DEP evaluation does not measure that outcome.

How meaningful is the average improvement?

Mean DEP RMSE fell from 4.930 to 4.886 g C m⁻² yr⁻¹ (0.89%). The absolute reduction was 0.044 g C m⁻² yr⁻¹ (95% interval [0.032, 0.056] under partition/site resampling). Always applying the same candidate gave mean RMSE 4.874 but worsened 41.56% of decisions, versus 8.04% with gating. Fixed amplitude λ = 0.25 gave a lower mean of 4.845 but worsened 40.46% of decisions. These results measure agreement with modeled carbon flux. The field management benefit remains untested.

Does the approach work in another application?

Your suggestion led us to test daily electricity demand on 308 held-out buildings. At 3,670 updates per policy, 18.39% of SafeAdapt updates worsened RMSE versus 24.98% under matched random selection (difference −6.59 percentage points, 95% building-bootstrap interval [−8.25, −5.09]). Ranking by estimated gain alone gave 18.39%. Buildings can share sites, so this result concerns held-out buildings (Appendix H).

What has changed in the paper, especially in term definitions and formatting?

Thank you also for your suggestions on the manuscript’s structure, logical flow, explanations, and formatting. The Introduction now explains the deployment decision, negative transfer, gate, and episode. Section 2.1 spells out SOC and defines watershed and HUC12 (the 12-digit Hydrologic Unit Code). Section 2.2 defines λ as the correction amplitude and k as the number of support labels. Section 3.2 gives λ ∈ {0.25, 0.50, 1.00}. The example after Figure 1 shows how k = 5 labels are split.

Figure 1(a) is a schematic. Inside the outlined target watershed, filled dots are hillslopes used for adaptation, and open dots are held out for evaluation. Table 3 gives exact values. Figure 2 plots the same estimates so readers can see how lower mean error can come with more harmful updates. Figure 4 plots threshold frequencies. Appendix G, Table 21 reports both denominators and the worst-decile means. We indented the abstract, added continuous review line numbers, and checked that the body font on page 9 is consistent.

We welcome further questions and will share relevant additional findings here. We hope these revisions are helpful as you assess the paper.

Thank you to the authors for the extensive response. Although the authors incorporated feedback from the reviews, the paper is still very difficult to read and I have significant concerns:

(1) There is still no good introduction to the topic of Spatial Soil Carbon Prediction. If the main contribution is intended to be mostly an ML/AI contribution, then the introduction needs to reflect that and several applications should be considered by evaluating on more diverse datasets. My advice to the authors is to decide whether the main focus and contribution is intended to be (a) an application on Spatial Soil Carbon Prediciton or (b) a fundamental ML approach and to structure the paper accordingly.

(2) While the idea of applying selective updates in order to increase global performance while avoiding local decreases in performance is a good one, my concerns about the lack of evidence or arguments on why "harm" is a suitable metric persist.

(3) There are no comparisons to existing approaches or to other baselines.

Therefore, my overall rating remains to reject.

Thank you for clarifying your concerns. Of the two directions you describe, our primary focus is (a), spatial soil carbon prediction. The revised Introduction now opens with the erosion process and the prediction task. In the Daily Erosion Project (DEP), we predict the modeled long-term mean annual lateral flux of soil organic carbon for each hillslope. A shared model already provides predictions throughout a watershed.

When modeled values become available for a few hillslopes, should we use them to revise predictions for the others?

Table 3 illustrates why we report how often a watershed’s prediction error increases. Always applying the local correction at a fixed amplitude of 0.25 gave the lowest mean RMSE among the policies in Table 3 (4.845). Yet in 40.46% of those decisions, the watershed would have had lower error with its existing predictor. SafeAdapt gave a slightly higher mean RMSE (4.886), and error increased in 8.04% of all decisions. We also report the size of these increases because small and large regressions count equally toward that frequency. What either increase costs in a field-management decision remains an open question. SafeAdapt applied corrections in 24.35% of all decisions in Table 3. To test whether it selects better updates at a fixed update count, Table 4 compares SafeAdapt with ranking the same candidate corrections by their estimated local gain. In the ten-seed DEP study, the share of applied updates that increased held-out RMSE was 2.35 percentage points lower with SafeAdapt on the hydrological split and 1.21 points lower on the spatial-grid split. The environmental split was inconclusive. To address your question about published methods, we added a direct comparison with the RSC-GNN architecture of Zeng et al. (2026) in Appendix I, Table 23. We refit the spatial graph model on the DEP target using the same 27 predictor variables, the same 20 labeled hillslopes per watershed and the same held-out hillslopes as SafeAdapt. With seed 42, mean watershed RMSE for SafeAdapt and RSC-GNN was 4.869 versus 5.364 on the hydrological split, 4.874 versus 5.143 on the spatial-grid split, and 3.617 versus 3.914 on the environmental split. Across these 1,681 decisions, SafeAdapt applied 575 corrections and increased held-out RMSE in 188 cases. RSC-GNN applied all 1,681 corrections and increased RMSE in 723 cases. Each increase is measured against that method’s own frozen predictor. SafeAdapt started with lower error on all three splits, and RSC-GNN achieved the larger reduction from its own starting error on each. This single-seed experiment compares the two complete procedures. Table 4 addresses the narrower question of which updates to select.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.