acceptodds
Under review as a conference paper at ICLR 2027

Conformalized Weather Forecasting: Distribution-Free Coverage Guarantees for Neural Weather Models

Abstract

Conformal calibration of a weather field must specify what is covered: an individual location, an average fraction of grid cells, or a field-level event. We study the event that at least 90% of unweighted grid cells lie inside their forecast intervals. A spatial order-statistic score gives the standard finite-sample split-conformal guarantee for this event under exchangeable forecast–verification pairs. A chronological Stormer audit instead tests where this assumption can fail: using 240 calibration cases from 2018–2019 and 336 test cases from 2020–2022, global calibration attains a 71.4% field-event rate. Six latitude bands attain 97.0% with mean width 4.27 K, but their worst-band score-event rate is only 48.2%; aggregate field coverage does not imply simultaneous regional validity. We also audit label timing using the same saved forecasts. For a fixed three-band rolling window, increasing an assumed label delay from zero to 90 days reduces field-event coverage from 98.2% to 89.6%, while width changes little. These are retrospective sensitivities using final archived targets, not recovered ERA5T vintages. The contribution is an explicit coverage-target and label-availability audit; operational validity under temporal shift remains outside the exchangeable guarantee.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
conformal prediction, weather forecasting, field coverage, temporal shift, label availability

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_dU6YOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper applies split conformal prediction to deterministic neural weather forecasting models like Stormer and ClimaX. The authors define a spatially aware nonconformity score based on the 90% percentile of the absolute error field. This score targets a field level coverage event to guarantee that at least 90% of grid cells fall within the prediction interval. The work also explores Mondrian conformal prediction by stratifying the globe into latitude bands to account for regional error differences. The authors evaluate these methods on ERA5 reanalysis data. They highlight the difference between mean grid coverage and field event rates. Furthermore, they conduct chronological stress tests to assess performance under temporal distribution shifts.

Critical strengths
  1. The paper articulates the difference between average spatial coverage and field level event coverage. This is a subtle but important distinction when evaluating spatial forecasts.

  2. The chronological stress test honestly exposes the limitations of the exchangeability assumption in standard conformal prediction when applied to real world time dependent weather data.

  3. The ablation studies on label delay sensitivity provide a realistic look at operational constraints.

Critical weaknesses
  1. Lack of Novelty: Applying standard split conformal prediction using a spatial quantile as the nonconformity score is a trivial methodological extension.

  2. Assumption Failure: The core theorem in Proposition 1 requires exchangeability. Weather data inherently violates this requirement. The chronological tests presented by the authors show the global conformal prediction field event rate drops to 71.4 percent compared to the 90 percent target. This renders the theoretical guarantee practically void.

  3. Metric Relevance: The field level event metric can mask severe regional failures. The paper admits that under the Mondrian approach the worst band coverage drops to 48.2 percent. This means the uncertainty quantification is highly unreliable in specific regions.

  4. Missing Baselines: The paper fails to compare against established probabilistic ensemble methods like GenCast or ECMWF ENS. This omission makes it impossible to assess if this post hoc calibration offers any competitive advantage over current standard practices.

  5. Poor Performance on Extremes: The method systematically fails to provide adequate coverage during extreme weather events. The coverage drops to 66.7 percent for 99th percentile anomalies. This is arguably the most important use case for uncertainty quantification.

Questions for authors
  1. Weather data inherently violates the exchangeability assumption as demonstrated by the chronological stress test dropping to 71.4 percent coverage. Under this condition what is the practical value of Proposition 1? Why not use adaptive conformal inference methods explicitly designed for time series?

  2. The worst band score event rate is only 48.2 percent under the Mondrian approach. How can an operational meteorologist trust this calibration method for regional warnings?

  3. There is no comparison against probabilistic neural weather models like GenCast or FuXi ENS. Comparing the quality of the uncertainty quantification is necessary to benchmark your approach against the state of the art even if these models use different backbones. Why was this comparison omitted?

  4. How do you justify the inclusion of the EVT fine tuning negative results? In what way does this failed experiment support the main narrative of conformal prediction?

3 comments

Thank you for the questions. The added experiments directly address regional coverage and adaptive calibration: joint regional calibration substantially improves simultaneous coverage, and ACI improves delayed-feedback coverage with finite intervals at gamma=0.005.

Regional and extreme-event results

We evaluated the same 336 Stormer test forecasts on six latitude bands. Alongside separate band calibration, we calibrated the maximum of the six band scores to target their simultaneous event. Both rules use the same 240 calibration forecasts.

Each band event requires coverage of at least 90% of its unweighted cells.

CalibrationWorst-band event rateAll six bands simultaneouslyMean width
Separate six-band calibration48.21%16.96%4.268 K
Joint calibration of the maximum score82.44%77.68%11.833 K

Joint calibration raises the worst-band rate from 48.21% to 82.44% and simultaneous coverage from 16.96% to 77.68%. The latter has a 30-day-block 95% interval of [70.54%, 84.04%], compared with [11.14%, 23.12%] for separate calibration. The joint rule uses one common width: it is substantially wider and still falls below 90% simultaneous coverage on this chronological split.

For cells exceeding the same calibration-defined 99th-percentile absolute-anomaly threshold:

MethodArea-weighted extreme-cell coverageOverall mean width
Fixed global66.72%5.681 K
Separate six-band79.98%4.268 K
Joint six-band91.96%11.833 K

These results show how changing the calibration target improves regional and extreme-cell coverage, together with its width cost. They do not turn the original 97.02% global field-event rate into a guarantee for regional warnings.

Adaptive calibration under temporal shift

Rolling calibration and adaptive conformal inference (ACI) use identical forecasts and the latest 60 eligible labels. Feedback arrives only at verification time plus the assumed delay.

Label delayMethodGlobal field-event rateMean width
0 daysRolling87.80%6.247 K
0 daysACI, gamma=0.00589.29%6.401 K
90 daysRolling85.42%6.231 K
90 daysACI, gamma=0.00589.29%6.608 K

All intervals in these rows are finite. At a 90-day delay, ACI gains 3.87 percentage points for an additional 0.377 K of width. We also tested gamma=0.01 and 0.02. Larger step sizes can raise coverage further, but introduce unbounded intervals; gamma=0.01 reaches 90.77% with 23.51% unbounded dates at the 90-day delay.

Proposition 1 specifies the event and conditions under which it is guaranteed. It does not certify chronological coverage. The adaptive comparison measures what can be recovered under temporal shift and delayed labels. Release delays remain retrospective assumptions applied to final archived targets.

Ensembles and EVT

The application is interval construction for an existing deterministic forecast without retraining or ensemble generation. We have not performed matched GenCast, FuXi ENS, or ECMWF ENS comparisons, so no advantage over those systems is claimed.

We will remove the EVT fine-tuning experiment from the main narrative. The revised empirical conclusion will focus on the measured gains from regional joint calibration and adaptive feedback, with interval width reported alongside coverage.

I have carefully considered the authors' rebuttal and the other reviewers' comments. Although the authors have resolved several of my initial concerns, the fundamental issue of limited novelty persists. Utilizing a spatial quantile as the nonconformity score in standard split conformal prediction represents only a marginal methodological increment. Consequently, I intend to keep my original rating.

Thank you for reconsidering the added evidence and for clarifying the remaining concern. The contribution beyond the standard split-conformal theorem is the specification and audit of operational coverage targets for weather fields. A complete forecast field is the sampling unit, spatial dependence is retained, and field, regional, chronological, and label-availability questions are evaluated within one framework.

The added experiments show why this matters. Separate six-band calibration attains 97.02% global field-event coverage, but only 16.96% simultaneous six-band coverage; calibrating the maximum regional score raises simultaneous coverage to 77.68%. Under a 90-day label delay, ACI raises field-event coverage from 85.42% to 89.29% with finite intervals and a 0.377 K width increase. We will foreground these target-specific and deployment-relevant findings in the revision.

Reviewer_hjQbOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies post-hoc interval calibration for deterministic neural weather forecasts, distinguishing pointwise coverage, average grid-cell coverage, and field-event coverage. The authors use a spatial quantile of forecast errors as the conformal score to construct a standard split-conformal guarantee for the event that at least 90% of grid cells are covered. Latitude-band calibration, chronological evaluation, and simulated label-release delays are used to audit empirical coverage and its limitations.

Critical strengths
  1. The paper clearly distinguishes different coverage targets, helping prevent aggregate coverage from being misinterpreted as reliability at every location.
  2. The theoretical scope is carefully stated. The authors explicitly acknowledge that the guarantee requires exchangeability and interpret chronological results as empirical stress tests.
  3. The label-delay audit is practically relevant. Reusing the same forecasts while varying label eligibility helps isolate the effect of calibration-label availability.
Critical weaknesses
  1. Reported widths, coverage rates, and sample counts differ across tables without sufficient explanation. A unified description of the experimental protocols would improve consistency.
  2. The sampling unit underlying the Wilson confidence intervals is unclear. Uncertainty estimates should distinguish grid coverage from field events and account for temporal dependence.
  3. A matched adaptive conformal baseline would strengthen the temporal-shift analysis.
  4. Since the theoretical construction follows standard split conformal prediction, the paper could more clearly establish the additional value of its auditing framework.
Questions for authors
  1. Could you provide a unified experimental protocol table explaining the differences between 15.47 K and 5.68 K, between 91.7% and 92.9% coverage, and the different ClimaX sample counts?
  2. Do the Wilson intervals in Table 5 describe average grid-cell coverage or binary field-event coverage? Could you provide uncertainty estimates that use forecast dates as the sampling unit and account for temporal dependence?
  3. Under identical label-delay rules, could an adaptive conformal baseline improve the coverage–width trade-off?
3 comments

Thank you for the questions. Under identical label-delay rules, adaptive conformal inference improves coverage with a modest width increase at gamma=0.005. We also added date-block uncertainty estimates and checked the protocol differences.

Matched adaptive comparison

Rolling calibration and adaptive conformal inference (ACI) use identical Stormer forecasts, spatial scores, and the latest 60 eligible labels. Labels become eligible at verification time plus the assumed delay, and ACI receives feedback only then. Both methods use the same finite-sample quantile rank.

Label delayMethodField-event rateMean width
0 daysRolling87.80%6.247 K
0 daysACI, gamma=0.00589.29%6.401 K
90 daysRolling85.42%6.231 K
90 daysACI, gamma=0.00589.29%6.608 K

All four rows have finite intervals on every date. With a 90-day delay, ACI improves coverage by 3.87 percentage points for a 0.377 K width increase, approaching but not reaching the 90% target.

We tested gamma=0.005, 0.01, and 0.02; the table highlights the setting with finite intervals throughout. Larger step sizes sometimes increase coverage further but introduce unbounded intervals. For example, gamma=0.01 at a 90-day delay reaches 90.77% coverage with 23.51% unbounded dates. We will report the full sweep alongside the finite-width comparison. An empirical-quantile sensitivity check also retains the delayed-feedback width problem.

These experiments use final archived targets with assumed release delays, rather than recovered historical release vintages.

Date-level uncertainty

The field event is binary for each forecast date: at least 90% of unweighted cells are covered. We used 10,000 bootstrap replicates of 30-, 60-, and 90-day calendar blocks, keeping each spatial field intact.

MethodField-event rate30-day-block 95% interval90-day-block 95% interval
Fixed global71.43%[63.20%, 79.18%][60.83%, 80.97%]
Fixed six-band97.02%[94.59%, 99.07%][94.56%, 99.11%]

The historical table reports average grid coverage, not binary field events. We will remove the Wilson intervals applied to those averages and use these date-block estimates for the chronological results. The intervals describe sampling variability under the block-resampling scheme, not a guarantee under temporal shift.

Protocol clarification

The earlier global result is 91.75% grid coverage at 14.29 K. The 15.47 K width belongs to an 80-case calibration ablation with 92.92% coverage; we will correct this pairing. The 5.68 K result uses the chronological split of 240 calibration cases from 2018–2019 and 336 test cases from 2020–2022. ClimaX uses 60 calibration and 60 test cases. A single protocol table will distinguish these experiments.

The added comparison shows that adaptation can improve the coverage–width trade-off, while the delay and step-size checks identify when higher coverage comes at an excessive width cost.

Thank you for the clarifications and additional experiments. The protocol correction affects the reported width comparison with the Gaussian baseline. My concern about the limited contribution beyond standard split conformal prediction also remains. Therefore, I maintain my recommendation to reject.

Thank you for pointing this out. The protocol correction changes the Gaussian width comparison. The historical global conformal result is 91.75% grid coverage at 14.295 K; 15.473 K and 92.92% belong to a separate 80-case calibration ablation. Relative to the 13.52 K Gaussian width, the corrected difference is 0.775 K, or 5.73%, rather than 14%. We will correct the comparison and separate the protocols in one table.

The contribution beyond standard split conformal prediction is a unified weather-field audit: it treats a complete field as the sampling unit, distinguishes mean grid coverage from field and simultaneous regional events, and incorporates label availability into chronological evaluation. This produces actionable comparisons. Six-band calibration raises the field-event rate from 71.43% to 97.02% while reducing mean width from 5.681 K to 4.268 K; joint regional calibration raises simultaneous six-band coverage from 16.96% to 77.68%. We will foreground these target-specific findings and the adaptive comparison in the revision.

Reviewer_9HkaOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The manuscript uses conformal prediction, a post-training uncertainty quantification method, on pre-trained medium term weather forecasting models. Using a spatially-aware non-conformity score and stratification by latitude bands the method achieves better local coverage. Compared to naive conformal predictions and pointwise Gaussian calibration the method shows better coverage when evaluated on subsets of the ERA5 data using Stormer and ClimaX backbone.

Critical strengths

The stratified version of conformal prediction using the spatially aware non-conformity score is to the best of my knowledge novel. Computationally efficient methods for uncertainty quantification on large foundational weather models is attractive. The method is evaluated on ERA5 data using relevant pre-trained deterministic backbones.

Critical weaknesses

The manuscript fails to properly motivate the need for the method. As a non-expert on weather and climate science, I find it difficult to get a clear picture of why a post-hoc calibration on deterministic models is something that the community is interested in. In light of the data driven weather forecasting field having moved to models that natively includes uncertainty quantification such as GenCast, I would have expected either a quantitative comparison or a strong motivation for why this fits a different application. The manuscript notes “Conformal calibration addresses a different question: whether a chosen prediction set attains a specified frequentist coverage target under explicit assumptions.” but then fails to follow up with motivation for why that particular question is interesting or why the “explicit assumptions” are relevant or reasonable for applications.

The manuscript has multiple critical presentation issues that makes it hard to read: The main figures 1 and 2 have no bulk references and it is also unclear what figure 2 adds to the understanding of the reader. The idea of having a figure that explains the concept of conformal prediction and how the proposed method fits in is good, but this needs to be clarified and properly put into context. Section 7 contains no conclusions on the comparison between the proposed method and baselines, this makes the reader wonder if there are no strong conclusions to be made and if so why the paper is interesting. There are multiple segments where where I encourage the authors to reformulate so that the reader is not worried that there has been no human supervision of the text, e.g. line 318 “This negative result is not part of the conformal method; we move its setup, ablations, and possible explanations to Appendix B.” See questions below for more examples.

Questions for authors

“Figure 1: Seasonal mean” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 2) There are no references to figure 1 in the bulk.

“exchangeability” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 2) At this point in the manuscript the reader is confused about what exchangeability refers to. Is the assumption that new/held-out data is "exchangeable" with the calibration set? Same generative process?

“Constructed example for definitions only — not ERA5 or model evaluation data” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 3) Duplicated in caption.

“Figure 2: A co” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 3) There is no reference to figure 2 in the bulk.

“Standard MSE training minimizes E[rs2,τ ], which is dominated by the bulk of the distribution and systematically neglects the tail {rs,τ : |rs,τ | > us} for a high threshold us” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 3) This statement does not have a follow-up? What is the implications for the problem at hand, or the method proposed?

“Extreme-value modeling is background to a separate fine-tuning diagnostic, not an assumption of the conformal guarantee. Its peaks-over-threshold formulation is deferred to Appendix N.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) I can't parse the first part of this, what does the statement "... is background to a separate fine-tuning diagnostic..." imply/mean?

“Definition 1 (Marginal Coverage). C has (1−α)-marginal coverage if P(y ∈ C(x)) ≥ 1 − α.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) It seems like the former section is written under the assumption that the reader has read this definition first?

“Given pre-trained fθ” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) There is no f_\theta in (1), where does the model enter? I assume there should be a definition of s including the prediction from f_\theta.

“Separate stratum-wise statements do not imply simultaneous validity over all strata; see Section 4.2” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) An example would be helpful.

“from the theorem” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) which theorem?

“q-th spatial quantile” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) Is there some interplay here between spatial correlation and the spatial quantiles? If the model is wrong about e.g. a weather system, the grid points in an entire region contributes to high error, and the quantile will be pushed by correlated spatial points.

“where we set q = 0.90” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 4) Is 0.9 grounded in application? Is it relevant to think about how this results depend on this parameter?

“conformal quantile” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 5) nothing in the statement above uses "conformal quantile"

“We use ERA5 reanalysis (Hersbach et al., 2020) 5-day forecasts with ncal = 240 samples as the calibration set” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 5) How are they distributed in time compared to the evaluation set?

“This negative result is not part of the conformal method; we move its setup, ablations, and possible explanations to Appendix B.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 6) There seems to be some archeology here of previous versions?

“Pointwise Gaussian residual interval” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 7) As far as I can see, there is no definition or reference to what exactly this baseline is computing. This is calibrating each grid site and each lead time separately against the same calibration data using a Gaussian prior?

“Table 1: Chronological day-5 T2m stress te” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 8) Which rows are the authors proposed method? I assume the last two.

“Field event” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 8) Would be good to clarify how this is different from the statement that the first two rows are below 90% coverage.

“Chronological stress-test results.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 8) From the way section 5, subsection data is written, I was expecting some results on the 2020 data as the main results, but here the "chronological stress-test results" is presented first?

“erval [0.88,6.41]); width increases by 0.387 K [0.318,0.456].” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 8) I would suggest other notation in a manuscript for the intervals.

“final-vintage” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 8) What does final vintage mean?

“We formulate deterministic neural-weather calibration as control of a field-level spatial-risk event. Under exchangeability, split conformal prediction gives finite-sample coverage for this explicit target; it is neither location conditional nor robust to arbitrary temporal shift. The chronological stress test shows that regional adaptation can improve empirical coverage, while the detailed results and assumption boundaries are reported in the appendix. A label-matured rolling-window reanalysis improves empirical coverage at the cost of wider intervals (Appendix J); it does not establish shiftrobust coverage.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 9) Why is there no conclusion about the comparison between the proposed method and baselines?

“Retained evaluation code and saved-prediction analyses distinguish mean grid coverage from the field-level event.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 10) What does "Retained evaluation code" mean? Is the code available?

“Available reanalyses do not establish validity under arbitrary temporal shifts or independently reproduce all backbone training” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 10) This sounds like a limitation rather than a reproducibility statement?

“Naive Gaussian vs. conformal intervals. The naïve Gaussian interval with ±1.645σ (theoretically 90% for Gaussian errors) achieves only 88.5% empirical gridpoint coverage, falling short of the 90% target. Real atmospheric residuals have fatter tails than a Gaussian, so the ±1.645σ quantile systematically underestimates the required interval width. Our Global Split CP achieves 91.7% coverage for T2m with only 14% wider intervals (15.47 K vs. 13.52 K). The split-conformal fieldevent guarantee requires exchangeability but makes no parametric assumption on the error distribution. Latitude-banded CP has higher empirical T2m grid coverage in this dataset, at 94.2%, with widths adapted by latitude. The corresponding Global/latitude-banded empirical coverages are 92.4%/94.4% for Z500 and 91.7%/93.8% for T850. We report the 88.5% Gaussian comparison only for T2m and do not extrapolate that value to the other variables. All results use ncal = 240 calibration samples and ntest = 183 test samples on 5-day forecasts.” (“CONFORMAL WEATHER FORECASTS: FIELD-COVERAGE AND LABEL-TIMING AUDITS”, p. 17) The core of this discussion would be helpful to include in the bulk manuscript, it motivates the need for the CP scheme.

2 comments

Thank you for the detailed comments. The matched chronological comparison shows that latitude-band calibration improves field-event coverage while reducing interval width. We also completed a spatial-target sensitivity analysis.

Matched results

All methods use 240 Stormer calibration forecasts from 2018–2019 and the same 336 test forecasts from 2020–2022. The field event requires at least 90% of unweighted grid cells to be covered on a forecast date.

MethodMean grid coverageField-event rateMean width
Fixed global95.16%71.43%5.681 K
Fixed three-band93.46%94.94%5.293 K
Fixed six-band92.21%97.02%4.268 K
Rolling three-band, no added label delay93.86%98.21%5.680 K

Six-band calibration raises the field-event rate from 71.43% to 97.02% and reduces mean width from 5.681 to 4.268 K. Mean grid coverage, which is area weighted, gives a different ordering. This is why the coverage target matters. Global field coverage still does not establish regional reliability: the worst-band event rate is 48.21%.

Application and assumptions

The application is to add intervals to an existing deterministic forecast without retraining or ensemble generation. This provides a directly measurable coverage target for users of that forecast; it is not a replacement for a probabilistic weather model.

The score is the ceil(qG)-th smallest absolute forecast error among G cells. Each complete forecast–verification field is one calibration observation, retaining its spatial dependence. Exchangeability concerns the joint distribution of calibration and test fields under permutations, not independent grid cells. The chronological results are empirical measurements, not exchangeability-based guarantees.

We tested q=0.80, 0.85, 0.90, 0.95, and 0.99 with the same 240 calibration cases. For global calibration, width rises from 3.420 K at q=0.80 to 14.685 K at q=0.99, while the corresponding field-event rates remain between 70.54% and 74.40%. Thus q controls spatial tolerance and width; increasing it does not resolve temporal shift. The choice 0.90 is not a universal weather-warning threshold.

Clarifications in the revision

The earlier global result is 91.75% grid coverage at 14.29 K; 15.47 K belongs to the 80-case calibration ablation with 92.92% coverage. We will correct the pairing and list these protocols separately from the chronological experiment. ClimaX uses 60 calibration and 60 test cases; we will exclude it from cross-backbone claims pending preprocessing validation.

The chronological Gaussian baseline is centered on the deterministic forecast, with half-width 1.64485 times the local calibration residual sample standard deviation. We will define it explicitly, introduce the coverage definitions before the method, reference both figures in the text, and remove the unrelated EVT discussion and the claim that MSE systematically neglects tails. “Final archived targets” will replace “final-vintage.”

These results support calibration of fixed deterministic forecasts. A matched GenCast or ECMWF ENS comparison has not been performed.

I thank the authors for their replies that helps sort out some of the concerns I raised. There are still some outstanding questions that I do not find further support for in the replies:

  1. Applications and value to the community: "The application is to add intervals to an existing deterministic forecast without retraining or ensemble generation. This provides a directly measurable coverage target for users of that forecast; it is not a replacement for a probabilistic weather model." Having systematic methods to add coverage capabilities to deterministic models sounds useful, but where is the domain of weather forecasting models where a user would use the deterministic models over ensembles or UQ-aware predictive models? It would be helpful for the reader to see a clear example of where this trade-off makes sense.

  2. In my review I noted that "Section 7 contains no conclusions on the comparison between the proposed method and baselines, this makes the reader wonder if there are no strong conclusions to be made and if so why the paper is interesting." I still do not see a clear statement regarding the sittuation in the replies. If there is such a conclusion that the authors can draw from their results, I would encourage a clarification and the inclusion of such statements in future revisions of the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.