- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper first trains a spatial combiner on historical forecasts and station observations. An online layer then combines raw forecasts, the learned combiner, and calibration baselines to produce station corrected probabilistic forecasts. The authors also derive a regret bound against the best fixed mixture in hindsight under the continuous ranked probability score and evaluate forecast quality across several station networks.
Critical strengths
The station level evaluation across regions is a concrete strength. Beyond average scores, the paper examines gains by terrain relief and freezes components of the online layer to separate weight adaptation from affine correction. These results make the improvements at mountain stations and their likely sources easier to assess than an aggregate score alone.
Critical weaknesses
In “Problem Formulation & Theoretical Results,” both the ideal prediction formula and its pseudocode put cumulative historical score in the exponent with a positive sign. The numerical algorithm instead uses a negative sign, as does the posterior update in the appendix proof. Since a larger cumulative score means a worse forecast, the positive sign gives greater weight to historically worse mixtures, opposite to the update required by the proof. This is more than a numerical integration difference: the stated ideal algorithm and the algorithm analyzed in the proof disagree, so the claimed regret guarantee cannot yet be verified for the algorithm as defined.
The table freezing components in “Ablations” shows that updating the affine correction alone captures most of the full online layer's gain, and the paper acknowledges the smaller effect of updating mixture weights alone. Meanwhile, “EMOS” describes fitting a Gaussian to consensus mean and spread, but does not establish whether that baseline receives the same station and lead specific online correction and feedback timing as the proposed method. The ablation helps explain the components within the proposed method, yet the comparison with this baseline could still mainly reflect different capacities for local correction. It therefore does not isolate the learned mixture's advantage over conventional postprocessing.
Questions for authors
Could the authors make the direction of the historical loss weight consistent across the ideal algorithm, numerical algorithm, and proof, and clarify which update the theoretical guarantee and reported experiments use?
Could the authors clarify how frozen weights are initialized in the ablation and provide a simple postprocessing comparison with the same station and lead specific feedback and online correction opportunities, so that local correction can be separated from learned mixing?
Thank you for fixing many of the mathematical issues I pointed out (sign flip, minimax, etc.). I think there are still a few issues that remain though:
I plan to maintain my score, as I think a few of my key concerns still stand.