- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
Suppose you have an existing forecaster that you trust to make good predictions about extreme events, but may be sub-optimal elsewhere. You also have access to other models that contain useful information, but might make errors on the extreme events. How should you combine them? Averaging the forecasts can destroy the predictive power of the original model on the extreme events. The solution in this paper is to fix quantiles of the baseline predictor, and allow the new models to update the forecast distribution between those quantiles. This is broadly sensible if you have reason to believe the selected baseline predictor quantiles.
For example given a collection of quantiles, this partitions the unit interval into a set of intervals of fixed probability. We freeze this distribution over the intervals, but then set the distribution within each interval to be the conditional distribution of a source model within that interval.
The paper uses a diagnostic measure called CRSD, and the main theorem controls how much updating the base predictor using the proposed algorithm changes this metric. It does not establish that the base predictor (or the updated predictor) is good in any absolute sense --- the premise is that the goal is to avoid substantially changing this metric. The proposed algorithm is just to optimize conditional squared error (Brier score) within each quantile interval, conditional on lying in that interval.
Critical strengths
The paper gives a well defined problem and the results seem to be correct.
Critical weaknesses
This is a very poorly written paper that is extremely difficult to understand. I suspect it is because very little care has been put into writing up the result. On a hunch given the poor quality of writing, I ran both the abstract and the introduction through pangram, and unsurprisingly it came back as 100% AI generated. This goes well beyond the copy-editing the authors suggest was used for the exposition in the AI disclosure. But I do not mean to AI-shame --- I do not think this is the main concern. Ultimately I do not think the authors have taken the care to make their result readable, and as a result the paper as written has very little value. I cannot recommend acceptance.
There are other strange expositional choices that make me question whether the paper was written with any care. For example, the body of the paper foregrounds an algorithm event-doubling QAST-E which obtains a sub-optimal regret bound, whereas its own Theorem B.1 gives a different algorithm with a substantially better regret bound. It is unclear if there is any reason to prefer the algorithm in the body --- the paper seems to give none.
The paper also gives essentially no evidence that the proposed algorithm improves over simply retraining the baseline. In the main experiment, the differences from the baseline seem to be essentially zero, with confidence intervals overlapping zero.
Questions for authors
N/A
Thank you for your suggestions on exposition, algorithm presentation, and the choice of comparisons. We have reorganized the narrative around preserving specified baseline probabilities while using the conditional shapes of source forecasts, and added direct comparisons of fixed-rate and event-doubling weights. The revision makes the connection between the construction, theoretical guarantees, and practical gains more explicit.
The revised abstract and introduction first explain the roles of the baseline and sources, then introduce the results and learning rules. Section 3.1 describes within-cell shape transfer; Section 3.2 states the anchor-count and CRSD guarantees and points to Appendix A.4 for the proof, while Section 3.3 explains the within-cell PIT displacement bound; and Section 4.2 introduces the fixed-rate aggregate-loss guarantee earlier and explains the role of event-doubling in source-weight analysis. Sections 3.3 and 4.3 now also contain short, blue-highlighted summaries of the anchor-selection and online-learning results, referring to Appendix D.2.5, Table 16, and Appendix D.2.4, Table 15, respectively. Appendix A.3 provides the algorithm, and the new Appendix D.2 connects the additional experiments to the review questions.
Regarding AI use, we used AI to polish the final wording, which may explain the AI-like phrasing you noted.
The revision introduces the constant-regret fixed-rate result earlier in Section 4.2, with the full proof in Appendix B.4, Theorem B.1. Under the full-feedback protocol, a fixed η∈(0,1/2] and no weight resets, exp-concavity of the square loss controls the actual aggregate forecast's cumulative loss relative to a fixed transferred expert.
Event-doubling connects the source-weight analysis to the distributional-improvement theorem. Its bound on the linear weighted expert loss is used in Corollary B.1 and subsequently in Theorem 4.2, explaining how an advantageous source receives greater weight and yields improvements in the full forecast distribution. The actual aggregate square loss equals the weighted expert loss minus a nonnegative diversity term, so these two loss controls serve different proof tasks.
We have added the direct weather replay in Appendix D.2.1, Tables 11–12, and the controlled simulation in Appendix D.2.4, Table 15. Section 4.3 now summarizes the simulated learning gains. With strongly differentiated sources and weights frozen after 800 historical events, both rules have positive pointwise Monte Carlo CRSD improvement intervals relative to static QAST: the mean improvement is 0.02293 for event-doubling and 0.05377 for fixed-rate EW (η=0.5). The weather replay uses η=0.25 as its primary fixed rate; the online increments there are small, with the main empirical gains arising from protection.
QAST therefore supplies the same protected forecast class for both rules. Fixed-rate EW has the stronger aggregate-loss regret order, while event-doubling supports the source-weight and usefulness arguments developed in the paper. Users can select the rule according to their learning objective while retaining the same pathwise protection.
QAST addresses a specific application need: multiple predictive distributions are available, and certain baseline probabilities have a meaning that should be preserved. It transfers source shapes at the distribution level, with explicit preservation constraints and an ex ante perturbation bound. This allows it to serve as a forecast-combination step on top of an existing baseline.
Appendix D.1.3 adds a further-training DRN comparator using physical-scale CRPS plus a tail-calibration penalty, with validation selection of the penalty strength. Table 1 and Appendix Tables 4 and 6 report selection and paired comparisons. At p95, the mean CRPS difference, QAST-E minus the fine-tuned DRN, is only +0.000172 mm, with a pointwise paired 95% interval of [−0.001374, +0.001477] mm; QAST also preserves the baseline probability structure specified by its construction.
The matched unprotected comparison further demonstrates the practical value of protection. At p95 with J=5 and κ=0.75, twCRPS decreases by 27.5% under EMOS and 27.2% under DRN, with paired intervals supporting both reductions. The static configurations in Section 5.3 also illustrate shape-adjustment space within the protected class: retaining two internal tail-quantile anchors gives p90/p95 CRSD point reductions of 9.68%/9.47% relative to the corresponding configuration-specific baseline. Together with the pathwise protection theorem, these results establish the value of QAST as a distributional forecast-combination method.