acceptodds
Under review as a conference paper at ICLR 2027

Sequential Regularized Ensemble Forecasting by Online Policy Learning

Abstract

Aggregating point forecasts from multiple experts often improves accuracy relative to relying on a single expert, but sequential ensemble forecasting becomes difficult for time series data when expert performances change over time. In this paper, we formulate sequential ensemble forecasting as an online policy learning problem and incorporate the recently proposed offline Regularized Ensemble Forecasting (REF) framework. At each forecasting period, the policy objective selects the optimal ensemble weight by balancing the variation of current forecasts, shrinkage toward historical performances, and stabilization toward preceding optimal weights. In the case of a large expert pool, we impose a sparse parameterization that permits exact zero weights and facilitates expert selection over time. Under suitable conditions, we develop an online regret bound that vanishes as the total number of periods goes to infinity. Numerical studies, including applications to the M5 competition and the Survey of Professional Forecasters (SPF), show that the proposed online REF policy performs favorably against standard ensemble benchmarks and offline REF.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Ensemble forecasting, Penalization, Wisdom of crowds, Entmax parameterization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_PsEUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This paper studies ensemble forecasting: given the forecasts of NN experts, how to combine them into one forecast when the relative accuracy of the experts changes over time. It builds on an offline regularized ensemble forecasting criterion and proposes an online version that, in each period, minimizes three terms: the dispersion of the ensemble around the contemporaneous average of the experts, a shrinkage toward a prior built from historical errors, and a proximal term that keeps the weights close to the previous period. Only one scalar, the common-correlation parameter of the prior, is updated online, and large expert pools are handled with an entmax parameterization for sparsity. The paper proves an O(1/T)O(1/\sqrt{T}) regret bound against the best fixed combination, and reports first place on M5 (154 series, 50 team forecasts) and on the SPF survey.

Critical strengths

Forecast combination is a practically important problem and the formulation is easy to read: each of the three terms has a clear meaning, and none of them is a black box. The experiments on real data are visible. There are two real forecasting settings, 50 real team forecasts in M5 and real professional forecasters in SPF, a broad baseline set, and a consistent first place, and the improvement over the simple mean is not trivial (about 8% on M5).

Critical weaknesses

I do not think there is online policy learning here. No state-to-action function and no value function are learned, and each period's weights are just the solution of a convex program, while the prior is a rolling-window statistic and the only learned object is the scalar correlation parameter. In substance this is a sequence of proximally regularized convex programs, i.e. online convex optimization with a shrinkage-plus-proximal regularizer, so the title overstates the contribution.

The theory is also mismatched to the motivation and weak relative to the online-learning literature. The regret is against the best fixed combination, and the assumptions (a time-invariant error second-moment matrix, a 1/t1/t convergence of the prior weights) essentially impose stationarity, which is the regime the paper claims to improve upon. The rate is the generic online convex optimization rate, so the proximal term, which is the only new structural ingredient, is never shown to help. The standard adaptive aggregation baselines (BOA, MLpol, discounted Hedge) are not compared against, and there is no dynamic-oracle regret, which would be the right target here.

The gains over the closest baseline are modest and not significance-tested (about 2.6% over offline REF on M5 and 0.7% on SPF GDP, over a 7-day evaluation window on M5), and there is no ablation that removes the proximal term.

Questions for authors
  1. What is learned here, beyond the scalar correlation ρ\rho? Please state precisely the state, action, and objective that make the problem an instance of policy learning rather than online convex optimization.
  2. Can you compare against BOA, MLpol, and discounted Hedge/EWA? These are the standard online-aggregation baselines; a comparison on M5/SPF would directly test the "online" claim.
3 comments

Thank you for reviewing our paper and providing valuable feedback. We have made several revisions to the manuscript, with the changes highlighted in blue. Below, we address the critical weaknesses (W1, W2, W3) and questions (Q1, Q2).

W1. & Q1. Online policy learning

Thank you for raising this point. We use “policy” as a context-to-action mapping (Hazan & Singh, 2021). At period tt, the state is St=(y^t,st,w^t−1)S_{t}=(\widehat{\mathbf{y}}_{t},\mathbf{s}_{t},\widehat{\mathbf{w}}_{t-1}), consisting of current forecasts, prior weights based on historical forecasting performance, and preceding weights. The action is the selected weight vector w^t=πt(St)∈ΔN\widehat{\mathbf{w}}_{t}=\pi_{t}(S_{t})\in\Delta_{N}, and the objective is to minimize cumulative squared forecasting loss.

Under this formulation, Eq.(1) defines the weight-selection policy through an optimization problem. The online learning component updates the common-correlation parameter ρt\rho_t in Eq.(3) using the realized aggregate forecasting loss; this parameter enters the CCR prior st\mathbf s_t in Eq.(2). The prior weights still reflect individual experts' historical performance and change over time. Updating ρt\rho_t changes the CCR prior and, consequently, the weights selected by Eq.(1). As discussed by Agrawal et al. (2020), optimization-based policies can be learned through their parameters. We agree that the within-period weight selection is an optimization problem; our use of “online policy learning” refers to the online adaptation of this parameterized weighting rule, without requiring a learned value function.

We have revised Sections 2.1–2.3 to clarify the state, action, objective, and online update.

W2. Theorem

Thank you for your comments. Theorem 1 compares online REF with the best fixed convex combination of expert forecasts selected in hindsight. This is a standard regret in online learning and forecast aggregation (e.g., Zinkevich, 2003; Shalev-Shwartz, 2012; Wintenberger, 2017; Gaillard et al., 2014). We have clarified this connection in Remark 1.

In this revision, we have relaxed both the second-moment and prior-weight assumptions (Assumptions 2(ii) and 4 in Appendix A). The conditional error second-moment matrix may now vary over time. For the projected prior weights, we replace the 1/t1/t mean-squared convergence requirement with a weaker bound on their cumulative expected distance from the population-optimal weights.

We agree that dynamic-oracle regret is also a relevant theoretical target (e.g., Zhao et al., 2020; Jacobsen and Orabona, 2024). Establishing such a bound for online REF would require substantial additional theoretical work, and we leave this extension for future work, as discussed in the Discussion section.

W3.& Q2. Real Data (M5/SPF)

Thank you for the suggestion! We have added BOA, MLpol, and discounted Hedge/EWA in both M5 and SPF applications. The results are given in Tables 1 and 2. Online REF achieves the best performance over all benchmarks.

We also report statistical significance tests for improvements in these tables. The improvements over the compared benchmarks are statistically significant in most comparisons.

We have also added Online REF with κ=0\kappa=0, which removes the preceding weights penalty. The results are given in Tables A.4 and A.5 of Appendix G.2. Removing the preceding weights penalty from Online REF generally increases forecast errors relative to the full Online REF, but it is generally better than Offline REF.

Thanks author for the revision. The theoretical concerns (W2) are now adequately addressed, and I appreciate the additional baselines, the significance tests and the ablation. However, the new results confirm that the gain over the closest baseline is modest and not significant on the SPF GDP comparison, and that the proximal term contributes only marginally, while the central framing of online policy learning is restated rather than resolved. I therefore keep my rating at 2, but I have raised my soundness score to 3 in recognition of the improved theory and experiments.

Thanks a lot for your reply and for raising the soundness score. We would like to provide a few clarifications.

  1. Although the numerical differences in forecast errors between comparable methods look small, which is very common in the M5 and SPF applications in the ensemble forecasting literature (e.g., Makridakis et al., 2022; Jose and Winkler, 2008; Su et al., 2026), the improvement of Online REF over the closest baseline, Offline REF, is statistically significant at the 0.05 level in most of the comparisons, including in the M5 data, SPF CPI inflation, and SPF change in the unemployment rate. SPF nominal GDP growth is the only exception where Online REF is not significantly better than Offline REF, but it still slightly reduces RMSSE. This demonstrates that overall, Online REF substantially improves over Offline REF. Moreover, across all 12 benchmarks under both M5 and SPF, the benefit of Online REF is statistically significant in 47 of 48 comparisons at the 0.05 significance level.

We have also performed statistical tests on the improvement of adding the proximal term. The contribution of the proximal term is significant at the 0.05 level in M5, SPF CPI inflation, and SPF change in the unemployment rate, as shown below:

DatasetIndicatorOnline REFOnline REF (κ=0\kappa=0)p-value
M5—0.3370.341<0.001*
SPFGDP1.0331.0400.282
SPFCPI0.8920.9270.033*
SPFUnemployment rate1.4681.5330.018*

We will add the new results in the finalized version.

  1. As noted in our reply to W1 & Q1. Online policy learning, we have revised Sections 2.1–2.3 to clarify the state, action, objective, and online update. At period tt, let St=(y^t,st,w^t−1)S_t=(\boldsymbol{\hat y}_t,\boldsymbol{s}_t,\boldsymbol{\hat{w}}_{t-1}) denote the state, consisting of current forecasts, prior weights based on historical forecasting performance, and preceding optimal weights. The decision maker selects the action w^t=πt(St)∈ΔN\boldsymbol{\hat{w}}_t=\pi_t(S_t)\in\Delta_N, where πt\pi_t is the policy that maps the current state to the selected weights (Hazan and Singh, 2021). The policy πt\pi_t is obtained by Eq. (1), specifying how forecast weights are selected in each period by solving an optimization problem. Optimization-based policy has been used in the policy learning literature, e.g., Agrawal et al. (2020).
Reviewer_jEZXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper extends regularized ensemble forecasting to an online setting. At each period, the weights minimize a criterion that combines the dispersion of current forecasts, shrinkage toward weights based on past performance, and a penalty on changes from the previous weights. An entmax parameterization is used to select a subset of experts. The paper gives a regret-type guarantee against the best fixed combination and experiments on M5 retail data and the Survey of Professional Forecasters.

Critical strengths

The paper addresses a useful problem. Combining forecasts while avoiding unnecessary changes in weights is a sensible approach when expert performance varies over time. The experiments use real forecasting applications, and the improvements over conventional forecast combinations are encouraging.

Critical weaknesses
  1. The sparsity mechanism does not follow from the stated objective. For α>1\alpha > 1, entmax maps onto the whole simplex, so minimizing Qt(entmaxα(θ))Q_t(\mathrm{entmax}_\alpha(\theta)) over θ∈RN\theta \in \mathbb{R}^N gives the same optimal weights as minimizing QtQ_t over the simplex. For the identity specification, QtQ_t is strictly convex, so these weights are unique and do not depend on α\alpha. The α\alpha-dependent number of selected experts in Fig. A.1, and the validation choice of α\alpha, must therefore come from the numerical procedure, which the paper does not describe. A likely cause is that entmax has zero gradient in the coordinates it sets to zero, so a gradient search over θ\theta can stop with experts excluded that belong in the optimum.

  2. The comparisons do not isolate the online contribution. There is no standard online weight-learning baseline such as exponential weights or online gradient descent on the simplex. This is notable because Theorem 1 is stated against the best fixed combination, which is exactly what those methods target. The comparison with offline REF also changes several things at once: offline REF averages six specifications, including entropy penalties, while online REF averages three and adds entmax and the online prior update. It is therefore unclear how much the penalty on weight changes contributes.

Questions for authors
  1. Do the gains, and the selected experts, remain when the weights are computed exactly, for example with a closed-form or KKT-verified solver for the identity case? Please report the optimality gap or KKT residual of the computed weights, and whether α\alpha still affects accuracy.
  2. Can you include an online weight-learning baseline such as exponential weights, and an ablation that removes only the penalty on weight changes with everything else matched?
1 comment

Thanks a lot for your time and efforts to review the paper and give us valuable suggestions. In response to your comments, we have made several revisions to the manuscript, highlighted in blue. Our replies to critical weaknesses (W1, W2) and questions (Q1, Q2) are below.

W1. & Q1. entmax

We agree that, for a given identity-L2L_2 objective, entmax with α>1\alpha>1 does not change its theoretical minimizer, since entmax maps onto the entire simplex (Peters et al., 2019) and QtQ_t is strictly convex in w\boldsymbol{w}. However, the reparameterized objective Qt(entmax⁡α(θ))Q_t(\operatorname{entmax}_{\alpha}(\theta)) may not be convex in θ\theta, and the corresponding numerical optimization may not return the same minimizer. We clarify that the observed α\alpha-dependent sparsity arises from the numerical optimization procedure rather than different exact minimizers.

We have compared the entmax and closed-form implementations of identity-L2L_2 Online REF in both M5 and SPF applications. We have added results in Appendix H. The entmax implementation achieves lower RMSSE on M5 and SPF (Tables A.6 and A.7). The entmax implementation selected different and generally fewer experts than the closed-form solution (Figure A.2). We have also calculated KKT residuals in Appendix H. The entmax implementation generally leads to larger KKT residuals than the closed-form solution (Figure A.2), which is not surprising since their solutions are different.

W2. & Q2. add more benchmarks and comparisons

Thank you for the suggestion! We have extended our comparisons in both M5 and SPF applications to include more online benchmarks, including Exponential Weights (EWA/Hedge) and Online Gradient Descent (OGD) on the simplex (see updated Tables 1 and 2). Overall, Online REF achieves the lowest avgRMSSE on both M5 and SPF applications.

The reason why Offline REF averages six specifications while Online REF averages three in the M5 and SPF applications is to reflect real-world situations where the decision maker does not have any preference on any specification and thus averages the offline REF or online REF over all of their corresponding available specifications. We have made this clear in Section 5.1.

To eliminate such difference in specifications in the comparison, we have added and compared the results of Offline REF and Online REF under the same three specifications, including identity-L2L_2, log-L2L_2, and shifted-log-L2L_2, both individually and through the average of their forecasts. We have also included online REF with κ=0\kappa=0, which removes the preceding weights penalty. The results are given in Tables A.4 and A.5 in Appendix G.2. For the three-specification average, Online REF outperforms Offline REF in both M5 and SPF applications. Removing the preceding weights penalty from Online REF generally increases forecast errors relative to the full Online REF, but it is generally better than Offline REF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.