acceptodds
Under review as a conference paper at ICLR 2027

Halo: Improving forecast accuracy through heteroscedastic estimation

Abstract

Heteroscedastic forecasting, where a network estimates a scale parameter alongside a location parameter, is normally motivated by uncertainty quantification. This paper shows it also improves the point estimate, in contrast to reported negative results for heteroscedastic estimation outside time series. Halo is a modification that reuses an existing deep forecaster's architecture, giving it a second output for the scale of its implied distribution and training it under the matching negative log likelihood. Adapting three state-of-the-art models — a transformer, a graph network paired with a variational autoencoder, and a single-layer convolutional network — under both Gaussian and Laplacian losses demonstrates the phenomenon. On the five electricity price markets of a standard forecasting benchmark, Halo improves MSE and MAE in 28 of 30 model-market-metric comparisons, cutting average MSE by 2.6% to 16.5% and average MAE by 1.7% to 11.0%. Two findings emerge: (1) whether the scale estimate comes from a second projection head or from a full parallel network matters far less than whether the network estimates scale, and (2) the improvement holds under the hyperparameters already tuned for the point-estimate baseline, so retuning is optional.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, heteroscadistic estimation, probabilistic forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_PaJAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Halo, a way of turning some pre-existing deep forecaster into a heterothedastic one; it does this by adding a scale output, either in its projection head or as another copy of the same network trained with the corresponding NLL. The main claim it does is that the outputted scale improves the point forecast itself, not just uncertainty. To show this, Halo is applied to TimeXer, GCGNet and CrossLinear on the five EPF electricity price markets, showing improvements in MSE/MAE in 28 of 30 comparisons, with average MSE reductions between 2.6% and 16.5%.

Critical strengths
  • The method is neat and simple, seems decently cheap and agnostic to the architecture. It can be added to any existing forecaster.
  • The results seem positive for 3 different architectures (transformer, GNN+VAE, CNN) and under both Gaussian and Laplacian losses.
  • The protocol is good: future exogenous inputs are excluded to prevent data leakage, the tuning is done only on validation, and the code with the hyperparameters has been released, which is always a plus.
Critical weaknesses

This paper should be rejected for the following reasons: (1) all results come from single runs, and (2) the gains are small enough to be seed noise; (3) the comparison to the baseline is not methodologically correct.

Reasons for rejection:

  • All result come from a single run with one seed, no variance is reported. For TimeXer and CrossLinear the average gains are around 2-4%, which are low enough to suspect that Halo is not actually the reason of improvement, since in these datasets, different runs for the same model can produce these results.

  • The tuning with respect to the baseline is unequal. Halo models get their own hyperparameter search whereas the baselines reuse the original authors' settings, even though everything is retrained under a new setup than what they were tuned for. GCGNet is the worst case. It was originally tuned in a case with access to future exogenous inputs, which this paper removes, yet only the Halo version is retuned. This makes the biggest paper empirical improvement of 16.5% MSE unsurprising. The parallel variant also doubles the parameter count, with the baseline not matching it.

Questions for authors
  • The appendix is quite limited, although it gives the training setup it leaves out reporting several basic things like: lookback, horizon, the hyperparameter search space and budget, the final per-model and per-market configurations, and data split sizes. I appreciate that the code and hyperparameters are in the anonymous repository, but it would be helpful to have these details in the appendix as well. Could you also say if the baselines got a comparable tuning budget in your setting?

  • All results are single runs. Could you report means and standard deviations over more seeds?

Reviewer_1vubOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a simple technique for improving point forecasting performance in time-series forecasting by allowing the model to additionally predict a scale parameter. The paper empirically demonstrates that this modification can improve forecasting accuracy when integrated with existing state-of-the-art neural forecasting architectures.

Critical strengths

None.

Critical weaknesses
  1. It is difficult to identify a clear and meaningful distinction between the proposed approach and existing well-established probabilistic forecasting models such as DeepAR [1]. The paper would benefit from a more explicit discussion of how the proposed method differs conceptually and methodologically from such prior work.

  2. The paper argues that parameterizing the model output in terms of location and scale parameters is not a necessary requirement for heteroscedastic loss formulations. However, the experiments and methodology appear to consider only the two-parameter location-scale setting. Other possible parameterizations, such as those based on Weibull or exponential distributions, are not investigated. As a result, the current empirical evidence does not appear sufficient to support the broader claim regarding alternative parameterizations.

  3. The β\beta-NLL objective used for model training is not newly introduced in this work. Therefore, it would be helpful for the paper to clarify more precisely where the methodological novelty lies beyond the adoption of this existing loss function.

  4. The paper claims that Halo improves forecasting accuracy, but the range of forecasting architectures and benchmark datasets considered in the experiments is relatively limited. For example, widely used forecasting models such as TimesNet, iTransformer, and PatchTST are not included. Similarly, standard benchmark datasets without explicit exogenous variables, such as the ETT and Traffic datasets, are not considered. Without evaluation across a broader set of architectures and datasets, I believe that the current claim regarding the general effectiveness of Halo is somewhat overstated.

[1] Salinas, David, et al. "DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks." International Journal of Forecasting, 2020.

Questions for authors
  1. How is the final point prediction y^\hat{y} computed from Equation (12)? Is y^\hat{y} simply given by a^\hat{a}? Since Halo allows the model to output two distributional parameters, it is currently unclear how these parameters are converted into a single point forecast at inference time. Please clarify the exact procedure used to obtain the reported point predictions.
Reviewer_LhLVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies heteroscedastic regression applied to time series forecasting problems. The paper claims that the proposed method, Halo, helps the point estimate to converge in contrast to previous works in heteroscedastic regression. Experiments are conducted on time-series data to reinforce this claim.

Critical strengths

The paper is written in an easy to understand manner and the material presented is correct.

Critical weaknesses
  1. The paper misses significant baselines and discussions [1, 2, 3, 4]. These are important since they precisely tackle the claim in the abstract, that "in contrast to reported negative results for heteroscedastic estimation outside time series"

  2. The method novelty is low. It is the same as Beta-NLL, which the authors cite. In addition the authors perform standardization of the time series, but this is not new novelty in my opinion.

[1] Immer et al., Effective Bayesian Heteroscedastic Regression with Deep Neural Networks, NeurIPS 2023 [2] Shukla et al., TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression, ICML 2024 [3] Sluijterman et al., Optimal training of Mean Variance Estimation neural networks, Neurocomputing 2024 [4] Wong-Toi et al., Understanding Pathologies of Deep Heteroskedastic Regression, UAI 2024.

Questions for authors

Please see above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.