- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses the problem of forecasting prices in electricity markets. The authors suggest modeling these via Stochastic Differential Equations (SDEs) driven by an -stable Lévy process where the drift and diffusion coefficients are parameterized by a learned neural network and term this method LévySDE-Net. The motivation for this is that renewable energy generation induces abrupt shocks in electricity prices which cannot be appropriately modeled via a Gaussian distribution but rather require a heavy tailed distribution. A similar approach has already been chosen in a previous paper to model stock indices, however, it has been only validated on single-run experiments. The main contribution consists of (1) the authors addressing this gap in their work and evaluating the method on three different seeds as well as (2) extending the idea of using Lyapunov time to test the divergence of forcing mechanisms beyond simply comparing the model’s accuracy before and after its predicted horizon. Additionally, several experiments are conducted such as testing whether the alpha parameter can be recovered in a controlled, synthetic experiment, one-step and multi-horizon forecasting as well as predictive uncertainty. The proposed method does not outperform the existing Gaussian baseline in terms of point prediction performance and has a larger predictive spread that it.
Critical strengths
The authors identify a gap in previous work, which however only uses a single seed in their experiments and which is thus susceptible to statistical miss-evaluations.
Critical weaknesses
- The contribution is marginal and the idea is not novel
- The idea of using -stable Lévy processes is already present in the literature as the authors write themselves in their paper. Simply running more seeds is not a significant extension of this work.
- The contribution of treating the Lyapunov time as marker for faithful forecasting is thin. In line 444 the authors choose and remarks that this is "not yet justified on independent grounds". Further, in line 466 they leave testing that this relationship holds to future work which would have strengthened this contribution.
- The results of the method are non-existing. As already mentioned the proposed method is not able to outperform the Gaussian baseline. The claimed improvement of having a wider ensemble are by itself marginal. The authors further fail to investigate this further and do not comment on why this is beneficial at all. The obvious explanation that this is only due to the heavy tails of the Lévy process is not addressed at all.
- The presentation of the paper is not good and lacks basic principles of scientific writing
- The citations are formatted not correctly: Each reference in the main text is indicated by a number and a link. The link does not lead to anywhere in the document. Further, the bibliography does not list the numbers of the references such that finding the corresponding work to a reference becomes a tedious counting.
- The tables are overflowing and lack clarity, for example due to missing separation bars. Some metrics are evaluated up to 6 decimals so simply rounding them appropriately would have addressed this overflowing issue in these cases.
- The subsection on "Limitations and positioning" is repeated in 146-158 and 159-172
Questions for authors
- What is the significance of having a larger spread for the ensemble? Does it correlate with prediction error and is thus a useful proxy for predictive uncertainty? Or is it just an artifact of using a heavy tails process?
- Can you elaborate on the claim of using Lyapunov-diagnostics for trajectory divergence?
3 comments
Follow-up (Reviewer 32py)
Dear Reviewer 32py, As the discussion period closes tomorrow, we wanted to check whether our response above and the revised PDF address your concerns. We would be glad to clarify anything before the deadline. Thank you again for your time.
I want to thank the reviewers for their detailed response and substantial revisions. I acknowledge that the presentation issues are partly fixed, the Lyapunov analysis has been removed, and my Q1 has received an answer.
Nevertheless, I stick with my score. The initial submission violated the ICLR formatting in several places (inconsistent citation style, overflowing margins), contained a duplicated subsection, and reported unverified results. The revision, on the other hand, changed the title, the central explanation, the theory, the baselines, and practically every reported number, after correcting errors in the original pipeline. Overall, an unfinished paper was submitted and a new one written during rebuttal, which cannot be reviewed with the necessary care at this time and is also not the purpose of a review process. Further, in my view, the contribution is not strong enough for this venue. It includes mainly a negative result for one small SDE-Net model that is uncalibrated and outperformed by the AR(1) model and its adaptation (+ GARCH(1,1)-t), as well as an interpretation of this in terms of the Forecaster's lemma, whose whole concept and theory are now central to the paper, but were added in retrospect.
I encourage the authors to rework the paper and resubmit, and to take the submission process more seriously.
Response to Reviewer 32py
Thank you for the review. We have substantially revised the paper in response; numbers refer to the revised PDF.
Novelty. We agree that running more seeds alone is not a sufficient contribution, and that α-stable neural SDEs are not new; we no longer claim to introduce the model. The revised paper contributes:
Q1. Is the larger spread useful, or just an artifact of heavy tails? We tested this directly, and your suspicion is largely correct.
The paper now states this explicitly (Sections 4.5–4.6).
Q2 and the Lyapunov contribution. We agree that it was thin. We removed the rollout study, the unjustified threshold and the claim that Lyapunov time marks a forecasting horizon. The remaining phase-space diagnostics are a short exploratory appendix for DK1 only, with no forecasting claim (Appendix J). Cao's criterion indicates predominantly stochastic dynamics, so the estimated exponent is reported only as a descriptive divergence scale.
Results. The main results are now: (i) no point-forecast benefit, with a theoretical explanation; (ii) a CRPS gain from compensating under-dispersion, not from better tails; (iii) GARCH-t outperforms both SDEs; and (iv) outcome-selected evaluation makes heavy-tailed forcing appear superior where a proper score shows it is not. We think (iv) is useful well beyond this model, since evaluating tail forecasts on observed extreme events is common practice.
Presentation.
Note that neural models now use five seeds throughout.