acceptodds
Under review as a conference paper at ICLR 2027

Easy Yet Effective: Direct Probabilistic Forecasting with Recurrent Volatility Memory

Abstract

Recent probabilistic forecasters have made sequence generation increasingly expressive, but this alone does not resolve how predictive uncertainty should evolve over a long horizon. We identify a design mismatch: future locations can be generated in parallel, whereas predictive scale benefits from memory of recent volatility. VolDy-VAE separates these computations. A patchwise variational encoder and direct latent projection produce all future latents in one pass; a shared decoder predicts location, while a single GRU carries a scale state from historical to future patches. The model requires neither iterative denoising nor recursive generation of predicted values. Heteroscedastic Gaussian likelihoods train historical reconstruction and future prediction, allowing learned scale to moderate the influence of variable observations on location learning. Across nine long-horizon benchmarks, VolDy-VAE improves CRPS over the listed probabilistic baselines in most settings, with ablations and efficiency results examining the role of recurrent volatility memory.

Primary area:
learning on time series and dynamical systems
Keywords:
Probabilistic Time Series Forecasting, Recurrent Volatility Memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tdPFOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VolDy-VAE combines direct latent forecasting with a recurrent scale decoder. Historical patches are encoded into variational latents and projected to the full future horizon; a shared decoder produces locations, while a GRU carries its historical state into future scale prediction. Reconstruction and forecasting use heteroscedastic Gaussian likelihoods. Evaluation includes nine long-horizon benchmarks, scale-head and likelihood ablations, calibration diagnostics, synthetic volatility examples, transfer experiments, and inference measurements.

Critical strengths
  1. Separating direct location generation from recurrent scale estimation is a useful, economical design. It preserves temporal structure in uncertainty without recursively generating observations. The manuscript also carefully distinguishes known-variance weighting results from the behavior of jointly learned scales, avoiding an unsupported theoretical guarantee.
  2. The empirical evaluation is substantial. VolDy-VAE achieves the lowest reported CRPS and NMAE on all nine datasets at the longest horizons, including comparisons with K2VAE, NsDiff, and PPM. The matched MLP/LSTM/GRU scale-head comparisons and calibration diagnostics support recurrent scale estimation beyond merely adding a heteroscedastic likelihood, while the efficiency results demonstrate practical value.
Critical weaknesses
  1. The promising evidence supports recurrent scale estimation more directly than historical-state transfer. Tables 5 and 15 do not retain future recurrence while removing the historical-state initialization, leaving the specific benefit of carrying history across the forecast boundary unisolated. Figure 5 also leaves forecast origins, held-out regions, and regime-transition generation unclear, limiting interpretation of its scale correlations as out-of-sample recovery. These are focused gaps in the mechanism's validation.
  2. Relevant accepted methods are absent from the comparisons: TimePrism (ICLR 2026), CW-Gen (ICLR 2026), DistPred (KDD 2025), and OCE-TS (AAAI 2026). TimePrism, DistPred, and OCE-TS offer simple direct probabilistic alternatives. OCE-TS is cited; the other three are not. All of these need to be cited and discussed; among them, the latest and most relevant (direct probability) ones are TimePrism and OCE-TS.
Questions for authors
  1. How much of the calibration and forecasting improvement remains when the GRU state is reset at the history-to-future boundary while retaining the same recurrent future scale decoder?
  2. For Figure 5, how are train/test sequences, forecast origins, and regime transitions defined? Are scale correlations computed exclusively on held-out future steps?
  3. As mentioned before, how does VolDy-VAE compare with other methods?
Reviewer_aWAFOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that long-horizon probabilistic forecasting suffers from a design mismatch in which "location can be generated directly in parallel, whereas scale should carry over historical volatility," and accordingly proposes VolDy-VAE, which uses a patched variational encoder plus an MLP to directly generate all future latents and means, uses a GRU that crosses the history/future boundary to output scales patch by patch, is trained with a heteroscedastic Gaussian NLL plus KL, and is accompanied by a theoretical analysis of inverse-variance weighting and an analogy to GARCH.

Critical strengths

The authors separate "what to generate" (location) from "how uncertainty evolves" (scale), introducing a GRU only on the scale path. This avoids both the iterative denoising of diffusion models and the step-by-step generation of autoregressive models, while preserving temporal coherence of the scale. The idea of "placing temporal machinery only where it is needed" is simple and elegant, and it can be added as a plug-in to other backbones (validated on PatchTST, TimeGAN, and K²VAE), making it highly practical.

Critical weaknesses
  1. The core concepts, the "location–scale mismatch" and "recurrent volatility memory," lack rigorous justification. The GRU acts more like an inductive bias or smoother than genuine volatility dynamics.
    • The future latent Zf=Fω(vec(Zh))Z^f=F_\omega(\mathrm{vec}(Z^h)) is a deterministic function of the historical latent, so any non-recurrent scale head taking ZhZ^h as input could in principle access the same historical information as the GRU. Thus "the scale needs a state that crosses from history into the future" is not an informational necessity but a choice of parameterization. The paper does not explain why this parameterization is in principle better than letting the scale depend directly on the whole history, e.g., via an MLP or attention. The so-called "mismatch" reads more like an assertion than a demonstrated structural problem.
    • Volatility persistence in GARCH fundamentally comes from feedback of realized residuals (σt2\sigma_t^2 depends on ϵt−12\epsilon_{t-1}^2). The GRU in VolDy-VAE receives no residuals or realized values during the forecast horizon; it merely rolls along a deterministic latent sequence. What it implements is "a temporally smoothed scale trajectory," not a conditional heteroscedastic recursion, so the analogy in C.1 is rather loose. The argument in C.2 that "temporal sharing reduces the variance of scale estimates" is also only a qualitative intuition and is not formalized.
    • Within a patch, the scale is produced by a linear readout, and the GRU updates only across patches, so the temporal resolution is limited by the patch length. This also does not fully match the narrative that "volatility evolves over time."
  2. The probabilistic model is not rigorously formulated, and the theoretical analysis is disconnected from the proposed mechanism.
    • The training objective (Eq. 8) is a concatenation of a historical reconstruction NLL, a future prediction NLL, and a KL term; it is not the ELBO of any generative model p(Y∣X)p(Y\mid X). The prior N(0,I)\mathcal{N}(0,I) does not depend on XX, yet at inference the method samples directly from the "posterior" qϕ(Zh∣X)q_\phi(Z^h\mid X) to form the predictive distribution. Consequently, the meaning of the predictive distribution (what uncertainty the randomness of ZhZ^h represents) and its relation to a Bayesian predictive distribution are unclear. The "ELBO derivation" in C.5 covers only the reconstruction term.
    • Given the latent, all steps and variables are conditionally independent, and temporal correlation arises only from sampling a shared latent. The paper emphasizes that "uncertainty evolves along the horizon," but does not discuss whether the generated sample paths are plausible in a joint sense.
    • The theoretical results (Theorem 1, Proposition 1, C.7) all analyze properties of "heteroscedastic NLL itself": constant mean, known variance, pointwise independent optimization. None of them concerns the recurrent scale state or the "parallel location + recurrent scale" structure, so they provide no support for the paper's core claims.
    • In terms of novelty, RNN-produced time-varying scales (DeepAR, DeepState), heteroscedastic VAEs (HeTVAE), and separate location–scale modeling (GAMLSS) already exist. The difference here lies mainly in the combination of "non-autoregressive location + recurrence only on the scale," which is a rather small increment.
Questions for authors

Clarifications (answers may affect my rating):

  1. Since ZfZ^f is a deterministic function of ZhZ^h, why is "a recurrent state crossing the history/future boundary" necessary for the scale at the information level? If it is only an inductive bias, what prior assumption does it correspond to, e.g., scales being smooth over time or monotonically reverting to some level?
  2. The GRU receives no residual feedback during the forecast horizon. Please clarify the fundamental mechanistic difference from GARCH-style conditional heteroscedasticity, and whether this design can express typical volatility dynamics such as "volatility clustering followed by reversion."
  3. Please specify the probabilistic model underlying VolDy-VAE's predictive distribution: Eq. 8 is a bound on which generative model? What justifies sampling from qϕ(Zh∣X)q_\phi(Z^h\mid X) at inference? Does the randomness of ZhZ^h capture epistemic or aleatoric uncertainty?
  4. Could you provide a theoretical result directly related to the recurrent scale structure, e.g., the error advantage of recurrent estimation over pointwise estimation under some volatility model with persistence?
  5. A brief experimental question: the MLP scale head used as a control only sees the latent of the current patch, whereas the GRU can read the history. Does this comparison confound "recurrent structure" with "information access"?

Suggestions:

  • Rephrase the GARCH analogy in C.1 more accurately, or genuinely introduce residual feedback or a stochastic-volatility prior.
  • Consider a more academic title.
Reviewer_dd81OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

VolDy-VAE combines a variational patch encoder, a direct projection to future latents, a shared location decoder, and a GRU scale decoder whose state carries over from the history into the forecast. Training uses Gaussian NLL for reconstruction and prediction plus a KL term. The paper reports the best CRPS and NMAE at the longest horizon on nine datasets. However, most baseline results are reused from K2K^2VAE while described as new runs.

Critical strengths
  1. Simple, well-motivated design. Direct location prediction with recurrent scale prediction avoids iterative denoising and autoregressive generation.
  2. Broad comparisons across nine datasets, four horizons, and diffusion, flow, VAE, and point-forecaster baselines (subject to W1).
  3. Useful scale-head ablations. On ETTh2 (H=192H=192), replacing the MLP scale head with a GRU improves CRPS from 0.179 to 0.160 and 95% coverage from 89.9% to 94.5%, while narrowing intervals from 1.83 to 1.50 (Tables 5 and 15).
  4. Appropriately qualified theory. The known-variance result is kept separate from learned scales, and the derivations are correct.
Critical weaknesses
  1. Baseline provenance is misreported. Section 4.1.2 and the captions of Tables 12 and 13 state that all baselines except PPM use six runs with standard deviations. All 692 mean/error-bar pairs for the ten baselines shared with K2K^2VAE (Wu et al., 2025, Tables 9 and 10) match that paper, including missing entries; its captions state five runs and standard errors. The K2K^2VAE and PatchTST "Base" rows in Tables 6 and 9 also match (48/48), contradicting the six-seed protocol of Section 4.5 and Appendix D.3. The four starred K2K^2VAE entries (Electricity, H=336,720H=336, 720: +0.03+0.03; Traffic, H=96,192H=96, 192: +0.10+0.10) are one-digit changes with unchanged error bars, which is implausible for a six-seed rerun. The originals may be typos, but all four corrections reverse the comparison with VolDy-VAE, and Table 1 uses the Electricity value unmarked, so the claim of best CRPS on all nine datasets depends on it. This is my main concern.
  2. The state-transfer mechanism is not isolated. The MLP ablation removes recurrence entirely, parameter counts are not reported, and each future latent already encodes the history through Eq. 2. The needed control is a GRU whose state is reset at the forecast boundary.
  3. The transfer experiments are underspecified. Table 8 does not say whether the recurrent scale head or historical state is used. The PatchTST base in Table 9 is the Gaussian-NLL model from Table 12, so "replace the point head" does not describe the comparison.
  4. Synthetic recovery is not shown out of sample. Appendix B omits the train/test split, forecast origins, and switching schedule. Its μt=sin⁡(t)\mu_t=\sin(t) has a period of ≈6.3\approx 6.3 steps, but the mean in Figure 5 has a period of ≈100\approx 100; the figure also lacks a forecast boundary and a head comparison. The reported correlations (0.985, 0.999) are invariant to scale error, so a magnitude-sensitive error is needed. Coverage alone cannot isolate the scale head, because intervals also reflect latent-driven variation in the mean.
  5. Only marginals are evaluated. Without CRPS-sum or an energy score, temporal and cross-variable dependence are untested.
  6. Two controls are questionable. On Electricity, the Vanilla VAE (MSE) NMAE (0.362 to 0.370) equals that of the variant without prediction loss and is about 4×4\times that of K2K^2VAE. Table 16 reports test-set values with no validation-based selection described, and Weather CRPS ranges from 0.067 to 0.086 across DD, more than the margin over K2K^2VAE.

Novelty: Heteroscedastic likelihoods and inverse-variance weighting are established, and VHVM already combines a VAE with recurrent volatility modeling. The contribution is the specific architecture and its empirical results.

Minor: Appendix A.2.3 attributes ODE solvers to HeTVAE, which uses temporal attention and feed-forward networks (Shukla and Marlin, 2022, Section 3).

Questions for authors
  1. Which baseline and transfer-base entries come from K2K^2VAE? How were the four K2K^2VAE entries reproduced, and why are their error bars unchanged? Please give per-seed results.
  2. Does state continuation outperform a GRU reset at the forecast boundary? What are the parameter counts of the scale heads?
  3. What exactly changes in each transfer experiment, and how is the historical scale state initialized?
  4. How were hyperparameters and learning rates selected?

I would reconsider my score if the provenance is corrected, the transfer experiments are specified, and the benefit of state transfer is isolated.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.