Easy Yet Effective: Direct Probabilistic Forecasting with Recurrent Volatility Memory

Recent probabilistic forecasters have made sequence generation increasingly expressive, but this alone does not resolve how predictive uncertainty should evolve over a long horizon. We identify a design mismatch: future locations can be generated in parallel, whereas predictive scale benefits from memory of recent volatility. VolDy-VAE separates these computations. A patchwise variational encoder and direct latent projection produce all future latents in one pass; a shared decoder predicts location, while a single GRU carries a scale state from historical to future patches. The model requires neither iterative denoising nor recursive generation of predicted values. Heteroscedastic Gaussian likelihoods train historical reconstruction and future prediction, allowing learned scale to moderate the influence of variable observations on location learning. Across nine long-horizon benchmarks, VolDy-VAE improves CRPS over the listed probabilistic baselines in most settings, with ablations and efficiency results examining the role of recurrent volatility memory.