acceptodds
Under review as a conference paper at ICLR 2027

DecoVAE: Decomposing Predictable Dynamics and Uncertainty for Efficient Probabilistic Time Series Forecasting

Abstract

Probabilistic forecasting of multivariate time series remains challenging because accurate prediction requires modeling both structured temporal dynamics and time-varying residual uncertainty. Existing methods may treat these sources uniformly or incur substantial computational overhead, making it difficult to jointly achieve accurate and efficient forecasting. We propose DecoVAE, a lightweight Trend–Seasonality–Residual variational framework that assigns distinct statistical roles to structured dynamics and residual uncertainty. Trend and seasonality determine the forecast mean, whereas the residual stream models its time-varying variance rather than predicting residual realizations directly. To reflect their different structures, DecoVAE represents trend through a smoothness-regularized latent trajectory, seasonality through a frequency-domain VAE, and residual uncertainty through a dedicated volatility stream. Across seven real-world datasets, DecoVAE achieves the lowest mean CRPS on every dataset, with average relative improvements over the strongest competing method of 10.05% for short-term and 12.28% for long-term forecasting, while remaining competitive in point accuracy (NMAE). Compared with the closest competing method, DecoVAE trains approximately faster and is about smaller on high-dimensional datasets. These results suggest that assigning distinct statistical roles to structured temporal dynamics and residual uncertainty enables accurate and efficient probabilistic time series forecasting.

Primary area:
learning on time series and dynamical systems
Keywords:
Probabilistic Time Series Forecasting, Multivariate Time Series, Variational Autoencoders, Time Series Decomposition

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_vGndOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

DecoVAE decomposes the input sequence into trend, seasonal, and residual components within a clear temporal structure. For determining uncertainty, DecoVAE employs the residual components to measure the variance of the prediction. Specifically, DecoVAE calculates the residual by removing the trend and seasonal components from the original series. Through three parallel VAEs for handling three corresponding components, DecoVAE gets the prediction mean from trend and seasonal predictors, combined with the prediction variance from the volatility predictor via the Gaussian distribution. Experiments on seven datasets report that DecoVAE achieves the lowest mean CRPS on all datasets and trains faster / uses less memory than competing probabilistic models. The ablation study supports the efficient of DecoVAE‘s design.

Critical strengths
  1. DecoVAE differentiates between predictable structure in the mean and in uncertainty, situating this distinction within existing econometric and neural volatility research.
  2. Trend/season and volatility are implemented with distinct inductive components within three parallel VAEs, which is conceptually coherent.
  3. Experiments show that DecoVAE outperforms baselines in 7 real-world datasets, within speed/memory/model-size comparisons, which do matter in probabilistic forecasting in practice.
  4. The paper is well organized, with clear statements and logical explanations. The motivation, methodology, and experimental results are easy to follow, making the proposed framework accessible to readers.
Critical weaknesses
  1. Novelty is arguably incremental and not established at the level of the presentation. The proposed decomposition is a general method in time series forecasting. Decomposition of the original series into trend/seasonal components is well used in

Wu, H., Xu, J., Wang, J., & Long, M. (2021). Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34, 22419-22430.

Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International conference on machine learning.

Treating the (de-trended, de-seasonalized) residual as the basis for time-varying prediction variance is usually used in

Kendall, A., & Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. Advances in neural information processing systems, 30.

Shukla, S. N., & Marlin, B. (2021). Heteroscedastic temporal variational autoencoder for irregularly sampled time series. In International Conference on Learning Representations.

  1. As stated in line 467, DecoVAE does not capture the dependency across features. Thus, can I deem DecoVAE a channel-independent (CI) method? This may stem from the FFT on the detrended series, as stated in line 165, and should be clearly stated in the method description. Although the authors claim this design is for a lightweight model, but the channel dependency is critical for time series forecasting. The FFT can be employed in Zs in seasonal predictor like following work:

Yi, K., Fei, J., Zhang, Q., He, H., Hao, S., Lian, D., & Fan, W. (2024). Filternet: Harnessing frequency filters for time series forecasting. Advances in neural information processing systems, 37, 55115-55140.

  1. For experiments, the results are lacking in Weather, which is the standard dataset in probabilistic and deterministic time series forecasting task.
  2. The ablation study clarifies the effectiveness of DecoVAE’s design, but I am more interested in the selection of poptp_{opt} in line 168. Since this relates to the generation of the residual components, and the cutoff of frequency modes is critical to balance useful frequency modes and noise.
Questions for authors

See weakness

3 comments

Q1. Novelty is arguably incremental and not established at the level of the presentation. The proposed decomposition is a general method in time series forecasting. Decomposition of the original series into trend/seasonal components is well used in Wu, H., Xu, J., Wang, J., & Long, M. (2021). Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting. Advances in neural information processing systems, 34, 22419-22430.

Zhou, T., Ma, Z., Wen, Q., Wang, X., Sun, L., & Jin, R. (2022). Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting. International conference on machine learning.

Treating the (de-trended, de-seasonalized) residual as the basis for time-varying prediction variance is usually used in

Kendall, A., & Gal, Y. (2017). What uncertainties do we need in bayesian deep learning for computer vision?. Advances in neural information processing systems, 30.

Shukla, S. N., & Marlin, B. (2021). Heteroscedastic temporal variational autoencoder for irregularly sampled time series. In International Conference on Learning Representations.

A1: Thank you for raising this concern. We agree that neither trend--seasonality decomposition nor heteroscedastic uncertainty modeling is novel in isolation. The contribution of DecoVAE is their role-specialized probabilistic organization: trend and seasonality parameterize the forecast mean, while a dedicated residual-volatility representation models time-varying forecast variance.

Autoformer [1] and FEDformer [2] use decomposition primarily to represent and predict temporal dynamics, but do not assign the residual a dedicated probabilistic role. Kendall & Gal [3] provide foundational work on input-dependent heteroscedastic uncertainty, although in a computer-vision setting rather than time-series forecasting. HeTVAE [4] is closer in the temporal domain, combining variational modeling with heteroscedastic outputs for irregular time series.

MethodTrend/SeasonHeteroscedasticResidual-volatilitySeparate mean/variance roles
Autoformer [1]✓–––
FEDformer [2]✓–––
Kendall & Gal [3]–✓––
HeTVAE [4]–✓––
DecoVAE✓✓✓✓

As summarized above, DecoVAE explicitly combines Trend--Seasonality--Residual decomposition with separate statistical roles: structured components model the forecast mean, while residual history is represented specifically to model future volatility.

Our new controlled ablation (https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/ResidualModeling.pdf) further clarifies this contribution. On ETTh2, introducing the structured deterministic residual representation improves CRPS from 0.444 to 0.308, while stochastic sampling and KL further improve it to 0.293. Thus, the main gain comes from structured residual-volatility modeling, with variational modeling providing an additional, dataset-dependent generalization benefit.

We will revise the manuscript to make this distinction explicit.

References

[1] H. Wu, J. Xu, J. Wang, and M. Long. Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting. NeurIPS, 2021.

[2] T. Zhou, Z. Ma, Q. Wen, X. Wang, L. Sun, and R. Jin. FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting. ICML, 2022.

[3] A. Kendall and Y. Gal. What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? NeurIPS, 2017.

[4] S. N. Shukla and B. M. Marlin. Heteroscedastic Temporal Variational Autoencoder for Irregularly Sampled Time Series. ICLR, 2022.

Q2. As stated in line 467, DecoVAE does not capture the dependency across features. Thus, can I deem DecoVAE a channel-independent (CI) method? This may stem from the FFT on the detrended series, as stated in line 165, and should be clearly stated in the method description. Although the authors claim this design is for a lightweight model, but the channel dependency is critical for time series forecasting. The FFT can be employed in Zs in seasonal predictor like following work: Yi, K., Fei, J., Zhang, Q., He, H., Hao, S., Lian, D., & Fan, W. (2024). Filternet: Harnessing frequency filters for time series forecasting. Advances in neural information processing systems, 37, 55115-55140.

A2. Thank you for raising this point. We agree that our wording in line 467 is ambiguous. The phrase “does not explicitly capture structured cross-channel dependencies” can be interpreted as implying that DecoVAE is channel-independent, which is not what we intended.

DecoVAE is not channel-independent. All input channels are jointly processed by the component encoders, which map the full multivariate feature vector into shared latent representations. Thus, cross-channel information can be captured at the representation level. In the seasonal stream, the FFT is applied only along the temporal dimension; the resulting FF-dimensional frequency representation is then jointly projected into the latent space, so the use of FFT does not imply channel independence.

What we intended to state in the limitation is that DecoVAE does not explicitly model structured cross-channel or cross-horizon dependence in the conditional predictive distribution. The output likelihood uses diagonal covariance, p(Y∣Z)=N(vec⁡(Y);vec⁡(μ),diag⁡(vec⁡(σ2))).p(Y\mid Z)=\mathcal{N}\bigl(\operatorname{vec}(Y);\operatorname{vec}(\mu),\operatorname{diag}(\operatorname{vec}(\sigma^2))\bigr). So cross-channel dependencies may be represented through the shared latent variables, but no explicit non-diagonal predictive covariance is parameterized.

FilterNet [1] could provide richer frequency-domain dependency modeling at the representation level, but it would not by itself remove this probabilistic limitation. We will revise the manuscript to clarify that DecoVAE captures cross-channel information through multivariate latent representations, while its limitation is the lack of explicit cross-channel or cross-horizon covariance in the predictive distribution.

References

[1] K. Yi, J. Fei, Q. Zhang, H. He, S. Hao, D. Lian, and W. Fan. FilterNet: Harnessing Frequency Filters for Time Series Forecasting. NeurIPS, 2024.

Q3. For experiments, the results are lacking in Weather, which is the standard dataset in probabilistic and deterministic time series forecasting task.

A3. Thank you for pointing this out. We have added experiments on the Weather benchmark for both short-term (L=24L=24) and long-term (L=720L=720) forecasting. The results are summarized below.

For L=24L=24, DecoVAE achieves the best performance in both metrics, with CRPS 0.286 ± 0.003 and NMAE 0.316 ± 0.001. For L=720L=720, DecoVAE achieves the best NMAE (0.459 ± 0.005) and the second-best CRPS (0.431 ± 0.002), behind TSDiff. Overall, DecoVAE remains highly competitive on this standard forecasting benchmark across both short and long horizons.

DatasetMetricDecoVAE (ours)CLaSTK2K^2VAELaSTDeepARTFMTSDiffDeNOTS
Weather (L=24L=24)CRPS0.286 ± 0.0030.287 ± 0.0040.683 ± 0.0110.538 ± 0.0570.360 ± 0.0251.008 ± 0.3220.327 ± 0.0600.434 ± 0.022
NMAE0.316 ± 0.0010.319 ± 0.0040.491 ± 0.0140.676 ± 0.0900.425 ± 0.0171.206 ± 0.4030.412 ± 0.0930.562 ± 0.030
Weather (L=720L=720)CRPS0.431 ± 0.0020.447 ± 0.0010.661 ± 0.0070.672 ± 0.0280.465 ± 0.0641.344 ± 0.2460.375 ± 0.0300.803 ± 0.071
NMAE0.459 ± 0.0050.482 ± 0.0010.606 ± 0.0080.835 ± 0.0430.555 ± 0.0671.542 ± 0.4220.465 ± 0.0330.865 ± 0.084

Bold denotes the best result and italics the second-best result; lower is better.

Q4. The ablation study clarifies the effectiveness of DecoVAE’s design, but I am more interested in the selection of poptp_{opt} in line 168. Since this relates to the generation of the residual components, and the cutoff of frequency modes is critical to balance useful frequency modes and noise.

A4. Thank you for raising this point. We treated poptp_{\mathrm{opt}} as a dataset-specific hyperparameter controlling the fraction of spectral energy retained when selecting the seasonal frequency modes. It was selected jointly with the other hyperparameters using Optuna over popt∈{0.5,0.6,0.7,0.8,0.9}p_{\mathrm{opt}} \in \left\lbrace 0.5, 0.6, 0.7, 0.8, 0.9 \right\rbrace.

Since poptp_{\mathrm{opt}} directly controls the separation between the seasonal and residual components, we additionally evaluated the sensitivity of DecoVAE to this parameter. We used the same challenging synthetic setting introduced in our robustness analysis of the nonzero residual conditional mean, as it contains substantial time-varying residual dynamics and therefore provides a suitable stress test for the frequency cutoff. The corresponding quantitative results and decomposition visualizations are provided in https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/vGnd_W4.pdf.

The results show that performance is worse at the boundary values, popt=0.5p_{\mathrm{opt}}=0.5 and popt=0.9p_{\mathrm{opt}}=0.9, while intermediate values provide the best forecasting quality. This behavior is consistent with the intended role of the cutoff: when poptp_{\mathrm{opt}} is too low, too few frequency modes are retained and part of the useful periodic structure remains in the residual component. Conversely, when poptp_{\mathrm{opt}} is too high, additional noisy modes are absorbed into the seasonal representation instead of remaining in the residual.

Thus, intermediate values provide a better balance between retaining informative seasonal modes and separating residual uncertainty.

Reviewer_mwLmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DecoVAE, a lightweight variational framework for probabilistic multivariate time series forecasting that explicitly separates predictable temporal dynamics from residual uncertainty. The key idea is to assign distinct statistical roles to decomposed time series components: trend and seasonality determine the forecast mean, while the residual component models time-varying predictive variance rather than directly predicting residual values.

Specifically, DecoVAE decomposes historical observations into trend, seasonal, and residual components. The trend branch employs a time-domain VAE with latent smoothness regularization, while the seasonal branch adopts a complex-valued frequency-domain VAE to capture periodic dynamics. The residual-volatility branch uses a dedicated VAE trained with a zero-mean Gaussian likelihood to learn historical volatility representations and predict future conditional variance. These components are combined to parameterize a heteroscedastic Gaussian predictive distribution, and the entire framework is optimized through component-specific variational objectives and a forecasting negative log-likelihood.

Experiments on seven real-world datasets demonstrate strong probabilistic forecasting performance, with the lowest reported mean CRPS across all datasets and competitive point-forecasting accuracy. Additional ablation studies investigate component-specific modeling, alternative residual uncertainty representations, and regularization strategies. The paper also reports favorable computational efficiency in terms of training time, memory consumption, and model size compared with competing variational forecasting methods.

Critical strengths

S1. A clear and meaningful role-specialized decomposition framework. The paper presents a well-motivated probabilistic decomposition framework that assigns distinct statistical roles to trend, seasonality, and residual components. In particular, modeling the residual stream through time-varying volatility rather than direct residual prediction provides an interesting perspective on separating predictable temporal dynamics from uncertainty.

S2. Comprehensive and informative ablation studies.

S3. Strong empirical results with a favorable accuracy–efficiency trade-off.

Critical weaknesses

W1. The specific contribution of variational latent modeling remains insufficiently isolated.

Although Table 3 provides meaningful comparisons against simpler residual uncertainty models, the proposed Residual-VAE combines several design choices, including stochastic latent representations, KL regularization, historical volatility reconstruction, and temporal smoothness regularization. Consequently, the observed improvements over the deterministic volatility MLP cannot be attributed specifically to variational latent modeling. While the additional regularization ablations are informative, the unique contribution of the variational representation itself remains insufficiently established.

W2. The proposed role-specialized decomposition assigns forecast mean prediction exclusively to the trend and seasonal components, while the residual-volatility stream assumes zero-mean Gaussian residuals and focuses solely on variance modeling. However, the decomposition process does not necessarily guarantee that the remaining residuals have zero conditional mean. When systematic or predictable residual patterns remain, the proposed volatility model may interpret these unmodeled location shifts as increased uncertainty, potentially affecting forecast calibration and accuracy. Although the paper evaluates synthetic data with varying volatility regimes, it does not explicitly investigate the robustness of this assumption when residuals exhibit nonzero conditional means.

Questions for authors

Q1. What is the specific contribution of variational latent modeling compared with deterministic volatility representations?

Table 3 provides useful comparisons between the proposed Residual-VAE and simpler volatility models, while Appendix J.2 further evaluates volatility regularization. However, the Residual-VAE differs from the Vol. MLP baseline in several aspects beyond stochastic latent modeling. Could the authors further clarify which improvements are attributable specifically to the variational representation? A controlled comparison using a deterministic residual autoencoder with the same architecture, historical volatility reconstruction objective, and smoothness regularization, but without latent sampling and KL regularization, would help isolate this contribution.

Q2. How robust is DecoVAE when the decomposition residual has a nonzero conditional mean?

The proposed framework assumes that trend and seasonality account for the predictable mean structure, leaving the residual stream to model only zero-mean uncertainty. However, this assumption may not always hold in practice. Could the authors evaluate the robustness of DecoVAE under controlled violations of the zero-mean residual assumption, for example, using synthetic data with varying degrees of systematic residual bias? Such an analysis would help clarify the applicability and limitations of the proposed role-specialized decomposition.

2 comments

Q1. What is the specific contribution of variational latent modeling compared with deterministic volatility representations?
Table 3 provides useful comparisons between the proposed Residual-VAE and simpler volatility models, while Appendix J.2 further evaluates volatility regularization. However, the Residual-VAE differs from the Vol. MLP baseline in several aspects beyond stochastic latent modeling. Could the authors further clarify which improvements are attributable specifically to the variational representation? A controlled comparison using a deterministic residual autoencoder with the same architecture, historical volatility reconstruction objective, and smoothness regularization, but without latent sampling and KL regularization, would help isolate this contribution.

A1: Thank you for this suggestion. We added the requested controlled deterministic residual autoencoder, which uses the same residual architecture, historical-volatility reconstruction, smoothness regularization, Trend+Season backbone, and shared training settings as DecoVAE, but removes latent sampling and KL regularization. We also include an intermediate stochastic variant without KL. Full results are provided in https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/ResidualModeling.pdf.

The results show that the improvement over Vol.MLP is only partly attributable to variational modeling. On ETTh2, the deterministic residual AE already improves CRPS from 0.444 to 0.308. Stochastic sampling further improves it to 0.301, and adding KL yields 0.293. Relative to the deterministic AE, the full Residual-VAE improves CRPS by approximately 4.9% and NMAE by 2.6%. Traffic shows the same trend with a smaller gain, while Electricity and ERCOT show little additional benefit from variationalization.

Thus, the main gain comes from structured historical-volatility modeling, with stochastic sampling and KL providing an additional, dataset-dependent improvement in generalization rather than a universal calibration benefit.

Q2. How robust is DecoVAE when the decomposition residual has a nonzero conditional mean?
The proposed framework assumes that trend and seasonality account for the predictable mean structure, leaving the residual stream to model only zero-mean uncertainty. However, this assumption may not always hold in practice. Could the authors evaluate the robustness of DecoVAE under controlled violations of the zero-mean residual assumption, for example, using synthetic data with varying degrees of systematic residual bias? Such an analysis would help clarify the applicability and limitations of the proposed role-specialized decomposition.

A2: Thank you for raising this point. We performed a controlled synthetic experiment in which the residual contains a systematic time-varying conditional mean,

ηt=γmt+εt,\eta_t=\gamma m_t+\varepsilon_t,

where mtm_t follows an AR(1) process and γ∈{0,0.25,0.5,0.75,1,1.5,2}\gamma \in \left\lbrace 0, 0.25, 0.5, 0.75, 1, 1.5, 2 \right\rbrace controls the strength of the violation. Full experimental details and visualizations are provided in the supplementary PDF: https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/mwLm_Q2.pdf.

The results reveal an important limitation of the role-specialized decomposition. When the systematic dynamics are relatively simple, DecoVAE successfully assigns predictable structure to the trend/seasonal components and leaves approximately zero-mean residual uncertainty. As γ\gamma increases, however, the additional dynamics become too complex to be represented as conventional trend or seasonality and therefore remain in the residual as a nonzero conditional mean. Since the residual branch is constrained to use zero mean, this remaining structure is absorbed into predictive variance, leading to wider prediction intervals and progressively worse forecasting performance. CRPS increases from 0.220 at γ=0\gamma=0 to 0.925 at γ=2\gamma=2, while NMAE increases from 0.311 to 1.303.

Thus, DecoVAE remains robust when the decomposition adequately separates predictable dynamics from residual uncertainty. The observed degradation under stronger violations also suggests a natural extension of the framework: allowing the residual branch to model both a conditional mean and a conditional variance, $ \eta_t \sim \mathcal{N}\left(\mu_{\varepsilon,t},\sigma_{\varepsilon,t}^{2}\right), $ rather than fixing με,t=0\mu_{\varepsilon,t}=0. Such an extension would preserve the role-specialized decomposition while allowing residual predictable structure that is not fully captured by trend and seasonality to be modeled explicitly instead of being absorbed into predictive variance.

The additional experiments and analyses have provided valuable clarification regarding the contributions and limitations of DecoVAE.

The controlled residual-modeling ablations are particularly informative. By introducing the deterministic autoencoder and the stochastic variant without KL regularization, the authors have successfully isolated the contributions of structured volatility modeling and variational latent representations. The results suggest that much of the improvement comes from structured residual modeling, while stochastic sampling and KL regularization provide additional, dataset-dependent benefits.

The synthetic experiments investigating non-zero-mean residuals also directly address my other concern. Although the results confirm that the zero-mean assumption can become restrictive when systematic dynamics remain in the residual component, they help clarify the conditions under which the proposed decomposition is effective. I also appreciate the authors' acknowledgment of this limitation.

Overall, the revisions have addressed my main questions and strengthened the technical and experimental justification of the framework. I remain positive about the paper and will maintain my rating.

Reviewer_8WDnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DecoVAE, which decomposes an input time series into trend, seasonal, and residual components and assigns them different statistical roles: the trend and seasonal branches generate the predictive mean conditioned on latent variables, while the residual branch predicts a time-varying conditional variance. The paper reports strong forecasting results in both short-term and long-term settings on seven real-world datasets and demonstrates advantages in model size and memory usage.

Critical strengths
  1. The model design is clear, and the modeling choices are reasonable. Separating trend, seasonality, and volatility, and assigning the prediction of the mean and conditional variance to different components, is a reasonable design. The time-domain trend representation, frequency-domain seasonal representation, and dedicated residual-variance branch are well matched to the characteristics of their respective components.

  2. The reported forecasting performance is strong, and the experimental coverage is broad. The authors evaluate short-term and long-term forecasting on seven real-world datasets, report CRPS and NMAE, and further examine ECE and calibration performance in the short-term component ablations on four datasets. The paper reports the lowest mean CRPS across all main experimental settings and includes several structural and regularization ablations, which provide some support for the effectiveness of the method.

  3. The architecture is compact and has practical value. The paper reports smaller model sizes and lower test-time memory usage in high-dimensional settings such as Electricity and Traffic, which is meaningful for deployment in resource-constrained scenarios.

Critical weaknesses
  1. The training cost per epoch may differ across methods, so the efficiency comparison needs further verification. In the submitted default code, the number of training batches per epoch is not fully consistent across methods. This affects the fairness of the efficiency comparison, and the authors should check the actual configurations used for that comparison.

  2. The model instantiates a dropout module inside forward(). Because of this, the module remains enabled during inference by default, introducing additional uncertainty and affecting the evaluation of model performance. The authors should either disable this dropout or redefine where it is instantiated and then re-evaluate the model, to ensure that the reported conclusions are reliable.

  3. The description of the volatility regularization in the main text is inconsistent with the code implementation. In the code, the second-order smoothing term is computed as second-order differences of the predicted future variance, whereas the main text states that it is applied to the reconstructed historical variance. The actual implementation needs to be clarified. In addition, there is an additional regularization term, shrink loss, whose effect on the results is not disclosed in the paper.

  4. The degradation of Vol. MLP on ETTh2 lacks sufficient explanation. In Table 3, the full model's CRPS advantage over Vol. MLP is mainly concentrated on ETTh2, and the CRPS of Vol. MLP on this dataset is even worse than that of Trend+Season. This degradation requires further analysis. The detailed configuration of Vol. MLP should be provided to distinguish whether the gain comes from the variational structure of the residual VAE or from different regularization and auxiliary training settings.

  5. The hyperparameter search budget is unclear. The paper searches five continuous loss weights and two discrete decomposition hyperparameters, which may affect the computational cost and practical applicability of the method. The authors should report the time spent on hyperparameter tuning and clarify whether the hyperparameters in the ablation experiments were independently retuned.

Questions for authors
  1. Did the efficiency experiments use the same number of training batches per epoch? Please provide the actual running configurations and add a running-time comparison under matched training workloads.

  2. Is retaining this dropout during inference intentional? Please report predictive performance and calibration results with this dropout disabled, while keeping the trained weights and other sampling mechanisms unchanged.

  3. Does the second-order volatility smoothing actually act on the reconstructed historical variance or on the predicted future variance? Which experiments used shrink loss? Please provide its definition, configuration, and corresponding ablation results.

  4. What is the main cause of the clear degradation of Vol. MLP on ETTh2? Which regularization terms and auxiliary objectives were used for this baseline, and how did its training settings differ from those of the full model?

  5. How many hyperparameter searches were actually performed for each method and ablation variant, what was the configuration-selection criterion, and what was the total search time? Were the ablation variants independently tuned?

4 comments

Q1: Did the efficiency experiments use the same number of training batches per epoch? Please provide the actual running configurations and add a running-time comparison under matched training workloads.

A1 We thank the reviewer for raising this important concern. We clarify that the efficiency experiments were conducted as a separate, dedicated study, independent of the main experiments, under an identical evaluation protocol across all models. Specifically, we used a limit of 100 training batches per epoch. Running time and memory consumption were measured and averaged over 5 epochs. Thus, all models were evaluated under matched computational workloads, ensuring a fair efficiency comparison. The discrepancies in the default configurations of the submitted code are artifacts of different preliminary experiments and development iterations and do not reflect the configurations used in the dedicated efficiency study. We will clarify these settings in the revised manuscript to improve transparency and reproducibility.

Q2: Is retaining this dropout during inference intentional? Please report predictive performance and calibration results with this dropout disabled, while keeping the trained weights and other sampling mechanisms unchanged.

A2: We acknowledge that retaining dropout during inference was an implementation error. We corrected it by properly registering the dropout layer and reusing it in the forward pass. To assess its impact, we repeated inference using the same trained weights while keeping all other sampling mechanisms unchanged. Most metrics remain exactly unchanged; the remaining differences in means or standard deviations are negligible. The detailed results are available at the following link: https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/8WDn_dropout.pdf

Thus, the corrected implementation has no meaningful impact on predictive performance or calibration, and the main conclusions remain unchanged.

Q3: Does the second-order volatility smoothing actually act on the reconstructed historical variance or on the predicted future variance? Which experiments used shrink loss? Please provide its definition, configuration, and corresponding ablation results.

A3: Thank you for highlighting these inconsistencies.

The implementation correctly regularizes the predicted future variance, whereas the manuscript mistakenly refers to historical variance. We will correct the text and equations accordingly.

The shrink loss is an inactive code artifact, unused in all experiments and ablations, and will be removed or marked as inactive.

Neither issue affects the reported results.

Q4: What is the main cause of the clear degradation of Vol. MLP on ETTh2? Which regularization terms and auxiliary objectives were used for this baseline, and how did its training settings differ from those of the full model?

A4: Thank you for raising this point. We provide a detailed analysis of the ETTh2 degradation, the Vol.~MLP configuration, and the controlled residual-modeling ablation in our supplementary rebuttal document: https://anonymous.4open.science/r/DecoVAE-3387/rebuttal/ResidualModeling.pdf

In brief, our analysis indicates that Vol.MLP overfits the pronounced anomalous regime present in the ETTh2 training data. The controlled ablation shows that introducing a structured historical-volatility representation removes most of this degradation, while stochastic latent sampling and KL regularization provide additional regularization and further improve generalization. Importantly, all variants use the same Trend+Season backbone and shared training settings, so the observed differences can be attributed to the residual-modeling formulation rather than to different optimization settings.

Q5: How many hyperparameter searches were actually performed for each method and ablation variant, what was the configuration-selection criterion, and what was the total search time? Were the ablation variants independently tuned?

A5: Thank you for raising this point.

We performed 100 Optuna trials per dataset, using validation CRPS for model selection; the test set was not used. The search space is already reported in the manuscript, and we will add the number of trials and selection criterion in the revision.

We did not record the total wall-clock search time because hyperparameter optimization was treated as a one-time model-selection stage, separate from the reported computational-efficiency evaluation. The reported efficiency results measure the cost of the selected model after configuration selection.

The ablation variants were not independently retuned. We kept all shared hyperparameters fixed to isolate the effect of adding or removing each component; component-specific hyperparameters were disabled when the corresponding component was removed.

We will clarify these details in the revised manuscript.

Thank you for the detailed responses and additional experiments. I accept the explanations about the efficiency evaluation settings, the dropout correction, and shrink loss not being used in the experiments. These responses have addressed most of my concerns. My remaining question is about the comparison with Vol. MLP.

The new ablations help clarify the contribution of each design choice. I also agree that structured residual modeling may help the model generalize better. However, Vol. MLP performs close to the full model on the validation set but much worse on the test set. The shared hyperparameter settings may be less suitable for Vol. MLP. Could you tune Vol. MLP independently with the same search budget and validation-based selection criterion as the full model, and check whether this improves its test performance? This would help clarify how much of the performance gap comes from the proposed structure.

Thank you for this important suggestion. We independently tuned Vol. MLP using the same optimization budget and model-selection protocol as the full DecoVAE model. Specifically, we performed 100 Optuna trials and selected the final configuration solely according to validation CRPS; the test set was not used during model selection.

On ETTh2, the independently tuned Vol. MLP improves test CRPS from 0.444 ± 0.031 to 0.417 ± 0.036 and ECE from 0.076 ± 0.012 to 0.064 ± 0.011, while NMAE remains essentially unchanged (0.423 ± 0.009 vs. 0.425 ± 0.008).

ModelCRPSNMAEECE
DecoVAE0.293 ± 0.0010.378 ± 0.0020.026 ± 0.003
Vol. MLP0.444 ± 0.0310.423 ± 0.0090.076 ± 0.012
Vol. MLP (independently tuned)0.417 ± 0.0360.425 ± 0.0080.064 ± 0.011

Thus, independent hyperparameter tuning partially improves Vol. MLP, particularly in CRPS and calibration, but does not close the substantial test-performance gap. The tuned Vol. MLP remains considerably worse than DecoVAE across all three metrics. This indicates that the gap cannot be explained primarily by the shared hyperparameter setting and provides further evidence that the structured residual representation contributes to improved generalization on ETTh2.

We will include these independently tuned results and clarify this point in the revised manuscript.

Thank you again for your follow-up. For convenience, we reproduce our complete response to your remaining question below.

Thank you for this important suggestion. We independently tuned Vol. MLP using the same optimization budget and model-selection protocol as the full DecoVAE model. Specifically, we performed 100 Optuna trials and selected the final configuration solely according to validation CRPS; the test set was not used during model selection.

On ETTh2, the independently tuned Vol. MLP improves test CRPS from 0.444 ± 0.031 to 0.417 ± 0.036 and ECE from 0.076 ± 0.012 to 0.064 ± 0.011, while NMAE remains essentially unchanged (0.423 ± 0.009 vs. 0.425 ± 0.008).

ModelCRPSNMAEECE
DecoVAE0.293 ± 0.0010.378 ± 0.0020.026 ± 0.003
Vol. MLP0.444 ± 0.0310.423 ± 0.0090.076 ± 0.012
Vol. MLP (independently tuned)0.417 ± 0.0360.425 ± 0.0080.064 ± 0.011

Thus, independent hyperparameter tuning partially improves Vol. MLP, particularly in CRPS and calibration, but does not close the substantial test-performance gap. The tuned Vol. MLP remains considerably worse than DecoVAE across all three metrics. This indicates that the gap cannot be explained primarily by the shared hyperparameter setting and provides further evidence that the structured residual representation contributes to improved generalization on ETTh2.

These independently tuned results and the corresponding clarification have already been incorporated into the updated manuscript PDF.

Please let us know if you have any remaining questions or concerns that we could address.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.