- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MDN, a multi-scale disentanglement network for time series forecasting that uses PoE to separate scale-shared information and MI regularization to reduce dependencies among scale-specific features. The framework also models intra- and inter-scale interactions, then performs multi-scale forecasting with auxiliary supervision. Experiments on seven benchmark datasets show competitive forecasting performance.
Critical strengths
- This paper explores the critical issue of cross-scale feature coupling in time series forecasting.
- The proposed framework is clearly articulated, with a detailed description of its key components and training process.
- Experiments are conducted on seven widely used datasets, comparing MDN against other benchmark forecasting models.
Critical weaknesses
- The methodological innovation is limited, as the framework primarily integrates existing techniques rather than proposing a fundamentally novel approach to decoupling.
- The cross-scale interaction module presented in Eq 6 merely combines scale-specific features with a shared representation; it does not explicitly model the interactions between the scale-specific features themselves across different scales.
- Although the paper employs CLUB to minimize an upper bound on the estimate of MI, it fails to examine the reliability of these estimates.
Questions for authors
See above
6 comments
Response to Reviewer 5DPC (Weaknesses 2)
Weakness 2. The cross-scale interaction module presented in Eq 6 merely combines scale-specific features with a shared representation; it does not explicitly model the interactions between the scale-specific features themselves across different scales.
Thanks for your insightful feedback and scientific rigor. We clarify that the cross-scale interaction module in Eq. (6) adopts a centralized interaction mechanism, which introduces a structural inductive bias that enables cross-scale interaction modeling to leverage global contextual information. Specifically, the shared representation , extracted from all scales through the PoE mechanism, captures scale-shared context and serves as a central representation for information exchange across scales. By fusing each scale-specific feature with , MDN facilitates indirect cross-scale interactions while integrating scale-specific patterns with global contextual information. This design also avoids the additional computational complexity associated with explicitly modeling pairwise interactions between scale-specific features.
(1) Explicit Cross-Scale Interaction Modeling Ablation.
To investigate whether explicitly modeling interactions between scale-specific features provides additional benefits, we conducted an ablation on ETTm2 with an input length of 96 and prediction lengths of 96, 192, 336, and 720. Specifically, we replaced the original centralized interaction mechanism with explicit pairwise interactions between scale-specific features (MDN-Explicit). For each scale, we independently modeled its interactions with the other scales using separate MLPs and averaged the resulting interaction features. MDN-Explicit used the same training configurations as MDN but had more trainable parameters.
Table 1.
| Prediction Length | MDN MSE ↓ | MDN-Explicit MSE ↓ | MDN MAE ↓ | MDN-Explicit MAE ↓ |
|---|---|---|---|---|
| 96 | 0.1688 | 0.1747 | 0.2471 | 0.2527 |
| 192 | 0.2339 | 0.2402 | 0.2904 | 0.2949 |
| 336 | 0.2932 | 0.3010 | 0.3294 | 0.3353 |
| 720 | 0.3884 | 0.3932 | 0.3869 | 0.3895 |
As shown in the table, MDN consistently outperforms MDN-Explicit across all four prediction lengths. Specifically, replacing the original centralized interaction mechanism with explicit pairwise interactions increases MSE and MAE by an average of 2.53% and 1.57%, respectively, across the four prediction horizons. This performance degradation may stem from the lack of direct access to scale-shared global context in explicit pairwise interactions, potentially limiting effective information integration across scales.
Overall, MDN employs a centralized interaction mechanism that enables scale-specific features to leverage global contextual information for indirect cross-scale interactions. The experimental results further support the effectiveness of this structural inductive bias.
Response to Reviewer 5DPC (Weaknesses 3)
Weakness 3. Although the paper employs CLUB to minimize an upper bound on the estimate of MI, it fails to examine the reliability of these estimates.
Thanks for your valuable comments and scientific rigor. Estimating mutual information (MI) between high-dimensional representations remains a challenging problem, particularly when complex nonlinear dependencies are involved. To examine the reliability of the CLUB estimator employed in MDN, we conducted additional experiments following the synthetic evaluation protocol of the original CLUB paper [1].
(1) CLUB Estimation on Synthetic Data.
We evaluated CLUB on Gaussian and nonlinear Cubic distributions with analytically known ground-truth MI. We set the feature dimension to 32, matching the residual feature dimension used in MDN, and varied the ground-truth MI over nats. Each experiment was repeated with five random seeds.
Table 2.
| Ground-truth MI | Gaussian CLUB | Cubic CLUB |
|---|---|---|
| 0 | 0.0001 ± 0.0002 | 0.0007 ± 0.0062 |
| 2 | 2.3943 ± 0.0133 | 2.5252 ± 0.0108 |
| 4 | 4.2888 ± 0.0265 | 4.4055 ± 0.0594 |
| 6 | 6.4951 ± 0.0462 | 6.6938 ± 0.1298 |
| 8 | 8.6924 ± 0.1246 | 8.7392 ± 0.1364 |
| 10 | 10.8940 ± 0.1840 | 10.9626 ± 0.1449 |
As shown in the table, CLUB estimates closely track the ground-truth MI across both Gaussian and Cubic distributions, exhibiting slight overestimation while consistently distinguishing different levels of MI. This slight overestimation may be attributed to the fact that CLUB estimates an upper bound on MI. Moreover, the small standard deviations across five random seeds indicate stable estimation performance. These results support the accuracy and reliability of CLUB estimation.
(2) Validation on Real-World Data.
In our response to Reviewer D1BQ (Question 4), we demonstrated that introducing MI regularization consistently reduces cross-scale entanglement on ETTm2, as independently verified by refitted CLUB estimators and normalized HSIC. These findings provide empirical support for the reliability of CLUB estimation in MDN.
Overall, these experiments demonstrate that CLUB provides accurate and stable dependence estimates on synthetic data with known MI, while independent evaluations on real-world data further support the reliability of CLUB estimation in measuring cross-scale entanglement.
[1] Cheng P, Hao W, Dai S, et al. CLUB: A contrastive log-ratio upper bound of Mutual Information. ICML, 2020.
Response to Reviewer 5DPC (Weaknesses 1)
Weakness 1. The methodological innovation is limited, as the framework primarily integrates existing techniques rather than proposing a fundamentally novel approach to decoupling.
Thanks for your valuable comments and scientific rigor. The primary contribution of MDN is not to propose a fundamentally new general-purpose disentanglement method, but to identify and address the underexplored problem of cross-scale entanglement in multi-scale time series forecasting. To the best of our knowledge, MDN is the first work to employ a mutual information-based disentanglement constraint to extract disentangled multi-scale temporal features for time series forecasting.
In addition, MDN introduces a novel and coordinated design to address two key challenges in extracting disentangled multi-scale temporal features: non-linear cross-scale entanglement and scale-shared information interference. Specifically, the PoE-based SSFS module adaptively extracts and separates scale-shared information to mitigate its interference with disentanglement while preserving global context for cross-scale interaction modeling. Subsequently, the MI-based SSFD module minimizes pairwise MI upper bounds among scale-specific residuals to mitigate non-linear cross-scale entanglement. These complementary mechanisms facilitate the extraction of disentangled multi-scale temporal features for subsequent interaction modeling. Furthermore, the ablation studies in Section 4.3 (Table 2) and Appendix D.4 (Table 9) demonstrate the effectiveness and necessity of MDN's key designs in improving forecasting performance.
Overall, the methodological novelty of MDN lies in identifying cross-scale entanglement as an important problem in multi-scale time series forecasting and developing a coordinated disentanglement framework to address its two key challenges, thereby improving time series forecasting performance.