acceptodds
Under review as a conference paper at ICLR 2027

MDN: A Multi-Scale Disentanglement Network for Time Series Forecasting

Abstract

Although multi-scale modeling methods have achieved success in time series forecasting, they often overlook cross-scale entanglement and directly model cross-scale interactions among entangled features, allowing entanglement to interfere with interaction modeling. Extracting disentangled multi-scale features before modeling interactions can mitigate such interference. However, extracting such features faces two challenges: (1) Non-linear cross-scale entanglement is difficult to disentangle effectively. (2) Scale-shared information confounds the disentanglement process. To this end, we propose a **M**ulti-scale **D**isentanglement **N**etwork (MDN) for time series forecasting. Specifically, MDN first employs a Product-of-Experts-based separation module to infer and remove a robust scale-shared feature, thereby reducing interference from scale-shared information. Then, a mutual information (MI)-based disentanglement constraint is designed to minimize pairwise MI upper bounds among entangled features, thereby mitigating non-linear cross-scale entanglement. Finally, MDN sequentially models the intra-scale and inter-scale interactions among the disentangled features. Extensive experiments on 7 real-world datasets demonstrate that MDN achieves state-of-the-art performance against 14 competitive baselines. Code is available at: https://anonymous.4open.science/r/MDN-6BF2.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Multi-Scale Modeling, Cross-Scale Entanglement, Disentangled Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_5DPCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MDN, a multi-scale disentanglement network for time series forecasting that uses PoE to separate scale-shared information and MI regularization to reduce dependencies among scale-specific features. The framework also models intra- and inter-scale interactions, then performs multi-scale forecasting with auxiliary supervision. Experiments on seven benchmark datasets show competitive forecasting performance.

Critical strengths
  1. This paper explores the critical issue of cross-scale feature coupling in time series forecasting.
  2. The proposed framework is clearly articulated, with a detailed description of its key components and training process.
  3. Experiments are conducted on seven widely used datasets, comparing MDN against other benchmark forecasting models.
Critical weaknesses
  1. The methodological innovation is limited, as the framework primarily integrates existing techniques rather than proposing a fundamentally novel approach to decoupling.
  2. The cross-scale interaction module presented in Eq 6 merely combines scale-specific features with a shared representation; it does not explicitly model the interactions between the scale-specific features themselves across different scales.
  3. Although the paper employs CLUB to minimize an upper bound on the estimate of MI, it fails to examine the reliability of these estimates.
Questions for authors

See above

6 comments
AuthorsOpenReview ↗

Response to Reviewer 5DPC (Weaknesses 1)

Weakness 1. The methodological innovation is limited, as the framework primarily integrates existing techniques rather than proposing a fundamentally novel approach to decoupling.

Thanks for your valuable comments and scientific rigor. The primary contribution of MDN is not to propose a fundamentally new general-purpose disentanglement method, but to identify and address the underexplored problem of cross-scale entanglement in multi-scale time series forecasting. To the best of our knowledge, MDN is the first work to employ a mutual information-based disentanglement constraint to extract disentangled multi-scale temporal features for time series forecasting.

In addition, MDN introduces a novel and coordinated design to address two key challenges in extracting disentangled multi-scale temporal features: non-linear cross-scale entanglement and scale-shared information interference. Specifically, the PoE-based SSFS module adaptively extracts and separates scale-shared information to mitigate its interference with disentanglement while preserving global context for cross-scale interaction modeling. Subsequently, the MI-based SSFD module minimizes pairwise MI upper bounds among scale-specific residuals to mitigate non-linear cross-scale entanglement. These complementary mechanisms facilitate the extraction of disentangled multi-scale temporal features for subsequent interaction modeling. Furthermore, the ablation studies in Section 4.3 (Table 2) and Appendix D.4 (Table 9) demonstrate the effectiveness and necessity of MDN's key designs in improving forecasting performance.

Overall, the methodological novelty of MDN lies in identifying cross-scale entanglement as an important problem in multi-scale time series forecasting and developing a coordinated disentanglement framework to address its two key challenges, thereby improving time series forecasting performance.

AuthorsOpenReview ↗

Response to Reviewer 5DPC (Weaknesses 2)

Weakness 2. The cross-scale interaction module presented in Eq 6 merely combines scale-specific features with a shared representation; it does not explicitly model the interactions between the scale-specific features themselves across different scales.

Thanks for your insightful feedback and scientific rigor. We clarify that the cross-scale interaction module in Eq. (6) adopts a centralized interaction mechanism, which introduces a structural inductive bias that enables cross-scale interaction modeling to leverage global contextual information. Specifically, the shared representation PP, extracted from all scales through the PoE mechanism, captures scale-shared context and serves as a central representation for information exchange across scales. By fusing each scale-specific feature with PP, MDN facilitates indirect cross-scale interactions while integrating scale-specific patterns with global contextual information. This design also avoids the additional computational complexity associated with explicitly modeling pairwise interactions between scale-specific features.

(1) Explicit Cross-Scale Interaction Modeling Ablation.

To investigate whether explicitly modeling interactions between scale-specific features provides additional benefits, we conducted an ablation on ETTm2 with an input length of 96 and prediction lengths of 96, 192, 336, and 720. Specifically, we replaced the original centralized interaction mechanism with explicit pairwise interactions between scale-specific features (MDN-Explicit). For each scale, we independently modeled its interactions with the other scales using separate MLPs and averaged the resulting interaction features. MDN-Explicit used the same training configurations as MDN but had more trainable parameters.

Table 1.

Prediction LengthMDN MSE ↓MDN-Explicit MSE ↓MDN MAE ↓MDN-Explicit MAE ↓
960.16880.17470.24710.2527
1920.23390.24020.29040.2949
3360.29320.30100.32940.3353
7200.38840.39320.38690.3895

As shown in the table, MDN consistently outperforms MDN-Explicit across all four prediction lengths. Specifically, replacing the original centralized interaction mechanism with explicit pairwise interactions increases MSE and MAE by an average of 2.53% and 1.57%, respectively, across the four prediction horizons. This performance degradation may stem from the lack of direct access to scale-shared global context in explicit pairwise interactions, potentially limiting effective information integration across scales.

Overall, MDN employs a centralized interaction mechanism that enables scale-specific features to leverage global contextual information for indirect cross-scale interactions. The experimental results further support the effectiveness of this structural inductive bias.

AuthorsOpenReview ↗

Response to Reviewer 5DPC (Weaknesses 3)

Weakness 3. Although the paper employs CLUB to minimize an upper bound on the estimate of MI, it fails to examine the reliability of these estimates.

Thanks for your valuable comments and scientific rigor. Estimating mutual information (MI) between high-dimensional representations remains a challenging problem, particularly when complex nonlinear dependencies are involved. To examine the reliability of the CLUB estimator employed in MDN, we conducted additional experiments following the synthetic evaluation protocol of the original CLUB paper [1].

(1) CLUB Estimation on Synthetic Data.

We evaluated CLUB on Gaussian and nonlinear Cubic distributions with analytically known ground-truth MI. We set the feature dimension to 32, matching the residual feature dimension used in MDN, and varied the ground-truth MI over 0,2,4,6,8,10\\{0,2,4,6,8,10\\} nats. Each experiment was repeated with five random seeds.

Table 2.

Ground-truth MIGaussian CLUBCubic CLUB
00.0001 ± 0.00020.0007 ± 0.0062
22.3943 ± 0.01332.5252 ± 0.0108
44.2888 ± 0.02654.4055 ± 0.0594
66.4951 ± 0.04626.6938 ± 0.1298
88.6924 ± 0.12468.7392 ± 0.1364
1010.8940 ± 0.184010.9626 ± 0.1449

As shown in the table, CLUB estimates closely track the ground-truth MI across both Gaussian and Cubic distributions, exhibiting slight overestimation while consistently distinguishing different levels of MI. This slight overestimation may be attributed to the fact that CLUB estimates an upper bound on MI. Moreover, the small standard deviations across five random seeds indicate stable estimation performance. These results support the accuracy and reliability of CLUB estimation.

(2) Validation on Real-World Data.

In our response to Reviewer D1BQ (Question 4), we demonstrated that introducing MI regularization consistently reduces cross-scale entanglement on ETTm2, as independently verified by refitted CLUB estimators and normalized HSIC. These findings provide empirical support for the reliability of CLUB estimation in MDN.

Overall, these experiments demonstrate that CLUB provides accurate and stable dependence estimates on synthetic data with known MI, while independent evaluations on real-world data further support the reliability of CLUB estimation in measuring cross-scale entanglement.

[1] Cheng P, Hao W, Dai S, et al. CLUB: A contrastive log-ratio upper bound of Mutual Information. ICML, 2020.

Reviewer_D1BQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MDN, a multivariate long-term forecasting architecture intended to reduce statistical dependence among multi-scale representations before modeling their interactions. Average pooling creates multiple temporal scales, and scale-specific embedders produce preliminary features. A Product-of-Experts-inspired module predicts a Gaussian mean and variance from each scale and combines the means by inverse-variance weighting to form a shared feature, which is subtracted from every scale. Pairwise CLUB estimators then supply a mutual-information upper-bound regularizer over the residual scale-specific features. The model applies variable-wise attention, fuses the shared feature back into each scale, predicts at every scale, and recursively fuses coarse-to-fine forecasts under time- and frequency-domain losses.

Critical strengths
  1. The paper gives the CLUB optimization schedule, search spaces, final settings, dataset splits, baseline sources, and an anonymous code repository. MDN is very small and computationally light relative to TimeKAN and AMD, while the training-only CLUB estimators are removed at inference. This makes the reported forecasting gains more compelling than gains obtained solely through a much larger model.

  2. MDN obtains the lowest average MAE on all seven datasets and the lowest average MSE on six. The paper also reports hyperparameter sensitivity, feature-dependence diagnostics, computational cost, and five-seed robustness for two representative datasets.

Critical weaknesses
  1. The paper does not substantiate the semantic “disentanglement” claim. The so-called shared feature is an inverse-variance-weighted combination learned only through the forecasting objective; there is no identifiability condition, reconstruction constraint, invariance test, or ground-truth factor supervision showing that it contains shared rather than arbitrary information. Subtracting this feature and minimizing pairwise MI can suppress valid dependencies between temporal scales, even though the introduction itself motivates cross-scale interactions as meaningful.

  2. The statistical comparison does not establish that the often-small gains are reliable. Most baseline numbers are taken from published releases or other repositories, while only TimeKAN and AMD are reproduced. MDN is tuned separately for every dataset/horizon over nontrivial grids, but competing methods are not uniformly retuned under the same budget. The main tables report single point estimates; the five-seed analysis covers only MDN on ETTm1 and Electricity and cannot quantify uncertainty in pairwise improvements. Several differences from strong baselines are only 0.001-0.005, so common preprocessing alone does not rule out seed, implementation, or tuning-budget effects.

Questions for authors
  1. What prevents the PoE-inspired module from learning a degenerate or arbitrary feature that merely helps the downstream forecaster? Please provide evidence that the inferred feature is actually shared across scales and that the residuals retain scale-specific predictive information.

  2. Does reducing pairwise MI ever remove useful cross-scale dependence? Please report the forecasting performance and retained target information as the MI weight increases.

  3. Are the 0.001-0.005 improvements over the strongest competitors stable on ETT and Weather?

  4. For the CLUB objective, mini-batch features are reshaped to (M=BN). Does treating variables from different series/windows as exchangeable samples introduce a variable-identity shortcut or conflate cross-variable dependence with cross-scale dependence?

6 comments
AuthorsOpenReview ↗

Response to Reviewer D1BQ (Question 1)

Question 1. What prevents the PoE-inspired module from learning a degenerate or arbitrary feature that merely helps the downstream forecaster? Please provide evidence that the inferred feature is actually shared across scales and that the residuals retain scale-specific predictive information.

Thanks for your insightful comments and scientific rigor. As described in Section 3.4, MDN employs a Product-of-Experts (PoE) mechanism to extract the scale-shared feature PP. Specifically, each scale adaptively generates a scale-shared feature candidate and its corresponding uncertainty, which are aggregated through uncertainty-weighted fusion across all scales. This mechanism provides an architectural inductive bias toward adaptively extracting and aggregating shared information distributed across different scales into the scale-shared feature PP. To verify whether PP is actually shared across scales and whether the residuals retain scale-specific predictive information, we conducted the following controlled experiments.

(1) Scale-shared Information in PP.

We constructed a synthetic dataset containing one known scale-shared factor, three known scale-specific factors, and additional noise. The shared factor generates temporal patterns observable across all scales. After freezing the trained MDN, we employed an independent ridge-regression probe to recover the scale-shared factor from PP. We further performed controlled interventions by reversing the sign of the scale-shared factor while keeping all other components unchanged.

Table 1. Scale-shared Factor Recovery and Intervention Results.

MetricR2R^2 ↑
Factor Recovery0.9128
Factor Intervention0.9182

The results show that the scale-shared factor can be accurately recovered from PP, achieving an R2R^2 of 0.9128. In addition, even after reversing the sign of the scale-shared factor, the recovery performance remains nearly unchanged, achieving an R2R^2 of 0.9182. These findings provide empirical evidence that PP captures meaningful information that is actually shared across scales.

(2) Scale-specific Predictive Information in Residuals.

In our response to Reviewer yEeW (Question 1), we demonstrate that the MI-based disentanglement constraint preserves the recoverability of corresponding scale-specific factors and the responses of residual representations to interventions on these factors. These findings indicate that the corresponding scale-specific factors can be accurately predicted from the scale-specific residuals.

Building on the same controlled three-scale synthetic dataset, we further conducted experiments to examine whether the residuals retain scale-specific predictive information. Specifically, we froze the trained MDN and employed independent ridge-regression probes to predict the future amplitudes of scale-specific temporal patterns under two settings: using PP alone (PP-only) and combining PP with the corresponding scale-specific residual (P+RsP+R_s).

Table 2. Scale-specific Predictive Information on Synthetic Data.

ScalePP-only MSE ↓P+RsP+R_s MSE ↓ΔMSEPP-only MAE ↓P+RsP+R_s MAE ↓ΔMAE
Scale 10.10350.0828−20.04%0.24160.2105−12.86%
Scale 20.10820.0692−36.03%0.25090.1998−20.35%
Scale 30.04460.0250−43.92%0.16350.1219−25.47%

ΔMSE and ΔMAE denote the relative changes in forecasting errors compared with PP-only

The results show that incorporating the corresponding scale-specific residual reduces prediction MSE and MAE across the three scales by an average of 33.33% and 19.56%, respectively. These findings demonstrate that each residual retains additional scale-specific predictive information beyond the scale-shared feature PP.

In addition, in our response to Reviewer yEeW (Question 3), we demonstrate that the scale-specific residuals retain additional predictive information beyond PP and make complementary contributions to forecasting through forecasting and information ablations on the real-world ETTm2 dataset. Finally, the controlled synthetic experiments and real-world ablations provide empirical evidence that the residuals retain meaningful scale-specific predictive information.

Overall, our analyses of the scale-shared feature and scale-specific residuals provide evidence that the PoE-inferred scale-shared feature is actually shared across scales, while the residuals preserve additional scale-specific predictive information for forecasting.

AuthorsOpenReview ↗

Response to Reviewer D1BQ (Question 2)

Question 2. Does reducing pairwise MI ever remove useful cross-scale dependence? Please report the forecasting performance and retained target information as the MI weight increases.

Thanks for your valuable comments and scientific rigor. The MI-based disentanglement constraint aims to mitigate cross-scale entanglement while preserving useful predictive information through forecasting supervision. To examine whether useful cross-scale dependence is retained as the MI weight increases, we conducted the following controlled experiments.

We constructed a controlled three-scale synthetic dataset containing a known inter-scale interaction term I=0.4z1z3I=0.4z_1z_3, where z1z_1 and z3z_3 represent scale-specific factors at scale 1 and scale 3, respectively. This interaction contributes to the future prediction target, allowing us to examine whether useful cross-scale dependence is preserved.

We varied the MI regularization weight λMID\lambda_{\mathrm{MID}} across six settings while keeping all other experimental settings identical. We evaluated forecasting performance using MSE and MAE. To assess retained target information, we froze the trained MDN and employed independent MLP probes to recover the known inter-scale interaction term II from the learned representations [P;R1;R2;R3][P;R_1;R_2;R_3].

Table 3. MI Weight Sensitivity and Retained Inter-scale Interaction Information.

MI WeightMSE ↓MAE ↓Interaction R2R^2 ↑
00.07910.20670.9329
0.00010.07770.20450.9330
0.00050.07590.20130.9332
0.0010.07530.20000.9350
0.0030.07670.20150.9330
0.0050.08000.20580.9344

The results show that moderate MI regularization can improve forecasting performance. Specifically, at an MI weight of 0.001, forecasting MSE and MAE decrease by 4.86% and 3.26% compared with the setting without MI regularization. As the MI weight further increases, forecasting errors begin to rise, indicating that excessively strong MI regularization may impair forecasting performance. Meanwhile, the known inter-scale interaction term remains highly recoverable across all six MI weights, with the probe R2R^2 consistently exceeding 0.93. These findings demonstrate that useful cross-scale predictive information is retained in the learned representations.

Overall, these experiments demonstrate that forecasting performance initially improves and then degrades as the MI weight increases, while useful cross-scale dependence remains highly recoverable from the learned representations across all tested weights. These findings indicate that moderate MI regularization can improve forecasting performance while retaining useful cross-scale dependence.

AuthorsOpenReview ↗

Response to Reviewer D1BQ (Question 3)

Question 3. Are the 0.001-0.005 improvements over the strongest competitors stable on ETT and Weather?

Thanks for your valuable comments and scientific rigor. We acknowledge that achieving substantial performance improvements on these extensively studied benchmarks is challenging, making it particularly important to assess the statistical significance of smaller improvements.

In our response to Reviewer yEeW (Question 2), we conducted multi-seed experiments and paired tests under a unified evaluation protocol. The results show that MDN achieves statistically significant improvements in both average MSE and MAE on ETTh2, ETTm1, and ETTm2 (p<0.05p < 0.05). On ETTh1, MDN achieves a significantly lower average MSE, while its improvement in MAE is not statistically significant. On Weather, TimeKAN achieves a significantly lower average MSE, whereas MDN achieves a significantly lower average MAE.

Overall, these results provide statistical evidence for the stability of MDN's average forecasting performance across the ETT and Weather datasets, while transparently acknowledging the exceptions observed on ETTh1 and Weather.

Reviewer_yEeWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes MDN for long-horizon multivariate forecasting. It obtains pooled representations at several temporal scales, estimates a common feature with a product-of-experts module, subtracts it from each scale, and minimizes pairwise CLUB mutual-information estimates among the residuals. The resulting features are processed by intra- and inter-scale modules and fused into forecasts. The paper reports strong MSE/MAE results on seven standard datasets and ablations of the components

Critical strengths
  • The model is clearly specified and the ablations cover the main proposed components: shared-feature separation, the mutual-information penalty, interaction modules, and multi-scale supervision

  • The empirical gains in the main table are broad, and the paper includes useful checks of seed stability for ETTm1 and Electricity as well as inference-cost reporting

Critical weaknesses
  • The central disentanglement claim is not established by the evidence. Pairwise CLUB penalties on learned residuals do not identify scale-specific causal factors; they can suppress forecast-relevant dependence or relocate common information into the shared representation. Lower Pearson correlation, HSIC, and visually separated t-SNE plots are expected consequences of the objective and are not an independent validation that the residual features are meaningful or that the claimed mechanism improves robustness. No controlled-factor or distribution-shift experiment tests this claim.

  • The comparison protocol weakens the strength of the forecasting conclusion. Only TimeKAN and AMD are reproduced; the remaining baselines are taken from published studies despite the paper claiming a common look-back/evaluation setup. The main table has no uncertainty or significance estimates, while five-seed results are limited to two dataset families. Several reported differences are small, so fair reimplementation and full multi-seed comparisons with the strongest current baselines are necessary

Questions for authors
  1. Can the authors demonstrate recovery or intervention on known multi-scale factors in a controlled dataset, and test whether MDN improves under a documented distribution shift rather than only reducing dependence statistics on an in-distribution test set?

  2. Please rerun the strongest baselines under the exact preprocessing, split, horizon, look-back, tuning budget, and seed protocol used for MDN. Provide mean, standard deviation/confidence intervals, and paired tests for all seven datasets.

  3. What prevents the shared feature from becoming a general information bypass while the MI penalty merely forces residuals to be independent? Please report information/forecasting ablations for the shared and each residual pathway.

5 comments
AuthorsOpenReview ↗

Response to Reviewer yEeW (Question 1) (Part 1/2)

Question 1. Can the authors demonstrate recovery or intervention on known multi-scale factors in a controlled dataset, and test whether MDN improves under a documented distribution shift rather than only reducing dependence statistics on an in-distribution test set?

Thanks for your valuable comments and scientific rigor. To provide independent evidence beyond feature-dependence statistics, we conducted two controlled experiments to evaluate (1) the recovery and intervention responses of known multi-scale factors and (2) forecasting improvement under a documented distribution shift.

(1) Controlled Multi-scale Factor Recovery and Intervention.

We constructed a synthetic dataset containing one known scale-shared factor, three known scale-specific factors, and additional noise. The scale-specific factors generate distinct temporal patterns at different scales, while the shared factor generates temporal patterns observable across all scales. We compared the full MDN with its variant without the MI-based disentanglement constraint (w/o MI), while keeping all other experimental settings identical.

Factor Recovery. After freezing the trained models, we employed independent ridge-regression probes to recover each ground-truth scale-specific factor from the residual representation of each scale. The recovery performance was evaluated using R2R^2 on the test set.

Table 1. Scale-specific Factor Recovery (R2R^2).

ResidualFactor 1Factor 2Factor 3
Scale 10.8826 → 0.88370.5651 → 0.15090.7176 → 0.2384
Scale 20.6639 → 0.50210.8604 → 0.86040.8259 → 0.4465
Scale 30.7063 → 0.48220.4601 → 0.20890.8406 → 0.8392

Each entry denotes w/o MI → MDN. Bold values indicate the recovery of corresponding scale-specific factors.

The results show that the MI-based disentanglement constraint consistently preserves the recoverability of corresponding scale-specific factors (diagonal entries), while reducing the recoverability of non-corresponding factors (off-diagonal entries). Specifically, the average cross-scale factor recovery R2R^2 decreases from 0.6565 to 0.3382 (a 48.49% reduction), whereas the average own-scale recovery remains nearly unchanged, from 0.8612 to 0.8611. These findings demonstrate that the MI-based disentanglement constraint effectively mitigates cross-scale entanglement while preserving the recoverability of scale-specific factors.

Factor Intervention. We further performed controlled interventions by individually reversing the sign of each scale-specific factor while keeping all other components unchanged. We quantified the responses of each scale-specific residual to these interventions in terms of the response slope, using the same frozen models and ridge-regression probes.

Table 2. Factor Intervention Results (Response Slope).

ResidualFactor 1Factor 2Factor 3
Scale 10.8716 → 0.87270.5383 → 0.12600.7299 → 0.2031
Scale 20.5773 → 0.39100.8222 → 0.82260.8370 → 0.3870
Scale 30.6510 → 0.42090.3660 → 0.14580.8534 → 0.8535

Each entry denotes w/o MI → MDN. Bold values indicate the responses to corresponding scale-specific factor interventions.

The results show that the MI-based disentanglement constraint consistently preserves the responses to corresponding scale-specific factor interventions (diagonal entries), while reducing the responses to non-corresponding factor interventions (off-diagonal entries). Specifically, the average cross-scale intervention response decreases from 0.6166 to 0.2790 (a 54.76% reduction), whereas the average own-scale response remains nearly unchanged, from 0.8491 to 0.8496. These findings demonstrate that the MI-based disentanglement constraint preserves the responses of residual representations to corresponding scale-specific factors while reducing their responses to other factors.

Overall, the factor recovery and intervention results demonstrate that the MI-based disentanglement constraint effectively mitigates cross-scale entanglement while identifying scale-specific factors.

AuthorsOpenReview ↗

Response to Reviewer yEeW (Question 1) (Part 2/2)

(2) Forecasting Improvement under a Documented Distribution Shift.

We further constructed a three-scale synthetic dataset with time-varying amplitude modulation to evaluate forecasting performance under distribution shift. Specifically, we introduced a controlled distribution shift by varying the amplitude-growth coefficient kk. The training and in-distribution (ID) test sets use k∈[0,0.30]k\in[0,0.30], while the out-of-distribution (OOD) test set uses k∈[0.35,0.60]k\in[0.35,0.60], introducing a documented distribution shift in amplitude growth.

Table 3. Forecasting Performance under Distribution Shift.

Metricw/o MIMDNImprovement
ID MSE2.36512.048913.37%
ID MAE1.17611.11135.51%
OOD MSE17.014913.486420.74%
OOD MAE2.98652.70009.59%

The results show that the MI-based disentanglement constraint improves forecasting performance under both ID and OOD conditions. Specifically, MDN achieves larger forecasting improvements under OOD (20.74% in MSE and 9.59% in MAE) than under ID (13.37% in MSE and 5.51% in MAE). We attribute the larger improvement under the documented distribution shift to the MI-based disentanglement constraint, which facilitates the extraction of scale-specific factors. Since these factors remain consistent across ID and OOD conditions, they can provide more reliable information for forecasting under distribution shift.

AuthorsOpenReview ↗

Response to Reviewer yEeW (Question 2)

Question 2. Please rerun the strongest baselines under the exact preprocessing, split, horizon, look-back, tuning budget, and seed protocol used for MDN. Provide mean, standard deviation/confidence intervals, and paired tests for all seven datasets.

Thanks for your valuable comments and scientific rigor. To address the concerns regarding fair baseline comparisons and missing statistical significance estimates, we conducted additional multi-seed experiments under a unified evaluation protocol. Specifically, we compared MDN with three of the strongest baselines (i.e., TimeKAN, Time-TK, and DUET) across all seven datasets.

For fair comparisons, all models were evaluated under a unified experimental protocol. Specifically, the four ETT datasets followed a chronological training/validation/test split of 6:2:2, while Weather, Traffic, and ECL datasets followed a split of 7:1:2. All variables were standardized using the mean and standard deviation computed exclusively from the training set. We adopted a look-back length of 96 and prediction horizons of 96, 192, 336, and 720. All models were assigned the same hyperparameter tuning budget of 64 trials, with the best configurations selected based on validation performance. Finally, all models were evaluated using the same five random seeds (i.e., 2021, 2022, 2023, 2024, and 2025).

To address the missing uncertainty and significance estimates, we report the mean and standard deviation of MSE and MAE across five random seeds, together with two-sided paired tt-tests. Specifically, we first averaged the forecasting results across four prediction horizons for each seed and then conducted paired tests across the five matched seeds. For each dataset, we report the strongest reproduced baseline selected according to its average MSE. The complete results for all seven datasets and four prediction horizons will be included in the revised manuscript.

Table 4.

DatasetBaselineMDN MSE ↓Baseline MSE ↓pp (MSE)MDN MAE ↓Baseline MAE ↓pp (MAE)
ETTh1TimeKAN0.4209 ± 0.00110.4278 ± 0.00210.00330.4277 ± 0.00110.4284 ± 0.00150.4473
ETTh2DUET0.3551 ± 0.00130.3709 ± 0.0023<0.001< 0.0010.3856 ± 0.00110.3982 ± 0.0008<0.001< 0.001
ETTm1TimeKAN0.3730 ± 0.00120.3784 ± 0.0009<0.001< 0.0010.3862 ± 0.00060.3945 ± 0.0007<0.001< 0.001
ETTm2Time-TK0.2709 ± 0.00050.2772 ± 0.0011<0.001< 0.0010.3129 ± 0.00050.3212 ± 0.0006<0.001< 0.001
WeatherTimeKAN0.2473 ± 0.00130.2444 ± 0.00140.00600.2688 ± 0.00060.2736 ± 0.00110.0025
TrafficTime-TK0.4131 ± 0.00080.4251 ± 0.0005<0.001< 0.0010.2685 ± 0.00070.2778 ± 0.0008<0.001< 0.001
ECLDUET0.1637 ± 0.00070.1720 ± 0.0012<0.001< 0.0010.2546 ± 0.00060.2592 ± 0.00140.0012

Results are reported as mean ± standard deviation across five random seeds, averaged over four prediction horizons. The pp-values are obtained from two-sided paired tt-tests.

As shown in the table, MDN achieves statistically significant (p<0.05p < 0.05) improvements in both average MSE and MAE over the strongest reproduced baselines on five of the seven datasets (i.e., ETTh2, ETTm1, ETTm2, Traffic, and ECL). Although some improvements are numerically small, the paired tests provide statistical evidence for the stability of these average improvements. On Weather, TimeKAN achieves a significantly lower average MSE, whereas MDN achieves a significantly lower average MAE. On ETTh1, MDN achieves a significantly lower average MSE, while its improvement in MAE is not statistically significant.

Overall, these additional experiments strengthen the fairness of baseline comparisons and provide statistical evidence for the stability of MDN's forecasting improvements across diverse datasets.

Comments on the paper

1 comment
Area_Chair_iabqOpenReview ↗

Reminder: Reviewer–Author Discussion

As the discussion deadline is approaching, I would kindly remind the reviewers to review the authors' rebuttal and provide their follow-up comments. In particular, it would be helpful to clarify whether your original concerns have been addressed and whether your assessment has changed. Even if your assessment remains unchanged, a brief response would be greatly appreciated.

Authors are also encouraged to remain engaged and respond to any follow-up questions. Thank you all for your time and cooperation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.