- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RegimeUnfold, a continual regime-space learning framework for deep hidden Markov process forecasting that does not assume a fixed regime set. Its offline regime discovery stage constructs the regime space without predefining its size. Its online regime evolution stage updates existing regimes or expands the regime space under stability constraints. Experiments on five synthetic and four real-world datasets are reported to show accurate forecasting, reliable regime inference, and effective adaptation.
Critical strengths
- Well-motivated problem. The paper targets a practical limitation of existing DL-HMM hybrids that assume a fixed regime space specified before training.
- Clear benefit when unseen dynamics emerge. On the Progressive dataset the full model raises regime inference accuracy from 51.56% to 91.49% over RegimeUnfold w/o Evolution.
- Detailed apendix with techinical details is provided.
Critical weaknesses
- Inconsistent baseline reporting. Tables 1 and 2 mix Adaptive and Non-adaptive rows across baselines and omit the Large-window variants, which overstates the gains reported in the main text.
- Regime-number sensitivity is shifted to thresholds. The method still requires a maximum allowed regime-space size along with many thresholds and loss weights, yet neither their values nor a sensitivity analysis is provided.
- Key claims and components are not directly validated. No experiment isolates the benefit of hard regime decoding or the ability to update drifting regimes, and the two ablations cannot attribute gains to individual components.
- Experimental setting does not match the stated motivation. Linear AR(1) synthetic data and 3- to 5-dimensional real-world datasets cannot test the claimed advantage on complex nonlinear predictive dependencies in high-dimensional multivariate time series, and online regime evolution is only shown qualitatively.
- Limited discussion of novelty and related work. The offline split and merge procedure closely resembles the birth and merge moves of [1] without discussion, and comparisons with deep switching dynamical systems and online forecasting methods are missing.
[1] Scalable adaptation of state complexity for nonparametric hidden Markov models.
Questions for authors
See weakness
8 comments
Weakness 3: Key claims and components are not directly validated. No experiment isolates the benefit of hard regime decoding or the ability to update drifting regimes, and the two ablations cannot attribute gains to individual components.
Response to Weakness 3: Thank you for the comment. To further validate the key mechanisms of RegimeUnfold, we conducted additional component-wise ablation studies covering hard regime decoding, adaptation of established regimes, and stability-aware control. Specifically, we consider the following ablations:
- Soft Forecast Aggregation: Replaces hard single-expert forecasting with a probability-weighted average of regime-specific predictions. Both approaches retain argmax-based regime identification, allowing us to examine the contribution of hard regime decoding to forecasting.
- w/o UPDATE: Prevents the incorporation of updates to established regimes to evaluate the contribution of adapting existing predictive dynamics.
- w/o Established-Regime Stability Checks at Admission: Removes the protection checks for established regimes during candidate acceptance to evaluate their role in preserving previously learned dynamics.
We evaluate these ablations on Fixed-K2, Fixed-K3, and Progressive. The results are reported below.
Table 1: Effects of hard regime decoding and online adaptation mechanisms on Fixed-K2.
| Model | Accuracy (%) | F1 (%) | MAE | MSE |
|---|---|---|---|---|
| Soft Forecast Aggregation | 95.78 ± 2.60 | 96.88 ± 2.27 | 0.0094 ± 0.0002 | 0.00022 ± 0.00001 |
| w/o UPDATE | 95.53 ± 2.45 | 96.84 ± 2.25 | 0.0092 ± 0.0002 | 0.00021 ± 0.00001 |
| w/o Established-Regime Stability Checks at Admission | 95.78 ± 2.60 | 96.88 ± 2.27 | 0.0093 ± 0.0002 | 0.00022 ± 0.00001 |
| RegimeUnfold | 97.85 ± 2.22 | 97.68 ± 2.08 | 0.0093 ± 0.0002 | 0.00022 ± 0.00001 |
Table 2: Effects of hard regime decoding and online adaptation mechanisms on Fixed-K3.
| Model | Accuracy (%) | F1 (%) | MAE | MSE |
|---|---|---|---|---|
| Soft Forecast Aggregation | 95.23 ± 1.12 | 95.32 ± 0.48 | 0.0098 ± 0.0002 | 0.00041 ± 0.00003 |
| w/o UPDATE | 94.71 ± 1.37 | 95.16 ± 0.47 | 0.0097 ± 0.0002 | 0.00041 ± 0.00003 |
| w/o Established-Regime Stability Checks at Admission | 95.23 ± 1.12 | 95.32 ± 0.48 | 0.0097 ± 0.0002 | 0.00041 ± 0.00003 |
| RegimeUnfold | 96.31 ± 0.23 | 95.88 ± 0.28 | 0.0093 ± 0.0001 | 0.00026 ± 0.00001 |
Table 3: Effects of hard regime decoding and online adaptation mechanisms on Progressive.
| Model | Accuracy (%) | F1 (%) | MAE | MSE |
|---|---|---|---|---|
| Soft Forecast Aggregation | 86.13 ± 4.01 | 86.23 ± 5.49 | 0.0146 ± 0.0010 | 0.00089 ± 0.00008 |
| w/o UPDATE | 86.90 ± 5.07 | 86.87 ± 6.26 | 0.0145 ± 0.0013 | 0.00094 ± 0.00011 |
| w/o Established-Regime Stability Checks at Admission | 66.25 ± 16.79 | 55.10 ± 25.17 | 0.0260 ± 0.0110 | 0.00243 ± 0.00157 |
| RegimeUnfold | 91.49 ± 0.61 | 91.19 ± 0.77 | 0.0134 ± 0.0000 | 0.00081 ± 0.00001 |
The full RegimeUnfold achieves higher regime inference accuracy than Soft Forecast Aggregation across all three datasets, while maintaining lower or comparable forecasting errors. On Progressive, replacing hard single-expert forecasting with soft aggregation reduces accuracy from 91.49% to 86.13% and increases MSE from 0.00081 to 0.00089. These results support the effectiveness of hard regime decoding within our regime-aware forecasting framework.
Disabling UPDATE also reduces regime inference accuracy across all three datasets. On Progressive, accuracy decreases from 91.49% to 86.90%, while MSE increases from 0.00081 to 0.00094, supporting the contribution of adapting established regimes. Furthermore, removing established-regime stability checks substantially degrades performance on Progressive, reducing accuracy to 66.25% and increasing MSE to 0.00243. These results demonstrate the importance of stability-aware control during online regime evolution.
The complete component-wise ablation studies, including additional training objectives, regime decoding mechanisms, regime selection components, and evolution assessment criteria, are provided in our response to Reviewer Ay5y, Weakness 1, and in Appendix G of the revised manuscript.
Weakness 4: Experimental setting does not match the stated motivation. Linear AR(1) synthetic data and 3- to 5-dimensional real-world datasets cannot test the claimed advantage on complex nonlinear predictive dependencies in high-dimensional multivariate time series, and online regime evolution is only shown qualitatively.
Response to Weakness 4: Thank you for the comment. To further evaluate RegimeUnfold under more complex predictive dynamics, we constructed an additional 64-dimensional nonlinear synthetic dataset with an evolving latent regime space.
The dataset contains 20,000 time steps and 64 dynamically coupled variables. Unlike the original linear autoregressive settings, the observations are generated using nonlinear cross-variable interactions and regime-dependent dynamics with four-step historical dependencies. Specifically, let denote the observation at time , and let denote its latent regime. We first construct nonlinear cross-variable features:
where is a sparse projection matrix with four nonzero entries per row, producing 32 nonlinear features from the observed variables. Each observation dimension is then generated as
y_{t,j}={}&a_{z_t}y_{t-1,j} +0.10\bar y_{t,j}\\ &+0.45s_j^{(0)}\phi_{t,c_j}\\ &+0.30s_j^{(z_t)}\phi_{t,r_{z_t,j}}\\ &+0.05\phi_{t,u_j}\phi_{t,v_j} \tanh(2.5\bar y_{t,w_j}) +\epsilon_{t,j}, \end{aligned}$$ where $a_{z_t}\in\{-0.45,0.45\}$ is a regime-dependent autoregressive coefficient, $s_j^{(0)}$ and $s_j^{(z_t)}$ are shared and regime-specific sign coefficients, and $c_j$, $r_{z_t,j}$, $u_j$, $v_j$, and $w_j$ specify cross-variable feature connections. The regime-specific connections vary across regimes, while $\epsilon_{t,j}\sim\mathcal{N}(0,0.12^2)$ denotes independent Gaussian noise. This construction combines shared nonlinear dynamics, regime-specific predictive mechanisms, and multiplicative nonlinear interactions. All 64 variables participate in the conditional dynamics rather than serving as additional noise dimensions. The dataset also retains the progressive regime evolution setting, with the number of available regimes increasing from 2 to 3 and then 4. New regimes are introduced at 50% and 70% of the sequence, respectively. Regime transitions follow second-order dependencies with state-dependent duration distributions. **Table 4: Regime inference and forecasting results on the 64-dimensional nonlinear Progressive dataset.** | Model | Accuracy (%) | F1 (%) | MAE | MSE | |---|---|---|---|---| | RegimeUnfold | 89.51 ± 2.94 | 84.01 ± 3.92 | 0.1564 ± 0.0117 | 0.04999 ± 0.00959 | RegimeUnfold achieves 89.51% regime inference accuracy and 84.01% F1, with MAE of 0.1564 and MSE of 0.04999, averaged over five runs. These results provide additional evidence that RegimeUnfold can discover evolving latent regimes and perform forecasting under higher-dimensional nonlinear predictive dynamics. Furthermore, we conducted additional quantitative ablation studies on UPDATE, CREATE, and stability-aware control, as detailed in our response to **Reviewer NbVW, Weakness 2** and **Appendix G** of the revised manuscript. These experiments further validate the contributions of the online regime evolution mechanisms.
Thank you very much for your valuable suggestions. We have provided point-by-point responses to address all the raised questions and concerns.
Weakness 1: Inconsistent baseline reporting. Tables 1 and 2 mix Adaptive and Non-adaptive rows across baselines and omit the Large-window variants, which overstates the gains reported in the main text.
Response to Weakness 1: Thank you for the suggestion. Due to space limitations, the original Tables 1 and 2 provided a compact summary of representative baseline results, while the complete results were reported in Appendix F. In addition, the Large-window variants use substantially longer historical inputs than the default configurations, making the comparisons less directly comparable under the same input settings. Therefore, these variants were initially presented only in the appendix.
We understand that a more complete presentation would help clarify the differences between experimental settings. To address this concern, we have revised Tables 1 and 2 to explicitly distinguish Adaptive and Non-adaptive settings and include the Large-window variants. We have also revised the corresponding comparisons and conclusions in the main text to provide a more comprehensive and fair evaluation.
Weakness 2: Regime-number sensitivity is shifted to thresholds. The method still requires a maximum allowed regime-space size along with many thresholds and loss weights, yet neither their values nor a sensitivity analysis is provided.
Response to Weakness 2: Thank you for the comment. The key distinction is that a fixed regime number K directly determines the size of the regime space, whereas the maximum allowed regime number Kmax only limits the search capacity without prescribing the final number of regimes. RegimeUnfold instead determines the regime-space size through evidence-based structural adaptation. During offline discovery, structural changes are evaluated based on predictive improvement, structural reliability, and regime-transition evidence. During online evolution, UPDATE and CREATE decisions are evaluated based on accumulated evolution evidence, predictive improvement, and stability requirements. Unlike a fixed regime number, these criteria allow the regime space to adapt when previously unseen dynamics emerge. In our experiments, Kmax was set above the number of regimes ultimately identified, and the limit was never reached.
To further examine parameter sensitivity, we conducted additional experiments covering eleven hyperparameters or groups of related parameters for offline regime discovery and four parameters for online regime evolution. The results show relatively limited sensitivity to the tested structural selection settings. On Fixed-K3, varying the split acceptance threshold γsplit from 0.015 to 0.06 yields mean regime inference accuracies between 99.07% and 99.11%. Similarly, varying Kmax over {4,8,12} yields accuracies of 97.32%, 99.09%, and 98.99%, respectively. These results indicate that regime discovery performance remains relatively stable across the examined ranges of these two structural controls, although other hyperparameters have a greater influence.
The complete sensitivity analysis, including loss weights, structural selection thresholds, and online adaptation parameters, is presented in our response to Reviewer Ay5y, Question 3, with the corresponding experimental results added to Appendix G of the revised manuscript. The relevant parameter settings are provided in Appendix F, while the detailed offline and online procedures are described in Appendices A and B, respectively.