- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces MiraDiff, a probabilistic time series forecasting method that combines an analytical local statistical reference with a neural correction network for diffusion-based forecasting. A gaussian mixture is fit once to non-overlapping segments which captures recurrant local patterns and this module is frozen. Then at each diffusion step, gaussian conditioning gives a closed form posterior mean estimating the clean segment from noisy observation. Experiments on 8 datasets report the lowest CRPS among the compared methods across all datasets. Also ablations of hierarchical reference prior configurations is provided with CRPS and Temporal Variogram Score (VS) for all 8 datasets.
Critical strengths
- Strong empirical results (Table 1) and well motivated theory, use of Gaussian mixture posterior inference to construct an analytical denoising reference is interesting. Equations (3)–(9) provide a clear formulation that integrates with standard diffusion training and sampling without modifying the forward process.
- The Standard, Diagonal, Single, and Mixture comparisons are useful and show how different reference configurations affect the results.
Critical weaknesses
- The evaluation only uses 32 test forecast origins and single input window and prediction window (L=168 and T=192) per dataset, which may not be enough to prove proposed method is better. Also the paper does not report statistical significance tests, mainly the paper mentions, "External baselines retain their original training and sampling settings." i believe this may not be a fair comparison, with my experience in time series, results might change by significant amount when trained with different settings, which questions the fairness of main results of paper (Table 1).
- The paper claims that the analytical reference captures local patterns while the neural residual focuses on global dependencies. However, the paper does not provide enough evidence to show that the neural residual actually learns global dependencies, additional ablations and analyses are needed to demonstrate that the two components actually capture their intended dependencies.
Questions for authors
- Were all baselines trained and evaluated with the same test split, and the same normalization?
- How was K selected ? Can you report Mixture with a fixed K across all datasets?
- Why are only 32 forecast origins used per dataset? How are these origins selected, and do the improvements persist with broader test coverage and multiple random seeds?
- What are the additional fitting cost, sampling latency, GPU memory requirements, and overall training costs (with parameters) compared with the Standard reference and leading baselines?
- Does the analytical reference improve convergence speed, training stability, or performance under reduced model capacity or limited training data?
2 comments
Response to Authors rebuttal
I thank the authors for the rebuttal, which addressed some of my concerns. However, some concerns remain open:
- The issue of a single input and prediction window (L=168, T=192) is still open. The revised paper still uses only L=168 and T=192. Is the same setting used for all baselines, and why is only one setting shown?
- In Table 1, considering the standard deviations, MiraDiff’s best results on Traffic, ETTm1, ETTm2, and ETTh2 are not clear wins (e.g., ETTm2: 0.276±0.016 vs. NsDiff 0.280±0.033) over the second-best methods, which makes the proposed model clearly better in only 3/8 datasets.
Considering the rebuttal, I keep my score.
Q1: All methods are evaluated on the exact same test split and under the same normalization. Each dataset is split chronologically (6/2/2 for ETT and 7/1/2 for the others), and the 256 test origins are spaced uniformly across the test split, so every method is scored on identical inputs with identical metrics. Channelwise means and standard deviations are computed on the training split only and applied unchanged to the validation and test sets for all methods. External baselines keep their original training and sampling settings, which is the standard convention for comparability in this benchmark literature, while the controlled comparison in Table 3 (Standard, Diagonal, Single, Mixture) shares the same architecture, training schedule, and diffusion setup, isolating the effect of the reference itself.
Q2: K is selected by minimizing validation CRPS before any test evaluation. For each candidate K from 1 to 8, we fit the mixture reference and train the full model, and the K with the lowest validation CRPS is kept (Fig. D.2, marked by red stars; selected values in Table B.1). Under this protocol, seven of eight datasets select K greater than or equal to 2, and ETTm2 is the only dataset selecting K = 1. The final evaluation under the selected configuration is run across the 5 independent training seeds. Regarding a fixed K, Fig. D.2 already reports validation CRPS for every fixed K from 1 to 8 on all eight datasets. The curves show that performance plateaus once K reaches 6, so a single fixed value such as K = 8 is near optimal on heterogeneous datasets while remaining close to optimal on simpler ones like ETTm2, where K = 1 and K = 2 give nearly identical results. When selecting K separately for each dataset is inconvenient, we therefore recommend a fixed K of 6 to 8 as a robust default.
Q3: The original 32 origins were spaced uniformly across the chronological test split of each dataset, so they spanned the entire test period, though sparsely. To further verify that the conclusions are robust to broader coverage, we expanded the main benchmark (Tables 1 and 2) to 256 uniformly spaced test origins per dataset and repeated the evaluation across 5 independent training seeds, with all methods evaluated on the exact same origins. All main conclusions remain consistent with low variance across seeds. In Table 1, MiraDiff achieves the best CRPS on 7 out of 8 datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic) and the second best on Solar. In Table 2, MiraDiff achieves the best QICE on 4 datasets (ETTm1, Weather, ECL, and Solar) and the second best on 3 datasets (ETTh1, ETTm2, and Traffic).
Q4: The overhead of the analytical reference relative to the Standard baseline is reported in our response to Reviewer VdRD Q3. Here we report only the comparison with other baselines, which is not covered there. On ETTh1 (8 test origins, 100 trajectories per origin, A800 80GB):
For detailed analyses of runtime, memory, and their balance with accuracy, please refer to Section 4.3 and Fig. 6 in the main text.
Q5: Yes, for convergence speed and accuracy at a fixed budget. We added convergence experiments in Appendix C tracking the training velocity MSE over the first 2,000 updates for MiraDiff and the Standard baseline under the same architecture and training setup on all eight datasets. The Standard baseline starts with a training MSE near 1.0 to 1.1, whereas MiraDiff starts at roughly 0.33 to 0.60 and converges within the first few hundred updates while maintaining lower or comparable loss throughout. Under a fixed budget (an identical backbone with 305,878 parameters trained for 2,000 updates on Weather), MiraDiff reduces the final validation CRPS from 0.2935 to 0.2229, a 24.1% improvement. Regarding reduced model capacity, preliminary ablations across model sizes show no significant difference in relative capacity sensitivity between MiraDiff and Standard. For training stability, the variance across the 5 independent training seeds is low on all datasets. Reduced training data remains untested. We expect the frozen statistical reference to be relatively more beneficial when data is scarce, since it provides local structure that does not depend on the amount of neural training data.