acceptodds
Under review as a conference paper at ICLR 2027

Mixture Reference Assisted Diffusion for Probabilistic Time Series Forecasting

Abstract

Probabilistic time series forecasting is essential for simulating future states and supporting decision making. However, real world time series often exhibit temporal heterogeneity, where local statistical patterns shift across time. While diffusion models provide a flexible framework for capturing predictive uncertainty, conventional formulations rely on neural networks to learn these varying local patterns implicitly, complicating the reverse denoising process. To address this limitation, we introduce Mixture Reference Assisted Diffusion (MiraDiff), a framework that decouples conditional denoising into an analytical local statistical reference and a learnable neural correction. Specifically, MiraDiff first fits a Gaussian mixture to training segments to capture recurring local patterns. This frozen reference supplies closed form clean signal estimates across diffusion timesteps, while a neural network conditioned on observed history learns residual corrections to capture long range contexts and cross variable dependencies. Extensive experiments across eight real world datasets demonstrate that MiraDiff delivers consistently competitive probabilistic forecasting performance on both distribution calibration and temporal dependency fidelity. Further ablation studies confirm that introducing closed form local statistics provides an effective statistical inductive bias, substantially alleviating the learning burden on the neural network.

Primary area:
learning on time series and dynamical systems
Keywords:
Probabilistic time series forecasting, Diffusion models, Temporal heterogeneity, Gaussian mixture models, Statistical inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bLU9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MiraDiff, a probabilistic time series forecasting method that combines an analytical local statistical reference with a neural correction network for diffusion-based forecasting. A gaussian mixture is fit once to non-overlapping segments which captures recurrant local patterns and this module is frozen. Then at each diffusion step, gaussian conditioning gives a closed form posterior mean estimating the clean segment from noisy observation. Experiments on 8 datasets report the lowest CRPS among the compared methods across all datasets. Also ablations of hierarchical reference prior configurations is provided with CRPS and Temporal Variogram Score (VS) for all 8 datasets.

Critical strengths
  1. Strong empirical results (Table 1) and well motivated theory, use of Gaussian mixture posterior inference to construct an analytical denoising reference is interesting. Equations (3)–(9) provide a clear formulation that integrates with standard diffusion training and sampling without modifying the forward process.
  2. The Standard, Diagonal, Single, and Mixture comparisons are useful and show how different reference configurations affect the results.
Critical weaknesses
  1. The evaluation only uses 32 test forecast origins and single input window and prediction window (L=168 and T=192) per dataset, which may not be enough to prove proposed method is better. Also the paper does not report statistical significance tests, mainly the paper mentions, "External baselines retain their original training and sampling settings." i believe this may not be a fair comparison, with my experience in time series, results might change by significant amount when trained with different settings, which questions the fairness of main results of paper (Table 1).
  2. The paper claims that the analytical reference captures local patterns while the neural residual focuses on global dependencies. However, the paper does not provide enough evidence to show that the neural residual actually learns global dependencies, additional ablations and analyses are needed to demonstrate that the two components actually capture their intended dependencies.
Questions for authors
  1. Were all baselines trained and evaluated with the same test split, and the same normalization?
  2. How was K selected ? Can you report Mixture with a fixed K across all datasets?
  3. Why are only 32 forecast origins used per dataset? How are these origins selected, and do the improvements persist with broader test coverage and multiple random seeds?
  4. What are the additional fitting cost, sampling latency, GPU memory requirements, and overall training costs (with parameters) compared with the Standard reference and leading baselines?
  5. Does the analytical reference improve convergence speed, training stability, or performance under reduced model capacity or limited training data?
2 comments

Q1: All methods are evaluated on the exact same test split and under the same normalization. Each dataset is split chronologically (6/2/2 for ETT and 7/1/2 for the others), and the 256 test origins are spaced uniformly across the test split, so every method is scored on identical inputs with identical metrics. Channelwise means and standard deviations are computed on the training split only and applied unchanged to the validation and test sets for all methods. External baselines keep their original training and sampling settings, which is the standard convention for comparability in this benchmark literature, while the controlled comparison in Table 3 (Standard, Diagonal, Single, Mixture) shares the same architecture, training schedule, and diffusion setup, isolating the effect of the reference itself.

Q2: K is selected by minimizing validation CRPS before any test evaluation. For each candidate K from 1 to 8, we fit the mixture reference and train the full model, and the K with the lowest validation CRPS is kept (Fig. D.2, marked by red stars; selected values in Table B.1). Under this protocol, seven of eight datasets select K greater than or equal to 2, and ETTm2 is the only dataset selecting K = 1. The final evaluation under the selected configuration is run across the 5 independent training seeds. Regarding a fixed K, Fig. D.2 already reports validation CRPS for every fixed K from 1 to 8 on all eight datasets. The curves show that performance plateaus once K reaches 6, so a single fixed value such as K = 8 is near optimal on heterogeneous datasets while remaining close to optimal on simpler ones like ETTm2, where K = 1 and K = 2 give nearly identical results. When selecting K separately for each dataset is inconvenient, we therefore recommend a fixed K of 6 to 8 as a robust default.

Q3: The original 32 origins were spaced uniformly across the chronological test split of each dataset, so they spanned the entire test period, though sparsely. To further verify that the conclusions are robust to broader coverage, we expanded the main benchmark (Tables 1 and 2) to 256 uniformly spaced test origins per dataset and repeated the evaluation across 5 independent training seeds, with all methods evaluated on the exact same origins. All main conclusions remain consistent with low variance across seeds. In Table 1, MiraDiff achieves the best CRPS on 7 out of 8 datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic) and the second best on Solar. In Table 2, MiraDiff achieves the best QICE on 4 datasets (ETTm1, Weather, ECL, and Solar) and the second best on 3 datasets (ETTh1, ETTm2, and Traffic).

Q4: The overhead of the analytical reference relative to the Standard baseline is reported in our response to Reviewer VdRD Q3. Here we report only the comparison with other baselines, which is not covered there. On ETTh1 (8 test origins, 100 trajectories per origin, A800 80GB):

MethodSampling (s/origin)Peak Mem (MiB)Params (M)
MiraDiff10.08113.480.306
TimeGrad77.3137.350.188
CSDI93.9249.630.610
TimeDiff92.9048.193.238
DiffusionTS110.54719.762.312
TMDM1.18107.559.068
NsDiff1.74100.539.468
D3VAE1.71838.6516.139
SSSD9.161115.7629.835
TSFlow3.23373.620.187
MG-TSD149.4339.150.340

For detailed analyses of runtime, memory, and their balance with accuracy, please refer to Section 4.3 and Fig. 6 in the main text.

Q5: Yes, for convergence speed and accuracy at a fixed budget. We added convergence experiments in Appendix C tracking the training velocity MSE over the first 2,000 updates for MiraDiff and the Standard baseline under the same architecture and training setup on all eight datasets. The Standard baseline starts with a training MSE near 1.0 to 1.1, whereas MiraDiff starts at roughly 0.33 to 0.60 and converges within the first few hundred updates while maintaining lower or comparable loss throughout. Under a fixed budget (an identical backbone with 305,878 parameters trained for 2,000 updates on Weather), MiraDiff reduces the final validation CRPS from 0.2935 to 0.2229, a 24.1% improvement. Regarding reduced model capacity, preliminary ablations across model sizes show no significant difference in relative capacity sensitivity between MiraDiff and Standard. For training stability, the variance across the 5 independent training seeds is low on all datasets. Reduced training data remains untested. We expect the frozen statistical reference to be relatively more beneficial when data is scarce, since it provides local structure that does not depend on the amount of neural training data.

Reviewer_bLU9OpenReview ↗

Response to Authors rebuttal

I thank the authors for the rebuttal, which addressed some of my concerns. However, some concerns remain open:

  1. The issue of a single input and prediction window (L=168, T=192) is still open. The revised paper still uses only L=168 and T=192. Is the same setting used for all baselines, and why is only one setting shown?
  2. In Table 1, considering the standard deviations, MiraDiff’s best results on Traffic, ETTm1, ETTm2, and ETTh2 are not clear wins (e.g., ETTm2: 0.276±0.016 vs. NsDiff 0.280±0.033) over the second-best methods, which makes the proposed model clearly better in only 3/8 datasets.

Considering the rebuttal, I keep my score.

Reviewer_6JmDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes MiraDiff, a probabilistic time series forecasting framework that decomposes conditional diffusion denoising into an analytical reference and a learned residual correction. A Gaussian mixture model fitted to training segments provides closed-form denoising estimates at different noise levels. A history-conditioned neural network then learns the correction needed to account for information not captured by this frozen local reference. The paper derives the residual parameterization and evaluates the method on eight datasets using CRPS, QICE, and temporal variogram scores. Controlled ablations compare no reference, a diagonal Gaussian, a single full-covariance Gaussian, and a Gaussian mixture.

Critical strengths
  1. The frozen mixture reference provides an interpretable denoising component, while the neural correction retains the ability to use forecasting context. The derivation explains the role of the residual rather than introducing it only as an architectural heuristic. This makes the approach a potentially useful design pattern for forecasting diffusion models.

  2. The results support the usefulness of covariance structure and, in many settings, mixtures. Reporting temporal variogram scores also provides evidence beyond marginal forecast accuracy.

Critical weaknesses
  1. The claimed reduction in optimization burden is stronger than the evidence provided. Section 3.2 describes a distinct optimization advantage and a reallocation of a capacity-constrained network's learning budget. The population identity for the optimal residual establishes representational equivalence, but does not establish an optimization or finite-capacity advantage. Table 2 demonstrates a useful inductive bias, yet does not directly measure convergence, the capacity needed to reach a given accuracy, or total cost including reference fitting and evaluation. This distinction matters because reduced learning burden is a central explanation for the method.

  2. The paper supports the value of structured covariance more directly than a general claim that a mixture is necessary. A clearer account of when the extra mixture complexity is beneficial would sharpen the contribution.

  3. The temporal variogram results are useful evidence, but the main evaluation does not separately establish cross-channel fidelity or continuity across reference-segment boundaries. This limits the strength of the global trajectory-quality interpretation.

  4. The reference-family ablation does not show whether the correction network can retain the benefit in windows poorly represented by the training mixture. This is a scope limitation for the proposed division between local statistical modeling and contextual correction, rather than evidence that a fixed prior is inherently unsuitable.

Questions for authors
  1. Can you report paired uncertainty for the principal comparisons in Tables 1–2, distinguishing variation across training runs from variation across forecast origins? Please clarify the common evaluation and model-selection protocol, particularly for the close ETTm2 result.

  2. At a fixed neural capacity or total training budget, does the analytical reference improve convergence or final accuracy? One representative comparison including mixture-fitting and reference-evaluation costs would directly test the explanation in Section 3.2.

  3. What determines the choice between a single Gaussian and multiple components? How should readers interpret the ETTm2 CRPS–variogram tradeoff, and were all reference-family choices made using validation data?

  4. Do the generated samples show better cross-channel dependence or continuity across segment boundaries? A focused analysis of the existing samples on one multivariate dataset would help connect the evaluation to the role assigned to the residual network.

  5. Does the advantage persist for test windows with relatively low likelihood under the fitted reference?

2 comments

Q1: We expanded the main benchmark (Tables 1 and 2) from 32 to 256 uniformly spaced forecast origins across 5 independent training seeds. Since all methods are evaluated on the exact same 256 test origins, the reported mean ± std directly captures variation across training runs after averaging over the shared origins. All datasets are split chronologically (6/2/2 for ETT and 7/1/2 for the others), and all models share the same preprocessing, where channelwise means and standard deviations computed on the training split are applied to the validation and test sets. The number of mixture components K is selected by minimizing validation CRPS (Appendix D). ETTm2 is the only dataset selecting K=1, which accounts for its close performance.

Q2: The analytical reference improves both training convergence and final predictive accuracy under a fixed neural capacity and training budget. Under an identical neural backbone with 305,878 parameters trained for 2,000 updates on Weather, MiraDiff reduces the final validation CRPS from 0.2935 (Standard) to 0.2229, a 24.1% improvement. MiraDiff also converges faster. Its training velocity MSE starts at 0.39 versus 1.08 for Standard, and it approaches its final loss within the first few hundred updates (curves for all eight datasets in Appendix C). Regarding computational overhead, fitting and evaluating the analytical reference slightly increases the total training and inference time, but this overhead remains moderate (total training time increases by only 1.2% and inference latency by 10.6% with negligible memory change). For the full breakdown, please refer to our response to Reviewer VdRD Q3.

Q3: Yes, all reference family choices and the number of mixture components K were determined strictly on validation data. In our experiments, K is selected by minimizing generation CRPS on the validation set, where seven of eight benchmarks favor K ≥ 2, with ETTm2 being the only case selecting K = 1 (Figure D.2).

On the ETTm2 CRPS–variogram tradeoff (Table 3), CRPS measures marginal accuracy at each timestamp, while the variogram score measures whether temporal increments along trajectories follow realistic dynamics. A single Gaussian fits ETTm2's homogeneous marginals slightly better, while mixtures capture composite local patterns and yield more coherent trajectories (better variogram scores)

Q4: Yes, we conducted a focused analysis on datasets to assess the specific role of the residual network, with full details and metrics now added to Section 4.3 and Appendix E.

For boundary continuity, we compare generated samples from the full model against samples in which the neural correction is gradually removed at sampling time, so that the statistical reference alone performs the denoising under identical noise. The reference only samples are accurate within each segment but show a clear discontinuity spike at segment boundaries, because segments are denoised independently. Restoring the correction smooths these transitions, cutting the boundary transition error by 28.7% in the CRPS of consecutive changes and by 54.1% in the variogram score (Fig.5). The same analysis is now extended to six additional datasets (Fig. E.1).

For cross-channel dependence, we compute the Pearson correlation between each pair of channels in the generated samples and compare the correlation matrix with the ground truth. Starting from the nearly diagonal matrix produced by the reference alone, realistic inter-channel structure progressively emerges as the neural correction is gradually added. With the full model, the correlation matrix closely matches the ground truth, confirming that the cross-channel dependence is learned by the neural residual (Fig. E.2).

Q5: The advantage is not consistently retained on low-likelihood test windows. We conducted a focused analysis on ETTh1, Weather, and Electricity. For each dataset, we selected the lowest-likelihood 25% of 256 test windows and compared 2,000 update Standard and MiraDiff models using 100 samples per window. Compared with windows having intermediate reference likelihood (the middle 50%), MiraDiff's CRPS increases by 11.30% on ETTh1, 12.99% on Weather, and 15.80% on Electricity on the lowest-likelihood windows. On these windows, MiraDiff's accuracy drops to a level close to the Standard baseline, so the advantage observed over the full test set largely disappears. This is consistent with the design of the method. The GMM is fitted to training data and then frozen. If distribution shift makes its learned statistics less representative of the test windows, the analytical reference becomes less accurate, and the neural residual may not fully compensate for the mismatch.

Reviewer_6JmDOpenReview ↗

Thank you for the additional experiments.

Thank you for the additional experiments. The comparison answers an important part of my question. However, as you report in Q5, MiraDiff's performance degenerates when the test examples look different from the patterns it learned during training, casting concerns on the method.

Overall, I think the idea is interesting, but the method formulation and performance is still not convincing (e.g. the hand-made noisy matrix in Eq.(4)). Therefore, I still rate this paper under ICLR acceptance.

Reviewer_VdRDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MiraDiff, a reference-assisted diffusion framework for probabilistic time series forecasting that addresses temporal heterogeneity by decoupling local statistical modeling from conditional neural denoising. The authors first fit a Gaussian mixture model (GMM) to historical time series segments to capture recurring local patterns and temporal correlations. During diffusion, the frozen GMM provides analytical clean-signal estimates through closed-form posterior inference, which are converted into reference velocities. A history-conditioned neural network then learns residual velocity corrections to capture dependencies beyond local segments. The resulting model retains the standard Gaussian diffusion framework.

Experiments on eight real-world datasets demonstrate competitive probabilistic forecasting performance in terms of CRPS and calibration. Additional hierarchical ablation studies investigate the contributions of marginal statistics, intra-segment covariance, and Gaussian mixture modeling to forecasting accuracy and temporal dependency fidelity.

Critical strengths

S1. The paper addresses the challenge of modeling heterogeneous local temporal patterns in probabilistic time series forecasting. Instead of relying entirely on neural networks to learn complex denoising functions, MiraDiff introduces an analytical statistical reference and a learnable conditional correction. This provides an interesting and well-motivated approach to incorporating explicit local statistical knowledge into diffusion-based forecasting.

S2. The proposed Gaussian mixture reference provides closed-form posterior clean-signal estimates under Gaussian corruption, which are subsequently incorporated into velocity prediction. The analytical reference and neural correction are combined without modifying the standard diffusion process. Moreover, the interpretation of the optimal correction as a score difference relative to the reference distribution provides useful theoretical insight into the proposed decomposition.

S3. The ablation experiments systematically compare standard diffusion, diagonal Gaussian references, a single full-covariance Gaussian reference, and Gaussian mixture references.

Critical weaknesses

Although the paper reports strong probabilistic forecasting results across eight datasets, Appendix B.3 indicates that the main evaluations use only 32 test origins per dataset. Given the substantially larger test sets, the representativeness of these selected origins and the sensitivity of the reported results to their selection remain unclear.

Questions for authors

Q1. Appendix B.3 states that the main evaluations use 32 forecasting origins per dataset, with 100 generated samples per origin. Could the authors clarify how these origins are selected and whether all baseline methods are evaluated using exactly the same forecasting origins and scoring protocols? Additionally, would the main performance conclusions remain consistent when evaluating a larger number of test origins? Such clarification would help assess the representativeness and reliability of the reported results.

Q2. A central motivation of MiraDiff is to alleviate the learning burden on the neural network by providing an analytical local statistical reference. The hierarchical ablation studies demonstrate clear improvements in forecasting performance, which support the effectiveness of the reference-assisted design. However, these results do not directly establish whether the reference simplifies neural optimization or reduces the required model capacity. Could the authors provide further evidence, such as training convergence comparisons or network-capacity ablations, to clarify this benefit?

Q3. Although the Gaussian mixture parameters are fitted and frozen before neural training, the analytical posterior reference must still be evaluated at each reverse diffusion step. Could the authors provide quantitative comparisons of training time, inference latency, and memory consumption between standard diffusion and the proposed reference-assisted configurations? Such comparisons would help assess the practical accuracy–efficiency trade-off introduced by the analytical reference.

2 comments

Q1: All methods, including MiraDiff and all baselines, are evaluated on the exact same forecasting origins and under identical scoring protocols. These test origins are selected by spacing them uniformly across the chronological test split of each dataset. To thoroughly verify the representativeness and reliability of our results on a larger scale, we expanded the main benchmark (Tables 1 and 2 in Section 4.1) to 256 uniformly spaced test origins per dataset, generating 100 trajectories per origin across 5 independent training seeds. All main conclusions remain consistent with low variance across seeds. Specifically, in Table 1, MiraDiff achieves the best CRPS on 7 out of 8 datasets (ETTh1, ETTh2, ETTm1, ETTm2, Weather, ECL, and Traffic) and the second best on Solar. In Table 2, MiraDiff achieves the best QICE on 4 datasets (ETTm1, Weather, ECL, and Solar) and the second best on 3 datasets (ETTh1, ETTm2, and Traffic).

Q2: We added training convergence experiments in Appendix C (Figure C.1) tracking the training velocity MSE over the first 2,000 updates (averaged every 20 updates) across all eight datasets for MiraDiff and the Standard baseline under the exact same neural architecture and training setup. Across all eight datasets, the Standard baseline starts with a high training velocity MSE near 1.0 to 1.1, whereas MiraDiff begins substantially lower at roughly 0.33 to 0.60, converges noticeably faster within the first few hundred updates, and maintains lower or comparable training loss throughout all 2,000 updates. Regarding network capacity, we also conducted preliminary ablations across different model sizes, where we did not observe a significant difference in relative capacity sensitivity between MiraDiff and Standard, indicating that the primary empirical benefit of the statistical reference lies in accelerating training convergence and improving forecasting accuracy under the same network capacity.

Q3: While the analytical posterior reference is evaluated at each reverse diffusion step, it does not add trainable neural parameters or increase the number of sampling steps. We compare the training time, inference latency, and GPU memory consumption between the Standard diffusion baseline and MiraDiff under an identical backbone below (training and sampling timed on Weather over 2,000 updates and 32 shared origins with 4 trajectories and 100 steps each, and memory checked on ETTh1):

MetricStandardMiraDiffDifference
Offline GMM Fitting Time—1.841 s+1.841 s
Total Training Time469.883 s475.678 s+1.2%
Sampling Time7.989 s8.834 s+10.6%
Trainable Neural Parameters305,878305,878+0.0%
Peak Sampling GPU Memory327.344 MiB327.379 MiB+0.035 MiB

In addition, we included an overall inference efficiency comparison across all baselines in Section 4.3 (Figure 5) showing that MiraDiff achieves a favorable balance among sampling speed, memory footprint, and parameter count.

Thank you to the authors for the detailed rebuttal and substantial revisions. I appreciate the effort devoted to addressing my concerns and strengthening the experimental analysis.

The expanded evaluation with 256 test origins and five independent training seeds satisfactorily addresses my main concern regarding the representativeness and reliability of the original results. The additional convergence experiments also provide useful evidence that the analytical reference reduces the residual learning burden and improves optimization behavior, although a reduction in the required neural network capacity has not been established.

I also appreciate the quantitative analysis of computational overhead, which adequately addresses my efficiency-related question. Furthermore, the newly added diagnostics on segment-boundary continuity and cross-channel dependencies provide more direct support for the complementary roles of the analytical reference and neural correction.

Overall, I believe the revisions have satisfactorily addressed my major concerns and strengthened the experimental support for the proposed approach. I will maintain my rating, with increased confidence in my positive assessment of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revision of the original manuscript with all changes marked in red

We thank all reviewers for their constructive comments. We have revised the manuscript accordingly. To make the revision easy to inspect, all changes are highlighted in red in the revised PDF. Newly added or substantially revised sections are indicated by red section titles, and newly added or revised figures and tables are indicated by red captions. The main changes are:

  1. Tables 1 and 2: expanded the main benchmark from 32 to 256 uniformly spaced test origins per dataset, evaluated across 5 independent training seeds, with mean and std reported.

  2. Section 4.3: residual diagnostics and inference efficiency, including a controlled analysis of segment boundary continuity across residual weights (Fig. 5) and an overall inference efficiency comparison across all baselines (Fig. 6).

  3. Appendix B.3: revised the evaluation protocol description for clarity, including the number of test origins, sampling settings, and model selection procedure.

  4. New Appendix C: training convergence curves for MiraDiff and the Standard baseline on all eight datasets.

  5. New Appendix D: reference family analysis and K selection, including validation curves across K for all datasets (Fig. D.1 and D.2) and the selected values (Table B.1).

  6. New Appendix E: diagnostic analysis of the neural residual, including the boundary continuity analysis extended to six additional datasets (Fig. E.1) and the cross channel correlation analysis (Fig. E.2).

Detailed responses to each reviewer are given in the individual replies below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.