acceptodds
Under review as a conference paper at ICLR 2027

Mixtures of Localized Diffusion Processes for Time Series Forecasting

Abstract

In many real-world long-horizon forecasting tasks, we observe only a limited subset of the variables governing a high-dimensional system, yet must predict its future evolution from these incomplete measurements. Under such partial observation, some localized temporal and frequency components of the future can be predicted reliably from the observed history, while others retain substantial residual variation. We propose MeLD, an end-to-end forecasting framework that adapts stochastic modeling to this localized unresolved structure. MeLD constructs a localized time-frequency frame over the observed history and uses context-dependent routing to identify components that are well explained by the available observations and those with larger unresolved uncertainty. The resolved components provide the conditional structure of the forecast, while the unresolved components define the subspace on which diffusion is applied as the central stochastic modeling mechanism. Across standard long-horizon forecasting benchmarks, we show that stochastic variation is not uniformly distributed across these components, and that diffusion becomes substantially more effective when concentrated on the few that remain unresolved.

Primary area:
learning on time series and dynamical systems
Keywords:
unobserved dynamics, diffusion, probabilistic modeling, long-term time series forecasting, localized generative modeling

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qYDpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Mixtures of Localized Diffusion Processes (MeLD), a framework for time-series forecasting that separates history-resolved structure from unresolved stochastic variation. The method employs a learnable localized time-frequency decomposition, context-dependent routing, and selective residual diffusion. Motivated by the Mori–Zwanzig formulation of partially observed dynamical systems, the authors aim to focus stochastic modeling on uncertain components rather than applying diffusion to the entire forecast.

Experiments on standard long-term forecasting benchmarks report competitive improvements over deterministic and diffusion-based baselines.

Critical strengths
  1. Intuitive modeling framework.\textbf{Intuitive modeling framework.} Separating history-supported predictive structure from residual stochastic variation provides a reasonable motivation for selectively allocating diffusion modeling capacity.

  2. Strong reported forecasting performance.\textbf{Strong reported forecasting performance.} The method achieves favorable MSE, MAE, and CRPS results across standard forecasting benchmarks. The deterministic evaluations cover multiple prediction horizons, and the authors provide additional controlled baseline comparisons and statistical analyses.

Critical weaknesses
  1. Insufficient differentiation from related structured generative forecasting methods.\textbf{Insufficient differentiation from related structured generative forecasting methods.} The paper motivates its resolved/unresolved decomposition through Mori–Zwanzig theory, but does not adequately position this formulation relative to existing stochastic dynamical and state-space approaches. For instance, LLapDiff [1] already introduces a stochastic port-Hamiltonian formulation and locally linearized latent dynamics, separating structured mean evolution from stochastic excitation while employing history-conditioned modal representations. Although MeLD differs by introducing learned time-frequency routing and selective residual diffusion, the paper does not clearly establish how its proposed decomposition advances beyond these existing dynamical principles.

  2. Insufficient theoretical justification for identifying unresolved components.\textbf{Insufficient theoretical justification for identifying unresolved components.} The paper defines conditional unresolved energy through the variance remaining after conditioning on the observed history. However, its implemented component-selection score is the product of learned routing weights and the temporal variance of the deterministic anchor. The relationship between this heuristic score and the formally defined conditional unresolved energy is not established. A component with high deterministic variation need not exhibit high residual uncertainty, and vice versa.

  3. Mismatch between the partial-observation motivation and empirical evaluation.\textbf{Mismatch between the partial-observation motivation and empirical evaluation.} The paper motivates forecasting under partial observation and incomplete measurements, yet its main evaluations use standard regularly sampled forecasting benchmarks without explicitly controlling the degree of partial observation. While incomplete observation of an underlying dynamical state is not equivalent to missing entries in a measured time series, the current experiments do not directly examine how the method behaves as relevant state information becomes unavailable. The synthetic experiment validates recovery of deterministic and stochastic frequency bands, but does not systematically study different observation operators or levels of partial observability.

  4. Limited validation of localized stochastic forecasting across physical time and prediction horizons.\textbf{Limited validation of localized stochastic forecasting across physical time and prediction horizons.} Although the proposed framework emphasizes localized unresolved dynamics, the implemented forecast-side decomposition uses a single forecast-window segment, with diffusion selection primarily operating over frequency and channel components. Consequently, the extent to which the method adaptively localizes stochastic variation across different physical-time regions of the forecast remains unclear. Furthermore, while the deterministic comparisons cover horizons of 96, 192, 336, and 720, the main comparisons against stochastic baselines are restricted to a single prediction horizon.

  5. Limitations in probabilistic evaluation and baseline comparability.\textbf{Limitations in probabilistic evaluation and baseline comparability.} The main stochastic comparisons rely on previously published baseline results rather than a unified reproduction protocol, making it difficult to fully assess differences arising from sampling and evaluation settings. Moreover, CRPS and minimum-over-N error do not provide a complete assessment of the joint temporal dependence of generated trajectories. Since the proposed method explicitly aims to model structured stochastic variation, additional joint probabilistic metrics, such as energy score and variogram score, together with calibration diagnostics, would provide stronger evidence for the quality of the learned forecast distribution. The statistical comparisons against published baseline point estimates also do not account for uncertainty in those baseline results.

[1] You, Z., Zheng, J. & Cartlidge, J.. (2026). Latent Laplace Diffusion for Irregular Multivariate Time Series. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:149909-149935.

Questions for authors
  1. On Novelty: How does MeLD's resolved/unresolved decomposition claim differ fundamentally from LLapDiff's Port Hamiltonian-SDE formulation and related structured state-space methods?

  2. Routing justification: What theoretical or empirical evidence supports using anchor variance and routing weights as a proxy for conditional unresolved uncertainty?

  3. Empirical validation: Can the authors evaluate the robustness of selective diffusion under controlled partial observability and across different probabilistic forecasting horizons?

7 comments

Response to Reviewer qYDp

(W1, Q1) Positioning, novelty and advances beyond existing stochastic dynamical and state-space methods, like LLapDiff [1]. How does MeLD's resolved/unresolved decomposition claim differ fundamentally from LLapDiff's Port Hamiltonian-SDE formulation and related structured state-space methods?

We appreciate the Reviewer bringing this method to our attention and pointing us to this line of work. The revision cites LLapDiff and the structured state-space methods in Related Work. Having studied LLapDiff closely, we find that it and MeLD differ as follows. Positioning. MeLD is a framework for deciding where stochastic modeling is needed in a forecast. Given the context window, it identifies the components of the forecast window that the context leaves unresolved; generative modeling is then only applied to those. LLapDiff, latent SDEs and deep state-space models address a different question, namely how a latent state evolves under noise, and they generate the whole state stochastically. The two questions are complementary. We evaluate MeLD on regularly sampled benchmarks, and LLapDiff is designed for and evaluated on irregularly sampled series.

Novelty. Compared to existing generative forecasting methods, as main contributions we view the selection mechanism and the localized generative processing based on its outputs. The selection step has no counterpart in LLapDiff or in latent SDE and state-space models. Beyond the selection mechanism, the formulations of MelD and LLapDiff share only the generic form of a mean term plus a noise term. In LLapDiff the noise term belongs to a single latent state and is always present. In MeLD it stands for the degrees of freedom that the history does not resolve, so it is tied to specific components and is absent from the others.

Advances over existing work. In our work we show that generative modeling of selected components of a time-series context window leads to better forecasts, compared to the modeling of the entire signal. For synthetic time series with a dominant predictable band, diffusion on all bands leads to a CRPS of 0.324. This number is worse than the result of the deterministic anchor alone (0.206), because generative modeling adds variation where the anchor is already correct. Diffusion on the selected band leads to the CRPS of 0.156, matching the oracle’s performance. On some real-world datasets (ring-transport, ETTm1) the selected band likewise leads to an improvement, compared to all-band diffusion, while on the others (Weather, ETTm2) it matches it, at about one fifth of the sampling time.

DataL / HAnchorRandomSelected (κ=1)All bandsOracleTime
Synthetic, dominant periodic band*96 / 1920.2060.1790.1560.3240.1565×
Synthetic, ring transport (6 stations)96 / 1920.5800.4480.4330.4660.4324.7×
Weather96 / 960.2260.2010.1870.1880.1875.0×
Weather96 / 1920.2580.2330.2170.2180.2174.9×
ETTm296 / 960.2530.2070.2000.2000.2005.1×
ETTm1336 / 960.3370.2920.2880.2950.288–

(W2, Q2) The selection score is not related to the conditional unresolved energy. What evidence supports anchor variance and routing weights as a proxy.

To respond to the Reviewer’s comment, we:

-Refer to the original submission’s table 4 that empirically shows positive Spearman rank correlations between the (routing weight times anchor variance) and conditional energy;

-Introduce Claim A.1 and Claim A.2 in Appendix section A.1 to show analytically that the Spearman rank correlations between the (routing weight times anchor variance) and conditional energy is positive;

Claim A.1 shows that the score, defined as routing weight times anchor variance, is positively rank-correlated with the residual energy, and gives the condition under which this holds.

Claim A.2 shows when the learned score is better than variance alone, when the router's weight increases with a component's noise-to-signal ratio.

(W3) The evaluations do not control the degree of partial observation, and the synthetic experiment does not study different observation operators or levels of observability.

To evaluate selective diffusion under controlled partial observability, we add an experiment in which the number of observed stations is varied.

To this end, we generated synthetic time series by combining a large predictable sinusoid (period 96) with hidden signals and white noise. The hidden signal was defined as s(t) = ρ s(t − 96) + (1 − ρ²)^{1/2} e(t) with an innovation e that was never observed. This way, time-series forecast window has the unresolved fraction of 1 − (ρ² + ρ⁴)/2. We found that, across the unresolved fractions (degrees of partial observation), MeLD has shown the strongest performance compared to the alternatives. The gain of MeLD over the anchor grows with the unresolved fraction, from 0.036 CRPS at ρ = 0.95 to 0.153 at ρ = 0.30, because the less the history determines, the more of the forecast error is unresolved variation that only a generative model can represent, and MeLD places that model on the components that carry it, which is why its agreement with the oracle rises from 48% to 90% at the same time.

ρ0.950.800.600.30
Unresolved fraction0.140.480.760.95
Anchor only0.2200.3870.4900.561
Oracle band (κ=1)0.1880.2880.3600.413
All bands (κ=5)0.2440.3150.3690.412
MeLD, selected κ0.184 (κ=1)0.290 (κ=1)0.360 (κ=1)0.408 (κ=2)
Top-1 = oracle48%69%78%90%

(W4) Stochastic variation is not localized in time within the forecast window, and only one horizon is used for stochastic comparisons.

To evaluate the role of localized variation across physical time, we referred to the Ablations table (Table 5) of the submitted manuscript. The ablation “w/o forecast seg” compares the forecast made on localized vs. global time segments. We observed that, across 6 considered datasets, the use of localized time segments has consistently led to better forecasting.

To extend the probabilistic evaluation across the forecast horizons, we trained MeLD on horizons of 96, 192, 336, and 720. In each case, to evaluate the contribution of stochastic modeling, we evaluated the CRPS of the (deterministic) anchor forecast and (deterministic + stochastic) full MeLD forecast. We found that, in each considered case, full MeLD results were better than the anchor results, suggesting that our proposed method is efficient across forecasting horizons. We have additionally compared the results of our model, across horizons, to TMDM and found that our model outperforms it across all the horizons, further strengthening our results.

DatasetModelH=96H=192H=336H=720
ExchangeTMDM0.2000.3060.4140.714
Anchor0.1980.2930.4030.792
MeLD (κ=1)0.1580.2450.3490.690
ETTm2TMDM0.2640.2940.3550.486
Anchor0.2530.2930.3340.398
MeLD (κ=1)0.2000.2340.2680.324
WeatherTMDM0.2020.2390.2750.333
Anchor0.2260.2580.2950.344
MeLD (κ=1)0.1870.2170.2520.298

(W5) Limitations in probabilistic evaluation and baseline compatibility.

The main stochastic comparisons rely on previously published baseline results rather than a unified reproduction protocol, making it difficult to fully assess differences arising from sampling and evaluation settings.

To address this point, we’ve implemented a unified evaluation protocol that is the same for MeLD and the baselines. The published baseline numbers in Table 2 use the same datasets, splits, normalization, context length and horizon as our runs (table below).

CRPS and minimum-over-N error do not provide a complete assessment of the joint temporal dependence of generated trajectories. Since the proposed method explicitly aims to model structured stochastic variation, additional joint probabilistic metrics, such as energy score and variogram score. Together with calibration diagnostics, would provide stronger evidence for the quality of the learned forecast distribution.

To address this point, we’ve evaluated the suggested metrics (table below). We found that MeLD consistently shows the best performance across the requested metrics. We thank the Reviewer for this suggestion that has further strengthened our results.

DatasetModelCRPSESVSCov. 50%Cov. 90%
ExchangeTMDM0.30615.40125.20.360.73
Anchor only0.29315.4953.2––
MeLD (κ=1)0.22911.48118.60.510.80
ETTm2TMDM (re-run)0.29415.22119.10.430.73
Anchor only0.29616.0079.9––
MeLD (κ=1)0.23612.18102.90.480.76
WeatherTMDM0.23923.59115.10.480.70
Anchor only0.25829.0557.5––
MeLD (κ=1)0.21622.13112.30.490.71
Reviewer_7jdQOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MeLD, a diffusion-based framework for time series probabilistic forecasting. The method employs a deterministic–stochastic decomposition to separate the predictable structural parts from the unresolved residual, and performs diffusion modeling only on the latter. The paper also introduces importance weighting to extracts time-varying intervariable interactions from the predictions and as conditions. Experimental results on multiple dataset demonstrate the effectiveness of the proposed method.

Critical strengths
  • Long-horizon prediction is a crucial problem in time series analysis. It makes sense to capture trend and seasonal components and then refine it further using a DM.
  • Limiting the scope of diffusion is also reasonable from an explainability standpoint.
  • The experiments are comprehensive and systematic, covering 8 datasets, numerous state-of-the-art baselines, ablation studies, and analyses of backbone generality.
Critical weaknesses

Writing:

  • The paper is poorly written due to insufficient explanations and overly compressed mathematical formulations. For instance, Definitions 1 and 2 introduce the localized time-frequency frame and its associated operator with minimal intuition of how they are constructed. This should be its own Appendix section rather than compressed to a definition.
  • "MeLD ... uses the best available approaches to model each structure." is an overclaim and an unprofessional statement that is discouraged in academic writing.

Theory:

  • Theorem 1 establishes an asymptotic relationship between the MoN error and the conditional distribution of future time steps. However, this result is irrelevant to the proposed method and model architecture. Its contribution to the paper is therefore minimal.
  • Moreover, MoN is a weird metric and should not be used here. It evaluates the best prediction among N generations rather than the quality of a typical forecast. Thus, it is of limited practical relevance.

Implementation:

  • Important architectural details, particularly the routing network and denoising network, remain unclear. The paper doesn't adequately explain their structures or how the conditioning is incorporated.
  • The proposed importance score in Equation 5 is not clearly connected to the conditional energy in Definition 3. The variance of the deterministic anchor doesn't always reflect the uncertainty of its components. For example, a highly predictable periodic component can have large variance but low conditional uncertainty.

Novelty:

  • The novelty of the proposed method is relatively limited, as the overall framework is closely related to the existing “deterministic + residual diffusion” paradigm. MeLD extends this paradigm through frequency decomposition and adaptive component selection, but the paper does not sufficiently establish why these additional mechanisms are necessary or superior to simpler fixed decomposition strategies.
Questions for authors

See weaknesses

5 comments

Response to Reviewer 7jdQ

(W1, W2) Writing. “Definitions 1 and 2 introduce the localized time-frequency frame and its associated operator with minimal intuition of how they are constructed. This should be its own Appendix section rather than compressed to a definition.”

We agree with the Reviewer that an intuitive, high-level overview of these definitions is important for conveying the paper's message to the broad ICLR community. The submitted manuscript gives this overview in two places. The Introduction explains the resolved-versus-unresolved view in plain terms, with the Mori–Zwanzig analogy and Figure 1, and the paragraph "Problem setup and MeLD overview" at the start of Section 3 introduces every object of Definitions 1 to 3 in words, with Figure 2 showing the decomposition on real data and Algorithm 1 giving the full pipeline, before the formal definitions follow. To address the Reviewer’s comment and improve the readability of the paper, we back-referenced the definitions to this high-level overview section.

"MeLD ... uses the best available approaches to model each structure." is an overclaim and an unprofessional statement that is discouraged in academic writing.

While the statement is fully correct as written (we’ve gone through literature to see the available approaches and compared them in our framework to determine the best one among them), we are not partial to this wording and have removed it following the Reviewer’s suggestion.

(W3, W4) Theory Theorem 1 establishes an asymptotic relationship between the MoN error and the conditional distribution of future time steps. However, this result is irrelevant to the proposed method and model architecture. Its contribution to the paper is therefore minimal.

We appreciate the Reviewer’s comments, as they highlight the parts of the paper that could benefit from additional explanations. In revision, we add the following explanation as to why the MoN metric is relevant and important (which, we agree, we should have had in the original submission).

Different metrics capture different properties of stochastic forecasts. Currently, time-series forecasting papers mostly report (and compare) the CRPS, a distributional metric of stochastic forecast quality, represented by the signed integral of the generative model’s cumulative density function around the actual observation. While CRPS is a well-substantiated metric, its integration approach gives a single-number overall evaluation of the entire forecast, preventing further analysis of what drives the discrepancy between the observed and forecasted data. The limitation of the single-number estimate of stochastic forecast quality has motivated the field to use additional metrics to have a more comprehensive view of the forecast [some of these metrics are mentioned by Reviewer qYDp], however, these metrics still did not allow to focus on specific parts of the forecast to see which ones the model succeeds or fails on. The MoN metric solves this problem. [as we describe in our response below].

Moreover, MoN is a weird metric and should not be used here. It evaluates the best prediction among N generations rather than the quality of a typical forecast. Thus, it is of limited practical relevance.

Here we continue the response that we’ve started with the question above. The MoN metric comes from trajectory forecasting in computer vision, where the best-of-N protocol introduced by Social GAN (Gupta et al., 2018), following DESIRE (Lee et al., 2017), is a well-established metric supported theoretically and successful experimentally, for example as minimum ADE and FDE over N samples in Trajectron++ (Salzmann et al., 2020) and PECNet (Mangalam et al., 2020). Despite the deceiving simplicity, the metric captures relative distributional properties of forecasts: put simply, with a fixed large number of rollouts, a model that approximates the data’s generative distribution better will generate a trajectory that’s closer to the one observed in the data. In that sense, it could be used similarly to the CRPS, generating a single number characterizing the stochastic forecast quality. However, beyond CRPS, the MoN’s computation process makes it interpretable: best rollouts can be directly overlaid with the observed trajectory, highlighting its parts that are systematically poorly predicted, guiding the further development of the models.

Our contribution here is bringing the well-established MoN metric from the domain of computer vision to the domain of time-series forecasting. As it is a new domain, theoretical guarantees had to be provided for this metric, which we contribute in Theorem 1.

(W5) Implementation (1)

Important architectural details, particularly the routing network and denoising network, remain unclear. The paper doesn't adequately explain their structures or how the conditioning is incorporated.

Following the Reviewer’s comment, we detail the implementation of the routing network and the denoising network in a plain text.

Routing network gθg_\theta. For each context chunk, component and channel, dm,k,cd_{m,k,c} has seven entries, six statistics of the localized response (mean, standard deviation, maximum absolute value, mean absolute first difference, mean absolute second difference and the ratio of the maximum to the standard deviation) and one scalar from a two-layer map of the chunk index and its normalized position in the window. gθg_\theta is a two-layer MLP (7→32→17 \to 32 \to 1, LeakyReLU), and the softmax over components with temperature τ\tau gives wm,k,cw_{m,k,c}. The router receives its gradient through the anchor loss, since the weighted responses are the anchor's input.

Denoising network ε_θ. The denoiser acts on the anchor-normalized coefficients of the selected components and is one cross-attention block. The queries are the noisy coefficients of each (channel, component) series, projected from the horizon length to a hidden size H, plus a two-layer embedding of the diffusion step n and a learned embedding of the component index, which is how n and M enter. The keys and values are the history h, namely the routed responses of every context chunk projected from the chunk length to H, concatenated with the anchor coefficients u^anc of every component projected from the horizon length to H, which is how the history and the anchor enter. A multi-head cross-attention layer with a residual connection and layer normalization is followed by a two-layer decoder from H back to the horizon length, which outputs the noise estimate. Components outside the selected set are returned at the anchor. The hidden size H, the number of heads, the number of diffusion steps and the sampler used at test time are added to Table A4 in the revision, and the code and the configuration of every reported run will be released.

(W6) Implementation (2)

The proposed importance score in Equation 5 is not clearly connected to the conditional energy in Definition 3. The variance of the deterministic anchor doesn't always reflect the uncertainty of its components.

To respond to the Reviewer’s comment, we:

-Refer to the original submission’s table 4 that empirically shows positive Spearman rank correlations between the (routing weight times anchor variance) and conditional energy;

-Introduce Claim A.1 and Claim A.2 in Appendix section A.1 to show analytically that the Spearman rank correlations between the (routing weight times anchor variance) and conditional energy is positive;

Claim A.1 shows that the score, defined as routing weight times anchor variance, is positively rank-correlated with the residual energy, and gives the condition under which this holds.

Claim A.2 shows when the learned score is better than variance alone, when the router's weight increases with a component's noise-to-signal ratio.

A highly predictable periodic component can have large variance but low conditional uncertainty.

To respond to the Reviewer’s comment, we added an experiment showing that MeLD successfully classifies as resolved periodic components with high variance and low uncertainty.

The example and the weight. We constructed the Reviewer's case, a large predictable periodic component with a weaker noise band, and ran MeLD on it (Table R4). The periodic band has the largest anchor variance (0.69) and almost no residual (0.0025), and the noise band has the largest residual (0.037). We show that MeLD matches the oracle performance, in line with our claims, theory, and previous empirical observations.

Selection ruleTop-1 = oracleρCRPSEnergy scoreVariogram score
Eq. 5 (MeLD)100%+0.890.1567.1248.2
Oracle (realized residual)100%10.1567.1348.4
Random––0.1788.2271.4
Anchor only––0.2069.4452.4
Variance only0%−0.850.26912.65165.1

(W7) Novelty

The overall framework is closely related to the existing “deterministic + residual diffusion” paradigm. MeLD extends this paradigm through frequency decomposition and adaptive component selection, but the paper does not sufficiently establish why these additional mechanisms are necessary or superior to simpler fixed decomposition strategies.

To address this response, we humbly point the Reviewer to the evidence already in the paper

Empirical evidence already in the paper.

  • Learnable decomposition leads to better forecasting than fixed decomposition, Table 5. Replacing the learnable localized decomposition with a fixed Haar or a fixed Gaussian basis, with everything else unchanged, raises the average MSE by 8.3% and 8.5% and the average MAE by 4.5% and 4.4% over five datasets (Table R5). The gap is largest on Exchange and ETTh2, whose spectral structure is irregular, and smallest on ETTm1 and ETTm2, where a fixed basis is already close to adequate.

  • Adaptive selection leads to better forecasting than no selection, Table 4. With the decomposition held fixed, diffusing the single component chosen by the selection score gives CRPS at or below an oracle that selects by the realized residual energy on all five datasets, while a random component at the same budget recovers only 65% of the gain over the anchor (average CRPS 0.261 selected, 0.276 random, 0.304 anchor).

  • Selection leads to better forecasting than diffusing everything, Table 3. Sweeping the budget κ from the anchor to all components shows that one selected component gives the lowest MoN on all five datasets and at least 88% of the best CRPS reduction on four of them, at about one fifth of the sampling cost of diffusing every component. Diffusing everything, which is what the existing deterministic plus residual diffusion methods do, is therefore not the best point on this curve.

To further substantiate this point per Reviewer’s request, we add the following theory to the updated version of the paper:

  • Spearman rank correlations between the (routing weight times anchor variance) and conditional energy is positive. This suggests that the novel architecture of MeLD correctly captures resolved vs. unresolved components of time-series.
Reviewer_x8AROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MELD, a time series forecasting framework that combines a deterministic forecast with selective residual diffusion. Motivated by reduced dynamics under partial observation, the method uses a learnable time-frequency decomposition and context-dependent routing to construct an anchor forecast. It then selects components using an importance score based on routing weights and anchor variability, and applies conditional diffusion to model their residuals. The paper evaluates both point and probabilistic forecasting on standard long-horizon benchmarks. Ablation studies examine the number of components receiving diffusion, the selection rule, and individual architectural choices. Synthetic experiments further investigate whether the learned decomposition captures the underlying frequency bands and whether the selection mechanism identifies components with stochastic variation.

Critical strengths

A major strength of this paper is its forecasting performance. Table 1 reports the best results in 15 of the 16 dataset–metric combinations averaged across prediction horizons, while Table 2 reports the best results in all 15 combinations evaluated against probabilistic baselines. This consistent performance across both point and distributional forecasting provides strong evidence of the method’s practical value.

Many of the questions I had about the design while reading the paper were addressed by the experiments. Table 3 demonstrates the benefit of adding diffusion to the anchor and examines the number of components receiving diffusion. Table 4 shows that the importance score is more effective than random selection or selection based only on variance, while Table 5 examines the contributions of individual components, including decomposition, routing, and forecast aggregation. These experiments help explain why the proposed design choices are useful, beyond demonstrating the performance of the complete model.

Critical weaknesses

Regarding novelty, I would have liked a clearer discussion of closely related work. D³U (ICLR 2025) also combines point forecasting with conditional residual diffusion, but I could not find a citation or discussion of this work in the paper. MELD differs in its dynamical-systems motivation and its use of a learnable localized decomposition and context-dependent routing to select components for diffusion. The usefulness of these design choices is also supported by the ablation studies. Explicitly discussing the similarities and differences would therefore help clarify MELD’s additional contribution beyond the existing combination of point forecasting and residual diffusion.

Questions for authors
  1. Could the authors distinguish more clearly between the general principle of separating and modeling structured uncertainty and MELD’s specific methodological contributions? Explaining how the learnable localized representation and context-dependent selection of components for diffusion extend existing residual diffusion approaches would help clarify the novelty claims throughout the paper.

  2. Table 3 already demonstrates the benefit of adding diffusion and examines the number of components receiving diffusion. Are results comparing the anchor alone with diffusion-enabled variants also available under the Table 2 setting of L=96 and H=192? If available, these results would help clarify the contribution of diffusion to the gains over probabilistic baselines.

1 comment

Response to Reviewer x8AR

(W1, Q1) Relation to D³U [1], and MeLD's specific contributions beyond the shared deterministic-plus-residual-diffusion principle. Please separate the general principle of modeling structured uncertainty from MeLD's specific contributions, the learnable localized representation and the context-dependent selection of components.

Thank you for pointing to D³U, which the revision cites and discusses in Related Work. We respond to the Reviewer’s comment in three parts, covering the shared principle between the two works; highlighting what MeLD adds as a novel forecasting framework; and concluding with a same-protocol comparison that shows the two methods are complementary.

Shared principle. The principle of separation of a predictable part from residual uncertainty is shared with D³U and with earlier decomposition-based forecasters. D³U realizes it by conditioning a diffusion model on a point forecaster and denoising the entire residual of the horizon with its PatchDN denoiser, while MeLD offers a mechanism for deciding which parts of that residual require stochastic modeling.

What MeLD adds. MeLD contributes two mechanisms that are pipelined with the denoiser and are independent of the denoiser’s architecture. The first mechanism is a learnable localized time-frequency representation in which the forecast and its residual are expressed. Table 5 shows that this mechanism leads to improvement in the forecasting performance, as replacing the learned decomposition with a fixed Haar or fixed Gaussian basis worsens the average MSE by about 8%. The second mechanism is a history-conditioned score that selects, for each input, which components are subjected to diffusion, while the rest of the components are forecasted deterministically, forming the deterministic anchor. Tables 3 and 4 show that these mechanisms also leads to improvements in the forecasting performance, since a single selected component recovers at least 88% of the best CRPS reduction on four of five datasets, closely matching the oracle that routes components to diffusion based on their realized residual energy.

Same-protocol comparison and complementarity. To show the complementarity of the methods, we add an experiment where we compare MeLD, D³U, and MeLD with D³U denoiser (PatchDN). We find that the best results are obtained with MeLD + D³U denoiser, outperforming both individual models. From this observation, we draw the following conclusion: While MeLD, with its diffusion module, already offers an advantage over other structured diffusion models due to its selection module, extending it with stronger diffusion modules further strengthens the results. We thank the Reviewer for this highly instrumental suggestion.

DatasetTMDM (paper)D³U (re-run)Anchor onlyMeLD (κ=1)MeLD + PatchDN
Exchange0.320.2310.2930.2450.228
ETTm20.370.2500.2960.2360.246
Electricity0.330.2000.2680.2240.197
Weather0.360.2030.2580.2170.200

(Q2) Are anchor-only versus diffusion results available under the Table 2 setting (L=96, H=192), to show how much of the gain over the probabilistic baselines comes from diffusion?

To address the Reviewer’s question, we’ve run the requested comparison. We observed that in all the considered cases diffusion has led to a noticeable improvement of the CRPS. We thank the Reviewer for this suggestion, as it further strengthens the claims of our paper.

DatasetTMDMAnchor onlyMeLD (κ=1)Reduction from diffusion
Exchange0.320.290.277%
ETTm20.370.290.2417%
Electricity0.330.260.2215%
Weather0.360.240.2113%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.