acceptodds
Under review as a conference paper at ICLR 2027

Dynamical State-Space Diffusion for Spatiotemporal Prediction

Abstract

Diffusion models for temporal prediction typically corrupt each future frame with independent noise, leaving temporal dependence entirely to the denoiser. We propose Dynamical State-Space Diffusion (DSSD), which learns the temporal structure of the corruption process itself. DSSD builds a temporal kernel from the damped oscillatory modes of a stable diagonal state-space model, whose learned decay rates and frequencies let the corruption capture multiple timescales and oscillatory correlations, while the denoiser architecture remains unchanged. Running this model on a finite clip, however, requires the state before the first frame, which is unobserved; starting from a zero state leaves the early frames with less noise than the diffusion schedule prescribes. We avoid this by using the Cholesky factor of the model's stationary covariance as the temporal operator. We prove that this operator is causal, exactly invertible, keeps each frame's noise variance equal to that of standard diffusion, and reduces to the identity at the clean end of the process, and that it is exactly the Kalman innovations form of the state-space model. The denoiser is trained to predict the temporally correlated noise, and sampling removes and reapplies the temporal mixing at each step. On scientific-field and robot-video datasets, DSSD outperforms diffusion baselines that share its denoiser, and the evolution of its learned modes during training shows that the temporal operator adapts to each dataset.

Primary area:
learning theory
Keywords:
diffusion models, spatiotemporal forecasting, state-space models, structured forward process, correlated noise

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_QHTvOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Dynamical State-Space Diffusion (DSSD), an extension of the paper Dynamical Diffusion (DyDiff). The authors regard the exponential moving average process in DyDiff as a special state-space operator and introduce a learnable operator with a small number of tunable parameters. The operator kernel is defined by damped modes to combine decay rates and oscillations, and can be obtained via Cholesky decomposition. Experiments on field prediction, video prediction and multivariate time-series forecasting show the effectiveness of the method.

Critical strengths
  1. DSSD proposes a unified framework to generalize the temporal mixing in diffusion prediction models. It introduces the operator framework to explain why DyDiff's temporal mixing and provides a more general learnable form to improve it with a small parameter overhead.
  2. The paper provides extensive experiments on multiple prediction tasks, supporting the effectiveness of the method. The visualizations of temporal modes are interesting, inspiring further investigations into powerful temporal modes.
Critical weaknesses
  1. The paper lacks of necessary ablation studies on key designs, such as the number of temporal modes, fixed versus learnable parameters, and oscillatory versus non-oscillatory modes. More ablation studies would help readers clearly find out which factors actually make a difference.
  2. The spatio-temproral experiments are limited to low resolutions and short prediction horizons, leaving the method’s performance at higher resolutions and longer horizons unclear. This limits the evidence for the effectiveness.
Questions for authors
  1. Does the proposed framework also apply to a flow-matching setup? Recently, there are more world models with flow matching setup.
  2. For for long-horizon prediction, how does DSSD’s practical computational cost compare with original diffusion? Does Cholesky decomposition remain computationally feasible?
5 comments
AuthorsOpenReview ↗

Response to W1.

W1. We ran the three ablations. We ran the additional experiments on the two datasets (Turbulence and SEVIR) we could cover within the rebuttal period. All runs use the setting of the main experiments and change only the operator.

Number of temporal modes (Turbulence). We trained DSSD with P∈{1,2,6,12}P\in\{1,2,6,12\} modes; P=6P=6 is the setting of the paper.

DatasetModesCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
TurbulenceP=1P=10.008150.005150.008490.97580.001680.012940.01741
P=2P=20.007810.004900.008340.97780.001500.012220.01641
P=6P=6 (paper)0.007560.004400.007570.97790.001470.011780.01582
P=12P=120.007590.004300.007530.97820.001470.011930.01596

A single mode is the worst on all seven metrics, and the scores saturate from P=6P=6: the differences between P=6P=6 and P=12P=12 are below 3% on every metric.

Fixed multimode operator and learned non-oscillatory modes. We ran the two variants on SEVIR: the six-mode operator frozen at its initialization, and a learned operator whose modes are constrained to be non-oscillatory (frequencies fixed to zero).

DatasetVariantCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
SEVIRDSSD (learned, oscillatory)3.9752.4215.9950.57072.12910.26314.585
frozen operator4.3472.8598.1340.44413.44012.06219.975
non-oscillatory5.6914.1317.9760.33893.65515.28536.686

The learned oscillatory operator is better than both variants on all seven metrics. Removing the oscillatory modes increases CRPS from 3.975 to 5.691 and RMSE from 14.59 to 36.69. Freezing the operator degrades every metric, most strongly the temporal variogram (2.129 vs. 3.440) and RMSE (14.59 vs. 19.98).

AuthorsOpenReview ↗

Response to W2.

W2. To evaluate longer horizons and longer context windows within the rebuttal period, we ran additional experiments on the Turbulence dataset varying the prediction horizon M∈{11,16,21}M \in \{11, 16, 21\} (with fixed context N=4N=4) and the context length N∈{4,8,12}N \in \{4, 8, 12\} (with fixed horizon M=11M=11). All runs use the same architecture, training budget, and sampling schedule as the main experiments.

(N,M)(N, M)MethodCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
(4,11)(4, 11) (paper)DPM0.011480.007510.010650.96410.002620.019200.02600
DyDiff0.011000.007520.011230.96740.002600.017910.02427
DSSD (Ours)0.007560.004400.007570.97790.001470.011780.01582
(4,16)(4, 16)DPM0.015040.012680.016120.96820.003230.023990.03342
DyDiff0.011010.007980.011450.97610.002510.017710.02406
DSSD (Ours)0.006910.004210.006400.98470.001100.010780.01460
(4,21)(4, 21)DPM0.017820.014070.018200.96180.004300.029550.04155
DyDiff0.011840.008240.011360.97480.002640.018960.02624
DSSD (Ours)0.008800.005930.008250.98160.001610.014190.02088

Prediction horizon scaling on Turbulence (M∈{11,16,21}M \in \{11, 16, 21\}).

(N,M)(N, M)MethodCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
(4,11)(4, 11) (paper)DPM0.011480.007510.010650.96410.002620.019200.02600
DyDiff0.011000.007520.011230.96740.002600.017910.02427
DSSD (Ours)0.007560.004400.007570.97790.001470.011780.01582
(8,11)(8, 11)DPM0.013840.011160.014300.97170.002830.022300.03016
DyDiff0.011440.007610.011150.97800.002320.018250.02435
DSSD (Ours)0.006680.003870.007010.98580.001190.010440.01394
(12,11)(12, 11)DPM0.011520.009220.013050.97580.002380.018360.02466
DyDiff0.009890.005990.009400.97880.002240.016010.02154
DSSD (Ours)0.006100.003290.006360.98670.001060.009530.01274

Context length scaling on Turbulence (N∈{4,8,12}N \in \{4, 8, 12\}).

Across all horizon and context configurations, DSSD outperforms both DPM and DyDiff on all seven metrics. Notably, as the prediction horizon doubles to M=21M=21, DPM's error grows sharply (RMSE 0.02600→0.041550.02600 \to 0.04155), while DSSD remains the best at every horizon, with the lowest error (RMSE 0.020880.02088 vs. 0.026240.02624 for DyDiff) and the best temporal consistency (temporal variogram 0.001610.00161 vs. 0.002640.00264 for DyDiff and 0.004300.00430 for DPM). Furthermore, DSSD effectively exploits longer context windows, monotonically improving all metrics as NN increases from 4 to 12. While training high-resolution backbones was not feasible within the rebuttal window, we note that DSSD's temporal operator acts purely along the time axis (L×LL \times L) independently of spatial resolution, making it directly compatible with higher-resolution architectures.

AuthorsOpenReview ↗

Response to Q1 and Q2.

Q1. Thank you for this question. DSSD can work with flow matching. The DSSD forward process is a linear interpolant between the data and the noise, zt=mt+Mtx+Ctϵ\mathbf{z}_t = \mathbf{m}_t + \mathbf{M}_t \mathbf{x} + \mathbf{C}_t \boldsymbol{\epsilon}, where the temporal operator enters through Mt\mathbf{M}_t and Ct\mathbf{C}_t. Proposition 4 already shows that our sampler follows a velocity field that transports the marginals of this interpolant. To move to flow matching, one can replace the variance-preserving path with a flow-matching path, such as the linear path of rectified flow, and tie the dynamics schedule γˉt\bar\gamma_t to that path in the same way. The model is then trained to predict the velocity m˙t+M˙tx+C˙tϵ\dot{\mathbf{m}}_t + \dot{\mathbf{M}}_t \mathbf{x} + \dot{\mathbf{C}}_t \boldsymbol{\epsilon}. The only difference from standard flow matching is that the velocity includes the time derivative of the temporal operator. The conditions that make the operator valid depend only on the dynamics schedule, not on the choice of path.

We also see world models as a promising application. Our operator is causal and can be extended one frame at a time: adding a frame does not change how earlier frames are corrupted, and the operator can be computed by a state-space recurrence. The corruption of each frame can therefore be defined without access to future frames, which fits world models that generate frames sequentially, often conditioned on actions. In our current experiments, the denoiser generates the future window jointly. Combining DSSD with frame-by-frame generation, as well as with flow matching, is future work.

Q2. The Cholesky decomposition remains feasible, and the overhead of DSSD over standard diffusion is small. In end-to-end terms, DSSD costs the same as standard diffusion: in Table 8 (one A100 GPU), the median time per training update of DSSD is within 5% of DPM on every dataset (e.g., 2.22 s vs. 2.22 s on SEVIR and 1.02 s vs. 1.06 s on Turbulence), and the reserved memory is identical. DSSD uses the same denoiser as standard diffusion and only adds the construction of the temporal operator. The operator is an L×LL\times L matrix along the time axis of the window, shared across all spatial locations and channels, so its cost does not depend on the spatial resolution. We measured the time to build the operator (mode computation and Cholesky factorization) together with its backward pass, for a batch of 16 noise levels on a single GPU (ms):

Window length LLMode computationCholeskyCholesky (forward + backward)Total (forward + backward)
152.840.060.458.41
2562.840.521.998.89
5122.961.7910.2915.06
10243.217.7869.2377.02

Up to L=256L = 256, the total cost stays nearly constant (8.4–8.9 ms), because it is dominated by the fixed cost of computing the modes. The Cholesky factorization grows as O(L3)O(L^3) and becomes the main cost only beyond L=256L = 256, reaching 77 ms at L=1024L = 1024. Part of this cost comes from computing the factorization in float64 for numerical stability, which is slow on consumer GPUs. In float32, the Cholesky factorization takes 1.7 ms at L=1024L = 1024. For the window lengths in our experiments (L≤48L \le 48), building the operator takes 8–11 ms, about 1% of a training step (0.8–2.2 s). Since the learned parameters are fixed at sampling time and the operator depends only on the noise level, we cache it for each noise level on the sampling grid, so it is computed only once per noise level and shared across all samples in a batch. The covariance is also well conditioned: its condition number stays below 75 up to L=1024L = 1024 and decreases after training (34 at L=1024L = 1024). For even longer windows, the state-space recurrence of Proposition 2 applies the operator at a cost linear in LL without forming the L×LL\times L matrix. We will add these measurements to the appendix.

Reviewer_wtRwOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This work introduces a method, DSSD, for spatiotemporal prediction based on a diffusion process that changes the geometry (with respect to standard diffusion) of the currupted signals seen by the denoiser. It does so by using a state-space model whose parameters are jointly learned with those of the denoiser. The new geometry allows the noise correlations to adapt to the data's temporal patterns. In particular, it supports both positive and negative temporal correlations that manifests in damped oscillatory modes; unlike baseline methods that the authors compare against. The methodology is applied to a wide range of datasets (with special focus on video prediction), showing consistent gains in probabilistic forecasts and temporal consistency.

Critical strengths

The authors present their method in a very easy-to-follow way. Although there are current methods that consider temporal noise correlations in diffusion models, the one presented in this work is novel, theoretically clean, and sounded. The authors show that, compared with standard diffusion, there is no information advantage from the temporal mixing at fixed noise levels. Despite this, the forecast skill provided by their method consistently give superior performance in the datasets considered.

Critical weaknesses
  • In certain comparisons in tables 1 and 2, it is hard to tell if there is actually a gain, since no measure of spread across seeds is used. Visually, the DPM and DSSD error fields (Fig 3a or Fig 6) look quite similar, which approximately seems to instantiate the equality in Proposition 5.

  • In the appendix, the authors apply their method to tabular time series on standard benchmarks. However, they don't compare their method with SOTA diffusion models tested on such benchmarks (e.g. NsDiff, TMDM).

Questions for authors
  • In sec 3.2, it is unclear what the input vector is: the authors say it is \bar{b}, however it seems to be e(\tau), since from the unrolled causal convolution, the kernel maps e to the output y. Is that correct?

  • How does your method compare to recent diffusion foreasters on tabular data?

  • Shouldn't section 5.5 provide a discussion of the main intuition behind the results? The learned temporal modes are what make DSSD different from DyDiff that only consider EMA decay.

4 comments
AuthorsOpenReview ↗

Response to W1, W2 and Q2.

W1. To measure the spread across seeds, we retrained all three methods on two datasets (SEVIR and Turbulence) over three seeds (42, 43 and 44) under the same setting as Table 1 (step 40k, same evaluation protocol):

DatasetMethodCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
SEVIRDPM4.264 ± 0.0952.810 ± 0.0957.474 ± 0.8580.4993 ± 0.04842.816 ± 0.46711.383 ± 0.60816.893 ± 2.092
DyDiff4.479 ± 0.1683.005 ± 0.1707.729 ± 0.4560.4096 ± 0.05063.529 ± 0.83912.559 ± 0.99123.723 ± 5.296
DSSD (Ours)4.031 ± 0.0552.501 ± 0.0836.072 ± 0.0670.5788 ± 0.00722.063 ± 0.06210.323 ± 0.08714.455 ± 0.171
TurbulenceDPM0.01168 ± 0.000330.00743 ± 0.000100.01075 ± 0.000120.9622 ± 0.00410.00282 ± 0.000330.01937 ± 0.000760.02620 ± 0.00098
DyDiff0.01109 ± 0.000120.00698 ± 0.000470.01075 ± 0.000580.9677 ± 0.00040.00268 ± 0.000070.01801 ± 0.000350.02439 ± 0.00028
DSSD (Ours)0.00763 ± 0.000120.00415 ± 0.000220.00739 ± 0.000160.9769 ± 0.00100.00155 ± 0.000090.01211 ± 0.000340.01632 ± 0.00046

DSSD has the best mean on all seven metrics on both datasets. Due to the limited time of the rebuttal period, we ran the additional seeds on the two datasets of Table 1; the runs for the datasets of Table 2 (BAIR and RoboNet) are not finished yet, and we will add them to the revised paper.

We agree that the difference between DPM and DSSD may not be clearly visible in the error maps of a single example (Fig. 3a, Fig. 6). However, we do not think these maps instantiate the equality in Proposition 5. Proposition 5 holds if the denoiser were ideal, that is, if it attained the Bayes-optimal error at a single noise level. Under this assumption, it states that temporal mixing does not add information about the future, and any reduction is confined to at most min⁡(N,M)\min(N, M) directions (Appendix A). The proposition thus clarifies that the benefit of DSSD does not come from additional information. Instead, DSSD changes the temporal coordinates in which the denoiser learns and the sampler generates the forecast. In practice, the denoiser has finite capacity and is trained on finite data, and a forecast is produced by a multi-step sampler rather than a single denoising step. Under these conditions, Proposition 5 does not determine which method forecasts better, and how the learning problem is posed can matter.

W2 & Q2. As in the main experiments, DSSD is an algorithm for the forward process of diffusion, not a complete forecasting model. The time-series experiments in the appendix therefore follow the same controlled design: DPM, DyDiff, and DSSD share the same denoiser, training budget, and sampler, so the comparison measures the effect of the forward process alone. We do not claim state-of-the-art performance on these benchmarks, and we will state this explicitly in the appendix.

NsDiff and TMDM are complete time-series forecasters. As we understand them, they improve the forward process by incorporating a conditional mean and variance for each time step. DSSD instead changes the correlation across time steps in the forecast window. The two address different aspects of the forward process, and they could in principle be combined. In addition, our time-series scores are computed in the normalized data space, so they are not directly comparable to the values reported in those papers.

AuthorsOpenReview ↗

Response to Q1 and Q3.

Q1. Thank you for the careful reading. You are correct. The input to the system is the scalar signal e(τ)e(\tau) (or eke_k after discretization), and the kernel Kˉℓ=c⊤Aˉℓbˉ\bar K_\ell = c^\top \bar A^\ell \bar b maps ee to the output yy. bˉ\bar b is the vector that multiplies the input and maps it into the state. Following the usual state-space convention, we called it the input matrix, which becomes a vector for a scalar input. We then wrote “input vector,” which is ambiguous with the input itself. We have revised the sentence to: “… where eke_k is the scalar input sequence and bˉ\bar b is the discretized input projection that maps it into the state.”

Q3. We agree, and we have extended Section 5.5 with an analysis of what the learned operator does. The main intuition is that the EMA operator of DyDiff applies the same decay structure to every dataset, whereas DSSD learns the temporal structure of the noise from the data. DSSD starts from a memory length similar to EMA; the difference is what training does to it. We measured the memory length of the operator (in frames, at the highest noise level) before and after training. On Turbulence the learned operator shortens its memory (0.86 →\to 0.54 frames), and on SEVIR it lengthens it (0.82 →\to 4.02 frames). Starting from the same initialization, training therefore moves the operator in opposite directions on the two datasets, which a single fixed kernel cannot do. The correlation of the noise two frames apart also changes sign with respect to the EMA operator, which gives +0.24+0.24: it becomes −0.07-0.07 on Turbulence and −0.23-0.23 on SEVIR, a structure that a decay-only kernel cannot represent and that requires the oscillatory modes. Both ingredients are supported by ablations on SEVIR. Constraining the learned modes to be non-oscillatory degrades every metric (CRPS 5.691 vs. 3.975). Learning the operator also matters: when the operator is frozen at its initialization, the temporal variogram degrades from 2.129 to 3.440 and RMSE from 14.59 to 19.98.

Reviewer_wtRwOpenReview ↗

Assessment

Thanks for the additional experiments, the clarification of proposition 5, and the extension of section 5.5 to explain the gains of the method more intuitively. They addressed most of my concerns and reinforce my score of weak accept.

Thank you for your follow-up and for maintaining your positive assessment. We are glad that the additional experiments, the clarification of Proposition 5, and the extended Section 5.5 addressed most of your concerns.

Reviewer_VGQqOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This was a very difficult paper to understand.

The goal is to develop a method for probabilistic forecasting of vector-valued time series in which the entries of the vectors have spatial correlations. The authors goal is to generate future sequences conditional on past observations. A very mature area which is used in areas such as climate and weather applications.

To explain the method let me write F(k) for the k-th frame. For simplicity, let's assume that we'll try to forecast only one frame based on two prior observations.

So, in the author's notation M=1 and N=2. We can form Z_{k,t+1}(phi ; eps(t+1)) and the training problem is simply (in population notation)

min_{theta, phi} E{ ||f_theta(Z_{t+1}, F(t-1), F(t)) - eps(t+1)||^2 },

where f_theta(.) is a decoder and Z_{t+1} is a linear function of F(t-1), F(t), F(t+1) and eps(t+1), in particular, for a suitable matrix A_phi we have

Z_{t+1} = A_phi*(F(t-1), F(t)) + b_phiF(t+1) + sigmaeps(t+1) and eps(t+1) is standard Gaussian. We can then compute E(F(t+1) | Z(t+1), F(t-1), F(t)).

The authors estimator is the natural extension of this to arbitrary N and M, they fix a discretization / time-step schedule (sigma = (1-alpha)^.5). I didn't see any particular justification for the schedule, so I'm keeping it constant here as this clarifies my understanding of what is it that the authors are doing.

Critical strengths

Experiments compare learned temporal corruption with independent noise and fixed exponential moving average corruption using the same denoiser and training and sampling budgets. They report gains of their proposed operator approach across scientific and video data sets.

Critical weaknesses
  1. The paper is not well written. I tried to reconstruct what the authors are doing, spent a lot of time trying to figure things.
  2. Experiments compare only independent-noise diffusion and fixed exponential moving-average corruption within shared architecture. The area is mature, no comparison to strong forecasting methods. And there are also new approaches like PreDiff and ABC, no mention of these either in the context of diffusions.
  3. From the experiments one can't tell what's the benefit of learning the operator from using multiple modes, for example. Maybe comparing with a fixed multimode operator and learned non-oscillatory modes can help.
Questions for authors

Can the authors please address items 2) and 3) above?

3 comments
AuthorsOpenReview ↗

Response to W1 and W2.

W1. We thank the reviewer for the careful reconstruction and apologize for the effort it required. The reconstruction is correct for a single future frame (M=1M=1) with a fixed noise level. In that setting, DSSD reduces to linearly mixing the history into the noisy future, as the reviewer describes. However, this special case removes the two components that distinguish DSSD:

  1. Temporally correlated noise. For M>1M>1, the noise is Wγˉt(M)ϵ\mathbf{W}^{(M)}_{\bar\gamma_t}\boldsymbol{\epsilon}, and the denoiser is trained to predict this correlated noise rather than ϵ\boldsymbol{\epsilon}. With M=1M=1, W(M)\mathbf{W}^{(M)} reduces to the scalar 1, so no temporal correlation remains.
  2. Step-dependent mixing. The mixing strength follows γˉt=1−η(1−αˉt)\bar\gamma_t = 1-\eta(1-\bar\alpha_t). Mixing is strongest at high noise levels and vanishes as the data become clean (γˉt→1\bar\gamma_t\to1). This schedule follows DyDiff and keeps every operator invertible (Proposition 1). Because the mixing changes with tt, DSSD is not a fixed reparameterization of standard diffusion (Section 4.5).

To make the method more intuitive, we replaced the main figure (Figure 1) with a new illustration of DSSD forward process, and we have uploaded the revised PDF.

W2. DSSD is an algorithm for the forward process of diffusion, not a specific model architecture. It changes only the temporal structure of the forward corruption, leaving the denoiser architecture unchanged. We therefore do not claim superiority over strong forecasters or state-of-the-art performance. Instead, our experiments isolate the effect of the forward process. DPM (independent noise), DyDiff (fixed EMA operator), and DSSD (learned operator) share the same VAE, denoiser, training budget, and sampling grid, so the comparison measures the forward process alone.

PreDiff is cited in the Introduction and Related Work of the submission. It mainly contributes a latent diffusion forecaster for precipitation nowcasting and a knowledge alignment step during sampling. Since its forward process is standard, with noise that is independent across frames, PreDiff and DSSD address different components of a diffusion framework. The two may be complementary, and DSSD could be applied to PreDiff's forward process.

ABC (Guo et al., 2026) was not discussed in the submission, and we thank the reviewer for pointing to it. ABC replaces the noise-to-data process by a data-to-data bridge that follows physical time, so, unlike PreDiff, it does change the corruption process, and it is the closest of the two to our work. We therefore trained ABC on SEVIR and Turbulence, the two datasets we could cover within the rebuttal period, using its public implementation (its own DiT backbone, constant volatility σ=0.25\sigma=0.25, the best SEVIR setting in the ABC paper), on the same data and the same VAE as our runs. Following the ABC recipe, we report its EMA weights.

DatasetMethodCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
SEVIRDPM4.1742.7376.5990.55182.33910.86215.420
DyDiff4.6423.1697.9550.39114.49213.02425.540
ABC4.7363.1077.5670.52732.30011.49115.704
DSSD (Ours)3.9752.4215.9950.57072.12910.26314.585
TurbulenceDPM0.011480.007510.010650.96410.002620.019200.02600
DyDiff0.011000.007520.011230.96740.002600.017910.02427
ABC0.027030.018400.025090.92840.009220.045650.06155
DSSD (Ours)0.007560.004400.007570.97790.001470.011780.01582

In this table, DSSD is better than ABC on all seven metrics on both datasets. Because ABC uses its own backbone, this row is not a forward-process-only comparison like the other three. We will add ABC to the Related Work section.

AuthorsOpenReview ↗

Response to W3.

W3. Thank you for suggesting the comparisons. Since the rebuttal period does not allow us to repeat every experiment on all datasets, we ran the additional experiments on one dataset each (Turbulence for the number of modes and SEVIR for the operator variants). All runs use the setting of the main experiments and change only the operator.

Number of temporal modes (Turbulence). We trained DSSD with P∈{1,2,6,12}P\in\{1,2,6,12\} modes; P=6P=6 is the setting of the paper.

DatasetModesCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
TurbulenceP=1P=10.008150.005150.008490.97580.001680.012940.01741
P=2P=20.007810.004900.008340.97780.001500.012220.01641
P=6P=6 (paper)0.007560.004400.007570.97790.001470.011780.01582
P=12P=120.007590.004300.007530.97820.001470.011930.01596

A single mode is the worst on all seven metrics, and the scores saturate from P=6P=6: the differences between P=6P=6 and P=12P=12 are below 3% on every metric.

Fixed multimode operator and learned non-oscillatory modes. We ran the two variants the reviewer proposed on SEVIR: the six-mode operator frozen at its initialization, and a learned operator whose modes are constrained to be non-oscillatory (frequencies fixed to zero).

DatasetVariantCRPS raw ↓CRPS w8avg ↓CRPS w8max ↓CSI w5 ↑Temporal variogram ↓Energy score ↓RMSE ↓
SEVIRDSSD (learned, oscillatory)3.9752.4215.9950.57072.12910.26314.585
frozen operator4.3472.8598.1340.44413.44012.06219.975
non-oscillatory5.6914.1317.9760.33893.65515.28536.686

The learned oscillatory operator is better than both variants on all seven metrics. Removing the oscillatory modes increases CRPS from 3.975 to 5.691 and RMSE from 14.59 to 36.69. Freezing the operator degrades every metric, most strongly the temporal variogram (2.129 vs. 3.440) and RMSE (14.59 vs. 19.98).

Dear Reviewer VGQq,

Thank you again for your detailed review. We would like to briefly summarize our responses:

  • W1 (clarity): We clarified how DSSD differs from linearly mixing the history into the noisy future (temporally correlated noise and step-dependent mixing), and replaced Figure 1 with a new illustration of the DSSD forward process in the revised PDF.
  • W2 (baselines): We added a comparison with ABC (Guo et al., 2026). DSSD outperforms ABC on all seven metrics on both SEVIR and Turbulence.
  • W3 (benefit of the learned operator): We added ablations on the number of temporal modes, a frozen operator, and non-oscillatory modes. The learned oscillatory operator outperforms all variants on all seven metrics.

We hope these responses address your concerns. If any concerns remain, we would appreciate the opportunity to address them further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and New Results

Dear Reviewers and Area Chairs,

We thank all reviewers for their time and constructive feedback. We were encouraged that the reviewers found our method novel and theoretically clean (wtRw), saw it as a unified framework that generalizes the temporal mixing of DyDiff with a small parameter overhead (QHTv), and noted that our experiments compare forward processes under the same denoiser and the same training and sampling budgets (VGQq). Below we summarize the main changes. Detailed results are in our individual responses.

1. New experiments

  • Operator ablations (QHTv W1, VGQq W3): We varied the number of temporal modes (P∈{1,2,6,12}P \in \{1, 2, 6, 12\}) on Turbulence, and we compared a frozen operator and learned non-oscillatory modes on SEVIR. The learned oscillatory operator is the best on all seven metrics. Removing the oscillatory modes increases RMSE from 14.59 to 36.69, and freezing the operator increases it to 19.98.
  • Comparison with ABC (VGQq W2): We trained ABC (Guo et al., 2026) with its public implementation on SEVIR and Turbulence. DSSD is better than ABC on all seven metrics on both datasets.
  • Spread across seeds (wtRw W1): We retrained DPM, DyDiff, and DSSD with three seeds on SEVIR and Turbulence. DSSD has the best mean on all seven metrics on both datasets, and on SEVIR it also has a much smaller spread (e.g., RMSE 14.455 ± 0.171 vs. 16.893 ± 2.092 for DPM).
  • Longer horizons and contexts (QHTv W2): On Turbulence, we varied the prediction horizon (M∈{11,16,21}M \in \{11, 16, 21\}) and the context length (N∈{4,8,12}N \in \{4, 8, 12\}). DSSD is the best on all seven metrics in every configuration, and it improves on all metrics as the context length increases.
  • Computational cost (QHTv Q2): Building the operator, including the Cholesky factorization, takes 8–11 ms for the window lengths in our experiments, which is about 1% of a training step. It stays below 80 ms even at L=1024L = 1024.

2. Revisions to the paper and clarifications

  • Figure 1 is replaced with a new illustration of the DSSD forward process (VGQq W1).
  • Section 5.5 is extended with an analysis of the learned operator (wtRw Q3). From the same initialization, training shortens the operator's memory on Turbulence (0.86 → 0.54 frames) and lengthens it on SEVIR (0.82 → 4.02 frames). It also produces negative correlations two frames apart, which a decay-only kernel cannot represent.
  • Section 3.2 now clarifies that eke_k is the scalar input sequence and bˉ\bar{b} is the input projection (wtRw Q1).
  • In the responses, we also clarified the role of Proposition 5 (wtRw W1), how DSSD differs from linearly mixing the history into the noisy future (VGQq W1), and how DSSD extends to flow matching (QHTv Q1).
  • Scope: DSSD is an algorithm for the forward process, not a complete forecasting model. It is complementary to complete forecasters such as PreDiff, NsDiff, and TMDM (VGQq W2, wtRw W2).

3. Planned for the final version

  • Add the new ablation, seed, horizon/context, ABC, and cost results to the paper and appendix.
  • Complete the seed runs on BAIR and RoboNet.
  • Add ABC to the Related Work section, and state explicitly in the appendix that we do not claim state-of-the-art performance on the time-series benchmarks.

We thank the reviewers and the Area Chairs again for their time and helpful feedback.

Sincerely, Authors of Submission58704

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.