acceptodds
Under review as a conference paper at ICLR 2027

DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters

Abstract

While generative modeling facilitates probabilistic time series forecasting, incorporating heterogeneous exogenous information remains challenging. Diffusion Transformers (DiT) provide a scalable framework for conditional generation, yet their adaptation to forecasting calls for conditioning mechanisms tailored to time series. Endogenous targets and exogenous covariates differ in sources, semantics, and statistical characteristics, while sharing temporal coordinates that support fine-grained conditional guidance. Covariates can describe future variability and temporal dependence beyond the conditional mean targeted by direct regression. Motivated by these considerations, we propose Diffusion Transformers for Time Series (DiTS), a Multimodal Diffusion Transformer for covariate-aware forecasting. DiTS models endogenous targets and exogenous covariates as distinct modalities, jointly conditioning future generation on target history and available covariates. Flow matching makes covariate-dependent distributional information relevant to velocity prediction conditioned on noisy future states. We introduce Time-aligned Modulation, extending AdaLN with the temporal-alignment prior to generate patch-wise modulation parameters from aligned covariates and diffusion time. Across diverse covariate-aware forecasting tasks, DiTS achieves strong performance in both deterministic and probabilistic forecasting, demonstrating the effectiveness of conditional generation for both point and distributional forecasting.

Primary area:
generative models
Keywords:
Time Series Forecasting, Diffusion Transformer, Covariate-aware Forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yKu1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

DiTS uses a standard Flow Matching method as its main algorithmic framework to develop a conditional generative model for time series. The paper adds a novel Time-aligned Modulation module to Transformer, which provides different conditional modulation parameters for different target patches and leads to better performance. The method achieves relatively strong generation performance on datasets with future exogenous variables, but performs less well when future exogenous variables are unavailable.

Critical strengths
  1. The paper introduces a novel Time-aligned Modulation module into the Transformer, allowing different target patches to use different conditional modulation parameters and improving model performance.
Critical weaknesses
  1. The main innovation lies in adding a Time-aligned Modulation module to the Transformer. However, this only makes a minor change to the Transformer architecture and does not introduce any new methods or algorithms, so the methodological novelty is somewhat limited.
  2. The method performs well when future exogenous variables are available, but its performance is relatively poor when they are unavailable. This may be because the method used to predict future exogenous variables is not sufficiently effective, which limits the overall performance. This suggests that the method relies heavily on the accuracy of future exogenous variable predictions. Could you develop a novel approach to predicting future exogenous variables that is specifically designed for your framework, so as to further improve its performance and make it more widely applicable?
Questions for authors
  1. Could you develop a novel approach to predicting future exogenous variables that is specifically designed for your framework, so as to further improve its performance and make it more widely applicable?
  2. You need to compare your method with some representative methods, such as KITE, Sundial and TimeDiff.
5 comments

Many thanks to Reviewer yKu1 for the thoughtful review and suggestions for extending our evaluation.

W1. What DiTS adds to DiT

DiTS adapts multimodal DiT to covariate-aware forecasting. Electricity demand and price have different meanings and units, yet their timestamps tell us which intervals correspond. DiTS uses this structure to define what the model generates and how it is conditioned. Only target patches evolve through the backbone. A separate encoder preserves the timing of the covariates that guide target generation under conditional flow matching.

Time-aligned Modulation changes where a covariate enters the velocity prediction. Window-level AdaLN gives morning and evening price patches the same modulation even when their load forecasts differ. DiTS uses a shared mapping to derive each patch's scales, shifts and gates from its aligned covariates and flow time. Revising the evening load forecast can therefore change the evening price update directly. Temporal attention can carry that information to later hours without adding covariate tokens. In Table 13, this design yields lower mean MSE across five markets than global modulation or any of the attention alternatives tested.

W2. Forecasting with past-only covariates

With only historical covariates, DiTS has the lowest five-market mean MSE for 24-step EPF forecasting (0.280 vs. 0.285 for GCGNet, Table 2). In this setting, a shared linear adapter maps past covariate embeddings to a latent condition for each future patch. The adapter is trained through the target velocity loss. For example, it can learn which aspects of past load help forecast tomorrow's prices without estimating tomorrow's load in physical units. When a load forecast is available, each price patch receives the external forecast for its own interval. In that setting, Table 1 reports a 15.1% reduction in mean MSE over the strongest baseline, averaged across two horizons. We address your second question in Q1, where we evaluate a DiTS extension that explicitly forecasts future covariates.

Q1. An explicit future-covariate predictor

We implemented your suggestion by swapping the target and covariate roles. Past-only DiTS generates ten 24-hour price paths from 168 hours of history. A second DiTS forecasts each covariate using the aligned price path through Time-aligned Modulation. Training uses paired observations. Inference uses history and generated paths.

We tested five EPF markets with three paired seeds for the exogenous models. A uses the same history, including past prices. B and C share weights and noise. B receives the mean price path and C receives individual paths.

Forecasting the future covariates

MethodMSEMAEWQL
A. History0.1190.2010.0845
B. Mean price0.1200.1980.0846
C. Price paths0.1200.1990.0844

Forecasting prices using those covariate predictions. A–C use the same frozen price model and ten output trajectories.

MethodMSEMAEWQL
Original DiTS0.2790.2550.1066
A. History0.2870.2580.1075
B. Mean price0.2820.2550.1067
C. Price paths0.2810.2540.1063

The extension supplies explicit forecasts of both external series. Price guidance reduced covariate MSE in PJM, FR and DE, with a small increase in the five-market mean. Feeding C's forecasts back improved price MSE over A at all three seeds. Relative to original DiTS, C lowered price MSE in four markets, but the increase in DE raised the overall mean. Its mean price WQL was slightly lower. End-to-end latency was 49.6 ms vs. 15.5 ms for original DiTS.

Q2. Comparisons with KITE, Sundial and TimeDiff

Sundial is discussed in Section 2.2 of the original submission. We now compare it, KITE and TimeDiff with DiTS on the 13 tasks and test windows of Figure 4. DiTS uses the original test-selected configurations (five steps, 50 samples). Baselines use default step counts. We report geometric means.

Metric ↓DiTSKITESundialTimeDiff
WQL0.08140.09310.08800.1175
MASE0.67810.78080.71810.8695

Sundial is closest on both metrics and achieves lower WQL and MASE on the Belgian electricity-price task. Across all 13 tasks, DiTS has the lowest aggregate error for both quantiles and point forecasts.

The separate full-test EPF comparison below uses covariates available throughout the forecast horizon. DiTS uses the paper's fixed configuration and averages ten trajectories for each point forecast.

Five-market mean ↓DiTSKITESundialTimeDiff
MSE0.2040.2460.2590.295
MAE0.2070.2320.2410.267

Dear Reviewer yKu1,

We hope our response has addressed the concerns you raised. We would welcome any further feedback before the discussion closes and would be glad to elaborate on any point. Thank you again for your careful reading and constructive suggestions.

Kind regards,
The Authors

Thank you for the detailed responses and additional experiments. The authors have made efforts to address my concerns. The new comparisons and future-covariate forecasting experiments provide a clearer picture of the method's performance. However, I still have concerns about the methodological novelty and the effectiveness of the proposed extension.

The additional explanations and ablation studies help clarify the role of Time-aligned Modulation. The results show that patch-wise conditioning can improve forecasting performance compared with several alternative conditioning mechanisms. However, the main contribution remains a modification of the existing AdaLN mechanism within a standard Transformer and flow-matching framework. The improvement over variate attention is also relatively small (0.204 versus 0.207 in average MSE). Therefore, my concern about the limited methodological novelty is only partially addressed.

The authors have also added an explicit future-covariate forecasting extension, but the new results do not show a clear improvement over the original DiTS. For future-covariate prediction, using generated price trajectories does not improve the average MSE compared with using historical information alone (0.120 versus 0.119). More importantly, when these predicted covariates are used for electricity price forecasting, the proposed extension achieves an average MSE of 0.281, slightly worse than the original DiTS (0.279). Although WQL improves slightly (0.1063 versus 0.1066), the difference is very small. Meanwhile, inference latency increases substantially from 15.5 ms to 49.6 ms. These results do not show a clear practical advantage of the proposed extension over the original DiTS.

I also have a concern about the training and inference procedures. The second DiTS is trained using paired observations, but during inference it uses price trajectories generated by the first model. This may lead to a distribution mismatch between training and inference. Prediction errors may also propagate through the covariate forecasting stage. It would be helpful if the authors could clarify how these issues are handled.

The additional comparisons with KITE, Sundial, and TimeDiff largely address my previous concern about insufficient experimental comparisons. The results show that DiTS performs competitively across the evaluated tasks. However, it would be helpful to clarify whether all methods use the same available covariate information and comparable inference budgets, especially since DiTS and the baselines use different sampling configurations.

Overall, the revisions have addressed several of my concerns and provided stronger experimental evidence for Time-aligned Modulation. However, the methodological novelty remains limited. The future-covariate forecasting extension also shows no clear improvement over the original DiTS, despite its much higher computational cost. Therefore, I maintain my original rating.

Reviewer_eJQtOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces DiTS, a multimodal-diffusion transformer based on flow matching for time series forecasting. Noisy patches and covariates are processed by DiTS blocks to predict the noise-free sample. In an empirical evaluation, the model outperforms baselines in point and probabilistic forecasting.

Critical strengths
  • The empirical evaluation shows improvements over selected baselines and covers deterministic and probabilistic forecasting.
  • The methodology is easy to understand.
Critical weaknesses
  • The major weakness of the work is the limited novelty. The methodological contribution seems to be mostly slight architectural changes. The generative model itself is a standard flow matching model, which has been applied to time series before.
  • The related work section is missing crucial works from the generative literature and only covers works up to 2024 with a few exceptions.
  • Important models are missing from the empirical evaluation. The probabilistic forecasting benchmark only covers TimeGrad from the diffusion literature, which is from 2021. There have been many advances since that work.
Questions for authors

See weaknesses

4 comments

We sincerely thank Reviewer eJQt for the careful reading and thoughtful questions about our work.

W1. Adapting DiT to covariate-aware forecasting

We agree that prior work has applied flow matching to time series. This still leaves open the architectural question our paper asks. Given a forecast of evening electricity demand, how should a DiT use it to generate the corresponding price trajectory? The timestamp already identifies the relevant price interval. Window-level AdaLN broadcasts the same modulation to every interval. Time-aligned Modulation uses the known correspondence to condition each target patch's velocity update directly. Temporal attention then learns how far the effect extends across time.

As iTransformer illustrates, a change in how an existing operator is applied can matter when it captures a property of the data. Table 13 tests this point with the flow-matching objective held fixed. Replacing window-level modulation with patch-wise modulation lowers mean MSE across five markets from 0.220 to 0.204 (7.3%), the lowest among the conditioning mechanisms tested. It does so without adding covariate tokens to temporal attention. This gives DiT a simple way to exploit information that a forecaster already has about when each condition applies.

W2. Coverage of generative forecasting methods

We have revised the Related Work section of the manuscript to discuss TimeDiff (ICML 2023), which uses future mixup and autoregressive initialization, D³U (ICLR 2025), which combines point prediction with diffusion for uncertain components, and KITE (ICML 2026), which directly addresses probabilistic forecasting with exogenous variables. Our submission also discusses Sundial and evaluates GCGNet. W3 gives the new comparisons, including KITE as a baseline designed for the same forecasting task.

W3. Additional comparisons

We evaluated KITE, TimeDiff and D³U on the same 13 tasks and test windows as Figure 4. We reused the DiTS configurations selected for Figure 4 using four test metrics (five steps, 50 samples). Each added method used its default step count. The entries are geometric means across tasks, and the ranking is exploratory. WQL measures quantile accuracy and MASE measures point accuracy.

Metric ↓DiTSKITETimeDiffD³U
WQL0.08140.09310.11750.1051
MASE0.67810.78080.86950.8851

We also report a separate comparison on the full EPF test sets with future covariates available at forecast time.. DiTS uses the fixed paper configuration, with the point forecast obtained by averaging ten trajectories.

Five-market mean ↓DiTSKITETimeDiffD³U
MSE0.2040.2460.2950.308
MAE0.2070.2320.2670.267

KITE is the closest of the three on both 13-task aggregates and has lower WQL than DiTS on some individual tasks, including the 15-minute ENTSO-E task. DiTS has the lowest aggregate WQL and MASE, and its advantage also appears when the criterion is electricity-price point accuracy.

Q1. What the ablations establish

Table 13 tests how to deliver each external observation to the corresponding target interval. Table 10 holds the forecasting inputs fixed within each setting and changes the prediction task. If two regimes have the same future mean but different volatility, a direct mean forecast can be optimal in both. At a given noisy state, the optimal velocity can still depend on the regime. In the ten-seed comparison, both covariate settings favor CFM (Holm-adjusted p < 0.001 in each), while the target-only interval spans zero. This is the pattern our analysis motivates. If you would find it helpful, we would welcome further discussion of any aspect of the paper.

Dear Reviewer eJQt,

Thank you again for the care you put into reviewing our paper. Your comments helped us see where our explanation needed more detail. We would welcome your thoughts on whether our response has addressed your main concerns and would be glad to clarify any remaining points.

Best regards,
The Authors

I thank the authors for their rebuttal. After reading the response and the other reviews, I agree with reviewer yKu1. The paper proposes only a minor architectural modification. While I appreciate the new baselines and extended related work, it is still not sufficient. Therefore, I will maintain my score.

Thank you for taking the time to read our rebuttal and explain your remaining concern. We appreciate your point about the scope of the architectural change. The methodological contribution lies in identifying which part of a standard DiT to adapt for covariate-aware forecasting. We analyzed how covariates inform velocity prediction under conditional flow matching and tested several ways to introduce them into the backbone. These comparisons led to Time-aligned Modulation, which uses the shared time grid to condition each target patch on the corresponding covariates.

Once effectiveness is established, we favor a targeted and parsimonious design that directly exploits the structure of the task. Our results across multiple benchmarks demonstrate the design’s effectiveness for both point and probabilistic forecasting. Introducing aligned covariates through modulation lets DiTS retain standard Transformer attention and feed-forward layers. This shows how a general-purpose architecture can be adapted to a specific task and gives the community a conditioning mechanism it can test, reuse, and extend within existing DiT architectures.

Thank you again for engaging with our response and giving us the opportunity to clarify the design. If you have any further questions, we would welcome the opportunity to address them before the discussion closes.

Reviewer_weVNOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces DiTS, a Transformer-based conditional flow-matching model for time series forecasting with exogenous covariates. The main idea is to learn a velocity field conditioned on target history and covariates, and use it to iteratively transform Gaussian noise into future target trajectories. The authors also propose Time-aligned Modulation, which combines temporally aligned covariate patches with diffusion-time embeddings to generate Patch-wise Modulation Parameters for temporal attention and feed-forward layers, thereby adapting target representations to the local covariate context and the current generation stage. Experiments cover five electricity price forecasting datasets and 13 selected probabilistic forecasting tasks, reporting strong performance in both deterministic and probabilistic forecasting.

Critical strengths

Strengths:

  • [S1] The paper introduces a novel and sensible way of incorporating exogenous information into the forecasting process, addressing the limited distributional supervision provided by direct mean regression. And the Time-aligned Modulation design is also intuitive and practically useful for preserving temporal correspondence in covariate injection.
  • [S2] The experiments are comprehensive, covering the training objective, conditioning mechanisms, sampling budget, and noise schedule. The empirical performance of DiTS is also competitive in both point and probabilistic forecasting tasks.
  • [S3] The paper is generally well-written and easy to follow, with the main idea and methodology presented clearly and in detail.
Critical weaknesses

Weaknesses:

  • [W1] Although the method is well-motivated and evaluated extensively, the paper lacks formal theoretical results supporting its effectiveness. For example, providing an analysis of when conditional flow matching can exploit covariate information beyond the conditional mean, and how this relates to forecasting error, would enhance the significance of this work and guide the application more broadly.
  • [W2] Minor: The direct regression baseline could be described more explicitly. Sec. 4.2 states that flow matching and direct MSE regression use the same backbone, architecture hyperparameters, and optimization protocol, but it is unclear what inputs are used at future patch positions and how diffusion-time conditioning is handled in the regression baseline.
Questions for authors

Questions:

  • [Q1] Could the authors report error bars or paired t-test results (with p-values) over multi-seed runs on a representative subset of the comparisons in Table 10, to assess whether the reported gains are statistically significant?
  • [Q2] The Covariate Encoder aggregates all covariate patches at each temporal position through flattening and an MLP. While Table 13 compares different covariate conditioning mechanisms, could the authors examine whether explicit cross-variate attention within the encoder is beneficial, while keeping Time-aligned Modulation unchanged?
  • [Q3] Figure 7 reports the time to generate one sample. I'm wondering how DiTS would compare with TimeXer or iTransformer in terms of full inference latency for multivariate point forecasting. Also, when multiple target variables need to be forecast, would DiTS generate each target separately?
3 comments

We sincerely thank Reviewer weVN for the thoughtful review and constructive comments.

W1. When variance information helps flow matching

Fix the target history. Let a covariate identify two regimes of positive probability with future laws Y∼N(0,a)Y\sim\mathcal N(0,a) and Y∼N(0,b)Y\sim\mathcal N(0,b), where 0<a<b0<a<b. The optimal point forecast under squared error is zero in both. On the DiTS path Zt=tY+(1−t)ϵZ_t=tY+(1-t)\epsilon, with independent standard Gaussian noise, write s∈{a,b}s\in\{a,b\} for the regime variance. At Zt=zZ_t=z, Gaussian conditioning gives

vs(z,t)=E[Y−ϵ∣Zt=z,s]=ts−(1−t)t2s+(1−t)2z.v_s(z,t)=\mathbb E[Y-\epsilon\mid Z_t=z,s]=\frac{ts-(1-t)}{t^2s+(1-t)^2}z.

For every 0<t<10<t<1, the coefficient of zz is strictly increasing in ss. Thus each nonzero state requires a different velocity in the two regimes. Both state densities are positive everywhere, so observing the state leaves both regimes possible. By conditional variance decomposition, omitting the covariate strictly increases the minimum velocity MSE. A covariate that leaves the optimal point forecast unchanged can still be necessary for optimal velocity prediction. Whether learning this velocity also improves a fitted model’s point forecasts is an empirical question, which we examine in Q1.

W2. Inputs to the direct regressor

Appendix D.1 now explains the direct regressor used in Table 10. Each future target patch is a fixed zero vector passed through the shared patch embedding, so its initial token equals the embedding bias. The regressor uses no diffusion time. With covariates, a zero time vector leaves the aligned covariate condition as the sole modulation input. The regressor predicts the full horizon in one pass and minimizes MSE on clean, history-normalized targets. A learned future embedding gave similar results.

Q1. Paired tests for Table 10

For each setting in Table 10, we froze the configuration chosen at seed 2021 and ran both objectives on ten matched seeds (2021–2030). The table reports mean MSE, the paired difference (Direct − CFM) with its 95% CI, and Holm-adjusted p-values from paired t-tests.

InputsCFMDirectDirect − CFM [95% CI]p
Target only0.3170.312−0.005 [−0.013, 0.004]0.225
Past covariates0.2950.3150.021 [0.015, 0.026]8.31e-5
Known future covariates0.2080.2380.031 [0.026, 0.035]7.86e-7

With target history alone, the estimated MSE difference is close to zero and its interval spans zero. With past or known future covariates, both intervals favor CFM. Relative to the target-only setting, the estimated CFM advantage grows by 0.025 and 0.035 MSE, respectively. These last two contrasts are descriptive. W1 gives one reason this pattern can arise. The same noisy state can require different updates under different external conditions.

Q2. Cross-variate attention inside the Covariate Encoder

We changed only the Covariate Encoder. Within each patch, self-attention mixes covariate tokens with variate embeddings. Mean pooling and projection yield one condition for Time-aligned Modulation.

EPF (MSE / MAE)MLPCross-variate attention
NP0.160 / 0.2070.161 / 0.210
PJM0.060 / 0.1410.074 / 0.175
BE0.330 / 0.2370.336 / 0.236
FR0.277 / 0.1710.265 / 0.172
DE0.216 / 0.2920.228 / 0.297
Mean0.209 / 0.2100.213 / 0.218

Q3. End-to-end latency and multiple targets

DiTS generates a trajectory for each target. In our three-target EPF benchmark, one model shares its weights across targets and processes them in parallel. Each target uses the other two series’ histories. All three futures are withheld. MSE and MAE average all three targets across five markets.

MethodMSEMAECRPSms
iTransformer0.1790.223—2.114
TimeXer0.1750.219—3.543
DiTS, 5 steps0.1590.1970.138912.544
DiTS, 1 step (original weights)0.1630.2150.16653.391
1-step CD0.1570.1980.15073.358
1-step DMD0.1590.2000.14613.386

Both students approach TimeXer’s latency, and iTransformer is fastest. Consistency distillation (CD) gives the lowest point MSE. Distribution matching distillation (DMD) is 3.7× faster than the teacher. We assessed predictive distributions on 256 fixed windows per market with 100 draws. DMD gives lower CRPS than CD in four markets. Empirical coverage of nominal 90% intervals averages 69.4% for DMD, 58.3% for CD and 77.2% for the teacher.

Thanks for the detailed response. I appreciate the additional theoretical insights and robustness checks. Most of my concerns have been addressed, and I will keep my current rating.

Thank you for your thoughtful review and for taking the time to consider our response. We are glad that the additional analysis and experiments helped address most of your concerns. Your questions helped us clarify the reasoning behind DiTS and how the experimental results support it. We will incorporate these clarifications and findings into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.