acceptodds
Under review as a conference paper at ICLR 2027

DCNet: Multivariate Time Series Forecasting via an Efficient Dual Context Network

Abstract

Multivariate time series forecasting is of great significance in fields such as energy scheduling, traffic management, and financial analysis. This paper proposes the Dual Context (DC) technique and reconstructs the retrieval space of the attention mechanism based on this technique; at the same time, combined with a novel multilayer perceptron, DCNet is constructed to fully exploit the coupling features between global and local contexts. DCNet not only avoids the isolated modeling state of global periodic patterns and local evolution dynamics in previous studies, but also effectively improves the problem of blind coupling of variables in existing inverted architectures under complex environments by introducing a global reference baseline. DCNet achieves overall leading accuracy on 12 widely used real-world datasets. In addition, further experiments show that the DC technique not only performs well on models based on the Transformer architecture, but also significantly improves linear models, proving that the technique has good universality and cross-architecture transferability.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Multivariate Time Series Forecasting, Dual Context, Transformer

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gt3wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DCNet, a dual-context framework for multivariate time-series forecasting. It combines global periodic patterns with local historical observations through a modified attention mechanism, using mixed queries and an expanded key–value space. A SwiGLU-based feed-forward module is also incorporated. Experiments across 12 datasets show competitive forecasting performance, with additional ablations, efficiency comparisons, and noise robustness evaluations. The results are encouraging, especially on high-dimensional PEMS datasets, although I have some concerns about the methodological novelty.

Critical strengths
  1. The paper addresses an important problem in multivariate forecasting. Combining global periodic information with local observations is a reasonable approach to modeling variable dependencies.

  2. I find the empirical results encouraging. DCNet achieves top-two performance on 22 of 24 aggregated metrics, with particularly strong results on the PEMS datasets.

  3. The ablation experiments support the usefulness of the dual-context mechanism. I also appreciate the additional efficiency and noise robustness analyses.

Critical weaknesses
  1. Novelty is somewhat limited. TQNet already uses global periodic representations to guide attention, while CycleNet provides the underlying periodic modeling approach. DCNet mainly extends these ideas through query mixing and key–value concatenation.

  2. The claim about reducing spurious correlations is not fully supported. Clearer attention maps do not necessarily indicate more meaningful variable relationships.

  3. The ablation study could be more specific. It is unclear how much improvement comes from query mixing versus simply adding periodic tokens to the attention mechanism.

  4. The robustness evaluation is relatively narrow. Noise experiments are conducted only on Electricity, and the comparison does not include TQNet.

Questions for authors
  1. What is the main methodological difference between DCNet and TQNet? Could the authors provide a more direct comparison under matched model capacity and experimental settings?

  2. Could the authors separately evaluate query mixing and key–value expansion? I would like to understand which component contributes most to the improvements.

  3. How can the authors demonstrate that DCNet actually reduces spurious inter-variable correlations rather than simply changing attention patterns?

  4. Would the robustness improvements remain similar on other datasets and against stronger baselines such as TQNet? How sensitive is DCNet to incorrectly specified periodic lengths?

5 comments

Response to Reviewer gt3w

We thank the reviewer for the careful evaluation. To address your concerns, we will answer your questions one by one below.

Q1. What is the main methodological difference between DCNet and TQNet? Could the authors provide a more direct comparison under matched model capacity and experimental settings?

We appreciate TQNet as an effective and efficient prior method. Its two core components are (i) a multi-head attention module that uses a periodic query, operating over the look-back dimension LL, and (ii) an MLP with two linear layers and two activations, operating over the hidden dimension DD. This design is simple and effective, but its two modules use different feature axes. As a result, it is less naturally organized as a conventional stack of variable-token Transformer blocks.

DCNet instead adopts a tokenized inverted-Transformer structure and operates over the variable dimension NN with hidden dimension DD. Both tokenized local and periodic contexts participate in query construction and key/value retrieval:

Q=βXenc+(1−β)Penc,K,V=[Xenc;Penc].Q=\beta X_{\mathrm{enc}}+(1-\beta)P_{\mathrm{enc}}, \qquad K,V=[X_{\mathrm{enc}};P_{\mathrm{enc}}].

This allows DCNet to use standard variable-token encoder stacking when necessary. In several settings, a single encoder layer is already sufficient, but the standard stacking interface also makes the model practical for larger-scale datasets.

We also note a difference in portability. TQNet's transfer experiments apply the attention block itself, for example through additive fusion in PatchTST and DLinear. In contrast, the DC construction combines local and periodic contexts at the representation level, so it can also be applied to purely linear backbones. For example, on PEMS08, the DC-augmented CycleNet outperforms TQNet in Table 3 and Table 6. The DLinear variant improved with DC technology also significantly outperforms the corresponding TQNet transfer variant. These comparisons reflect different design goals: TQNet is intentionally compact, while DC is designed as a portable dual-context interface.

For quantitative comparisons, all models use the same data split, look-back length, prediction lengths, and training protocol. Baseline-specific settings follow their official implementations. Exact parameter matching is difficult because the architectures operate on different axes; we therefore report parameter counts and use within-DCNet component ablations to isolate the mechanism. The Electricity result in Table 2 and Table 8 uses a single encoder layer, which is comparable to TQNet's compact setting.

Q2. Could the authors separately evaluate query mixing and key–value expansion? I would like to understand which component contributes most to the improvements.

We added a component-level ablation over the composite query QQ, the expanded key/value branch KVKV, and the MLP. The variants are:

  • QQ: retain the composite query; remove the global periodic context (GPC) from the expanded KV branch and replace the new MLP with a standard linear-activation stack.
  • KVKV: reduce the composite query to the local evolution context (LEC) and remove the MLP.
  • MLP: remove both the composite query and the expanded KV branch, using standard self-attention instead.
  • QQ+MLP: remove GPC from the KV branch.
  • KVKV+MLP: reduce the composite query to LEC.
  • QQ+KVKV: remove the MLP.
  • W/o Both: remove all three components.

The complete results are reported below. Each cell is MSE/MAE.

DatasetStepDCNetQKVMLPQ+MLPKV+MLPQ+KVW/o BothTQNetiTransformer
Metric-MSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAE
Electricity960.131/0.2210.134/0.2290.138/0.2330.145/0.2370.133/0.2270.136/0.2280.134/0.2290.145/0.2380.134/0.2290.148/0.240
-1920.150/0.2380.155/0.2470.156/0.2470.163/0.2540.153/0.2430.153/0.2420.152/0.2450.163/0.2560.154/0.2470.162/0.253
-3360.165/0.2540.168/0.2650.171/0.2640.177/0.2690.168/0.2660.166/0.2610.168/0.2650.178/0.2730.169/0.2640.178/0.269
-7200.192/0.2790.208/0.3010.204/0.2970.207/0.2980.199/0.2890.194/0.2830.221/0.3110.228/0.3120.201/0.2940.225/0.317
-Avg0.159/0.2480.166/0.2610.167/0.2610.173/0.2650.163/0.2560.162/0.2540.169/0.2630.179/0.2700.165/0.2590.178/0.270
PEMS03120.056/0.1550.058/0.1590.061/0.1590.064/0.1660.059/0.1600.059/0.1590.058/0.1590.064/0.1670.060/0.1610.071/0.174
-240.068/0.1710.075/0.1800.074/0.1780.081/0.1880.075/0.1810.072/0.1760.069/0.1740.082/0.1900.077/0.1820.093/0.201
-480.089/0.1960.102/0.2110.098/0.2040.114/0.2250.098/0.2060.098/0.2030.091/0.1980.114/0.2250.104/0.2150.125/0.236
-960.126/0.2290.144/0.2470.136/0.2360.154/0.2650.145/0.2460.133/0.2340.136/0.2350.162/0.2760.148/0.2530.164/0.275
-Avg0.084/0.1870.095/0.1990.092/0.1940.103/0.2110.094/0.1980.091/0.1930.089/0.1920.106/0.2150.097/0.2030.113/0.222
PEMS04120.063/0.1610.065/0.1640.067/0.1660.073/0.1760.064/0.1620.065/0.1630.065/0.1650.074/0.1760.067/0.1660.078/0.183
-240.072/0.1710.075/0.1770.078/0.1780.089/0.1950.075/0.1750.079/0.1790.074/0.1750.089/0.1950.077/0.1810.095/0.205
-480.083/0.1850.096/0.2060.091/0.1950.115/0.2230.091/0.1960.089/0.1920.086/0.1960.117/0.2270.097/0.2060.120/0.233
-960.097/0.2010.126/0.2390.107/0.2140.145/0.2520.123/0.2330.101/0.2080.104/0.2090.147/0.2560.123/0.2330.150/0.262
-Avg0.078/0.1790.091/0.1970.086/0.1880.106/0.2120.088/0.1920.084/0.1860.082/0.1860.107/0.2140.091/0.1970.111/0.221
PEMS07120.050/0.1400.052/0.1430.053/0.1450.056/0.1480.052/0.1420.051/0.1410.052/0.1430.057/0.1510.051/0.1430.067/0.165
-240.058/0.1500.064/0.1610.064/0.1580.069/0.1660.065/0.1630.061/0.1540.059/0.1520.073/0.1730.063/0.1590.088/0.190
-480.070/0.1630.080/0.1780.078/0.1710.091/0.1890.077/0.1730.077/0.1710.073/0.1680.097/0.2010.081/0.1790.110/0.215
-960.087/0.1800.099/0.1970.093/0.1880.113/0.2040.096/0.1920.094/0.1890.091/0.1870.123/0.2270.103/0.2030.215/0.245
-Avg0.066/0.1580.074/0.1700.072/0.1660.082/0.1770.073/0.1680.071/0.1640.069/0.1630.088/0.1880.075/0.1710.120/0.204
PEMS08120.067/0.1620.073/0.1720.068/0.1640.077/0.1710.071/0.1700.068/0.1630.069/0.1650.078/0.1740.071/0.1700.079/0.182
-240.083/0.1740.097/0.1920.086/0.1800.105/0.1970.093/0.1870.086/0.1780.083/0.1740.109/0.2020.096/0.1960.115/0.219
-480.109/0.1910.138/0.2240.117/0.2000.153/0.2310.139/0.2260.115/0.1980.115/0.1980.154/0.2380.149/0.2440.186/0.235
-960.138/0.2050.179/0.2520.149/0.2190.181/0.2560.166/0.2370.146/0.2140.147/0.2160.188/0.2630.253/0.3090.221/0.267
-Avg0.099/0.1830.122/0.2100.105/0.1910.129/0.2140.117/0.2050.104/0.1880.104/0.1880.132/0.2190.142/0.2300.150/0.226

The KV-only variant provides the largest single-component gain on all four PEMS datasets. On Electricity, the Q-only variant is slightly stronger than KV-only. The complete DCNet remains best on every dataset, indicating that the components are complementary rather than redundant.

Q3.How can the authors demonstrate that DCNet actually reduces spurious inter-variable correlations rather than simply changing attention patterns?

We thank the reviewer for this important correction. We agree that a clearer attention map does not, by itself, demonstrate that the learned variable relations are more meaningful. To our knowledge, none of the twelve benchmark datasets provides a verified variable-level causal graph. We therefore use a controlled synthetic benchmark with a known dependency graph.

The latent variables follow zt+1=Azt+pt+ϵtz_{t+1}=Az_t+p_t+\epsilon_t, where AA is a known sparse graph and pt,i=γisin⁡(2πt/24+ϕi)p_{t,i}=\gamma_i\sin(2\pi t/24+\phi_i) is a periodic component with period T=24T=24. The observed input is xt=zt+Crut+ηtx_t=z_t+C_r u_t+\eta_t: the disturbance CrutC_r u_t changes the observed Pearson correlations across regimes but does not affect the future latent target. Thus, spurious correlations can be present in the input without being valid predictive edges.

We use 7 variables, 9 nonzero off-diagonal edges among 42 directed candidate pairs, look-back length 96, and prediction length 96. The experiment is repeated with three seeds. We construct a 7×77\times7 dependency matrix by perturbing one input variable at a time and measuring its first-step effect on each output variable. The resulting matrix is compared with the known graph. We report AUROC, the area under the receiver operating characteristic curve: it is the probability that a true edge receives a higher dependency score than a non-edge. Random ranking gives approximately 0.5; higher is better. We also report AUPRC, the area under the precision-recall curve, which focuses on true edges among the highest-ranked pairs. Since 9 of 42 candidate pairs are true, random ranking gives an AUPRC baseline of approximately 9/42=0.2149/42=0.214.

ModelTest MSETest MAEDependency AUROCDependency AUPRC
DCNet0.3899 +/- 0.01050.4934 +/- 0.00680.6790 +/- 0.07320.3621 +/- 0.0852
iTransformer0.3990 +/- 0.00920.4992 +/- 0.00580.5039 +/- 0.01660.2888 +/- 0.0302
TQNet0.4150 +/- 0.00860.5088 +/- 0.00550.5488 +/- 0.01010.2544 +/- 0.0070

DCNet obtains lower test MSE and MAE than both baselines. Its dependency AUROC is higher than iTransformer by 0.175 and than TQNet by 0.130. Its AUPRC is also higher than iTransformer by 0.073 and than TQNet by 0.108.

As an additional control, we evaluate both models with the correct phase and a shuffled phase.

ModelPhaseAUROCAUPRCMSE degradation
DCNetCorrect0.6790 +/- 0.07320.3621 +/- 0.0852-
DCNetShuffled0.5533 +/- 0.04570.3176 +/- 0.031332.31% +/- 8.80%
TQNetCorrect0.5488 +/- 0.01010.2544 +/- 0.0070-
TQNetShuffled0.5488 +/- 0.01010.2544 +/- 0.00700.00%

The TQNet temporal query and query/key projections collapse to nearly zero during training under this setting, making the learned query phase-insensitive.This also explains why TQNet, which incorporates periodic representation, has higher forecasting error than iTransformer. In contrast, phase corruption substantially degrades both DCNet's forecasting error and dependency recovery. These results indicate that DCNet effectively uses the phase-aligned periodic context and recovers the known dependency graph more closely than both a model without periodic context and the TQNet baseline.

Reviewer_6FJgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DCNet, an inverted Transformer model for multivariate time series forecasting that combines local observations with a learned global periodic context. Queries mix the two contexts, while keys and values are constructed from their concatenation. The authors argue that the periodic reference helps suppress noise-induced spurious correlations. The model also uses a SwiGLU feed-forward network. DCNet achieves competitive results across the evaluated benchmarks, particularly on PEMS. The paper includes ablation studies, experiments applying DC to DLinear and CycleNet, and analyses of efficiency, sensitivity, and noise robustness.

Critical strengths

Competitive performance across diverse datasets. DCNet performs competitively across the evaluated benchmarks, with particularly notable gains on PEMS. I consider the consistently strong empirical results across different forecasting settings the main strength of the paper.

Simple design with practical value. The method introduces global periodic information into inverted attention through a relatively simple modification. The improvements reported for DLinear and CycleNet also suggest that the DC idea is useful beyond the proposed architecture. On Electricity, DCNet offers a favorable accuracy–cost trade-off compared with iTransformer and EMAformer.

Critical weaknesses

Unclear contribution beyond related work. TQNet already uses periodic queries to combine global information with local observations. DCNet adds query mixing and an expanded key–value space, but the comparisons do not isolate the benefit of the latter under otherwise matched settings. Figure 5 varies the mixing coefficient, but the main ablation removes DC entirely, so the individual contribution of the expanded key–value space remains unclear. In addition, the MLP described as novel uses the standard SwiGLU formulation. Together, these issues make it difficult to assess the significance of the proposed changes beyond existing methods.

Insufficient evidence for the claimed correction of spurious variable relationships. Figure 2 compares variable correlations, but does not establish which relationships are caused by noise or whether DCNet corrects them. Table 4 shows improved robustness to Gaussian noise on Electricity, but does not show that this improvement comes from correcting variable relationships. Since this explanation is central to the paper's motivation, it should be supported by direct evidence. The current results demonstrate improved forecasting performance but leave this claim unverified

Questions for authors

Please see weaknesses.

5 comments

Response to Reviewer 6FJg

Thank you for your detailed and thoughtful review. We respond to the two main concerns below:

1. More fine-grained evaluation of DCNet's contribution

We have added the following component-level ablation groups:

  • (Q): retain the composite query; remove the global periodic context (GPC) from the expanded key/value branch and replace the MLP with a standard linear-activation stack.
  • (KV): reduce the composite query to the local evolution context (LEC) and replace the MLP with a standard linear-activation stack.
  • MLP: remove both the composite query and the expanded key/value branch, using standard self-attention instead.
  • (Q)+MLP: remove GPC from the key/value branch.
  • (KV)+MLP: reduce the composite query to LEC.
  • (Q)+(KV): replace the MLP with a standard linear-activation stack.
  • W/o All: remove all three components.
DatasetStepDCNetQKVMLPQ+MLPKV+MLPQ+KVW/o AllTQNetiTransformer
Metric-MSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAE
Electricity960.131/0.2210.134/0.2290.138/0.2330.145/0.2370.133/0.2270.136/0.2280.134/0.2290.145/0.2380.134/0.2290.148/0.240
-1920.150/0.2380.155/0.2470.156/0.2470.163/0.2540.153/0.2430.153/0.2420.152/0.2450.163/0.2560.154/0.2470.162/0.253
-3360.165/0.2540.168/0.2650.171/0.2640.177/0.2690.168/0.2660.166/0.2610.168/0.2650.178/0.2730.169/0.2640.178/0.269
-7200.192/0.2790.208/0.3010.204/0.2970.207/0.2980.199/0.2890.194/0.2830.221/0.3110.228/0.3120.201/0.2940.225/0.317
-Avg0.159/0.2480.166/0.2610.167/0.2610.173/0.2650.163/0.2560.162/0.2540.169/0.2630.179/0.2700.165/0.2590.178/0.270
PEMS03120.056/0.1550.058/0.1590.061/0.1590.064/0.1660.059/0.1600.059/0.1590.058/0.1590.064/0.1670.060/0.1610.071/0.174
-240.068/0.1710.075/0.1800.074/0.1780.081/0.1880.075/0.1810.072/0.1760.069/0.1740.082/0.1900.077/0.1820.093/0.201
-480.089/0.1960.102/0.2110.098/0.2040.114/0.2250.098/0.2060.098/0.2030.091/0.1980.114/0.2250.104/0.2150.125/0.236
-960.126/0.2290.144/0.2470.136/0.2360.154/0.2650.145/0.2460.133/0.2340.136/0.2350.162/0.2760.148/0.2530.164/0.275
-Avg0.084/0.1870.095/0.1990.092/0.1940.103/0.2110.094/0.1980.091/0.1930.089/0.1920.106/0.2150.097/0.2030.113/0.222
PEMS04120.063/0.1610.065/0.1640.067/0.1660.073/0.1760.064/0.1620.065/0.1630.065/0.1650.074/0.1760.067/0.1660.078/0.183
-240.072/0.1710.075/0.1770.078/0.1780.089/0.1950.075/0.1750.079/0.1790.074/0.1750.089/0.1950.077/0.1810.095/0.205
-480.083/0.1850.096/0.2060.091/0.1950.115/0.2230.091/0.1960.089/0.1920.086/0.1960.117/0.2270.097/0.2060.120/0.233
-960.097/0.2010.126/0.2390.107/0.2140.145/0.2520.123/0.2330.101/0.2080.104/0.2090.147/0.2560.123/0.2330.150/0.262
-Avg0.078/0.1790.091/0.1970.086/0.1880.106/0.2120.088/0.1920.084/0.1860.082/0.1860.107/0.2140.091/0.1970.111/0.221
PEMS07120.050/0.1400.052/0.1430.053/0.1450.056/0.1480.052/0.1420.051/0.1410.052/0.1430.057/0.1510.051/0.1430.067/0.165
-240.058/0.1500.064/0.1610.064/0.1580.069/0.1660.065/0.1630.061/0.1540.059/0.1520.073/0.1730.063/0.1590.088/0.190
-480.070/0.1630.080/0.1780.078/0.1710.091/0.1890.077/0.1730.077/0.1710.073/0.1680.097/0.2010.081/0.1790.110/0.215
-960.087/0.1800.099/0.1970.093/0.1880.113/0.2040.096/0.1920.094/0.1890.091/0.1870.123/0.2270.103/0.2030.215/0.245
-Avg0.066/0.1580.074/0.1700.072/0.1660.082/0.1770.073/0.1680.071/0.1640.069/0.1630.088/0.1880.075/0.1710.120/0.204
PEMS08120.067/0.1620.073/0.1720.068/0.1640.077/0.1710.071/0.1700.068/0.1630.069/0.1650.078/0.1740.071/0.1700.079/0.182
-240.083/0.1740.097/0.1920.086/0.1800.105/0.1970.093/0.1870.086/0.1780.083/0.1740.109/0.2020.096/0.1960.115/0.219
-480.109/0.1910.138/0.2240.117/0.2000.153/0.2310.139/0.2260.115/0.1980.115/0.1980.154/0.2380.149/0.2440.186/0.235
-960.138/0.2050.179/0.2520.149/0.2190.181/0.2560.166/0.2370.146/0.2140.147/0.2160.188/0.2630.253/0.3090.221/0.267
-Avg0.099/0.1830.122/0.2100.105/0.1910.129/0.2140.117/0.2050.104/0.1880.104/0.1880.132/0.2190.142/0.2300.150/0.226

(Due to space limitations, more content is presented here.)

The KV-only variant provides the largest single-component gain on all four PEMS datasets. On Electricity, the Q-only variant is slightly stronger than KV-only. The complete DCNet remains best on every dataset, indicating that the components are complementary rather than redundant.

We recognize TQNet as a strong and efficient prior method. Its two core components are: (i) a multi-head attention module that uses a periodic query and operates over the look-back dimension LL; and (ii) an MLP containing two linear layers and two activation functions, operating over the hidden dimension DD. This design is simple and effective, but its two modules use different feature dimensions. Consequently, the original block is less naturally suited to conventional module stacking, and this compact design may limit capacity scaling in large-scale scenarios.

DCNet instead adopts a tokenized inverted-Transformer structure operating at hidden dimension DD. Both the tokenized local context and periodic context participate in query construction and key/value retrieval.

We also note a difference in portability. TQNet's transfer experiments are closely tied to its attention block, for example through additive fusion in PatchTST and DLinear. In contrast, the DC technique combines local and periodic contexts at the representation level, so it can also be applied to purely linear backbones. For example, on PEMS08, the DC-augmented CycleNet outperforms TQNet in Table 3 / 6. The DC-improved DLinear variant also substantially outperforms the corresponding TQNet transfer variant. Therefore, our contribution is not merely a new basic attention unit, but a dual-context retrieval structure that can be transferred to linear backbones.

Since TQNet also uses periodic representations, we additionally compare DCNet and TQNet under noisy conditions. For a fair robustness comparison, we evaluate both models on two large datasets, PEMS03 and PEMS07, and two smaller datasets, ETTh1 and ETTm1. We use prediction length 12 for PEMS and 96 for ETT. Under these settings, DCNet does not substantially outperform TQNet on clean data, which enables a more conservative comparison of their relative degradation.

The complete noise results are as follows. SNR denotes signal-to-noise ratio.

ModelNoise levelETTh1---ETTm1---PEMS03---PEMS07---
--MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)
DCNetRaw0.366620.39290--0.306820.34938--0.056320.15533--0.050210.14018--
-40dB0.366630.392900.000.000.306840.349390.010.000.056330.155360.020.020.050220.140240.030.04
-30dB0.366740.393000.030.030.306960.349520.050.040.056410.155600.150.170.050330.140710.250.38
-20dB0.367920.394240.360.340.308320.351120.490.500.057200.157951.561.680.051430.144742.423.25
-10dB0.380770.407383.863.690.320710.366244.534.830.065990.1780117.1514.600.062770.1722125.0122.84
-5dB0.412690.4362212.5711.030.352830.3992614.9914.280.088990.2162158.0039.190.090680.2194180.6056.51
TQNetRaw0.371240.39302--0.311490.35267--0.060280.16078--0.051650.14353--
-40dB0.371440.393190.060.040.311420.35286-0.020.050.060330.160890.070.070.051700.143750.110.16
-30dB0.371860.393570.170.140.312000.353960.170.370.060550.161470.450.430.052000.144870.680.93
-20dB0.374850.396570.970.900.315000.358091.131.540.062950.167174.423.980.055060.154026.617.31
-10dB0.397150.418496.986.480.336300.382487.978.450.076720.1933427.2720.250.071530.1876238.5030.72
-5dB0.434430.4509617.0214.740.375810.4189520.6518.790.112160.2429686.0651.120.111000.24396114.9269.97

Finally, we agree that the introduced MLP is a standard SwiGLU structure. To the best of our knowledge, it has not been systematically adopted in multivariate time series forecasting (MTSF). Our ablation shows that it provides a small but consistent gain. We do not regard it as a core contribution and it occupies a limited portion of the paper, but we hope this choice offers a useful option for future work.

(Due to space limitations, more content is presented here.)

2. More direct evidence to justify the motivation.

We agree that a clearer attention map does not, by itself, demonstrate that the learned variable relations are more meaningful. To our knowledge, none of the twelve benchmark datasets provides a verified variable-level causal graph. Some PEMS datasets provide sensor adjacency or distance information, but that is a spatial structural prior, not a causal graph.We therefore use a controlled synthetic benchmark with a known dependency graph.

The latent variables follow zt+1=Azt+pt+ϵtz_{t+1}=Az_t+p_t+\epsilon_t, where AA is a known sparse graph and pt,i=γisin⁡(2πt/24+ϕi)p_{t,i}=\gamma_i\sin(2\pi t/24+\phi_i) is a periodic component with period T=24T=24. The observed input is xt=zt+Crut+ηtx_t=z_t+C_r u_t+\eta_t: the disturbance CrutC_r u_t changes the observed Pearson correlations across regimes but does not affect the future latent target. Thus, spurious correlations can be present in the input without being valid predictive edges.

We use 7 variables, 9 nonzero off-diagonal edges among 42 directed candidate pairs, look-back length 96, and prediction length 96. The experiment is repeated with three seeds. We construct a 7×77\times7 dependency matrix by perturbing one input variable at a time and measuring its first-step effect on each output variable. The resulting matrix is compared with the known graph. We report AUROC, the area under the receiver operating characteristic curve: it is the probability that a true edge receives a higher dependency score than a non-edge. Random ranking gives approximately 0.5; higher is better. We also report AUPRC, the area under the precision-recall curve, which focuses on true edges among the highest-ranked pairs. Since 9 of 42 candidate pairs are true, random ranking gives an AUPRC baseline of approximately 9/42=0.2149/42=0.214.

ModelTest MSETest MAEDependency AUROCDependency AUPRC
DCNet0.3899 +/- 0.01050.4934 +/- 0.00680.6790 +/- 0.07320.3621 +/- 0.0852
iTransformer0.3990 +/- 0.00920.4992 +/- 0.00580.5039 +/- 0.01660.2888 +/- 0.0302
TQNet0.4150 +/- 0.00860.5088 +/- 0.00550.5488 +/- 0.01010.2544 +/- 0.0070

DCNet obtains lower test MSE and MAE than both baselines. Its dependency AUROC is higher than iTransformer by 0.175 and than TQNet by 0.130. Its AUPRC is also higher than iTransformer by 0.073 and than TQNet by 0.108.

As an additional control, we evaluate both models with the correct phase and a shuffled phase.

ModelPhaseAUROCAUPRCMSE degradation
DCNetCorrect0.6790 +/- 0.07320.3621 +/- 0.0852-
DCNetShuffled0.5533 +/- 0.04570.3176 +/- 0.031332.31% +/- 8.80%
TQNetCorrect0.5488 +/- 0.01010.2544 +/- 0.0070-
TQNetShuffled0.5488 +/- 0.01010.2544 +/- 0.00700.00%

The TQNet temporal query and query/key projections collapse to nearly zero during training under this setting, making the learned query phase-insensitive.This also explains why TQNet, which incorporates periodic representation, has higher forecasting error than iTransformer. In contrast, phase corruption substantially degrades both DCNet's forecasting error and dependency recovery. These results indicate that DCNet effectively uses the phase-aligned periodic context and recovers the known dependency graph more closely than both a model without periodic context and the TQNet baseline.

Reviewer_gNDJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposed DCNet, and it's DC attention mixes local and periodic representations to form the query and concatenates them for key/value construction, together with SwiGLU MLP and RevIN. The model is evaluated on 12 real-world multivariate time series datasets, with claims of improved accuracy, computational efficiency, and robustness.

Critical strengths
  • The motivation is clear and is a reasonable direction for multivariate forecasting.
  • The empirical evaluation covers a broad set of real-world datasets and several strong baselines.
  • The DC design is non-invasive in output shape and can be combined with existing backbones.
Critical weaknesses

1.Method positioning is insufficiently clarified.

  • The DC query mixing and KV concatenation are closely related to context concatenation / memory attention, but the paper does not adequately analyze or discuss these connections.
  • Mechanistically, the design resembles “iTransformer + CycleNet periodic branch + concat/cross-attention.” The authors should clarify whether there is an essential difference and what the novel contribution of DCNet is beyond this combination.

2.Complexity analysis and efficiency experiments are insufficient.

  • The paper claims “excellent computational efficiency,” but the theoretical complexity is only described by a single sentence O(N^2), without memory complexity analysis or comparison with linear/sparse attention.
  • Figure 4(a) covers only one dataset and one prediction length, lacking cross-dataset and cross-hyperparameter efficiency experiments.
  • Table 7 does not report standard deviations over repeated runs. Moreover, since DCNet uses 2N KV tokens, its attention constant should be higher than that of iTransformer; yet Table 7 reports lower MACs (0.90G vs. 1.65G) and fewer parameters (2.30M vs. 5.15M). The authors need to explain whether the configurations are fair and provide an architecture-level analysis.
Questions for authors

1.Please provide more ablation experiments.

  • The current ablation only reports W/o DC and W/o Both, lacking component-level ablation inside DC.
  • The paper does not separately remove the GPC branch, the ββ-mixed query, or the KV concatenation.
  • It also lacks controls such as random period, shuffled period, and fixed non-learnable period, making it difficult to establish that the gains come from “dual-context collaboration” rather than extra periodic parameters or larger KV capacity.

2.Please provide more Robustness experiments for “excellent robustness.”

  • The evaluation covers only one dataset, one prediction length, and one noise type, which is too narrow.
  • Only iTransformer is compared, without robustness comparisons against more strong baselines.
  • There is no DCNet w/o DC control, so the contribution of DC cannot be isolated from RevIN or other modules.
  • At 5dB, DCNet’s MSE still degrades by about 40.95% and MAE by 34.93%; this can only be described as “relatively more stable,” not “excellent.”

3.Please add more key hyperparameter analyses.

  • The paper lacks sensitivity analyses for the lookback length L, number of attention heads h, number of layers, and hidden dimension D.
  • The efficiency impact across different d_model values is also insufficiently studied.
8 comments

Response to Reviewer gNDj

Thank you for your detailed and thoughtful review. We answer your concerns in the following four parts.

1. More fine-grained ablation study

We agree that the previous ablation analysis was insufficient. We have added the following component-level groups:

  • (Q): retain the composite query; remove the global periodic context (GPC) from the expanded KV branch and replace the new MLP with a standard linear-activation stack.
  • (KV): reduce the composite query to the local evolution context (LEC) and replace the new MLP with a standard linear-activation stack.
  • MLP: remove both the composite query and the expanded KV branch, using standard self-attention instead.
  • (Q)+MLP: remove GPC from the KV branch.
  • (KV)+MLP: reduce the composite query to LEC.
  • (Q)+(KV): replace the new MLP with a standard linear-activation stack.
  • W/o All: remove all three components.
DatasetStepDCNetQKVMLPQ+MLPKV+MLPQ+KVW/o AllTQNetiTransformer
Metric-MSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAEMSE/MAE
Electricity960.131/0.2210.134/0.2290.138/0.2330.145/0.2370.133/0.2270.136/0.2280.134/0.2290.145/0.2380.134/0.2290.148/0.240
-1920.150/0.2380.155/0.2470.156/0.2470.163/0.2540.153/0.2430.153/0.2420.152/0.2450.163/0.2560.154/0.2470.162/0.253
-3360.165/0.2540.168/0.2650.171/0.2640.177/0.2690.168/0.2660.166/0.2610.168/0.2650.178/0.2730.169/0.2640.178/0.269
-7200.192/0.2790.208/0.3010.204/0.2970.207/0.2980.199/0.2890.194/0.2830.221/0.3110.228/0.3120.201/0.2940.225/0.317
-Avg0.159/0.2480.166/0.2610.167/0.2610.173/0.2650.163/0.2560.162/0.2540.169/0.2630.179/0.2700.165/0.2590.178/0.270
PEMS03120.056/0.1550.058/0.1590.061/0.1590.064/0.1660.059/0.1600.059/0.1590.058/0.1590.064/0.1670.060/0.1610.071/0.174
-240.068/0.1710.075/0.1800.074/0.1780.081/0.1880.075/0.1810.072/0.1760.069/0.1740.082/0.1900.077/0.1820.093/0.201
-480.089/0.1960.102/0.2110.098/0.2040.114/0.2250.098/0.2060.098/0.2030.091/0.1980.114/0.2250.104/0.2150.125/0.236
-960.126/0.2290.144/0.2470.136/0.2360.154/0.2650.145/0.2460.133/0.2340.136/0.2350.162/0.2760.148/0.2530.164/0.275
-Avg0.084/0.1870.095/0.1990.092/0.1940.103/0.2110.094/0.1980.091/0.1930.089/0.1920.106/0.2150.097/0.2030.113/0.222
PEMS04120.063/0.1610.065/0.1640.067/0.1660.073/0.1760.064/0.1620.065/0.1630.065/0.1650.074/0.1760.067/0.1660.078/0.183
-240.072/0.1710.075/0.1770.078/0.1780.089/0.1950.075/0.1750.079/0.1790.074/0.1750.089/0.1950.077/0.1810.095/0.205
-480.083/0.1850.096/0.2060.091/0.1950.115/0.2230.091/0.1960.089/0.1920.086/0.1960.117/0.2270.097/0.2060.120/0.233
-960.097/0.2010.126/0.2390.107/0.2140.145/0.2520.123/0.2330.101/0.2080.104/0.2090.147/0.2560.123/0.2330.150/0.262
-Avg0.078/0.1790.091/0.1970.086/0.1880.106/0.2120.088/0.1920.084/0.1860.082/0.1860.107/0.2140.091/0.1970.111/0.221
PEMS07120.050/0.1400.052/0.1430.053/0.1450.056/0.1480.052/0.1420.051/0.1410.052/0.1430.057/0.1510.051/0.1430.067/0.165
-240.058/0.1500.064/0.1610.064/0.1580.069/0.1660.065/0.1630.061/0.1540.059/0.1520.073/0.1730.063/0.1590.088/0.190
-480.070/0.1630.080/0.1780.078/0.1710.091/0.1890.077/0.1730.077/0.1710.073/0.1680.097/0.2010.081/0.1790.110/0.215
-960.087/0.1800.099/0.1970.093/0.1880.113/0.2040.096/0.1920.094/0.1890.091/0.1870.123/0.2270.103/0.2030.215/0.245
-Avg0.066/0.1580.074/0.1700.072/0.1660.082/0.1770.073/0.1680.071/0.1640.069/0.1630.088/0.1880.075/0.1710.120/0.204
PEMS08120.067/0.1620.073/0.1720.068/0.1640.077/0.1710.071/0.1700.068/0.1630.069/0.1650.078/0.1740.071/0.1700.079/0.182
-240.083/0.1740.097/0.1920.086/0.1800.105/0.1970.093/0.1870.086/0.1780.083/0.1740.109/0.2020.096/0.1960.115/0.219
-480.109/0.1910.138/0.2240.117/0.2000.153/0.2310.139/0.2260.115/0.1980.115/0.1980.154/0.2380.149/0.2440.186/0.235
-960.138/0.2050.179/0.2520.149/0.2190.181/0.2560.166/0.2370.146/0.2140.147/0.2160.188/0.2630.253/0.3090.221/0.267
-Avg0.099/0.1830.122/0.2100.105/0.1910.129/0.2140.117/0.2050.104/0.1880.104/0.1880.132/0.2190.142/0.2300.150/0.226

(Due to space limitations, more content is presented here.)

The KV-only variant provides the largest single-component gain on all four PEMS datasets. On Electricity, the Q-only variant is slightly stronger than KV-only. The complete DCNet remains best on every dataset, indicating that the components are complementary rather than redundant.

2. Period-prior controls

DC is not intended as a new attention primitive. It introduces a phase-aligned periodic context into local variable-token retrieval. To verify that the gain comes from the useful periodic structure rather than simply from additional parameters, we introduce the following controls:

  • Random period: for an already trained model, replace its periodic matrix with a parameter-matched random matrix.
  • Shuffled period template: preserve the learned periodic values but independently permute the cycle dimension for each variable, destroying the learned phase arrangement.
  • Shuffled phase: preserve the original periodic matrix but randomize the phase index of each sample.
  • Fixed non-learnable period: make the periodic matrix non-learnable and train the model from scratch.
DatasetControlMSEMAEΔ MSE (%)ΔMAE (%)
ETTh1Original0.3666270.392901--
-Random period0.4175490.42381013.8893207.866867
-Shuffled template0.4139860.42163712.9174887.313801
-Shuffled phase0.4394030.44178019.85014712.440538
-Fixed non-learnable period0.3779980.3996013.1011511.705264
ETTm1Original0.3068260.349381--
-Random period0.3685050.39219520.10217812.254491
-Shuffled template0.3717960.39424821.17497112.841904
-Shuffled phase0.4067620.40445732.57085815.764135
-Fixed non-learnable period0.3237550.3623835.5174593.721439
PEMS03Original0.0563280.155336--
-Random period0.1067010.21660889.42909739.445434
-Shuffled template0.1028100.21283982.52212637.018579
-Shuffled phase0.1399310.249126148.42312560.378997
-Fixed non-learnable period0.0663780.17202817.84192610.745738
PEMS07Original0.0502150.140188--
-Random period0.0960910.21143591.36046450.822665
-Shuffled template0.0945710.20815288.33343248.481287
-Shuffled phase0.1147160.229978128.45124164.050174
-Fixed non-learnable period0.0624450.16299324.35527216.267441

The MSE and MAE of all control groups are higher than those of the original model. These results indicate that DCNet benefits from phase-aligned periodic information rather than relying solely on additional parameters.

3. Broader noise analysis

TQNet also uses periodic representations. We therefore add a comparison between DCNet and TQNet under noisy conditions. For a fair robustness comparison, we evaluate both models on two large datasets, PEMS03 and PEMS07, and two smaller datasets, ETTh1 and ETTm1. We use prediction length 12 for PEMS and 96 for ETT. Under these settings, DCNet does not substantially outperform TQNet on clean data, which makes the comparison of relative degradation more conservative.

(Due to space limitations, more content is presented here.)

ModelNoise levelETTh1---ETTm1---PEMS03---PEMS07---
--MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)MSEMAEΔMSE (%)ΔMAE (%)
DCNetRaw0.366620.39290--0.306820.34938--0.056320.15533--0.050210.14018--
-40dB0.366630.392900.000.000.306840.349390.010.000.056330.155360.020.020.050220.140240.030.04
-30dB0.366740.393000.030.030.306960.349520.050.040.056410.155600.150.170.050330.140710.250.38
-20dB0.367920.394240.360.340.308320.351120.490.500.057200.157951.561.680.051430.144742.423.25
-10dB0.380770.407383.863.690.320710.366244.534.830.065990.1780117.1514.600.062770.1722125.0122.84
-5dB0.412690.4362212.5711.030.352830.3992614.9914.280.088990.2162158.0039.190.090680.2194180.6056.51
TQNetRaw0.371240.39302--0.311490.35267--0.060280.16078--0.051650.14353--
-40dB0.371440.393190.060.040.311420.35286-0.020.050.060330.160890.070.070.051700.143750.110.16
-30dB0.371860.393570.170.140.312000.353960.170.370.060550.161470.450.430.052000.144870.680.93
-20dB0.374850.396570.970.900.315000.358091.131.540.062950.167174.423.980.055060.154026.617.31
-10dB0.397150.418496.986.480.336300.382487.978.450.076720.1933427.2720.250.071530.1876238.5030.72
-5dB0.434430.4509617.0214.740.375810.4189520.6518.790.112160.2429686.0651.120.111000.24396114.9269.97

DCNet performs better than TQNet, which also uses periodic representations. Although DCNet exhibits a noticeable performance drop at 5dB, we consider this acceptable because such a low SNR would render most practical sensor streams nearly unusable.

4. Additional hyperparameter analysis

Look-back length. We compare five models with six input lengths (48,96,192,336,504,720)(48,96,192,336,504,720) on Traffic, with prediction length 96.

-DCNet-EMAformer-TQNet-CycleNet-iTransformer-
LMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE
480.4640.2840.4590.2480.4690.2790.5320.3110.4840.303
960.3840.2570.3970.2380.4130.2610.4580.2970.3950.268
1920.3490.2460.3780.2350.3770.2540.4060.2750.3640.258
3360.3430.2440.3660.2350.3670.2520.3870.2680.3550.258
5040.3430.2450.3750.2400.3690.2660.3820.2700.3540.257
7200.3430.2460.3640.2370.3970.2960.3720.2610.3490.256

Number of attention heads. We compare four datasets. ETT uses prediction length 96 and PEMS uses prediction length 12.

-ETTh1-ETTm1-PEMS03-PEMS07-
HeadsMSEMAEMSEMAEMSEMAEMSEMAE
10.3640.3900.3120.3530.0570.1560.0500.139
20.3640.3900.3100.3520.0560.1550.0500.138
40.3660.3920.3060.3490.0560.1550.0500.140
80.3670.3930.3040.3480.0560.1540.0500.139

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.