acceptodds
Under review as a conference paper at ICLR 2027

Shared-Memory Axis Inversion for Long-term Time Series Forecasting

Abstract

Multivariate long-term series forecasting (LTSF) involves both temporal and variate axes, whereas efficient sequence models typically model only one axis explicitly. We characterize the resulting resolution–efficiency trade-off in terms of native-step alignment and cross-axis coupling cost, and formulate Step-indexed Completion (SInC) as a design criterion. To realize SInC, we propose DAMIE, which models the two axes sequentially: it first builds shared temporal memory indexed by native time steps, then uses Flip Attention to form a variate-to-time alignment distribution for each target variate and read out its representation from the memory. Across 11 datasets from five benchmark families, DAMIE achieves strong forecasting performance and a favorable accuracy–efficiency trade-off, particularly in settings with many variates. Further analyses indicate that step-indexed cross-axis readout helps target variates select relevant historical information and connect temporal and variate representations.

Primary area:
learning on time series and dynamical systems
Keywords:
Long-term Time Series Forecasting, Axis Inversion, Flip Attention, Step-indexed Completion

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Xy5EOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes DAMIE, a memory-efficient architecture for long-term multivariate time-series forecasting. The paper introduces Step-Indexed Completion (SInC), a design criterion that enables cross-axis modeling within a single-axis computational budget while preserving native-axis-level alignment. DAMIE instantiates SInC through a shared temporal memory and Flip Attention, allowing each target variate to selectively aggregate information from individual time steps, together with Taylor-form residual completion for further refined representations. Experiments across multiple datasets and forecasting horizons show that DAMIE achieves strong overall forecasting performance compared with existing multivariate time-series forecasting baselines while demonstrating memory efficiency.

Critical strengths

S1. Memory-efficient cross-axis modeling. DAMIE enables effective cross-axis modeling within a single-axis budget by combining a shared temporal memory with Flip Attention. The efficiency results in Section 4.3 (Q1) show that this design substantially reduces memory consumption compared with existing multivariate time-series forecasting baselines. It provides a practical way to capture both temporal and cross-variate dependencies without the memory cost of explicit two-axis modeling.

S2. Robust forecasting performance. DAMIE achieves the best overall forecasting performance across diverse datasets and prediction horizons, compared with both Transformer-based and non-Transformer baselines. It also remains robust across different combinations of lookback-window and prediction-horizon lengths. (Section 4.3, Q2) Together, these results suggest that its gains generalize across forecasting settings rather than depending on a particular dataset, horizon, or input length.

Critical weaknesses

W1. Incomplete empirical comparison and scaling analysis. The main comparison omits several recent and relevant long-term forecasting methods, such as TimeMixer and TimeFilter. This makes it difficult to determine whether DAMIE’s reported gains persist against stronger baselines. Moreover, the paper does not provide a controlled analysis of how its accuracy and efficiency scale with the variate dimension although the proposed cross-axis coupling can explicitly depend on the number of variates.

W2. Limited evidence for the generality of SInC. DAMIE is evaluated only in the time-major configuration, where the time axis forms the shared memory and variates serve as targets. The paper therefore provides limited evidence that the proposed principle generalizes to alternative axis choices.

Questions for authors

Q1. Could you compare DAMIE with more recent long-term forecasting baselines, particularly such as TimeMixer and TimeFilter? Results for both forecasting accuracy and memory efficiency would materially strengthen the empirical claims. Please refer TSlib, which is the github repository including a lot of recent long-term forecasting baselines.

Q2. Could you evaluate a variate-major counterpart in which the variate axis forms the explicitly indexed shared memory, under a matched parameter and computational budget?

Q3. How do DAMIE’s accuracy, memory consumption, and runtime scale with the number of variates? A controlled experiment that varies the variate dimension while holding the temporal setting and other properties fixed would help establish whether the claimed efficiency and effectiveness persist in both low- and high-dimensional settings.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 3

We thank the reviewer for the thoughtful questions and constructive feedback.

Q1. Additional Baselines

We further compare DAMIE with TimeMixer and TimeFilter on both low- and high-dimensional multivariate time-series datasets.

DatasetDAMIE MSEDAMIE MAETimeMixer MSETimeMixer MAETimeFilter MSETimeFilter MAE
ETTh20.3620.3950.3670.3980.3650.398
ECL0.1740.2630.1860.2780.1610.259
Traffic0.4070.2650.4840.2970.4090.260

The computational results are reported as Size (Time), measured in GB (s/iter).

DatasetDAMIETimeMixerTimeFilter
ETTh20.43 (0.0109)0.42 (0.0271)0.54 (0.0121)
ECL0.74 (0.042)2.8 (0.108)5.5 (0.156)
Traffic1.7 (0.089)15.5 (0.487)6.8 (0.207)

DAMIE outperforms TimeMixer across all three datasets and achieves comparable forecasting accuracy to TimeFilter. In efficiency, DAMIE achieves substantial memory and runtime savings on the high-dimensional datasets.

These results demonstrate DAMIE's favorable trade-off and validate the effectiveness of explicit cross-axis alignment under a single-axis budget. We have also included recent competitive models, including SOFTS, S-Mamba, and TimePro, in our baseline comparisons (In the main result and appendix).

Q2: Indexing Order

In theory, prior studies suggest that variate-token (N×DN\times D) representations are well suited to Transformer-based forecasting architectures (e.g., iTransformer).

Experimentally, We design and evaluate a Variable-Major indexing scheme. Under matched parameter counts and computational budgets, we compare Variable-Major (encoder outputs L×DL\times D) and Time-Major indexing (encoder output N×DN\times D).

PTime-Major MSETime-Major MAEiTransformer MSEiTransformer MAEVariable-Major MSEVariable-Major MAE
960.2880.3410.3000.3490.3320.379
1920.3560.3880.3790.3980.4030.415
3360.3990.4190.4220.4310.4460.454
7200.4060.4300.4280.4460.4500.459
Avg0.3620.3950.3820.4060.4070.426

The results show that Time-Major indexing achieves the best forecasting performance. This suggests that Time-Major indexing organizes temporal information more effectively within Transformer-style architectures, consistent with the itransformer conclusion.

Therefore, shared-memory indexing affects not only information access and interaction patterns but also forecasting performance, making Time-Major indexing a more effective choice for DAMIE.

Q3. Scalability Analysis

DAMIE models interactions across both variate (NN) and temporal (LL) dimensions. We therefore evaluate its forecasting performance and computational efficiency along both dimensions.

(1) Variate Dimension. *Directly varying the number of variates within a dataset disrupt existing inter-variate dependencies and affect forecasting results. * We therefore evaluate datasets with different variate dimensions: ETTh2 (N=7N=7), ECL (N=321N=321), and Traffic (N=862N=862). We use iTransformer as the baseline under identical settings.

DatasetDAMIEiTransformer
ETTh20.43 (0.0109)0.36 (0.087)
ECL0.74 (0.042)0.85 (0.046)
Traffic1.7 (0.089)4.33 (0.107)

The computational results are reported as Size (Time), measured in GB (s/iter).

The main forecasting results show that DAMIE achieves better accuracy than iTransformer. In efficiency, DAMIE incurs lower memory consumption and runtime on high-dimensional datasets, demonstrating favorable scalability with increasing variate dimensions.

(2) Temporal Dimension (LL). We further evaluate computational efficiency across different lookback lengths, using a standard Transformer with the same configuration as the baseline on ETTh2.

Lookback LengthDAMIETransformer
960.429 (0.0109)0.613 (0.0208)
1920.535 (0.0137)0.885 (0.0245)
3360.865 (0.0218)1.507 (0.0323)
7202.493 (0.0415)4.547 (0.0605)

DAMIE consistently incurs lower memory consumption and runtime as the lookback length increases, demonstrating favorable scalability along the temporal dimension.

*Theoretical Analysis. These results are consistent with the interaction complexity (in Table 12). DAMIE adopts a constrained interaction structure to reduce computational and memory overhead, enabling favorable scalability along both dimensions.

Thank you for the detailed responses and additional experiments. They helped address several of my concerns.

However, the Q2 results leave me uncertain about how broadly SInC can be instantiated with different choices of explicit axis. Furthermore, although the suggested method, DAMIE, demonstrates effective forecasting performance, it still appears to combine several established components in a relatively complex architecture, so my concern about its methodological distinctiveness remains.

For these reasons, I will maintain my score.

Generality

The generality of SInC does not rely on instantiating multiple models, but on its architecture-agnostic abstraction of cross-axis information transfer under a single-axis budget (section 2). Through the View–Weights–Readout interface, it formalizes the trade-off between native time-step resolution and cross-axis interaction cost as an explicit design constraint. DAMIE is a concrete Transformer-style implementation following this principle.

Clarification.

We cite iTransformer and RLinear to draw on prior studies demonstrating the effectiveness of variate-level representations ((N\times D)) in multivariate time series forecasting. Accordingly, the performance differences between Variable-Major and Time-Major should also be understood in light of their output representations. The (N\times D) variate-level output representation is also an important factor influencing these differences, providing a theoretical rationale for choosing Time-Major.

Note: These models are cited solely as related work and external baselines; their components are not incorporated into DAMIE.

We hope this clarification can address the your concerns.

Thank you for the clarification.

I understand your points, but my concerns remain. I will maintain my weak acceptance recommendation.

Reviewer_h6quOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to model temporal and cross-variate dependencies efficiently in multivariate time series forecasting. The authors propose SInC as a design principle for maintaining access to individual historical time steps without explicitly modeling a two-dimensional time-variate token grid. To implement this, DAMIE first encodes the temporal dimension into shared memory and then uses Flip Attention to aggregate historical information into variate representations. The main contribution is an efficient way to connect temporal and variate representations through cross-attention. Experiments on 11 datasets demonstrate competitive forecasting accuracy, particularly with many variates, while ablation and efficiency analyses provide evidence for the effectiveness of the architecture.

Critical strengths

S1. The paper addresses a meaningful problem in multivariate time series forecasting. The trade-off between fine-grained temporal modeling and computational efficiency is well motivated, particularly for datasets with many variates.

S2. The proposed architecture provides a simple and efficient approach to cross-axis information aggregation. By combining shared temporal memory with Flip Attention, DAMIE allows each variate to selectively retrieve historical information without maintaining an expensive two-dimensional token grid.

S3. The method achieves promising forecasting performance with favorable computational efficiency. DAMIE performs competitively across multiple benchmarks, with particularly strong results on high-dimensional datasets. The reported runtime and memory usage further demonstrate its practical potential.

Critical weaknesses

While the problem is meaningful and the proposed architecture is reasonable, my main concerns are the gap between the motivation and the actual design, and the lack of experiments that directly validate the claimed benefits. I also have a minor concern about the efficiency evaluation.

W1. The claimed ability to capture fine-grained cross-axis dependencies is not fully supported by the proposed design. As formulated in Eq. (1), the paper aims to model how the prediction of variate nn at future step pp depends on historical variate mm at time step tt. However, DAMIE first mixes information across variates through linear projection and across time steps through temporal self-attention. Flip Attention then assigns weights to historical time positions for each target variate, rather than explicitly modeling dependencies between specific historical variates and time steps. Thus, preserving native-step indices does not necessarily preserve the fine-grained cross-axis dependencies emphasized in the motivation.

W2. The experiments do not sufficiently isolate the benefits of the proposed design. The current ablations cannot clearly distinguish the contributions of native-step resolution and Flip Attention from those of combining temporal and variate modeling or other architectural choices. Controlled comparisons with patch-based memory, standard cross-attention, and dual-axis self-attention would help clarify the sources of improvement.

W3. The efficiency evaluation does not sufficiently demonstrate scalability across different sequence lengths. The experiments mainly use a fixed lookback length of 96, while the efficiency comparison is limited to two datasets. A more systematic analysis varying both the lookback length and the number of variates would better support the claimed efficiency advantages.

Questions for authors

Q1. Although the shared memory retains LL tokens, each token has already mixed information across variates and time steps. How can the attention weights over these tokens be interpreted as fine-grained access to the original historical time steps? Could the authors explain how the model distinguishes information from different historical time steps and variates after such mixing? If this cannot be established, the claims about fine-grained dependency modeling should be clarified or appropriately narrowed.

Q2. The current ablations do not sufficiently isolate the benefits of the two core design choices.

(1) Is native-step resolution really necessary? The current ablations do not examine intermediate temporal resolutions. Comparisons with DAMIE variants using different temporal patch sizes would help establish whether native-step resolution provides meaningful benefits over coarser representations.

(2) Are the improvements really attributable to Flip Attention? The Time-only and Variate-only settings in Table 4 are not sufficiently described. For example, the Time-only encoder produces L×DL\times D representations, whereas the full model predicts from N×DN\times D variate representations. Generating predictions of shape P×NP\times N therefore requires an additional transformation or a different prediction head. The exact architectures and readout mechanisms should be disclosed to establish that the comparisons are fair. More importantly, the experiments should demonstrate that the improvements arise from the proposed cross-axis attention mechanism, rather than simply incorporating both temporal and variate representations. For example, comparing Flip Attention with a simpler alternative that combines temporal and variate representations through simple feature fusion (e.g., concatenation followed by an MLP) would help isolate its contribution.

Q3. How does the accuracy–efficiency trade-off change as the lookback length LL increases? Figure 5 examines forecasting accuracy across different lookback lengths but does not report the corresponding computational costs. DAMIE retains LL temporal tokens and uses temporal self-attention with O(L2)O(L^2) complexity, so native-step resolution may become increasingly costly for longer sequences. It would be helpful to examine whether its accuracy gains justify this cost as LL grows, and when coarser temporal representations might become preferable.

3 comments
AuthorsOpenReview ↗

Response to Reviewer: We Hope Our Responses Address Your Concerns

We thank the reviewer for the feedback and address each concern below..

Q1. Fine-grained

Fine-grained retrieval refers to variate-conditioned selective reading from shared memory that preserves native temporal resolution, rather than independently modeling each variable at every historical time step. It is reflected in two aspects:

(1) Native-step Indexing: Given an input X∈RL×NX\in\mathbb{R}^{L\times N}, Temporal Self-Attention generates shared memory indexed by the original time steps, preserving all LL temporal positions.

(2) Variate-conditioned Alignment: T2V-Q constructs new queries (eq.7), establishing an N×LN\times L cross-axis alignment space where each target variate has its own attention weights over all historical time steps, enabling selective retrieval. Fig. 6 illustrates these alignment patterns.

Q2. Design Benefits

Q2.1. Necessity of Native-step Resolution**

Theoretically, theorem A.1 proves that information aggregation may incurs irreducible prediction errors under specific coarse-grained readout constraints. Thus, native temporal resolution preserves step-level distinguishability and adaptive selection, rather than merely increasing the number of tokens.

Empirically, we evaluate different patch sizes, where each patch aggregates consecutive time steps by averaging, reducing the temporal length from LL to L/pL/p positions.

PSize=1Size=2Size=4Size=8
960.288 (0.341)0.294 (0.345)0.295 (0.346)0.298 (0.352)
1920.356 (0.388)0.369 (0.392)0.376 (0.394)0.380 (0.398)
3360.399 (0.419)0.409 (0.418)0.411 (0.423)0.419 (0.429)
7200.406 (0.430)0.417 (0.439)0.419 (0.438)0.428 (0.442)
Avg0.362 (0.395)0.372 (0.399)0.375 (0.401)0.381 (0.405)

The results show an overall decline in forecasting performance as patch size increases. Native-step resolution achieves the best performance across all prediction horizons.

Q2.2. Flip Attention

First, all ablation variants use linear prediction heads tailored to their respective representation dimensions, without additional nonlinear prediction modules. Thus, Table 4 adopts progressive ablations to assess the contributions of individual components, rather than differences in prediction head designs.

Second, to evaluate the effectiveness of the query construction in Flip Attention, we compare three strategies: fixed learned variate queries (Learned), which lack input dependence; queries derived solely from shared temporal memory (Query); and the proposed T2V-Q.

POurs MSEOurs MAELearned MSELearned MAEQuery MSEQuery MAE
960.2960.3480.3110.3560.3450.384
1920.3780.4050.3960.4210.4120.435
3360.4150.4380.4350.4580.4460.457
7200.4360.4510.4570.4630.4890.516
Avg0.3810.4110.4000.4260.4210.448

T2V-Q achieves the best results across all prediction horizons, validating the contribution of T2V-Q to cross-axis information retrieval.

Finally, DAMIE performs sequential cross-axis modeling along the L×D→N×DL\times D\rightarrow N\times D : Variate representations are generated from temporal representations, rather than being independently modeled in parallel. To further validate the necessity of cross-axis interaction, we replace the T2V-Q with an MLP. Results are reported as MSE (MAE).

PiTransformerDAMIE (MLP)DAMIE (T2V-Q)
960.300 (0.349)0.331 (0.376)0.288 (0.341)
1920.379 (0.398)0.420 (0.423)0.356 (0.388)
3360.422 (0.431)0.460 (0.459)0.399 (0.419)
7200.428 (0.446)0.482 (0.476)0.406 (0.430)
Avg0.382 (0.406)0.423 (0.433)0.362 (0.395)

The MLP alternative performs substantially worse than T2V-Q, indicating that DAMIE's gains arise from the cross-axis alignment and variate-conditioned information retrieval enabled by Flip Attention.

Q3. Lookback Efficiency

To assess the computational cost of lookback windows, we compare DAMIE against a single Transformer with the same configuration as a single-axis baseline. DAMIE results are in parentheses.

Lookback (L)Memory (GB)Time (ms/iter)
960.613 (0.429)0.0208 (0.0109)
1920.885 (0.535)0.0245 (0.0137)
3361.507 (0.865)0.0323 (0.0218)
7204.547 (2.493)0.0605 (0.0415)

DAMIE consistently incurs lower memory and runtime across all lookback lengths. Together, Fig. 5 and the efficiency analysis show that DAMIE maintains favorable forecasting performance with lower computational costs as the lookback length increases.

Dear Reviewer h6qu,

Thank you again for your feedback. As the discussion period is approaching its conclusion, we would like to briefly summarize the additional experiments and analyses provided in our response to address your three main concerns:

  • Fine-grained Temporal Retrieval Mechanism (Q1): We further clarified the mechanisms of Native-step Indexing and Variate-conditioned Alignment, explaining how DAMIE preserves the original temporal resolution and enables fine-grained, variate-conditioned Alignment of historical information.
  • Effectiveness of Key Design Choices (Q2): We conducted additional comparisons across different temporal resolutions, query construction strategies, and an MLP-based alternative, further validating the necessity and effectiveness of T2V-Q.
  • Computational Efficiency and Scalability (Q3): We provided additional comparisons against a standard Transformer across different lookback lengths. The results demonstrate that DAMIE consistently achieves lower memory consumption and runtime across all tested lengths.

As the discussion period is scheduled to conclude in approximately 12 hours, we would sincerely appreciate it if you could take a moment to review our responses. We fully understand that you may have a busy schedule, and we sincerely appreciate your time and feedback.

Best regards, The Authors

Dear Reviewer,

Please engage in the discussion with authors, thank you.

AC

Reviewer_VVDyOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Multivariate forecasting is a different problem than univariate forecasting, as the interactions between variates need to be modeled as well. This is exactly what the authors propose in the paper. They claim that efficient sequence models typically only model one axis explicitly, and they try to bridge this gap by proposing DAMIE, which models the two axes sequentially. Hereby, it constructs a shared memory and uses Flip Attention to form a variate-to-time alignment distribution for each target variate and read out its representation from it. They outperform the selected subset of baselines on the standard TSLib and PEMS benchmarks, while also being efficient.

Critical strengths

1. Efficiency trade-off. The authors showcase the efficiency trade-off quite decently (Figure 4) and showcase their advantage even compared to a state-space model (S-Mamba): on Traffic, DAMIE has a lower MSE with less peak memory (1.73 vs 2.85 GB), and on ECL, it is on par in MSE (0.141 vs 0.140, Table 10) with slightly lower training time and memory.

2. Coherent architecture. I think the architecture is practically motivated and coherent.

Critical weaknesses

DAMIE is a sensible forecasting architecture with competitive results, but the paper makes a stronger case for the complete model's usefulness than for its proposed general principle, "Step-Indexed Completion." The most substantial concerns are the complexity analysis and the lack of experiments that isolate why the proposed mechanism helps.

1. Complexity analysis. The stated per-layer complexity of DAMIE is O(L² + NL), but as far as I can tell, it does not include all components. The gating projection g in Eq. 7 maps each variate's length-L history to a length-L coefficient vector, i.e., a linear layer over the L time steps applied to each of the N variates, which costs O(NL²). Appendix E also describes TFRC as producing an M × N "feature–time view" with M > L, whereas Sec. 3.3 defines TFRC on the N × D variate representations. For the baselines, Crossformer is listed with O(NS² + SN²), although its cross-dimension stage uses a small fixed number of routers, which makes it linear in N. In addition, SOFTS, the closest competitor in accuracy and also linear in N, is missing from Figure 4.

2. No experiment isolates why the mechanism helps. The readout in Flip Attention is cross-attention from variate queries to the shared temporal memory; what is new is the query construction. The ablation in Table 4 adds components incrementally (time-only, variate-only, Flip, FFN, TFRC), but it has no variant that keeps the shared temporal memory and uses standard queries instead, for example the variate embeddings X_var from Eq. 11 or learned per-variate queries. The lookback analysis (Figure 5) only compares against iTransformer on ETTh2, although the claimed advantage is in settings with many variates. On exactly these datasets (ECL, Traffic, PEMS03/04/07/08), the margin over SOFTS is relativly small, and no standard deviations over seeds are reported. Finally, SInC is presented as a general design principle, but it is instantiated only once, and the theorems in Appendix A are counterexamples for restricted scalar readouts that, as the authors note themselves, do not establish necessity for general architectures.

Questions for authors

Clarification requests

  1. Could the authors discuss/evaluate the relevant xLSTM-Mixer (Kraus et al., 2025, NeurIPS)? I think, in general, the xLSTM architecture is missing from the comparison, as it is efficient by design, similar to state-space models. Also, xLSTM-Mixer models time and variates in a sequential manner, so it is definitely worth including/discussing.
  2. I am a bit unsure about Equation 7 and the structure of g. What is the complete complexity after including it? Given that we are talking about complexity, can the authors outline why Crossformer has an SN² complexity for the variable interaction (Table 12)? As far as I recall, standard Crossformer uses a fixed number of routers, which reduces it from quadratic to linear.
  3. As far as I can see, your proposed readout is very similar to cross-attention, and you only alter the query construction; there is no experiment that showcases the empirical benefit of it. Could you comment on this? If not, a simplification might be beneficial.

Suggestions for presentation

  • Figure 5 is just too small.
  • Line 470 (Related Work): a period is missing, and a reference did not resolve (we see a "?").
  • I appreciate the authors moving a lot of related work to the appendix, but given the sheer amount of related work, I don't think it is a good idea to only discuss 6 works in the main paper.
  • The figure text in general is extremely small; just using subfigures (e.g., Figure 1) and placing the subfigure titles in the caption would already help.
  • Smaller issues: an unresolved citation command ("S-Mamba citewang2025mamba", line 931); the abstract mentions 11 datasets, while Appendix D.1 says six; and the "Gain vs. M/I" is averaged over six datasets (line 1123), while Table 3 refers to seven benchmarks.

References

  • Kraus, Divo, Dhami & Kersting (2025). xLSTM-Mixer: Multivariate Time Series Forecasting by Mixing via Scalar Memories. NeurIPS 2025.
4 comments
AuthorsOpenReview ↗

Response to the Reviewers: We Hope Our Responses Address Your Concerns

We thank the reviewer for the detailed and constructive suggestions. We address each point below.

Q1. xLSTM-Mixer vs. DAMIE

xLSTM-Mixer (Kraus et al., NeurIPS 2025) and DAMIE fundamentally differ in their sequence modeling axes.

  • SInC is a design principle for budget-efficient cross-axis information transfer, instantiated by DAMIE in a Transformer-style architecture.

  • xLSTM-Mixer follows the variate-token paradigm, performing independent temporal mixing before modeling N×DN \times D variate tokens.

In particular, sequential modeling strategy of xLSTM-Mixer does not consider the trade-off of cross-axis interaction cost. We further evaluate them under the same settings (L=96,P=96L=96, P=96).

DatasetMSEMAEMemory (GB)Time (s/iter)
ECL0.149 (0.141)0.237 (0.233)1.9 (0.74)0.662 (0.042)
Traffic0.396 (0.378)0.262 (0.250)4.8 (1.7)1.65 (0.089)

The results show that DAMIE outperforms xLSTM-Mixer in both forecasting accuracy and efficiency on high-dimensional multivariate datasets, consistent with the limitation acknowledged in the original paper of xLSTM-Mixer.

We will therefore include xLSTM-Mixer in the Related Work to further highlight the need for efficient cross-axis information transfer.

Q2. Complexity Analysis

Table 12 reports the dominant interaction complexity of different architectures rather than full-model FLOPs, following established practice in Informer (AAAI 2021), Crossformer (ICLR 2023), and SOFTS (NeurIPS 2024).

##(1) The gg in Eq. (7)##

gg is a learnable temporal coefficient generator based on historical observations, used to construct variate-conditioned queries.

In implementation, gg involves neither inter-variate information exchange nor additional 2D token interactions. It serves as an auxiliary temporal coefficient generation step, while cross-axis coupling is performed by Eqs. (8)–(10).

Thus, it is excluded from the cross-axis interaction complexity.

(2) Crossformer Complexity

We thank the reviewer for pointing this out. Upon re-examination, the complexity of Crossformer is O(NS2+cSN)O(NS^2+cSN). We will accordingly revise Table 12 to ensure consistency with the original paper.

Q3. Readout and T2V-Q Ablation

Although DAMIE and standard Cross-Attention share similar Q-K-V computations, they differ fundamentally in information organization and query construction.

In theory. Inspired by LLM-style KV Memory Retrieval, DAMIE enables cross-axis information transfer by generating variate representations from temporal memory rather than fusing.

In implementation. our queries qq are generated by gg in Eq. (7). Motivated by SInC, T2V-Q addresses the need for both input dependence and target conditioning in step-indexed evidence selection (Appendix A).

We compare three query construction strategies: fixed learned variate queries (Learned), which lack input dependence; queries derived solely from shared temporal memory (Query); and Our T2V-Q.

PT2V-Q MSET2V-Q MAELearned MSELearned MAEQuery MSEQuery MAE
960.2960.3480.3110.3560.3450.384
1920.3780.4050.3960.4210.4120.435
3360.4150.4380.4350.4580.4460.457
7200.4360.4510.4570.4630.4890.516
Avg0.3810.4110.4000.4260.4210.448

T2V-Q consistently achieves the best results across all prediction horizons, validating the contribution of our queries to shared memory retrieval.

Other weakness

    1. Additional experiments show that SOFTS achieves computational efficiency comparable to DAMIE (DAMIE results in parentheses).
DatasetMemory (GB)Time (s/iter)
Traffic1.79 (1.73)0.092 (0.089)
ECL0.72 (0.74)0.044 (0.042)

These results further demonstrate DAMIE's competitive accuracy–efficiency trade-off while enabling dual-axis information transfer.

    1. Results across Random Seeds
DatasetMSEMAE
PEMS030.101 ± 0.0030.208 ± 0.002
PEMS040.099 ± 0.0020.203 ± 0.003
PEMS070.085 ± 0.0030.185 ± 0.003
PEMS080.137 ± 0.0020.201 ± 0.002
Traffic0.407 ± 0.0020.265 ± 0.001
ECL0.174 ± 0.0020.263 ± 0.002
    1. Generality of SInC

The generality of SInC does not rely on instantiating multiple models, but on its architecture-agnostic abstraction of cross-axis information transfer under a single-axis budget (section 2). Through the View–Weights–Readout interface, it formalizes the trade-off between native time-step resolution and cross-axis interaction cost as an explicit design constraint. DAMIE is a concrete Transformer-style implementation following this principle.

Thank you for running the additional experiments. The learned-query ablation directly addresses part of my concern, and the xLSTM-Mixer and SOFTS comparisons are strengthening the work. Unfortunately, the authors missed to upload a revised pdf with the changes made, which makes judging the final paper a bit difficult.

My main complexity question remains open. Eq. 7 describes a dense LL-to-LL projection applied to each of the NN variates, which costs O(NL2)O(NL^2) per example each time it is evaluated. Would it be possible to report the full forward-pass complexity including this term and clarify whether gg is recomputed per layer and also label the interaction-only bound explicitly.

For the query ablation i think it would be great to see standard cross-attention using XvarX_{\mathrm{var}} from Eq. 11 as the queries with the same shared keys and values, ideally on ECL or Traffic as this is where the paper emphasizes its advantages.

We thank the reviewer for the follow-up questions and address them below.

  • 1 Revised Manuscript.

We have resubmitted the revised manuscript.

    1. In the full computational complexity, gg consists of two linear layers, each mapping from LL to LL, with GELU activation. Its parameter complexity is O(L2)O(L^2), while its computational complexity per invocation is O(BNL2)O(BNL^2). In the implementation, gg is recomputed at each interaction layer.

Accordingly, we provide a module-wise Computational complexity analysis of DAMIE, with the Computational complexity reported per module invocation.

ModuleOperationComputational Complexity
Temporal Self-AttentionMulti-head self-attention over B×L×DB\times L\times DO(B(LD2+L2D))O(B(LD^2+L^2D))
Query ConstructionTwo Linear(L,LL,L) layers with GELUO(BNL2)O(BNL^2)
T2V-QTemporal query rewriting(O(BNLD))
AlignmentQuery-Key matching and Softmax(O(BNLD))
ReadoutShared temporal value retrieval(O(BNLD))
Variate FFNTwo-layer feature transformationO(BNDDf)O(BNDD_f)
Taylor CompletionLinear(D,RDD,RD) projection and Taylor aggregationO(BND2)O(BND^2)

Note that the original (O(L^2+NL)) expression captures only the core interaction complexity, rather than the complete computational complexity of all modules. We have further added a module-wise computational complexity analysis in Appendix E of the revised manuscript, distinguishing it from the interaction complexity.

    1. We further introduce a standard Cross-Attention baseline using (X_{\mathrm{var}}) as Query while keeping the shared Keys and Values unchanged.
PT2V-Q MSET2V-Q MAELearned MSELearned MAECross MSECross MAE
960.2960.3480.3110.3560.3010.349
1920.3780.4050.3960.4210.3860.402
3360.4150.4380.4350.4580.4360.445
7200.4360.4510.4570.4630.4460.457
Avg0.3810.4110.4000.4260.3920.413

**The results show that Cross-Attention improves upon Learned Query, while T2V-Q achieves better performance. **

We further evaluate this comparison on ETTh2, ECL, and Traffic. T2V-Q consistently outperforms standard Cross-Attention on all three datasets, supporting the effectiveness of our query construction mechanism.

| Dataset | Cross-attention | T2V-Q | |:---:|:---:|:---:|:---:| | ETTh2 | 0.381 (0.403) | 0.362 (0.395) | | ECL | 0.182 (0.275) | 0.174 (0.263) | | Traffic | 0.455 (0.293) | 0.407 (0.265) |

All values are averaged results, reported as MSE (MAE).

We hope this response can address the your concerns.

Reviewer_VVDyOpenReview ↗

Raise my Score

I thank the authors for the strong rebuttal and also for uploading the revised pdf. The complexity table and the cross-attention baseline directly address my remaining questions and I appreciate that you labeled the interaction-only bound.

A few smaller cosmetic points for the final version. Since g constructs the queries and therefore the alignment weights, its O(NL**2) cost is arguably part of the cross-axis coupling cost in Definition 1(B). It becomes the dominant term for longer lookbacks, so I would qualify the single-axis budget claim in the main text accordingly. Section 3.2 also still describes g as a single linear layer and app E.1 still looks like still describes TFRC as producing an M × N feature-time view. Second, it would be great to include the ECL and Traffic results. Third with the seed deviations, the margin over SOFTS on the high-dimensional datasets is within about one standard deviation, while the efficiency is comparable. I think the paper would benefit from framing its advantage as comparable accuracy with explicit step-indexed alignment, rather than as a clear accuracy gain.

Overall, my main concerns are addressed, and I will raise my score.

Best, VVDy

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.