acceptodds
Under review as a conference paper at ICLR 2027

DESA: Discrete Event-Sparse Activation for Lightweight Multivariate Time-Series Forecasting

Abstract

Recent advances in multivariate time-series forecasting have primarily focused on improving predictive accuracy. However, these improvements often come at the cost of substantially increased computation, limiting the applicability of such models in resource-constrained industrial environments. In this study, drawing on the event-centric perspective of Discrete Event System Specification (DEVS), we propose DESA (Discrete Event-Sparse Activation), which combines a low-cost base forecast with selective nonlinear correction. The Inertia Block processes the entire input sequence at low computational cost for every forecast, while the Event Block corrects the base forecast using local windows centered at selected time steps. Connecting the two blocks, the Surprise Gate uses reconstruction residuals from the Inertia Block and temporal differences to guide time-step selection and modulate the strength of the Event Block’s input. The Event Block gathers the selected time steps for processing and skips its core computations at unselected time steps. To learn this selective processing, we first train a reference model with the same architecture, applying the Event Block at every time step. We then use its predictions and intermediate signals as supervision to train a sparse model that performs selective event processing. Experimental results show that the sparse model achieves an average forecasting error close to that of the reference model while reducing the computational cost of inference. Overall, DESA lies on the empirical forecasting-error–FLOPs Pareto frontier among the compared models, providing a favorable trade-off between forecasting accuracy and computational cost.

Primary area:
learning on time series and dynamical systems
Keywords:
Multivariate time-series forecasting, Conditional computation, Computational efficiency, Knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZuQHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DESA, an inertia-correction architecture for multivariate time series forecasting. It combines an always-executed Inertia Block that produces a low-cost base forecast from the full input history with a selectively executed Event Block that applies nonlinear correction only at input-dependent selected time steps. A Surprise Gate, driven by reconstruction residuals and temporal differences, modulates Event inputs and guides time-step selection. A reference model with all centers active supervises the sparse model via knowledge distillation. Experiments on nine benchmarks show DESA achieves comparable accuracy while reducing FLOPs from 150.90M to 55.74M.

Critical strengths
  1. The lightweight design is important, especially in resource-constrained scenarios.
  2. The authors attempt to improve inference efficiency through a sparse activation module, which is a reasonable solution, and the ablation studies further validate the effectiveness of this strategy.
  3. The repeated experiments report both means and standard deviations, which strengthens the reliability of the empirical results.
Critical weaknesses
  1. The main text is not self-contained and is hard to follow. Notation is not properly connected across sections, and several technical details remain unclear. For example, X̂_{I} appears in the text but is not defined in any equation. In Section 3.3, the symbols x̃_{t,c}, x̂_{I,t,c}, u_c, and v_c are introduced without any explanation of their meanings or subscripts. Sections 3.4 and 3.5 lack formalized descriptions, and the purely textual presentation makes it difficult to understand the proposed method precisely.
  2. The comparison results are not fully convincing. DESA does not achieve state-of-the-art performance (in most metrics it is not even the runner-up) and mainly attains an accuracy-efficiency trade-off, which may not be regarded as sufficient by the time series community. In addition, several recent forecasting baselines are missing, such as TimeMixer++ [1] and OLinear [2], as well as lightweight baselines such as RLinear [3], SparseTSF [4], and TimeBase [5]. In particular, TimeBase achieves state-of-the-art performance with an extremely small number of parameters.
  3. Several hyperparameter settings lack a clear rationale. For example, why are the kernel sizes of the Inertia Block chosen as 7, 31, and 51? Why is the sharpness of the sigmoid in the S-Gate fixed at κ=10κ = 10?
  4. Figure 4 would be more informative if it included horizontal comparisons with other methods, to examine whether they can respond to event changes. Such a comparison would better highlight the advantage of Event correction.

[1] Wang S, Li J, Shi X, et al. (2025). Timemixer++: A general time series pattern machine for universal predictive analysis. In International Conference on Learning Representations (Vol. 2025, pp. 16980-17016).

[2] Yue W, Liu Y, Wang H, et al. (2025). Olinear: A linear model for time series forecasting in orthogonally transformed domain. Advances in Neural Information Processing Systems, 38, 60573-60632.

[3] Li, Z., Qi, S., et al. (2023). Revisiting long-term time series forecasting: An investigation on linear mapping. arXiv preprint arXiv:2305.10721.

[4] Lin, S., Lin, W., Wu, W, et al. (2024). SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters. Proceedings of the 41st International Conference on Machine Learning 235:30211-30226

[5] Huang, Q., Zhou, Z., et al. (2025). TimeBase: The Power of Minimalism in Efficient Long-term Time Series Forecasting. In ICML.

Questions for authors

Please refer to Weaknesses.

Reviewer_nzC5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DESA, a forecaster for long-term multivariate time-series forecasting that splits prediction into an always-executed, low-cost Inertia Block (rFFT spectral filter plus depth-wise temporal convolutions with kernels 7/31/51, followed by linear projections) and an Event Block that adds a nonlinear correction computed only at selected input time steps. A continuous Surprise Gate built from the Inertia reconstruction residual and first differences modulates the Event inputs and feeds a learned threshold-based time-step selector. The Event Block processes radius-2 local windows around the selected centers with a latent-token channel mixer that is linear in the channel count. Training first fits a dense reference model with all centers active (113 epochs), then distills it into the sparse model in three stages using a straight-through estimator and a MAC-based computation penalty. On nine benchmarks with L=96, the sparse model activates about 25% of time steps and reduces FLOPs from 150.9M to 55.7M while staying close to the reference in MSE (0.3457 vs. 0.3440). The authors claim it lies on the empirical MSE–FLOPs Pareto frontier among the compared models.

The decomposition into a cheap base forecast plus selectively executed correction is intuitive, and the implementation is carefully documented. The appendix gives full architecture equations, losses, stage-wise schedules, checkpoint rules, ablation-control protocols, and FLOPs/runtime protocols. Measured CPU/GPU latency and memory are reported alongside FLOPs, and the reported numbers are internally consistent across tables.

However, the per-dataset ablation shows that the Event correction helps mainly on the high-dimensional datasets (Solar, Traffic, Electricity) and hurts on most low-dimensional ones, so most of the accuracy and the frontier position may come from the Inertia backbone. The efficiency claim rests on operator-counted FLOPs that may omit FFT and element-wise operations and that do not appear to be profiled under one protocol across models. There is also no latency comparison between sparse and dense execution of the same model. The Pareto claim is made against a baseline set in which DLinear is the only lightweight model and no model covers the range between 55.7M and 1.6G FLOPs. The DEVS framing is not reflected in the method.

Critical strengths
  • Sparsity without discarding input: The inertia–correction split (Eq. 1) applies conditional computation along the time axis in a way that never drops observations. The channel-wise Inertia Block always sees the full history, and only the extra nonlinear correction is sparse. This differs from token pruning or early exit, where skipped tokens are lost. When K=0 the model returns exactly the base forecast (Eq. 15), so the design degrades to a cheap linear-spectral forecaster instead of failing.
  • Input-dependent compute for forecasting: Most efficiency work in long-term forecasting builds a fixed, small model (DLinear, FITS). DESA instead asks where in each input window extra computation is worth spending, and makes that cost vary per sample (K ranges from 0 to L, Eq. 6). The Random matched-K control, which keeps each sample's K but moves the centers, is clearly worse on average (0.3631 vs. 0.3457, Table 3), so position does matter at least in aggregate. This is a useful question for the field even if the current answer is incomplete.
Critical weaknesses
  • The gains do not come mainly from event-selective correction: The paper's thesis is that the model should learn where to apply nonlinear correction (§1). However, the Inertia Block is channel-wise (§3.2), so the Event Block is the model's only cross-channel path (Eq. 9). Its benefit appears where channel mixing is known to help, not where events are.
    • Averaged over horizons (Table 9), DESA beats Inertia-only mainly on Solar (0.337→0.239), Traffic (0.604→0.505), and Electricity (0.204→0.189). On 5 of 9 datasets Inertia-only is better (e.g., ETTh1 0.443 vs. 0.462, ETTm1 0.396 vs. 0.410), and the dense reference is also worse there (ETTh1 0.465).
    • On Solar and Traffic, Random matched-K already recovers about two thirds (0.337→0.272) and half (0.604→0.553) of the gain. Teacher-free, which just adds the Event Block, beats DESA on Traffic (0.495 vs. 0.505) but is far worse than Inertia-only on the ETT datasets (e.g., ETTh1 0.547 vs. 0.443).
    • So the 36-condition average in §4.5 does not show that learned event selection drives the accuracy. A channel-mixing control applied at all or random positions is needed to separate "where" from "cross-channel".
  • The efficiency and Pareto claims are not established: The central claim is a position on the MSE–FLOPs frontier, but the frontier point can be explained by a strong cheap backbone, and the FLOPs saving is not shown in wall-clock time.
    • Inertia-only (12.48M, 0.3647) already dominates DLinear (19.74M, 0.398), the only cheaper baseline. No model is compared between 55.7M and 1.6G FLOPs, and FITS and FilterNet are cited in §2 but not compared (Table 1).
    • Appendix B states that FFT and element-wise operations "may be omitted", although the Inertia Block, S-Gate, and selector consist largely of such operations. Baselines "retain their recorded profiling settings", and TimesNet's 798G FLOPs per sample against 16.07 ms per batch of 16 on an RTX 5060 Ti is far above that GPU's peak.
    • Table 1 reports no latency for the dense reference or Inertia-only. DESA's GPU time (3.52 ms) is 13× DLinear's and only about 2× faster than iTransformer's despite 47× fewer FLOPs.
  • The "event/surprise" framing does not match the mechanism: The DEVS perspective and the Surprise Gate motivate selection by surprising time steps. The selector that is actually trained is a standard learned hard gate with STE, a compute penalty, and distillation, close to ACT/Mixture-of-Depths-style routing that the paper does not discuss (§2 cites only BranchyNet and DynamicViT).
    • The paper states twice that it does not implement DEVS (§1, §2), and no DEVS element (time advance, transition functions) appears in §3. By its own description, the binary mask "rather than S-Gate magnitude" decides the centers (§1).
    • Shuffling the S-Gate in time changes MSE only from 0.3457 to 0.3471 (Table 3), while the selector also sees aligned residual/difference features f_t and a learned per-position bias b_t (Eq. 5). How much of the selection reflects surprise rather than position is not shown.
Questions for authors
  • Sparse vs. dense latency: Under the Table 1 protocol, what are the latency and memory of the dense reference model and of Inertia-only? Is sparse execution actually faster in wall-clock time? How does latency change with batch size and with K/L?
  • Per-dataset value of Event correction: Why do both the reference and the sparse model underperform Inertia-only on ETTh1, ETTm1, ETTm2, Weather, and Exchange? What are the active-step rates on these datasets? What happens if validation is allowed to disable the Event Block (K=0)?
  • Consistent FLOPs and baselines: Were all baselines re-profiled with the same counter, batch size, and horizons as DESA, and how is the TimesNet entry (798G FLOPs vs. 16.07 ms per batch) explained? If FFT and element-wise operations are counted for all models, does the frontier hold when FITS, SparseTSF, CycleNet, or a mid-cost model such as SOFTS is added and Inertia-only is plotted in Fig. 1/3?
  • Value of the learned selector and budget control: How does the learned selector compare with heuristic top-K selection by |r_t| or by the S-Gate summary e_t? What is the per-sample distribution of K/L, and does a hard cap on K change accuracy? Can Table 12 report FLOPs, given that γ=0.01 gives lower MSE than the reference (0.3436 vs 0.3440) at 64% active steps?
  • Distillation-only control and training cost: What is the result of the same three-stage recipe with λp=λe=λg=0? What is the total GPU-hour cost of the reference and sparse training compared with the baselines?
  • Baseline sources: Which baseline numbers in Tables 2 and 13 are published and which are re-run? Are the test-set windows (e.g., drop_last) identical for DESA and the baselines?
1 comment
Reviewer_nzC5OpenReview ↗

Response to Authors

Thank you for the response. Q1, Q4, Q5, and Q6 are addressed. The measured latency (CPU 2.92x, GPU 1.83x, memory 363.6 -> 107.4 MiB, Table 16), the FLOPs recount including FFT and element-wise ops (+5%), the top-K and KD controls, and the baseline sources are useful additions.

Corrections on my side: TiDE (1,236.5M FLOPs) does lie between 55.7M and 1.6G, so that statement was wrong. I also withdraw the strength that DESA degrades to a cheap forecaster at K=0, since disabling the Event path raises MSE from 0.3457 to 0.6473 (App. C).

Remaining concerns:

  • Per-dataset value of the Event path: Inertia-only is still better on 5/9 datasets (Table 28), while the selector stays active on 19-35% of the steps there. The control that separates "where to correct" from "cross-channel mixing" is not provided, and the top-K comparison (Tables 21-22) is reported only on average.
  • Efficiency: CycleNet, with 16% more FLOPs (68.0M vs. 58.7M), beats DESA on 8/9 datasets (Table 28) at about 1/77 of the training cost (0.117 vs. 8.979 GPU-h, Table 20). The GPU speedup drops to 1.30x on RTX 4090 (Table 19). DESA stays on the frontier formally, but the practical margin is narrow.
  • Consistency of cost reporting: App. B still says FFT and element-wise ops "may be omitted" and baselines "retain their recorded profiling settings", next to the new statement that all baselines use the same counter. Table 1 and Fig. 1 remain on the old counting.
  • Framing: The DEVS/surprise framing is unchanged, and the selected positions are less enriched for large changes than a simple residual top-K (1.24 vs. 1.37, Table 23).

I will keep my score.

Reviewer_GBS1OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DESA, a lightweight framework for multivariate time-series forecasting that combines low-cost base prediction with selective nonlinear correction. The Inertia Block generates base forecasts, while the Surprise Gate and time-step selector identify positions for additional processing by the Event Block. A reference-guided training strategy is introduced to transfer predictive knowledge from a fully activated model to its sparse counterpart. Experiments on nine benchmarks demonstrate a favorable accuracy–efficiency trade-off, with substantially reduced FLOPs and forecasting accuracy close to that of the reference model.

Critical strengths
  • DESA achieves substantial reductions in inference FLOPs while maintaining forecasting performance close to that of the fully activated reference model. This is practically relevant for resource-constrained forecasting applications.

  • The paper evaluates nine datasets across four prediction horizons and reports forecasting accuracy, computational costs, runtime measurements, and several ablation studies. These experiments provide useful preliminary evidence for the effectiveness of selective computation.

Critical weaknesses
  • The proposed framework primarily integrates existing techniques, including FFT-based modeling, temporal convolutions, learned gating, and knowledge distillation. Although motivated by discrete-event systems, the proposed event mechanism essentially performs learned time-step selection without explicitly modeling event-driven state transitions. Moreover, the reconstruction residuals used by the Surprise Gate are obtained from a reconstruction head without an explicit reconstruction objective, making their interpretation as meaningful surprise signals unclear. The experiments do not sufficiently demonstrate that the selected positions correspond to informative temporal events or that the proposed mechanism offers clear advantages over simpler residual-based or fixed-budget selection strategies.

  • Although DESA demonstrates favorable FLOPs–accuracy trade-offs, the evaluation lacks comparisons with several particularly relevant lightweight forecasting methods and computationally matched alternatives. The FLOPs accounting also excludes certain operations, making the relationship between theoretical savings and practical efficiency less clear. Furthermore, the Teacher-free ablation changes the optimization procedure and computation penalty, preventing a clean attribution of improvements to knowledge distillation. Stronger controlled experiments are needed to isolate the benefits of the proposed components and establish the robustness of the efficiency claims.

Questions for authors
  • What distinguishes the proposed event-driven selection mechanism from conventional residual-based gating or dynamic token selection? Can the authors provide controlled comparisons with simpler selection strategies under matched computational budgets?

  • Since the reconstruction head is not explicitly optimized for reconstruction accuracy, how can its residuals reliably indicate meaningful temporal events? Can the authors provide quantitative evidence linking selected time steps to important changes in temporal dynamics?

  • Can the authors provide a controlled distillation ablation in which the training schedule, initialization, computation penalty, and checkpoint selection remain unchanged, with only the distillation objectives removed?

  • Could the authors include additional lightweight forecasting baselines, such as FITS and NLinear, and compare them under matched inference budgets? Reporting the reference-to-sparse wall-clock speedup would also help clarify the practical benefits.

  • How sensitive is the proposed selection mechanism to distribution shifts and different input patterns? Does its efficiency advantage remain consistent across individual datasets and inference environments?

Comments on the paper

1 comment
Area_Chair_DiPpOpenReview ↗

Discussion Reminder

Dear Reviewers,

Thank you for your efforts so far. This is a friendly reminder to review the authors' rebuttal and respond to their comments as soon as possible, before the discussion deadline (November 18 AoE). Please also indicate whether your concerns have been addressed. Thank you for your time and cooperation!

Best regards,
AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.