Learnable Difference-Aware Embedding: A Lightweight Plug-and-Play Module for Multivariate Time Series Forecasting

Multivariate time series forecasting supports decision-making in energy, transportation, and other dynamic systems, where models must capture both global dependencies and local temporal changes. However, standard embeddings in representative forecasting backbones do not explicitly encode first- and second-order temporal differences at the initial representation stage. We introduce Learnable Difference-Aware Embedding (LDAE), a lightweight plug-and-play module that constructs channel-wise rate-of-change and curvature features using learnable convolutions initialized with finite-difference stencils. LDAE fuses these derivative cues with the backbone representation at the initial embedding interface, while leaving the backbone's internal temporal modeling unchanged. Across 12 real-world datasets, LDAE improves forecasting accuracy in low-to-moderate-noise, non-stationary settings and transfers across Transformer-, CNN-, and MLP-based backbones. These results demonstrate that explicit derivative cues provide a lightweight and transferable inductive bias for multivariate time series forecasting.