CRFormer: Covariance-Enhanced Rough Transformer for Multivariate Time Series Modeling

Multivariate time series modeling faces the dual challenges of capturing complex spatial dependencies and maintaining computational efficiency over long sequences. Existing signature-based methods, including the Rough Transformer (RFormer), often overlook the covariance structure of multivariate sequences, leading to inadequate modeling of spatial dependencies. In this work, we propose Covariance-Enhanced Rough Transformer (CRFormer), which extends RFormer by introducing a Covariance Rough Path Module (CRP Module) that integrates the geometric structure of covariance matrices with path signature theory. CRP Module explicitly constructs a dynamic covariance path alongside the raw temporal path, forming a dual-signature stream that effectively captures complex spatiotemporal inter-dependencies. We further establish the theoretical equivalence between CRFormer and Neural CDEs in the continuous-time limit, achieving comparable expressiveness with substantially fewer layers while providing rigorous interpretability and convergence guarantees. More importantly, the CRP Module is backbone-nonintrusive and can be seamlessly integrated into benchmark time series backbones to enhance their spatial dependency modeling capability. Extensive experiments demonstrate that CRFormer significantly outperforms competing baselines across diverse benchmarks, while CRP-enhanced models deliver consistent accuracy gains alongside dramatic reductions in computational cost and memory consumption across multiple backbone architectures, validating the effectiveness and broad applicability of the CRP Module. The code is available at a repository: https://anonymous.4open.science/r/CRFormer-FDC8/.