Decoupling Time and Space in Controlled Differential Equations for Exact Interaction-Order Decomposition

Selective state-space models (SSMs) and linear controlled differential equations (LNCDEs) have emerged as effective approaches for long-sequence modeling. However, existing models face a trade-off: diagonal state transitions offer computational efficiency but struggle to capture cross-channel interactions, whereas dense transitions successfully capture these interactions at a computational cost that structured alternatives have since aimed to reduce. To address this, we propose the Tensor-Decoupled CDE (TD-CDE), an architecture that decouples temporal updating from spatial mixing. By restricting time evolution to diagonal, data-dependent decays, TD-CDE maintains efficient parallel scans. Cross-channel interactions are modeled along the depth dimension using Tensor Train (TT) decomposition, extracting high-order signature-type features without exponential parameter growth. Furthermore, the model employs gating via raw inputs, avoiding normalizations to preserve causality and exact, additive interpretability. Theoretically, we prove that models with commuting generators—including diagonal SSMs—realise only the shuffle subalgebra and cannot represent complex interactions like Lévy area at any width, and that TD-CDE's graded state decomposes predictions exactly by interaction order and feature words. Additionally, we formalize an order contract for selective discretisation. Empirically, TD-CDE captures cross-channel dynamics in path-dependent synthetic tasks that challenge existing models like Mamba. Evaluated on the UEA archive, TD-CDE achieves state-of-the-art accuracy with up to two orders of magnitude fewer parameters specifically on the datasets our order profiler flags as high-order, while showing no advantage on low-order ones—a separation predicted prior to training. Crucially, we demonstrate that a high-order discretisation scheme is essential for resolution invariance: TD-CDE preserves robust predictions and stable exact attributions even under severe sequence subsampling, where standard endpoint approximations fail.