DVSI: Dynamical Variational System Identification with High-dimensional State

We address the problem of identifying a nonlinear stochastic Markov dynamical system that has many state variables represented together by a high-dimensional state vector. We focus on the case where the state-transition model (STM) of the system has no known analytical structure due to lack of physics knowledge, is modeled using a neural network, and is identified from noisy measurements through a known measurement model. Particle-filter-based system identification, including differentiable particle filters (DPFs), suffers from the curse of dimensionality in this setting. To address the curse of dimensionality, we develop Dynamical Variational System Identification (DVSI): transformer-based dynamical variational inference methods that jointly learn the STM and an amortized posterior of the state trajectory by maximizing an evidence lower bound. We develop four DVSI methods, spanning autoregressive versus factorized and smoothing versus filtering posteriors. Through experiments on stochastic Lorenz-63 and Lorenz-96 systems and real magnetoencephalography (MEG) data, we evaluate the DVSI methods for identifying dynamical systems that have up to 200 state variables, where DPFs have significant limitations due to the curse of dimensionality.