Semi-Supervised Learning and Output Prediction with Mode-Dependent Dynamical Variational Autoencoders

Predicting multidimensional time series with hidden switching dynamics is central to safety-critical decision making and data augmentation. However, standard deep generative models rarely represent operating modes explicitly. This paper augments Dynamical Variational Autoencoders (DVAEs) with a sequence of discrete latent variables governing the switching prior dynamics of the continuous latent state. The resulting Mode-Dependent DVAEs (MD-DVAEs) can be trained in a semi-supervised manner. Namely, a labeled data subset anchors discrete latent values to known modes of operation and resolves the permutation ambiguity intrinsic to unsupervised discrete latents. A novel evidence lower bound is derived that includes a term on the categorical mode encoder for supervision. Once trained, MD-DVAEs directly predict future output sequences under either a known or stochastically sampled future mode context. The approach is validated on synthetic datasets, including a comparison with switching-dynamics models and unsupervised baselines.