Hierarchical Latent Factorization for Non-invasive Sequential Motor Decoding

Non-invasive neural decoding is challenged by context-dependent representation drift: during sequential movements, the same finger movement can produce different neural patterns across participants, sequences, and serial positions. These sources of variation operate over different temporal scales, and mix nonlinearly, hindering stable decoding. We formulate this setting as primary-task prediction from mixed, context-dependent observations and propose HiLa, a hierarchically latent factorization framework for stable decoding during sequential movements. HiLa uses hierarchical labels as supervisory signals and conditional branch splitting to learn context-robust representations that preserve action identity across contexts. HiLa achieves 62.02% movement identity accuracy during sequential finger tapping and 50.44% keystroke accuracy during typing. HiLa's backbone outperforms benchmarks, while hierarchical supervision improves accuracy, with gains remaining stable in cross-participant and cross-context tests. Latent factorization yields more decontextualized and biologically plausible representations without sacrificing decoding accuracy. Together, these results show that hierarchically supervised latent factorization can improve primary-task decoding under structured contextual variation in non-invasive neural recordings.