K-PRISM: Kalman-based Physics-informed Residual Innovation with Sensor Multimodality

Classical Kalman filtering is optimal under known linear-Gaussian dynamics, but real systems exhibit partially-known dynamics, unknown observation models, and time-varying sensor reliability. Neural-augmented filters such as KalmanNet relax the need for known noise statistics, yet rely on recurrent gain estimation that adapts slowly to abrupt sensor degradation and tends to memorize training-specific spatial corrections. We propose K-PRISM (Kalman-based Physics-informed Residual Innovation with Sensor Multimodality), which (i) decomposes the state transition into a known physics prior and a learned residual, (ii) computes a translationally invariant gain from instantaneous velocity and innovation features, enabling single-timestep adaptation without spatial overfitting, and (iii) extends to heterogeneous multi-sensor fusion via cross-modal gating. The resulting update generalizes the structure of the classical multi-sensor Kalman update, with Lipschitz-bounded innovations and formal error contraction. Empirically, K-PRISM matches or exceeds KalmanNet's accuracy while using $20$ to $200\times$ fewer parameters and eliminating its catastrophic-failure mode. On the Lorenz attractor, it matches steady-state accuracy while recovering from sensor dropout with zero failures; in a controlled two-sensor study under alternating reliability, the cross-modal gate identifies the informative sensor online, beating independent fusion by $5.2$ dB. On real-world NCLT localization it converges in every run, whereas our KalmanNet reproduction diverges in $47\%$, and it transfers to visual-odometry velocity filtering on EuRoC. These results position compact, translationally invariant filtering as a robust alternative to recurrent gain estimation.