Your Probabilistic JEPA Is Secretly a Hidden Markov Model: A State-Space Interpretation of Joint-Embedding Predictive Learning

A hidden Markov model (HMM) combines three roles: inference of a hidden-state belief from observations, propagation through a Markov transition, and emission back to observation space. We show that full, time-indexed Predictive Information Bottleneck VJEPA (PIB-VJEPA) exposes the same computational structure: a stochastic context encoder plays the role of an amortized filtering distribution, a probabilistic predictor defines latent-state dynamics, and a decoder, inverse target encoder, or induced implicit conditional supplies the emission direction. We formalize four progressively stronger levels of correspondence, from computational analogy to exact sequence-level and model-and-objective equivalence, and give sufficient conditions under which a probabilistic temporal JEPA admits an exact HMM representation. We instantiate this view with Markov-Chain JEPA (MCJEPA), which replaces the latent predictor by a learned transition matrix; in the finite time-homogeneous case, matrix powers guarantee exact multi-horizon Chapman–Kolmogorov consistency. The framework extends naturally to conditioned discrete-state transitions, continuous-state Markov kernels, continuous-time dynamics, and deterministic Dirac-kernel dynamics. We further reinterpret predictive information-bottleneck learning as Markov-state construction: compression promotes minimality, while residual predictability diagnoses insufficiency. Controlled experiments support transition recovery and composition, the filtering interpretation of the context encoder, recovery of a known minimal predictive state, and HMM-style sequence training of probabilistic JEPA, including hybrid JEPA–HMM objectives. Together, these results give probabilistic temporal JEPA a principled state-space interpretation and connect joint-embedding predictive learning with probabilistic dynamical modeling.