RiJEPA: Rule-Informed Neuro-Symbolic Joint-Embedding Predictive Architecture

Modern Joint-Embedding Predictive Architectures (JEPAs) learn powerful representations from data but lack explicit mechanisms for incorporating symbolic knowledge, while traditional rule systems are interpretable but rely on rigid discrete representations and combinatorial search. We introduce RiJEPA, a Rule-Informed Neuro-Symbolic JEPA that connects symbolic reasoning and predictive representation learning in both directions. In the rules-for-JEPA direction, RiJEPA combines Energy-Based Constraints (EBC) with a multi-modal dual-encoder architecture to align continuous observations and symbolic rules in a shared latent space, shaping valid rules into low-energy basins while repelling contradictory or invalid rules. In the reverse JEPA-for-rules direction, the resulting differentiable rule-energy landscape replaces discrete rule search with gradient-guided exploration, supporting joint rule generation, forward inference, abductive reasoning, and marginal-predictive translation. On a controlled simulation, RiJEPA preserves near-zero energy for valid rules while increasing the energy of invalid OOD rules by approximately $85\times$ relative to a classic JEPA. On the UCI Heart Disease case study, RiJEPA organizes patient representations around symbolic diagnostic poles, enabling prediction by latent-space proximity without training a downstream classifier and supporting generative rule reasoning. Together, these results establish a bidirectional neuro-symbolic principle: symbolic rules can shape predictive representations, while the resulting representations provide a continuous, differentiable space for reasoning over and discovering rules.