acceptodds
Under review as a conference paper at ICLR 2027

RiJEPA: Rule-Informed Neuro-Symbolic Joint-Embedding Predictive Architecture

Abstract

Modern Joint-Embedding Predictive Architectures (JEPAs) learn powerful representations from data but lack explicit mechanisms for incorporating symbolic knowledge, while traditional rule systems are interpretable but rely on rigid discrete representations and combinatorial search. We introduce RiJEPA, a Rule-Informed Neuro-Symbolic JEPA that connects symbolic reasoning and predictive representation learning in both directions. In the rules-for-JEPA direction, RiJEPA combines Energy-Based Constraints (EBC) with a multi-modal dual-encoder architecture to align continuous observations and symbolic rules in a shared latent space, shaping valid rules into low-energy basins while repelling contradictory or invalid rules. In the reverse JEPA-for-rules direction, the resulting differentiable rule-energy landscape replaces discrete rule search with gradient-guided exploration, supporting joint rule generation, forward inference, abductive reasoning, and marginal-predictive translation. On a controlled simulation, RiJEPA preserves near-zero energy for valid rules while increasing the energy of invalid OOD rules by approximately relative to a classic JEPA. On the UCI Heart Disease case study, RiJEPA organizes patient representations around symbolic diagnostic poles, enabling prediction by latent-space proximity without training a downstream classifier and supporting generative rule reasoning. Together, these results establish a bidirectional neuro-symbolic principle: symbolic rules can shape predictive representations, while the resulting representations provide a continuous, differentiable space for reasoning over and discovering rules.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Joint-Embedding Predictive Architectures, Neuro-Symbolic Learning, Rule-Informed Representation Learning, Energy-Based Model, Self-Supervised Learning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_cjfJOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper combines formal rules with JEPA and explores the bidirectional relationship between symbolic rules and the latent space. It adds energy-based constraints to JEPA training so that valid rules have low energy in the latent space and invalid rules have high energy, and then uses this energy landscape to generate rules and perform reasoning via Langevin sampling. The paper conducts two experiments, a 3D simulation and a study on the UCI Heart Disease dataset: the simulation compares the energies of valid and invalid rules, while the UCI study evaluates the accuracy of classification by distance to the poles and shows examples of generated rules.

Critical strengths
  1. The writing is generally fluent.
  2. The description of the method in Section 3 is fairly clear.
  3. The paper is fairly candid about its limitations.
Critical weaknesses
  1. The motivation is unclear. The paper never explains why JEPA in particular is chosen: the "JEPA" in the experiments is a few MLP layers predicting between features of tabular data, without using any property that distinguishes JEPA from generic embedding learning. The energy-based constraint is essentially the margin loss commonly used in metric learning and could be implemented with any dual-encoder architecture, so the connection between the method and JEPA is weak. In addition, the claim that "logical validity equates to geometric proximity" (p. 2) is too strong.

  2. Section 4 proposes generating rules and performing forward and abductive reasoning in the latent space, and lists this as one of the main contributions, but these claims are not sufficiently tested: Table 5 gives only one example per mode, with no quantitative evaluation. The energy EctxE_{\text{ctx}} in Paradigm 2 is never defined, and the experiments only execute the step z^t=g(zc)\hat z_t = g(z_c), without the sampling and acceptance steps.

  3. The experiments are small in scale and do not show significant results. The simulation uses synthetic 3D toy data, and its outcome appears to be a natural consequence of contrastive learning, with limited novelty and significance. In the heart disease experiment, the classic JEPA and RiJEPA both reach 100% accuracy, which does not demonstrate any advantage of RiJEPA. RiJEPA also uses the ground-truth labels directly during training through the anchor loss as I understand, so its pole-based classification cannot be called zero-shot. More importantly, RiJEPA adds both the rule loss and the anchor loss on top of the classic JEPA, and the anchor loss alone could produce the diagnosis-based clusters and pole-based classification. Without more ablations, the effect cannot be attributed to the rules.

Questions for authors
  1. Could the authors provide more experiments and comparisons with more baselines to further demonstrate the effectiveness of the method?
  2. For the claims in Section 4, could the authors provide stronger evidence of their actual contribution?
  3. In the heart disease experiment, which columns are included in the context and the target on the data side, and is the diagnosis label excluded from the input? Could the authors explain why both the classic JEPA and RiJEPA reach 100% accuracy?
4 comments

Thank you for the constructive review. We have clarified the theory, quantitatively tested both reasoning paradigms, and added matched ablations and broader benchmarks. The central results now appear in Tables 3–5 and Figure 4, with details in Appendices G–K.

1. Why JEPA, and what is novel? JEPA supplies masked-context latent prediction and an EMA target encoder. RiJEPA connects this data-prediction pathway to symbolic antecedent–consequent pairs using modality-specific encoders and a shared predictor. EBC shapes their energies through positive and sampled negative pairs; a separate supervised anchor aligns observations with class poles. We agree the margin loss is not unique to JEPA. In a first-fold ablation over 15 datasets, removing the JEPA data loss changes pole accuracy +0.006, silhouette +0.012, and matched-versus-flipped rule AUROC −0.003; replacing the shared predictor with separate predictors changes these by −0.003, −0.006, −0.002 (Appendix H). Thus we do not claim JEPA is necessary for these tabular outcomes. The contribution is an integrated predictive data–rule query interface, rather than a new contrastive-loss identity. We have also replaced “logical validity equates to geometric proximity”: Proposition 1 ensures separation only for constrained rule pairs at zero loss, not validity of arbitrary unseen low-energy rules (Sec. 3; Appendix I).

2. Quantitative evidence for Section 4. Rule completion on previously unmined antecedents achieves 0.836 accuracy versus 0.750 for lookup; class-1 Langevin abduction yields 0.567 held-out-valid candidates versus 0.120 for frequency-matched random antecedents (Appendix J). For Paradigm 2, Sec. 4 now defines Ectx(zc)=−log⁡p^ctx(zc)E_{\rm ctx}(z_c)=-\log\hat p_{\rm ctx}(z_c) using a Gaussian KDE of training-rule antecedent embeddings. The implemented procedure explores this energy by Langevin updates, predicts with g(zc)g(z_c), decodes items/poles, and applies an optional training-only confidence filter. Held-out rule validity is 0.647 before filtering and 0.677 after, versus 0.246 for joint Langevin and 0.237 for random proposals (Table 5; Fig. 4(b)). 71.7% of accepted pairs are absent from mined training rules, but novelty does not guarantee validity. Acceptance is a Metropolis-style heuristic, not exact MH without proposal-density corrections; plausible context proposals explain most of the improvement.

3. Does EBC contribute beyond the supervised anchor? Across 15 exploratorily selected tabular datasets (2×5 stratified folds), matched JEPA + supervised poles without rules reaches 0.863 mean accuracy, versus 0.868 for full RiJEPA (Holm-adjusted p=0.767p=0.767). Supervised contrastive JEPA + probe reaches 0.865 (adjusted p=1.0p=1.0). Thus the anchor explains most classification gain; EBC's more distinctive result is rule discrimination (Table 3; Appendix H). In the unanchored staged comparison, adding negatives raises matched-versus-flipped rule-energy AUROC 0.735→0.966, and completion 0.704→0.839 (Fig. 4(a)); within the fully anchored model, the negative-margin AUROC contrast is smaller (0.941→0.956). For arbitrary held-out rule candidates, energy ranks empirical validity at AUROC 0.845, below training-row confidence 0.924 (Appendix I). These scores are not soundness or causal-explanation certificates.

4. Broader comparisons and statistical limits. We evaluate 15 tabular datasets with ten test folds and five fixed-split video benchmarks using frozen ResNet-18 features and five seeds (Table 4). RiJEPA pole-readout gains over JEPA probes are +13.8, +16.8, and +10.9 percentage points on KTH, UT-Interaction, and MNNIT; HMDB51 and RLVS show negligible changes. The broader 115-dataset exploratory screen finds higher accuracy on 47/115 datasets and higher balanced accuracy on 81/115 (Appendix G). We acknowledge dataset screening, small video test sets, seed-only variability, and the absence of end-to-end video training.

5. Clarification of the previous 100% result. The original comparison was invalid because the diagnosis label inadvertently remained in two unmasked one-hot input columns. We have removed that result and its related figures, as they do not support the revised claims. In the retained experiments, class labels are excluded from observation inputs and used only for explicitly supervised training-rule consequents and the pole anchor. The pole read-out is therefore classifier-free but label-informed, not zero-shot (Secs. 3, 5; Appendix G).

We thank the reviewer for motivating these theoretical corrections, controlled comparisons, and quantitative reasoning experiments.

Area_Chair_rK56OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Thank you for the substantial revisions and additional experiments. These address my concerns about missing ablations and quantitative evaluation, and I appreciate the clearer scope of the revised claims.

My remaining reservation concerns the strength of the contribution. I acknowledge the demonstrated capabilities, but remain unconvinced that their added value is substantial enough to support acceptance. I therefore maintain my original rating.

Thank you for revisiting the revised manuscript and for confirming that the additional ablations and quantitative evaluations address your earlier concerns. We understand that your remaining reservation is the significance of the contribution, and would like to clarify why we believe RiJEPA adds a meaningful capability beyond conventional predictive representation learning.

1. Rule supervision provides a capability that the supervised anchor alone does not. In matched ablations, the label-anchored rule-pole control without paired-rule training (C1) obtains 0.517 matched-versus-flipped rule AUROC, versus 0.956 for full RiJEPA (S3). The anchor-only JEPA without a rule pathway (C2) attains similar classification accuracy but does not provide rule-energy querying. In the unanchored comparison, adding negative rule pairs increases rule AUROC from 0.735 to 0.966, isolating their effect on sampled rule-pair discrimination.

2. The learned rule pathway supports quantitative reasoning tasks. On previously unmined antecedents, consequent-completion accuracy is 0.836, versus 0.750 for lookup. In class-conditional abduction, 0.567 of generated rule candidates satisfy the held-out empirical-validity criterion, versus 0.120 for frequency-matched random antecedents. The separate context-marginal sampling evaluation attains 0.677 validity after filtering, versus 0.237 for random proposals. These experiments demonstrate complementary completion and proposal capabilities, not a logical-validity certificate or superiority to every discrete search method.

3. These capabilities coexist with comparable predictive performance. Across 15 selected tabular datasets, full RiJEPA reaches 0.868 nearest-pole accuracy, compared with 0.863 for the anchor-only control (Holm-adjusted p=0.767p=0.767) and 0.865 for label-matched supervised contrastive JEPA with a probe. The rule-query benefit is thus distinct from a claim of statistically significant accuracy improvement. The shared interface also operates on five frozen-feature video benchmarks, with held-out rule AUROCs above 0.84 on four; we explicitly report the fifth failure case (0.365).

4. The novelty is the integrated predictive interface, not the margin loss in isolation. To our knowledge, RiJEPA is the first JEPA-style architecture to use one learned predictor for both observation prediction and explicit antecedent–consequent rule queries, thereby supporting symbolic-pole read-out, rule completion, and abductive proposals within one framework. We do not claim the JEPA objective or shared weights are uniquely necessary for every measured tabular gain; the controlled component ablations clarify that scope. The contribution is the unified, quantitatively evaluated data-rule interface rather than a new contrastive margin or replacement for conventional rule mining.

5. Two new figures further clarify the learned representation geometry. We added Fig. 4, comparing classic JEPA and RiJEPA latent representations across all 15 tabular datasets, and Fig. 9, extending this comparison to five video-feature benchmarks. The visualizations illustrate how RiJEPA organizes predicted states around symbolic class poles, with clearer class separation on most datasets. Quantitatively, the mean tabular silhouette improves from 0.101 to 0.563 relative to classic JEPA, while RiJEPA achieves higher test silhouettes on four of five video benchmarks. These figures make the role of the complete rule-informed, label-anchored architecture more tangible and complement the quantitative evidence for its predictive and symbolic capabilities.

We have also updated a new rebuttal PDF version (v3).

We believe this architectural and functional contribution outweighs the remaining limitations of the present evaluation, and respectfully hope the reviewer will reconsider its significance in light of the revised evidence.

Reviewer_DC5FOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3(+2)
Read review
Paper summary

The paper introduces RbJEPA (rule-based JEPA) and then RiJEPA (rule-informed JEPA), the hybrid which accepts both data and rules instead of purely data or purely rules. RiJEPA has a energy-based constraint (EBC) that assigns valid rules low energy and invalid rules high energy. This differentiable energy is then used to perform gradient-guided rule discovery. The architecture uses a data encoder, a rule encoder and a shared predictor which shares the same embedding space. This allows the eventual guided discovery of rules to be a continuous gradient-guided search (Langevin diffusion) instead of a combinatorial problem. Synthetic data and UCI heart disease dataset was used to demonstrate properties of their model.

Critical strengths

Paper is presented in an easy to read manner, with appropriate figures and tables.

RiJEPA, the hybrid which creates a shared latent embedding space with rules and data, is to the best of my knowledge novel and shows some promise in their preliminary results.

Critical weaknesses

It is confusing what the paper is trying to evaluate. After all the prior explanation and setup, there is a confounding inclusion of an anchor in the loss in Eq (6). Is it difficult to tell the effects of anchor and EBC without robust ablation studies and comparison.

The interpretability claims are not convincing. The plausible rules that have been retrieved are merely plausible associations, but has not explained the prediction itself. It would also be nice to see the rule being verified by an additional forward pass to see if it is indeed faithful to the learned model.

I would like to see the comparison against combinatorial search. While a full exhaustive search is expensive, a sampled search could be conducted and a comparison of compute resources used would provide fairer evidence for the claims made in the paper.

The evaluation is limited to a single non-synthetic dataset, which provides weak evidence on whether RiJEPA results would extend to other datasets.

Furthermore, the lack of error bars and significance tests does not make the claims strong. I would prioritize more datasets and repeated trials to provide stronger evidence for the empirical claims.

Questions for authors

Please address the weaknesses above.

4 comments

Thank you for identifying the need to disentangle supervised anchoring from EBC, verify rule faithfulness, compare continuous discovery with discrete search, and establish reliability beyond a single dataset. We have reorganized the evaluation around three questions: (i) does rule-informed training improve predictive representations beyond supervised anchoring? (ii) do learned rule energies support faithful symbolic queries? and (iii) how does continuous rule discovery compare with discrete search? The revised paper addresses these through matched ablations, forward-pass tests, search comparisons, and repeated tabular and video experiments.

1. Matched anchor and rule controls. Across 15 selected tabular datasets (2×5 stratified test folds), data-only JEPA + probe attains mean accuracy 0.843, JEPA + the same supervised pole anchor but no rules attains 0.863, and full RiJEPA poles attain 0.868 (main Table 3; Appendix H, Table 8). The incremental +0.005 over anchor-only is not statistically established (Holm-adjusted p=0.767p=0.767); balanced accuracy is 0.839 for both. Thus we do not attribute the classification gain to EBC alone. Its rule-specific effect is clearer: adding a negative margin to an unanchored positive-pair model improves matched-versus-flipped rule-energy AUROC 0.735→0.966 and completion on previously unmined antecedents 0.704→0.839 (Fig. 4(a); Appendix H). Within the fully anchored model, removing negatives gives a smaller contrast (0.941→0.956 AUROC). We distinguish these comparisons and explicitly describe the label-supervised anchor in Sec. 3 and Fig. 2.

2. Forward-pass consistency and explanation fidelity. Feeding decoded abductive antecedents through the rule encoder and shared predictor recovers their consequents in 0.950 of cases (strongest mined rules: 1.000; random antecedents: 0.498). However, only 0.651 of generated antecedents match at least three held-out records. Among these supported cases, data-pathway predictions agree with the proposed class in 0.845 and observed labels agree in 0.822. For held-out records, the lowest-energy applicable mined rule agrees with the data model's prediction in 0.775 of cases, versus 0.861 for the highest-confidence applicable rule (Appendix J). These are measured forward consistency and local fidelity—not proof of causal explanation or logical soundness.

3. Discrete search and measured compute. We compared methods requesting 64 rules per class with three-condition antecedents. The following are averages across 15 datasets on one CPU core (Appendix J, Table 10):

ProcedureCandidate evaluationsHeld-out validTime (s)
Latent Langevin abduction6,4000.5400.055
Sampled search, energy-ranked64,0000.3320.608
Sampled search, training-count ranked64,0000.7300.671
FP-Growth, full frequent-rule baseNot matched0.7877.201

Langevin is fast under this implementation and budget, but its validity is below count-ranked search and FP-Growth. This is not a compute-matched benchmark: Langevin uses 6,400 candidate evaluations, sampled search uses 64,000, and FP-Growth produces an entire frequent-rule base. We therefore present latent search as a complementary differentiable proposal mechanism, not as universally faster or more valid. On general held-out rule candidates, learned energy also trails training-row confidence in validity discrimination (AUROC 0.845 vs. 0.924; Sec. 5.3, Appendix I).

4. Broader evaluation, variance, and significance. The paper now includes 15 tabular datasets with ten test folds each and five video benchmarks with five model seeds on fixed test splits (main Tables 3–4; Appendices G, H, K). Video test accuracy improves from 66.3±3.6% to 80.1±0.7% on KTH (p=0.001p=0.001), 51.9±6.7% to 68.6±5.3% on UT-Interaction (p=0.001p=0.001), and 66.2±4.9% to 77.1±2.1% on MNNIT Shoplifting (p=0.009p=0.009); HMDB51 and RLVS show no statistically established pole-readout improvement. These pp-values are paired tests across five seeds on each fixed split, not evidence of generalization across new splits. The 15 datasets were selected after exploratory screening, video inputs are frozen ResNet-18 features, and UT/MNNIT have only 37/55 test clips. We state these limits explicitly.

Together, the experiments establish what the learned interface can do, i.e. rule discrimination, completion, and measured predictive consistency, while distinguishing these capabilities from classification gains due to supervision or guaranteed superiority to discrete search. We appreciate the reviewer's suggestions, which substantially improved the experimental design and interpretation.

Area_Chair_rK56OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

I thank the authors for their revision and rebuttal, and have also read the other rebuttals along with the corresponding reviews. I think the manuscript has improved significantly, especially on the evaluation (i.e., significance and datasets tested). I have updated the soundness score as it is now better backed by evidence.

Thank you for revisiting the paper, and for recognizing the substantial improvement in the evaluation and updating your assessment. The reviewers suggestions directly motivated the matched anchor-versus-EBC ablations, forward-consistency checks, discrete-search comparisons, and repeated evaluations on 15 tabular datasets and five frozen-feature video benchmarks.

We hope this unified interface between predictive representation learning and symbolic reasoning represents a meaningful contribution beyond improvements in classification accuracy. Thank you again for helping us make both its value and limitations more precise.

Reviewer_y1x4OpenReview ↗
Rating
1
Confidence
2
Informativeness
3
Presentation
3
Soundness
1
Read review
Paper summary

This paper suggests the RiJEPA loss formulation that adds conformity with symbolic rules to the training of Joint Embedding Predictive Architectures (JEPAs). The key tricks are to (1) consider the learned transition function g(a) = b as model of an implication a->b on latents a, b; and (2) to embed and align formal superclass descriptions A, B as prototypes into the observations' latent space. A simple contrastive loss formulation on sampled correct and incorrect antecedent-consequent pairs is suggested. As interesting applications of this latent space structure, the authors suggest decoding of discovered consequents into formal symbols for human assessment (Paradigm 2); and to locally optimize either antecedent or consequent to not only do forward inference, but also abduction and rule discovery. Experiments on a tiny 3D controlled simulation evidence a strong effect of the rule-based regularization on latent space structure, with similar results on a real-world few-sample medical dataset (UCI Heart Disease case study).

Critical strengths
  1. The use of a push-pull Energy-Based Constraint (EBC) to align continuous latent spaces with discrete logical rules is to my knowledge novel and clearly an interesting approach.
  2. The neat idea of aligning superclass prototypes to instances in latent space for joint implication learning is nicely formalized.
  3. The experimental validation also suggests an interesting real-world dataset, and features promising preliminary results.
Critical weaknesses

My key concerns summarize as:

  1. Too small experimental setup where overfitting might be the core cause for the observed clear difference between model spaces; only two small-scale setups, no standard deviation, relevant baselines missing. This is also the key reason for my low soundness rating and reject decision. Adding respective results would be a major revision in my opinion (if I haven't completely overlooked something).
  2. The limitations arising from the (possibly extensive) need for negative antecedent-consequent samples are neither explored, nor discussed.
  3. Several alternative formulations are introduced in the paper but not considered in the experiments.

This is detailed in the following.

Experiments: Experimental validation is restricted to a low-dimensional 3D simulation and a small tabular dataset of 303 patients. Given that JEPAs are fundamentally designed to process high dimensional, unstructured data (images and video), the architecture's scalability to target modalities remains unproven. Furthermore and more severely, experiments are not only on rather small setups, but do not report any standard deviation, and partly not even quantitative results. Two single experimental runs cannot count as any experimental proof, but rather a small case study and (interesting and promising, but) preliminary results. Hence, in its current form, I don't see sufficient evidence for the strong claims of the paper about superiority over classic JEPA and would recommend a major revision with substantial addition of experiments. Lastly and maybe most severely on the experimental side, I am missing the direct comparison against contrastive learning of JEPA. It is known that contrastive learning regimes yield better latent space structures in terms of separating contrasted classes. It might be that contrastive training without the anchor loss already yields most of the favorable properties, even without the alignment to interpretable superclass symbols.

Alternative formulations: Several alternative formulations like RbJEPA and paradigm 2 are introduced in the paper but not at all considered in the experiments or otherwise validated: Particularly the contribution of RbJEPA is in not clear in the paper. The paper introduces RbJEPA as one of the contributions and makes claims but doesn't say anything about it. It's briefly touched upon in section 3 but no mention of it in the results. This space could be better used.

Apart from the major concerns, the structure requires further polishing. Most critically, Eq. (7) only in the experimental section introduces a key part of the approach, namely the "anchor loss" that ensures that the instance embeddings are aligned with the superclass embeddings. Also, the separation of Secs. 3 & 4 is unclear. Sec. 3 introduces the learning objective and parts of the architecture, Sec. 4 suggests inference strategies and adds a decoder to the architecture (missing from Fig. 2!). Table 1 is never referenced in the text.

The authors claim that "without rule encoders its latent coordinates cannot be translated into discrete clinical concepts.". There is an abundance of literature in the direction of post-hoc concept-based explainable AI that provides methods and techniques for probing exactly that, particularly if the symbolic terms are supposed to origin from a predefined dictionary. The practical benefit of this ante-hoc observation-symbol alignment over classical multi-modal alignment (like in vision-language models) or post-hoc techniques is therefore unclear and should be argued.

Questions for authors
  1. Details on experimental results:
    1. What were standard deviations of the two experimental settings?
    2. How sensitive is the resulting RiJEPA latent geometry to the hyperparameters of rule extraction phase (minimum support and confidence thresholds)?
    3. Does the nearest-neighbor decoding remain robust in more densely populated latent spaces?
    4. In results section (line 323), the statement "85x times increase" refers to RiJEPA's OOD energy compared to ID energy. Could you please clarify: Relative to what? And if it is relative to ID energy, then the number seems to be way higher. How is this statement then calculated?
  2. Will the rule-mining scripts be released to ensure full end to end reproducibility?
2 comments

Thank you for the detailed review. We expanded the evaluation, separated EBC from the supervised anchor, and tested previously unvalidated alternatives.

1. Scale, variance, and baselines. We evaluate 15 public binary tabular datasets, with 2×5 stratified folds and training-only preprocessing/rule mining (Sec. 5.2; App. G–H). Mean accuracy/balanced accuracy is 0.843/0.798 for data-only JEPA + probe versus 0.868/0.839 for RiJEPA's label-informed poles (Table 3). These datasets were selected after screening; across 115 screened datasets, pole accuracy was higher on 47/115 datasets (median difference −0.002), and balanced accuracy on 81/115 (median +0.012). We do not claim universal superiority.

Five video tasks use 8 frozen ResNet-18 frames (4,096 features), five seeds and fixed splits (Table 4; App. K): KTH 66.3±3.6% → 80.1±0.7% (p=.001p=.001), HMDB51 23.2±0.8% → 23.3±1.1% (n.s.), UT-Interaction 51.9±6.7% → 68.6±5.3% (p=.001p=.001), MNNIT 66.2±4.9% → 77.1±2.1% (p=.009p=.009), and RLVS 77.5±1.4% → 78.3±1.8% (n.s.), comparing JEPA probes to RiJEPA poles. These are not end-to-end video results; UT/MNNIT have 37/55 test clips, and seed tests do not measure split variance. For the 3D simulation, 20 additional seeds yield invalid-region energy 14.640±3.610 for RiJEPA versus 0.142±0.032 for JEPA (App. I).

2. Anchor versus EBC and contrastive controls. A label-matched supervised contrastive JEPA + probe achieves 0.865 mean accuracy, versus RiJEPA poles 0.868 (Holm-adjusted p=1.0p=1.0). JEPA with the same supervised pole anchor but no rules achieves 0.863 (full RiJEPA 0.868, p=.767p=.767); both have balanced accuracy 0.839. Thus the anchor explains most classification gains. EBC contributes rule-query discrimination: in an unanchored staged comparison, adding negatives raises matched-versus-flipped rule-energy AUROC 0.735→0.966 and new-antecedent completion 0.704→0.839; within the full anchored model, the margin's smaller AUROC contrast is 0.941→0.956 (Table 8; Fig. 4(a)). We do not attribute class separation entirely to EBC.

3. RbJEPA, negatives, thresholds, decoding. Positive-pair-only RbJEPA has nearly coincident class poles (cosine >0.99>0.99 in 14/15 datasets); adding EBC to rules-only RbJEPA yields rule AUROC 0.962 (App. I). A one-fold negative-coverage sweep finds 0.859 pole accuracy / 0.932 rule AUROC with negatives on 10% of antecedents, versus 0.856/0.952 at 100%. Coverage placement also matters: in the simulation, unseen-region FPR95 is 0.215 with concentrated negatives versus 0.123 with broader coverage. Flipped consequents do not cover all invalid rules or guarantee OOD detection. Across nine support/confidence settings, the median-dataset pole-accuracy range is 0.018, although rule AUROC is more sensitive. Nearest-item recall drops to 0.452/0.447 for novel two-/three-condition antecedents in denser vocabularies. Supervised post-hoc JEPA concept probes achieve AUROC 0.931, versus 0.552 for direct RiJEPA item decoding under different read-out procedures: we withdraw the claim that JEPA concepts cannot be decoded (App. I).

4. Both reasoning paradigms. Rule completion on unmined antecedents achieves 0.836 accuracy versus 0.750 for lookup. Section 4 now defines the Paradigm-2 context energy Ectx(zc)=−log⁡p^ctx(zc)E_{\rm ctx}(z_c)=-\log\hat p_{\rm ctx}(z_c) from a KDE of training antecedents, followed by Langevin proposals, shared prediction, decoding and training-confidence filtering. Held-out validity is 0.647 before filtering and 0.677 after, versus 0.246 for joint Langevin (Paradigm 1) and 0.237 for random proposals (Table 5; Fig. 4(b); App. J). This acceptance is Metropolis-style, not exact MH. Count-ranked search and FP-Growth can yield more valid rules; latent search is a differentiable proposal method, not guaranteed-valid or superior discrete search.

5. 85×, structure, and reproducibility. The submitted 85× is 12.713/0.150=84.8, between models on the same invalid pairs, not within-RiJEPA invalid/valid energy; across 20 further seeds the ratio is 111±46 (App. I). The complete three-term loss, including the label-supervised anchor, is now in Sec. 3; Fig. 2 shows the decoder and anchor, Table 1 is referenced, and Secs. 3–4 distinguish learning from inference. Pole read-out is classifier-free but label-informed, not zero-shot. Appendices E/G document preprocessing, mining thresholds, canonical rule sorting and training; we will release the rule-mining and evaluation scripts.

The revised evidence supports a differentiable data–rule interface with explicit limits on classification and decoding claims.

Area_Chair_rK56OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_PYfpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes RiJEPA, which incorporates symbolic rules with Joint-Embedding Predictive Architectures (JEPAs). RiJEPA learns from raw data and symbolic rules simultaneously. The key component of incorporating symbolic rules is Energy-Based Constraints (EBC), which embeds combinatorial rule spaces into differentiable embedding manifolds, where valid rules correspond to low energy. Experiments on controlled simulation and a clinical case study indicate the good performance of RiJEPA.

Critical strengths

None

Critical weaknesses

I would be open-minded if the authors could clarify the following points.

  • Proposition 1 proves that zero loss implies zero energy for valid rules. However, it seems that this proposition does not prove that low energy implies rule validity. Can we extract valid rules from low energy? This is important since the energy never goes to exact zero according to experimental results.
  • Line 357-368: An anchor loss is added in addition to EBC loss, which raises the concern about which component is effective. Due to the anchor loss, the experiments do not isolate the effect brought by symbolic rules. A comparison between JEPA loss + anchor loss and JEPA loss + anchor loss + EBC loss would help. An experiment with JEPA loss + EBC loss would further help in understanding the effect of symbolic rules.
Questions for authors

See Weaknesses.

2 comments

Thank you for identifying two central questions: whether low rule energy can recover valid rules, and whether EBC contributes beyond the supervised anchor. We have clarified Proposition 1 and added matched ablations addressing both comparisons requested in the review.

1. What does Proposition 1 guarantee, and if low-energy rules can be extracted. Proposition 1 states a zero-loss implication for finitely constrained pairs: positive pairs then have zero energy and sampled negatives have energy at least the margin. It does not establish the converse, nor require training to reach exactly zero loss. We have corrected this scope in Sec. 3. In practice, learned energy ranks candidate rules; after decoding, their validity must be checked against held-out data or explicit logical constraints.

Across 15 datasets, low energy ranks general held-out candidate validity (confidence ≥0.7\geq 0.7, among candidates with sufficient antecedent support) with mean AUROC 0.845 (chance: 0.5). The lowest-energy 5% contain 53.4% empirically valid rules, against a 29.5% candidate base rate. Training-row confidence is a stronger comparator (AUROC 0.924; 90.2% valid in its top 5%). These are empirical association tests, not logical soundness guarantees (Sec. 5.3; Appendix I, Table 9).

2. Both anchor/EBC comparisons requested by the reviewer. We directly evaluated (i) JEPA + anchor versus JEPA + anchor + EBC, and (ii) JEPA + EBC without the anchor, within a seven-variant controlled study using matched folds and training settings (Appendix H, Table 8). Selected mean results across 15 exploratorily selected tabular datasets are:

Training configurationPole accuracyRule AUROC
Data-only JEPA + probe— (probe: 0.843)—
JEPA + positive rule pairs, no anchor0.4940.735
JEPA + positive/negative EBC, no anchor0.4840.966
JEPA + supervised poles, no rules0.863—
Rule poles + anchor, no paired-rule loss0.8420.517
Full RiJEPA: JEPA + EBC + anchor0.8680.956
Full RiJEPA without negative margin0.8450.941

Rule AUROC here tests held-out matched-versus-flipped consequents, not the general-candidate validity AUROC in point 1.

For comparison (i), adding the rule pathway and EBC to the anchored model changes pole accuracy from 0.863 to 0.868; balanced accuracy remains 0.839 and the accuracy difference is not established statistically (Holm-adjusted p=0.767p=0.767). Thus we do not attribute the classification gain uniquely to EBC. For comparison (ii), JEPA + EBC without anchoring achieves 0.484 pole accuracy but 0.840 supervised-probe accuracy and 0.966 rule AUROC. Its poor pole read-out reflects missing supervised pole alignment, not an absence of useful learned representations or rule energies.

The role of EBC is clearest for symbolic queries: adding negatives to unanchored positive-pair supervision improves rule AUROC 0.735→0.966 and completion of previously unmined antecedents 0.704→0.839. Within the fully anchored model, removing negatives produces a smaller rule-AUROC contrast (0.941→0.956). We distinguish these two ablations rather than treating them as the same effect (main Table 3, Fig. 4(a); Appendix H). The 15 datasets followed exploratory screening, so their statistics are not independent confirmatory evidence.

3. Objective terms. Section 3 now states the three-term objective, including the label-supervised anchor, mean-squared rule energy over 32 latent dimensions, and averaging over rule pairs. Binary tabular negatives reuse each mined antecedent with the opposite class consequent. The separate 3D simulation uses its original squared-norm energy and no anchor. The retained empirical evaluation spans the controlled simulation, 15 tabular datasets, 5 frozen-feature video benchmarks, and rule completion/validity tests (Secs. 5.1–5.3; Appendices G–K).

We thank the reviewer for prompting us to distinguish finite-pair theoretical separation, empirically assessed validity of unseen candidates, and the separate roles of supervised anchoring and rule-energy constraints.

Area_Chair_rK56OpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Comments on the paper

2 comments
AuthorsOpenReview ↗

manuscript PDF updated (rebuttal version)

We thank all reviewers for their constructive feedback. We have revised RiJEPA to address theory, the anchor–EBC confound, experimental scale, rule faithfulness, and reasoning evaluation. Changes appear in red in the updated PDF; key results are in Tables 3–5 and Figure 4, with full evidence in Appendices G–K.

1. Clarified the theory and scope of rule validity (Sec. 3). Proposition 1 guarantees zero-loss separation only for constrained positive and negative pairs, not validity of arbitrary unseen low-energy rules. Decoded candidates require independent verification. We clarified energy scaling and negative construction.

2. Specified the complete training objective and architecture (Secs. 3–4; Fig. 2). The main method now explicitly includes masked JEPA prediction, positive/negative EBC, and the label-supervised symbolic-pole anchor. Figure 2 shows the anchor and symbolic decoder. We distinguish JEPA prediction from the metric-learning margin; pole classification is classifier-free but label-informed, not zero-shot.

3. Separated anchoring from rule supervision with matched baselines (Table 3; Appendix H). We evaluated seven objective variants, including JEPA + anchor without rules and JEPA + EBC without anchor, plus a supervised contrastive baseline. Mean pole accuracy across 15 tabular datasets is 0.863 for anchor-only versus 0.868 for full RiJEPA (Holm-adjusted p=0.767p=0.767); supervised contrastive JEPA + probe reaches 0.865. The classification gain is not uniquely attributable to EBC. For rule discrimination, in an unanchored staged ablation, matched-versus-flipped rule AUROC improves 0.735→0.966 (Fig. 4(a)); the incremental difference within the fully anchored model is smaller (0.941→0.956).

4. Expanded the empirical evaluation (Secs. 5.2–5.3; Appendices G, H, K). We evaluate 15 tabular datasets with two stratified five-fold repeats and five video-action/event benchmarks with five seeds. RiJEPA's pole read-out improves over JEPA probes on KTH (+13.8 pp), UT-Interaction (+16.8 pp), and MNNIT (+10.9 pp), but changes on HMDB51 and RLVS are negligible (Table 4). Video tests use frozen ResNet-18 frame features, not end-to-end video JEPA. We disclose exploratory selection, the 115-dataset screen, and fixed-split limitations.

5. Tested rule supervision, negative coverage, and decoding robustness (Appendices H–I). Rules-only RbJEPA, negative-margin ablations, rule-mining thresholds, and negative-sample coverage are now evaluated. Positive-only RbJEPA frequently exhibits nearly coincident class poles; negative EBC improves rule-pair discrimination. At 10% negative coverage, rule AUROC is 0.932 versus 0.952 at full coverage; nearest-item recall for novel two-/three-condition antecedents is 0.452/0.447. We also compare supervised post-hoc concept probes.

6. Evaluated whether low energy predicts unseen rule validity (Sec. 5.3; Appendix I). General-candidate validity AUROC is 0.845 for learned energy versus 0.924 for training-row confidence. The lowest-energy 5% of candidates are 53.4% empirically valid versus 29.5% overall prevalence. Energy helps rank proposals but does not certify validity.

7. Quantified both generative reasoning paradigms (Sec. 4; Table 5; Fig. 4(b); Appendix J). Rule completion reaches 0.836 versus 0.750 for lookup. Paradigm 2 now specifies KDE context energy, sampling, decoding, and a training-only confidence filter. Held-out rule validity is 0.677 after filtering (0.647 before), versus 0.246 for joint Langevin and 0.237 for random proposals. Acceptance is Metropolis-style, not exact Metropolis–Hastings.

8. Added faithfulness, discrete-search, replication, and reproducibility checks (Appendices I–J). Generated-rule forward consistency is 0.950; lowest-energy applicable mined rules agree with model predictions in 0.775 of held-out cases, versus 0.861 for a confidence-ranked rule. Count-ranked search and FP-Growth outperform Langevin in rule validity under unmatched compute budgets; we claim no universal search advantage. We replicate the simulation over 20 additional seeds, clarify the 84.8× between-model OOD-energy comparison, and document evaluation and rule-mining protocols.

we hope these revisions establish RiJEPA's contributions more precisely: a shared predictive interface for observations and symbolic rule queries, with measurable rule-energy and reasoning capabilities. We separate supervised classification, empirical validity, and theoretical guarantees. We thank all reviewers for helping us improve the paper.

AuthorsOpenReview ↗

updated rebuttal manuscript (v3)

We thank all reviewers for their constructive feedback and suggestions for improving this work. The updated v3 revision builds on the original submission and rebuttal v2. Changes remain highlighted in red.
Improvements relative to the original submission (introduced in the rebuttal):

  1. Clarified the theory and complete objective (Secs. 3–4). Proposition 1 establishes separation only for finitely constrained positive/negative rule pairs at zero loss, not logical validity of arbitrary low-energy rules. The method now explicitly distinguishes masked JEPA prediction, the positive/negative EBC margin, and the label-supervised class-pole anchor; pole read-out is classifier-free but not label-free.

  2. Added matched evaluations of rule supervision and anchoring (Table 3; Appendix H). Across 15 selected tabular datasets with two stratified five-fold repeats, full RiJEPA achieves 0.868 pole accuracy versus 0.863 for anchor-only JEPA (Holm-adjusted p=0.767p=0.767); label-matched supervised contrastive JEPA reaches 0.865 with a probe. Adding negatives without anchoring improves held-out matched-versus-flipped rule AUROC from 0.735 to 0.966. In the fully anchored setting, AUROC reaches 0.956 versus 0.517 for the rule-pole anchor control without paired-rule training. These results distinguish symbolic-query benefits from classification effects.

  3. Expanded experimental scope and symbolic-query evaluation (Sec. 5; Appendices G–K). We evaluate five frozen-feature video benchmarks (five seeds), negative-coverage and decoder robustness, general-candidate rule validity, forward consistency, and quantitative reasoning. Completion of previously unmined antecedents reaches 0.836 versus 0.750 for lookup. Context-marginal proposals achieve 0.677 held-out empirical validity after training-only filtering, versus 0.246 for joint Langevin and 0.237 for random proposals. Learned energy ranks general-candidate validity at 0.845 AUROC versus 0.924 for training-row confidence; we also report where conventional discrete search performs better. Further improvements in v3 relative to rebuttal v2:

  4. Expanded dataset and protocol transparency (Appendix G). Detailed tables now describe all 15 tabular datasets and five video benchmarks, including features, targets, data splits, rule counts, thresholds, and shared optimization settings. We also retain the broader 115-dataset exploratory-screen context to clarify the scope of the reported averages.

  5. Added comprehensive latent-space visualizations (Fig. 4; Fig. 9). Side-by-side t-SNE visualizations compare classic JEPA and RiJEPA across all 15 tabular datasets and five frozen-feature video benchmarks. They illustrate the organization of predicted representations around symbolic class poles, while also displaying datasets with overlapping classes. Quantitative silhouette scores complement these visual comparisons.

  6. Provided quantitative evidence of improved representation separation (Sec. 5; Appendix H). The mean predicted-state silhouette increases from 0.101 for classic JEPA to 0.563 for RiJEPA across the tabular datasets, and RiJEPA achieves higher test silhouettes on four of five video benchmarks. These findings demonstrate stronger class separation under the complete RiJEPA objective, including supervised symbolic-pole anchoring, rather than attributing the effect to EBC alone.

  7. Refined the methodological conclusion and mathematical presentation. We emphasize the distinction between label-aligned classification and the additional rule-query capabilities enabled by paired-rule training, clarify the novelty of the shared data–rule predictive interface, and tighten the formulation of continuous rule-energy regions. We also completed an anonymity audit of the manuscript source.

Overall significance. To our knowledge, RiJEPA introduces a first-of-its-kind JEPA-style framework connecting observation prediction and explicit symbolic rule queries through a shared learned predictor. Its demonstrated rule-energy discrimination, rule completion, abductive candidate generation, and improved organization of latent representations around symbolic poles provide complementary capabilities beyond conventional JEPA prediction. We believe this unified neural-symbolic modeling perspective, supported by the expanded experimental evidence and theoretical analysis, represents a useful methodological contribution whose value outweighs the remaining limitations. We respectfully hope the reviewers and Area Chair will reconsider the paper's overall significance in light of these improvements, and thank them again for their constructive guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.