acceptodds
Under review as a conference paper at ICLR 2027

MoJEPA: Joint-Embedding Predictive Learning for Dynamic Graphs with Discrete Morse Theory

Abstract

Self-supervised pretraining on dynamic graphs can reduce reliance on task-specific labels and support transfer across datasets. In joint-embedding predictive architectures (JEPAs), the learning task is jointly defined by what is masked and what context remains available for prediction. On dynamic graphs, these choices face two fundamental coupled challenges. First, masking interactions also removes relational evidence needed for prediction. Second, changing interactions limit the context available at each time, leaving useful relations observed at other times unavailable for prediction. To address these challenges, we design _MoJEPA_ which combines partial interaction masking with a shared Morse reference forest. We partially mask interactions so that the model can predict missing evidence from the relations that remain visible. To enrich this predictive context, we introduce _Morse reference context_, which leverages discrete Morse theory from low-dimensional topology to reduce the relations observed across a window to a sparse rooted forest. This allows the relations observed at other times to support information exchange at the target time. A relation observed at one time can subsequently support the exchange of node states at another, even when the corresponding interaction is absent. This combination allows prediction to use context beyond the interactions present at an individual time. Our experiments demonstrate that MoJEPA leads on all 9 within-dataset tasks and 15 of 18 transfer directions, with relative gains over the strongest external baseline up to 13.13% and 12.27% for within-dataset prediction and transfer respectively.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Dynamic Graph Representation Learning, Joint-Embedding Predictive Architecture, Discrete Morse Theory, Self-Supervised Learning, Temporal Link Prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_HDjGOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper describes a JEPA approach for prediction within dynamic networks addressing the issue that masking edges for testing hides additional relational structure beyond the removed edge. Using a teacher with access to additional reference context, a student can pre-learn representations to predictor those from the teacher. The leads to big improvements for prediction within a dataset but also creates an embedding function that allows for transfer between similar dynamic networks.

The paper focusses on a Morse reference context provided by an spanning forest derived from expected random-walk hitting times. It is unclear what the benefits of describing the forest in this language, primarily as the details required to understand the point of Theorem 4.2 is lost due to missing notation. As I understand the work, any spanning forest could be used as a reference context and further work to explore which spanning forests give better results is needed.

As the authors claim, this is a first step in incorporating JEPA into dynamic networks by including network topology. This is a very interesting and potentially fruitful approach that is highly worthy of further developments.

Critical strengths

The JEPA architecture to use context windows within dynamic networks is a good idea and the setup is very well-explained in the paper with helpful diagrams throughout. Overall, the research was easy to follow for someone new to JEPA methods, which made it more noticeable when some details were missing (see below).

Critical weaknesses

My main concern is that MoJEPA may be restricting itself to only using one type of reference context when other approaches to spanning forest selection could be explored. It gives the impression that only this one approach works and could limit further developments. It was unclear from the ablation studies in Section 5 whether this is actually being considered already.

Some sections lacked necessary detail, for instance, more should be provided in Section 3 (Preliminaries) although the rest of the paper was written well enough to fill in some of the missing gaps here. Noticeably, Theorem 4.2 lacked definitions for dSkd_\text{Sk} and S(U,g)\mathcal{S}(U,g) necessary for explaining its purpose, when motivation for random-walk hitting times based reference forests was needed over alternate approaches.

Questions for authors
  • Does the theory extend to any rooted forest for provide JEPA benefits?
  • How does link prediction perform in the case where η=0\eta = 0 in Equation 5? It would be interesting to know if the historical statistics dominate the scoring and evaluate the benefit from the MoJEPA node representations. It was unclear if this was being performed in the ablation study.
  • The paper describes within-dataset prediction (transductive learning) but goes to the extreme when considering domain transfer (inductive learning). How does MoJEPA perform within-dataset inductive learning?
4 comments

W1: Flexibility of reference forest construction.

Thank you for this question. MoJEPA is not limited to hitting time. Scalar fields guide reference-route selection. Original Table 2 already tests hitting time, hub distance, positive/negative strength and negative net flow. We also test alternative spanning forests.

1. Matched reference forests

  • Hitting time: Weighted random-walk-guided routes.
  • Same-hub BFS: Shortest-path routes to the same hubs.
  • Random: Random spanning forests.
  • Maximum-weight: Forests favoring high-weight historical edges.

All use identical history, one tree per observed component, edge counts, relation-type slots, and training/evaluation settings. MRR ×100, mean ± SD over three seeds (W: within; T: transfer).

Table 1. Matched reference-forest comparison.

ForestToken WToken TQA WQA T
Hitting time78.14±0.4776.11±0.2957.39±0.0856.58±0.07
Same-hub BFS77.80±0.3375.89±0.2857.37±0.0756.54±0.11
Random77.52±0.2875.19±0.2057.20±0.0656.50±0.05
Maximum-weight77.49±0.3775.84±0.5057.25±0.0456.47±0.11

Finding. All four forests can be used within MoJEPA. Hitting time has the highest mean across these settings, with larger gaps on Token than QA. Because component coverage and edge budgets are fixed, the comparison examines which historical routes are selected, rather than how many reference edges are supplied. In fact, BFS is a special case of scalar-field-guided construction: hub distance produces shortest-path trees, and matching tie-breaking recovers BFS parent choices. Random and maximum-weight forests show architectural flexibility, although the minimum-neighbor rule does not generate every rooted forest.

W2: Clarifying Theorem 4.2 and the hitting-time motivation.

Thank you for requesting clearer definitions. Theorem 4.2 asks how the selected forest changes when graph support or node scores change.

1. Skeleton distance and structural stability

Let S(U,g)S(U,g) be the Morse skeleton selected from support graph U=(V,E)U=(V,E) by injective comparison scores gg, comprising extended parent pointers p~\tilde p and unmatched (critical) edges EcritE_{crit}. We define

dSk(S,S′)=∥v:p~(v)≠p~′(v)∥+∥Ecrit△Ecrit′∥.d_{Sk}(S,S')=\|{v:\tilde p(v)\ne\tilde p'(v)}\|+\|E_{crit}\triangle E'_{crit}\|.

The first term records parent, root, and vertex-presence changes; the second records changes in critical-edge membership. For each vertex (v)(v), the local margin mvU(g)m_v^U(g) is the gap between the two smallest scores in its closed neighborhood, and is infinite for isolated vertices. On fixed support, a pointer is unchanged when the maximum score perturbation in that neighborhood is less than mvU(g)/2m_v^U(g)/2.

Theorem 4.2 combines score and graph changes:

dSk(S(U,g),S(U′,g′))≤∥V△V′∥+2∥E△E′∥+3min⁡(∥HU∥,∥HU′∥),d_{Sk}(S(U,g),S(U',g'))\le \|V\triangle V'\|+2\|E\triangle E'\|+3\min(\|H_U\|,\|H_{U'}\|),

where HUH_U contains vertices whose local score changes reach their margin threshold (and likewise for U′U'). The bound separates support edits from changes in the field used to select parents. If hitting times are recomputed after observations change, their score changes must also enter the bound; a small support edit alone does not guarantee unchanged routes. This is a conditional guarantee about pointers and critical edges, not full root paths, embeddings, or prediction error.

2. Why hitting time is a motivated default

For hub hh, the expected random-walk hitting time satisfies H(h)=0H(h)=0 and H(v)=1+∑uP(v,u)H(u))for(v≠hH(v)=1+\sum_u P(v,u)H(u)) for (v\ne h, including self-transitions. Thus the weighted average of neighboring scores is H(v)−1<H(v)H(v)-1<H(v), so every non-hub vertex has a distinct neighbor with strictly smaller hitting time. Strict descent cannot cycle, and the hub is the unique root. Proposition 4.1 therefore guarantees one spanning tree per observed component with exactly ∥V∥−β0(U)\|V\|-\beta_0(U) reference edges.

Hitting time also reflects interaction weights through random-walk transition probabilities. Although each parent is chosen locally, its score incorporates connectivity throughout the component, allowing routes beyond shortest-hop paths. This gives a principled way to select sparse historical communication routes when their predictive usefulness is unknown in advance.

For fixed support and a fixed hub, hitting times vary continuously with positive edge weights through the grounded linear system; the local-margin condition then identifies when parent choices persist. A change of hub may alter the field more broadly. We distinguish raw hitting times from the injective comparison scores obtained by deterministic tie-breaking. Hub distance likewise guarantees one root per component, while other fields can produce more roots; hitting time is not assumed universally optimal.

We will update the notation, stability discussion and hitting-time motivation in the revised manuscript as soon as possible.

Q1: Scope of the Morse construction and its role in JEPA.

Morse theory relates rooted forests to discrete gradients; scalar fields guide the historical routes selected for learning.

  1. General correspondence. Theorem B.3 establishes a bijection between discrete gradients on a graph and rooted spanning forests. For any rooted forest with roots R, the identities \|E_F\| \= \|V\| − \|R\| and \|E \\ E_F\| \= β₁(U) + \|R\| − β₀(U) hold independently of its construction.
  2. Scalar-field selection. Our minimum-neighbor rule uses a node field to select descending parent edges. Theorem 4.2 then quantifies how these choices change with graph-support and score perturbations. This makes reference-route selection explicit and analyzable. Different forest algorithms can also supply reference edges to the same encoder, as demonstrated by the matched comparisons in our W1 response.
  3. Representation learning. The selected forest supplies sparse historical routes shared across snapshots, allowing node states to be exchanged even when particular real interactions are absent. On cyclic graphs, different forests preserve different routes for message passing; their predictive usefulness is assessed experimentally.

The No Forest ablation shows gains of +1.88/+2.33 MRR on Token and +0.17/+0.18 on QA (within/transfer). Matched forest comparisons further show that selecting different routes at the same edge budget changes predictive performance. These results connect the general construction to its practical role in representation learning.

Q2: Contribution of neural representations when η=0η = 0.

Setting η \= 0 in Equation (5) removes the recency-weighted neural term, leaving only the TPPR neighborhood inner product. This corresponds exactly to the TPPR-only rows of original Tables 15–16. The original MoJEPA rows, including random initialization, use the full fused score.

To quantify the independent contribution of representation learning, we cross random versus JEPA-pretrained initialization with neural-only versus fused scoring:

  • Random: No JEPA pretraining.
  • Pretrained: 6,000 JEPA pretraining iterations.
  • Neural-only: Neural score without TPPR or recency weighting.
  • Fused: Neural score combined with TPPR and recency.

All settings share the architecture, Morse forest, training queries and fine-tuning budget. Each initialization uses the same neural-selected checkpoint and candidates for both scores; fusion is calibrated on source validation.

Table 1. Pretraining and score fusion, MRR ×100 (mean ± SD, three seeds).

ScorePToken WToken TQA WQA T
TPPR-onlyn/a33.41±0.0034.20±0.0046.21±0.0046.43±0.00
Neural-only066.25±0.2258.83±0.5039.39±0.1238.49±0.14
Neural-only600077.38±0.3275.03±0.2150.91±0.2049.97±0.24
Fused067.11±0.3160.08±0.3847.22±0.0543.32±0.13
Fused600078.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07

Finding. Pretraining improves neural-only MRR by 11.13/16.20 points on Token and 11.52/11.48 on QA (within/transfer). These gains demonstrate a representation-learning benefit without score fusion. Fusion provides additional gains, especially on QA, where pretrained MoJEPA improves by 6.48/6.61 points. The experiment thus separates two complementary sources of predictive performance.

Q3: Inductive link prediction involving new nodes.

We test links involving naturally emerging nodes on Bytom. A node is new if it never appeared in histories used for pretraining, supervised training, or fitting training statistics. Models remain frozen; predictions use only prior observations and the original candidate sets and ranking protocol.

The test contains 7,150 positive links involving at least one new node across 59 test dates. We group positive links by endpoint status and report neural-only MRR over three seeds.

Table 2. Bytom new-node prediction.

MethodOld → NewNew → OldNew → New
EAGLE46.24±0.1895.36±0.1377.98±0.20
WinGNN27.54±0.0262.45±0.5658.58±0.35
EGCH32.70±0.6291.11±0.1766.30±0.62
ROLAND40.94±0.5594.28±0.6371.65±3.58
SiST-GNN32.17±2.8992.69±0.6559.28±6.57
SG-JEPA48.55±3.9995.26±0.0879.94±0.28
MoJEPA62.49±0.5596.35±0.2288.04±0.50

Finding. MoJEPA achieves the highest MRR in all three groups, improving over the strongest baseline by 13.94 points on Old → New, 0.99 on New → Old, and 8.10 on New → New. Across all new-node links, it reaches 84.21 MRR, 7.22 points above the strongest baseline. Interaction-derived features, rather than learned node-ID embeddings, allow MoJEPA to represent newly observed nodes.

We will incorporate these results and theoretical clarifications into the revised manuscript.

Thank you to the authors for their rebuttal. My initial concern mirrors those shared with the other reviewers as it appears that the largest gains comes from the wider JEPA framework. While there are additional benefits using hitting time reference forests, these seem less significant, which you mention as part of Table 1.

However, I feel that the JEPA framework alone is a strong selling point for this paper, with the investigation into alternative reference forests as the secondary part. There is still research to be done, for example, other sparse graphs with cycles beyond just spanning forests. Along with other additions for the revised manuscript, I believe this reframed paper is worth accepting and I will increase my rating accordingly.

AuthorsOpenReview ↗

Thank you very much, Reviewer HDjG!

Dear Reviewer HDjG,

Thank you very much for your feedback, appreciating the novelty of our approach and, of course, for raising the score! We do agree that JEPA is a bigger machinery and topological tools can enhance this machinery further, our paper is just one step toward this direction. We'll make sure we address it well in the discussion of future directions.

The Authors.

Reviewer_BVyrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MoJEPA, a self-supervised pretraining framework for dynamic graphs that combines partial interaction masking with a shared Morse reference forest. By leveraging partially masked interactions to utilize remaining visible associations and incorporating the "Morse Reference Context," it enables interactions observed at other timestamps to support information exchange at the target timestamp. Experimental results demonstrate that MoJEPA achieves the best performance on all 9 within-dataset tasks across 3 domains, and outperforms competitors in 15 out of 18 transfer tasks.

Critical strengths
  1. The paper combines joint-embedding predictive learning for dynamic graphs with a sparse reference forest constructed using discrete Morse theory, to capture temporal dependencies while providing additional structural communication paths across snapshots. The method is technically innovative because it connects a self-supervised temporal learning objective with a principled graph-structural construction, rather than relying solely on a conventional temporal encoder. This will be a potentially useful methodological contribution.

  2. The authors evaluates MoJEPA on nine datasets spanning three domains, covering both within-dataset link prediction and 18 within-domain cross-dataset transfer directions. MoJEPA achieves the best results on all nine within-dataset tasks and 15 of the 18 transfer directions, with reported relative gains over the strongest external baseline of up to 13.13% and 12.27%, respectively. The evaluation also reports results across three random seeds and includes statistical analyses, strengthening the empirical evidence for the method's effectiveness across diverse datasets and evaluation settings. Although the transfer evaluation is limited to within-domain settings, the breadth and consistency of the reported results constitute a strength of the paper.

Critical weaknesses
  1. Although MoJEPA outperforms baseline models, the ablation studies lack a direct comparison between the proposed Morse-based reference forest and alternative sparse reference structures. Consequently, it remains unclear whether the performance gains stem specifically from the discrete Morse construction or can be achieved through simpler alternatives.
  2. The paper reports strong predictive results and experiments with different training budgets, but these do not establish whether its improvements justify its computational cost relative to the baselines. In particular, the paper does not sufficiently quantify the runtime and resource overhead associated with constructing and using the reference forest, nor does it provide a clear performance comparison under matched computational budgets. This is consequential because the practical value of the proposed approach depends not only on its predictive gains but also on the resources required to achieve them.
Questions for authors
  1. The relationship between the theoretical construction and empirical gains remains unclear. The paper gives a formal construction and stability analysis, but it is not clear how the theoretical properties translate into better representations or predictions. The authors should clarify which properties of the hitting-time forest are expected to be useful for message passing, and whether the empirical gains correlate with these properties.

2.Results are reported over three seeds, and the paper uses one-tailed Holm-adjusted tests. More seeds or uncertainty estimates across additional independent evaluation units would strengthen the claims, particularly when differences between scalar fields are small.

7 comments

W1: Distinguishing Morse reference forests from simpler sparse structures.

Thank you for highlighting the need for direct controls. The question is whether MoJEPA benefits simply from adding sparse historical edges, or whether which historical routes are selected also matters. We compare four forests under a matched reference-edge budget.

1. Matched reference-forest comparison

  • Hitting time: Our weighted random-walk-guided construction.
  • Same-hub BFS: Shortest-path trees rooted at the same hubs.
  • Random: Random spanning forests.
  • Maximum-weight: Spanning forests favoring stronger historical edges.

All variants use the same aggregated historical graph, one tree per observed connected component, identical edge counts and relation-type slots, and the same training and evaluation protocol. Results are MRR ×100, mean ± sample SD over three seeds; W and T denote within-dataset and transfer.

Table 1. Reference forests with matched edge counts.

ForestToken WToken TQA WQA T
Hitting time78.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07
Same-hub BFS77.80 ± 0.3375.89 ± 0.2857.37 ± 0.0756.54 ± 0.11
Random77.52 ± 0.2875.19 ± 0.2057.20 ± 0.0656.50 ± 0.05
Maximum-weight77.49 ± 0.3775.84 ± 0.5057.25 ± 0.0456.47 ± 0.11

Finding. Hitting time achieves the highest mean in all four settings, with more pronounced differences on Token than QA. Because coverage, edge count, and relation encoding are matched, this comparison isolates the choice of reference routes rather than the mere presence of extra historical edges. The competitive alternatives also indicate that the benefit is not exclusive to hitting time.

2. What the Morse formulation contributes

The purpose of the Morse construction is to make historical route selection explicit, configurable, and mathematically analyzable. A node scalar field specifies how routes are selected; the observed graph determines the resulting forest. Hitting time is a motivated choice because it reflects weighted random-walk connectivity and produces a component-spanning forest with the minimum number of edges. Importantly, using hub distance instead recovers a BFS-type shortest-path forest, so BFS is a specialization of the same construction rather than an unrelated alternative.

Thus, our contribution is not a claim that hitting time must outperform every spanning forest on every dataset. It is a principled framework for constructing and analyzing candidate historical reference structures, whose predictive value can be tested under the same encoder. We will add the matched controls and sharpen this distinction in the revision.

W2: Computational cost of constructing and using reference forests.

Thank you for requesting a cost-aware assessment. We quantify the reference forest's construction overhead, the training cost across architectures, and predictive performance under a common wall-clock budget. This distinguishes the cost of obtaining historical reference routes from the cost of learning with them.

1. Model size and training cost

We report parameter counts, total optimizer updates across all training stages, and measured training time on Bytom, averaged over three seeds. Epochs and optimizer updates are not treated as equivalent computational units.

Table 1. Bytom model and training costs.

MethodParametersUpdatesTime (s)
EGCH41.97M16,6034,655.26
SiST-GNN12.05M17,500577.19
EAGLE122K7,910498.24
TNCN224K8,8003,414.06
SG-JEPA85K8,000315.25
CPDG141K8,000990.08
MoJEPA87K8,0001,555.98
MoJEPA, no forest87K8,0001,407.75

Finding. MoJEPA uses only 87K parameters, comparable to SG-JEPA and far fewer than EGCH or SiST-GNN. Its training time is lower than EGCH and TNCN. Adding the reference forest requires no additional trainable parameters and increases measured training time by approximately 10.5% relative to No Forest. On Token, this accompanies +1.88/+2.33 MRR within-dataset/transfer. Since SG-JEPA trains faster, a matched-time comparison is needed to evaluate whether MoJEPA's gain is worth its additional runtime.

2. Reference forest construction time

We separately time one hitting-time forest construction using one CPU thread, sampling 30 seven-day history windows per dataset and repeating each measurement five times.

Table 2. Mean construction time per forest (ms).

BytomDecentralandGolemMathOverflowAskUbuntuSuperUser
389.81339.74451.0134.53120.14184.07

Finding. A single forest takes approximately 35–451 ms to construct across these datasets. Forests are built once per historical context window, not at every optimizer update. These measurements isolate CPU preprocessing from training overhead rather than treating forest construction as cost-free.

3. Performance at matched training time

We compare MoJEPA and SG-JEPA on the same RTX 4090 with a 1,373.55-second cap, including training and validation, but excluding preprocessing, final calibration, and test inference. Both use seed 21.

Table 3. Matched-time Bytom performance (MRR ×100).

ModelActual time (s)PretrainFine-tuneNeural-onlyFused
SG-JEPA1,334.5816,30014,10077.6978.32
MoJEPA1,373.556,0002,00083.9684.39

Finding. Within nearly equal training time, SG-JEPA completes 30,400 updates versus MoJEPA's 8,000, yet MoJEPA achieves +6.27 neural-only and +6.07 fused MRR. The gain therefore persists under a matched training-time budget and does not depend on giving MoJEPA an unmatched scoring rule. This comparison is specifically about training and validation time, not a claim of matched end-to-end cost including preprocessing.

We will add these runtime, construction-cost, and matched-budget results to the revised manuscript.

Q1: Connecting the theoretical construction to representation learning.

Morse construction turns observed historical relations into sparse routes shared across snapshots. For JEPA, masking changes both the prediction target and the paths through which the student obtains evidence. Four properties explain how the forest supports learning.

  1. Component coverage with minimal edges. Hitting-time descent gives one spanning tree per observed component with exactly |V| − β₀(U) edges (Proposition 4.1). It retains historical connectivity without adding the dense aggregate graph. The encoder exchanges messages along these routes within its finite receptive field.
  2. Weight-sensitive route selection. Hitting time reflects interaction weights and broader connectivity through expected random-walk travel to a hub. Each non-hub node has a lower-valued neighbor, ensuring descent to the hub. Unlike BFS, hitting time can choose non-shortest-hop routes on cyclic graphs. Thus, the field determines which historical routes carry messages.
  3. Shared historical context across time. Constructed only from observed context snapshots, the forest is attached to every snapshot as bidirectional reference edges with a separate relation type. A relation observed at one time can thus carry node states at another time, even if the real interaction is absent or masked there. Student and teacher share these routes; the student predicts complete-view embeddings from retained interactions and historical context.
  4. Conditional structural stability. With fixed tie-breaking, construction is deterministic. Theorem 4.2 bounds changes in parent pointers and critical edges using support edits and local score margins, characterizing when reference routes persist under perturbations.

Empirical connection. Appendix H.1 reports median retained-edge fractions of 88–94% on Token, 66–73% on QA, and 44–62% on Reddit. Relative to No Forest, MRR changes by +1.88/+2.33 on Token, +0.17/+0.18 on QA, and −0.13/−0.18 on Reddit (within/transfer). These differences have a structural interpretation: on nearly tree-like contexts, the forest retains most available relations, whereas on cyclic contexts it must choose among many alternative routes. Because the encoder has only two spatial message-passing layers, the selected short routes can matter even when every forest spans its component. The largest gains coincide with higher edge retention on Token, but retention alone does not establish causation. The W1 matched-forest experiments isolate route selection at fixed coverage and edge budget.

The theory explains how historical evidence enters message passing; the experiments evaluate its predictive value.

Q2: Statistical reliability across seeds and test periods.

We assess training randomness and temporal dependence.

1. Ten-seed tests. We reran nine Token settings with ten seeds. Two-sided Welch tests against the second-best baseline use Holm correction.

Table 1. Token MRR ×100, ten-seed mean ± sample SD.

MethodBYDEGOBY→DEBY→GODE→BYDE→GOGO→BYGO→DE
HawGCN66.75±0.1950.54±0.4357.12±0.2251.40±0.0054.04±0.0062.00±0.0054.01±0.0062.01±0.0351.39±0.05
EAGLE75.95±0.1565.09±0.2264.41±0.3762.88±0.1462.41±0.2374.67±0.2061.03±0.5973.26±0.7261.56±0.75
SG-JEPA77.77±0.7665.60±1.3164.02±0.8464.26±1.0461.17±0.7572.92±0.8656.79±2.0274.00±0.9662.44±1.62
MoJEPA84.29±0.19***75.17±0.52***74.81±0.62***74.13±0.61***74.37±0.55***81.52±0.42***73.07±0.63***80.73±0.59***71.63±0.69***

BY, DE, GO denote Bytom, Decentraland, Golem; first three columns are within-dataset and six are transfer. *** denotes Holm-adjusted p<0.01.

Finding. MoJEPA leads in all nine settings, with every improvement over the second-best baseline significant at Holm-adjusted p<0.01.

2. Temporal block bootstrap. We resample contiguous test-date blocks with indices synchronized across methods and seeds, recomputing paired MRR differences. Across nine settings versus SG-JEPA, EAGLE and HawGCN, all 27 two-sided 95% CIs have positive lower bounds (5.33 to 22.09). All pass Holm correction, even across the full 162-test family. Halving or doubling block length preserves these results.

Finding. Ten-seed tests support consistency across runs; block bootstrap supports the gains under temporal dependence. Full CIs and resampling details will appear in the appendix.

We will incorporate these theoretical clarifications and expanded statistical analyses into the revised manuscript as soon as possible.

Reviewer_kv3uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents MoJEPA, a joint-embedding predictive framework for dynamic graphs that combines partial interaction masking with a shared Morse reference forest. While the method is reasonably motivated and achieves competitive results across multiple datasets, the evidence does not sufficiently establish the necessity or independent contribution of its core mechanisms. Partial masking does not guarantee that nodes retain local relational evidence, leaving the proposed explanation for its benefits insufficiently validated. The absence of controls such as random spanning trees also makes it difficult to attribute improvements specifically to the Morse construction, particularly given its mixed effects across domains. Overall, the experiments demonstrate the effectiveness of the complete system more convincingly than they explain the sources of its gains.

Critical strengths

1.Coherent, problem-driven design. The combination of partial interaction masking and a historical reference forest provides a well-motivated approach to addressing missing relational evidence in dynamic graph prediction.

2.Broad evaluation and competitive performance. The method is evaluated on nine datasets and eighteen within-domain transfer directions, achieving the best results on all nine within-dataset tasks and fifteen transfer directions.

Critical weaknesses

1.The independent contributions of the core mechanisms remain insufficiently established. Missing controls, such as random spanning trees, and incomplete score-fusion ablations make it difficult to attribute the gains to the Morse construction, JEPA pretraining, or historical scoring.

2.A gap remains between the masking motivation and the supporting evidence. Partial masking can still remove all incident edges of a node, and the experiments do not adequately establish that the gains arise from preserving local relational evidence.

Questions for authors

1.Clarification of the masking mechanism. Equation (1) removes the union of records sampled independently by their incident nodes. Consequently, a node may lose all its incident interactions, and degree-one nodes necessarily do so. Appendix E.4 further reports realized removal rates of approximately 65%–74%. How should this behavior be reconciled with the motivation that partial masking preserves local relational evidence? What fraction of eligible target nodes retain at least one interaction, and does the benefit persist under a comparison with uniform edge masking at the same realized removal rate?

2.Evidence for the specific value of the Morse construction . Table 2 compares scalar fields and a no-forest control, but does not isolate the proposed construction from simpler historical spanning forests. Moreover, hitting-time forests improve Token performance but slightly underperform the no-forest variant on Reddit. Have you compared against random spanning trees or maximum-weight spanning trees constructed from the same context, with identical edge budgets and training settings? Such evidence would materially affect whether I view the contribution as a specific advance in reference construction or primarily as historical graph augmentation.

3.Clarification and isolation of score fusion. Equation (5) combines TPPR similarity, endpoint recency, and neural predictions. However, Tables 15–16 do not explicitly identify neural-only or recency-free variants, while Appendix E.4 describes G.1 as demonstrating improvements in “neural-only prediction.” Which scoring function is used in each reported row? Can you provide the neural-only results before and after pretraining, together with the effect of adding recency and TPPR? This distinction is necessary to assess how much of the reported advantage comes from the proposed representation-learning mechanisms.

9 comments

W1: Isolating the contributions of the core mechanisms.

Q2: The specific value of Morse reference construction.

Thank you for suggesting these controls. The central question is whether the reference forest provides useful historical structure merely by adding sparse edges, or whether the choice of reference routes also matters. We address this through matched forest comparisons and clarify the scope of our structural contribution.

1. Comparing reference forests under matched edge budgets

We compare four constructions using the same aggregated historical context:

  • Hitting time: The proposed weighted random-walk-based forest.
  • BFS: A shortest-path forest rooted at the same hubs.
  • Random: A random spanning forest.
  • Maximum-weight: A spanning forest favoring stronger historical interactions.

All constructions produce one tree per observed connected component, with identical reference-edge counts, relation-type slots, and training and evaluation protocols.

Table 1. Matched reference-forest comparison, MRR ×100 (mean ± SD, three seeds).

ForestToken WToken TQA WQA T
Hitting time78.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07
Same-hub BFS77.80 ± 0.3375.89 ± 0.2857.37 ± 0.0756.54 ± 0.11
Random77.52 ± 0.2875.19 ± 0.2057.20 ± 0.0656.50 ± 0.05
Maximum-weight77.49 ± 0.3775.84 ± 0.5057.25 ± 0.0456.47 ± 0.11

W denotes within-dataset prediction and T denotes transfer.

Finding. Hitting time achieves the highest mean MRR in all four settings, with clearer differences on Token than QA. Because the forests share the same historical context, component coverage, and edge budget, these differences arise from which reference routes are selected, rather than simply how many historical edges are added. The competitive performance of alternative forests also confirms that useful historical augmentation is not exclusive to hitting time.

2. Why the Morse construction matters

Our contribution is not restricted to one particular spanning forest. The Morse formulation provides a general way to turn a node scalar field into a sparse, analyzable reference structure. Different fields express different criteria for selecting historical communication routes, while the same learning architecture can evaluate their predictive usefulness.

Importantly, BFS itself is a special case of this construction. Using graph distance to a designated hub as the scalar field produces a shortest-path spanning tree, with the exact BFS parent choices recovered through consistent tie-breaking. Hitting time extends this construction by incorporating weighted random-walk connectivity, allowing reference routes to depart from shortest paths.

This distinction matters because the historical graph determines which routes are available. On an observed component that is already a tree, component-spanning forests necessarily select the same undirected edges. On cyclic components, alternative scalar fields can preserve different communication routes, potentially affecting the information exchanged by the encoder.

3. Interpreting the domain-dependent gains

The existing No Forest ablation shows that the incremental benefit of reference context is largest on Token, much smaller on QA, and slightly negative on Reddit. We therefore distinguish two contributions:

  • Historical reference context: Adding selected historical relations can improve prediction, but its benefit depends on the dataset.
  • Structural selection: The scalar-field-guided construction determines which relations form the reference forest, with hitting time performing best in the four matched settings evaluated here.

The results support Morse construction as an explicit, configurable framework for selecting and analyzing historical reference context, rather than claiming that hitting time must be optimal for every graph.

We will incorporate these matched controls and clarify the corresponding theoretical contribution and domain-dependent findings in the revised manuscript.

W2: Evidence supporting the partial-masking mechanism.

Q1: Neighbor retention and comparisons at matched removal rates.

Thank you for identifying this distinction. We clarify that partial masking is designed to preserve some relational evidence within the target block, not to guarantee that every target node retains an incident interaction. We further quantify actual neighbor retention and compare against uniform masking at the same realized removal rate.

1. What partial masking preserves

Under Equation (1), each active node samples a fraction of its incident records, and the union of these selections is removed. Consequently, a degree-one node necessarily loses its only interaction, while nodes with higher degree can also lose all incident interactions when their neighbors select the corresponding records.

The masking parameter therefore controls the per-node sampling rule, not the realized global edge-removal fraction or guaranteed per-node retention.

Crucially, MoJEPA predicts target representations using a temporal context. Eligible target nodes belong to both the target and context node sets, so they remain observable at earlier, unmasked context positions. The reference forest provides additional historical communication routes, although it does not reveal or reconstruct the withheld target-time interactions.

Thus, target-block neighbor retention and historical contextual evidence are complementary, rather than interchangeable.

2. How often do eligible nodes retain interactions?

We measure the fraction of eligible target-node occurrences retaining at least one target-block interaction after masking on Bytom.

  • All eligible nodes: 12.55% under MoJEPA masking versus 13.61% under uniform masking.
  • Two neighbors: 58.04% versus 21.83%.
  • Three to five neighbors: 65.54% versus 32.73%.

Finding. Overall retention is low because the data contain many sparse nodes, especially degree-one nodes. However, among nodes with multiple original neighbors, the proposed masking rule preserves local relational evidence substantially more often than uniform masking. This is the setting in which partial masking can exploit the availability of alternative local interactions.

3. Predictive performance at matched removal rates

To determine whether the gains arise from the sampling structure rather than a different amount of removed information, we compare MoJEPA masking against uniform edge masking at the same realized removal rate. The architecture, Morse forest, training budget, and evaluation settings remain fixed.

Table 1. Matched-removal masking comparison, MRR ×100 (mean ± SD, three seeds).

MaskingToken WToken TQA WQA T
MoJEPA78.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07
Uniform76.26 ± 0.1174.94 ± 0.1656.37 ± 0.0355.51 ± 0.13

Finding. MoJEPA masking improves MRR by 1.88/1.16 points on Token and 1.02/1.07 on QA (within/transfer). Since both methods remove the same overall fraction of records, the improvement cannot be explained simply by retaining more edges globally. The way removed interactions are distributed across local neighborhoods also matters.

4. Representation prediction error by neighborhood size

We further examine nodes retaining at least one neighbor after masking. Errors are averaged across three datasets per domain and three seeds; lower values indicate more accurate representation prediction.

Table 2. Representation prediction error by original neighborhood size (mean ± SD).

DomainMasking123–5≥6
TokenMoJEPA0.1814 ± 0.01900.1815 ± 0.01600.1927 ± 0.01660.1554 ± 0.0027
TokenUniform0.1778 ± 0.00570.2215 ± 0.03000.2381 ± 0.02320.1768 ± 0.0097
QAMoJEPA0.2221 ± 0.01520.1804 ± 0.01490.1598 ± 0.01480.1370 ± 0.0137
QAUniform0.2424 ± 0.01030.2130 ± 0.01320.2038 ± 0.01080.1634 ± 0.0048

Finding. MoJEPA yields lower representation prediction error for every group with at least two original neighbors, across both domains. This aligns with the retention analysis: the masking rule is particularly beneficial when a node has multiple local interactions that can provide complementary evidence. The degree-one case remains inherently constrained because no alternative target-time neighbor can be retained.

We will clarify the scope of the partial-masking motivation and add these retention, matched-removal, and representation-error analyses to the revised manuscript.

Q3: Clarification and isolation of score fusion.

Thank you for highlighting the ambiguity between neural representation learning and the final scoring function. We clarify the scoring semantics of the original ablations and conduct a matched experiment to separate the contribution of JEPA pretraining from historical score fusion.

1. Clarifying the original scoring protocol

The MoJEPA rows in the original Tables 15–16, including both randomly initialized and pretrained variants, use the full fused score in Equation (5). They combine neural predictions with TPPR neighborhood similarity and endpoint recency.

The TPPR-only rows instead use historical neighborhood similarity without neural predictions or recency weighting, corresponding to Equation (5) with the neural contribution removed.

Accordingly, the description in Appendix E.4 referring to improvements in “neural-only prediction” requires correction: the original results demonstrate improvements under the fused scoring pipeline, rather than isolating neural predictions.

2. Separating pretraining and score fusion

To establish how much of the improvement comes from representation learning itself, we conduct a matched 2×2 experiment crossing initialization and inference scoring:

  • Random: No JEPA pretraining.
  • Pretrained: 6,000 JEPA pretraining updates.
  • Neural-only: Neural predictions without TPPR or recency weighting.
  • Fused: Neural predictions combined with TPPR and endpoint recency.

All variants use the same downstream architecture, Morse reference forest, training queries, and fine-tuning budget. For each initialization, neural-only and fused scores are evaluated on the same neural-selected checkpoint and identical candidate sets. The fusion coefficient is calibrated on source validation data.

Table 1. Pretraining and score-fusion decomposition, MRR ×100 (mean ± SD, three seeds).

ScoringPretrainToken WToken TQA WQA T
TPPR-onlyN/A33.41 ± 0.0034.20 ± 0.0046.21 ± 0.0046.43 ± 0.00
Neural-only066.25 ± 0.2258.83 ± 0.5039.39 ± 0.1238.49 ± 0.14
Neural-only600077.38 ± 0.3275.03 ± 0.2150.91 ± 0.2049.97 ± 0.24
Fused067.11 ± 0.3160.08 ± 0.3847.22 ± 0.0543.32 ± 0.13
Fused600078.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07

Finding.

  • JEPA pretraining: Under neural-only scoring, pretraining improves MRR by 11.13/16.20 points on Token and 11.52/11.48 on QA (within/transfer). These improvements are obtained without TPPR or recency fusion, demonstrating a substantial independent contribution from learned representations.
  • Historical score fusion: For pretrained MoJEPA, fusion provides an additional 0.76/1.08 points on Token and 6.48/6.61 on QA. Its larger contribution on QA indicates that the value of historical scoring differs across domains.
  • Complementary mechanisms: TPPR-only scoring is substantially weaker than the complete model, while neural-only pretraining already produces strong gains. The final performance reflects complementary contributions from learned representations and historical statistics.

These results separate the contribution of JEPA pretraining from the combined TPPR and recency scoring pipeline. They do not individually isolate TPPR from endpoint recency, which would require an additional scoring ablation.

We will correct the scoring descriptions in Tables 15–16 and Appendix E.4 and incorporate the matched decomposition into the revised manuscript.

Reviewer_yoAAOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how useful node representations can be learned from observed interactions in dynamic graphs. Masking interactions removes useful relational context, while relations observed at earlier times are no longer visible at the current timestamp. MoJEPA selects a contiguous target block inside a pretraining window, randomly masks part of the interactions inside that block, and compresses relations from the observed snapshots outside the block into a small shared reference skeleton reused across the window. The student view predicts the teacher's complete-view node representations at the target positions. Experiments on several daily dynamic graph datasets report competitive results on within-dataset prediction and on most transfer directions, together with an analysis of the structural stability of the reference skeleton.

Critical strengths
  1. The distinction between the target block and the observed snapshots on either side is stated clearly. The pretraining skeleton is built only from snapshots outside the target block and is injected as a separate relation type alongside real interactions; under cross-dataset transfer the source encoder, link head, and calibration are frozen, while the target dataset rebuilds the skeleton from its own history. The reported mean scores are highest on all nine within-dataset tasks and on fifteen of eighteen transfer directions.
  2. Partial interaction masking combined with a shared reference skeleton addresses a concrete weakness of dynamic-graph self-supervised pretraining. The student and teacher share the same reference pathways, but each view keeps its own retained real interactions and the node features derived from them. The appendices also report realized masking rates, skeleton construction cost, solver choices, and several negative controls.
Critical weaknesses
  1. The source of the reference skeleton's gain is not cleanly isolated. Relative to a no-skeleton variant, the skeleton adds about 1.9 to 2.3 MRR points on Token, about 0.2 on QA, and slightly reduces performance on Reddit; the no-skeleton variant already approaches or exceeds the strongest external baseline in all three domains. Table 2 includes a hub-distance field, an approximately BFS-like tree to a hub, but its gap to the hitting-time field is only about 0.3 MRR points on Token and under 0.1 on QA; matched edge-budget controls with a random or maximum-weight skeleton are not reported, so the current ablations do not justify attributing the overall advantage to this particular Morse structure.

  2. Comparable scoring and compute budgets are not established by the reported setup. The final score combines a TPPR neighborhood inner product with a sigmoid-transformed neural score scaled by endpoint recency and a calibration coefficient. EAGLE itself already uses TPPR, but the paper does not say whether other baselines could receive a comparable fusion, and no matched control is given. The method is reported in pretraining iterations plus fine-tuning updates, whereas non-pretraining baselines are reported in training epochs, and the two units cannot be converted directy into a comparable computational cost. SG-JEPA, another pretraining method, uses the same 6000 pretraining steps plus at most 2000 fine-tuning updates, but identical update counts do not imply identical compute; whether the budget relative to non-pretraining baselines is matched cannot be determined from the current description.

  3. The statistical support for small gains is limited. Only three seeds are run, a one-tailed Holm correction is used, and the reported thresholds include p below 0.10; some positive transfer gains have no significance marker. Under this protocol, the count of highest means does not correspond tightly to statistical significance.

Questions for authors
  1. Could a random skeleton, a BFS tree rooted at the same hub, and a maximum-weight skeleton be compared against the hitting-time skeleton in a single table with matched reference-edge counts and the same relation-type slot? If hitting time does not consistently win, should Contribution 2 be narrowed accordingly?
  2. In the two appendix ablations of pretraining and score fusion, is each row reported with the pure neural score or with the fused historical neighborhood and recency score? Would a matched two-by-two decomposition over random versus pretrained initialization and neural-only versus fused scoring, with fixed queries and checkpoints, separate the pretraining contribution from the scoring pipeline?
  3. Could comparable training-budget information be reported, such as total update counts, processed sample counts, or wall-clock time, together with matched control experiments at a common budget? Under the current units, is it possible to tell how much of the lead comes from differences in training amount?
  4. Because the positive pairs used for ranking are allowed to have appeared in earlier history, would the authors report MRR on the subset of pairs that have never appeared before, to distinguish prediction of genuinely new links from prediction of recurring links?
  5. The appendix specifies the latest 15 incident records only for endpoint recency. Could the authors clarify the history span used for TPPR and whether comparable history inputs and fusion are applied across baselines?
6 comments

W1: Reference forest contribution.

Q1: Matched forest controls and the scope of Contribution 2.

Thank you for suggesting these controls. We examine route selection, connectivity, and hitting-time guidance in three experiments. All results are mean MRR ×100 over three seeds; W denotes within-dataset prediction and T denotes transfer.

1. Reference forest selection under matched edge budgets

We compare hitting-time, same-hub BFS, random spanning, and maximum-weight spanning forests. All share the historical graph, component coverage, edge count, relation-type slot, and training/evaluation protocol.

Table 1. Matched reference-forest comparison.

ForestToken WToken TQA WQA T
Hitting time78.1476.1157.3956.58
Same-hub BFS77.8075.8957.3756.54
Random spanning77.5275.1957.2056.50
Maximum-weight77.4975.8457.2556.47

Finding. Hitting time achieves the highest mean MRR in all four settings, with clearer differences on Token than QA. At fixed component coverage and edge count, the difference lies in which historical routes are selected. Competitive alternatives show that reference context helps beyond any single rule, while route choice can add value.

2. Forest connectivity versus edge addition

A Morse forest spans each observed component with minimal reference edges. To test the value of connectivity beyond edge count, we fine-tune the same Morse-pretrained encoder with:

  • Intact: Original forest.
  • Cut: Randomly remove 25% of forest edges.
  • Within: Add historical edges inside the remaining components.
  • Across: Restore deleted edges between components.

Within and Across add equal numbers of edges under identical fine-tuning settings.

Table 2. Connectivity and matched edge restoration.

ForestToken WToken TQA WQA T
Intact78.1476.1157.3956.58
Cut77.1875.0957.3056.44
Within77.2775.2057.2856.45
Across77.4975.3257.3256.49

Finding. Removing forest edges reduces MRR, while reconnecting separated components recovers more performance than adding equally many edges within them. This links theoretical component coverage to predictive benefit: reference edges provide routes between otherwise separated parts of the reference structure. The intact forest remains best, indicating that connectivity alone does not account for all of its value.

3. How hitting time extends BFS-type construction

To understand what hitting time adds, note that BFS is a special case of our scalar-field-guided Morse construction. Using distance to the hub as the scalar field makes each non-hub node choose a parent one hop closer, yielding a shortest-path tree. Matching the tie-breaking to a BFS traversal recovers its exact parent choices.

BFS selects routes by hop distance; hitting time uses weighted random-walk connectivity and can depart from shortest paths. We compare:

  • BFS: Parents follow shortest-path levels.
  • BFS-HT: Hitting time selects among neighbors one hop closer to the hub.
  • Full HT: Hitting-time descent without the shortest-path constraint.

All use the same Morse-pretrained encoder, hubs, edge counts, relation encoding, and fine-tuning protocol.

Table 3. Hitting-time guidance beyond BFS.

ForestToken WToken TQA WQA T
BFS77.8975.9857.3856.52
BFS-HT78.0976.0057.3756.53
Full HT78.1476.1157.3956.58

Finding.

  • Separating the mechanisms: BFS-HT tests hitting-time-guided parent choice while retaining shortest-path levels; Full HT additionally allows departures from those levels. Full HT has the highest mean in all four settings, while BFS-HT improves Token W over BFS.
  • What changes structurally: Both BFS and hitting time produce one root per observed component, and their forest edges coincide when that component is already a tree. On cyclic components, hitting time can favor weighted historical connections that hop distance misses. Thus, the comparison concerns route selection, not root counts.

Overall Interpretation

  • Adding context: No Forest ablations show gains largest on Token, small on QA, and slightly negative on Reddit.
  • Selecting its structure: Matched-budget and connectivity controls show that both the selected routes and their ability to connect components matter beyond edge count.
  • Contribution 2: MoJEPA makes historical route selection explicit, configurable, and mathematically analyzable through scalar fields. BFS is one specialization; hitting time uses weighted connectivity to choose alternative routes. Its usefulness depends on the observed graph and task, rather than universal optimality.

We will add these results and clarify the theory in the revised manuscript.

W2: Comparable scoring mechanisms.
Q2: Separating pretraining from score fusion.

Thank you for raising this issue. The key question is whether MoJEPA's predictive gains come from representations learned through JEPA or from the TPPR and recency terms added at inference. We address it first within MoJEPA, then against SG-JEPA using identical scoring rules. All results are MRR ×100, averaged over three seeds; W denotes within-dataset and T denotes transfer.

1. Separating JEPA pretraining and score fusion

We first clarify the original appendix: both randomly initialized and pretrained MoJEPA rows use the full fused score. TPPR-only uses historical neighborhood similarity alone, without neural prediction or recency weighting. Therefore, the original ablations do not by themselves isolate the gain from pretraining under neural-only scoring.

We now cross two initialization choices with two inference scoring rules:

  • Random: No JEPA pretraining (P=0).
  • Pretrained: 6,000 JEPA pretraining updates.
  • Neural-only: Use neural predictions without TPPR or recency weighting.
  • Fused: Combine neural predictions with TPPR similarity and endpoint recency.

All variants share the downstream architecture, Morse forest, training queries and fine-tuning budget. Within each initialization, neural-only and fused scoring use the same neural-selected checkpoint, queries and candidates; the fusion coefficient is calibrated on source validation data.

Table 1. Matched 2×2 ablation, mean MRR.

ScoringPToken WToken TQA WQA T
TPPR-onlyn/a33.41 ± 0.0034.20 ± 0.0046.21 ± 0.0046.43 ± 0.00
Neural-only066.25 ± 0.2258.83 ± 0.5039.39 ± 0.1238.49 ± 0.14
Neural-only600077.38 ± 0.3275.03 ± 0.2150.91 ± 0.2049.97 ± 0.24
Fused067.11 ± 0.3160.08 ± 0.3847.22 ± 0.0543.32 ± 0.13
Fused600078.14 ± 0.4776.11 ± 0.2957.39 ± 0.0856.58 ± 0.07

Finding. Pretraining improves the neural-only MRR by 11.13/16.20 points on Token and 11.52/11.48 on QA (W/T). These gains arise before any historical score is added, demonstrating that JEPA learns more predictive representations. Fusion provides a separate improvement: for pretrained MoJEPA, it adds 0.76/1.08 on Token but 6.48/6.61 on QA. Thus, the relative roles of representation learning and historical scoring differ by domain, and neither component should be credited with the other's gains.

2. Applying the same scoring procedure to SG-JEPA

The internal decomposition establishes what each component contributes within MoJEPA, but it does not answer whether SG-JEPA could also gain from TPPR and recency. To make the external comparison fair, we evaluate both methods with neural-only scoring and with the same fusion procedure, keeping the evaluation queries and candidates fixed.

Table 2. Matched scoring on Bytom, mean MRR.

MethodScoringWithin→DE→GO
SG-JEPANeural-only77.25 ± 1.0365.08 ± 0.2761.50 ± 0.69
SG-JEPAFused78.10 ± 0.8066.10 ± 0.5463.82 ± 0.69
MoJEPANeural-only83.97 ± 0.0473.83 ± 0.1373.63 ± 0.47
MoJEPAFused84.37 ± 0.1374.43 ± 0.1574.46 ± 0.51

Finding. SG-JEPA benefits from fusion too, showing why it is important to give the baseline the same scoring option. Crucially, MoJEPA already leads by 6.72, 8.75 and 12.13 MRR points under neural-only scoring for Bytom within-dataset and transfer to Decentraland and Golem. Its advantage therefore persists without the historical fusion mechanism. When fusion is applied to both, MoJEPA remains ahead in all three settings.

Overall Interpretation

The 2×2 ablation identifies a substantial JEPA pretraining contribution independent of inference-time scoring. The matched SG-JEPA experiment then establishes that MoJEPA's advantage is not an artifact of giving it a scoring mechanism unavailable to the baseline. Computational fairness is examined separately using measured training times and matched wall-clock budgets in our response to Q3. We will clarify the scoring semantics of the original appendix and add both controlled comparisons to the revision.

W2: Comparing computational budgets across architectures.
Q3: Training cost and matched-budget evaluation.

Thank you for requesting a compute-aware comparison. Reporting pretraining iterations, fine-tuning updates and baseline epochs separately does not reveal their relative cost. We therefore evaluate MoJEPA along two complementary dimensions: its actual model and training overhead, and its predictive performance when a competing JEPA method receives the same wall-clock budget.

1. Model size and measured training costs

Parameter count indicates model size, but cannot replace measured runtime. We report both, together with total optimizer updates across all training stages, on Bytom. Training times are averaged over three seeds; the source results do not report their standard deviations.

Table 1. Model size and training costs on Bytom.

MethodParamsUpdatesTime (s)
EGCH41.97M16,6034,655
SiST-GNN12.05M17,500577
EAGLE122K7,910498
TNCN224K8,8003,414
SG-JEPA85K8,000315
CPDG141K8,000990
MoJEPA87K8,0001,556
MoJEPA (no forest)87K8,0001,408

Finding. Three observations clarify the cost of MoJEPA:

  • Model complexity: With 87K trainable parameters, MoJEPA is close in size to SG-JEPA and much smaller than EGCH and SiST-GNN. Its predictive gains do not depend on scaling up the number of learned parameters.
  • Practical training cost: MoJEPA trains faster than EGCH and TNCN, although other baselines, including SG-JEPA, require less time. This is why comparing update counts alone is insufficient: 8,000 updates correspond to substantially different runtimes across architectures.
  • Reference forest overhead: Adding the forest introduces no trainable parameters. Relative to the no-forest variant, measured training time rises from 1,408 to 1,556 seconds, approximately 10.5%. This makes the additional cost of the reference structure explicit rather than treating its contribution as free.

These measurements establish the actual resource trade-off, but a runtime table alone cannot determine whether the resulting predictive improvement justifies the cost. We therefore conduct a matched-time evaluation.

2. Performance under a common wall-clock budget

We compare MoJEPA and SG-JEPA on the same RTX 4090 GPU, using a 1,373.55-second training-time cap, including training and validation. Each method uses the available budget for its own pretraining and fine-tuning schedule. We evaluate both neural-only and fused scores so that the result does not depend on granting MoJEPA an extra inference-time scoring mechanism.

Table 2. Matched-time Bytom prediction, MRR ×100.

MethodTime (s)PretrainFine-tuneNeuralFused
SG-JEPA1,334.5816,30014,10077.6978.32
MoJEPA1,373.556,0002,00083.9684.39

Finding. The comparison directly tests whether MoJEPA's Bytom advantage is simply a consequence of more training. Within nearly identical elapsed time, SG-JEPA completes 30,400 updates, versus MoJEPA's 8,000, yet MoJEPA achieves 6.27 points higher neural-only MRR and 6.07 points higher fused MRR. The improvement therefore persists even when SG-JEPA receives substantially more optimization steps under the same time budget. The neural-only result is especially important because it does not rely on score fusion to establish the advantage.

Overall Interpretation

MoJEPA is parameter-efficient, and the additional training overhead associated with the reference forest is measurable and limited relative to its no-forest variant. More importantly, its Bytom predictive advantage over SG-JEPA persists under matched wall-clock time and matched scoring choices. We will report both the cross-baseline cost measurements and the matched-budget experiment, and distinguish optimizer-update counts from actual computational cost in the revised paper.

Comments on the paper

1 comment

Dear Reviewers,

We have uploaded an updated version of the manuscript incorporating the theoretical clarifications, additional experiments, and revisions discussed in our rebuttal.

On the theoretical side, we clarified the connection between discrete Morse gradients and rooted spanning forests, established BFS as a special case of our scalar-field-guided construction, and further explained how hitting time extends shortest-path-based selection through weighted random-walk connectivity. We also refined the structural stability analysis, including the definitions of Morse skeleton distance and local score margins, and clarified the scope of the theoretical guarantees. These revisions help distinguish the generality of the Morse framework from the specific choice of hitting time.

On the experimental side, we incorporated matched reference-forest comparisons, connectivity and route-selection controls, pretraining and score-fusion ablations, inductive and previously unseen link prediction, computational cost and matched-time evaluations, and expanded statistical reliability analyses.

Most additional experiments are included in the appendix, with the corresponding findings summarized in the main text. Revisions are highlighted in blue for ease of review.

We sincerely thank all reviewers for their constructive comments, which have helped us strengthen both the theoretical foundations and empirical evaluation of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.