- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper describes a JEPA approach for prediction within dynamic networks addressing the issue that masking edges for testing hides additional relational structure beyond the removed edge. Using a teacher with access to additional reference context, a student can pre-learn representations to predictor those from the teacher. The leads to big improvements for prediction within a dataset but also creates an embedding function that allows for transfer between similar dynamic networks.
The paper focusses on a Morse reference context provided by an spanning forest derived from expected random-walk hitting times. It is unclear what the benefits of describing the forest in this language, primarily as the details required to understand the point of Theorem 4.2 is lost due to missing notation. As I understand the work, any spanning forest could be used as a reference context and further work to explore which spanning forests give better results is needed.
As the authors claim, this is a first step in incorporating JEPA into dynamic networks by including network topology. This is a very interesting and potentially fruitful approach that is highly worthy of further developments.
Critical strengths
The JEPA architecture to use context windows within dynamic networks is a good idea and the setup is very well-explained in the paper with helpful diagrams throughout. Overall, the research was easy to follow for someone new to JEPA methods, which made it more noticeable when some details were missing (see below).
Critical weaknesses
My main concern is that MoJEPA may be restricting itself to only using one type of reference context when other approaches to spanning forest selection could be explored. It gives the impression that only this one approach works and could limit further developments. It was unclear from the ablation studies in Section 5 whether this is actually being considered already.
Some sections lacked necessary detail, for instance, more should be provided in Section 3 (Preliminaries) although the rest of the paper was written well enough to fill in some of the missing gaps here. Noticeably, Theorem 4.2 lacked definitions for and necessary for explaining its purpose, when motivation for random-walk hitting times based reference forests was needed over alternate approaches.
Questions for authors
- Does the theory extend to any rooted forest for provide JEPA benefits?
- How does link prediction perform in the case where in Equation 5? It would be interesting to know if the historical statistics dominate the scoring and evaluate the benefit from the MoJEPA node representations. It was unclear if this was being performed in the ablation study.
- The paper describes within-dataset prediction (transductive learning) but goes to the extreme when considering domain transfer (inductive learning). How does MoJEPA perform within-dataset inductive learning?
4 comments
Q1: Scope of the Morse construction and its role in JEPA.
Morse theory relates rooted forests to discrete gradients; scalar fields guide the historical routes selected for learning.
- General correspondence. Theorem B.3 establishes a bijection between discrete gradients on a graph and rooted spanning forests. For any rooted forest with roots R, the identities \|E_F\| \= \|V\| − \|R\| and \|E \\ E_F\| \= β₁(U) + \|R\| − β₀(U) hold independently of its construction.
- Scalar-field selection. Our minimum-neighbor rule uses a node field to select descending parent edges. Theorem 4.2 then quantifies how these choices change with graph-support and score perturbations. This makes reference-route selection explicit and analyzable. Different forest algorithms can also supply reference edges to the same encoder, as demonstrated by the matched comparisons in our W1 response.
- Representation learning. The selected forest supplies sparse historical routes shared across snapshots, allowing node states to be exchanged even when particular real interactions are absent. On cyclic graphs, different forests preserve different routes for message passing; their predictive usefulness is assessed experimentally.
The No Forest ablation shows gains of +1.88/+2.33 MRR on Token and +0.17/+0.18 on QA (within/transfer). Matched forest comparisons further show that selecting different routes at the same edge budget changes predictive performance. These results connect the general construction to its practical role in representation learning.
Q2: Contribution of neural representations when .
Setting η \= 0 in Equation (5) removes the recency-weighted neural term, leaving only the TPPR neighborhood inner product. This corresponds exactly to the TPPR-only rows of original Tables 15–16. The original MoJEPA rows, including random initialization, use the full fused score.
To quantify the independent contribution of representation learning, we cross random versus JEPA-pretrained initialization with neural-only versus fused scoring:
- Random: No JEPA pretraining.
- Pretrained: 6,000 JEPA pretraining iterations.
- Neural-only: Neural score without TPPR or recency weighting.
- Fused: Neural score combined with TPPR and recency.
All settings share the architecture, Morse forest, training queries and fine-tuning budget. Each initialization uses the same neural-selected checkpoint and candidates for both scores; fusion is calibrated on source validation.
Table 1. Pretraining and score fusion, MRR ×100 (mean ± SD, three seeds).
| Score | P | Token W | Token T | QA W | QA T |
|---|---|---|---|---|---|
| TPPR-only | n/a | 33.41±0.00 | 34.20±0.00 | 46.21±0.00 | 46.43±0.00 |
| Neural-only | 0 | 66.25±0.22 | 58.83±0.50 | 39.39±0.12 | 38.49±0.14 |
| Neural-only | 6000 | 77.38±0.32 | 75.03±0.21 | 50.91±0.20 | 49.97±0.24 |
| Fused | 0 | 67.11±0.31 | 60.08±0.38 | 47.22±0.05 | 43.32±0.13 |
| Fused | 6000 | 78.14 ± 0.47 | 76.11 ± 0.29 | 57.39 ± 0.08 | 56.58 ± 0.07 |
Finding. Pretraining improves neural-only MRR by 11.13/16.20 points on Token and 11.52/11.48 on QA (within/transfer). These gains demonstrate a representation-learning benefit without score fusion. Fusion provides additional gains, especially on QA, where pretrained MoJEPA improves by 6.48/6.61 points. The experiment thus separates two complementary sources of predictive performance.
Q3: Inductive link prediction involving new nodes.
We test links involving naturally emerging nodes on Bytom. A node is new if it never appeared in histories used for pretraining, supervised training, or fitting training statistics. Models remain frozen; predictions use only prior observations and the original candidate sets and ranking protocol.
The test contains 7,150 positive links involving at least one new node across 59 test dates. We group positive links by endpoint status and report neural-only MRR over three seeds.
Table 2. Bytom new-node prediction.
| Method | Old → New | New → Old | New → New |
|---|---|---|---|
| EAGLE | 46.24±0.18 | 95.36±0.13 | 77.98±0.20 |
| WinGNN | 27.54±0.02 | 62.45±0.56 | 58.58±0.35 |
| EGCH | 32.70±0.62 | 91.11±0.17 | 66.30±0.62 |
| ROLAND | 40.94±0.55 | 94.28±0.63 | 71.65±3.58 |
| SiST-GNN | 32.17±2.89 | 92.69±0.65 | 59.28±6.57 |
| SG-JEPA | 48.55±3.99 | 95.26±0.08 | 79.94±0.28 |
| MoJEPA | 62.49±0.55 | 96.35±0.22 | 88.04±0.50 |
Finding. MoJEPA achieves the highest MRR in all three groups, improving over the strongest baseline by 13.94 points on Old → New, 0.99 on New → Old, and 8.10 on New → New. Across all new-node links, it reaches 84.21 MRR, 7.22 points above the strongest baseline. Interaction-derived features, rather than learned node-ID embeddings, allow MoJEPA to represent newly observed nodes.
We will incorporate these results and theoretical clarifications into the revised manuscript.
Thank you to the authors for their rebuttal. My initial concern mirrors those shared with the other reviewers as it appears that the largest gains comes from the wider JEPA framework. While there are additional benefits using hitting time reference forests, these seem less significant, which you mention as part of Table 1.
However, I feel that the JEPA framework alone is a strong selling point for this paper, with the investigation into alternative reference forests as the secondary part. There is still research to be done, for example, other sparse graphs with cycles beyond just spanning forests. Along with other additions for the revised manuscript, I believe this reframed paper is worth accepting and I will increase my rating accordingly.
Thank you very much, Reviewer HDjG!
Dear Reviewer HDjG,
Thank you very much for your feedback, appreciating the novelty of our approach and, of course, for raising the score! We do agree that JEPA is a bigger machinery and topological tools can enhance this machinery further, our paper is just one step toward this direction. We'll make sure we address it well in the discussion of future directions.
The Authors.
W1: Flexibility of reference forest construction.
Thank you for this question. MoJEPA is not limited to hitting time. Scalar fields guide reference-route selection. Original Table 2 already tests hitting time, hub distance, positive/negative strength and negative net flow. We also test alternative spanning forests.
1. Matched reference forests
All use identical history, one tree per observed component, edge counts, relation-type slots, and training/evaluation settings. MRR ×100, mean ± SD over three seeds (W: within; T: transfer).
Table 1. Matched reference-forest comparison.
Finding. All four forests can be used within MoJEPA. Hitting time has the highest mean across these settings, with larger gaps on Token than QA. Because component coverage and edge budgets are fixed, the comparison examines which historical routes are selected, rather than how many reference edges are supplied. In fact, BFS is a special case of scalar-field-guided construction: hub distance produces shortest-path trees, and matching tie-breaking recovers BFS parent choices. Random and maximum-weight forests show architectural flexibility, although the minimum-neighbor rule does not generate every rooted forest.
W2: Clarifying Theorem 4.2 and the hitting-time motivation.
Thank you for requesting clearer definitions. Theorem 4.2 asks how the selected forest changes when graph support or node scores change.
1. Skeleton distance and structural stability
Let S(U,g) be the Morse skeleton selected from support graph U=(V,E) by injective comparison scores g, comprising extended parent pointers p~ and unmatched (critical) edges Ecrit. We define
dSk(S,S′)=∥v:p~(v)=p~′(v)∥+∥Ecrit△Ecrit′∥.
The first term records parent, root, and vertex-presence changes; the second records changes in critical-edge membership. For each vertex (v), the local margin mvU(g) is the gap between the two smallest scores in its closed neighborhood, and is infinite for isolated vertices. On fixed support, a pointer is unchanged when the maximum score perturbation in that neighborhood is less than mvU(g)/2.
Theorem 4.2 combines score and graph changes:
dSk(S(U,g),S(U′,g′))≤∥V△V′∥+2∥E△E′∥+3min(∥HU∥,∥HU′∥),
where HU contains vertices whose local score changes reach their margin threshold (and likewise for U′). The bound separates support edits from changes in the field used to select parents. If hitting times are recomputed after observations change, their score changes must also enter the bound; a small support edit alone does not guarantee unchanged routes. This is a conditional guarantee about pointers and critical edges, not full root paths, embeddings, or prediction error.
2. Why hitting time is a motivated default
For hub h, the expected random-walk hitting time satisfies H(h)=0 and H(v)=1+∑uP(v,u)H(u))for(v=h, including self-transitions. Thus the weighted average of neighboring scores is H(v)−1<H(v), so every non-hub vertex has a distinct neighbor with strictly smaller hitting time. Strict descent cannot cycle, and the hub is the unique root. Proposition 4.1 therefore guarantees one spanning tree per observed component with exactly ∥V∥−β0(U) reference edges.
Hitting time also reflects interaction weights through random-walk transition probabilities. Although each parent is chosen locally, its score incorporates connectivity throughout the component, allowing routes beyond shortest-hop paths. This gives a principled way to select sparse historical communication routes when their predictive usefulness is unknown in advance.
For fixed support and a fixed hub, hitting times vary continuously with positive edge weights through the grounded linear system; the local-margin condition then identifies when parent choices persist. A change of hub may alter the field more broadly. We distinguish raw hitting times from the injective comparison scores obtained by deterministic tie-breaking. Hub distance likewise guarantees one root per component, while other fields can produce more roots; hitting time is not assumed universally optimal.
We will update the notation, stability discussion and hitting-time motivation in the revised manuscript as soon as possible.