- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies temporal graph link prediction. The authors argue that the structural components in existing TGL models, and their performance gains attributed to them, are largely explained by two simpler signals: memory (whether this edge or event has appeared in the history) and recency (how long ago the endpoints last interacted).
To test this hypothesis, the paper separates memory, recency, and structure and measures how much each component contributes under different settings: logistic regression and gradient-boosted trees as scorers, and evaluation with the TGL models DyGFormer, TGN, and Hawkeye.
Critical strengths
- The paper identifies construction mistakes (Section 2) and quantifies their impact (Table 1). That contribution should be especially useful to the temporal graph learning community.
Critical weaknesses
-
The evaluation protocol used in this paper, in which each positive edge is scored against a single negative edge with AP reported, is outdated and no longer the literature standard. As highlighted in TGB [3], this strategy tends to produce negatives that are easy to predict given the structure and sparsity of real-world networks [1], leading to inflated performance estimates [2]. This evaluation limitation significantly affects how broadly the paper’s insights apply.
-
Writing issues negtively affect readability:
- The definitions of , , and (lines 120–123) use , , and without clearly defining them.
- The legend of Figure 1 is confusing and not informative enough. Under the legend, labels such as “added to popularity + pair count” and “added after memory (incl. whether they ever met)” do not state which component or feature is being added. With vertical space available, the figure could be more explicit so readers do not have to guess.
- At lines 305–306, the authors name four variants: the published co-occurrence channel, the no structural channel, the pair-time channel alone, and both, but Table 3 uses different names, which is confusing (for example, what is the “none” variant?).
- The paper’s structure feels disconnected and lacks a clear narrative flow. For example, some sections move straight into methodology without stating what the section is for or how it connects to the previous one.
-
The benefit of evaluating with logistic regression and gradient-boosted trees is not clear to me, since these scorers may not reflect whether the results transfer to actual TGL models. TGL models may use different readouts or transform structural features in ways that diverge from logistic regression and gradient-boosted trees.
-
The abstract and title suggest that structural gain in temporal link prediction is mostly memory and recency, but I find this overclaimed. Lines 261–262 and Section 6 describe cases where gains from structural additions after memory and recency remain considerable, especially on repeat-free datasets in TGB-Seq.
-
The introduction lists many TGL baselines, but the empirical results cover only a limited set: DyGFormer, Hawkeye, and TGN. Generalizability of the findings is therefore limited by baseline coverage. Extending the discussion in Section 4 to include additional baselines, such as TPNet, TNCN, or CAWN, besides DyGFormer and TGN, would improve the generalizability of the paper.
-
The main takeaway I draw is that different components matter depending on dataset characteristics (e.g., repeated-edge frequency). The paper calls this out and analyzes it, but the finding feels incremental and somewhat trivial.
-
The only conclusion I can draw is that different components matter depending on dataset characteristics (e.g., repeated-edge frequency). The paper calls this out and analyzes it, but the finding feels incremental and somewhat trivial.
[1] A.-L. Barabási. Network science. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 371(1987):20120375, 2013
[2] F. Poursafaei, S. Huang, K. Pelrine, and R. Rabbany. Towards better evaluation for dynamic link prediction. Advances in Neural Information Processing Systems, 35:32928–32941, 2022.
[3] Huang, S., Poursafaei, F., Danovitch, J., Fey, M., Hu, W., Rossi, E., Leskovec, J., Bronstein, M., Rabusseau, G. and Rabbany, R., 2023. Temporal graph benchmark for machine learning on temporal graphs. Advances in Neural Information Processing Systems, 36, pp.2056-2073.
Questions for authors
-
Line 099 states that a batch sees only events strictly earlier than its first event. Suppose the evaluation set contains multiple batches. For batches after the first batch, does the model also see events from earlier batches?
-
Lines 108–109 state that “structure reads the neighbors of the endpoints, in three blocks.” Can the authors clarify whether structural information is taken from temporal neighbors (e.g., recent interaction history) or only from neighbors within the same timestamp/snapshot?
-
Table 2 reports how many cases structural features help after memory and after recency. To assess more fully whether structural features are mostly memory and recency, could the authors also report how many structural features still help after both memory and recency (i.e., the structural residual)?
We thank the reviewer for the detailed comments. We answer the three weaknesses and the three questions together, with results measured since submission.
W1 / Q1. Why separate memory and recency from structure, if their overlap is expected. We agree that some overlap is expected. Our contribution is its size and its consequence. The size: under historical negatives, DyGFormer's co-occurrence encoding keeps a gain above 0.01 AP after memory and recency on 0 of 14 benchmarks. The consequence is practical, because the three sources have very different costs. The memory and recency of a pair are a table lookup, while a structural encoder must read neighborhoods. Inside the trained model, nine numbers about the pair's own history replace the co-occurrence channel on 11 of 14 benchmarks (extended since submission to 14 benchmarks, 3 seeds). The channel makes each batch 3.4 to 6.9 times slower, and the nine numbers at most 12% slower. A designer who reads the usual ablation concludes that neighborhoods are needed. The decomposition shows where they are not, and the test says in advance where they are: repeat-free streams and the cold part of a benchmark. We separate the sources by what a feature reads (the pair's own events, their timestamps, or other nodes' events). This is an operational definition, not a claim that the sources are independent, and we will say so explicitly.
W2 / Q2. Hand-crafted blocks; degree as a structural statistic. The memory block is deliberately minimal. It reads only the events of the pair and of the candidate; the candidate's degree and event count are the popularity statistics used by simple baselines such as EdgeBank, and they read no other node's neighborhood. We accept that the boundary is a choice, so we measured how much the conclusions depend on it. (i) Order-free attribution. Under Shapley values, the structural signals keep 34% to 41% of the gain they show when entering first, so about three fifths of that gain is shared with memory and recency. We will report this next to the sequential numbers and soften our "four fifths" sentence. (ii) Moving the "never met" indicator from memory to structure changes the historical-protocol residual from +0.000 to a median of +0.058 AP. We will report both assignments. (iii) Threshold. At 0.005, 0.01, 0.02 and 0.03 AP, 21, 14, 6 and 3 of 112 cells survive. (iv) A second scorer and a shuffled-feature control are already in the paper. We have not tested richer definitions of memory or recency, such as decayed counts. Richer blocks can only absorb more, so the current blocks give an upper estimate of what structure adds.
W3 / Q3. Generalization beyond DyGFormer. Since submission we ran the trained-model experiment on four backbones, 14 benchmarks and 3 seeds. The pair-time channel improves TGN, GraphMixer and TCL on 13, 12 and 13 of 14 benchmarks (median +0.011, +0.030 and +0.044 AP, up to +0.199). We have not run CAWN, TPNet or TNCN, and the revision will state that the model-level claim covers DyGFormer's co-occurrence channel and these three backbones only. At the feature level, the test already covers four structural signals, including a published cohesion channel. We also agree that the conclusion depends on data and protocol, and that is the claim we intend. We will scope the title and abstract to standard benchmarks. The paper names the regimes where structure survives, for example GoogleLocal, where overlap keeps +0.120 AP after recency over 5 seeds (standard deviation 0.0004).
We hope the added backbones, the order-free attribution and the narrower claim address the soundness concerns, and we would be grateful if the reviewer would reconsider the rating in their light.