acceptodds
Under review as a conference paper at ICLR 2027

Does Future Link Prediction on Temporal Text-Attributed Graphs Need Heavyweight Modeling?

Abstract

Future link prediction on Temporal Text-Attributed Graphs (TTAGs) increasingly relies on heavyweight models that process fine-grained interaction histories with temporal Transformers or large language models (LLMs). This modeling complexity incurs substantial computational cost, yet it remains unclear how much is necessary. We explore this question empirically and find that our lightweight Snapshot GNN built on reusable temporal snapshots and inexpensive global heuristics remains surprisingly competitive with substantially heavier models, while being much more efficient. To explain this result, we derive a theoretical bound on the optimal AUC advantage of the full temporal history over a compressed additive summary. Guided by the analysis, we perform controlled experiments to separate historical evidence from processing machinery and observe limited returns from both longer, finer-grained histories and heavier graph processing. LLMs still retain an advantage given comparable historical information, and graph models require additional structural cues to close the gap. This motivates us to examine where the remaining LLM advantage arises. We discover that GNN uncertainty alone can localize the predictions on which the LLM is particularly beneficial, which tend to correlate with weak structural support from the graph. When used for selective LLM invocation, these localized gains are substantially larger than their aggregate effect on global performance, while uncertainty remains highly competitive with learned localizers. Overall, our results suggest that future link prediction on TTAGs can often achieve strong accuracy at considerably lower computational cost across real-world benchmarks, with LLMs providing localized gain where graph models are least certain. Our code is available at https://anonymous.4open.science/r/dyg-6686

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Temporal Graphs, Graph Neural Networks, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ocWHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper asks whether future link prediction on temporal text-attributed graphs (TTAGs) requires heavy TGNN or LLM architectures. It proposes a snapshot GNN: a batch-shared, decay weighted graph snapshot with a shallow GIN on E5 features. This lightweight approach competes with heavy architectures and is computationally faster end-to-end. This paper provides theoretical and empirical evidence to describe when and why this methodology is sufficient.

Critical strengths
  • Their methodology gives competitive performance while having significantly lower computation cost (training, inference and preprocessing) than other temporal graph models for TTAGs.
    • With the onset of LLMs many work may rightfully apply these architectures to gain predictive performance at the cost of computation. This strength shows that this is not necessarily the case for TTAGs, showing that well motivated but lightweight methods are still competitive.
    • Given that applied work on TTAGs may require learning on particularly large-scale graphs, having a computationally light method helps scale to such datasets.
  • A controlled experimentation for revealing why this lightweight architecture is competitive against previous computationally expensive methods on TTAGs.
    • This paper's methodology shows an interesting "saturation on both dimensions" between finer-grained histories and heavier processing of evidence. We see that there are diminishing returns with increasing the amount of historical information, and with heavier processing of events.
    • This paper discusses and empirically shows cases where an LLM provides a clear advantage. In particular, GNN uncertainty can be used as a proxy for this and LLM advantage occurs where graph support is weak (e.g low homophily.)
Critical weaknesses

The paper makes a claim that LLM (Entity) processing provides additionally entity-specific associations which can be learned during pre-training and can thus provide improvements on predictive performance. - This is somewhat circular because the embeddings initialized in the graph operator (GIN) are provided by E5 embeddings which may similarly leverage semantic information from pre-training.

  • More difficult metrics on ROC-AUC may be required. The results use one random negative per positive, which averaged per batch.
    • There are harder evaluation methods which can provide larger granularity of performance differences than (0.97-0.99) on Tables 1-3. For example, MRR/Hits@k. This can make the empirical evidence more substantial.
Questions for authors

Clarifications for weaknesses above.

Reviewer_2w26OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether heavyweight temporal modeling is necessary for future link prediction on temporal text-attributed graphs (TTAGs). The authors propose a lightweight Snapshot GNN that aggregates historical interactions into temporally weighted graph snapshots, encodes node representations using a GIN and combines representations with four inexpensive temporal and structural heuristics (recency, pair interaction frequency, target activity, and Resource Allocation) for link prediction. The paper derives a theoretical bound on the optimal AUC gap between full history prediction and prediction based on an additive historical summary. Evaluations on benchmarks show that the proposed model achieves competitive performance at substantially lower computational cost.

Critical strengths
  1. The authors evaluate the proposed method on four TTAG benchmarks under both transductive and inductive settings.
Critical weaknesses
  1. Limited connection between Theorem 1 and the proposed method. The theoretical connection between Theorem 1 and the proposed Snapshot GNN remains insufficiently established. The theorem analyzes an abstract additive history summary under specific assumptions on the reconstruction error and history-dependent corrections. However, the paper does not explicitly characterize the proposed Snapshot GNN in terms of the corresponding event encoder and reconstruction map, nor does it establish that the resulting representation satisfies the fidelity condition required by the theorem. Consequently, it remains unclear whether the theoretical guarantee applies to the proposed method, and Theorem 1 does not directly establish that Snapshot GNN can achieve an AUC close to that of an optimal full-history predictor. A more explicit connection between the theorem's assumptions and the actual model architecture would strengthen the theoretical justification of the proposed approach.

  2. The methodological novelty of Snapshot GNN is relatively limited. The proposed architecture combines established components, including exponentially decayed temporal aggregation, a GIN, pretrained text embeddings, and hand-crafted temporal and structural heuristics. Its primary contribution therefore appears to lie in the empirical demonstration that a lightweight combination of these techniques can remain competitive with more complex models, rather than in a novel modeling mechanism. The paper could more clearly articulate its technical contributions beyond the integration of existing components. Moreover, the ablation studies do not fully disentangle the contributions of individual components in the final architecture across all four datasets. In particular, comparisons among heuristic-only, snapshot-only, text-only, and full-model variants would help clarify the respective roles of historical statistics, temporal graph structure, and pretrained semantic representations.

  3. Figure 1 has a generic, seemingly AI-generated visual style, which detracts from the figure's professionalism and makes the methodological illustration less clear and informative.

Questions for authors

See the weaknesses

Reviewer_UASZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors of this submission consider the task of future link prediction inTemporal Text-Attributed Graphs, i.e. continuous time temporal graphs where nodes have additional textual attributes. Such data combine three non-trivial modalities, namely graph structure, text and temporal patterns. For this problem, the authors address the question whether it requires sophisticated models like temporal Transformers or large language models (LLMs), or whether more lightweight models are sufficient. To this end they propose a lightweight "Snapshot GNN" that uses temporal snapshots with exponential decay weighting, combined with a shallow GIN encoder, frozen text embeddings and inexpensive global historical heuristics like recency or previous interaction counts. The authors evaluate their approach in four real-world benchmarks, showing that the performance of their approach matches that of the best method, with considerable speedups. The authors further derive a theoretical bound on the optimal AUC advantage of full temporal history over additive summaries, and conduct controlled experiments showing limited returns from fine-grained histories and heavier processing. They finally show that LLM advantages concentrate on queries where the uncertainty of a GNN-based prediction is high and that a simple uncertainty-based selective LLM invocation captures much of this benefit.

Critical strengths
  • Addressing future link prediction in temporal text-attributed graphs, the authors consider a timely and relevant problem, showing that complex and costly previously proposed solutions to address the task might not be needed. As such, the proposed approach is of high practical use.
  • The evidence for the claim that complex models may provide little to no advantage is strong and both the theoretical analysis and the experiments are well-executed. I particularly liked the approach to not only show but also understand the strong performance of the proposed lightweight approach, in terms of (i) testing how much historical information is actually needed, (ii) to what extent the processing of that historical information matters, and (iii) characterizing the advantage of LLMs.
  • The theorem establishing a theoretical bound on the AUC advantage of a full history over additive summaries of temporal embeddings is genuinely interesting and constitutes a contribution by itself.
  • The paper is well-written, clear and the level of reporting of experimental details is in my view above average.
Critical weaknesses
  • Results are based on a selection of four data sets only, and for one of those several of the competing methods actually did not yield results. As such, the findings for the some of the baselines only rest on three data sets, which may question generalizability.
  • I do not see this is not a critical weakness, but I want to mention it anyway: The key novelty of the work is not the lightweight link prediction architecture, but rather the analysis of why it works so well, see my question/comment Q1
  • I have some open questions regarding the generality of the theoretical finding, see my question Q2 and Q3 below.
  • In the lightweight approach proposed by the authors, textual or semantic information is only used in a rather minimal way by incorporating static text embeddings in the nodes for the GIN encoder, which does not seem to support (temporal) textual attributes for edges, see my question Q4 on this below.
Questions for authors

Q1: This may be more of a comment than a question, but it might help the authors to better frame their contribution: As mentioned above, I do not see the key novelty of this work in a simplified model for future link prediction, which appears to be a rather straight-forward combination of existing techniques. I find the analysis of WHY this simpler baseline works well, and how it can be combined with LLMs the most interesting aspect of the work, and one that can help us to design better temporal GNNs for trimodal graph data combining topology, time and text.

Q2: In the proof of theorem 1, the authors make the assumption of balanced query labels, i.e. P(Y=1|B)=1/2 and the theoretical findings seem to depend on this balance. If I understood correctly, the authors used the default negative sampling protocol of the DTGB benchmark which assumes one negative sampled for each positive edge, thus satisfying the balance assumption. I guess this could be made clear, which also implies that whether the theoretical results apply depends on the negative sampling strategy employed.

Q3: Related to Q2, the proof of theorem 1 further depends on histories with uniformly bounded temporal accumulation quantity Mβ,TM_{\beta,T}. Did the authors evaluate this quantity in the benchmark data sets? Does this constitute a practical limitation for the applicability of the theoretical findings in practice?

Q4: If I understood the approach correctly, in the Snapshot GNN textual or semantic information is only used in a rather minimal way by incorporating static text embeddings in the GIN encoder. I would argue that this limits the text modality of the temporal graph to be static, which implies that textual attributes of time-stamped edges cannot directly incorporated. At the same time, several of the data sets from the DTGB benchmark actually feature text attributes on edges. Did the authors consider these textual attributes? If not, could it be accounted for in the proposed architecture?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.