- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper asks whether future link prediction on temporal text-attributed graphs (TTAGs) requires heavy TGNN or LLM architectures. It proposes a snapshot GNN: a batch-shared, decay weighted graph snapshot with a shallow GIN on E5 features. This lightweight approach competes with heavy architectures and is computationally faster end-to-end. This paper provides theoretical and empirical evidence to describe when and why this methodology is sufficient.
Critical strengths
- Their methodology gives competitive performance while having significantly lower computation cost (training, inference and preprocessing) than other temporal graph models for TTAGs.
- With the onset of LLMs many work may rightfully apply these architectures to gain predictive performance at the cost of computation. This strength shows that this is not necessarily the case for TTAGs, showing that well motivated but lightweight methods are still competitive.
- Given that applied work on TTAGs may require learning on particularly large-scale graphs, having a computationally light method helps scale to such datasets.
- A controlled experimentation for revealing why this lightweight architecture is competitive against previous computationally expensive methods on TTAGs.
- This paper's methodology shows an interesting "saturation on both dimensions" between finer-grained histories and heavier processing of evidence. We see that there are diminishing returns with increasing the amount of historical information, and with heavier processing of events.
- This paper discusses and empirically shows cases where an LLM provides a clear advantage. In particular, GNN uncertainty can be used as a proxy for this and LLM advantage occurs where graph support is weak (e.g low homophily.)
Critical weaknesses
The paper makes a claim that LLM (Entity) processing provides additionally entity-specific associations which can be learned during pre-training and can thus provide improvements on predictive performance. - This is somewhat circular because the embeddings initialized in the graph operator (GIN) are provided by E5 embeddings which may similarly leverage semantic information from pre-training.
- More difficult metrics on ROC-AUC may be required. The results use one random negative per positive, which averaged per batch.
- There are harder evaluation methods which can provide larger granularity of performance differences than (0.97-0.99) on Tables 1-3. For example, MRR/Hits@k. This can make the empirical evidence more substantial.
Questions for authors
Clarifications for weaknesses above.